Building a data warehouse

It is time to give my personal AI access to more of my data, so it can start answering questions with context and in a more relevant way. As part of this I wanted to address something I have been thinking about for a long time, funneling most of my data into a searchable PostgreSQL database. Obviously this will not be your typical data warehouse, I do t expect more than a few gigabyte of data to accumulate over a year. The stack will be Python and PostgreSQL. I could certainly make vectorization and ranking work in Go, but I refuse to fight the stack all day long. For most of ML and AI Python is and will likely be the lingua franca. Data One of the most important subjects to me is messages and email, so I obviously started there. I will consolidate all in a single model, there is not much difference but the way they were transferred over the wire. There is a sender, a recipient, a body and some meta data. Most importantly a source and external ID so I can make sure messages are unique…