My first RAG experiment
After a year of hand-feeding custom GPTs, I finally tried
The AI news this month has been impossible to keep up with. DeepSeek crashed the stock market, reasoning models are suddenly everywhere, and every newsletter I read says this is the year of agents. Meanwhile the thing that actually changed my work is quieter and about two years less fashionable: I finally got around to RAG.
I wrote in December about my custom GPT problem. The assistants are only as current as the knowledge I upload, and keeping that current is a manual job I do by hand, badly, on whatever day I remember to. Retrieval augmented generation is the standard answer to exactly this. Instead of uploading documents into the assistant ahead of time, the system searches your documents at the moment you ask a question and hands the relevant pieces to the model along with your prompt. The knowledge lives outside the AI, so updating the knowledge means updating the documents.
I understood the concept for a while. Building even a rough version taught me things the concept didn't.
Retrieval quality matters more than model quality
My first version pulled the wrong context constantly. The model then did a beautiful job answering based on the wrong documents. The model itself needed almost nothing from me, and swapping to a different one barely changed my results. Where all my time went was what gets retrieved and how: how documents get split up, what makes a chunk findable, which documents deserve to be in there at all. The model was never the problem.
It turns the context problem into a product problem
Keeping GPT knowledge current was a chore. Retrieval turns it into a pipeline: what sources feed in, how often, who owns them, what happens when two documents disagree. Those are product questions, and I know how to work on product questions.
The system picks the context now
With my GPTs, I chose every piece of context myself. Here, the search picks, and the model sees whatever got retrieved, whether or not I ever look at it. I still spot-check when an answer seems off, but the selection isn't mine anymore. Last summer I wrote that Andrew Ng's agent patterns described AI one step past where I was working. Building this closed some of that distance. The system retrieves and answers on its own, and everything else is still me.
There's plenty I want to improve. Smarter chunking, a bigger source set, and it has to prove itself against more than my own questions before I'd put it in front of a team. But it already answers product questions better than my most carefully maintained GPT, and I built it myself in my spare time.