Rag

Sooner or later every RAG system sends you the same bill, and it’s written in gigabytes of RAM. The model wants memory for its weights and its KV cache. …