[ Ai and llms ] Part 5 of 6 10 min read
How LLMs Actually WorkA Diary It Rereads Every Time: Context Windows and LLM "Memory"
A language model has no memory in the sense you'd expect. Nearly everything that looks like memory is text being resent, summarized, searched, or stored by the software around it, and the diary gets heavy fast. Part five of six.