Agent Memory

The successor to prompt engineering: deliberately curating what enters the model's context window — system prompt, retrieved docs, tools, memory. Goal is max accuracy on the fewest tokens. A model only knows what you put in front of it.

A proxy layer between your app and LLM providers (OpenAI, Anthropic): routing, retries, caching, rate-limits, key management, cost tracking and failover. One place to see your whole AI bill — and no lock-in to a single vendor.

Send each request to the cheapest model that can handle it: a small model for easy queries, a frontier model for hard ones — often decided by a classifier. Cuts inference cost dramatically, frequently 5-10× on real traffic.

A RAG variant that retrieves over a knowledge graph (entities + relationships) instead of flat text chunks. Lets the model answer multi-hop questions ("how is X connected to Y?") that pure vector search misses.

Model-generated training and eval data for when real data is scarce, sensitive (GDPR), or imbalanced. Useful, but you must check quality and diversity — otherwise you bake the model's own blind spots into your system.

An AI architecture where the model retrieves relevant documents from your own data before answering, and only reasons over that context. Kills ~80% of hallucinations.