Production agent systems, end to end: orchestration, retrieval, and the evals that decide what ships.
how i build products
I work where applied engineering meets evaluation research.
At TheAgentic I’ve shipped multi-agent systems across four client products, from an AI COO for founders to an automotive repair assistant. The lesson that stuck: an agent is only as good as the context it retrieves and the evidence it’s judged on.
So I measure where systems fall short as carefully as I build them: ungrounded retrieval, unverifiable claims, and the gap between a fluent answer and a correct one.
Four client products, end to end: agent design, retrieval, evals and the infrastructure underneath.
- AI COO platform: fit-score AUC 0.51 → 0.82 on 1,200 planted assessments; integration API calls cut ~97%.
- Repair assistant: wrong-vehicle pages 60% → 4.4%, manual lookups 30.6 s → 0.1 ms, page reads 12.8× faster.
- Football scouting: backtest 53% → 71% across 106 replayed transfers over 5 seasons; discovery ~50 s → 7 s.
- Kubernetes RCA: 8/12 strict on held-out faults, with PII redacted at every model, tool and stream boundary.
LangGraphPydanticAIpgvectorNeo4jPostgreSQLFastAPI
selected work
Client names held back. What each one does, and the call that made it work.
AI COO platform
Embeddings pick candidates, SQL enforces filters and a fixed ladder broadens: reproducible investor search over 150k investors.
PydanticAI / pgvector HNSW / PostgreSQL
Repair assistant
Routes each vehicle to a 2.7M-chunk Neo4j graph, 70K OEM manuals or live web research, and scopes retrieval before it ranks.
Neo4j GraphRAG / pgvector / PydanticAI
Football scouting
Discovery, valuation, transfer-impact and tactics agents over 572K players, with every verdict backtested point-in-time.
PostgreSQL / guarded SQL / LLM critics
LiveGraph
A planner draws the agent DAG live; edits recompute only what changed, like a build system, at 38–59% of a fresh run.
PydanticAI / React Flow / WebSocket
specifications
What I use, and where it actually runs.
- agents
- LangGraphPydanticAItool callingstructured outputs
- retrieval
- pgvector HNSWNeo4j GraphRAGpg_trgmhybrid ranking
- data
- PostgreSQLSQLRedisKafka
- backend
- PythonFastAPIDockerKubernetes
- frontend
- TypeScriptReactNext.jsReact Flow
- evaluation
- LLM judgesbacktestingplanted answerspytest
LangGraph runs the incident-RCA agent; PydanticAI the rest of the client work.
Scope-first GraphRAG behind the repair assistant; embeddings-then-SQL investor search in the COO platform.
A player warehouse the scouting agents query in SQL.
FastAPI under everything; Kubernetes for deploys and incident work.
LiveGraph’s live, steerable canvas.
How I know a change helped before it ships.
Building something
that has to be right?
Agents, retrieval or evals that need to hold up in production. I’d like to hear about it.
off the clock · still learning
- 01Consumer techlaunches, teardowns and the details that make hardware feel rightkeynotesteardownsindustrial design
- 02Sportshover for the lineupCricketFootballTennisF1Basketball
- 03Geopoliticswho controls the chips, the energy and the routestradesemiconductorsenergysupply chains
- 04Musicaudiophile · 1950 → presentIndianOld classicsRetroJazzHip-hopRap