I read this: http://ampcode.com/how-to-build-an-agent.
I followed along and pasted the code and it is nice to see how, underlying all the hype around “agents”, the essential concept is very simple (and very powerful):
- LLMs can understand text very well (they are called large language models after all)
- We expose tools to the LLMs, by literally using text to say “these are the tools available, give me a reply that looks like this if you want to use the tool” (in the example, Anthropic/Claude basically has a preset format for how these tools are exposed)
- When we get this text, we basically “use the tools” by calling a function with the specified parameters. This is why “tools” is interchangeable with “function call”.
- We feed the output of the function call back to the LLM, which can continue the “conversation”.
While a “conversation” is the underlying primitive of LLMs, one can easily imagine building different UX based on the specific use case.
I then took some time to delve more deeply into A2A (TLDR: it’s a protocol for agent-to-agent communication, still early and I don’t think I have seen much actual adoption yet) and MCP (a standard for “exposing tools and resources to LLMs”).
Specifically, I watched this https://www.youtube.com/watch?v=m3IE6JygT1o. Interesting notes:
- Remote MCP servers are going to be way more interesting than local ones. The ones I’ve seen so far are all built using this: https://github.com/geelen/mcp-remote
- David basically created Sentry’s remote MCP server using Cloudflare. I am interested in Cloudflare’s AI play. It seems like there are still rough edges with the OAuth to be worked out
- I still don’t really understand evals for LLMs (e.g. evalite, BrainTrust, it seems like Sentry rolled their own because evalite doesn’t play that nicely with Vitest?)
Some other MCP stuff I’ve seen and intend to spend more time to explore more deeply:
I am convinced that we will see many many MCP servers shortly.
Meanwhile, I have started on a new side project to build a quick AI agent using Cloudflare’s Agent SDK. Look forward to reporting back once that’s ready.
Came across this on evals by Andrew Ng
https://x.com/andrewyng/status/1912908679344693711?s=46