A coworker shared this video called 12-Factor Agents: Patterns of reliable LLM applications which for me was a total "ah ha!" moment. Not so much as any of it was new, but a concise synthesis of lots of loose thoughts I've been having while designing and building AI systems over the last few years, which to me is even more exciting. If you're building or thinking of building an AI system, I highly recommend watching it. It's 17 minutes well spent.

Here are the three most important ideas / principles that I wish I'd known before I started designing and developing agentic systems:

Key idea: Effective AI systems are built using a series of agents - distinct prompts and models - with each agent given specific context - broad or deep but rarely both - and the system composed of multiple in both dimensions. The "monoprompt" approach is very rarely going to work well.

For a simple example: In a bot designed to discuss the pros and cons of various retirement planning strategies, one agent might summarize a user query broadly, while another dives deeply into financial datasets relevant to the asset classes being discussed, and a third would dive deep into the user's own specific profile and goals. Only the first agent would be given broad user and multi-domain context in its prompt. The other two would be isolated to context that supports depth in their focus areas.

Key idea: AI product development skateboard-level prototyping is really easy, almost too easy. It's really simple to get the first 80% going and assume that the next 20% will be just as easy as the first 80%. In a way it follows most software development in that most competent web developers could build 80% of Facebook in a couple weeks - CRUD with persistence on a handful of entities being a very solved problem - but getting the full next 20% - an app that scales to billions of users - takes thousands of engineers upwards of a decade.

Again, be wary of the monoprompt. It's really easy to get 80% of the way there with one and assume the other 20% is simply edge case handling.

Key idea: Not every problem is a good problem for an agent-driven system. In the video the presenter frames it as "realize this isn't a good problem for agents". My experience is that there's more nuance, and that it's actually more of a distinction between building an AI-driven system with a human-in-the-loop, or a human-driven system with AI-in-the-loop. I've touched on this before, but I'll write an updated post soon.