Building your first AI agent: what it is and what it isn't

"AI agent" currently describes everything from a chatbot to a fully autonomous system. That vagueness is why so many agent projects disappoint.

Here's the definition I find useful, and how to build one that survives contact with reality.

A useful definition

An agent is a system that, given a goal, decides what steps to take rather than following a fixed sequence.

That's the distinction. A workflow follows a path you designed. An agent chooses a path.

By that definition, most things marketed as agents are workflows with a language model inside them — which is completely fine, and usually the better engineering choice. Just call it what it is.

The three layers

Every reliable automation I've built separates these before any building starts:

Directive — the actual process, written out clearly. What's happening, in what order, and why. This is documentation, not code, and skipping it is why most automations are unmaintainable.

Orchestration — where the system genuinely has to reason. Route this enquiry, classify this message, decide whether this is urgent.

Execution — the repeatable steps that need no intelligence at all. Send an email, write a row, call an API.

Most failures come from mixing these. People ask a model to handle execution — "and then send the email" — where deterministic code would be faster, cheaper and correct every time.

Use the model for judgement. Use code for everything else.

What to build first

Not a general-purpose assistant. Pick one narrow, repetitive, judgement-light task:

• Classifying inbound enquiries by type and routing them
• Summarising a call transcript into a structured format
• Drafting a first-pass response for a human to approve
• Extracting structured data from messy documents

One task, clear input, clear output, human review at the end. That last part matters more than people want it to.

The architecture that works

1. Trigger — webhook, schedule, or new record
2. Gather context — pull the data the decision needs
3. The model call — one focused prompt with clear constraints
4. Validate the output — check it's the shape you expected
5. Act — deterministic code doing the actual work
6. Log everything — inputs, outputs, decisions

Step four is the one people skip. Models return unexpected shapes sometimes. If your next step assumes valid JSON and doesn't check, you'll find out in production.

Constraints belong in the prompt

The most valuable parts of a production prompt are usually the restrictions:

• What to do when the input is missing or malformed
• What never to invent
• What format the output must take
• When to escalate to a human instead of guessing

I build audit reports with a prompt that spends more words on "don't fabricate metrics, say the data was unavailable" than on what to write. That's the part that makes it trustworthy.

Cost, honestly

Model calls cost per token, and long context is where cost lives. Truncate inputs. Cap outputs. Choose a cheaper model where quality allows — the gap between tiers is often smaller than the price difference implies.

A real number: the audit tool on this site costs roughly four US cents per run, with inputs capped at around 26,000 characters. Capping is what makes the cost predictable regardless of what someone submits.

When not to use an agent

• The task is fully deterministic — write code
• Errors are expensive and can't be reviewed
• You can't describe the process clearly yourself
• Volume is so low that manual is genuinely cheaper

That last one gets ignored constantly. Automating a five-minute weekly task is a hobby, not an efficiency gain.