AI agents are easy to demo and harder to run well in production. The difference between the two is where most of the cost sits. This guide explains the drivers behind building and running an AI agent or chatbot, so you can scope a first agent that is useful, safe and sustainable to operate.
Why is an AI agent more than a prompt?
A production agent is a system: it retrieves the right data, calls tools with permissions, follows guardrails, hands over to people, logs everything, and is measured against real questions before each release. The language model is one component. Most of the engineering effort is in the parts around it.
What are the main cost drivers?
| Cost driver | Lower effort when… | Higher effort when… |
|---|---|---|
| Use-case scope | One topic area, answers only | Many topics, mixed intents, multi-step tasks |
| Data sources | A set of documents | Several databases, APIs and live systems with permissions |
| Tools & actions | Read-only lookups | Writes to CRM, ERP or booking systems, with approvals |
| Channels | Website chat | WhatsApp, email, voice, in-app and internal tools together |
| Guardrails | Internal users, low risk | Customer-facing, regulated topics, financial actions |
| Evaluation | Small test set | Large test sets, multiple languages, regular regression runs |
| Integration | Standard APIs | Legacy systems without APIs, custom connectors |
| Infrastructure & hosting | Cloud model APIs | Self-hosted models, data residency, private networking |
| Model usage | Low volumes, short answers | High volumes, long documents, large models |
| Compliance & support | Basic logging | Audit trails, retention rules, extended support |
Use-case scope
The number and variety of things the agent must handle sets the base effort. “Answer delivery and returns questions” is narrow and measurable. “Handle all customer messages” is broad, harder to evaluate and riskier. Starting narrow costs less effort and is safer.
Data sources and retrieval
Answers grounded in your data need retrieval: documents are chunked and indexed, live data is fetched through APIs, and permissions decide what each user may see. Messy or scattered content needs preparation. The RAG chatbots guide explains how this works.
Tools, actions and guardrails
An agent that only answers is simpler than one that acts. Each tool, such as “create ticket” or “reschedule booking”, needs an API connection, permission checks, validation and sometimes human confirmation. Customer-facing agents need stronger guardrails: scope limits, refusal behaviour and escalation rules.
Evaluation
Evaluation means building a test set of real questions with expected answers and scoring the agent on every change. It is essential for trust and catches regressions when prompts, data or models change.
Infrastructure, model usage and support
Hosting covers the application, vector index, logs and monitoring. Model usage is charged per request by the provider and depends on model size, answer length and context size. Self-hosted models replace per-request fees with infrastructure and operations effort. Support covers evaluation runs, tuning and new capabilities.
Which costs continue after launch?
- Model usage, which scales with conversations and context size.
- Hosting for the application, vector index, logs and monitoring.
- Channel fees, such as WhatsApp per-message fees set by Meta.
- Regular evaluation, tuning and content updates.
- Support and new tools or channels.
How can you keep AI agent cost under control?
- Pick one high-volume, low-risk use case with clear right answers.
- Ground answers in curated content rather than everything in your shared drive.
- Route by difficulty: small, fast models for routine questions; larger models only when needed.
- Cache and trim context: reuse frequent answers and send only relevant passages.
- Measure value: track resolved conversations and hours saved against running cost each month.
Questions to ask before you start
- Which questions or tasks consume the most staff time today?
- Where do the correct answers live, and who keeps them current?
- Should the agent only answer, or also take actions?
- Which channels do customers or staff use?
- What must the agent never say or do?
- When should it hand over to a person, and to whom?
- What data may be sent to a cloud model, and what must stay in-house?
- How many conversations per month do we expect at launch and in a year?
Get a scoped estimate
Every system is different. Send us your requirements and we’ll come back with a module-by-module scope, timeline and infrastructure plan, including expected running costs. We reply within 1 business day.
Book a scoping call or message an engineer on WhatsApp, and see how we build AI agents and chatbots.