AI Agent Platform Development: Requirements, Architecture, and Design Principles You Must Define Before You Build

It looks as though we could build the platform — but we still haven’t decided who is allowed to do what.

The moment Sato, from the IT department of a certain manufacturer, said that in the meeting room, I fell briefly silent. On the screen was an architecture diagram for an AI agent: the model, the APIs, the internal databases, workflow management, the audit log. Technically, it was a rather well-considered diagram. Yet the sales team wanted to hand over quotation drafting, the administrative team wanted to automate application processing, and the development team was worried about the safety of the data integrations. The security lead quietly asked, “And whose responsibility is the execution permission, exactly?”

Six months earlier, this company had been driving a proof-of-concept for an AI execution environment, led by the technical team. The trouble was that the business-side use cases and operational requirements never firmed up: of the twelve prototype flows built during the three-month PoC, only three made it into real operation. Today, they have sorted out — up front — which tasks to execute, the permissions, the audit log and the human approval step for exceptions, and they now have a clear map of their business AI architecture and a phased plan for building an autonomous AI platform.

In this article, I want to set out what you should settle as requirements before you properly build a task-executing AI agent. The aim is not a state in which the AI runs off and does as it pleases, but one in which work moves safely forward within a scope that people can manage. That said, simply building the platform will not automate every task. If you are unsure about your own scope of work, do read on while mapping each point back onto your own operations.

A task-executing AI agent platform needs its requirements settled before the technology

A task-executing AI agent platform is an environment in which the AI does more than merely answer questions: it connects to internal systems and business data and, within defined conditions, assists with and carries out work.

Tasks of the following sort, for instance, fall within its remit:

• Referring to customer information to draft a proposal
• Checking internal rules to judge whether an application has any deficiencies
• Classifying enquiries and routing them to the relevant department
• Organising information from meeting notes for entry into a CRM
• Automating notifications and report creation in line with pre-approved conditions

Raise this sort of thing and the first questions are often “Which AI model should we use?” and “Which tools can it integrate with?” Technology selection matters, of course. I came up through engineering myself, so I know the pull well: start thinking about architecture and you want to nail down the API design and data structures straight away.

But the first thing to decide for a task-executing AI agent is not “what the AI can do.” It is “what the AI may be entrusted with.”

An AI agent’s reach into the business is wider than a chat AI’s. Beyond simply giving a wrong answer, it brings the risk of registering incorrect data, sending unnecessary notifications, or pushing ahead with a process before it has been approved. For that reason, building an autonomous AI platform calls for defining the business requirements, permissions, auditing, security and operational rules before you ever select a model or wire up tools. The principle of advancing AI adoption and risk management as two sides of the same coin is reflected in established frameworks such as the NIST AI Risk Management Framework and the OECD AI Principles.

What is the difference between a chat AI and a task-executing AI agent?

A chat AI mainly responds to human input with prose. A person asks, a person reads the answer, and a person decides what to do next.

A task-executing AI agent, by contrast, has an “action” sitting beyond the answer. It takes on part of the business process: referring to external systems, updating data, starting a workflow, notifying the people concerned.

That difference changes the lens you bring to requirements definition as well.

With a chat AI, the considerations are largely answer accuracy and ease of use. With a task-executing AI agent, you add execution permissions, scope of operations, logging, approval, exception handling and stop conditions to that list.

On the projects I support, things often begin with “we’d like the AI to read our internal data.” But dig a little deeper and what people actually want is rarely just “reading.” Themes much closer to task execution surface — “we want to answer enquiries,” “we want to route applications,” “we want to suggest the next action to a sales rep,” “we want to cut the burden of CRM data entry.”

In short, building a task-executing AI agent platform is not about creating “an environment where the AI can be used.” It is about making clear the scope in which the AI touches the business, and building an execution environment that people can manage.

The first thing to decide is the use case

The starting point for AI agent requirements definition is the use case.

But broad units such as “make sales more efficient” or “automate the back office” are not enough. To entrust work to a task-executing AI agent, you need to make the task one notch more concrete.

In sales, for example, you might break it down like this:

• Summarise meeting notes
• Classify customer issues
• Draft an outline for a proposal
• Extract the fields to be entered into the CRM
• Notify the sales rep of the next action

You then decide how far along that chain you hand things to the AI.

The question I most often put on site is: “Is this a task the AI may carry through to the very end?” In most cases there is no need to hand over the whole thing from start to finish. Even if the AI merely proposes candidates and a person checks them, you may well be able to lighten the workload.

Think from the outset in terms of “fully automating everything from proposal to contacting the customer” and the requirements balloon. In the early stages, the realistic approach is to start with tasks where the criteria are clear, the impact is contained, and people can easily check the output.

My recommendation is to lay out the following for each use case:

• What the target task is
• What information the AI refers to
• What the AI creates, updates or notifies
• At what points human confirmation is required
• How far the impact reaches if it fails
• How you judge what “success” looks like

Only once you have sorted this out do the functions your AI execution environment actually needs come into view.

The next thing to decide is permissions

Permissions are the thing most likely to stay vague in a task-executing AI agent.

Human user permissions and the AI agent’s execution permissions need to be thought of separately. May the AI refer to the same information a person can view? May it update the fields a person can update? Implement while this stays vague and you tend to invite security and audit problems later on.

On one project, a business-side colleague said to me, “Couldn’t we just give the AI the same permissions I have?” I replied on the spot: “A person has the capacity to sense that something is off and stay their hand; it is wiser not to expect too much of that from an AI.”

In permission design, settle at least these four:

View permission

Decide which data the AI may access. Treat each kind of data differently — customer information, contract information, internal rules, deal histories, enquiry histories and so on.

Execution permission

Decide how far the AI may go in its operations. Make it clear whether it stops at drafting, goes as far as registering in a system, or extends to sending notifications.

Approval permission

Decide whether a person needs to check what the AI has produced, or whether it may proceed automatically under certain conditions.

Stop permission

Decide who halts the AI agent when something goes wrong.

With a task-executing AI agent, making clear “what it must not do” matters more than “expanding what it can do.” It is safer to start with least privilege and widen the scope in stages as operational experience accumulates.

Design data integration with reads and writes kept separate

Data integration is unavoidable when thinking about a business AI architecture. For an AI agent to carry out work, it needs connections to internal systems and external tools.

Lumping all data integration together, though, is dangerous. Start by separating “reads” from “writes”.

Read integration

This is the AI referring to information. Internal manuals, knowledge bases, customer information, product information, FAQs, past deal histories and the like fall within scope.

Write integration

This is the AI registering, updating or sending some piece of information — CRM entry, raising a ticket, sending an email, requesting approval, saving a report, and so on.

In the early stages it is safer to begin with read integration. If the AI refers to information and a person ultimately checks and acts on it, you can verify the value while keeping risk contained.

When you do move on to write integration, be sure to settle the following requirements:

• Which system you write to
• Which fields you update
• Whether a human check sits before the update
• Whether a mistaken entry can be corrected
• Which logs you keep
• Who is notified when an integration fails

When AI agents are first introduced, I often advise against rushing into write integration. The moment writing becomes possible, the AI is no longer a mere support tool — it becomes part of the business system.

The more useful an AI agent becomes, the greater its effect on the system. Precisely for that reason, it is important to design data integration on the assumption that it will be widened in stages.

Design the audit log from the start, not as an afterthought

When you run a task-executing AI agent in production, the audit log is a crucial design element.

If you cannot trace what the AI judged, which data it referred to, which operation it performed and who approved it, pinning down the cause when something goes wrong becomes difficult.

For audit requirements, make sure you can retain the following:

• Date and time of execution
• The user or AI agent that executed
• The instruction that was entered
• The data that was referred to
• The output that was generated
• The operation that was performed
• The approver
• The error details
• The result of the operation

What matters especially is being able to trace not just the AI’s decision but the information used to reach it.

If the AI judges that “this enquiry should be routed to legal,” for instance, then unless you can see why it judged so — which classification rule or which past data it drew on — you can neither improve nor audit it.

In practice this matters a great deal. A failure by an AI agent must not be left at “the AI got it wrong.” Was the input poor? Was the referenced data stale? Was the permission design too broad? Were the business rules ambiguous? If you cannot break the cause down, it leads nowhere useful.

The audit log is both your evidence trail when something goes wrong and material for operational improvement. By understanding which tasks the AI is hesitating over and which exception patterns crop up most, you can feed that back into improving prompts, rules and data preparation.

Set security requirements together with the business departments

Security for an AI agent cannot be wrapped up by the IT department alone.

That is because which information is confidential, which tasks are high-risk and which operations need approval are things only the business departments can judge.

For security requirements, sort out at least the following points:

• Whether it handles data containing personal information
• Whether it handles confidential customer information
• Whether it handles information relating to contracts, legal matters or finance
• Whether anything is sent outside the organisation
• Whether information access beyond the granted permissions can be prevented
• Whether a mis-operation can be halted or undone
• How long logs are retained

The important thing here is not to “think specially because it’s AI,” but to look at what changes when AI is added to your existing business risks.

Add AI to work people had been doing by hand and the processing speeds up. The speed at which mistakes propagate goes up too. For that reason, security requirements need to be built in at the requirements-definition stage, not bolted on after go-live.

When people come to me for advice on adopting AI, I recommend first deciding “what must not go in.” Deciding only which information to handle is not enough. By separating the information you will not handle, the information you will mask if you do handle it, and the information that needs a specialist department’s check, the people on the ground find it easier and more reassuring to use.

LLM applications carry risks that differ from those of conventional web applications — prompt injection and excessive permission grants among them. So when you let an AI agent operate external tools or business systems, you need to design with input controls, separation of permissions, logging and approval flows in combination. For reference, the OWASP Top 10 for LLM Applications sets out the risks specific to LLM applications.

Leave room for people to intervene in the AI execution environment

The phrase “task-executing AI agent” gives an impression of “acting autonomously.” But there is no need to automate everything.

In the early stages, it is more realistic to design things so that a person can check partway through.

Designs of the following sort, for example:

• The AI drafts and a person sends
• The AI offers classification candidates and a person makes the final decision
• The AI prepares a draft CRM entry and a person presses the register button
• The AI detects an anomaly and asks a person to confirm
• The AI proceeds to the next step only when the approval conditions are met

With this kind of human-in-the-loop design, the business departments can start using it with confidence.

My sense is that the projects most prone to failure in AI adoption are the ones where the word “automation” grows too strong too early. The projects that go well, by contrast, keep to an order: “first bring it into the work in a form people can check,” then “once the criteria are settled, widen the scope of automation.”

The same applies when you use Kanata, which our company provides: rather than aiming for full automation straight away, one approach is to combine AI chat, AI summarisation, project-level training-data management and prompt management, and first embed it into the work in a form people can check. For example, organise internal documents and FAQs as training data, have the AI produce draft answers or summaries, and use them for internal sharing or customer responses only after a person has checked them. Going through such stages makes it easier to widen AI adoption while keeping the team’s apprehension in check.

A task-executing AI agent platform is not an environment that is complete with AI alone. It needs to be designed as an operating environment in which people, AI, data, systems and approval flows all connect.

Draw up a phased build roadmap

Building an autonomous AI platform is not something you complete in one go. Widening it in stages keeps both the risk and the rework in check.

Stage 1: Sorting out the work and read integration

In the first stage, you sort out the target tasks and prepare the information the AI refers to. Organise internal documents, FAQs, manuals, past enquiries, sales materials and the like so that the AI can produce answers and drafts.

At this stage, the AI supports human judgement rather than carrying out the work itself.

In enquiry handling, for instance, the AI refers to internal FAQs and past handling records and offers a draft answer. The person in charge checks that draft, amends it as needed, and then replies. Even this alone can sometimes lighten the load on the ground.

Stage 2: A semi-automated flow with human checks

Next, you move on to a stage where a person checks the AI’s output and puts it to use in the work.

The AI might draft meeting minutes, proposal outlines, enquiry classifications or CRM entries, for example, with a person amending and approving them.

At this stage you verify the output quality, the exception patterns and the ease of use for the business departments.

I would not make light of this second stage. It is here that the people on the ground come to understand the AI’s quirks and learn to tell the work that can be entrusted to it from the work that cannot. AI adoption is a learning process for the business side, not just a technical matter.

Stage 3: Automated execution of low-risk tasks

Once operational experience has built up, begin automated execution with the tasks whose impact is smallest.

Internal notifications, raising tasks, periodic report generation and applying classification labels, for example.

At this stage the audit log, the stop conditions and error notifications become important.

When you move to automated execution, always check “can we put it straight back if it fails?” Operations that cannot be undone, that affect parties outside the organisation, or that involve amounts of money should be handled with care.

Stage 4: Task execution spanning multiple systems

Finally, you widen things to task execution that spans multiple systems.

The AI might classify an enquiry, refer to the relevant customer information, check the past handling history, present a draft answer to the person in charge, and record it in the CRM after approval. A sequence of that sort is conceivable.

At this stage, however, the impact on the business is large, so approval flows, rollback, security review and continuous monitoring are indispensable.

Dividing things into stages like this lets you grow the platform without strain while confirming the value of the AI agent along the way.

A checklist you can use in requirements definition

Before building a task-executing AI agent platform, going through the following items helps reduce the chance of overlooking something.

Business requirements

• Is the target task clear?
• Have the initial use cases been narrowed down?
• Are the success conditions defined?
• Have the exception cases been identified?
• Have the points where a person checks been decided?

Permission requirements

• Has the data the AI may refer to been decided?
• Have the operations the AI may perform been decided?
• Has the relationship between user permissions and AI permissions been sorted out?
• Are the operations that need approval clear?
• Has the person who holds the stop permission been decided?

Data integration requirements

• Are the systems to integrate with clear?
• Are read integration and write integration kept separate?
• Has the frequency of data updates been decided?
• Has the notification recipient for integration failures been decided?
• Is there a way to correct a mistaken entry?

Audit and security requirements

• Can operation logs be retained?
• Can the referenced data and the output be traced?
• Has the handling of personal and confidential information been defined?
• Is there a halt procedure for incidents?
• Have the log retention period and viewing permissions been decided?

Operational requirements

• Has the business-side owner been decided?
• Is there someone to review the AI output?
• Has the frequency of periodic review been decided?
• Are there rules for updating prompts and training data?
• Has the approach to educating users been decided?

This checklist is not just for marking a finished requirements document. It is better used as a common language so that the business departments, IT, the DX team and the security lead can talk at the same table.

When I join a project, I do not try to produce a tidy design document from the outset either. I first lay out questions like these one by one, and make visible which can be answered, which cannot, and which still reveal a gap in understanding between departments. Requirements definition for an AI agent is, before it is a technical document, a process of building consensus among the people involved.

In closing: build a task-executing AI agent platform only after you have defined the work and the scope of responsibility

When building a task-executing AI agent platform, you will be tempted to hurry into technology selection. Which AI model to use, which tools to integrate with, which cloud environment to place it in — these do matter.

But there are things to settle before that.

• Which tasks to entrust to the AI
• Which data to show it
• Which operations to permit
• Where a person checks
• What triggers a halt
• Who holds responsibility

Unless these are settled, however fine an AI execution environment you prepare, the people on the ground will not be able to make full use of it.

I regard AI adoption not as “a project to bring in technology” but as “a project to redesign the scope of responsibility within the business.” With a task-executing AI agent, that character comes through all the more strongly. The more the AI does, the human role does not thereby diminish; rather, you need to be clearer about what people judge and what they are responsible for.

A task-executing AI agent is not an all-purpose automation device. Only once you understand the work, design the permissions, and operate it in an auditable form does it deliver value safely within a company.

There is no need to build big from the start. The first thing is for the business and technical sides to hold the same map. Begin with small use cases, improve while watching the logs, and widen the permissions and the scope of automation in stages. That accumulation is what leads to a business AI architecture suited to your own organisation.

Q&A: Common questions on requirements definition for a task-executing AI agent platform