I run AI agents on my own server - and client data never goes near them

A chatbot answers when you ask. An agent takes a task, searches, reads, runs tools and comes back with a result - while you do something else. That is the whole point. It is also exactly why “can we let it loose on our data?” is the wrong first question.

The right first question is: which data may an autonomous process touch, and what may it do without a human? I had to answer it for myself before building anything. Here is the answer, the setup that came out of it, and what it taught me.

Two zones, one hard rule

Zone	What’s in it	Agents
Client zone	Third-party data - ERP, finance, mailboxes of the companies I work for. Under NDA and GDPR.	Not allowed. Only interactive work, with me in the loop on every step.
Private zone	My own business and personal data	Allowed. Autonomous for information tasks; manual confirmation for anything with consequences - money, outgoing messages.

The agent server lives entirely in the private zone. Client data never gets there. That’s not a setting that could be switched - it’s the reason the server exists as a separate machine at all.

The same split works inside a company. Before choosing a tool, decide which systems are “client zone” for your agents - payroll, customer data, anything regulated - and keep autonomous processes out of them by architecture, not by a policy document.

What the setup is

A small private Linux server in Azure. It has no public inbound port - nobody can connect to it from the internet; I reach it over a private network (Tailscale). The agents - Claude Code from Anthropic and Codex from OpenAI - run under an account without administrator rights. Even a confused or manipulated agent stays unprivileged.

I talk to it through Telegram. Three bots: one for each agent, and one for a pipeline that makes the two work together. Only my Telegram account is accepted; anyone else is rejected and logged.

Task (Telegram)
   │
   ▼
Gateway on the server
   │
   ├─► Agent A - Claude   (research, draft)  ──► DRAFT ─┐
   │                                                    │  full text, passed inside the process
   ├─► Agent B - Codex    (critic)          ◄───────────┘
   │
   ▼
Draft + critique back to me

Infrastructure cost is roughly EUR 60-75 a month (VM, outbound gateway, IP), plus the AI subscriptions. Hardware was never the hard part.

The agent that checks the agent

The pipeline is a fixed line: agent A researches and writes a draft, agent B gets the complete draft and attacks it. I only appear at the start (the task) and the end (draft plus critique) - not in the middle, copying text from one chat window to another.

That handoff was the first real lesson. In the early version the draft travelled through the chat itself, and the chat’s message limit (4,000 characters) quietly split it - agent B received only the first chunk and had no way of knowing the rest existed. Now the draft is passed inside the process and nothing gets lost.

Two modes: /auto runs the whole line; /review stops after the draft so I can read it before the critic sees it.

What the critic reliably catches:

• wrong version numbers and dates, checked against the actual changelog;
• confabulation - two real releases merged into one that never existed;
• inference presented as fact - a judgement dressed up as evidence;
• and, importantly, it says what it could not verify instead of guessing.

An unplanned division of labour appeared on its own: the first agent synthesises and flags its own uncertainty; the second goes to the web and checks against primary sources. That’s stronger than two agents doing the same thing twice - and it’s the pattern I’d build into any business use of AI where the numbers matter.

The two prompts that do most of the work

The critic’s instruction:

You are a critic. Below is the output of another agent. Find FACTUAL weaknesses:
unsupported claims, wrong numbers / versions / dates, confabulation, inference
presented as fact. Verify against primary sources. Whatever you cannot verify,
mark as unverified - do not claim certainty you don't have. Don't critique style.

=== OUTPUT TO REVIEW ===
{draft}

And the skeleton of the research task:

Task: research + synthesis.

TOPIC: {topic}

METHOD:
1. Find 5-7 specific items. Prefer primary sources (changelog, release notes,
   official blog) over aggregators.
2. VERIFY: confirm date and existence of each item from a primary source.
   If you only find it in one secondary source and can't confirm it, DON'T drop it -
   include it, mark it [UNVERIFIED] and say why.
3. For each item: what it is (1 sentence) + WHY it matters in MY context
   + link to the primary source.
4. Section "DOES THIS CHANGE ANY OF MY DECISIONS?" - flag anything with impact.

RULES:
- Paraphrase, don't paste paragraphs. Don't invent links or facts.
- If the topic doesn't fit the scope, better 4 verified items than 7 with filler.

The last line is the most important one. Ask for seven items and a model will give you seven - padding with whatever sounds plausible. Explicit permission to return fewer is what separates a useful agent from a confident one.

What it can’t do (yet)

Honest limits of version one:

• No memory across restarts. A draft waiting for review lives in memory; restart the service and it’s gone.
• No initiative. Everything starts with my message. No schedules, no “it noticed something”.
• No safe place to run arbitrary code. The agent account is deliberately tied down; a separate sandbox machine is planned, not built.
• Single user. Sharing the bot with someone else would mean giving them code execution on my server under my accounts. “Sharing” an agent is not like sharing a document. The safe way to share is to share the output.

What I’d tell a company

The model is the easy part. The work is in three decisions: which data the agent may see, what it may do without a human, and who checks its output. Get those wrong and a better model only gets you wrong answers faster.

The technical side - the locked-down server, the permission model and the mistakes I made building it - is in a separate, longer note: Claude Code and Codex on a locked-down Azure VM.