Try a cheaper workflow beside a live one

Some jobs run the same way many times a day, and each run is one long agent turn. You can often do the same job more cheaply with a pilot workflow: a few small steps written from what the full agent did on past runs. Before you trust it, you want figures: what each run costs on each path, and how often the pilot gives up.

This page shows how to run a pilot beside the live job, on the same input, without letting it change anything. Then one command compares the two.

A workflow is a list of steps AgentX runs in order. One pass through it is a run. A trace is the record AgentX keeps of one agent turn, with the tokens it used.

How it works

• The live workflow keeps doing every run, as today.
• After its agent step, the live workflow starts the pilot workflow with the same input. The pilot's agent steps run with autonomy report, so they read, decide and answer, but change nothing.
• When the pilot cannot handle a case, its step answers with the result token RESULT: fallback. That is a hand-back: if the pilot were live, the full agent would take the case.
• agentx workflow pilot-report reads the traces of both runs and prices their tokens with the same table the costs page uses. Requests over 200,000 input tokens are priced at 1.5 times, as the provider bills them.
• Each trace is priced at the model it ran on. A trace written by this version or later records that model. For an older trace, the report uses the model set for that agent in agentx.json. When neither is known, or the model is not in the price table, the trace is priced at Sonnet rates and the report says how many traces that was.

Steps that run code (scripts, branches) make no model call and count as zero on both paths.

Set it up

You need workflows turned on and the live workflow saved in .agentx/workflows/. The examples use a live workflow owner-sweep with a script step sweep and an agent step act.

Know this before the first run: the live run waits at the new pilot step until the pilot run ends. If a pilot run fails, the live run stays paused at pilot instead of ending as completed. The live agent's work is already done by then, but a scheduled job's run will show as paused. See If something is wrong for how to clear it.

1. In your editor, create .agentx/workflows/owner-sweep-pilot.yaml. Its trigger is manual, because the live workflow starts it:

   id: owner-sweep-pilot
   version: 2
   title: Owner sweep pilot (dry-run)
   
   nodes:
     - id: start
       type: trigger.manual
       config: {}
   
     - id: decide
       type: agent
       config:
         agentId: example-agent
         autonomy: report
         prompt: |
           For each line below, say which action you would take and why.
           Do not change anything. If a line does not fit these rules,
           end your reply with "RESULT: fallback".
   
           {{start.items}}
   
     - id: done
       type: end
       config: { status: completed }
   
   edges:
     - { from: start, to: decide }
     - { from: decide, to: done }

2. In your editor, open the live workflow and add a step after its agent step that starts the pilot with the same input:

     - id: pilot
       type: subProcess
       config:
         workflowId: owner-sweep-pilot
         inputMap:
           start: { items: "{{sweep.output}}" }

3. In the same file, send the agent step to the new step, and the new step to the end. Replace the edge { from: act, to: done } with:

     - { from: act, to: pilot }
     - { from: pilot, to: done }

4. In the terminal, check both files:

   agentx workflow validate .agentx/workflows/owner-sweep-pilot.yaml
   agentx workflow validate .agentx/workflows/owner-sweep.yaml

5. In the terminal, reload the workflows:

   agentx workflow add .agentx/workflows/owner-sweep-pilot.yaml
   agentx workflow add .agentx/workflows/owner-sweep.yaml

The pilot runs after the live agent has finished, so it never delays the live work.

Read the figures

1. In the terminal, after a few days of runs:

   agentx workflow pilot-report owner-sweep owner-sweep-pilot --since 7d

2. Read the table:

   Line	What it is
   owner-sweep (all)	Every live run that reached a model in the period. Runs that stopped at a script step are left out.
   live, paired runs	The live runs that have a pilot run beside them.
   pilot path, paired	What the same runs would cost on the pilot path: the pilot's cost, plus the live cost for each hand-back.

   Below the table: how many runs were handed back, and the saving in percent on the paired runs.

   A pilot run that has not ended yet is left out, and counted on a Not compared yet line. It is compared on the next report, once it ends.

   If you see No price known for N of M model call(s), those calls were priced at Sonnet rates, so their cost may be wrong. Either the call has no model on record, or its model is not in the price table. When some calls have no model, the line says how many and asks you to set the agent's model in agentx.json. A Models with no price line names the models of the second kind.

3. In the terminal, to check runs one by one, list each pair with its trace ids, then open a trace:

   agentx workflow pilot-report owner-sweep owner-sweep-pilot --since 7d --pairs
   agentx trace show <trace-id>

Whether each answer was right is for you to judge: compare what the pilot said it would do with what the live agent did on the same lines. A hand-back is not an error; it is counted in the hand-back share and in the cost.

Add --json to get the same report as data, with every pair.

Check it worked

• agentx workflow runs owner-sweep-pilot lists pilot runs, one after each live run that reached its agent step.
• agentx workflow pilot-report owner-sweep owner-sweep-pilot shows a pilot path, paired line with at least one run.
• The pilot changed nothing: the issues, messages or files the job works on show only the live agent's actions.

If something is wrong

• No pilot runs: the live workflow was not reloaded, or the edge from act still goes to done. Run agentx workflow show owner-sweep and check the pilot step and its edges.
• The pilot changed something: autonomy: report is enforced only for agents on the claude-code tier. Use such an agent for the pilot, or take the tools that change things away from it.
• Live runs stay paused at pilot: the live run waits for the pilot to reach an end step. A pilot run that fails leaves the live run paused. Give every path in the pilot an end step, and cancel stuck runs with agentx workflow cancel <runId>.
• Not compared: … pilot run(s) without a live run: those pilot runs were started by hand, or their live run is older than --since. They are left out of the figures.
• No price known for … model call(s), with no Models with no price line: those traces are older than this version and their agent has no model in agentx.json. Add "model" to each agent the two workflows use, then run the report again. New traces record their model, so the line goes away for runs made after the update.
• Models with no price: …: the price table has rates for Claude models only. Calls on another model (for example an agent on a non-Claude tier) are priced at Sonnet rates, so treat that side's cost as an estimate.
• Costs look too low: check for the No price known line first. Then: only agent turns are priced. A step that calls a model some other way (for example action.builtin with extract.structured, or classify) leaves no trace with tokens, so it is not counted. Keep the pilot's model calls in agent steps.
• No trace database: run the command in the folder the daemon runs in, or pass --db <path>.