Skip to content
All resources

What can this agent actually change?

Before an unattended run, name the task, the authority it needs, and the evidence that will establish its effects. Include scripts, descendants, alternate tools and background services. An allowed tool invocation and a final “done” message answer only part of that review.

Free local review aid · v0.2 · October 1, 2026

Download the harness review record (.md)

Complete it offline. It includes blank fields, checks marked NOT RUN, and a fictional timeline. It does not scan, enforce policy, stop processes or establish that a system is safe.

Choose the boundary you need to review

Start with one workflow and three to five relevant scenarios. The field guide compares existing controls and their limits; the scenario pack specifies fixtures, independent evidence, pass/fail/unknown conditions and a useful-task control for each case.

24 of 24 scenarios have a reference-fixture run (2026-10-23). Each scenario stays a specification. A run grades enforcement and the useful task separately against a fixture we built; a mocked boundary establishes only that fixture's behavior, not a deployed harness or product. The eight executed filesystem cases below are a separate lab.

Files, configuration and descendants

Can another representation or execution route change the same protected resource?

Compare shell, scripts, descendants and separate file tools. Review workspace files that can activate code later. Record transient changes as well as final state; restored bytes do not prove that nothing happened.

Scenario specifications HB-01–04 · 4 of 4 run passed both axes · run record

Process and service ownership

Which session owns the workload—and the server your test reached?

Bind processes and service endpoints to an owner and generation. Test busy ports, descendants, retained-service leases and truthful completion. An occupied port does not grant permission to stop its listener.

Scenario specifications HB-05–08 · 4 of 4 run passed both axes · run record

Network, credentials and remote authority

Does the allowed connection carry only the authority this task needs?

Distinguish mandatory routing, secret custody and receiver permissions. Check the tenant, object, action and output destination. An allowed domain or brokered credential can still permit an out-of-scope effect.

Scenario specifications HB-09–12 · 4 of 4 run passed both axes · run record

Budgets, retries and recovery

What bounds in-flight work when accounting or an acknowledgement is missing?

Compare actual charges with reservations across concurrent calls, retries and fallbacks. Reconcile uncertain operations before replay. Ending a local process does not establish that a remote job stopped or its cost was refunded.

Scenario specifications HB-13–16 · 0 of 4 run passed both axes · run record

Evidence quality and privacy

Would the conclusion survive a wrong tool response or missing trace?

Compare tool claims with independently observed effects, test capture loss and protect the observer's record. Grade detection separately from prevention: an alert or stopped process can follow an already committed effect.

Scenario specifications HB-17–20 · 4 of 4 run passed both axes · run record

Adapters, plugins and policy changes

Does an upgrade preserve authority and useful work?

Review adapter schemas, fallback routes, plugin provenance and changed grants. Keep terminal denials, transient failures and unknown outcomes distinct. Shared evidence should preserve correlation while excluding unnecessary sensitive payloads.

Scenario specifications HB-21–24 · 4 of 4 run passed both axes · run record

Record enforcement, evidence quality and useful-task completion separately. A missing receipt stays unknown. Correctly detecting a forbidden effect does not turn failed prevention into a pass.

One denial, another attempt, an unfinished server

This worked timeline is entirely fictional, NOT RUN. Task T-17 permits a parser patch and local tests. Its server belongs to session S-17.

  1. Attempt A-1 is denied. A hook refuses an out-of-scope write and records D-17. That decision covers this invocation.
  2. Attempt A-2 runs a script. The invocation passes. A filesystem boundary still refuses the same write inside the script; an independent observer records unchanged synthetic state.
  3. The agent says “finished.” The harness inventory still shows S-17's listener. A code-work summary does not establish that the server ended.
  4. The next port is occupied by S-18. S-17 selects another approved port. A port conflict gives it no authority to terminate another session's process.

Give each conclusion its evidence

Invocation decision
What request was inspected, under which effective policy? A hook decision does not inventory every operation inside a script.
Resource boundary
What prevents the effect across descendants and alternative tools? Name who can change the boundary and any unsandboxed fallback.
Independent observation
Who checked the file or receiving service, when, and with what coverage? Tool stdout remains a statement. Missing capture or an uncertain remote outcome stays visible.
Workload ownership
Which session owns the process, descendants, listener and teardown authority? A process name, PID or port alone is insufficient.

Compare the boundary with a real effect

Our separate filesystem lab ran eight fixed-script cases with synthetic files. Three forms passed a deliberately weak command filter; a read-only mount prevented their canary write while a separate workspace task completed. No model, coding client or process-lifecycle mechanism was tested.

Read the analysis, inspect the lab method and limits, or download the reproduction script (.py). It requires Docker and a pre-pulled Python image; it uses temporary synthetic fixtures and starts no persistent server.

Record what changes the next decision

To work through completion, retained services and unsettled costs, use the harness operations kit. It adds an executable reference lab and editable handoff records. Its checks are separate from the 24 proposed scenarios above.

Keep terminal denials separate from transient failures and unknown effects. Identify permitted recovery paths, operator diagnostics and exception expiry. Re-review model adapters, harness versions, plugin authority, credentials and budget controls when they change.

For the initial private-repository grant, use the access review. If an external action timed out, use the uncertain-action record. Keep private traces and credentials out of shared examples.