Skip to content

pilotctl: render next-steps after every appstore call#396

Open
Alexgodoroja wants to merge 3 commits into
mainfrom
feat/next-steps-graph
Open

pilotctl: render next-steps after every appstore call#396
Alexgodoroja wants to merge 3 commits into
mainfrom
feat/next-steps-graph

Conversation

@Alexgodoroja

Copy link
Copy Markdown
Collaborator

What & why

Autonomous agents install an app and stop caring, because nothing in a call's output tells them where the flow goes. The product demo covers install→first-call, once. This covers every call after it: each pilotctl appstore call now ends by naming the small set of recommended next commands for where the agent stands — the next step on success, and the specific fix on failure (needs_signup → the gateway, 402 → balance, missing param → a corrected call).

This is the client half. The authoring half (schema, validation, CI gates, and the 19 backfilled graphs) is app-template#89. This PR is inert until an app's metadata.json actually carries a next_steps — nothing regresses when the field is absent, which is every app today.

$ pilotctl appstore call io.pilot.primitive primitive.send_email '{...}'
{ "needs_signup": true, "activate": "primitive.signup", ... }
next: you called primitive.send_email but this host has no account yet — the call
      soft-failed and nothing was sent; sign up once and it works from then on
  1. pilotctl appstore call io.pilot.primitive primitive.signup '{}'
     why: free, zero-argument, idempotent — then re-run the call above unchanged (required first)

Design

The data lives in the sha-verified metadata; this is only the insertion mechanism. Same discipline as the product-demo render: a pure resolver + renderer that knows nothing about any specific app, plus the hook.

  • No network on the call path. The graph is cached to $APP/next-steps.json at install, out of the already-sha-verified metadata.json. Every call is a local file read — no fetch, no DNS, no added latency, works offline. A call is the hottest path in the app store; it must not grow a network dependency to print a hint.
  • stderr, never stdout. stdout is the call's JSON result and agents pipe it into jq; one line of prose there breaks the very workflow this exists to encourage. Verified jq still parses cleanly. --json callers get a structured next_steps object instead of prose (on the error envelope for a failed call; on stderr for a successful one).
  • Failure steps drain through fatalHint via a package var, so they land after the error they fix — the same pattern motd's importantUpdate already uses, and necessary because only fatalHint knows where output ends (it owns the exit).
  • No trust surface added. The bytes came out of the sha-pinned metadata; $APP already holds the app's signed manifest. A graph produces text, never an action — nothing here ever executes a recommended command.
  • Fail-silent, recover()-guarded. Absent, malformed, future-schema or unmatched → the call behaves exactly as today. A hint is a nicety; a call is not. PILOT_NEXT_STEPS=off opts out.

Matching is over the outcome PAYLOAD

match tests the error text on failure and the result body on success. That is load-bearing, not a nicety: the scaffold's requireKey soft-fails an unauthenticated call with exit 0 and {"needs_signup":true}, and the CLI adapter wraps a non-zero tool exit into a successful call with {"exit":1,"stderr":...}. An outcome-only matcher would stay silent for precisely the cold agent who most needs the gateway named.

Note appstore call has exactly two observable outcomes — exit 0, or exit 1 via fatalHint. The exit code carries no classification, so everything finer must come from the payload.

Precedence

An edge that matched the actual situation beats one that merely matched the method name. Every discriminated edge outranks every undiscriminated one, however exact its from. This is a regression fix: scoring originally ranked from-exact above a match, so a cold agent calling primitive.send_email got the flow edge ("read your inbox") instead of the gateway ("sign up first").

The resolver is a deliberate duplicate of app-template's nextsteps.Resolve (pilotctl must not depend on that module — the same call appstore_demo.go's ProductDemo mirror makes). The cost of duplication is drift, and drift happened immediately: the two diverged on exactly the shadowing case, because the parity test didn't cover it. TestResolveMatchesAppTemplateSemantics now pins the shared cases, and I verified by mutation that re-introducing the old scoring fails the suite.

Verification

  • go test ./cmd/pilotctl/ green; gofmt/go vet clean.
  • Proven end-to-end against the live daemon and real apps, with the real authored graphs: a cold primitive.send_email renders the signup gateway marked (required first); a real sqlite.query missing-param failure renders the fix, and the recommended command then works.
  • Verified stdout still parses under jq, --json emits a structured envelope, and PILOT_NEXT_STEPS=off suppresses.

Alexgodoroja added 2 commits July 15, 2026 14:06
An agent installs an app and stops caring: nothing in a call's output says
where the flow goes next. The product demo covers install->first-call, once.
This covers every call after it.

Each `appstore call` now ends by naming the small set of recommended next
commands for where the agent stands: the next step in the flow on success,
and the specific fix on failure (402 -> top up, needs_signup -> the gateway,
missing param -> a corrected call).

The graph is authored in submission.json, rides into the catalogue's
metadata.json (pinned by metadata_sha256 under the catalogue signature), and
is cached at install to $APP/next-steps.json. The call path is therefore a
local file read: no fetch, no DNS, no added latency, works offline.

Matching is over the outcome PAYLOAD, not just the outcome: the error text on
failure, the result body on success. That is load-bearing. The scaffold's
requireKey wrapper soft-fails an unauthenticated call with exit 0 and
{"needs_signup":true,...}, so "you must sign up first" is not an error at all
-- an outcome-only matcher would stay silent for precisely the cold agent who
most needs the gateway named.

Output goes to stderr, never stdout: stdout is the JSON result agents pipe
into jq, and one line of prose there breaks the workflow this exists to
encourage. --json callers get a structured next_steps object instead.

Additive and fail-silent throughout: absent, malformed, future-schema or
unmatched means the call behaves exactly as before. A hint is a nicety; a
call is not.

- appstore_nextsteps.go: mirrored wire contract, resolver, renderer, cache
- appstore.go: hooks on both outcomes; success resolves before the review
  prompt can replace the body
- main.go: fatalHint drains exitNextSteps so steps land after the error they
  fix (same pattern motd's importantUpdate already uses)
- PILOT_NEXT_STEPS=off opts out
Ports the precedence fix from app-template's nextsteps.Resolve, which validates
the graphs this resolves. The two had drifted: this side still ranked from-exact
above a match, so for any method with both a flow edge and a soft-failing
gateway, the flow edge won — a cold agent calling primitive.send_email was told
to read its inbox instead of to sign up.

The parity test passed through the drift because it never covered exact-bare vs
wildcard-match, which is precisely the shadowing case. Added it; verified by
mutation that re-introducing the old scoring now fails the suite.

Proven end-to-end against the real app with the real authored graph: a cold
send_email now renders the signup gateway, marked (required first).
Comment thread cmd/pilotctl/appstore_nextsteps.go Fixed
Comment thread cmd/pilotctl/appstore_nextsteps.go Fixed
gosec flagged both ends as G703 path traversal, correctly. appID arrives
straight from argv, so filepath.Join alone is not containment:
`pilotctl appstore call ../../etc/x ...` resolves outside the install root and
we would read a file there and print pieces of it back as "next steps".

Use resolveUnder on both the read and the write — the same guard install and the
supervisor already rely on, whose own comment says filepath.Join does NOT block
traversal.

The regression test needed a second pass. Its first draft passed even with the
naive join restored: the graph's app-id-mismatch check rejected the planted file
first, so the test was green for the wrong reason and proved nothing. The
planted graph's `app` now equals the traversing id, leaving containment as the
only thing that can stop the load — verified by mutation that removing the guard
fails the suite.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants