MAINT: Unify scenario initialize_async's into common parameter bag#2132
Merged
rlundeen2 merged 5 commits intoJul 9, 2026
Merged
Conversation
Make Scenario.initialize_async argument-free by reading all common run inputs from a single param bag (self.params), and share the param-bag create-and-configure flow between the scenario and initializer registries. Resolve objective_target names against TargetRegistry the same way the constructor path resolves converter/scorer references, removing the confusing "must be a PromptTarget instance" rejection. Update scanner and registry/scenario docs plus tests to the new API. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Fix pyrit_scan --start-server hanging any caller that captures its output: the detached pyrit_backend inherited the parent's stdout, so the pipe never reached EOF. Redirect the child's stdout/stderr to a temp log file (pyrit_backend.log) and reference it in the error paths. Re-execute and clean up the affected doc notebooks, add server start/stop cells to 1_pyrit_scan, and align target-catalog model_var names in the initializer + .env_example. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
…gistry-init # Conflicts: # tests/unit/scenario/airt/test_cyber.py # tests/unit/scenario/airt/test_scam.py # tests/unit/scenario/garak/test_doctor.py
behnam-o
approved these changes
Jul 8, 2026
behnam-o
left a comment
behnam-o
left a comment
Contributor
There was a problem hiding this comment.
looks good. some little comments
…async Address PR review: replace the super().supported_parameters() + [...] override pattern with an additional_parameters() hook the base composes into supported_parameters (common inputs + additional), so the common "just add a parameter" case can't silently drop the common inputs. supported_parameters remains overridable as the escape hatch for removing/replacing a common input. Mark initialize_async @Final since no subclass overrides it. Migrate Scam, TextAdaptive, and AdversarialBenchmark onto the new hook, update the scenario instructions and 2_custom_scenario_parameters doc, and add tests for the compose/remove paths. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
…gistry-init # Conflicts: # doc/code/scenarios/3_adaptive_scenarios.ipynb # doc/code/scenarios/3_adaptive_scenarios.py # pyrit/scenario/scenarios/foundry/red_team_agent.py # tests/unit/scenario/foundry/test_red_team_agent.py
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Unifies how scenarios receive configuration by collapsing
initialize_async's typed keyword arguments into the single param-bag path already used for scenario-specific parameters.What changed
Scenariodeclares the common, run-resolved inputs (objective_target,scenario_strategies,dataset_config,memory_labels,include_baseline) as a default parameter list surfaced fromsupported_parameters(); subclasses extend rather than replace it.set_params_from_args->resolve_declared_paramspath, then calls an argument-freeinitialize_async.initialize_asyncreads its inputs fromself.paramsinstead of its own signature.objective_targetnames are resolved to instances at the registry layer (viaTargetRegistry) before the scenario is initialized, with a clearer error when a name isn't registered.ScenarioContextboundary is unchanged — only the source of the common inputs moves (param bag, not a typed signature).This is Phase F of the scenario-simplification effort. Design and rationale (including the newly added Phase H tracking two pre-existing adaptive-scenario bugs found while validating this work): https://gist.github.com/rlundeen2/d2fa5f91eb411ba9b1d0bbf98412f700
Also folds in a fix for
pyrit_scan --start-serverhanging any caller that captured its output (the detached backend inherited the parent's stdout) and re-validates the affecteddoc/notebooks.