optimize: prewire task dependencies on orchestrator side#1263
Merged
Conversation
|
Important Review skippedAuto incremental reviews are disabled on this repository. Please check the settings in the CodeRabbit UI or the ⚙️ Run configurationConfiguration used: Organization UI Review profile: CHILL Plan: Pro Run ID: You can disable this status message by setting the Use the checkbox below for a quick retry:
📝 WalkthroughWalkthroughThis PR reworks the orchestrator/scheduler dependency-wiring pipeline: adding an explicit ChangesInline Wiring, Ready-State, and Wake-List Discovery
Estimated code review effort: 5 (Critical) | ~110 minutes Sequence Diagram(s)sequenceDiagram
participant Caller
participant submit_task_common
participant all_claimed_fanin_completed
participant try_orch_prewire_task
participant route_ready_once
participant WiringQueue
Caller->>submit_task_common: submit task
submit_task_common->>all_claimed_fanin_completed: check claimed producers
alt zero fanin or all completed
submit_task_common->>route_ready_once: route inline-ready
else fanin pending
submit_task_common->>try_orch_prewire_task: attempt prewire (dep_pool_lock)
alt prewire succeeds
try_orch_prewire_task-->>submit_task_common: wired
else prewire fails
submit_task_common->>WiringQueue: push task (spin if full)
end
end
sequenceDiagram
participant Task
participant on_task_complete
participant WakeListState
participant route_ready_once
participant pending_ready
Task->>on_task_complete: task finishes
on_task_complete->>on_task_complete: slot_state.mark_completed()
alt discovery enabled
on_task_complete->>WakeListState: drain_wake_list
WakeListState->>route_ready_once: route woken consumer
route_ready_once->>pending_ready: requeue if fanin missing
end
Possibly related PRs
Poem
🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
![gemini-code-assist[bot]](https://avatars.githubusercontent.com/in/956858?s=60&v=4){kind=small}
There was a problem hiding this comment.
Code Review
This pull request optimizes the task scheduling and wiring pipeline by allowing the orchestrator to inline already-ready tasks and prewire tasks directly, bypassing the scheduler's wiring queue. To support this, thread-safe locking is added to the dependency pool. Additionally, a disabled-by-default "discovery" wake-up mechanism is introduced to optimize task wake-ups, and dummy ready queue draining is parallelized across multiple scheduler threads. There are no review comments, so I have no feedback to provide.
Important
The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 1
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@src/a2a3/runtime/tensormap_and_ringbuffer/runtime/pto_orchestrator.cpp`:
- Around line 906-925: Preserve the dep_pool_mark for inline-ready tasks in PTO
orchestrator flow: in the inline-ready branches around route_orch_inline_ready
and try_orch_prewire_task, the slot is being made ready without going through
wire_task(), so the oldest live slot can end up with a zero mark and block
PTO2DepListPool::reclaim(). Update the inline path to assign the same
dep_pool_mark that wire_task() would have established before marking the task
ready, using the existing fanin_builder/current slot state helpers in
pto_orchestrator.cpp so dep-pool tail advancement still works when an inline
task becomes the head of the live window.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro
Run ID: 73457185-c6e8-4a0a-b250-b01562f2d41a
📒 Files selected for processing (5)
src/a2a3/runtime/tensormap_and_ringbuffer/runtime/pto_orchestrator.cppsrc/a2a3/runtime/tensormap_and_ringbuffer/runtime/pto_runtime2_types.hsrc/a2a3/runtime/tensormap_and_ringbuffer/runtime/scheduler/pto_scheduler.hsrc/a2a3/runtime/tensormap_and_ringbuffer/runtime/scheduler/scheduler_dispatch.cppsrc/a2a3/runtime/tensormap_and_ringbuffer/runtime/shared/pto_runtime2_init.cpp
4 tasks
TaoZQY
added a commit
to Crane-Liu/simpler
that referenced
this pull request
Jul 8, 2026
Resolve a2a3 Orch-side wiring conflicts with direct-only early-dispatch. Keep wire_fanin_task and mark_dep_pool_position owned by Orch, and pack allow_early_resolve into ready_state to preserve the 64-byte slot_state. Co-authored-by: Crane-Liu <c.wliu@outlook.com>
TaoZQY
added a commit
to Crane-Liu/simpler
that referenced
this pull request
Jul 8, 2026
Resolve the a2a3 early-dispatch conflict while keeping allow_early_resolve as an independent PTO2TaskSlotState bool. Move the deferred-completion marker into ready_state so the slot layout stays 64 bytes, and keep the new upstream alloc-creator early-dispatch behavior on the Orch-side wire path. Co-authored-by: Crane-Liu <c.wliu@outlook.com>
ChaoWao
added a commit
to Crane-Liu/simpler
that referenced
this pull request
Jul 9, 2026
…eap/task detection Rebase of PR hw-native-sys#1263 (20260702_PP) onto current main, plus follow-up fixes to the dep-pool deadlock diagnostic and the swimlane. Rebased content (PR hw-native-sys#1263): - Move fanin/fanout wiring out of the scheduler's SPSC wiring queue into the orchestrator submit path (wire_fanin_task / orch_wire_live_fanin_task). Drop PTO2SpscQueue, WiringState, drain_wiring_queue, wire_task, and the scheduler-side wiring-deadlock detector. - Serialize ready publication with a per-slot claim-once ready_state (a2a3); a5 keeps the exact-refcount single-pusher (no early-dispatch). - Reconcile with main hw-native-sys#1297's rename (SpecReleaseSink -> EarlyDispatchReleaseSink, try_speculative_release -> try_early_dispatch_release, spec_state -> early_dispatch_state). Fix 1 - dep-pool deadlock detection: the orchestrator's dep-pool wiring wait latched PTO2_ERROR_DEP_POOL_OVERFLOW with no structured report, unlike the heap/task-window allocator that shares the same last_task_alive watermark. Route orch_wire_live_fanin_task through PTO2DepListPool::ensure_space and give it the allocator's detection: an immediate structural head-of-line check (head COMPLETED, all consumers released, scope still open) plus a wall-clock PTO2_ALLOC_DEADLOCK_TIMEOUT_CYCLES backstop, each emitting a scope_gated-aware report_deadlock with the head-task dump and the PTO2_RING_DEP_POOL sizing hint (slot_states null-guarded like the allocator). Supersedes the ea65ecd scheduler-timeout deferral in favor of parity with heap/task/fanin-spill detection. Removes the dead PTO2_DEP_POOL_SPIN_LIMIT. Fix 2 - swimlane: the scheduler Wire phase is no longer emitted (wiring left the scheduler). Remove the L2SwimlaneSchedPhaseKind::Wire enumerator (slot 3 now unused; Dummy=4.. keep their trace values), its collector name case, and the swimlane_converter "wire" mapping. Orch-side wiring has no swimlane lane; its cost is g_orch_fanin_cycle in the device-log orch breakdown -- documented in l2-swimlane-profiling.md. Fix scope-stats.md (snapshots are now orchestrator-published, wiring synchronous). Validation: full a2a3+a5 build (onboard+sim); cpput test_wiring, test_a5_wiring, test_dep_list_pool, test_task_allocator pass; a2a3sim dependency-heavy scene tests + runtime_fatal_codes[dep_pool_overflow] + --enable-l2-swimlane (level 4) pass; onboard a2a3 dependency-heavy scene tests pass on silicon. Co-authored-by: TaoZQY <zhangtaolqy@mail.ustc.edu.cn> Co-authored-by: Crane-Liu <c.wliu@outlook.com> Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…eap/task detection Rebase of PR hw-native-sys#1263 (20260702_PP) onto current main, plus follow-up fixes to the dep-pool deadlock diagnostic and the swimlane. Rebased content (PR hw-native-sys#1263): - Move fanin/fanout wiring out of the scheduler's SPSC wiring queue into the orchestrator submit path (wire_fanin_task / orch_wire_live_fanin_task). Drop PTO2SpscQueue, WiringState, drain_wiring_queue, wire_task, and the scheduler-side wiring-deadlock detector. - Serialize ready publication with a per-slot claim-once ready_state (a2a3); a5 keeps the exact-refcount single-pusher (no early-dispatch). - Reconcile with main hw-native-sys#1297's rename (SpecReleaseSink -> EarlyDispatchReleaseSink, try_speculative_release -> try_early_dispatch_release, spec_state -> early_dispatch_state). Fix 1 - dep-pool deadlock detection: the orchestrator's dep-pool wiring wait latched PTO2_ERROR_DEP_POOL_OVERFLOW with no structured report, unlike the heap/task-window allocator that shares the same last_task_alive watermark. Route orch_wire_live_fanin_task through PTO2DepListPool::ensure_space and give it the allocator's detection: an immediate structural head-of-line check (head COMPLETED, all consumers released, scope still open) plus a wall-clock PTO2_ALLOC_DEADLOCK_TIMEOUT_CYCLES backstop, each emitting a scope_gated-aware report_deadlock with the head-task dump and the PTO2_RING_DEP_POOL sizing hint (slot_states null-guarded like the allocator). Supersedes the ea65ecd scheduler-timeout deferral in favor of parity with heap/task/fanin-spill detection. Removes the dead PTO2_DEP_POOL_SPIN_LIMIT. Fix 2 - swimlane: the scheduler Wire phase is no longer emitted (wiring left the scheduler). Remove the L2SwimlaneSchedPhaseKind::Wire enumerator (slot 3 now unused; Dummy=4.. keep their trace values), its collector name case, and the swimlane_converter "wire" mapping. Orch-side wiring has no swimlane lane; its cost is g_orch_fanin_cycle in the device-log orch breakdown -- documented in l2-swimlane-profiling.md. Fix scope-stats.md (snapshots are now orchestrator-published, wiring synchronous). Validation: full a2a3+a5 build (onboard+sim); cpput test_wiring, test_a5_wiring, test_dep_list_pool, test_task_allocator pass; a2a3sim dependency-heavy scene tests + runtime_fatal_codes[dep_pool_overflow] + --enable-l2-swimlane (level 4) pass; onboard a2a3 dependency-heavy scene tests pass on silicon. Co-authored-by: TaoZQY <zhangtaolqy@mail.ustc.edu.cn> Co-authored-by: Crane-Liu <c.wliu@outlook.com> Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
ChaoZheng109
reviewed
Jul 9, 2026
| // signalled inline here. The ready queue is MPMC, and the fanout path | ||
| // uses per-slot locks/atomics, so multiple scheduler threads can share | ||
| // the dependency-only resolve work. | ||
| if (thread_idx < 3) { |
Collaborator
There was a problem hiding this comment.
进入调度逻辑的线程一定是小于3的，这里是多余的判断
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Move dependency wiring to the orchestrator side for the a2a3 and a5
tensormap_and_ringbufferruntimes, and remove the old scheduler-side deferred wiring fallback.The latest cleanup makes this PR a single Orch-side wiring design instead of a mixed O/S fallback path:
Modified Code
a2a3 runtime
src/a2a3/runtime/tensormap_and_ringbuffer/runtime/pto_orchestrator.cppsrc/a2a3/runtime/tensormap_and_ringbuffer/runtime/scheduler/pto_scheduler.hsrc/a2a3/runtime/tensormap_and_ringbuffer/runtime/scheduler/scheduler_dispatch.cppsrc/a2a3/runtime/tensormap_and_ringbuffer/runtime/scheduler/scheduler_cold_path.cppsrc/a2a3/runtime/tensormap_and_ringbuffer/runtime/shared/pto_runtime2_init.cppsrc/a2a3/runtime/tensormap_and_ringbuffer/runtime/pto_runtime2.{cpp,h}src/a2a3/runtime/tensormap_and_ringbuffer/runtime/pto_runtime2_types.hsrc/a2a3/runtime/tensormap_and_ringbuffer/runtime/pto_ring_buffer.hsrc/a2a3/runtime/tensormap_and_ringbuffer/runtime/scheduler/{scheduler_context.h,scheduler_types.h}a5 runtime
src/a5/runtime/tensormap_and_ringbuffer/runtime/.Shared profiling / tooling
src/a2a3/platform/include/common/l2_swimlane_profiling.hsrc/a5/platform/include/common/l2_swimlane_profiling.hsimpler_setup/tools/swimlane_converter.pyThese were updated so the old scheduler-wire swimlane/profiling label is treated as legacy scheduler wire phase metadata rather than an active deferred wiring phase.
Documentation Updates
Updated both a2a3 and a5 runtime docs:
src/a2a3/runtime/tensormap_and_ringbuffer/docs/RUNTIME_LOGIC.mdsrc/a2a3/runtime/tensormap_and_ringbuffer/docs/MULTI_RING.mdsrc/a2a3/runtime/tensormap_and_ringbuffer/docs/SUBMIT_BY_CLUSTER.mdsrc/a5/runtime/tensormap_and_ringbuffer/docs/RUNTIME_LOGIC.mdsrc/a5/runtime/tensormap_and_ringbuffer/docs/MULTI_RING.mdsrc/a5/runtime/tensormap_and_ringbuffer/docs/SUBMIT_BY_CLUSTER.mdThe docs now describe:
Test Updates
tests/ut/cpp/a2a3/test_wiring.cpptests/ut/cpp/a5/test_wiring.cppUpdated wiring tests to exercise Orch-side helpers directly.
Removed obsolete tests for the deleted SPSC wiring queue:
tests/ut/cpp/a2a3/test_spsc_queue.cpptests/ut/cpp/a5/test_spsc_queue.cppUpdated:
tests/ut/cpp/CMakeLists.txtCurrent Behavior
wfanin == 0: Orch seeds fanin state, records the dep-pool position, and routes the task ready.wfanin > 0and all producers are already complete: Orch seeds fanin/dispatch state and routes the task ready.wfanin > 0with unfinished producers: Orch wires fanout entries through the shared wiring helper before the task becomes visible to schedulers.Results
Previous 100-round Strategy Sweep
Benchmark:
qwen3_14b_decode,StressBatch16Seq3500,a2a3, device 4,tensormap_and_ringbuffer, 100 rounds per run. Values below are trimmed averages, dropping 10 low and 10 high rounds.Negative delta means faster.
simpler-base)Readout:
Latest pypto-lib qwen14b Decode 40L Check
Command:
task-submit --device 0 --max-time 0 --timeout 0 --run 'cd /data/pyptouser/zhangtao/zt/pypto-lb && .venv-bench/bin/python models/qwen3/14b/decode_layer.py -p a2a3 -d $TASK_DEVICE --validate-fwd --fwd-layers 40'Result from
after_lock_cleanup_qwen14b_40l_20260703_162637_dev0.log:device_wallorchschedrunner_runCorrectness:
argmax match 16/16sample match 16/16logits 100.0000% within 5e-2max_abs_err=0.0199Compared with the earlier device-0 baseline average for this 40L check,
device_wallandschedare within about 0.5%, so the cleanup does not show an obvious performance regression.Tests
git diff --checkcmake --build tests/ut/cpp/build -j$(nproc)ctest --test-dir tests/ut/cpp/build --output-on-failure -R '^(test_wiring|test_a2a3_orchestrator_fanin|test_a5_wiring|test_a5_orchestrator_fanin)$'models/qwen3/14b/decode_layer.py -p a2a3 -d $TASK_DEVICE --validate-fwd --fwd-layers 40