Skip to content

Fix/camera init#368

Merged
JohnYanxinLiu merged 3 commits into
developfrom
fix/camera-init
Jul 7, 2026
Merged

Fix/camera init#368
JohnYanxinLiu merged 3 commits into
developfrom
fix/camera-init

Conversation

@krrishj18

Copy link
Copy Markdown
Collaborator

I found the issue causing the drone to act stuck. It was cause the drone camera was being launched in mono and not stereo so droan_gl was failing. I did 15 multi-agent runs and none of them failed vs previously when testing for raven-multi i had over 50% failure rate.

In this file below which part of isaac-sim it'll default to mono if it doesn't get info from both right and left camera
/isaac-sim/exts/isaacsim.ros2.bridge/isaacsim/ros2/bridge/ogn/python/nodes/OgnROS2CameraInfoHelper.py

is_stereo = False # ← set: default mono
if not db.inputs.renderProductPath: # left retry-guard
db.per_instance_state.initialized = False
return False
if db.inputs.renderProductPathRight: # ← set: flips True IF right input is populated
is_stereo = True
...
if is_stereo: # ← used: gate for the whole right-eye branch
camera_info_right, camera_right = read_camera_info(db.inputs.renderProductPathRight)
... stereoRectify ...
add_camera_info_writer(... topicNameRight ...) # right writer only created here
add_camera_info_writer(... topicName ...) # left writer always created

in simulation/isaac-sim/extensions/PegasusSimulator/extensions/pegasus.simulator/pegasus/simulator/ogn/api/spawn_zed_camera.py
The current line
(f"{nodes['playback']}.outputs:tick", f"{nodes['info_helper']}.inputs:execIn"),

  - SOURCE = playback.outputs:tick — the OnPlaybackTick node's tick output, which fires once every simulation frame while the sim is playing.
  - DESTINATION = info_helper.inputs:execIn — the camera-info-helper's "run now" trigger.
  The problem: the two IsaacCreateRenderProduct nodes are wired to that same playback.tick (lines 222–223), so all three are triggered by one pulse
  with no guaranteed order between them:

  playback.tick ──┬──► left_create_rp.execIn     (builds LEFT render product)
                  ├──► right_create_rp.execIn     (builds RIGHT render product)
                  └──► info_helper.execIn          (reads both — but can run before RIGHT is ready)

  On the tick where the info-helper commits its one-time init, the right render product often isn't produced yet → db.inputs.renderProductPathRight is
  empty → is_stereo = False → right writer never created.

  The new line
(f"{right_nodes['create_rp']}.outputs:execOut", f"{nodes['info_helper']}.inputs:execIn"),

  - SOURCE = right_create_rp.outputs:execOut — a node's execOut fires after that node's compute() finishes. So this pulse fires only after the right
  render product has been created and its renderProductPath output is populated.
  - DESTINATION = same info_helper.inputs:execIn.
  playback.tick ──┬──► left_create_rp.execIn
                  └──► right_create_rp.execIn ──(execOut)──► info_helper.execIn
                                                              (now runs AFTER right product exists)

  Now when compute() reads db.inputs.renderProductPathRight, it's already valid → the Isaac node's own if db.inputs.renderProductPathRight: is_stereo =
  True evaluates True → right writer is built. 

@krrishj18 krrishj18 marked this pull request as ready for review July 3, 2026 18:56
@krrishj18 krrishj18 requested a review from JohnYanxinLiu July 3, 2026 18:56
@JohnYanxinLiu JohnYanxinLiu merged commit 6279bea into develop Jul 7, 2026
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants