feat: nav2 sensor fix OTA demo#58
Conversation
bburda
changed the title
bburda
force-pushed
the
feat/ota-nav2-sensor-fix
branch
from
39a0925 to
fff9581
Compare
bburda
changed the title
bburda
force-pushed
the
feat/ota-nav2-sensor-fix
branch
from
fff9581 to
7a37cc6
Compare
…pdate, publish-and-apply a hotfix Dev-grade OTA demo on the ros2_medkit gateway's ota_update_plugin and the SOVD /updates resource. An RB-Theron AMR runs Nav2 in an AWS small-warehouse world. A regressing lidar update (broken_lidar_3_0_0) is auto-applied at boot; a few metres into a mission the scan sensor develops a stuck sector, Nav2 can no longer make progress, and navigate_to_pose aborts. Generic log/action-status bridges surface that as SOVD faults on bt-navigator and controller-server with a freeze-frame and an MCAP capture - the sensor node never reports itself. The operator diagnoses over SOVD: downloads the MCAP, runs a suite of health-check operations (lidar/localization/drivetrain/costmap) to confirm the lidar is the cause, then publishes the forward hotfix (fixed_lidar_3_0_1) as a hand-provided update descriptor via POST /updates and applies it (prepare + execute), and clears the latched faults. A custom nav_to_pose behaviour tree keeps the abort prompt. Foxglove panels + curl scripts drive the loop; smoke tests cover the plugin, the SOVD envelope, the publish-then-apply flow, and the single-publisher /scan remap. Runs on CycloneDDS (FastDDS segfaults amcl/controller_server on Jazzy).
bburda
force-pushed
the
feat/ota-nav2-sensor-fix
branch
from
7a37cc6 to
7a85e20
Compare
The default recovery (spin, wait 5s, back-up) made a stuck robot thrash for 20-30s before navigate_to_pose aborted. Strip the recovery RoundRobin down to a costmap clear (no motion), keep the top-level retry so the goal still aborts, and shorten the progress-checker window (movement_time_allowance 10 -> 4s). The robot now drives, stalls on the phantom, and aborts in ~15-19s with no spinning.
Group the health-check app under a new 'Health Checks' function (category diagnostics) so the differential-diagnosis operations (lidar / localization / drivetrain / costmap) show up as a first-class functional grouping in the SOVD entity tree, alongside Fleet Diagnostics.
…e gz race The gz_ros2_control cold start races the controller spawners; when it loses, the robot has no odometry/TF and Nav2 aborts every goal instantly. run-demo.sh now waits for the odom->base_footprint TF to actually flow before reporting the demo up, and restarts the sim (bounded to 3 tries) if it does not - so one run-demo.sh hands off a genuinely drive-ready robot instead of leaving it to chance.
… health checks into Fleet Diagnostics The fault_manager defaults snapshots.rosbag.topics to 'entity', which narrows the captured bag to the fault source node's subgraph (bt_navigator: /odom /tf /bond /navigate_to_pose/*) and drops /scan, /cmd_vel and /local_costmap - so the downloaded MCAP had none of the diagnostic topics and Foxglove playback showed nothing. Set topics 'explicit' (write exactly include_topics) + qos_match so the capture records /scan (the phantom), /cmd_vel, /tf and the local costmap. Also add the health-check app to the Fleet Diagnostics function so its operations surface there (dropped the redundant standalone function).
…cute The descriptor shipped automated:false, which the Updates panel surfaces as 'not automated' and uses to disable the Execute button after a completed Prepare. The OTA plugin does apply the update itself on execute (kill + respawn), so it is an automated application - set automated:true. Verified: Prepare alone does NOT auto-swap (broken stays), Execute swaps to fixed - the two-step operator flow is preserved.
…abort BT Add a latched_relay node that re-publishes the latched /robot_description (URDF) and /tf_static at 2 Hz so the fault_manager's post-hoc volatile rosbag capture records them - without them a downloaded MCAP had no robot mesh and no static TF to place /scan against, so playback showed only floating obstacles. Add /robot_description to the capture include_topics. Also bake the nav2 default behavior-tree overwrite into Dockerfile.gateway (was only applied at runtime) so a clean build ships the fast-abort tree.
…hecks operation The 4 std_srvs/Trigger checks each showed a meaningless auto-generated 'structure_needs_at_least_one_member' request field and needed 4 separate clicks. Replace them with a single health_check_msgs/RunHealthChecks service: request bools lidar/localization/drivetrain/costmap (default true) select which checks to run, response returns ok / checks_run / checks_failed and a per-check report. One operator operation, meaningful request fields, richer diagnostic output; the per-check logic and message wording are unchanged.
The relay called declare_parameter('use_sim_time') but rclpy auto-declares that
parameter and the launch already sets it via -p, so the node raised
ParameterAlreadyDeclaredException and died on startup every boot. Rely on the
launch-provided value instead of re-declaring.
…in the MCAP supports_automated() returned false, so the UI's combined 'Prepare & execute' reported 'package does not support automatic updates' even though the plugin applies updates itself on execute - return true. Also capture /local_costmap/published_footprint so MCAP playback shows the robot's outline and pose; the URDF mesh cannot render on replay (Foxglove resolves package:// meshes via the live foxglove_bridge asset service, absent in a bag), so the footprint + /tf frames are how you see where the robot is in the recording.
…pose freshness AMCL only republishes /amcl_pose on a motion-triggered filter update, so once the robot reaches its goal and sits still the pose goes stale even though AMCL is still localized (it keeps broadcasting map->odom). The freshness check then reported 'AMCL pose stale/absent' after the OTA fix, collapsing the whole post-fix 'all healthy' result on a false positive. Judge the cached pose's xy covariance instead - small = converged, regardless of age; a real divergence still trips the bound.
… medkit workspace The gateway image ran a bare 'colcon build' over the full 15-package ros2_medkit clone - including ros2_medkit_opcua (open62541pp, very slow to compile), integration_tests, and the discovery beacons the demo never loads (it only loads ota_update_plugin). Restrict the build to --packages-up-to the four medkit packages the demo actually runs (gateway, fault_manager, log/action-status bridges) plus the demo's own packages, so their deps come in transitively and everything else is skipped.
There was a problem hiding this comment.
Pull request overview
This PR adds a new end-to-end “OTA over SOVD” demo (demos/ota_nav2_sensor_fix/) that exercises the /updates lifecycle against a live ros2_medkit gateway, including a dev-grade OTA plugin, an artifact server, and ROS 2 demo packages used to demonstrate a forward hotfix of a regressed lidar node.
Changes:
- Introduces a complete OTA demo stack (gateway image + C++ OTA plugin + FastAPI artifact server + ROS 2 demo packages + operator scripts).
- Adds automated validation (new OTA smoke tests + CI jobs) to cover
/updatesbehavior, spec-shaped payloads, and process swapping. - Documents the new demo in the repository README and includes third-party notice pinning for build-fetched assets.
Reviewed changes
Copilot reviewed 75 out of 77 changed files in this pull request and generated 2 comments.
Show a summary per file
| File | Description |
|---|---|
tests/smoke_test_ota.sh |
New host-driven smoke test for the OTA demo’s /updates and process swap flow |
tests/smoke_lib.sh |
Makes smoke harness exit code reflect recorded failures (and preserve crash exit codes) |
README.md |
Adds the new OTA demo and includes it in the smoke test list |
demos/ota_nav2_sensor_fix/updates/README.md |
Documents manual publishing of updates via POST /updates |
demos/ota_nav2_sensor_fix/updates/fixed_lidar_3_0_1.json |
Ships the hotfix update descriptor used by publish/apply scripts |
demos/ota_nav2_sensor_fix/trigger-bad-update.sh |
Script to re-apply the regressing update via prepare/execute |
demos/ota_nav2_sensor_fix/THIRD_PARTY_NOTICES.md |
Documents pinned upstream assets fetched at image build time |
demos/ota_nav2_sensor_fix/stop-demo.sh |
Demo teardown helper (compose down + optional cleanup) |
demos/ota_nav2_sensor_fix/send-goal.sh |
Helper to publish a Nav2 goal inside the gateway container |
demos/ota_nav2_sensor_fix/scripts/test_pack_artifact.py |
Pytest coverage for pack_artifact.py |
demos/ota_nav2_sensor_fix/scripts/pyrightconfig.json |
Pyright config for demo scripts |
demos/ota_nav2_sensor_fix/scripts/pack_artifact.py |
CLI to build tarballs + SOVD-shaped catalog/update entries |
demos/ota_nav2_sensor_fix/scripts/e2e_webui_smoke.mjs |
Optional Playwright-based E2E smoke driver for the web UI flow |
demos/ota_nav2_sensor_fix/scripts/conftest.py |
Pytest path setup for local script tests |
demos/ota_nav2_sensor_fix/scripts/build_artifacts.sh |
Optional host-side artifact build helper for maintainers |
demos/ota_nav2_sensor_fix/scripts/.gitignore |
Ignores local Python/pytest artifacts for scripts |
demos/ota_nav2_sensor_fix/run-demo.sh |
Main runner for building/starting the demo stack |
demos/ota_nav2_sensor_fix/ros2_ws/.gitignore |
Ignores ROS 2 workspace build/install/log trees |
demos/ota_nav2_sensor_fix/ros2_packages/ota_nav2_sensor_fix_demo/urdf/warehouse_rbtheron.urdf.xacro |
RB-Theron URDF wrapper to avoid closed controller config and keep sensors |
demos/ota_nav2_sensor_fix/ros2_packages/ota_nav2_sensor_fix_demo/package.xml |
Package metadata for the demo launch/config glue |
demos/ota_nav2_sensor_fix/ros2_packages/ota_nav2_sensor_fix_demo/models/person_obstacle/model.sdf |
Simple box obstacle model used in the simulation narrative |
demos/ota_nav2_sensor_fix/ros2_packages/ota_nav2_sensor_fix_demo/models/person_obstacle/model.config |
Model config for person_obstacle |
demos/ota_nav2_sensor_fix/ros2_packages/ota_nav2_sensor_fix_demo/models/person_human/model.sdf |
Human-like obstacle model built from primitives (mesh-free) |
demos/ota_nav2_sensor_fix/ros2_packages/ota_nav2_sensor_fix_demo/models/person_human/model.config |
Model config for person_human |
demos/ota_nav2_sensor_fix/ros2_packages/ota_nav2_sensor_fix_demo/models/aws_small_warehouse/worlds/warehouse.sdf |
gz-port world file referencing AWS warehouse models |
demos/ota_nav2_sensor_fix/ros2_packages/ota_nav2_sensor_fix_demo/config/warehouse_map.yaml |
Map server YAML for the warehouse environment |
demos/ota_nav2_sensor_fix/ros2_packages/ota_nav2_sensor_fix_demo/config/ros_gz_bridge.yaml |
Bridge config, including /scan_sim mapping to avoid /scan stomping |
demos/ota_nav2_sensor_fix/ros2_packages/ota_nav2_sensor_fix_demo/config/navigate_to_pose_fast_abort.xml |
Nav2 BT tuned to abort faster for the demo narrative |
demos/ota_nav2_sensor_fix/ros2_packages/ota_nav2_sensor_fix_demo/config/nav2_params.yaml |
Nav2 params tailored for the RB-Theron + warehouse scenario |
demos/ota_nav2_sensor_fix/ros2_packages/ota_nav2_sensor_fix_demo/config/controllers.yaml |
ros2_control controller config for diff-drive + joint states |
demos/ota_nav2_sensor_fix/ros2_packages/ota_nav2_sensor_fix_demo/CMakeLists.txt |
Installs launch/config/URDF/models/maps into the package share dir |
demos/ota_nav2_sensor_fix/ros2_packages/health_check/src/health_check_node.cpp |
Operator-invoked Trigger services for differential diagnosis checks |
demos/ota_nav2_sensor_fix/ros2_packages/health_check/package.xml |
health_check package manifest |
demos/ota_nav2_sensor_fix/ros2_packages/health_check/CMakeLists.txt |
Builds/installs health_check_node |
demos/ota_nav2_sensor_fix/ros2_packages/fixed_lidar/src/fixed_lidar_node.cpp |
Post-fix clean /scan_sim → /scan republisher |
demos/ota_nav2_sensor_fix/ros2_packages/fixed_lidar/package.xml |
fixed_lidar package manifest |
demos/ota_nav2_sensor_fix/ros2_packages/fixed_lidar/CMakeLists.txt |
Builds/installs fixed_lidar_node |
demos/ota_nav2_sensor_fix/ros2_packages/broken_lidar/src/broken_lidar_node.cpp |
Regressed scan node that overlays a phantom sector after a driven distance |
demos/ota_nav2_sensor_fix/ros2_packages/broken_lidar/package.xml |
broken_lidar package manifest |
demos/ota_nav2_sensor_fix/ros2_packages/broken_lidar/CMakeLists.txt |
Builds/installs broken_lidar_node |
demos/ota_nav2_sensor_fix/README.md |
Full demo narrative + curl equivalents + Foxglove walkthrough |
demos/ota_nav2_sensor_fix/publish-fix.sh |
Registers the hotfix via POST /updates |
demos/ota_nav2_sensor_fix/ota_update_server/tests/test_main.py |
Tests for update server catalog + artifact endpoints |
demos/ota_nav2_sensor_fix/ota_update_server/tests/__init__.py |
Marks test package |
demos/ota_nav2_sensor_fix/ota_update_server/pyrightconfig.json |
Pyright config for the update server |
demos/ota_nav2_sensor_fix/ota_update_server/pyproject.toml |
Python packaging for the update server |
demos/ota_nav2_sensor_fix/ota_update_server/ota_update_server/main.py |
FastAPI app serving /catalog and /artifacts/* |
demos/ota_nav2_sensor_fix/ota_update_server/ota_update_server/__init__.py |
Exposes create_app |
demos/ota_nav2_sensor_fix/ota_update_server/Dockerfile |
Multi-stage build: builds demo artifacts, then serves via FastAPI |
demos/ota_nav2_sensor_fix/ota_update_server/.gitignore |
Ignores venv/build artifacts |
demos/ota_nav2_sensor_fix/ota_update_plugin/test/test_plugin_smoke.cpp |
GTest suite validating plugin behaviors and ProcessRunner spawn failure |
demos/ota_nav2_sensor_fix/ota_update_plugin/test/test_operation_dispatcher.cpp |
Tests operation-kind classification from metadata component arrays |
demos/ota_nav2_sensor_fix/ota_update_plugin/test/test_catalog_client.cpp |
Tests URL parsing logic for the CatalogClient |
demos/ota_nav2_sensor_fix/ota_update_plugin/src/process_runner.hpp |
Declares process-management seam used by the plugin |
demos/ota_nav2_sensor_fix/ota_update_plugin/src/process_runner.cpp |
Implements pgrep/kill/spawn with cmdline-based matching and double-fork |
demos/ota_nav2_sensor_fix/ota_update_plugin/src/plugin_exports.cpp |
Gateway plugin entrypoints + UpdateProvider accessor |
demos/ota_nav2_sensor_fix/ota_update_plugin/src/operation_dispatcher.hpp |
Defines operation-kind classification API |
demos/ota_nav2_sensor_fix/ota_update_plugin/src/operation_dispatcher.cpp |
Implements operation-kind classification |
demos/ota_nav2_sensor_fix/ota_update_plugin/src/catalog_client.hpp |
Declares catalog/artifact HTTP client interface |
demos/ota_nav2_sensor_fix/ota_update_plugin/src/catalog_client.cpp |
Implements HTTP GET for /catalog and artifact downloads |
demos/ota_nav2_sensor_fix/ota_update_plugin/package.xml |
Plugin package manifest |
demos/ota_nav2_sensor_fix/ota_update_plugin/include/ota_update_plugin/ota_update_plugin.hpp |
Plugin class interface (GatewayPlugin + UpdateProvider) |
demos/ota_nav2_sensor_fix/ota_update_plugin/CMakeLists.txt |
Builds static core lib + dlopen-loaded module + tests |
demos/ota_nav2_sensor_fix/manifest.yaml |
Demo manifest defining components/apps/functions (hybrid discovery) |
demos/ota_nav2_sensor_fix/gateway_config.yaml |
Gateway config enabling updates and loading ota_update_plugin |
demos/ota_nav2_sensor_fix/entrypoint.sh |
Launches demo and auto-applies the bad update at boot |
demos/ota_nav2_sensor_fix/Dockerfile.gateway |
Builds the full gateway + Nav2 + gz-sim + demo packages image |
demos/ota_nav2_sensor_fix/docker-compose.yml |
Two-service stack: gateway + update server |
demos/ota_nav2_sensor_fix/clear-fault.sh |
Clears the latched faults after applying the hotfix |
demos/ota_nav2_sensor_fix/check-demo.sh |
Displays current update/fault/process state for the operator |
demos/ota_nav2_sensor_fix/artifacts/.gitignore |
Ignores locally built artifacts |
demos/ota_nav2_sensor_fix/apply-fix.sh |
Applies the published fix via prepare/execute |
.github/workflows/ci.yml |
Adds CI jobs to build/run the OTA demo and execute smoke tests |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| parser.add_argument( | ||
| "--version", | ||
| default="0.0.0", | ||
| help="Semantic version of the artifact (pass '' for uninstall).", | ||
| ) |
| ProcessRunner(ProcessRunner &&) = delete; | ||
| ProcessRunner & operator=(ProcessRunner &&) = delete; | ||
|
|
||
| /// Find PIDs of processes whose /proc/<pid>/comm matches the given basename. |
smoke_test_ota.sh asserted broken_lidar_node was the live scan_sensor_node with a one-shot pgrep, but the entrypoint applies broken_lidar_3_0_0 asynchronously after the catalog registers, so the check raced the async apply and flaked. Gate on broken_lidar_3_0_0 reaching 'completed' before the "Initial process state" section, and poll the process assertions instead of checking once (matches the boot gate smoke_test_demo_narrative.sh already uses).
The gateway container multiplexes ~15 nodes into one stdout, so 'docker compose logs gateway --tail=N' on failure showed only discovery chatter and never the nav2 goal handling. Add tests/dump_ota_diagnostics.sh and run it from both OTA jobs on failure: it captures the full compose log plus ROS introspection (lifecycle, /navigate_to_pose server, /scan, /amcl_pose, TF, controllers, faults) and uploads it as an artifact.
… publish send-goal.sh published /goal_pose once with 'ros2 topic pub --once' from a transient docker exec node. On a cold CI runner that single volatile publish raced DDS discovery and was dropped before bt_navigator matched the subscription, so nav2 never drove, never aborted, and the reactive-fault scenario raised nothing (ota-demo-narrative timed out waiting for ACTION_NAVIGATE_TO_POSE_ABORTED). Send through the /navigate_to_pose action instead: the client waits for the server and confirms the goal is accepted, guaranteeing delivery, and returns on acceptance so a healthy-resume goal does not block on the drive. Update the README/run-demo help to match.
Sending via the action exposed a second cold-start race: bt_navigator is lifecycle-active and its action server is discoverable before it can actually accept a goal (amcl has not published map->odom yet, so it has no pose to plan from), so it rejects the goal. A goal fired right after boot was rejected once and dropped. Retry send_goal_async until the goal is accepted (up to 90s), so the reactive-fault step reliably starts the mission on a cold CI runner.
| return tl::make_unexpected("artifact GET returned status " + std::to_string(res->status)); | ||
| } | ||
|
|
||
| std::ofstream o(out_path, std::ios::binary); |
There was a problem hiding this comment.
download_artifact writes the HTTP response body to out_path with no checksum or signature verification, and execute() then extracts and spawns it (ota_update_plugin.cpp:256-261). Catalog entries carry no digest field (pack_artifact.py emits only an advisory size), and the fetch is plaintext HTTP with TLS explicitly rejected (parse_url:144). Any MITM on the catalog/artifact channel, or a rogue ota_update_server, delivers an arbitrary tarball that runs as root. Add a mandatory sha256 field in the catalog entry, verify it against the downloaded bytes before extraction, and sign the catalog itself.
| fs::remove_all(staging_extracted, ec); | ||
| fs::create_directories(staging_extracted, ec); | ||
|
|
||
| const std::string cmd = "tar -xzf " + staged_tarball + " -C " + staging_extracted; |
There was a problem hiding this comment.
extract_and_swap builds tar -xzf <staged_tarball> -C <dir> and runs it via std::system, and staged_tarball embeds the catalog-controlled id (staged_path = staging_dir_ + "/" + id + ".tar.gz", prepare() line 201). The id is taken verbatim from the catalog fetched over plain HTTP with no charset check in register_update, so an entry whose id is x; rm -rf / ; yields shell execution as root at execute() time. Drop the shell: fork+execvp (or an absolute tar path) passing an argv vector, and reject any id not matching ^[A-Za-z0-9._-]+$ in register_update.
| --skip-keys ur_description \ | ||
| --skip-keys ur_simulation_gz && \ | ||
| colcon build \ | ||
| --cmake-args -DCMAKE_BUILD_TYPE=Release -DBUILD_TESTING=OFF && \ |
There was a problem hiding this comment.
The gateway image builds the plugin with -DBUILD_TESTING=OFF, and no CI job runs colcon test or pytest. All three test suites this PR adds (ota_update_plugin gtest in ota_update_plugin/test/, ota_update_server/tests/test_main.py, scripts/test_pack_artifact.py) are therefore never executed anywhere in the pipeline; CI only builds the Docker images and runs the shell smoke tests. A regression in the dispatcher/catalog client/packer parsing passes CI green. Add a job that runs the gtest suite (BUILD_TESTING=ON + colcon test) and the two pytest suites.
| RMW_IMPLEMENTATION=rmw_cyclonedds_cpp | ||
|
|
||
| EXPOSE 8080 8765 | ||
| ENTRYPOINT ["/usr/local/bin/entrypoint.sh"] |
There was a problem hiding this comment.
Neither this image nor ota_update_server/Dockerfile sets a USER, so the gateway runs as uid 0. The OTA execute() flow (kill_by_executable, tar extraction of the fetched tarball, exec of the artifact binary) therefore runs with full in-container root and no capability restrictions. A tampered catalog or artifact escalates directly to root inside the container. Add a non-root USER for the runtime stage, and set cap_drop: [ALL] plus security_opt: [no-new-privileges:true] on the gateway service in docker-compose.yml.
|
|
||
| # CORS so an external Foxglove panel or browser can hit the API. | ||
| cors: | ||
| allowed_origins: ["*"] |
There was a problem hiding this comment.
The gateway binds 0.0.0.0:8080 with allowed_origins [*] and no authentication while updates.enabled is true, and 8080 is published to the host in docker-compose.yml. Any host or browser on the network can PUT /updates/{id}/prepare and /execute, driving the kill/extract/exec chain. The real OTA product must require auth on the update endpoints; even in the demo, note this or bind the API to the internal network only.
| if (!target_package.empty()) { | ||
| // Best-effort kill: legacy nodes may use the package name as their executable basename. | ||
| // Failures are tolerated since the process may already be gone. | ||
| auto kr = process_runner_->kill_by_executable(target_package); |
There was a problem hiding this comment.
Install spawns the node via install_dir/target_package/lib/target_package/executable, so its argv[0] basename is x_medkit_executable. Uninstall calls kill_by_executable(target_package), which pgrep matches on argv[0] basename and never matches the running node (basename is the executable, not the package). The files are then removed out from under a still-running process, leaving an orphaned live node. Kill by the executable basename (record it at install), not target_package.
| // Entities defined in the base manifest stay - fragments only | ||
| // ADD, they can't remove base-manifest declarations - those | ||
| // entries just go offline. | ||
| if (auto fr = remove_install_fragment(id); !fr) { |
There was a problem hiding this comment.
Install writes the fragment as ${update_id}.yaml where update_id is the install entry id (pack_artifact slug, e.g. foo_1_0_0), but Uninstall calls remove_install_fragment(id) with the uninstall entry id (${package}_remove, e.g. foo_remove). remove_install_fragment builds a different filename, hits no_such_file_or_directory (tolerated), and the install fragment is left in fragments_dir. The uninstalled app therefore stays in the entity tree forever. Key the fragment filename on the target package/component (shared across install and uninstall), not the per-entry update id.
| return tl::make_unexpected("create fragments_dir failed: " + ec.message()); | ||
| } | ||
|
|
||
| const std::string final_path = fragments_dir_ + "/" + update_id + ".yaml"; |
There was a problem hiding this comment.
update_id (the catalog id) is concatenated into final_path/tmp_path = fragments_dir_ + "/" + update_id + ".yaml" with no sanitization, so an id containing '../' writes a gateway manifest fragment to an arbitrary path as root. Separately, render_install_fragment interpolates app_id (added_components[0]) and node_name (x_medkit_executable) into the YAML body unquoted, so a value carrying a newline injects arbitrary apps/keys into the merged manifest that drives discovery. Validate update_id against [A-Za-z0-9._-]+ and emit the fragment with a real YAML serializer.
| // gzipped archive) but the kill step runs first - that is what we are | ||
| // checking here. | ||
| auto rc = plugin.execute("u_replaces", reporter); | ||
| (void)rc; |
There was a problem hiding this comment.
Both execute() tests feed a non-gzip staged tarball so extract_and_swap fails, then drop the result with (void)rc and assert only last_kill_target. Nothing asserts execute() success, the extract->swap->spawn path, or recovery after a mid-update failure, so these pass green even if the swap silently corrupts the install or spawn is removed. execute() kills the old process (ota_update_plugin.cpp:251) before extract/spawn and has no rollback on failure, so the exact scenario these tests trigger leaves the robot with no lidar node running - and no test catches it. Add a test that stages a real tarball and asserts rc succeeds and the binary is swapped, plus one asserting the failed-swap-after-kill behavior.
| # ros2 is not on the container's default PATH and `docker exec` does not run the | ||
| # image entrypoint that sources it, so source the ROS overlay inside the exec. | ||
| docker exec ota_demo_gateway bash -c \ | ||
| "source /opt/ros/jazzy/setup.bash && source /ws/install/setup.bash && ros2 topic pub --once /goal_pose geometry_msgs/PoseStamped '${GOAL}'" |
There was a problem hiding this comment.
X and Y (positional args, send-goal.sh:5) are interpolated into GOAL and then into a docker exec ota_demo_gateway bash -c "... '${GOAL}'" string where GOAL sits inside single quotes. A single quote in the argument breaks out of that quoting and runs arbitrary commands as root in the container, e.g. ./send-goal.sh "0'; touch /pwned; echo '0" executes touch /pwned. Validate that X and Y are numeric before use, or pass the goal to ros2 topic pub via a file/stdin instead of string-interpolating it into bash -c.
- pack_artifact.py: --version defaults to '' so an omitted version fails loud for update/install instead of silently packing a 0.0.0 artifact. - process_runner: correct the pgrep doc to cmdline argv0 (not comm, which the kernel truncates to 15 chars). - docker-compose: block privilege escalation on the gateway (no-new-privileges); cap_drop / non-root USER are documented as production hardening, not applied here (they risk breaking gz-sim / DDS in this all-in-one demo container).
…e-then-swap) - Reject catalog ids/components/executables/target packages that are not [A-Za-z0-9._-] before they reach a path or exec (blocks '../' traversal and argument injection). - Extract the artifact with fork+execvp instead of std::system, so a catalog-controlled name can never be interpreted by a shell. - Stage-then-swap: extract and verify the new tree before killing the running node, so a bad tarball no longer leaves the sensor with no lidar. - Track processes the plugin spawned (keyed by component) and also kill the recorded pid on re-apply, guarded by a /proc/<pid>/cmdline argv0 check against pid reuse - fixes the duplicate scan_sensor_node on a repeated apply. - Uninstall kills the recorded executable (not the package name, which never matched argv0) and removes the install fragment keyed on the component, so an uninstalled app no longer lingers in the entity tree. - Emit install-fragment scalars as quoted YAML. - Rewrite the process-runner test stub into a stateful fake covering the swap, duplicate-spawn, uninstall, traversal-rejection and quoting paths. - README: document the demo's trusted-local security model and what a production deployment would add; correct the swap note.
…n CI The plugin image built with -DBUILD_TESTING=OFF and no job ran colcon test or pytest, so the plugin gtest, the ota_update_server tests and the pack_artifact tests never executed anywhere. Add a Dockerfile 'ota-plugin-test' stage that rebuilds the plugin with BUILD_TESTING=ON and runs its gtest, and a 'plugin-tests' CI job that builds that stage and runs both pytest suites.
Description
Adds
demos/ota_nav2_sensor_fix/- an end-to-end OTA-over-SOVD demo (gateway plugin + FastAPI artifact server + 4 ROS 2 demo packages) exercising the/updateslifecycle: update a broken lidar node, install a new safety classifier, and uninstall a deprecated package, all over HTTP and spec-compliant.OTA demo
demos/ota_nav2_sensor_fix/end-to-end OTA demoota_update_pluginC++ gateway plugin (UpdateProvider+GatewayPlugin)updated_components/added_components/removed_components)pack_artifact.pyCLI for building tarballs and catalog entriesdocker-compose.yml(gateway + update server); nav2 / Foxglove are bring-your-own (documented in README)Out of scope (deliberate, dev-grade positioning)
Test plan / verification
Unit & integration tests (all clean):
pytest -vforpack_artifact.py(16 tests)pytest -vforota_update_server(5 tests)colcon testforota_update_plugin(24 GTest cases)-Wall -Wextra -Wpedantic -Wshadow -Wconversionbuild_artifacts.shproduces a 3-entry catalog + tarballs end-to-endEnd-to-end smoke:
UpdateProvider(gateway logs: "Update backend provided by plugin")/catalogand registers all 3 catalog entries/updates/fixed_lidar_2_1_0/prepare && /executekillsbroken_lidar_nodeand spawnsfixed_lidar_node/updates/obstacle_classifier_v2_1_0_0/prepare && /executeswaps files and spawnsobstacle_classifier_node/updates/broken_lidar_legacy_remove/prepare && /executereturnsstatus: completedand the legacy process is gonetests/smoke_test_ota.sh- 25/25 pass on a fresh stacktests/smoke_test_demo_narrative.sh- 8/8 pass on a fresh stacktrigger-update.sh,trigger-install.sh,trigger-uninstall.sh,check-demo.sh,stop-demo.shexercised end-to-endNotes
selfpatch/ros2_medkitmainfor the gateway sources (clone happens at image build time)pgrepmatches against/proc/<pid>/cmdlineargv[0] basename (notcomm, which the kernel truncates to 15 chars)artifacts/is baked into theupdate_serverimage at build timeros2_medkit_foxglove_extensionUpdates panel PR (fix(docker): add missing ros2_medkit components and submodules #6 of that repo)