Sensor and robotics simulation (preview)
Dawn Engine SensorSim is a planned first-class simulation Gem family (first-release entry Gems/SensorSim, with planned Ros2Bridge / RotorCraft / Replay related tracks). It builds on Atom rendering and PhysX scene queries to emit configurable camera, lidar, radar, and infrared sensor streams, and bridges them to external autonomy stacks through ROS 2.
Maturity: planned / product preview. This page describes target capabilities and rollout stages—not available product APIs. Enabling any future Registry gate must leave existing PhysX, Atom, Navigation3D, Flight, and ClusterDisplay paths unchanged when SensorSim is off.
It does not replace an autonomy or robotics algorithm stack: SensorSim provides sensor facts and a clock; planning, control, and learning pipelines stay in ROS 2 / the project layer. Fixed-wing flight dynamics remain owned by the planned Flight Gem (JSBSim).
What problem it solves
Autonomy, UAV, robotics, and training simulation need the world the engine renders to line up with the sensors an algorithm stack can read. SensorSim is intended to provide:
- Configurable sensor components (pinhole / fisheye cameras, spinning / solid-state lidar, simplified radar and IR approximations);
- Sampling contracts consistent with double-precision large worlds, PolyTerrain, and Weather (visibility / precipitation);
- A ROS 2 bridge (topics, TF, clock) that takes the O3DE-ecosystem ROS2 prototype in-tree, localized, and continuously maintained;
- Optional synthetic-data export hooks (images / point clouds / annotation metadata) for offline training pipelines;
- Coordination with Navigation3D, ClusterDisplay, and DataLink for trainers, CAVE walls, and twin co-simulation.
Planned capabilities
Sensor core (SS0–SS5)
| Milestone | Focus |
|---|---|
| SS0 | Gem scaffold, Registry opt-in (default off), sensor component model, headless unit tests |
| SS1 | RGB camera (with distortion model), depth-buffer export, fixed simulation clock |
| SS2 | Lidar ray / depth approximation, point-cloud topics, noise and drop models |
| SS3 | IR / night-vision approximation, simplified radar returns, Weather visibility coupling |
| SS4 | Extrinsic stability under large-world / render-origin shifts; multi-sensor sync gates |
| SS5 | Performance tiers, batch offline recording, shipping docs and sample levels |
ROS 2 bridge track (RB1–RB3)
| Track | Focus |
|---|---|
| RB1 | ROS 2 Humble/Jazzy dynamic bridge, basic sensor topics and /clock |
| RB2 | TF tree, joint / chassis state, bidirectional adapter for Navigation3D expectations |
| RB3 | Chinese editor workflows, Windows/Linux packages, ongoing maintenance gates |
Follow-on tracks (after first exit)
| Track | Focus |
|---|---|
| RotorCraft | Multirotor / helicopter dynamics complementary to fixed-wing Flight; Navigation3D airspace |
| Replay | Deterministic record/playback, timeline scrubbing, multi-view debrief (training evaluation) |
| DIS/HLA | Distributed simulation interconnect (defense / institute training) with ClusterDisplay |
Rotorcraft dynamics, DIS/HLA, and full Replay are outside SensorSim’s first-release scope, but the architecture reserves clock and topic contracts for them.
How it will work
Atom views / PhysX queries / Weather environment facts
│
▼
SensorSim Core (fixed-step sensor clock)
│
├─ Camera / Lidar / Radar / IR components
├─ Noise / sync / extrinsics (double → sensor frame)
├─ Ros2Bridge ──► ROS 2 topics / TF / clock
└─ ExportHook (optional) ──► synthetic datasets
Design rules in the specs:
- Render
dtdoes not define sensor cadence—sensors use a fixed-step accumulator for replay and CI. - Declared fidelity: each sensor asset declares
DeclaredFidelityLevel; MVP targets procedural usefulness, not metrology certification. - One-way Weather / Flight facts: visibility and atmosphere come from Weather/Flight; SensorSim never writes weather.
- Default off: zero cost when disabled; ROS 2 is an optional dependency and must not be forced into every shipping package.
Explicitly out of scope for v1
| Not in v1 | Notes |
|---|---|
| Certified sensor digital twins | No metrology / ISO compliance claims |
| Full EM / optical solvers | Rasterization and simplified physics models dominate |
| Built-in autonomy planners | Algorithms stay in ROS 2 / project layer |
| Fixed-wing FDM | Owned by Flight |
| Turnkey cloud annotation platforms | Export hooks only |
Related systems
| Topic | Relationship |
|---|---|
| Flight (planned) | Authoritative fixed-wing dynamics; SensorSim can mount onboard sensors |
| Weather (planned) | Visibility, precipitation, and wind effects on noise and detection range |
| Navigation3D | Air / underwater path expectations → autonomy stack or simple controllers |
| PhysX Vehicle | Ground-vehicle chassis; sensor extrinsic attach points |
| ClusterDisplay | Multi-channel visuals for trainers |
| DataLink (planned) | Simulated telemetry can write into the twin binding bus |
| Double-precision rendering | Stable large-world extrinsics and point-cloud coordinates |
Status and next steps
This chapter is a product preview. When SensorSim ships, expect Quick start (enable Gem, mount sample sensor Prefabs, start the ROS 2 bridge), Registry / packaging notes, and debugging panels consistent with other user-guide chapters.