Skip to content

feat(motion): add fixed-scene expert trajectory generation - #591

Open
yuecideng wants to merge 5 commits into
mainfrom
feat/fixed-scene-trajectory-generation
Open

feat(motion): add fixed-scene expert trajectory generation#591
yuecideng wants to merge 5 commits into
mainfrom
feat/fixed-scene-trajectory-generation

Conversation

@yuecideng

Copy link
Copy Markdown
Contributor

Description

Add a fixed-scene expert trajectory generation pipeline that restores the same physical initial state between attempts, augments explicitly permitted motion phases, validates the executed motion, and counts an episode only after its LeRobot shard has been sealed and read back.

The parallel cube example keeps the cube's initial pose fixed while varying 0°/90° grasp orientations and transit paths. Collection checks full-joint collision geometry, native contacts at every physics substep, bilateral finger contact, lift height, and held-object drift before persisting an episode.

  • Group solvers, planners, workspace, and augmentation under the lazily loaded embodichain.lab.sim.motion package. Update imports, configuration references, API docs, examples, benchmarks, tests, and agent context.
  • Add immutable trajectory contracts, protected-phase residual/retiming operators, coverage bookkeeping, bounded generation sessions, and confirmed commit receipts.
  • Add sim/Gym initial-state ownership and restoration, candidate planning, measured rollout evidence, synchronous LeRobot persistence, and initial candidate hooks for handwritten/atomic sources.
  • Add offline MoveEndEffector → PickUp template export, a bounded CPU-physics PickUp validator, repeated parallel collection, and synchronized H.264 video examples.

Breaking API migration: replace embodichain.lab.sim.{solvers,planners,workspace} imports with embodichain.lab.sim.motion.{solvers,planners,workspace}. Legacy import packages are removed; low-level Warp kernels remain under utils/warp/kinematics.

Dependencies: the optional trajectory-generation extra adds python-fcl, trimesh, and yourdfpy for PickUp collision validation. Collection uses the existing DexSim, LeRobot, and video dependencies; the free-motion example additionally uses cuRobo.

This draft contains the implemented portions of the design and implementation plan under docs/design/fixed_scene_expert_trajectory_augmentation_*. Full Atomic Runtime candidate consumption/tracking/recovery, Gym PickUp collection, the unified configuration launcher, and asynchronous persistence remain follow-up work. The current PickUp integration supports one unscaled fixed-base URDF robot and cuboid rigid objects with CPU physics; sampled geometry checks do not guarantee continuous collision avoidance.

Type of change

  • New feature
  • Breaking change (motion import paths)
  • Documentation update

Validation

  • After rebasing onto current main (f4ad72de): 1542 passed, 1 skipped, 168 deselected across motion, atomic actions, trajectory generation, Gym preparation/demo and simulation integrations, SimulationManager, articulation/gizmo, scene semantics, and motion benchmarks (excluding GPU/slow/requires_sim tests).
  • 3 passed, 1 deselected for the real parallel PickUp collection, failure cleanup, and existing visualization example:
    pytest tests/lab/trajectory_generation/test_pickup_collection.py \
      tests/lab/trajectory_generation/test_cube_grasp_parallel.py --run-gpu -m gpu -q
  • Real collection: 8 proposed / 8 attempted / 8 committed, four physical rows across two restored batches; 1.5 s hold with 100% bilateral contact coverage, minimum lift 17.90–17.92 cm, maximum relative translation drift 0.35–0.73 mm. Tests read back every LeRobot shard and the 544-frame, 20 fps, 1280×1056 H.264 video.
  • black . and git diff --check passed. Public API coverage: 1761/1761; checker tests: 8 passed. Sphinx dummy build completed with documentation warnings.
  • All 43 new Python files pass header/future-import/export checks; moved legacy modules retain their existing style. Changed canonical skills pass quick_validate.py.
  • The full repository suite and complete M1 source/host acceptance matrix were not run.

Screenshots

Generate the synchronized four-view video and verified dataset locally:

pip install -e '.[trajectory-generation]'
python examples/sim/motion/trajectory_generation/cube_pickup_collection.py \
  --output /tmp/cube-pickup-collection --episodes 8 --record-video

Outputs include preview.mp4, generation_report.json, pickup_report.json, manifest.json, and individual LeRobot episode shards. The video includes all attempted rollouts; the manifest includes only confirmed accepted episodes.

Checklist

  • I have run the black . command to format the code base.
  • I have made corresponding changes to the documentation.
  • Public API changes are reflected in the API docs (python docs/scripts/check_api_docs.py).
  • I have added tests that prove my fix is effective or that my feature works.
  • Dependencies have been updated.

Group solvers, planners, workspace and augmentation under sim.motion. Add fixed-scene preparation, bounded generation and confirmed LeRobot persistence, plus physically validated parallel cube PickUp collection and video examples.

BREAKING CHANGE: import robot motion APIs from embodichain.lab.sim.motion; previous sim.solvers, sim.planners and sim.workspace paths are removed.
@yuecideng yuecideng added docs Improvements or additions to documentation enhancement New feature or request motion gen Things related to motion generation for robot refactor breaking dataset labels Sep 5, 2026
@yuecideng
yuecideng marked this pull request as ready for review September 6, 2026 17:00
@greptile-apps

greptile-apps Bot commented Sep 6, 2026

Copy link
Copy Markdown

RetriggerConfidence Score: 5/5

The PR appears safe to merge because no blocking failure remains within the scope of this follow-up review.

Summary

  • Adds generation leases, preparation epochs, fixed-scene restoration, and measured rollout execution.
  • Adds trajectory contracts, augmentation operators, planning and contact-validation integrations, and synchronous LeRobot persistence.
  • Migrates solver, planner, and workspace APIs beneath the lazily loaded simulation motion package.

Diagram

sequenceDiagram
    participant R as GenerationRunner
    participant H as FixedSceneHost
    participant P as Planner
    participant E as RolloutExecutor
    participant V as Validators
    participant S as LeRobotEpisodeSink
    R->>H: Acquire generation lease
    R->>H: Restore and verify fixed initial state
    H-->>R: PreparedBatch with epoch
    R->>P: Generate augmented candidates
    P-->>R: Planned candidate batch
    R->>E: Execute candidate commands
    E->>V: Validate measured motion and contacts
    V-->>R: Acceptance evidence
    R->>S: Write and seal accepted episode
    S->>S: Read back shard and manifest
    S-->>R: Confirmed commit receipt
    R->>R: Update committed count and coverage
    R->>H: Release generation lease
Loading

Reviews (4) · Last reviewed commit: "fix(trajectory): resolve CI compatibilit..."

@yuecideng
yuecideng requested a review from matafela September 7, 2026 04:04
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

breaking dataset docs Improvements or additions to documentation enhancement New feature or request motion gen Things related to motion generation for robot refactor

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant