Skip to content

Visualize per-point manipulability with inspection ellipsoids - #631

Open
Yuan-Xinyi wants to merge 12 commits into
mainfrom
xinyi/workspace-viz
Open

Yuan-Xinyi wants to merge 12 commits into
mainfrom
xinyi/workspace-viz

Conversation

@Yuan-Xinyi

@Yuan-Xinyi Yuan-Xinyi commented Sep 15, 2026

Copy link
Copy Markdown
Collaborator

Description

Adds the manipulability visualization requested as item 3 of the review on #614. That PR made WorkspaceAnalyzer produce a true per-point Yoshikawa score; this one makes the score readable.

Follow-up to #614 (now merged); this PR contains only the visualization.

The module keeps a pure mapping layer separate from a thin matplotlib renderer, so normalization and alignment are unit-testable without a display:

Function Responsibility
align_manipulability_scores Tie the compact score vector to the drawn points
normalize_manipulability Percentile clipping, optional log10, reports raw and clipped ranges
map_manipulability_colors Color reachable points, keep unreachable ones distinct
select_inspection_indices / inspect_points Pick top/bottom/explicit points and gather their diagnostics
translational_manipulability_ellipsoid Velocity ellipsoid from the translational Jacobian rows

How each review point is met

  • Alignment. Joint-space scores map 1:1 onto workspace_points; Cartesian and plane scores map 1:1 onto reachable_points and are scattered onto all_points through reachability_mask. The scatter verifies all_points[mask] == reachable_points and raises rather than drawing a mis-colored figure if that ordering assumption is ever broken.
  • Robust / log normalization with an honest color bar. Percentile clipping (default 2–98) optionally on log10. clip_range is converted back into raw w units, so the bar, the array, and the footer annotation always agree. The untouched raw_range and the saturated-point count are printed next to every figure.
  • Unreachable points stay distinct — small translucent grey crosses, not dropped and not blended into the ramp.
  • Ellipsoids only where asked. inspect_points() calls jacobian_fn once, with only the selected rows, and uses select_jacobian_rows(J, "translational"). A 6000-point workspace costs a handful of Jacobians, not 6000.
  • Degenerate inputs are explicit: empty, fully-invalid, and constant score vectors are handled instead of dividing by a zero span.

Type of change

  • New feature (non-breaking change which adds functionality)

Screenshots

All three are produced by scripts/tutorials/sim/workspace_manipulability_visualization.py on a UR5 and are the figures embedded in the docs page.

1. Cartesian workspace colored by manipulability — 6000 sampled points: the 981 reachable ones carry the viridis ramp, the 5019 unreachable ones remain grey crosses that outline the sampling box. The footer reports raw w range [0, 0.1036] against the percentile-clipped bar span [0, 0.09719] and the 20 saturated points, so the clipping is visible rather than silent. Red circles mark the two inspected points.

Cartesian workspace colored by manipulability

2. Joint-space workspace on a log color scale — 3541 FK samples with decade ticks on the bar. This is the figure that justifies offering log normalization: the raw range starts at 0 while the clipped span starts at 5.07e-05, and on a log ramp the near-singular shell stays separable from the dexterous mid-range instead of collapsing into one dark color.

Joint-space workspace on a log manipulability scale

3. Translational manipulability ellipsoids at the best and worst points — each panel prints w, cond(J), the position, the semi-axes, the anisotropy and the full qpos. The best point (w = 0.104, cond = 10.5) is a near-round ellipsoid; the worst (w = 0, cond = 2.06e9) is a flat disc whose shortest semi-axis is 0.13 m/s against 0.87.

Translational manipulability ellipsoids at selected points

Worth noting from that last figure: the w = 0 configuration still has a three-dimensional translational ellipsoid. Its rank deficiency lives in the rotational rows, so reading w = 0 alone would suggest the end-effector cannot move at all, when in fact only the orientation directions are degenerate. That is a concrete argument for showing the ellipsoid next to the scalar, and it is called out in the docs page.

Checklist

  • I have run the black . command to format the code base.
  • I have made corresponding changes to the documentation
  • Public API changes are reflected in the API docs
  • I have added tests that prove my feature works
  • Dependencies have been updated, if applicable (none)

Validation

pytest tests/sim/motion/workspace/ tests/compute/test_manipulability.py  -> 162 passed
python scripts/tutorials/sim/workspace_manipulability_visualization.py    -> 3 figures written
black --fast --check .                                                     -> 982 files unchanged
python docs/scripts/check_api_docs.py                                      -> 2103/2103 documented
context.py check                                                           -> agent context map: ok

69 of those tests are new and cover the mapping and normalization directly: percentile clipping, log scaling, empty / single-point / all-equal inputs, the unreachable mask, index alignment in both result modes, and that the ellipsoid is computed only for the selected subset.

Known limitations

  • The color bar and the range annotation are matplotlib-only. The Viser and sim_manager backends forward RGB alone, so an in-scene legend would be a separate change.
  • ManipulabilityVisualizer deliberately lets scores override a colors array passed by the generic analyzer.visualize() path, which hands reachability colors to every visualizer. Documented and covered by a test.

yuecideng and others added 11 commits September 10, 2026 08:07
Wire the metrics module into WorkspaceAnalyzer (closing the long-standing
_compute_metrics TODO) and compute true per-configuration Yoshikawa
manipulability w = sqrt(det(J J^T)) from the active solver's Jacobian after
every analysis mode. Scores are row-aligned with joint_configurations (and
with reachable points in Cartesian/plane modes), stored in results.npz,
restored on cache hits, and aggregated under metrics["manipulability"].
Computation is gated on MetricConfig.enabled_metrics and costs ~10 ms per
470 configurations on GPU.

Remove ManipulabilityMetric's centroid-distance placeholder: measured on
Franka it is negatively correlated with true manipulability (corr = -0.37),
so consumers ranking by it preferred worse configurations. Without
Jacobians or precomputed scores the metric now warns and returns no
statistics instead of fabricating them.

The batching test's mock robot now returns None from get_solver, faithful
to Robot.get_solver with no solvers attached.

Also documented in the robot-workspace context: enabling the #599
seed-selection sampler speeds Cartesian reachability analysis 3.3x at
unchanged num_samples=30 while detecting slightly more reachable points
(measured on Franka, 4000 identical targets), with no analyzer changes.

Covered by tests/sim/motion/workspace/test_manipulability.py; the full
workspace suite passes (68 passed).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Address two review findings on the manipulability integration:

1. Metric settings are deliberately not part of the results-cache key (a
   metric toggle must not invalidate the expensive sampling/IK work), so a
   cache entry written under a different metric configuration — or before
   manipulability existed — could be returned without scores. The cache-hit
   path now runs the same _apply_manipulability step as fresh analysis,
   recomputing scores and aggregates from the cached joint configurations
   in milliseconds and repairing such entries transparently.

2. The analyzer reduced Jacobians to Yoshikawa scalars and discarded them,
   so the default compute_isotropy=True could never produce its documented
   condition statistics. The chunked Jacobian sweep now also collects
   condition numbers (max/min singular value) when isotropy is enabled, and
   ManipulabilityMetric accepts them precomputed.

Tests cover the repair path (an entry written with manipulability disabled
is repaired by a later default-enabled hit on the same key), isotropy
presence via the analyzer, and precomputed condition-number passthrough.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The repair-on-load path handled a disabled-producer entry hit by an
enabled run, but not the symmetric direction: an enabled-producer entry
hit by a disabled run leaked stale manipulability_scores and
metrics["manipulability"] into the returned results, diverging from the
fresh-analysis contract. _apply_manipulability now strips both fields
when the metric is disabled, and keeps cached scores when the metric is
enabled but locally not computable (they remain valid for the same joint
configurations).

Covered by test_cache_hit_strips_fields_when_metric_disabled, which
writes a score-bearing entry first so the strip path is genuinely
exercised on the hit.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…config

On a cache hit where the current robot has no solver, the previous branch
kept both the cached scores and the producer's aggregate metrics, so the
returned means/counts could reflect a different jacobian_threshold and
condition statistics could be present with isotropy disabled (or stale
when enabled). Cached scores are pure kinematics and stay valid, but
aggregates now always go through ManipulabilityMetric under the CURRENT
configuration; per-point condition numbers are not cached, so condition
statistics are correctly absent on this path instead of leaking through.

Covered by test_no_solver_hit_recomputes_aggregates_under_current_config:
a mock no-solver robot hits an entry carrying producer aggregates from a
different configuration, and the returned aggregates honour the current
threshold while the stale mean_condition is removed.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…unt report

Address review items R5, R2, and R1 on PR #614.

R5 — extract numerical calculation into a public compute module. Move the
Jacobian-to-score math out of WorkspaceAnalyzer into
embodichain.compute.kinematics.manipulability as pure batched Torch helpers:
- yoshikawa_manipulability(J) = sqrt(det(J @ J^T)), clamped at zero;
- condition_number(J) = sigma_max / sigma_min;
- select_jacobian_rows(J, "translational"|"rotational"|indices) for
  task-specific row subsets.
They preserve input dtype/device and never import simulation/workspace code.
The analyzer, the metric class, and future IK candidate-ranking now share
this single implementation; aggregation stays in the workspace layer.

R2 — report zero valid points correctly. When no score clears
jacobian_threshold (all singular, or threshold above every score),
ManipulabilityMetric previously substituted a single 0.0 score and reported
num_valid_points == 1. It now reports num_valid_points == 0 with NaN
statistics. Added regression tests for all-below-threshold and all-zero.

R1 — keep the public metric contract consistent. Updated the metrics guide:
its heuristic examples (which now raise KeyError) are replaced with
Jacobian-based usage, and the zero-valid-point / empty-return behaviour is
documented. Added the compute Kinematics API section and page.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Address review items R5 (IK path must consume the shared compute helpers)
and R4 (application-oriented example) on PR #614.

PytorchSolver gains an opt-in ik_solution_selection="manipulability" mode.
The multi-seed pipeline previously always collapsed candidates to the one
nearest the caller seed; the new mode scores every successful candidate
with embodichain.compute.kinematics.yoshikawa_manipulability before the
collapse and keeps the best-conditioned posture. The default ("nearest")
is unchanged; invalid values are rejected at construction.

scripts/tutorials/sim/ik_manipulability_selection.py demonstrates the
decision on the DexforceW1 left arm (50 targets, 30 seeds each) and
separates the two effects: seed selection (iksel) improves which
candidates exist, re-ranking improves which candidate is kept.

  variant                  success  mean w   mean cond  time ms
  nearest (default)         100.0%  0.0125     165.2     180.6
  manipulability re-rank    100.0%  0.0162      75.2     176.9
  iksel + nearest           100.0%  0.0127     164.8     132.4
  iksel + manipulability    100.0%  0.0163      78.9     122.5

Re-ranking lifts mean manipulability +30% and halves the mean condition
number at no measurable cost; combined with iksel it keeps the +30% gain
at the lowest solve time. Focused tests cover mode validation, target
accuracy preservation, dominance over nearest under an identical
candidate pool, and deterministic selection.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The compute-helper refactor dropped the pre-existing LinAlgError
degradation path. Keep the shared batched implementation as the primary
route, but on SVD failure fall back to per-matrix numpy computation with
inf on failure, exactly as before the refactor.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This branch was stacked on #606, which main has since merged, so every
file #606 touched conflicted between that branch's original commit and
main's merged version.

Resolution rule: for files this branch never edited on top of #606
(base_solver.py, ur_solver.py, robot.py, test_analytic_batching.py,
the ik-solvers context page), take main's version wholesale — it is the
reviewed outcome of #606 plus later refactors, and keeping the older
pre-review copy would have reverted them. For files this branch does
own, keep its additions:

- compute/kinematics/__init__.py: main extended the kernel list with
  trapezoidal/Double-S profiles while this branch documented the
  manipulability helpers; both statements are kept.
- the robot-workspace context page and test_analysis_batching.py carry
  manipulability sections that main has no counterpart for.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…llipsoids

Address the remaining review item on PR #614: a focused visualization for
the per-point Yoshikawa scores that analysis already produces.

The module separates a pure mapping layer from a thin matplotlib renderer,
so normalization and alignment are testable without a display:

- align_manipulability_scores() ties the compact score vector to the drawn
  points. Joint-space scores map 1:1 onto workspace_points; Cartesian and
  plane scores map 1:1 onto reachable_points and are scattered onto
  all_points through reachability_mask. That scatter verifies
  all_points[mask] == reachable_points and raises instead of drawing a
  mis-colored figure if the ordering assumption is ever broken.
- normalize_manipulability() clips to percentile bounds, optionally on a
  log10 scale, and reports the untouched raw range next to the clipped
  span so the color bar never hides the true magnitudes. Empty, fully
  invalid, and constant inputs are handled explicitly rather than dividing
  by a zero span.
- map_manipulability_colors() keeps unreachable points visually distinct as
  small translucent grey crosses instead of dropping them.
- translational_manipulability_ellipsoid() takes the translational rows via
  select_jacobian_rows() and is evaluated only for the inspected subset:
  inspect_points() calls jacobian_fn once with the selected rows, so a
  6000-point workspace still costs a handful of Jacobians.

scripts/tutorials/sim/workspace_manipulability_visualization.py renders the
three documentation figures headlessly. It drives the kinematic chain
directly rather than a SimulationManager, so the doc figures regenerate
without a GPU or renderer.

The analyzer change is one elif branch that forwards scores and the
reachability mask for the new visualization type; every existing type
keeps its current arguments.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
# Conflicts:
#	agent_context/MAP.yaml
#	agent_context/topics/robot-workspace/analysis-and-cache.md
#	tests/sim/motion/workspace/test_manipulability.py
@Yuan-Xinyi Yuan-Xinyi added the enhancement New feature or request label Sep 15, 2026
@greptile-apps

greptile-apps Bot commented Sep 15, 2026

Copy link
Copy Markdown

RetriggerConfidence Score: 5/5

The PR appears safe to merge, with two non-blocking inspection-helper issues still tracked in existing review threads.

Fix All in CodexFindings

  1. P2 Top ties reverse index order
  2. P2 Inspection arrays lack validation
Fix with agent prompt
### Issue 1
embodichain/lab/sim/motion/workspace/visualizers/manipulability_visualizer.py:undefined-732
When several points share the highest score, reversing the ascending array also reverses their index order. If `top_k` truncates that tied group, this selects the highest-index point instead of the lowest-index point promised by the public docstring, so a different point and Jacobian are inspected.

```suggestion
        for index in valid_indices[
            np.lexsort((valid_indices, -values[valid_indices]))
        ][:top_k]:
```

Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!

### Issue 2
embodichain/lab/sim/motion/workspace/visualizers/manipulability_visualizer.py:822-838
This public helper accepts independently supplied point, score, and score-index arrays, but it checks selected indices only against `points`. A shorter `scores` or `score_indices` vector therefore causes an incidental `IndexError`, while arrays that are long enough but misaligned can pair a point with the wrong score or joint configuration. Validate all point-aligned lengths and require one label per selected index before indexing.

```suggestion
    point_array = _as_points(points)
    score_array = _as_float_vector(scores)
    if len(score_array) != len(point_array):
        raise ValueError(
            f"scores has {len(score_array)} entries but points has "
            f"{len(point_array)} rows."
        )
    if len(labels) != len(indices):
        raise ValueError(
            f"selection has {len(indices)} indices but {len(labels)} labels."
        )
    if indices.size and (indices.min() < 0 or indices.max() >= len(point_array)):
        raise ValueError(
            f"inspection indices out of range for {len(point_array)} points."
        )

    qpos = (
        None
        if joint_configurations is None
        else np.asarray(_to_numpy(joint_configurations), dtype=np.float64)
    )
    if score_indices is None:
        rows = indices
    else:
        row_map = np.asarray(_to_numpy(score_indices), dtype=np.int64).reshape(-1)
        if len(row_map) != len(point_array):
            raise ValueError(
                f"score_indices has {len(row_map)} entries but points has "
                f"{len(point_array)} rows."
            )
        rows = row_map[indices] if indices.size else indices
```

---

For each issue above, determine whether it is valid and should be fixed. If so, fix it directly.

Summary

Adds manipulability-aware workspace visualization with score alignment, percentile/log normalization, distinct unreachable-point styling, and selected-point translational ellipsoids.

  • Registers the new visualizer with WorkspaceAnalyzer and VisualizerFactory.
  • Adds a headless tutorial, generated figures, public API documentation, and focused numerical tests.
  • Changes since the previous review only regenerate the three documented PNG artifacts.

Diagram

%%{init: {'theme': 'neutral'}}%%
flowchart LR
    A[WorkspaceAnalyzer results] --> B[align_manipulability_scores]
    B --> C[Aligned points, scores, mask]
    C --> D[normalize_manipulability]
    D --> E[map_manipulability_colors]
    E --> F[Workspace visualization]
    C --> G[select_inspection_indices]
    G --> H[inspect_points]
    H --> I[Selected Jacobians]
    I --> J[Translational ellipsoids]
    J --> K[Inspection visualization]
Loading

Reviews (2) · Last reviewed commit: "docs(workspace): track the manipulabilit..."

# Stable order: primary key score, secondary key index.
order = np.lexsort((valid_indices, values[valid_indices]))
ascending = valid_indices[order]
for index in ascending[::-1][:top_k]:

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Top ties reverse index order

When several points share the highest score, reversing the ascending array also reverses their index order. If top_k truncates that tied group, this selects the highest-index point instead of the lowest-index point promised by the public docstring, so a different point and Jacobian are inspected.

Suggested change
for index in ascending[::-1][:top_k]:
for index in valid_indices[
np.lexsort((valid_indices, -values[valid_indices]))
][:top_k]:
Prompt To Fix With AI
This is a comment left during a code review.
Path: embodichain/lab/sim/motion/workspace/visualizers/manipulability_visualizer.py
Line: 732

Comment:
**Top ties reverse index order**

When several points share the highest score, reversing the ascending array also reverses their index order. If `top_k` truncates that tied group, this selects the highest-index point instead of the lowest-index point promised by the public docstring, so a different point and Jacobian are inspected.

```suggestion
        for index in valid_indices[
            np.lexsort((valid_indices, -values[valid_indices]))
        ][:top_k]:
```

---

For each issue above, determine whether it is valid and should be fixed. If so, fix it directly.

Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!

Fix in Codex Fix in Claude Code

Comment on lines +822 to +838
point_array = _as_points(points)
score_array = _as_float_vector(scores)
if indices.size and (indices.min() < 0 or indices.max() >= len(point_array)):
raise ValueError(
f"inspection indices out of range for {len(point_array)} points."
)

qpos = (
None
if joint_configurations is None
else np.asarray(_to_numpy(joint_configurations), dtype=np.float64)
)
if score_indices is None:
rows = indices
else:
row_map = np.asarray(_to_numpy(score_indices), dtype=np.int64).reshape(-1)
rows = row_map[indices] if indices.size else indices

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Inspection arrays lack validation

This public helper accepts independently supplied point, score, and score-index arrays, but it checks selected indices only against points. A shorter scores or score_indices vector therefore causes an incidental IndexError, while arrays that are long enough but misaligned can pair a point with the wrong score or joint configuration. Validate all point-aligned lengths and require one label per selected index before indexing.

Suggested change
point_array = _as_points(points)
score_array = _as_float_vector(scores)
if indices.size and (indices.min() < 0 or indices.max() >= len(point_array)):
raise ValueError(
f"inspection indices out of range for {len(point_array)} points."
)
qpos = (
None
if joint_configurations is None
else np.asarray(_to_numpy(joint_configurations), dtype=np.float64)
)
if score_indices is None:
rows = indices
else:
row_map = np.asarray(_to_numpy(score_indices), dtype=np.int64).reshape(-1)
rows = row_map[indices] if indices.size else indices
point_array = _as_points(points)
score_array = _as_float_vector(scores)
if len(score_array) != len(point_array):
raise ValueError(
f"scores has {len(score_array)} entries but points has "
f"{len(point_array)} rows."
)
if len(labels) != len(indices):
raise ValueError(
f"selection has {len(indices)} indices but {len(labels)} labels."
)
if indices.size and (indices.min() < 0 or indices.max() >= len(point_array)):
raise ValueError(
f"inspection indices out of range for {len(point_array)} points."
)
qpos = (
None
if joint_configurations is None
else np.asarray(_to_numpy(joint_configurations), dtype=np.float64)
)
if score_indices is None:
rows = indices
else:
row_map = np.asarray(_to_numpy(score_indices), dtype=np.int64).reshape(-1)
if len(row_map) != len(point_array):
raise ValueError(
f"score_indices has {len(row_map)} entries but points has "
f"{len(point_array)} rows."
)
rows = row_map[indices] if indices.size else indices
Prompt To Fix With AI
This is a comment left during a code review.
Path: embodichain/lab/sim/motion/workspace/visualizers/manipulability_visualizer.py
Line: 822-838

Comment:
**Inspection arrays lack validation**

This public helper accepts independently supplied point, score, and score-index arrays, but it checks selected indices only against `points`. A shorter `scores` or `score_indices` vector therefore causes an incidental `IndexError`, while arrays that are long enough but misaligned can pair a point with the wrong score or joint configuration. Validate all point-aligned lengths and require one label per selected index before indexing.

```suggestion
    point_array = _as_points(points)
    score_array = _as_float_vector(scores)
    if len(score_array) != len(point_array):
        raise ValueError(
            f"scores has {len(score_array)} entries but points has "
            f"{len(point_array)} rows."
        )
    if len(labels) != len(indices):
        raise ValueError(
            f"selection has {len(indices)} indices but {len(labels)} labels."
        )
    if indices.size and (indices.min() < 0 or indices.max() >= len(point_array)):
        raise ValueError(
            f"inspection indices out of range for {len(point_array)} points."
        )

    qpos = (
        None
        if joint_configurations is None
        else np.asarray(_to_numpy(joint_configurations), dtype=np.float64)
    )
    if score_indices is None:
        rows = indices
    else:
        row_map = np.asarray(_to_numpy(score_indices), dtype=np.int64).reshape(-1)
        if len(row_map) != len(point_array):
            raise ValueError(
                f"score_indices has {len(row_map)} entries but points has "
                f"{len(point_array)} rows."
            )
        rows = row_map[indices] if indices.size else indices
```

---

For each issue above, determine whether it is valid and should be fixed. If so, fix it directly.

Fix in Codex Fix in Claude Code

.gitignore carries a blanket *.png rule, so the three generated figures
were silently skipped and the documentation page (and the pull request)
referenced files that do not exist in the repository. Force-add them the
way the existing tutorial figures are tracked.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants