Skip to content

chore(synthesis): bundle MediaPipe deps + hand_landmarker model for HandGestureDetector block - #509

Open
harsh839 wants to merge 1 commit into
JdeRobot:masterfrom
harsh839:chore/bundle-mediapipe-block
Open

harsh839 wants to merge 1 commit into
JdeRobot:masterfrom
harsh839:chore/bundle-mediapipe-block

Conversation

@harsh839

Copy link
Copy Markdown

Summary

Completes the backend side of the MediaPipe HandGestureDetector custom block
for the Build & Download flow, addressing the remaining bundling gap in
#470 (the block itself, HandGestureDetector.vc3 + hand_landmarker.task, is
already merged upstream via VisualCircuit-resources#17).

When a circuit that uses the hand-gesture block is built, the generated ZIP must
ship its third-party runtime and model. This PR wires that up by mirroring the
exact pattern already used for FaceDetector (haar) and ObjectDetector
(yolov3):

  1. requirements.txt — added
    'HandGestureDetector': ['mediapipe', 'opencv-python', 'numpy'] to
    BLOCK_DEPENDENCIES. mediapipe has no entry in COMMON_PIP_ALIASES, so the
    AST sweep alone would only add an unresolved mediapipe name; listing it here
    guarantees the real pip package lands in the generated requirements file.
  2. Model bundling — added
    'HandGestureDetector': 'utils/models/hand_landmarker/**/*' to
    OPTIONAL_FILES, mirroring how FaceDetector/ObjectDetector pull their
    model assets into the ZIP (via the paths_to_include whitelist in
    syntheize_executioner, keyed off the optional_files[script_name] flag set
    in syntheize_modules).
  3. Model asset — ships
    backend/staticfiles/synthesis/utils/models/hand_landmarker/hand_landmarker.task
    (7.6 MB, from Google's official MediaPipe model zoo — same convention as the
    committed yolov3-tiny.weights), plus a README.md noting source/license.

Verification

Backend CI does not run tests, so correctness is verified locally:

  • python -m py_compile backend/synthesis/synthesis.py — passes.
  • Reimplemented the bundler's exact include/exclude matching
    (Path.relative_to('synthesis').match(...) against the OPTIONAL_FILES
    globs, as done in syntheize_executioner) against the on-disk
    hand_landmarker.task path:
    • glob utils/models/hand_landmarker/**/* includes the model when the
      block is present (True),
    • it is not swallowed by the FaceDetector/ObjectDetector exclude globs
      (True),
    • resolved deps = ['mediapipe', 'opencv-python', 'numpy'].
  • git diff origin/master...HEAD touches only synthesis.py (2 dict lines) and
    the new model directory — no unrelated churn.

Notes

Closes the bundling portion of #470.

When the HandGestureDetector custom block is used in a circuit, the Build &
Download bundle must include its third-party runtime:

- add mediapipe to requirements.txt through BLOCK_DEPENDENCIES (mediapipe is
  not a COMMON_PIP_ALIASES alias, so the AST sweep alone cannot rewrite it)
- ship the hand_landmarker.task model with the generated ZIP through
  OPTIONAL_FILES, mirroring how FaceDetector bundles haar_cascade and
  ObjectDetector bundles yolov3

Release note: none needed; behavioral only for circuits using the MediaPipe
hand-gesture block (VisualCircuit-resources#470).
@harsh839

Copy link
Copy Markdown
Author

@jmplaza could you take a look when you get a chance? This bundles the MediaPipe hand_landmarker model so the HandGestureDetector block's synthesized project ZIP includes the model file (see OPTIONAL_FILES / BLOCK_DEPENDENCIES in backend/synthesis/synthesis.py).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant