adaptive-batching

Here are 3 public repositories matching this topic...

Real-time ML inference engine with adaptive batching, model hot-swap, and circuit breaker patterns

python machine-learning streaming real-time deep-learning inference pytorch model-serving mlops adaptive-batching

Local inference serving with adaptive batching, benchmark sweeps, and regression gates for batching tradeoffs.

Deliver real-time streaming ML inference with adaptive batching, backpressure control, and zero-downtime model hot-swapping.

python machine-learning streaming real-time deep-learning inference pytorch model-serving mlops adaptive-batching

To associate your repository with the adaptive-batching topic, visit your repo's landing page and select "manage topics."