NOTE: this is just food for thought, it's kind of a non-trivial feature from the user's perspective.
OpenAI has the v1/models/* endpoints, see here. We might make use of these as follows (also serves for the purpose of OpenAI compat #3)
- GET
/v1/models/list lists the supported models
- GET
/v1/models/{model} prepares the topology for the given model, akin to calling prepare_topology
- we would need some extra request parameters (such as
strategy, but we could embed them into the {model} param too, e.g. ring:gpt-oss-20b); bit smelly though
- DELETE
/v1/models/ unloads the model, akin to calling unload_model on API.
The idea here is to allow the entire user-flow of dnet to be achievable with openai compatibility, which enables us to use all existing OpenAI-compatible clients out there.
NOTE: this is just food for thought, it's kind of a non-trivial feature from the user's perspective.
OpenAI has the
v1/models/*endpoints, see here. We might make use of these as follows (also serves for the purpose of OpenAI compat #3)/v1/models/listlists the supported models/v1/models/{model}prepares the topology for the given model, akin to callingprepare_topologystrategy, but we could embed them into the{model}param too, e.g.ring:gpt-oss-20b); bit smelly though/v1/models/unloads the model, akin to callingunload_modelon API.The idea here is to allow the entire user-flow of dnet to be achievable with openai compatibility, which enables us to use all existing OpenAI-compatible clients out there.