Skip to content

feat: Add OrcaRouter as a supported LLM provider - #231

Open
clementguarino06510-glitch wants to merge 1 commit into
docker:mainfrom
clementguarino06510-glitch:add-orcarouter-provider
Open

feat: Add OrcaRouter as a supported LLM provider#231
clementguarino06510-glitch wants to merge 1 commit into
docker:mainfrom
clementguarino06510-glitch:add-orcarouter-provider

Conversation

@clementguarino06510-glitch

Copy link
Copy Markdown

Add OrcaRouter as a supported LLM provider

The GenAI Stack connects to OpenAI through LangChain's ChatOpenAI in chains.py, and to Bedrock through ChatBedrock and Ollama through ChatOllama. OrcaRouter speaks the same OpenAI-compatible protocol on https://api.orcarouter.ai/v1, so this PR wires it in as a first-class provider using the exact same ChatOpenAI integration — no custom base URL juggling, no generic any-endpoint passthrough.

What's added

  • chains.py — a new named branch in load_llm(): any LLM=orcarouter/<model> (e.g. orcarouter/fusion-mini) is routed to ChatOpenAI pointed at the OrcaRouter base URL, mirroring the existing OpenAI case directly above it.
  • api.py, bot.py, pdf_bot.py — read ORCAROUTER_BASE_URL and ORCAROUTER_API_KEY from the environment and pass them into load_llm()'s config, exactly like OLLAMA_BASE_URL is threaded through today.
  • docker-compose.yml — forward ORCAROUTER_BASE_URL (default https://api.orcarouter.ai/v1) and ORCAROUTER_API_KEY to the bot, pdf_bot, and api services, alongside the existing OPENAI_API_KEY / OLLAMA_BASE_URL entries.
  • env.example — a new OrcaRouter section documenting both variables.
  • readme.md — documented ORCAROUTER_BASE_URL / ORCAROUTER_API_KEY in the env table and added a note under LLM Configuration.

Why OrcaRouter

OrcaRouter is an OpenAI-compatible AI gateway built for both models and agents. Like OpenRouter, it exposes a provider/model namespace across many models — but it also combines adaptive routing, automatic failover, zero-markup inference, observability, guardrails, and agent-tool governance behind the same endpoint. Adding orcarouter as a first-class provider means this project's users can use that stack directly, without treating OrcaRouter as an anonymous custom base URL.

It also runs gateway-level, zero-trust security for AI agents on the same endpoint — screening every prompt/response and governing every tool call on a default-deny basis, with no application code changes.

Verification

  • python -m py_compile passes on all changed Python files; docker-compose.yml parses cleanly.
  • Live-tested the new load_llm() path with a real key against https://api.orcarouter.ai/v1LLM=orcarouter/fusion-mini returns HTTP 200 and a valid completion.

To try it: set LLM=orcarouter/fusion-mini and ORCAROUTER_API_KEY=sk-orca-... in your .env, then docker compose up.


I'm an engineer on the OrcaRouter team.

Discord: discord.gg/YEubt8enRA · X: https://x.com/OrcaRouter

Mirror the existing OpenAI provider wiring so the GenAI Stack can use
OrcaRouter models through the same LangChain ChatOpenAI integration.
Set LLM=orcarouter/<model> (for example orcarouter/fusion-mini) and add
ORCAROUTER_API_KEY to the .env file.

Live-tested against the OrcaRouter API (HTTP 200).

Signed-off-by: clementguarino06510-glitch <clementguarino06510-glitch@users.noreply.github.com>

Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: clementguarino06510-glitch <clementguarino06510-glitch@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant