Skip to content

Fix OpenAI API request option handling - #2295

Open
patrick-andstar wants to merge 1 commit into
QwenLM:mainfrom
patrick-andstar:fix/openai-request-options
Open

Fix OpenAI API request option handling#2295
patrick-andstar wants to merge 1 commit into
QwenLM:mainfrom
patrick-andstar:fix/openai-request-options

Conversation

@patrick-andstar

Copy link
Copy Markdown

Summary

  • Forward the existing max_length request option to the underlying chat generation call.
  • Accept stop as either a string or a list of strings, and normalize a scalar before applying the existing leading-newline workaround.

Motivation

max_length was accepted by the request model but silently ignored. In addition, clients using the standard scalar form of stop received a validation error even though this script exposes an OpenAI-compatible chat-completions endpoint.

Compatibility

Existing list-valued stop input keeps its current behavior. No model loading, streaming protocol, or authentication behavior changes.

Testing

  • Ran a focused stubbed endpoint regression check covering scalar stop, existing list stop, and forwarding max_length to model.chat.
  • Ran an AST syntax check for openai_api.py.
  • Ran git diff --check.
  • Did not run the Docker/model integration test because it requires Docker and an external checkpoint download.

Duplicate Work Check

Searched open Issues and PRs for openai_api, max_length, and stop words; no matching open change was found. Historical PR #1038 added streaming stop-word support but does not cover these request options. Open PR #2255 changes the unrelated plugin-evaluation parser.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant