update api - #2488
Open
Yongbozzz wants to merge 1 commit into
Open
Conversation
Signed-off-by: Yongbozzz <yongbo.zhu@intel.com>
Dependency Review✅ No vulnerabilities or license issues found.Scanned Files
|
Contributor
There was a problem hiding this comment.
Pull request overview
Adds chat/completion API support, dynamic JSON agent configuration, UI updates, deployment tooling, Helm resources, and ARC integration tests.
Changes:
- Adds chat generation, readiness checks, and agent streaming.
- Adds JSON configuration editing and validation.
- Updates Docker, model tooling, Helm deployments, documentation, and ARC tests.
Reviewed changes
Copilot reviewed 39 out of 39 changed files in this pull request and generated 17 comments.
Show a summary per file
| File | Summary | Final findings |
|---|---|---|
EdgeCraftRAG/ui/vue/src/views/settings/components/Agent/components/UpdateDialog.vue |
Adds dynamic configuration validation. | No final comment. |
EdgeCraftRAG/ui/vue/src/views/settings/components/Agent/components/DynamicConfigs.vue |
Adds JSON editing and validation. | Moderate (2 votes): Normalize JSON.stringify’s possible undefined result before assigning to jsonText; strict TypeScript currently fails. |
EdgeCraftRAG/ui/vue/src/views/chatbot/components/Chatbot/MessageItem.vue |
Handles image preview clicks. | No final comment. |
EdgeCraftRAG/ui/vue/src/views/chatbot/components/Chatbot/Chat.vue |
Updates scrolling and observer cleanup. | No final comment. |
EdgeCraftRAG/ui/vue/src/i18n/zh.ts |
Adds Chinese validation messages. | No final comment. |
EdgeCraftRAG/ui/vue/src/i18n/en.ts |
Adds English validation messages. | No final comment. |
EdgeCraftRAG/ui/docker/replace-env.sh |
Renders runtime Nginx configuration. | No final comment. |
EdgeCraftRAG/ui/docker/nginx.conf |
Adds configurable proxy targets. | No final comment. |
EdgeCraftRAG/ui/docker/Dockerfile.ui |
Installs runtime Nginx configuration. | No final comment. |
EdgeCraftRAG/tools/README.md |
Documents model-download options. | Nit (3 votes): DOWNLOAD_EMBEDDING_RERANKER=0 is documented but not implemented by model_download.sh. |
EdgeCraftRAG/tools/README_zh.md |
Documents model downloads in Chinese. | Nit (3 votes): DOWNLOAD_EMBEDDING_RERANKER=0 is documented but not implemented by model_download.sh. |
EdgeCraftRAG/tools/model_download.sh |
Adds OpenVINO model handling. | No final comment. |
EdgeCraftRAG/tests/test_compose_vllm_on_arc.sh |
Adds ARC vLLM validation. | Moderate (3 votes): Validate response content, not only HTTP status. Moderate (2 votes): Prefix validation URLs with http://. |
EdgeCraftRAG/tests/test_compose_vllm_on_arc_b60.sh |
Adds B60 vLLM validation. | Moderate (3 votes): Validate response content, not only HTTP status. Moderate (2 votes): Prefix validation URLs with http://. |
EdgeCraftRAG/tests/test_compose_on_arc.sh |
Adds ARC integration validation. | Moderate (2 votes): Prefix validation URLs with http://. Moderate (2 votes): Compare EXPECTED_RESULT and CONTENT. |
EdgeCraftRAG/kubernetes/helm/values.yaml |
Adds Helm deployment configuration. | No final comment. |
EdgeCraftRAG/kubernetes/helm/templates/service-llm-serving-xpu.yaml |
Exposes the LLM-serving service. | No final comment. |
EdgeCraftRAG/kubernetes/helm/templates/service-edgecraftrag-ui.yaml |
Exposes the UI service. | No final comment. |
EdgeCraftRAG/kubernetes/helm/templates/service-edgecraftrag-server.yaml |
Exposes the pipeline server. | No final comment. |
EdgeCraftRAG/kubernetes/helm/templates/service-ecrag.yaml |
Exposes the mega service. | No final comment. |
EdgeCraftRAG/kubernetes/helm/templates/deployment-edgecraftrag-ui.yaml |
Deploys the UI. | No final comment. |
EdgeCraftRAG/kubernetes/helm/templates/deployment-ecrag.yaml |
Deploys the mega service. | No final comment. |
EdgeCraftRAG/kubernetes/helm/templates/daemonset-llm-serving-xpu.yaml |
Deploys LLM serving. | No final comment. |
EdgeCraftRAG/kubernetes/helm/templates/daemonset-edgecraftrag-server.yaml |
Deploys pipeline servers. | Critical (1 vote): A multi-node DaemonSet can route requests to pods without the in-memory pipeline state; use a single replica or shared state/request routing. |
EdgeCraftRAG/kubernetes/helm/templates/configmap-env.yaml |
Defines shared environment settings. | Critical (1 vote): Set vLLM_ENDPOINT to the llm-serving-xpu service; otherwise the server falls back to localhost. |
EdgeCraftRAG/kubernetes/helm/README.md |
Documents Helm deployment. | Nit (3 votes): Replace nonexistent edgecraftrag/values.yaml references with the chart’s current values.yaml path. |
EdgeCraftRAG/kubernetes/helm/README_zh.md |
Adds Chinese Helm documentation. | Moderate (3 votes): Install the current chart with ./, not nonexistent ./edgecraftrag. Nit (3 votes): Correct the invalid edgecraftrag/values.yaml references. |
EdgeCraftRAG/kubernetes/helm/Chart.yaml |
Defines the Helm chart. | No final comment. |
EdgeCraftRAG/edgecraftrag/utils.py |
Supports chat prompt templates. | No final comment. |
EdgeCraftRAG/edgecraftrag/requirements.txt |
Updates runtime dependencies. | No final comment. |
EdgeCraftRAG/edgecraftrag/components/model.py |
Adds benchmark chat methods. | Critical (3 votes): chat_with_bench returns only the first character because complete_with_bench returns completion[0]; return the full decoded completion. |
EdgeCraftRAG/edgecraftrag/components/generator.py |
Adds chat/completion execution paths. | No final comment. |
EdgeCraftRAG/edgecraftrag/components/agents/simple.py |
Propagates stream writers. | No final comment. |
EdgeCraftRAG/edgecraftrag/components/agent.py |
Adds writer fallback and propagation. | No final comment. |
EdgeCraftRAG/edgecraftrag/api/v1/chatqna.py |
Adds the readiness endpoint. | Moderate (3 votes): Validate every supported inference type and fail when no usable generator exists, rather than probing only vLLM. |
EdgeCraftRAG/docs/API_Guide.md |
Updates API documentation. | Nit (2 votes): Retain or update documentation for the implemented /v1/data/files endpoints. |
EdgeCraftRAG/docker_compose/intel/gpu/arc/README.md |
Updates ARC deployment documentation. | No final comment. |
EdgeCraftRAG/docker_compose/intel/gpu/arc/README_zh.md |
Updates Chinese ARC documentation. | No final comment. |
EdgeCraftRAG/docker_compose/intel/gpu/arc/compose.yaml |
Configures API mode. | No final comment. |
Suppressed comments (9)
EdgeCraftRAG/edgecraftrag/components/generator.py:122
- When chat mode is enabled, this branch passes a
list[ChatMessage]to the local model. The benchmark code below later sends that same list directly toBenchmark.cal_input_token_size, whose tokenizer expects text, so enabled local chat streaming ends with acode:0000error after generation instead of recording metrics. Convert the messages to a prompt string before tokenizing.
response = await llm.astream_chat_with_bench(prompt_str)
else:
response = await llm.astream_complete_with_bench(prompt_str)
EdgeCraftRAG/kubernetes/helm/README.md:3
intrudocesis misspelled; useintroducesin this new README sentence.
This doc intrudoces the Helm chart for deploying EdgeCraft RAG (ecrag) on a Kubernetes cluster.
EdgeCraftRAG/kubernetes/helm/README.md:42
Persistantis misspelled; usePersistent.
4. **Persistant Paths**: Ensure the host paths exist for mounting.
EdgeCraftRAG/kubernetes/helm/README.md:58
- This new sentence contains the misspellings
there'reandavaliable; usethere areandavailable.
If there're different clusters avaliable, please install the chart with specific kube config, e.g. :
EdgeCraftRAG/kubernetes/helm/README.md:90
- This sentence repeats the misspellings
there'reandavaliable; usethere areandavailable.
If there're different clusters avaliable, please uninstall the chart with specific kube config, e.g. :
EdgeCraftRAG/kubernetes/helm/templates/daemonset-edgecraftrag-server.yaml:42
- This DaemonSet mounts
/dev/dribut has no node selector or affinity, so it will also be scheduled on CPU-only nodes in a heterogeneous cluster. On nodes without that host path the pod cannot mount its volume and remains unavailable. Restrict the server to nodes that provide the device (or remove the device dependency for CPU deployments).
- name: dri-device
mountPath: /dev/dri
EdgeCraftRAG/kubernetes/helm/templates/daemonset-llm-serving-xpu.yaml:58
- This toleration makes GPU-tainted nodes eligible, but it does not restrict the DaemonSet to GPU nodes. It will also be scheduled on ordinary nodes, where the
/dev/drihostPath and model mount may be absent and the vLLM pod cannot start. Add a node selector or affinity matching the cluster's GPU-node label.
tolerations:
- key: "gpu"
operator: "Exists"
effect: "NoSchedule"
EdgeCraftRAG/tools/model_download.sh:574
- The post-download validation only checks
openvino_model.xmlbecauseopenvino_model_existsdoes not require the matching.binweights file. A partial download can therefore be reported as ready and fail later when the model is loaded; validate the weights file too (or strengthen the shared predicate).
if ! openvino_model_exists "${ov_llm_dir}"; then
echo "[Model Check] ERROR: Download completed but OpenVINO model is incomplete: ${ov_llm_dir}"
exit 1
fi
EdgeCraftRAG/ui/vue/src/views/settings/components/Agent/components/DynamicConfigs.vue:190
- This watcher only assigns the new configuration and never removes keys that disappeared from it. Switching an agent type from a schema with extra keys to one without them therefore leaves stale values in
form, and those values are emitted and submitted with the new agent. Remove keys absent fromdatabefore assigning the replacement.
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
| -F "file=@/path/to/your/document.pdf" | jq '.' | ||
| ``` | ||
|
|
||
| --- |
Comment on lines
+237
to
+240
| generator = pipeline.get_generator(GeneratorType.CHATQNA) | ||
| if generator is not None and generator.inference_type == InferenceType.VLLM: | ||
| try: | ||
| response = requests.get(f"{generator.vllm_endpoint.rstrip('/')}/v1/models", timeout=2) |
Comment on lines
+379
to
+386
| def chat_with_bench(self, messages: Any, **kwargs: Any) -> CompletionResponse: | ||
| """Chat endpoint with benchmark. | ||
|
|
||
| Applies the model's own chat template via messages_to_prompt (local | ||
| tokenizer), then reuses complete_with_bench. | ||
| """ | ||
| prompt = self.messages_to_prompt(messages) | ||
| return self.complete_with_bench(prompt, formatted=True, **kwargs) |
|
|
||
| ## Configuration | ||
|
|
||
| Before installing, you should configure the `edgecraftrag/values.yaml` file according to your environment. |
|
|
||
| ```bash | ||
| cd kubernetes/helm | ||
| helm install edgecraftrag ./edgecraftrag |
| local HTTP_STATUS="${RESPONSE: -3}" | ||
| local CONTENT=$(cat ${LOG_PATH}/${SERVICE_NAME}.log) | ||
|
|
||
| if [ "$HTTP_STATUS" -eq 200 ]; then |
|
|
||
| # setup pipeline | ||
| validate_services \ | ||
| "${HOST_IP}:${EC_RAG_SERVICE_PORT}/v1/settings/pipelines" \ |
Comment on lines
+311
to
+313
| - `DOWNLOAD_EMBEDDING_RERANKER` controls embedding/reranker download checks: | ||
| - `1` (default): keep current embedding/reranker download/check logic | ||
| - `0`: skip embedding/reranker download checks (LLM processing still runs based on mode) |
Comment on lines
+298
to
+300
| - `DOWNLOAD_EMBEDDING_RERANKER` 用于控制 embedding/reranker 下载检查： | ||
| - `1`（默认）：保持当前 embedding/reranker 下载/检查逻辑 | ||
| - `0`：跳过 embedding/reranker 下载检查（LLM 仍按当前 mode 执行） |
|
|
||
| const handleExpand = (field: Field) => { | ||
| if (field.type === "json") { | ||
| jsonText[field.key] = stringifyJsonValue(form[field.key]); |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
New feature support