Skip to content

Update the documentation for both microservices to match muEd #34

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
neagualexa merged 3 commits into from
Aug 4, 2026
Merged

Update the documentation for both microservices to match muEd #34

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
94377a2
update muEd chat functions usage
neagualexa Jul 30, 2026
51ecb48
consistent μEd api naming
neagualexa Jul 30, 2026
c95425e
Updated eval function docs for muEd and the new workflow
m-messer Aug 4, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
14 changes: 9 additions & 5 deletions docs/advanced/chat_functions/local.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,5 +1,7 @@
# Running and Testing Chat function Locally

All request payloads below follow the [μEd API](https://mued.org/) `ChatRequest` schema, which is the specification chat functions on Lambda Feedback follow. Only `messages` is required; `conversationId`, `user`, `context` and `configuration` are optional.

## Run Unit Tests

You can run the unit tests using `pytest`:
Expand Down Expand Up @@ -44,12 +46,12 @@ docker run -e OPENAI_API_KEY={your key} -e OPENAI_MODEL={your LLM chosen model n
docker run --env-file .env -it --name my-lambda-container -p 8080:8080 llm_chat
```

This will start the evaluation function and expose it on port `8080` and it will be open to be curl:
This will start the chat function and expose it on port `8080` and it will be open to be curl:

```bash
curl --location 'http://localhost:8080/2015-03-31/functions/function/invocations' \
--header 'Content-Type: application/json' \
--data '{"body":"{\"conversationId\": \"12345Test\", \"messages\": [{\"role\": \"USER\", \"content\": \"hi\"}], \"user\": {\"type\": \"LEARNER\"}}"}'
--data '{"body":"{\"messages\": [{\"role\": \"USER\", \"content\": \"hi\"}]}"}'
```

### Call Docker Container
Expand All @@ -70,13 +72,15 @@ POST URL:
http://localhost:8080/2015-03-31/functions/function/invocations
```

Body (stringified within `body` for the API request):
Minimal body — only the required `messages` field:

```JSON
{"body":"{\"conversationId\": \"12345Test\", \"messages\": [{\"role\": \"USER\", \"content\": \"hi\"}], \"user\": {\"type\": \"LEARNER\"}}"}
{
"messages": [{ "role": "USER", "content": "hi" }]
}
```

Input Body with optional fields:
Full body as Lambda Feedback sends it, with all optional μEd API fields populated:
```json
{
"conversationId": "<uuid>",
Expand Down
284 changes: 152 additions & 132 deletions docs/advanced/chat_functions/quickstart.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -9,150 +9,170 @@ A chat function is a function which calls Large Language Models (LLMs) to respon

Chat functions host a chatbot. Chatbots capture and automate the process of assisting students during their learning process when outside of classroom.

## The μEd API specification

Chat functions on Lambda Feedback consume the [μEd API](https://mued.org/) request schema. The `messages`, `user`, `context` and `configuration` fields of an incoming request follow the μEd `ChatRequest` format, and the chat function translates them into a tutoring prompt (in the boilerplate this is done by `src/agent/context.py`).

Per the μEd `ChatRequest` schema, **only `messages` is required** — `conversationId`, `user`, `context` and `configuration` are all optional. Refer to [mued.org](https://mued.org/) for the authoritative field definitions.

## Getting Setup for Development

1. Get the code on your local machine (Using github desktop or the `git` cli)

- For new functions: clone the template repo for [chat-function-boilerplate](https://github.com/lambda-feedback/chat-function-boilerplate). **Make sure the new repository is set to public (it needs access to organisation secrets)**.
- For new functions: use the template repo [chat-function-boilerplate](https://github.com/lambda-feedback/chat-function-boilerplate) via *Use this template > Create a new repository*, choosing the `Lambda Feedback` organisation as the owner. **Make sure the new repository is set to public (it needs access to organisation secrets and GitHub deployment protection rules)**.
- For existing functions: please make your changes on a new separate branch

2. _If you are creating a new chatbot_, you can either edit the `src/agents/base_agent` or copy it and rename it based on the name of your chatbot.
3. You are now ready to start making changes and implementing features by editing each of the main function-logic files:
2. Add your LLM credentials to a `.env` file in the root of the repository. OpenAI, Google AI and Ollama are supported out of the box:

1. **`src/agents/{base_agent}/{base}_agent.py`**: This file contains the main LLM pipeline using [LangGraph](https://langchain-ai.github.io/langgraph/) and [LangChain](https://python.langchain.com/docs/introduction/).

- the chat function expects the following arguments when it being called:

Body with necessary fields:

```JSON
{
"conversationId": "12345Test",
"messages": [{ "role": "USER", "content": "hi" }],
"user": { "type": "LEARNER" }
}
```

Body with optional fields:

```JSON
{
"conversationId": "12345Test",
"messages": [
{ "role": "USER", "content": "<previous user message>" },
{ "role": "ASSISTANT", "content": "<previous assistant reply>" },
{ "role": "USER", "content": "hi" }
],
"user": {
"type": "LEARNER",
"preference": { "conversationalStyle": "<stored style string>" },
"taskProgress": {
"timeSpentOnQuestion": "30 minutes",
"accessStatus": "a good amount of time spent on this question today.",
"markedDone": "This question is still being worked on.",
"currentPart": {
"position": 0,
"timeSpentOnPart": "10 minutes",
"markedDone": "This part is not marked done.",
"responseAreas": [
{
"responseType": "EXPRESSION",
"totalSubmissions": 3,
"wrongSubmissions": 2,
"latestSubmission": {
"submission": "<student's last answer>",
"feedback": "<feedback text from evaluator>",
"answer": "<reference answer used for evaluation>"
}
}
]
}
}
},
"context": {
"summary": "<compressed chat history>",
"set": { "title": "Fundamentals", "number": 2, "description": "<set description>" },
"question": {
"title": "Understanding Polymorphism",
"number": 3,
"guidance": "<teacher guidance>",
"content": "<master question content>",
"estimatedTime": "15-25 minutes",
"parts": [
{
"position": 0,
"content": "<part prompt>",
"answerContent": "<part answer>",
"workedSolutionSections": [
{ "position": 0, "title": "Step 1", "content": "..." }
],
"structuredTutorialSections": [
{ "position": 0, "title": "Hint", "content": "..." }
],
"responseAreas": [
{
"position": 0,
"responseType": "EXPRESSION",
"answer": "<reference answer>",
"preResponseText": "<label shown before input>"
}
]
}
]
}
}
}
```

Expected response:

```JSON
{
"output": { "role": "ASSISTANT", "content": "<assistant reply text>" },
"metadata": {
"summary": "<updated chat summary>",
"conversationalStyle": "<updated style string>",
"processingTimeMs": 1234
}
}
```

2. **`src/agents/{base_agent}/{base}_prompts.py`**: This is where you can write the system prompts that describe how your AI Assistant should behave and respond to the user.

3. _If you edited the chatbot agent file name_, make sure to add your chatbot `invoke()` function to the `module.py` file.

4. Update the `config.json` file with the name of the chat function.

5. Please add a `README.md` file to describe the use and behaviour of your chatbot.

4. Changes can be tested locally by running the pipeline tests using:
```bash
pytest
# If you use OpenAI:
OPENAI_API_KEY
OPENAI_MODEL

# If you use GoogleAI:
GOOGLE_AI_API_KEY
GOOGLE_AI_MODEL

# If you use Ollama:
OLLAMA_MODEL
OLLAMA_BASE_URL
OLLAMA_API_KEY
```
[Running and Testing Chat Functions Locally](local.md){ .md-button }


5. Merge commits into dev branch will trigger the `dev.yml` workflow, which will build the docker image, push it to a shared `dev` ECR repository and deploy an AWS Lambda function available to any http requests. In order to make your new chatbot available on the `dev` environment of the Lambda Feedback platform, you will have to get in contact with the ADMINS on the platform.
!!! note
If you use another endpoint than the one set in the template, update the `.github/workflows/{staging-deploy,production-deploy,test-lint}.yml` files so they pass the right secrets and variables during testing and deployment.

6. You can now test the deployed chat function using your preferred request client (such as [Insomnia](https://insomnia.rest/) or [Postman](https://www.postman.com/) or simply `curl` from a terminal). `DEV` Functions are made available at:
```url
https://<***>.execute-api.eu-west-2.amazonaws.com/default/chat/<function name as defined in config.json>
```
3. You are now ready to start making changes and implementing features by editing each of the main function-logic files:

!!! example "Example Request to chatFunctionBoilerplate-dev"
curl --location 'https://<***>.execute-api.eu-west-2.amazonaws.com/default/chat/chatFunctionBoilerplate-dev' \
--header 'Content-Type: application/json' \
--data '{
"conversationId": "12345Test",
"messages": [
{
"role": "USER",
"content": "hi"
}
1. **`src/agent/agent.py`**: This file contains the main LLM pipeline using [LangGraph](https://langchain-ai.github.io/langgraph/) and [LangChain](https://python.langchain.com/docs/introduction/). If you want a custom agent, copy or update this file and import the new invocation in `src/module.py`.

The agent uses **two separate LLM instances** — one for chat responses and one for conversation summarisation and style analysis. By default both use the same provider, but you can point them at different models (e.g. a cheaper or faster model for summarisation) by changing the class used in `agent.py`.

2. **`src/agent/prompts.py`**: This is where you can write the system prompts that describe how your AI Assistant should behave and respond to the user.

3. **`src/agent/llm_factory.py`**: factory classes for each LLM provider. Add your own provider here if it is not supported out of the box.

4. **`src/agent/context.py`**: converts the μEd API `context` and `user` dictionaries into the prompt text given to the LLM.

5. Update the `config.json` file with the name of the chat function.

6. Please add a `README.md` file (and update `docs/`) to describe the use and behaviour of your chatbot.

## Request and response schema

Minimal request — only the required [μEd API](https://mued.org/) fields populated:

```JSON
{
"messages": [{ "role": "USER", "content": "hi" }]
}
```

Full request as Lambda Feedback sends it, with all optional [μEd API](https://mued.org/) fields populated:

```JSON
{
"conversationId": "<uuid>",
"messages": [
{ "role": "USER", "content": "<previous user message>" },
{ "role": "ASSISTANT", "content": "<previous assistant reply>" },
{ "role": "USER", "content": "<current message>" }
],
"user": {
"type": "LEARNER",
"preference": { "conversationalStyle": "<stored style string>" },
"taskProgress": {
"timeSpentOnQuestion": "30 minutes",
"accessStatus": "a good amount of time spent on this question today.",
"markedDone": "This question is still being worked on.",
"currentPart": {
"position": 0,
"timeSpentOnPart": "10 minutes",
"markedDone": "This part is not marked done.",
"responseAreas": [
{
"responseType": "EXPRESSION",
"totalSubmissions": 3,
"wrongSubmissions": 2,
"latestSubmission": {
"submission": "<student's last answer>",
"feedback": "<feedback text from evaluator>",
"answer": "<reference answer used for evaluation>"
}
}
]
}
}
},
"context": {
"summary": "<compressed chat history>",
"set": { "title": "Fundamentals", "number": 2, "description": "<set description>" },
"question": {
"title": "Understanding Polymorphism",
"number": 3,
"guidance": "<teacher guidance>",
"content": "<master question content>",
"estimatedTime": "15-25 minutes",
"parts": [
{
"position": 0,
"content": "<part prompt>",
"answerContent": "<part answer>",
"workedSolutionSections": [
{ "position": 0, "title": "Step 1", "content": "..." }
],
"structuredTutorialSections": [
{ "position": 0, "title": "Hint", "content": "..." }
],
"user": { "type": "LEARNER" }
}'
"responseAreas": [
{
"position": 0,
"responseType": "EXPRESSION",
"answer": "<reference answer>",
"preResponseText": "<label shown before input>"
}
]
}
]
}
}
}
```

Expected response:

```JSON
{
"output": { "role": "ASSISTANT", "content": "<assistant reply text>" },
"metadata": {
"summary": "<updated chat summary>",
"conversationalStyle": "<updated style string>",
"processingTimeMs": 1234
}
}
```

## Testing your changes

Changes can be tested locally by running the pipeline tests using:

```bash
pytest
```

[Running and Testing Chat Functions Locally](local.md){ .md-button }

## Deploying to Lambda Feedback

Deployment is handled by GitHub Actions, as long as the repository is within the [Lambda Feedback organisation](https://github.com/lambda-feedback). Add your API key as a repository **secret** and your LLM model name as a repository **variable** under `Settings > Secrets and variables > Actions`, using the same names as in your `.env` file.

The pipeline has two environments:

- **Staging** — pushing to the `main` branch triggers the `staging-deploy.yml` workflow, which runs the test suite and (on success) deploys the chat function to AWS. After deploying, contact one of the Lambda Feedback admins to make the function accessible on `lambdafeedback.com`.

- **Production** — once you are happy with the staging deployment, create a `Release Request` using the button on the `README.md`, and run the `production-deploy.yml` workflow manually from the GitHub Actions tab. Pick a `version-bump` (`patch`/`minor`/`major`); the workflow redeploys staging, pauses on a manual approval stage (to be reviewed by a Lambda Feedback admin), then creates a `vX.Y.Z` git tag and GitHub Release and deploys to the main [Lambda Feedback platform](https://www.lambdafeedback.com/). Finally, contact one of the Lambda Feedback admins to make the function accessible on `lambdafeedback.com`.

7. Once the `dev` chat function is fully tested, you can merge the code to the default branch (`main`). This will trigger the `main.yml` workflow, which will deploy the `staging` and `prod` versions of your chat function. Please contact the ADMIN to provide you the URLS for the `staging` and `prod` versions of your chat function.
Pull requests trigger the `test-lint.yml` workflow, which runs the test suite only — no deploy.

8. In order to make your new chat function available on any of the environments of the Lambda Feedback platform, you will have to get in contact with the ADMINS on the platform.
!!! note
Once a deployment has been successful, share the necessary environment variables (e.g. API key and LLM model) with one of the Lambda Feedback team members so the function can be enabled on the platform.
36 changes: 26 additions & 10 deletions docs/advanced/evaluation_functions/quickstart.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -31,23 +31,39 @@ python -m unittest app/evaluation_tests.py
```
[Running and Testing Functions Locally](local.md){ .md-button }

5. Merge commits into the default branch will trigger the `test-and-deploy.yml` workflow, which will build the docker image, push it to a shared ECR repository, then call the backend `grading-function/ensure` route to build the necessary infrastructure to make the function available from the client app.
5. The pipeline has two environments:

- **Staging** — pushing to the `main` branch triggers the `staging-deploy.yml` workflow, which runs the test suite and (on success) builds and deploys the docker image to staging.

- **Production** — once you are happy with the staging deployment, run the `production-deploy.yml` workflow manually from the GitHub Actions tab, picking a `version-bump` (`patch`/`minor`/`major`).

Pull requests trigger the `test-lint.yml` workflow, which runs the test suite only — no deploy.

!!! note
The build and deploy steps are implemented as reusable workflows maintained in [lambda-feedback/evaluation-function-workflows](https://github.com/lambda-feedback/evaluation-function-workflows).

6. You can now test the deployed evaluation function using your prefered request client (such as [Insomnia](https://insomnia.rest/) or [Postman](https://www.postman.com/) or simply `curl` from a terminal). Functions are made available at:
```url
https://c1o0u8se7b.execute-api.eu-west-2.amazonaws.com/default/<function name as defined in config.json>
```

!!! example "Example Request to SymbolicEqual"
```
curl --request GET \
--url https://c1o0u8se7b.execute-api.eu-west-2.amazonaws.com/default/symbolicEqual \
--header 'Content-Type: application/json' \
--header 'command: eval' \
--data '{"response": "x + x", "answer": "2*x"}'
```
!!! example "Example µEd Request to SymbolicEqual"
```bash
curl --request POST \
--url https://c1o0u8se7b.execute-api.eu-west-2.amazonaws.com/default/symbolicEqual/evaluate \
--header 'Content-Type: application/json' \
--data '{
"submission": { "type": "MATH", "content": { "expression": "x + x" } },
"task": { "referenceSolution": { "expression": "2*x" } }
}'
```

See the [µEd API](specification.md#ed-api) section of the specification for full request/response details. Functions still running the **Legacy** API instead use the `command` header — see [Legacy API](specification.md#legacy-api).

7. In order to make your new function available on the LambdaFeedback platform, you have to register it via the [Admin Panel]({{ urls.client }}admin/functions). This is done by supplying its name, url (the same as the one above) and supported response types.

7. In order to make your new function available on the LambdaFeedback platform, you have to register it via the [Admin Panel]({{ urls.client }}admin/functions). This is done by supplying its name, url (the same as the one above) and supported response types.
!!! note
New evaluation functions should be registered as **µEd** (a standard, path-based API — see [Chat Functions](../chat_functions/quickstart.md) for a general introduction to µEd on Lambda Feedback, and [mued.org](https://mued.org/) for the specification). The **Legacy** command-header API documented on this page is being phased out — only a small number of functions that haven't yet migrated still use it.

## More Info

Expand Down
Loading
Loading