feat(prompt): --environment flag

[Cedric / ViaDézo1er (viadezo1er)]

Adds bt prompts ...:

• versions subcommand
• (un)assign subcommands
• --environment flag for list/view/assign/unassign and bt functions view
• --version flag for view

Usage in the examples bellow.

Prompt version ids have a long form (19 characters) and a short form (16 characters), both are valid when requested.

In case you were unaware of the existence of environments like me, they are in https://www.braintrust.dev/app/your-org/p/your-project/configuration/org/environments

Examples:

Details

❯ bt prompts view test-varied-claude-haiku-short --environment test-1
Viewing Test Varied - Claude Haiku Short
Environment: test-1
Version: 1000197749656649734
Model: claude-3-5-haiku-20241022
  max_tokens:             180
  stop_sequences:         [
]
  temperature:            0

│ Classify {{input}} as positive, neutral, or negative. Return one label.

❯ bt prompts list --environment test-1
4 prompts found in ced / My Project for environment test-1

Name                                    Description                            Slug                                   Version                  
Test Varied - GPT-4o Creative           Synthetic prompt with varied           test-varied-gpt-4o-creative            1000197744929341442      
                                        model settings for CLI testing.                                                                        
Test Varied - GPT-4o Mini               Synthetic deterministic prompt,        test-varied-gpt-4o-mini-determini      1000197749656518660      
Deterministic                           revised for environment tes…           stic                                                            
Test Varied - Gemini Flash              Synthetic prompt with varied           test-varied-gemini-flash               1000197744929341442      
                                        model settings for CLI testing.                                                                        
Test Varied - Claude Haiku Short        Synthetic short-form prompt,           test-varied-claude-haiku-short         1000197749656649734      
                                        revised for environment testin…                                                                       

❯ bt prompts list --environment incorrect-env-name
error: request failed (400 Bad Request): {"Code":"BadRequestError","Message":"Environment with slug 'incorrect-env-name' not found [user_email=cedric@braintrustdata.com] [timestamp=1787685069.048]"}

❯ bt prompts assign test-varied-claude-haiku-short --version 1000197749656649734 --environment test-2
✓ Assigned prompt 'test-varied-claude-haiku-short' version 1000197749656649734 to environment 'test-2'

❯ bt prompts unassign test-varied-claude-haiku-short --environment test-2
✓ Unassigned prompt 'test-varied-claude-haiku-short' from environment 'test-2'

❯ bt prompts versions
✔ Select prompt · q
2 versions found for q-b138

 Version          
 4304fee66c80c5c8 
 8124d96283514d0f

https://linear.app/braintrustdata/issue/SDK-244/environment-tags-in-the-bt-cli
See also the backend PR, and the environment management pr.

[github-actions]

Latest downloadable build artifacts for this PR commit 0d4b76a94c63:

• Workflow run: https://github.com/braintrustdata/bt/actions/runs/33002742198
• Download all artifacts (GitHub CLI): gh run download 33002742198 --repo braintrustdata/bt
• Installers are published from main automatically. To publish one for a PR branch, run release-canary manually via workflow_dispatch.

Available artifact names

• artifacts-build-global
• artifacts-build-local-x86_64-apple-darwin
• artifacts-build-local-aarch64-pc-windows-msvc
• artifacts-build-local-x86_64-pc-windows-msvc
• artifacts-build-local-x86_64-unknown-linux-musl
• artifacts-build-local-aarch64-apple-darwin
• artifacts-build-local-x86_64-unknown-linux-gnu
• artifacts-build-local-aarch64-unknown-linux-gnu
• artifacts-plan-dist-manifest
• cargo-dist-cache

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 1a1b66369d

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "Codex (@codex) review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "Codex (@codex) address that feedback".

[Luca Forstner (lforst)]

one thing

[Luca Forstner (lforst)]

are we properly paginating here?

[Cedric / ViaDézo1er (viadezo1er)]

No.

(lines numbers from the branch cedric/endpoint-check-reject-unknown-slug for braintrust)

Audit logs don't support pagination:
braintrust/api-ts/src/btql.ts:1610:

  if (body.audit_log) {
    // Audit log queries do not support pagination.
    boundQuery.limit = undefined;
    boundFilter.limit = undefined;
  }

And cursors are not supported braintrust/api-ts/src/btql.ts:1971:

  if (!useBrainstore && body.audit_log) {
    adjustBoundQueryForAuditLog({
      boundQuery: boundFilter,
      objectType: objectType,
      testingOnlyAllowQueryFullAuditLog:
        body._testing_only_allow_query_full_audit_log,
    });
    if (cursorParams) {
      throw new BadRequestError("Audit log queries do not support cursors");
    }
  }

Btw the python SDK doesn't paginate either when querying audit logs braintrust-sdk-python/py/src/braintrust/logger.py:6146:

    resp = _state.api_conn().post(
        "btql",
        json={
            "query": query,
            "audit_log": True,
            "use_columnstore": False,
            "brainstore_realtime": True,
        },
        headers={"Accept-Encoding": "gzip"},
    )

.... all according to the slop

[Luca Forstner (lforst)]

I don't think this code branch is related to audit logs in any way right? I was more asking about pagination for prompts list. Why are audit logs relevant?