test: align test suite with SDK baseline, and honor the retries option - #598
Merged
Conversation
Bring the Python test suite in line with the scope and strategy used by seamapi/javascript-http: tests are organized by SDK concern, exercise the SDK against fake-seam-connect, and assert against seeded records rather than hand-written stubs. Fixture: - Start the fake directly from node_modules/.bin instead of `npm run start`, so teardown signals the server rather than an npm wrapper. - Poll /health until the server is ready and fail loudly if it exits early, replacing the connect-retry on /_fake/default_seed that made startup flaky. - Scope PORT to the subprocess instead of mutating os.environ. - Add a recording_server fixture for the two things the fake cannot do: asserting what the SDK puts on the wire, and driving retry responses. Scope: - Drop test/workspaces, which covered generated route methods rather than SDK behavior. - Move deep_attr_dict_test.py under test/ so all tests live together. Coverage: - Add serialization, retry, and client tests, and cover the multi workspace client against the fake. - Replace the mocked niquests.Session in the headers test with assertions on the request the server actually received. - Assert against seed ids instead of `len(devices) > 0`. - Cover paginator construction, cursor validation, and last page. - Replace the personal access token xfail with passing tests. The fake rejects that token on /devices/list but authorizes it on /devices/get, which is the route the JavaScript SDK tests use. Two retry tests are marked xfail(strict=True): SeamHttpClient sets self.retries after calling niquests.Session.__init__ without forwarding it, so the mounted HTTPAdapter keeps its default and the retries option is silently ignored. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_011DzapiU8A9NMdyoTybL9xB
SeamHttpClient assigned self.retries after calling niquests.Session.__init__, by which point the session had already mounted its adapters with the default max_retries. The retries option was therefore silently ignored: a caller passing Retry(total=5, status_forcelist=[503]) still got exactly one attempt. Pass retries through to niquests.Session so the mounted adapters are built with it. Seam and SeamMultiWorkspace default the option to None, which now falls back to DEFAULT_RETRIES rather than being dropped. Drops the xfail markers from the two retry tests that covered this. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_011DzapiU8A9NMdyoTybL9xB
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Brings the Python test suite in line with the scope and strategy used by seamapi/javascript-http, then fixes the one SDK bug that work uncovered.
Two commits, split so the fix is reviewable on its own:
test: align test suite with the JavaScript SDK baselinefix: honor the retries optionseam/client.py+ drops two xfail markersThe baseline this follows
The JavaScript SDK has one fixture,
getTestServer, that gives every test a freshly seeded fake. Its test files are named after SDK concerns — auth, env, headers, http errors, pagination, retry, serialization, waiting on action attempts — and they reach for routes likedevices.getonly as a means of exercising the client. There are no per-route tests, because routes are generated.nockappears only where the fake cannot help.Fixture
node_modules/.bininstead ofnpm run start, so teardown signals the server rather than an npm wrapper that may leave the server behind./healthuntil the server is ready and fail loudly if it exits early. Previously the only wait was a connect-retry on/_fake/default_seedwith a 0.1s backoff, which gave up after roughly 1.5s — that is what made startup flaky.PORTto the subprocess instead of mutatingos.environ.recording_serverfixture for the two things the fake cannot do: asserting what the SDK puts on the wire, and driving retry responses.Scope
test/workspaces, which covered generated route methods rather than SDK behavior.deep_attr_dict_test.pyout ofseam/so all tests live undertest/.Coverage
serialization_test.py,retry_test.py, andclient_test.py, and coverSeamMultiWorkspaceagainst the fake.niquests.Sessionin the headers test with assertions on the request the server actually received, including theseam-workspaceheader.len(devices) > 0, so the tests would notice a client talking to the wrong workspace.The personal access token xfail is gone
test/env_test.pycarriedxfail(reason="Fake does not support personal access token."). The fake does support it — it authorizes a personal access token on/devices/getand rejects it only on/devices/list. The JavaScript SDK's equivalent test usesdevices.get, which is why it passes there. Switching the route turns that skipped test into a passing one and adds real coverage forSeam.from_personal_access_token.The fix: retries were silently ignored
SeamHttpClientassignedself.retriesafter callingniquests.Session.__init__. By that point the session had already mounted its adapters with the defaultmax_retries, so the assignment had no effect on anything that actually sends a request. A caller passingRetry(total=5, status_forcelist=[503])got exactly one attempt.The fix passes
retriesthrough toniquests.Sessionso the mounted adapters are built with it.SeamandSeamMultiWorkspaceboth default the option toNone, which previously meant "drop it entirely" and now falls back toDEFAULT_RETRIES.Worth knowing when reviewing: this changes runtime behavior. Requests that never retried will now retry according to whatever policy is configured.
DEFAULT_RETRIESisurllib3.Retry(), whoseallowed_methodscovers idempotent verbs only — and the SDK sendsPOST— so the default path retries connection errors but not status codes, and callers who never passedretriesshould see no change in retry behavior on HTTP responses.Verification
Down from 55s, and the run that produced the previous baseline also hit one flaky startup error.
black --check,pylint ./seam ./test(10.00/10), andrstcheck README.rstare all clean.