Unattended save backup via sysmodule, TLS verification, and server hardening - #7
Open
dmuiX wants to merge 8 commits into
Open
Unattended save backup via sysmodule, TLS verification, and server hardening#7dmuiX wants to merge 8 commits into
dmuiX wants to merge 8 commits into
Conversation
- feat: Skip titles whose save data has not changed since the last upload
- feat: Give each account only the saves that belong to it
The push / pull routines were lambdas inside MainScreen, tied to the GUI
only by their log callback. Moving them into sync.{hpp,cpp} lets a headless
caller reuse the exact same code path, so an app and a background service
cannot drift apart. No behavioural change on its own.
Change detection compares the newest mtime inside each save against
saves/.syncstate and filters during the probe, so unchanged titles are not
even archived. countChangedTitles() answers the same question without
touching the network, which lets a caller decide whether there is anything
to do before opening a socket. collectTargetTitles() is shared so that
"nothing changed" and "nothing to upload" cannot disagree - they were
separate copies of the same filter before.
The console returns every save on the system when asked for a list, without
separating them by user, and that was never filtered. So every account
received every title and spent the round trying to open saves belonging to
someone else. Mounting a save that does not exist also returned the same
value as failing to mount one that does, so a healthy round always ended
"with errors" - and a round that ends with errors never records the time it
succeeded, which meant the next round packed and uploaded everything again
from scratch.
archiveAllSaveData() also could not report failures: it returned OK once it
had walked the list, and per-title errors only reached a callback that
always answered true. A server that was down still produced a return value
of 0, which both callers read as success. Failures are counted and returned
negatively; SAVEDATA_* codes are positive, so the two cannot be confused.
The credentials sit in the URL and libcurl offers them as an Authorization header on the first request. Without verification, anyone who answers the handshake receives the password in the clear. bcrypt protects the hash at rest on the server; it does nothing for this leg. Let's Encrypt validates against the console's built-in list - ISRG Root X1 has been there since 10.1.0 - so the common case needs nothing extra. For a private CA there is sdmc:/uNSS/cacert.pem: this curl uses the libnx SSL backend, which passes CAINFO to sslContextImportServerPki, so a single file can make the console trust a root the firmware never shipped. That is the first thing to try; turning verification off is the last resort, because its failure mode looks like "cannot connect" rather than "certificate refused". Failures now carry the underlying result alongside the -1. That -1 covered "server said 400" and "never connected" with one value, which is what made a DNS problem take a day to find.
Author
|
Sorry first pr version were a bit too many commits. Now its densed down to the important ones. Is working fine on my switch as of now. Have been testing it intensively. And had some problems as you definetly must slim down the installed sysmodule otherwise it will not run. But I suppose this is standing inside the README |
dmuiX
force-pushed
the
main
branch
2 times, most recently
from
August 2, 2026 18:03
a38b27c to
d83c68a
Compare
An NRO only runs while it is open - start a game and it is gone - so unattended backups need a sysmodule. This one boots with the console and stays resident, waking every five minutes to ask whether any save data changed since the last upload. If nothing changed it does nothing at all, not even opening a socket, which is the normal case and costs a few file reads. When a game ends it checks straight away rather than waiting out the interval, since the moment after someone saves and quits is the best time to copy a save. While a game runs it stays off the save files entirely. Backing up at boot instead, which the first version did, sounds reasonable and works badly: a Switch is closed, not switched off, so reboots are weeks apart and the module would finish before the first game even started. Config is re-read every round, so editing config.ini takes effect without a reboot. Shared sources are symlinked from client/source rather than copied, so both binaries always build from the same code. build-all.sh builds the module first and refreshes the copy the app embeds; it also drops the NRO, because the NRO rule depends on the elf and the nacp but never on romfs contents - without that, fixing the module and running make produced an app that still installed the previous one. Notes from bringing this up on hardware (22.5.0 / Atmosphere 1.11.2): - The program id must sit in the custom range. 0x0100000000554E53 was never loaded by boot2 at all; 0x4200000000554E53 works. - Services must be opened in __appInit before smExit(). But opening network, ns and account there runs alongside HID, which then failed to get resources and died with 2001-0132, boot-looping the console. They move to initLateServices() after a 30 second grace period, and the session is kept rather than released - sfdnsres has no initialize function and opens itself per request, so closing the session disabled getaddrinfo while the log still said "network ready". - INNER_HEAP_SIZE is 1 MiB, measured rather than guessed. Sysmodules share one small pool and a static heap leaves it the moment the module loads. 6 MiB starved hid, 2 MiB killed am. 256 KB is too tight in the other direction: a round peaking at 189 KB with a roomy heap peaked at 245 KB with a tight one, and compression, title lookup and upload failed at once. - NsApplicationControlData is 0x24000 bytes and the main thread stack is 16 KB, so asking for a title name overflowed it in the function prologue. Since the module can take the console down, it does not rely on being correct. It parks its own boot2.flag before anything risky and restores it only after a clean round, so a crash costs one error screen rather than a boot loop. Crashes are attributed by program id - ours land in crash_reports, a dying system process in fatal_reports, and the two real incidents landed in different directories. A crash during a backup pauses in growing steps (5, 20, 80 min, capped at two hours) and then retries on its own, because nobody watches a console for notifications and a module that switched itself off would stay off. Only a crash during startup, where retrying means a boot loop, still stands down.
- feat: Show the service log inside the app - feat: Automatic backup on launch, throttled by change rather than time Installing a sysmodule by hand means copying two files into a hex-named folder under atmosphere/contents - easy to get wrong. The module is embedded in the NRO's romfs and the app writes it out itself: "Install background service" copies exefs.nsp, creates the boot2 flag and records the bundled version; "Remove background service" deletes them again. An already installed but outdated module is updated silently on launch, while the first install stays a deliberate button press, since it starts a process at every boot. When the module has stood itself down after a crash, the app offers to re-enable it rather than leaving it stuck. The sysmodule has no screen, so finding out what it had been doing meant pulling the SD card or fetching the log over FTP - which is exactly what once delayed a diagnosis by a day, with the module failing every upload for hours and the only witness a file nobody could see. "Service log" follows the end of that file while new lines arrive and lets go as soon as you scroll up, because a view that jumps around cannot be read. Lines mentioning a failure are red. Only the last 500 lines are held in memory. With sync.autoPushOnLaunch the client starts pushing as soon as it opens. autoPushIntervalHours is a brake rather than a trigger and defaults to 0: what decides whether anything is uploaded is whether the save data changed, so backups happen several times a day while playing, or not for weeks. A fixed interval could only ever be wrong in one of the two directions. The timestamp is written only after a successful run, so a failed backup is retried on the next launch instead of counting as done. A running game keeps its save open, so an automatic push waits for it to end (pm:dmnt); the manual Push button stays unrestricted. pm:dmnt is opened before smExit, because the lazy open inside isGameRunning() failed afterwards and the fail-open path then reported "no game running" - which would have let a backup archive a save file the game still held open. remote.insecureSkipVerify is read here and passed to the HTTP layer.
- chore: Add tests for the revision transaction and the wire format created_at is CURRENT_TIMESTAMP and only has second resolution, so two uploads of the same title within one second get identical timestamps. ORDER BY created_at DESC then has no defined order, and SQLite was measured returning the oldest row - so restoring could hand back a previous save, and the save list could advertise a revision that is not the current one. Both queries were affected. GROUP BY title_id HAVING MAX(created_at) reduces each title to one row but never says which row, and returned the oldest for the same reason. Both now select the newest row explicitly and break ties on rowid, which grows with insertion order. Using one rule in both places matters: a disagreement between the list and the per-title lookup would mean the client downloads a different revision than the one it chose from the list. The tests cover this and the rest of the revision transaction - a pending revision must not hide the last completed one, a completed revision must not be writable again - plus the plain text wire format the C++ client parses by hand, where an empty list has to be an empty body rather than "[]". Both new revision tests fail against the previous queries, which is the point of them. Each test gets a fresh server in a temporary directory, so they neither touch nor need a real database.
- fix: Run the server as a non-root user - feat: Declare the server port in the image - chore: exec uvicorn so signals reach the server main.py requests loop="uvloop" but the package was not listed in requirements.txt, so the Docker container crashed on startup. run-linux.sh now execs uvicorn, so signals reach the server rather than the wrapping shell - killing that shell used to leave uvicorn orphaned. Switching the container off root is not just a USER line: the database path is relative, so metadata.sqlite lives in /app, and SQLite writes its journal next to the database. Without write access to the directory - not merely the file - every write fails with "unable to open database file" even when the bind mount is correct. So /app is chowned at build time and the image drops to that account. UID and GID are build args (default 1000:100) and can also be overridden per deployment with the compose user: key; the mounted database and savedata directory have to belong to whichever is used. EXPOSE opens nothing on the host; it records the port in the image metadata, where reverse proxies read it. Traefik picks it up on its own as long as exactly one port is declared, so a deployment no longer needs a port label to keep in sync. Without either, Traefik builds no router and answers 404, which reads like a routing problem rather than a missing port.
Builds server/Dockerfile on pushes to main and on version tags, for amd64 and arm64. Uses the automatically provided GITHUB_TOKEN, so no secret has to be configured. Images pushed to ghcr.io are private until their visibility is changed by hand. The owner name is lowercased before it reaches a tag - ghcr.io rejects upper case, and account names can contain it. There is deliberately no paths filter. It would apply to the tag push as well: there is only one push block, and GitHub evaluates the filter for every ref in it. Tagging a release on a commit that happens not to touch server/ - a docs or client commit, which is the common case - would then silently produce no image and no version tags at all. The cost of leaving it out is a rebuild on commits that change nothing in the image, which takes well under a minute; a missing release does not announce itself.
- docs: Add compose file for running the server behind Traefik
Covers the new [sync] keys, what the automatic push does and the two rules
that bound it, and a section on the background service: how to install it
from the app, what it reads, where it logs, and why restoring is
deliberately not part of it.
The memory section deliberately states no threshold. Ten module starts were
logged on hardware: the run that took the console down had 4780 KB free in
the system pool at its low point, while five runs that survived a full
round reached 3448-3960 KB. Free memory does not separate them, so no
figure can honestly be drawn from the data, and the text says that rather
than quoting one that would look authoritative and be wrong. What the
crashes had in common was fixed in code, not in free space. The advice that
remains is qualitative - fewer resident sysmodules is more headroom - plus
how to read the module's own pool samples to compare a console against
itself before and after a change.
compose.traefik.yml is for setups that already terminate TLS in a proxy: it
joins an external network and publishes no port of its own. Access control
is basicAuth rather than a login portal because the client cannot follow
redirects, so anything redirect-based can never complete, while libcurl
does send credentials taken straight from the URL.
Getting the hash to Traefik is the awkward part, because a container label
cannot be filled from a secret store - Compose resolves ${...} while
parsing, long before a store injects anything, and Traefik discards a
router whose middleware is invalid, so the service answers 404 rather than
prompting. A helper container bridges that: it reads the credentials from
its environment, hashes the password with bcrypt cost 12 and writes the
middleware into Traefik's dynamic configuration, which Traefik reads on its
own. It stays resident with a healthcheck that tests the file, so the
server can depend on service_healthy rather than on it having run once, and
it refuses to produce a middleware that lets everyone in.
Also records which characters the password may contain, since it travels
inside a URL in config.ini where @ splits the host off and # truncates the
rest, and the ini parser strips quotes from both ends of a value.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Adds unattended save backup, hardens the server for real deployment, and
brings the docs in line with what the code actually does.
Everything here has been run on hardware — firmware 22.5.0 with Atmosphere
1.11.2, backing up three accounts against a server behind Traefik.
Background service (the main addition)
An NRO only runs while it is open, so an app can never back up a save that
changed while you were playing. This adds a sysmodule that starts with the
console and stays resident.
It wakes every five minutes and asks one question: has any save data changed
since the last upload? If not it does nothing at all — not even opening a
socket — which is the normal case and costs a few file reads. When a game ends
it checks immediately instead of waiting out the interval. While a game runs it
stays off the save files entirely, because the game holds them open.
Backing up at boot, which the first version did, sounds reasonable and works
badly: a Switch is closed, not switched off, so reboots are weeks apart and the
module would almost never run. What triggers a backup is change, not the clock.
NRO's romfs, so there is no copying files into hex-named folders by hand.
Service login the app reads the module's log back — a sysmodule has noscreen, and the alternative was pulling the SD card.
MainScreenintosync.{hpp,cpp}so the app and theservice run the exact same code path and cannot drift apart.
It can take the console down, and it is built to survive that
It has killed
hidandamduring development, each time via2001-0132(kernel
LimitReached) — an error that names the victim, not the cause. So themodule does not rely on being correct:
boot2.flagbefore anything risky and puts it back onlyafter a clean round. If it dies, the flag stays parked and the next boot
comes up without it. A boot loop is structurally impossible rather than
merely unlikely.
crash_reports, our program id in thefilename) from "a system process died" (
fatal_reports) — the two realincidents landed in different directories.
console for notifications, so a module that switched itself off would stay
off. It backs off in steps instead — 5 min, 20, 80, capped at two hours — and
retries on its own. Only a crash during startup, where retrying means a boot
loop, still stands down.
The README documents the memory situation in detail, including the measurements
that refused to support a threshold: the run that killed the console had more
free pool at its low point than five runs that survived. It says so rather than
quoting a number that would look authoritative and be wrong.
It also documents a second limit that is not the pool. Adding one more resident
sysmodule alongside this one made every homebrew launch panic the console
immediately after boot —
Atmosphere panic occurred!, program id0100000000000034. That id isfatal, Atmosphère's own crash reporter, whichcould not get a session for
lblto turn the backlight on and aborted; thereport decodes to
2021-0003,sm::ResultOutOfSessions. It lands inatmosphere/fatal_errors/, whilecrash_reports/andfatal_reports/stayempty, so the obvious places suggest nothing happened. Removing the extra
sysmodule fixed it. The README writes down how to read that report, and is
explicit that this is one A/B on one console: it shows one module too many
was enough, not that any particular module is at fault.
Client
libcurl offers them preemptively, so without verification anyone answering
the handshake gets the password. Private CAs go in
sdmc:/uNSS/cacert.pem;remote.insecureSkipVerifyis the last resort.pm:dmnt); the manualPush button stays unrestricted.
system when asked for a list, so all accounts were previously trying to open
each other's — and "this account never played that game" was being reported
as a failure, which kept a healthy round from ever recording success.
Server
text wire format the C++ client parses by hand, and the
P→C/Drevision transaction. Both new tests fail against the previous queries.
created_athas one-secondresolution, and
ORDER BY created_at DESCthen returned the oldest row, soa restore could hand back a previous save. Ties now break on rowid, and the
list and per-title lookup use the same rule so they cannot disagree.
uvloopdependency —main.pyrequested it, so thecontainer crashed on startup.
arm64, and declare the port in the image.
and no login flow can ever complete. The stack takes the password in plain
text and hashes it itself with bcrypt cost 12, so only one representation of
the secret has to be maintained.
Notes
tests pinning it.
both binaries always build from the same code.
build-all.shbuilds the module first and refreshes the copy inside theNRO — the NRO rule depends on the elf and nacp, never on romfs contents, so
fixing the module and running
makeinclient/otherwise produced an appthat still installed the previous one.
🤖 Generated with Claude Code