diff --git a/CHANGELOG.md b/CHANGELOG.md index 4999e4a1d3..b97349e43d 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -18,6 +18,46 @@ --- +## Unreleased + +- ✨ Three changes published from the console now end the running session, so they reach the + visitor at their next interaction instead of waiting for that session to end on its own: a + session sample rate of 0 while the visitor is being collected — the emergency stop — a rate of + 100 while they are not, and a stricter Session Replay privacy level while they are being + collected. The session that ends is collected to its end as it began, so no recording is left + masked in one half and plain in the other. Every other change — any rate between 0 and 100, a + loosening privacy level, the replay and trace rates — still waits for the next session. Custom + values wait on their own too, but not once `beforeSampling` turns them into one of the three: a + callback answering 0 for the values just published ends the session exactly as a published 0 + would. Nothing here happens without `remoteConfigurationEnabled: true`. +- 📝 What you will see on the day you publish one of those three: session counts rise and average + session length drops, because each affected visitor's running session is split at that moment; a + replay in progress ends at the split, and the session that follows draws again, so it carries a + new recording only if that draw keeps one; a rate of 100 makes previously invisible visitors + appear within hours rather than the next day, so collected volume climbs the same day. That is + the change taking effect, not a defect. +- 📝 "At once" means "as soon as this client hears of the change". Settings are fetched at page load + and at each new session, never on a timer, so a page nobody reloads hears of a publish at its next + session boundary — at most four hours away, the cap on a session's life. Opening a tab or + reloading any page fetches immediately and ends the session every tab shares, which is why a + visitor who touches the site converges in seconds. A change that is not one of the three still + takes effect one session after that. +- 📝 The three act on what actually changed, not on the activation mode recorded with the publish: + a change the console files as "next session" still ends the running session if it is one of them. +- 📝 `beforeSampling` is now also consulted when settings arrive, away from any draw, to work out + which rate would apply. It must stay free of side effects and answer the same way for the same + input: a callback that draws its own lottery — answering 0 or 100 at random — can end a session + that a steady answer would have left running. +- 📝 A session forced with `setForcedSession()` is not ended by a rate while it is being collected: + forcing decides whether this visitor is collected, and every draw the page makes is collected + whatever the console says, so ending it would only produce the same session again. A + stricter privacy level still ends it, because forcing says nothing about how much of the page may + be uploaded in the clear. The page forces the next session on its own, so the visit continues as + two sessions. +- 📝 Turning remote configuration off is itself a change: the rates go back to the ones passed to + `init`. On a site whose init rate is 0, switching it off stops collection at once rather than at + the next session. + ## v0.2.0 - 💥 **Breaking**: `remoteConfigurationId` is gone from `RumInitConfiguration`. It fetched a diff --git a/packages/rum-core/src/domain/configuration/configuration.ts b/packages/rum-core/src/domain/configuration/configuration.ts index ee15cedb80..62cc56db14 100644 --- a/packages/rum-core/src/domain/configuration/configuration.ts +++ b/packages/rum-core/src/domain/configuration/configuration.ts @@ -62,6 +62,12 @@ export interface RumInitConfiguration extends InitConfiguration { * a single session. Keep it a pure decision: side effects will be repeated, and only the last * call's return value is used. * + * The SDK also calls it away from a draw: when new settings arrive it asks which rate would + * apply now, to decide whether the running session has to end for them to take effect. So it + * must answer the same way for the same input — one that answers differently each time can end a + * session that a steady one would have left running — and anything it does besides returning a + * rate (a metric, a log, a counter) happens more often than there are sessions. + * * Its failure modes never reach session creation: a thrown error or an out-of-range value leaves * the incoming rate in place, and a value that is not a function at all is reported once and * then ignored rather than refusing `init`. @@ -86,9 +92,19 @@ export interface RumInitConfiguration extends InitConfiguration { * Take the sampling rates from the application's settings in the console instead of only from the * values passed here, so they can be changed without releasing a new version of this site. * - * A change applies to sessions started after it arrives; a session already under way keeps the - * decision it was created with. The values below stay in use until the first settings arrive, and - * whenever the settings cannot be reached. + * A change applies to sessions started after it arrives, and a session already under way is never + * re-decided in place. Three changes do not wait for that session to end on its own, because + * their effect on it can be told without drawing again: a session sample rate of 0 while the + * visitor is being collected, a rate of 100 while they are not, and a stricter + * `defaultPrivacyLevel` while they are being collected — a visitor who is not being collected + * records nothing, so a stricter level has no plaintext to catch there. Each of those ends the + * current session, and the visitor's next action starts a new one under the new settings — the + * old session is collected to its end as it was begun, so no recording is left masked in one + * half and plain in the other. Every other change, a loosening privacy level included, waits for + * the next session. + * + * The values below stay in use until the first settings arrive, and whenever the settings cannot + * be reached. * * Requires `localStorage`. Sessions themselves are kept in a cookie unless `sessionPersistence` * says otherwise, but this SDK already reads one `localStorage` entry on every site — the record diff --git a/packages/rum-core/src/domain/configuration/remoteConfiguration.spec.ts b/packages/rum-core/src/domain/configuration/remoteConfiguration.spec.ts index 214a097342..7ba6456c20 100644 --- a/packages/rum-core/src/domain/configuration/remoteConfiguration.spec.ts +++ b/packages/rum-core/src/domain/configuration/remoteConfiguration.spec.ts @@ -196,6 +196,83 @@ describe('remoteConfiguration', () => { }) }) + describe('announcing that new settings are in storage', () => { + function watchStoredNotifications() { + const notified = jasmine.createSpy('remoteConfigurationStored') + lifeCycle.subscribe(LifeCycleEventType.REMOTE_CONFIGURATION_STORED, notified) + return notified + } + + it('announces settings that reached storage, so a subscriber can act on them', (done) => { + const notified = watchStoredNotifications() + + interceptor.withMockXhr((xhr) => { + xhr.complete(200, body({ rum: { sessionSampleRate: 0 } })) + + expect(notified).toHaveBeenCalledTimes(1) + done() + }) + start(configurationWith()) + }) + + it('stays silent about settings it refused as older than the ones it holds', (done) => { + localStorage.setItem(setup!.storeKey, JSON.stringify({ sessionSampleRate: 42, version: 8 })) + const notified = watchStoredNotifications() + + interceptor.withMockXhr((xhr) => { + xhr.complete(200, body({ version: 7, rum: { sessionSampleRate: 0 } })) + + // Nothing changed in storage, so nothing downstream may behave as though it had. + expect(notified).not.toHaveBeenCalled() + done() + }) + start(configurationWith()) + }) + + it('stays silent about an answer that repeats the settings it already holds', (done) => { + localStorage.setItem(setup!.storeKey, JSON.stringify({ sessionSampleRate: 42, version: 7 })) + const notified = watchStoredNotifications() + + interceptor.withMockXhr((xhr) => { + xhr.complete(200, body({ version: 7, rum: { sessionSampleRate: 42 } })) + + // The ordinary answer: every new session asks again and most find nothing changed. A + // subscriber woken by those would act on no news, once per session, for as long as the + // visitor stays. + expect(notified).not.toHaveBeenCalled() + done() + }) + start(configurationWith()) + }) + + it('stays silent when the answer never reached storage', (done) => { + const notified = watchStoredNotifications() + spyOn(Storage.prototype, 'setItem').and.throwError('storage is full') + + interceptor.withMockXhr((xhr) => { + xhr.complete(200, body({ rum: { sessionSampleRate: 0 } })) + + // The next draw will not find these settings, so ending a session for their sake would end + // it for nothing. + expect(notified).not.toHaveBeenCalled() + done() + }) + start(configurationWith()) + }) + + it('stays silent about an answer that never made it', (done) => { + const notified = watchStoredNotifications() + + interceptor.withMockXhr((xhr) => { + xhr.complete(500) + + expect(notified).not.toHaveBeenCalled() + done() + }) + start(configurationWith()) + }) + }) + describe('refusing a payload it cannot read', () => { const STORED = { sessionSampleRate: 42, version: 2 } diff --git a/packages/rum-core/src/domain/configuration/remoteConfiguration.ts b/packages/rum-core/src/domain/configuration/remoteConfiguration.ts index 2180dfbc8f..190e6df96f 100644 --- a/packages/rum-core/src/domain/configuration/remoteConfiguration.ts +++ b/packages/rum-core/src/domain/configuration/remoteConfiguration.ts @@ -20,17 +20,23 @@ declare const __BUILD_ENV__SDK_VERSION__: string * masks a page by default. * * A change only affects sessions created after it arrives, so a visitor is never dropped halfway - * through and never starts being recorded halfway through. Fetching follows the same rhythm: once - * at start-up and once whenever a new session begins — a change can only matter at the next draw, - * so asking more often than sessions are drawn would be requests for nothing. There is no timer - * between sessions. + * through and never starts being recorded halfway through. What "immediately" means for the + * handful of changes that cannot wait is therefore not a flip of the running session but its end: + * see `endSessionIfSettingsAreDecisive` in the session manager, which subscribes to the event this + * module emits once new settings are in storage. + * + * Fetching follows the session's rhythm: once at start-up and once whenever a new session begins — + * a change can only matter at a draw, and every draw is a new session — so asking more often than + * sessions are drawn would be requests for nothing. There is no timer between sessions. The cost + * of that rhythm is that a visitor who never goes idle stays on one session, and so on one set of + * settings, for as long as they keep using the site. * * Three fields the server sends are accepted and ignored, deliberately: `ttl` and * `refresh_on_foreground`, which describe when to ask again and are moot without a timer, and - * `activation`, which offers to end a running session so a change applies at once. Everything here - * is next-session, so a console that ever offers "apply immediately" would not be obeyed by this - * build — named here so the mismatch is found by reading rather than by an operator wondering why - * nothing happened. + * `activation`, which offers to end a running session so a change applies at once. This build ends + * a running session on its own reading of what changed rather than on the server's say-so, so a + * console that offers "apply immediately" as a switch would not be obeyed — named here so the + * mismatch is found by reading rather than by an operator wondering why nothing happened. * * Nothing here runs unless `remoteConfigurationEnabled: true`. Left off — the default — the SDK makes no * extra request and behaves exactly as it did before this existed. @@ -112,6 +118,9 @@ export interface BeforeSamplingContext { * The application's last word on the sampling of the session about to be drawn — see the * `beforeSampling` init option. Returning nothing, or an out-of-range rate, leaves the incoming * value in place. + * + * Must be free of side effects and answer the same way for the same input: it is also called away + * from a draw, to work out which rate newly delivered settings would actually apply. */ export type BeforeSamplingCallback = ( context: BeforeSamplingContext @@ -277,7 +286,13 @@ function keepConfigFresh(configuration: RumConfiguration, setup: RemoteConfigSet } if (response) { failedAttempts = 0 - store(setup, response) + if (store(setup, response)) { + // Announced only once new settings are in storage, because that is where the next draw + // reads them: a subscriber that ends the running session so the new values can take + // effect immediately has to be sure the draw that follows will find them, and must not + // be woken by an answer that changed nothing. + lifeCycle.notify(LifeCycleEventType.REMOTE_CONFIGURATION_STORED) + } return } if (failedAttempts < RETRY_DELAYS.length) { @@ -387,6 +402,12 @@ function fetchRemoteConfiguration( } } +/** + * Writes the response to storage, and answers whether it brought settings this client did not + * already hold. A refused or unwritable response answers `false`, and so does one that repeats the + * version already stored: settings only ever change under a higher number, so by that contract a + * repeat leaves the next draw reading what it would have read anyway. + */ function store(setup: RemoteConfigSetup, response: RemoteConfigurationResponse) { // Settings are published under a number that only ever goes up — rolling back republishes the // old settings under a new, higher one — so a response numbered below what is already stored is @@ -403,9 +424,17 @@ function store(setup: RemoteConfigSetup, response: RemoteConfigurationResponse) // requests that can cross are two pages, and storage is the only thing they share. const storedVersion = readRemoteConfig(setup).version if (storedVersion !== undefined && response.version < storedVersion) { - return + return false } + // Settings only ever change under a higher number, so a response repeating the number already + // stored carries nothing new — and that is the ordinary answer, since every new session refetches + // and most of them find the settings unchanged. It is written anyway, which costs one small + // `setItem` and keeps the entry in the shape this build writes, but it is not announced: a + // subscriber that ends the running session must hear about changes only, or an unchanged answer + // arriving at every renewal would end a session per renewal, forever. + const isNew = storedVersion === undefined || response.version > storedVersion + const values: RemoteConfigValues = { version: response.version } if (response.enabled && response.rum) { // Each value is copied only when the server actually sent it. A knob nobody configured must @@ -437,10 +466,13 @@ function store(setup: RemoteConfigSetup, response: RemoteConfigurationResponse) // settings" looks like — so that the version is kept either way and the console can still see // that this client is up to date with the change that turned it off. localStorage.setItem(setup.storeKey, JSON.stringify(values)) + return isNew } catch { // Storage unavailable, or the origin is out of room. The previous entry stays as it is, which // is the same "keep what is already working" answer a failed request gets — the client goes on - // applying the settings it last stored, and goes on reporting their version. + // applying the settings it last stored, and goes on reporting their version. Reported as a + // failure all the same: nothing downstream may act on settings the next draw will not find. + return false } } diff --git a/packages/rum-core/src/domain/lifeCycle.ts b/packages/rum-core/src/domain/lifeCycle.ts index b1abd3fb46..78b6d9fccb 100644 --- a/packages/rum-core/src/domain/lifeCycle.ts +++ b/packages/rum-core/src/domain/lifeCycle.ts @@ -37,6 +37,18 @@ export const enum LifeCycleEventType { RAW_RUM_EVENT_COLLECTED, RUM_EVENT_COLLECTED, RAW_ERROR_COLLECTED, + + // FLASHCAT FORK - a remote configuration response has just changed what is in storage. Emitted + // only when the write actually happened and actually changed something, so a response refused as + // stale, one that merely repeats the settings already held, and a storage failure all stay + // silent: a subscriber acting on settings the next draw would have read anyway would be acting + // on no news at all. + // + // Added last on purpose. The values of a const enum are inlined at build time and shift when an + // entry is inserted, and everything above this line is upstream's — keeping the fork's own entry + // at the end leaves upstream's numbering alone and keeps this file out of the way of the next + // upstream merge. + REMOTE_CONFIGURATION_STORED, } // This is a workaround for an issue occurring when the Browser SDK is included in a TypeScript @@ -69,6 +81,7 @@ declare const LifeCycleEventTypeAsConst: { RAW_RUM_EVENT_COLLECTED: LifeCycleEventType.RAW_RUM_EVENT_COLLECTED RUM_EVENT_COLLECTED: LifeCycleEventType.RUM_EVENT_COLLECTED RAW_ERROR_COLLECTED: LifeCycleEventType.RAW_ERROR_COLLECTED + REMOTE_CONFIGURATION_STORED: LifeCycleEventType.REMOTE_CONFIGURATION_STORED } // Note: this interface needs to be exported even if it is not used outside of this module, else TS @@ -93,6 +106,7 @@ export interface LifeCycleEventMap { error: RawError customerContext?: Context } + [LifeCycleEventTypeAsConst.REMOTE_CONFIGURATION_STORED]: void } export interface RawRumEventCollectedData { diff --git a/packages/rum-core/src/domain/rumSessionManager.spec.ts b/packages/rum-core/src/domain/rumSessionManager.spec.ts index f122708907..cfa6debf94 100644 --- a/packages/rum-core/src/domain/rumSessionManager.spec.ts +++ b/packages/rum-core/src/domain/rumSessionManager.spec.ts @@ -820,6 +820,399 @@ describe('rum session manager', () => { }) }) + describe('restarting the session when the settings are decisive', () => { + const STORE_KEY = 'test-decisive-settings' + const DRAW_KEY = 'test-decisive-settings-draw' + const REMOTE_SETUP = { + buildUrl: () => 'https://example.com/config', + storeKey: STORE_KEY, + fetchTimeout: 3000, + } + + afterEach(() => localStorage.removeItem(DRAW_KEY)) + + function storeRemote(stored: object) { + localStorage.setItem(STORE_KEY, JSON.stringify(stored)) + registerCleanupTask(() => localStorage.removeItem(STORE_KEY)) + } + + function startWith(configuration: Partial = {}) { + return startRumSessionManagerWithDefaults({ + configuration: { remoteConfig: REMOTE_SETUP, drawStoreKey: DRAW_KEY, ...configuration }, + }) + } + + // Settings reach storage first and are announced afterwards, the order the fetcher uses: the + // draw that may follow reads storage, so it has to find them already there. + function deliver(stored: object) { + storeRemote(stored) + lifeCycle.notify(LifeCycleEventType.REMOTE_CONFIGURATION_STORED) + } + + function isSessionEnded() { + return getSessionState(SESSION_STORE_KEY).isExpired === '1' + } + + describe('the three changes it can decide on its own', () => { + it('ends a session being collected when the rate goes to zero', () => { + storeRemote({ version: 1, sessionSampleRate: 100, sessionReplaySampleRate: 100 }) + startWith({ sessionSampleRate: 100 }) + expect(getSessionState(SESSION_STORE_KEY)[RUM_SESSION_KEY]).toBe(RumTrackingType.TRACKED_WITH_SESSION_REPLAY) + + deliver({ version: 2, sessionSampleRate: 0, sessionReplaySampleRate: 0 }) + + expect(isSessionEnded()).toBeTrue() + }) + + it('ends a session that is not being collected when the rate goes to a hundred', () => { + storeRemote({ version: 1, sessionSampleRate: 0 }) + startWith({ sessionSampleRate: 0 }) + expect(getSessionState(SESSION_STORE_KEY)[RUM_SESSION_KEY]).toBe(RumTrackingType.NOT_TRACKED) + + deliver({ version: 2, sessionSampleRate: 100, sessionReplaySampleRate: 100 }) + + expect(isSessionEnded()).toBeTrue() + }) + + it('ends the session when the privacy level tightens', () => { + storeRemote({ version: 1, sessionSampleRate: 100, defaultPrivacyLevel: 'allow' }) + startWith({ sessionSampleRate: 100, defaultPrivacyLevel: 'allow' }) + + deliver({ version: 2, sessionSampleRate: 100, defaultPrivacyLevel: 'mask-user-input' }) + + expect(isSessionEnded()).toBeTrue() + }) + + it('ends the session on the tightening step that masks everything', () => { + storeRemote({ version: 1, sessionSampleRate: 100, defaultPrivacyLevel: 'mask-user-input' }) + startWith({ sessionSampleRate: 100, defaultPrivacyLevel: 'allow' }) + + deliver({ version: 2, sessionSampleRate: 100, defaultPrivacyLevel: 'mask' }) + + expect(isSessionEnded()).toBeTrue() + }) + + it('ends a session drawn before any settings arrived when the first ones tighten the level', () => { + // Nothing in storage yet, so this session was drawn on the init values — and a draw that + // lands exactly on them records nothing, which is why the level it runs under can only be + // read back off init. The recorder falls back the same way, so this is the level the page + // is really being masked with. + startWith({ sessionSampleRate: 100, defaultPrivacyLevel: 'allow' }) + + deliver({ version: 1, sessionSampleRate: 100, defaultPrivacyLevel: 'mask' }) + + expect(isSessionEnded()).toBeTrue() + }) + + it('ends a collected session when the callback turns the delivered values into a zero', () => { + storeRemote({ version: 1, sessionSampleRate: 100 }) + startWith({ + sessionSampleRate: 100, + beforeSampling: ({ custom }) => (custom?.optOut === true ? { sessionSampleRate: 0 } : undefined), + }) + expect(getSessionState(SESSION_STORE_KEY)[RUM_SESSION_KEY]).not.toBe(RumTrackingType.NOT_TRACKED) + + // The response carries no rate at all: the console ships the data and the application's own + // code turns it into the decision. Asking the callback away from a draw is the whole reason + // that decision can reach the session already running. + deliver({ version: 2, custom: { optOut: true } }) + + expect(isSessionEnded()).toBeTrue() + }) + + it('ends a collected session when the settings are switched off and init never collected', () => { + storeRemote({ version: 1, sessionSampleRate: 100 }) + startWith({ sessionSampleRate: 0 }) + expect(getSessionState(SESSION_STORE_KEY)[RUM_SESSION_KEY]).not.toBe(RumTrackingType.NOT_TRACKED) + + // Turning remote configuration off in the console stores the version and nothing else, so + // the rates go back to the ones the site passed to init. That is a change like any other, + // and here it is the decisive one. + deliver({ version: 2 }) + + expect(isSessionEnded()).toBeTrue() + }) + + it('draws the session that follows on the settings that have just landed', () => { + storeRemote({ version: 1, sessionSampleRate: 0 }) + startWith({ sessionSampleRate: 0 }) + + deliver({ version: 2, sessionSampleRate: 100, sessionReplaySampleRate: 100 }) + clock.tick(STORAGE_POLL_DELAY) + document.dispatchEvent(createNewEvent(DOM_EVENT.CLICK)) + + expect(getSessionState(SESSION_STORE_KEY)[RUM_SESSION_KEY]).toBe(RumTrackingType.TRACKED_WITH_SESSION_REPLAY) + }) + }) + + describe('everything else waits for the next session', () => { + it('leaves the session alone when the rate moves to a value it cannot decide on', () => { + storeRemote({ version: 1, sessionSampleRate: 100 }) + startWith({ sessionSampleRate: 100 }) + + deliver({ version: 2, sessionSampleRate: 30 }) + + expect(expireSessionSpy).not.toHaveBeenCalled() + expect(isSessionEnded()).toBeFalse() + }) + + it('leaves a session that is not collected alone when the rate merely rises', () => { + storeRemote({ version: 1, sessionSampleRate: 0 }) + startWith({ sessionSampleRate: 0 }) + + deliver({ version: 2, sessionSampleRate: 30 }) + + expect(expireSessionSpy).not.toHaveBeenCalled() + expect(isSessionEnded()).toBeFalse() + }) + + it('leaves a session that is not being collected alone when the privacy level tightens', () => { + storeRemote({ version: 1, sessionSampleRate: 0, defaultPrivacyLevel: 'allow' }) + startWith({ sessionSampleRate: 0, defaultPrivacyLevel: 'allow' }) + expect(getSessionState(SESSION_STORE_KEY)[RUM_SESSION_KEY]).toBe(RumTrackingType.NOT_TRACKED) + + // Nothing is being recorded for this visitor, so there is no plaintext for the stricter + // level to catch and nothing to gain by ending their session. + deliver({ version: 2, sessionSampleRate: 0, defaultPrivacyLevel: 'mask' }) + + expect(expireSessionSpy).not.toHaveBeenCalled() + expect(isSessionEnded()).toBeFalse() + }) + + it('does not end one sampled-out session after another as settings keep arriving', () => { + // A session that is not collected is given no id, so no record of its draw is kept and the + // level it was drawn under cannot be read back. Ending it would not change that, so acting + // on the comparison would end every session this visitor is ever given. + storeRemote({ version: 1, sessionSampleRate: 0, defaultPrivacyLevel: 'allow' }) + startWith({ sessionSampleRate: 0, defaultPrivacyLevel: 'allow' }) + + deliver({ version: 2, sessionSampleRate: 0, defaultPrivacyLevel: 'mask' }) + clock.tick(STORAGE_POLL_DELAY) + document.dispatchEvent(createNewEvent(DOM_EVENT.CLICK)) + deliver({ version: 3, sessionSampleRate: 0, defaultPrivacyLevel: 'mask' }) + + expect(expireSessionSpy).not.toHaveBeenCalled() + expect(isSessionEnded()).toBeFalse() + }) + + it('leaves the session alone when the privacy level loosens', () => { + storeRemote({ version: 1, sessionSampleRate: 100, defaultPrivacyLevel: 'mask' }) + startWith({ sessionSampleRate: 100 }) + + // Being slow here is the point: it leaves an operator time to undo a mistake, and what it + // costs meanwhile is more of the data already being collected. + deliver({ version: 2, sessionSampleRate: 100, defaultPrivacyLevel: 'allow' }) + + expect(expireSessionSpy).not.toHaveBeenCalled() + expect(isSessionEnded()).toBeFalse() + }) + + it('leaves the session alone when only the custom bag changed', () => { + storeRemote({ version: 1, sessionSampleRate: 100, custom: { cohort: 'a' } }) + startWith({ sessionSampleRate: 100 }) + + deliver({ version: 2, sessionSampleRate: 100, custom: { cohort: 'b' } }) + + expect(expireSessionSpy).not.toHaveBeenCalled() + expect(isSessionEnded()).toBeFalse() + }) + + it('leaves the session alone when only the trace rate changed', () => { + storeRemote({ version: 1, sessionSampleRate: 100, traceSampleRate: 10 }) + startWith({ sessionSampleRate: 100 }) + + deliver({ version: 2, sessionSampleRate: 100, traceSampleRate: 90 }) + + expect(expireSessionSpy).not.toHaveBeenCalled() + expect(isSessionEnded()).toBeFalse() + }) + + it('leaves the session alone when only the replay rate changed', () => { + storeRemote({ version: 1, sessionSampleRate: 100, sessionReplaySampleRate: 100 }) + startWith({ sessionSampleRate: 100 }) + + // The replay rate is deliberately not one of the three: it decides a draw nested inside the + // session draw, and a rule for it would have to say what happens to a replay the host + // application forced on. Until that is settled, a replay rate change waits for the next + // session like every other change. + deliver({ version: 2, sessionSampleRate: 100, sessionReplaySampleRate: 0 }) + + expect(expireSessionSpy).not.toHaveBeenCalled() + expect(isSessionEnded()).toBeFalse() + }) + + it('does not fall over when the site never opted in and has no settings store', () => { + storeRemote({ version: 1, sessionSampleRate: 100 }) + startRumSessionManagerWithDefaults({ configuration: { sessionSampleRate: 0, drawStoreKey: DRAW_KEY } }) + + // Such a site never fetches, so the announcement can only ever be reached by hand and the + // store key below is one nothing would look under. All this pins down is that the decision + // survives `remoteConfig` being undefined; that the opt-out is respected is settled where + // the fetcher is never started, not here. + deliver({ version: 2, sessionSampleRate: 100 }) + + expect(expireSessionSpy).not.toHaveBeenCalled() + expect(isSessionEnded()).toBeFalse() + }) + + it('has nothing to end when the session is already over', () => { + storeRemote({ version: 1, sessionSampleRate: 0 }) + startWith({ sessionSampleRate: 0 }) + expireCookie() + clock.tick(STORAGE_POLL_DELAY) + expireSessionSpy.calls.reset() + + // There is no session to read a decision off, and nothing to end: the next activity draws + // on what has just been stored, which is all this change needs. + expect(() => deliver({ version: 2, sessionSampleRate: 100 })).not.toThrow() + expect(expireSessionSpy).not.toHaveBeenCalled() + }) + }) + + describe('what it compares', () => { + it('never draws again to reach its decision', () => { + storeRemote({ version: 1, sessionSampleRate: 100 }) + startWith({ sessionSampleRate: 100 }) + const draw = spyOn(Math, 'random').and.callThrough() + + deliver({ version: 2, sessionSampleRate: 30 }) + + // Drawing here would be a second lottery on top of the one the next session runs, quietly + // turning a rate p into p². + expect(draw).not.toHaveBeenCalled() + }) + + it('compares against the level the session was drawn under, not the settings stored since', () => { + storeRemote({ version: 1, sessionSampleRate: 100, defaultPrivacyLevel: 'mask' }) + startWith({ sessionSampleRate: 100, defaultPrivacyLevel: 'allow' }) + + // A loosening leaves the running session masking everything, as it was drawn to. + deliver({ version: 2, sessionSampleRate: 100, defaultPrivacyLevel: 'allow' }) + expect(expireSessionSpy).not.toHaveBeenCalled() + + // Stricter than what was stored a moment ago, still looser than what this session actually + // masks with. Judged against the stored settings it would end a session with nothing to + // gain from restarting. + deliver({ version: 3, sessionSampleRate: 100, defaultPrivacyLevel: 'mask-user-input' }) + + expect(expireSessionSpy).not.toHaveBeenCalled() + expect(isSessionEnded()).toBeFalse() + }) + + it('lets beforeSampling have the last word on the rate it judges', () => { + storeRemote({ version: 1, sessionSampleRate: 100 }) + startWith({ + sessionSampleRate: 100, + beforeSampling: ({ sessionSampleRate }) => ({ sessionSampleRate: sessionSampleRate === 0 ? 50 : 100 }), + }) + + // The console says zero, the application puts it back in the middle: the rate that would + // actually apply is fifty, which decides nothing. + deliver({ version: 2, sessionSampleRate: 0 }) + + expect(expireSessionSpy).not.toHaveBeenCalled() + expect(isSessionEnded()).toBeFalse() + }) + }) + + describe('arriving more than once', () => { + it('does not end the session a second time when the same settings arrive again', () => { + storeRemote({ version: 1, sessionSampleRate: 100, sessionReplaySampleRate: 100 }) + startWith({ sessionSampleRate: 100 }) + + deliver({ version: 2, sessionSampleRate: 0 }) + expect(isSessionEnded()).toBeTrue() + + clock.tick(STORAGE_POLL_DELAY) + document.dispatchEvent(createNewEvent(DOM_EVENT.CLICK)) + expect(getSessionState(SESSION_STORE_KEY)[RUM_SESSION_KEY]).toBe(RumTrackingType.NOT_TRACKED) + expireSessionSpy.calls.reset() + + // Another tab, a retry, a reload: the same answer arrives again and finds the difference + // that justified ending a session already gone. + lifeCycle.notify(LifeCycleEventType.REMOTE_CONFIGURATION_STORED) + lifeCycle.notify(LifeCycleEventType.REMOTE_CONFIGURATION_STORED) + + expect(expireSessionSpy).not.toHaveBeenCalled() + expect(isSessionEnded()).toBeFalse() + }) + + it('stops tightening the privacy level once the session is drawn under it', () => { + storeRemote({ version: 1, sessionSampleRate: 100, defaultPrivacyLevel: 'allow' }) + startWith({ sessionSampleRate: 100, defaultPrivacyLevel: 'allow' }) + + deliver({ version: 2, sessionSampleRate: 100, defaultPrivacyLevel: 'mask' }) + expect(isSessionEnded()).toBeTrue() + + clock.tick(STORAGE_POLL_DELAY) + document.dispatchEvent(createNewEvent(DOM_EVENT.CLICK)) + expireSessionSpy.calls.reset() + + lifeCycle.notify(LifeCycleEventType.REMOTE_CONFIGURATION_STORED) + + expect(expireSessionSpy).not.toHaveBeenCalled() + expect(isSessionEnded()).toBeFalse() + }) + }) + + describe('a session the host application forced', () => { + function startForced(configuration: Partial = {}) { + const rumSessionManager = startWith({ sessionSampleRate: 0, ...configuration }) + rumSessionManager.setForcedSession() + clock.tick(STORAGE_POLL_DELAY) + document.dispatchEvent(createNewEvent(DOM_EVENT.CLICK)) + expect(getSessionState(SESSION_STORE_KEY)[RUM_SESSION_KEY]).toBe(RumTrackingType.TRACKED_WITH_SESSION_REPLAY) + expireSessionSpy.calls.reset() + return rumSessionManager + } + + it('is still ended by a rate of a hundred when the session it adopted collects nothing', () => { + storeRemote({ version: 1, sessionSampleRate: 0 }) + setCookie(SESSION_STORE_KEY, `rum=0&created=${Date.now()}&expire=${Date.now() + DURATION}`, DURATION) + const rumSessionManager = startWith({ sessionSampleRate: 0 }) + + // Forcing ends a session that collects nothing, so that the next draw can be the forced + // one. Before that draw happens, a tab that never forced anything starts a session of its + // own, and this page adopts it: the page is forced while the session it holds is not. + rumSessionManager.setForcedSession() + setCookie(SESSION_STORE_KEY, `rum=0&created=${Date.now()}&expire=${Date.now() + DURATION}`, DURATION) + clock.tick(STORAGE_POLL_DELAY) + document.dispatchEvent(createNewEvent(DOM_EVENT.CLICK)) + expect(getSessionState(SESSION_STORE_KEY)[RUM_SESSION_KEY]).toBe(RumTrackingType.NOT_TRACKED) + expireSessionSpy.calls.reset() + + // Here the rate has something to change, so the exemption does not apply: ending the + // session is what lets the next draw be the forced one this page asked for. + deliver({ version: 2, sessionSampleRate: 100 }) + + expect(isSessionEnded()).toBeTrue() + }) + + it('is not ended by a rate, since every draw it makes is collected anyway', () => { + storeRemote({ version: 1, sessionSampleRate: 0 }) + startForced() + + // Ending it would only replace it with another forced session — the same difference, for + // as long as the page lives. + deliver({ version: 2, sessionSampleRate: 0 }) + + expect(expireSessionSpy).not.toHaveBeenCalled() + expect(isSessionEnded()).toBeFalse() + }) + + it('is still ended when the privacy level tightens', () => { + storeRemote({ version: 1, sessionSampleRate: 0, defaultPrivacyLevel: 'allow' }) + startForced({ defaultPrivacyLevel: 'allow' }) + + // Forcing decides whether this visitor is collected. It says nothing about how much of + // their page may be uploaded in the clear. + deliver({ version: 2, sessionSampleRate: 0, defaultPrivacyLevel: 'mask' }) + + expect(isSessionEnded()).toBeTrue() + }) + }) + }) + function startRumSessionManagerWithDefaults({ configuration, trackingConsentState = createTrackingConsentState(TrackingConsent.GRANTED), diff --git a/packages/rum-core/src/domain/rumSessionManager.ts b/packages/rum-core/src/domain/rumSessionManager.ts index 423c3c9c27..dc82b415c9 100644 --- a/packages/rum-core/src/domain/rumSessionManager.ts +++ b/packages/rum-core/src/domain/rumSessionManager.ts @@ -1,6 +1,7 @@ -import type { DefaultPrivacyLevel, RelativeTime, TrackingConsentState } from '@flashcatcloud/browser-core' +import type { RelativeTime, TrackingConsentState } from '@flashcatcloud/browser-core' import { BridgeCapability, + DefaultPrivacyLevel, Observable, SESSION_TIME_OUT_DELAY, STORAGE_POLL_DELAY, @@ -206,6 +207,88 @@ export function startRumSessionManager( lifeCycle.notify(LifeCycleEventType.SESSION_RENEWED) }) + // FLASHCAT FORK - a change published mid-session normally waits for that session to end on its + // own, which for a visitor who never goes idle is hours away. Three changes cannot afford the + // wait, and what makes exactly those three special is that their outcome for the running session + // can be asserted without drawing again: + // + // - a session sample rate of 0 while this session is being collected: nothing is meant to be + // collected any more, and this is the emergency stop the console offers; + // - a session sample rate of 100 while this session is not: everything is meant to be + // collected, and this visitor is the exception; + // - a stricter default privacy level while this session is being collected: every further + // second recorded is a second of plaintext uploaded, and masking cannot reach back for it. + // + // No other rate says anything about whether THIS session should have been kept — only a second + // draw could, and drawing twice silently turns a rate p into p². So everything else waits for + // the next session, a loosening privacy level included. Loosening waits on purpose: the delay + // is what leaves an operator room to undo a mistake, and what it costs meanwhile is more of the + // data already being collected. + // + // The action is always to end the session and let the next activity start a new one — never to + // flip the running one, which would leave a replay masked in its first half and plain in its + // second, or invent a session that begins in the middle of a visit. + // + // It stays idempotent with no bookkeeping at all: it compares what this session was drawn under + // against what a draw would use now, and ending the session is exactly what makes that + // difference disappear. The same response arriving again — another tab, a retry, a reload — + // finds nothing left to act on. + function endSessionIfSettingsAreDecisive() { + const session = sessionManager.findSession() + if (!session) { + // Nothing to end. Whatever starts the next session draws on the settings just stored, which + // is the ordinary path and already gives them their effect. + return + } + + const remote = readRemoteConfig(configuration.remoteConfig) + + // Whether this session is collected is read off the session itself rather than reconstructed + // by comparing rates: the session IS the outcome its draw produced, and an outcome is the only + // thing 0 and 100 let us assert anything about. + const isCollected = isTypeTracked(session.trackingType) + + // Only a session that is being collected can be recording, and only a recording can be too + // plain. A sampled-out visitor uploads nothing, so a stricter level has nothing to protect + // there — and nothing to compare against either: a session that is not collected is given no + // id, so no draw is recorded for it and what it was drawn under cannot be read back here. The + // comparison would fall through to the init value on every announcement and keep answering + // "tighter", ending one empty session after another for as long as the visitor stays. + if (isCollected) { + // What this session is masking pages with right now, which is not the previously stored + // settings: settings are stored while a session runs, and the session was drawn under + // whatever was stored before that. No record means the draw used the init value, and so does + // the recorder — see `startRecording`, which falls back the same way. + const drawnPrivacyLevel = drawnHistory.find()?.defaultPrivacyLevel ?? configuration.defaultPrivacyLevel + const nextPrivacyLevel = remote.defaultPrivacyLevel ?? configuration.defaultPrivacyLevel + if (PRIVACY_LEVEL_STRICTNESS[nextPrivacyLevel] > PRIVACY_LEVEL_STRICTNESS[drawnPrivacyLevel]) { + sessionManager.expire() + return + } + } + + if (forcedSession && isCollected) { + // The host application has taken this page off the rates deliberately, and every draw it + // makes from now on is collected whatever the console says. Ending a collected session on a + // rate would only replace it with another collected one — the same difference, forever. That + // reasoning runs out when the session is not collected: this page can adopt one an unforced + // tab drew, and there a rate of 100 has something to change, so it is left to the rule + // below. The flag is this page's either way — another tab that never called + // `setForcedSession` reads the shared session as an ordinary one. + return + } + + const { sessionSampleRate } = resolveSampleRates(configuration, remote) + if ((sessionSampleRate === 0 && isCollected) || (sessionSampleRate === 100 && !isCollected)) { + sessionManager.expire() + } + } + + const remoteConfigSubscription = lifeCycle.subscribe( + LifeCycleEventType.REMOTE_CONFIGURATION_STORED, + endSessionIfSettingsAreDecisive + ) + sessionManager.sessionStateUpdateObservable.subscribe(({ previousState, newState }) => { if (!previousState.forcedReplay && newState.forcedReplay) { const sessionEntity = sessionManager.findSession() @@ -238,6 +321,7 @@ export function startRumSessionManager( expireObservable: sessionManager.expireObservable, stop: () => { consentSubscription.unsubscribe() + remoteConfigSubscription.unsubscribe() drawnHistory.stop() }, setForcedReplay: () => sessionManager.updateSessionState({ forcedReplay: '1' }), @@ -387,34 +471,7 @@ function computeSessionState( // the decision it was created with: settings arriving mid-session never start or stop // collecting for a visitor already on the site. const remote = readRemoteConfig(configuration.remoteConfig) - - let sessionSampleRate = remote.sessionSampleRate ?? configuration.sessionSampleRate - let sessionReplaySampleRate = remote.sessionReplaySampleRate ?? configuration.sessionReplaySampleRate - - // FLASHCAT FORK - the application gets the last word, right at the draw. This is what turns the - // delivered custom values into sampling decisions without a wasted first draw or a session - // restart: the console ships the data (an allow-list, a cohort rule), the application's own - // code interprets it here. Its failure modes must never reach session creation, so a thrown - // error or a value outside 0..100 leaves the incoming rate in place. - if (configuration.beforeSampling) { - try { - const override = configuration.beforeSampling({ - sessionSampleRate, - sessionReplaySampleRate, - custom: remote.custom, - }) - if (override) { - if (isRate(override.sessionSampleRate)) { - sessionSampleRate = override.sessionSampleRate - } - if (isRate(override.sessionReplaySampleRate)) { - sessionReplaySampleRate = override.sessionReplaySampleRate - } - } - } catch (e) { - display.error('beforeSampling threw an error:', e) - } - } + const { sessionSampleRate, sessionReplaySampleRate } = resolveSampleRates(configuration, remote) reportDraw(configuration, remote, sessionSampleRate, sessionReplaySampleRate, onDraw) @@ -432,6 +489,56 @@ function computeSessionState( } } +/** + * FLASHCAT FORK - the rates a draw would use right now: what the console delivered, falling back to + * what the site passed to init, with the application's `beforeSampling` given the last word. This + * is what turns the delivered custom values into sampling decisions without a wasted first draw or + * a session restart: the console ships the data (an allow-list, a cohort rule), the application's + * own code interprets it here. Its failure modes must never reach session creation, so a thrown + * error or a value outside 0..100 leaves the incoming rate in place. + * + * It resolves rates and never draws on them, which is what lets the same question be asked away + * from a draw — see `endSessionIfSettingsAreDecisive`, which needs to know which rate would apply + * without spending a lottery ticket to find out. + */ +function resolveSampleRates(configuration: RumConfiguration, remote: RemoteConfigValues) { + let sessionSampleRate = remote.sessionSampleRate ?? configuration.sessionSampleRate + let sessionReplaySampleRate = remote.sessionReplaySampleRate ?? configuration.sessionReplaySampleRate + + if (configuration.beforeSampling) { + try { + const override = configuration.beforeSampling({ + sessionSampleRate, + sessionReplaySampleRate, + custom: remote.custom, + }) + if (override) { + if (isRate(override.sessionSampleRate)) { + sessionSampleRate = override.sessionSampleRate + } + if (isRate(override.sessionReplaySampleRate)) { + sessionReplaySampleRate = override.sessionReplaySampleRate + } + } + } catch (e) { + display.error('beforeSampling threw an error:', e) + } + } + + return { sessionSampleRate, sessionReplaySampleRate } +} + +/** + * FLASHCAT FORK - how much of a page each level keeps out of a recording, ordered so two levels can + * be compared. Only the direction matters: tightening is the change that cannot be undone after the + * fact, because a second already recorded in the clear has already been uploaded in the clear. + */ +const PRIVACY_LEVEL_STRICTNESS: { [level in DefaultPrivacyLevel]: number } = { + [DefaultPrivacyLevel.ALLOW]: 0, + [DefaultPrivacyLevel.MASK_USER_INPUT]: 1, + [DefaultPrivacyLevel.MASK]: 2, +} + /** * FLASHCAT FORK - hands the draw that just happened to whoever records it. Both draw branches * report the same shape and differ only in the rates: forcing pins them, an ordinary draw uses