Key rotation has two readers and one writing authority

The new key worked. A malformed old-key fallback still made a valid current-key credential fail before the reader got a chance to try it.

That bug came from evaluation order. The decrypt function prepared every candidate key before trying any ciphertext. If the previous operator key was malformed, candidate construction failed before decryption under the current key began. We were asking fallback configuration to be valid even when the primary credential was sufficient.

I moved key preparation inside the per-candidate attempt. The reader now tries the current key and returns immediately when authentication succeeds; only a failed current attempt proceeds to the previous key. This is a small code change with a broad reliability lesson: fallback configuration should not be able to break a working primary path.

Two readers, one writer

During an operator-managed migration, the service may need two read keys: the current key and a previous key that can still open older ciphertext. New credentials should be encrypted under the current key only. That gives the system overlap for reading existing records without creating a second write authority.

The order matters. This is the core of the final reader:

for (const secretValue of [secret, ...(previous ? [previous] : [])]) {
  try {
    const candidate = await key(secretValue);
    const [version, nonce, cipher, extra] = value.split('.');
    if (version !== 'v1' || extra !== undefined) throw Error();
    const iv = Uint8Array.from(atob(nonce), (c) => c.charCodeAt(0));
    if (iv.length !== 12) throw Error();
    return new TextDecoder().decode(
      await crypto.subtle.decrypt(
        { name: 'AES-GCM', iv, additionalData: new TextEncoder().encode(id) },
        candidate,
        Uint8Array.from(atob(cipher), (c) => c.charCodeAt(0)),
      ),
    );
  } catch {
    // Try the previous operator key during a bounded rotation window.
  }
}

The try covers key decoding and import, envelope validation, nonce decoding and authenticated decryption. So a malformed previous key is not touched when the current key opens the ciphertext. If the current key cannot open it, the previous candidate gets its own attempt. New writes use the current key through a separate encrypt path; the previous key is only an input to decryption.

That separation is easy to lose in a refactor. If candidate keys are eagerly imported into an array, the fallback can fail before the primary attempt. If a writer tries both keys, the service can create fresh records that prolong the migration indefinitely. The read path may be tolerant for a bounded operator-run migration; the write path should remain singular and explicit.

Bind ciphertext to its account

The envelope is versioned as v1, with a 12-byte random nonce and AES-GCM authenticated encryption. The account ID is passed as additional authenticated data. That means the encrypted value is bound to the row identity: copying a ciphertext from one account into another does not make it valid under the other account’s ID. The authentication check fails.

The implementation validates the configured key as 32 bytes before importing it. The nonce comes from the cryptographic random source. Those are concrete properties of this code path, not a claim that the whole key-management program is complete. The environment still has to deliver the intended current and previous values, the operator must control who can invoke re-encryption, and the migration must verify its inventory.

I want tests at both the cryptographic helper and the API route. The helper test proves the account-binding property directly. The route test exercises the bug that motivated the change: it stores an account credential under the current key, supplies malformed previous-key values, and expects the claimed worker route to still return the credential. A separate rotation test uses two account fixtures, reads old ciphertext with the previous key, re-encrypts both under the current key, checks that two records were rewritten and none skipped, then confirms the old key no longer opens those newly written ciphertexts. It also checks that the audit contains account identifiers without credential material.

That fixture result is not a production migration count. It demonstrates the expected state transition on controlled data and the authorization boundary around the operator route.

Rotation is an operation, not a fallback forever

A safe rotation sequence has an owner and an end condition. First, introduce the new current key while retaining the previous key for reads. Then use the authorized re-encryption path to rewrite the known inventory. Read back the inventory, verify every record under the new key, and confirm the previous key is no longer needed. Only then remove the previous value from the deployment configuration.

The reader loop itself does not enforce a clock or automatically retire the previous key. “Bounded window” describes the operator’s migration plan, not an expiry check in this function. If the old key remains configured indefinitely, the fallback becomes a second long-lived decryption authority. If it is removed before all records are re-encrypted, older credentials become unreadable. The retirement decision therefore belongs to an explicit runbook with inventory and acceptance evidence, not to the code’s catch block.

The order of an actual rehearsal should preserve a path back to readability. Before moving the current pointer, record the known account inventory and a non-secret integrity reference for each stored envelope. Deploy the new key as current while retaining the old key as previous. Re-encrypt through the operator route, then read the complete inventory back with the new key alone in a controlled configuration. If one record fails, stop and preserve both configured candidates while you reconcile that row. Do not remove the old key because a summary says “mostly complete.”

Rollback also deserves an explicit exercise. Once some rows are encrypted under the new key, returning to the old key as the only reader can strand those rows. The fallback order can be reversed for a controlled recovery, but that is a separate configuration and acceptance path that should be tested before it is needed. Keep the two key values available to the authorized operator during the rehearsal, and avoid logging either value, plaintext credentials, or full ciphertexts while diagnosing failures. The useful audit record is which account identifiers were processed, which state transition completed, and what remained unresolved.

I also separate candidate failure from total failure in the operator view. The decryption function intentionally tries another key after an attempt throws, so a single candidate error should not be treated as a final credential outage. At the same time, successful fallback use is a signal that the migration is not finished. A bounded, access-controlled count of previous-key reads can help the operator find stragglers; it should never include the key material or turn into an automatic expiry rule unsupported by the implementation.

I would test the exact failure matrix before a staging rehearsal:

• A valid current ciphertext opens even if the previous value is malformed.
• A ciphertext encrypted under the previous key opens only while that candidate is configured.
• The wrong account ID fails authentication.
• A malformed envelope version, extra field or nonce length is rejected.
• Re-encryption requires the operator route, rewrites the entire known inventory, and leaves no old-key-only records.
• After the readback, removing the previous key still leaves every record readable under current.

The test should report its denominator and keep fixture results separate from a real staging inventory. If one record cannot be read or rewritten, hold the retirement step; do not silently skip it and declare rotation complete.

The core design is modest: two read candidates, one current write key, account-bound authenticated ciphertext, and an explicit operator-owned retirement step. It makes recovery more available without hiding the migration’s remaining work.

How this was verified: I read the pinned source and test objects at the recorded commit and checked their SHA-256 values against the dossier. The code tests cover current-key reads with malformed previous configuration, wrong-account binding and a controlled two-account re-encryption fixture. They do not establish a completed staging or production rotation, automatic previous-key expiry, or a secret-management review.