Digging Deeper

Cache

Introduction#

Some answers take long to compute and change rarely: the numbers of a dashboard, a feed from another service. The cache keeps such a value under a key for a while, so the next request reads it instead of computing it again. Inject the Cache of @marmeon/cache, which every new app has:

import type { Authenticated } from '@marmeon/auth';
import { Cache } from '@marmeon/cache';
import { Controller, type HttpContext } from '@marmeon/http';
import { NoteRepository } from '../NoteRepository.ts';

export class ShowNoteStatsController extends Controller {
  readonly #cache: Cache;
  readonly #notes: NoteRepository;

  constructor(cache: Cache, notes: NoteRepository) {
    super();
    this.#cache = cache;
    this.#notes = notes;
  }

  async handle(ctx: HttpContext<Authenticated>) {
    const stats = await this.#cache.remember(`notes:stats:${ctx.user.id}`, 600, () => this.#notes.statsFor(ctx.user.id));
    return this.view('notes/Stats', { stats });
  }
}

remember() returns the cached value, or computes it, stores it for 600 seconds and returns it. The same cache keeps the framework's rate limits and locks, and the rate limiting page covers the limits.

Configuration#

A new app's .env.test sets CACHE_DRIVER=memory.

Drivers#

The database driver needs @marmeon/database and the redis driver @marmeon/redis, which brings the Redis client. Without the package the app fails at the start and names it: CACHE_DRIVER=redis needs @marmeon/redis, which is not installed in this app: pnpm add @marmeon/redis. The Redis page covers its connection.

The database driver needs its tables once:

pnpm marmeon make:cache-table
pnpm marmeon migrate

make:cache-table writes the migration into the system module, or the module that --module names. The migration creates the tables on the cache's connection, CACHE_CONNECTION.

The database driver and transactions#

The database driver never writes inside the app's transactions. A failed sign-in counted inside a transaction that rolls back would be uncounted, and a rate limit would let the next guess through. A lock taken inside one would stay invisible to other processes until the commit. So the cache writes on a connection of its own:

A SQLite file has one writer. A second connection to the app's file would wait for the open transaction, forever if the transaction waits for the cache. So on SQLite give the cache its own file: a second connection in config/database.ts, such as system for storage/system.sqlite, and CACHE_CONNECTION=system. Outside a transaction, the normal case, the app's own file works too. The database page shows how to define a connection.

Using the cache#

Values are stored as JSON, so they must survive JSON.stringify(): a Date comes back as a string. Expiry follows the app's clock, so app.travel() in a test moves it, except on Redis, whose keys expire by Redis' own clock.

add() and increment() are atomic in every store, and across processes in the file, database and redis stores. Of two add() calls at once, exactly one stores its value. Two processes that count at once never lose a count. A new key of increment() gets the time to live, and an existing one keeps its expiry, so a window does not start over with every hit.

remember() is no lock: when two requests miss at once, both compute the value. When the work must run only once, take a lock.

Locks#

A lock lets one process do a piece of work while others wait or give up. It is held for a number of seconds at most, so a holder that crashes cannot block it forever:

import type { Authenticated } from '@marmeon/auth';
import { Cache } from '@marmeon/cache';
import { abort, Controller, type HttpContext } from '@marmeon/http';
import { NoteExports } from '../NoteExports.ts';

export class ExportNotesController extends Controller {
  readonly #cache: Cache;
  readonly #exports: NoteExports;

  constructor(cache: Cache, exports: NoteExports) {
    super();
    this.#cache = cache;
    this.#exports = exports;
  }

  async handle(ctx: HttpContext<Authenticated>) {
    const lock = this.#cache.lock(`notes:export:${ctx.user.id}`, 60);
    if (!(await lock.get())) abort(423);
    try {
      await this.#exports.build(ctx.user.id);
    } finally {
      await lock.release();
    }
    return this.back();
  }
}

• get() tries once and says whether it took the lock. get(callback) runs the callback while holding it, releases it and returns the callback's result, or false when the lock was taken.
• block(seconds, callback?) waits up to seconds for the lock, trying every 250 milliseconds, then throws LockTimeoutError. With a callback it runs it, releases the lock and returns the result.
• release() frees the lock only while this lock object holds it. A lock that expired and was taken by another stays theirs.
• forceRelease() frees it, whoever holds it. owned() says whether this object holds it now.

Each lock object has a random owner token. cache.lock(name, seconds, owner) makes an object with a known owner, so a job can release a lock that a request took, given the lock's owner. The number of seconds must be positive.

A lock taken for work inside a transaction must go when the transaction rolls back. afterRollback(() => lock.release()) of @marmeon/database does that, and the transactions page explains it.

What else the cache holds#

The framework keeps its own counters and locks in the same cache:

So the warning above holds for all of them: with more than one server, the cache must be one they share.

Expired entries and clearing#

The memory, file and database stores keep an expired entry until it is read or overwritten. cache:prune deletes expired entries and locks, and never touches one that is still valid. Redis expires keys by itself, and the command says so. Schedule it in a module's schedule, which counts once the module lists it as schedule in its definition, as the task scheduling page shows:

import { defineSchedule } from '@marmeon/scheduler';

export const schedule = defineSchedule((s) => {
  s.command('cache:prune').hourlyAt(25).withoutOverlapping().onOneServer();
});

cache:clear empties the cache: every entry, never a lock. Rate-limit windows, the queue's job.once markers and the scheduler's record of its last run go too. Clear it after migrate:fresh, when ids start over and an old marker would match a new record. On Redis it deletes only the keys of the cache, never the whole database.

Testing#

Tests usually run on CACHE_DRIVER=memory, which .env.test sets. The cache, its locks and the rate limits read the time from the app's clock: app.travel({ minutes: 1 }) lets an entry expire or a rate-limit window pass without waiting. The time page covers the clock in tests.