One Fat Function Was Making Every Page Slow

Stem Shovel is a SvelteKit app deployed to Vercel. In September its song and project pages sometimes took five seconds to open. The database was not the problem: Turso answers in a few milliseconds. The time went to a cold start of the serverless function that renders the page. We measured 5 to 6 seconds for a cold request against an idle deployment and 0.15 to 0.35 seconds for a warm one. With one or two people using the app at a time, the function was idle most of the day, so most sessions began with the slow kind.

This is the story of why the function was slow to start, what we moved, and the three follow-up problems that moving it created. None of them were hard once we could see them. Seeing them took a while.

Why the Function Was Fat

When a band uploads a stem, the server renders an MP3 of it for playback with ffmpeg. When a song's stems are in, the server bounces a mix, and transcribes the tonal stems to notes with Basic Pitch, a TensorFlow.js model, for the chart draft. All of that is background work, and we had done the convenient thing with it: it ran inside the page function after the response, under Vercel's waitUntil. No queue, no worker, no extra deploy target.

The cost was in the bundle. Importing ffmpeg means shipping the static binary, 77 MB of it, and importing the transcription stack means shipping TensorFlow.js. SvelteKit's Vercel adapter traces the imports of every route and packs what they reach, so every page function carried both. After a build, the page function's node_modules came to 131 MB. A cold start downloads that, unpacks it, and imports it before the first line of a load function runs.

There was a second, quieter cost in the same bundle. Our server-side Sentry setup imported the SvelteKit server entry, which re-exports the Vite plugin, which drags Vite, esbuild and Babel into every cold start. Its ESM loader hook also slowed every import after it. Neither did anything useful at runtime.

And there was a bug hiding under the weight. The transcription runs in a child process that resolves its packages at runtime, which the adapter's tracer does not follow, so they were never packed at all. Every song page's background job on Vercel failed with "Cannot find package '@tensorflow/tfjs'", quietly, after the response had already gone out. Nobody had noticed because the page was fine.

Move the Work Behind a Door

The fix was to give the background work a function of its own. SvelteKit's Vercel adapter lets a route export a config, and split: true puts that route in its own function. The jobs route is the only route that imports ffmpeg, the transcription stack, or (since October) Verovio and pdfkit for engraving notation PDFs:

export const config: Config = { split: true, maxDuration: 800 };

export const POST: RequestHandler = async ({ request }) => {
	const bearer = request.headers.get("authorization")?.replace(/^Bearer\s+/i, "") ?? null;
	if (!isJobsToken(bearer)) error(401, "Not this app's jobs token");
	const parsed = v.safeParse(JobSchema, await request.json().catch(() => null));
	if (!parsed.success) error(400, "kind and ids are required");
	const { kind, ids } = parsed.output;
	background(() => RUNNERS[kind](ids), { kind, count: ids.length });
	return Response.json({ accepted: kind, count: ids.length }, { status: 202 });
};

The page function no longer does the work. It posts a small JSON body, a job kind and a list of row ids, to POST /api/jobs after its own response has gone out. The jobs route validates the body, answers 202 at once, and runs the job under its own waitUntil. In development one dev server serves both sides. The jobs function started with the 300 second budget and now has 800, the Pro plan's ceiling, because a whole show's recording can take that long to decode, measure and encode (that is its own article: Running ffmpeg on Serverless).

The page function's node_modules went from 131 MB to 33 MB. The jobs function carries 117 MB, TensorFlow.js and Basic Pitch included, and now that they are packed, notes get transcribed on Vercel. One detail of that packing is worth passing on: the tracer only packs a package it can see in a literal import(), so the jobs route calls a function that names the transcription packages that way, behind a condition that is never true. It is a line of code whose only job is to be read by the build.

Sentry on the server became @sentry/node, errors only, without the loader hook and without the SvelteKit entry. We measure the cold start locally by importing the built function with plain Node on our VM; that figure went from 1.8 seconds to 1.2 seconds before the first request, and Lambda multiplies whatever that number is.

A Token Without a New Secret

The jobs route is for this app only, and it had to prove that without adding a secret to configure per stage. The token is an HMAC: a SHA-256 MAC of a fixed label, keyed by a secret the app already holds for its sign-in system. The page function derives it to post the job, the jobs function derives it to check the bearer, and nothing new is stored anywhere. The check compares in constant time:

export function isJobsToken(bearer: string | null): boolean {
	const expected = Buffer.from(jobsToken());
	const given = Buffer.from(bearer ?? "");
	return given.length === expected.length && timingSafeEqual(given, expected);
}

The post itself has a ten second timeout and one retry, and the retry only fires on transport failures: a network error, a 5xx, the timeout. A 401 means the token is wrong and a 400 means the body is, and neither gets better on a second try.

The Cron That Warmed the Wrong Function

The third change in September was a cron. Vercel runs GET /api/warm every five minutes on production; the route does one select 1 and answers ok, which keeps the page function and its database path warm. For the song and project pages it worked. Then, in late October, the Idea Recorder was slow to open after every idle spell. A signed-out request to /idea-recorder, which is a redirect and nothing else, took 3.4 seconds cold and 0.1 seconds warm.

The reason is how the adapter groups routes. @sveltejs/adapter-vercel bundles routes into functions by their exported config: routes whose configs are JSON-equal share one function. The Idea Recorder, the project and song pages, and the ready and mix endpoints all declare maxDuration: 300, so they were a second function beside the default one, and the cron only ever reached the default. The build output tells you this if you ask: readlink -f .vercel/output/functions/idea-recorder.func resolves to the group it landed in.

So there are now two warm routes. /api/warm/long declares the same config as the recorder, lands in the same function, and the cron calls it too. The test that keeps this from drifting is one line:

it("declares the Idea Recorder's config, so the adapter bundles it into that function", async () => {
	const recorder = await import("../../../idea-recorder/+page.server");
	expect(config).toEqual(recorder.config);
});

We had met the same adapter behavior from the other side on a different project. There, one route's config for a longer maxDuration split a build and sent every unmatched path to a nearly empty function, and the whole site returned 404 (Finding No. 50). The split is a real mechanism with real consequences in both directions. The rule we keep now is to know how many functions a build produces and to warm each one on purpose.

Scripts Cannot Knock

The last follow-up came from a maintenance script. Some recordings had been rendered with sound on the left channel only, and we wrote a script that clears their rendition status and posts the ids to /api/jobs with the jobs token, exactly as the app does from inside a page function. Run from a laptop against production, it got a 429.

The 429 came from Vercel's managed bot protection, in challenge mode, which challenges requests from non-browser clients. Bun's fetch cannot run the challenge, so it gets the 429 before the route ever sees the token. The app's own dispatch from the page function reaches the jobs function fine. But a fetch from inside the jobs function for the transcription model's files was also challenged, back in September, which is why that model now travels inside the module as JSON and is written to the job's temp directory. So the rule is not "requests from Vercel get through". We do not know the exact rule, and we stopped trying to learn it.

What we did instead is make the rows the source of truth. A script can change rows. It cannot drive the jobs function. So every kind of background work has a page-load backstop: the page that shows the thing looks at the rows and schedules whatever they say is owed. For stems it is a small filter:

return stems
	.filter((s) => s.status === "ready" && s.url)
	.filter((s) => {
		const age = now - (s.playbackStartedAt?.getTime() ?? 0);
		if (s.playbackStatus === null) return true;
		if (s.playbackStatus === "failed") return age > PLAYBACK_RETRY_MS;
		if (s.playbackStatus === "pending") return age > PLAYBACK_STALE_MS;
		return false;
	})
	.map((s) => s.id);

Never tried, failed a while ago, or stuck in pending: schedule it. The song page does this for stems, demos and mixes, the Idea Recorder for takes, and the Sessions page for session recordings. The script now records which kinds it could not post, carries on, and names the page to open at the end. The same backstop is what picks up a job the original post lost, and it is how a failed row that a deploy has since fixed gets retried without anyone touching it.

What We Would Tell Another Team

Look at the size of each function's node_modules after a build. If one route imports a binary or a machine learning runtime, every route pays for it on every cold start unless that route is split out. The split costs one HTTP post and one token derivation, and the token can come from a secret you already have.

Count your functions. A per-route config makes a new one, warming is per function, and a cron that keeps the default warm says nothing about the others. A test that pins two routes' configs together is cheap and has already paid for itself.

And do not assume your own token gets a script past your own front door. Design background work so that any page load can look at the rows and schedule what is owed. Then the scripts only need to change rows, the firewall can do what it likes, and the work still gets done.