Vercel Deployment

use-e.md
- wiki/engineering/vercel-deployment/client-brain-4b04b31-parallelize-sentiment-cron-in-batches-of-5-to-avoi.md
- wiki/engineering/vercel-deployment/client-brain-84ae9b2-fix-vercel-deploy-remove-functions-pattern-use-rou.md
- wiki/engineering/vercel-deployment/asymxray-0b078e6-restore-webpack-workaround-for-vercel-builds.md
- wiki/engineering/vercel-deployment/contentcommand-41fc832-fix-vercel-build-force-dynamic-rendering-on-auth-p.md
- wiki/engineering/vercel-deployment/client-brain-259bafb-fix-verceljson-remove-invalid-custom-runtime.md
contradicting_sources: []
contradicting_count: 0
first_seen: "unknown"
last_updated: "2025-07-14"
hypothesis: false
rate_of_change: high
web_monitoring_frequency: weekly
fragment_count: 35
evolution_timeline: []
evolution_start: null
superseded_by: null
superseded_date: null
deprecation_notice: null
tags: []

Summary

Vercel's serverless and edge architecture breaks assumptions that hold in traditional server deployments — in-memory state doesn't survive across instances, Edge Runtime excludes Node.js core modules entirely, and database connection pools must be sized to 1 per instance or you'll exhaust your database. The Security Checkpoint is aggressive enough to block GitHub Actions IPs with 429s, making external health checks from CI unreliable; use UptimeRobot or Vercel's native cron monitoring instead. vercel.json configuration for App Router projects and API-only projects has enough sharp edges that you should expect 2-3 iterative fixes before a new project deploys cleanly.

TL;DR

What we've learned
- In-memory rate limiting, caches, and counters silently break on Vercel — each cold start resets them and concurrent instances never share state. Use Upstash Redis for anything that needs to persist.
- Database connection pools must be set to max: 1 per serverless instance. At higher values, concurrent invocations exhaust the database's connection limit.
- Vercel's Security Checkpoint blocks GitHub Actions IPs with 429s — don't use GitHub Actions for health checks or uptime monitoring against Vercel-hosted endpoints.
- Edge Runtime has no access to perf_hooks, process.memoryUsage, or process.cpuUsage. Replace with the global performance API.
- Cron jobs processing more than ~15 items sequentially will approach Vercel's 300s function timeout. Batch and parallelize early.

External insights

No external sources ingested yet for this topic.

Common Failure Modes

In-memory rate limiting silently stops working at scale

In-memory rate limiting counters reset on every cold start and are never shared between concurrent serverless instances. The failure is invisible — no errors, no logs — but rate limits simply don't enforce correctly under load. In eydn-app, this was discovered after deploying rate-limited API routes and observing that limits weren't being respected across requests hitting different instances.

Fix: replace in-memory counters with Upstash Redis. For local development where Redis credentials aren't available, fall back to in-memory with an explicit warning:

// Graceful fallback pattern from eydn-app
const redis = process.env.UPSTASH_REDIS_REST_URL
  ? new Redis({
      url: process.env.UPSTASH_REDIS_REST_URL,
      token: process.env.UPSTASH_REDIS_REST_TOKEN!,
    })
  : null; // falls back to in-memory for local dev

^[1]

Database connection pool exhaustion under concurrent load

Each serverless function instance creates its own connection pool. With max: 10 (a common default), 10 concurrent invocations open 100 connections simultaneously — enough to exhaust most managed database plans. In stride-v2, this manifested as connection timeout errors under normal production load.

Fix: set max: 1 in your pool configuration for serverless:

// stride-v2 fix
const pool = new Pool({
  connectionString: process.env.DATABASE_URL,
  max: 1, // one connection per serverless instance
});

^[2]

Vercel Security Checkpoint returns 429 for GitHub Actions IPs

Vercel's Security Checkpoint treats GitHub Actions runner IPs as suspicious and blocks them with 429 responses. This affects all external HTTP requests to your deployment — including public, unauthenticated endpoints. In asymxray, health check jobs in GitHub Actions were returning 429s consistently, making the monitoring pipeline unreliable.

The fix is to stop using GitHub Actions for health checks entirely. UptimeRobot (free tier: 50 monitors, 5-minute intervals) bypasses the checkpoint. Vercel's native cron jobs also bypass it since they originate from Vercel's own infrastructure.

Note: GitHub Actions CI/CD pipelines (build, lint, test, deploy) are unaffected — the block only applies to outbound HTTP requests to your Vercel deployment, not to the Vercel deployment pipeline itself.

^[3]

Edge Runtime rejects Node.js core modules at build time

Edge Runtime does not include perf_hooks, process.memoryUsage(), or process.cpuUsage(). If your middleware or edge functions import these — even transitively through a monitoring or instrumentation library — the build fails or the function throws at runtime. In asymxray, the monitoring system used perf_hooks for timing and had to be rewritten to use the global performance API.

// Before (breaks on Edge Runtime)
import { performance } from 'perf_hooks';

// After (works everywhere)
const start = performance.now(); // global, no import needed

^[4]

Cron jobs timing out on large sequential workloads

Sequential processing of per-client or per-record tasks hits Vercel's 300-second function timeout faster than expected. In client-brain, processing 23 clients sequentially took ~230 seconds — within 70 seconds of the limit, with no headroom for slow API responses or retries.

Fix: batch and parallelize. Processing in groups of 5 concurrent tasks reduced wall-clock time from ~230s to ~50s:

// client-brain batch parallelization pattern
const BATCH_SIZE = 5;
for (let i = 0; i < clients.length; i += BATCH_SIZE) {
  const batch = clients.slice(i, i + BATCH_SIZE);
  await Promise.all(batch.map(client => processSentiment(client)));
}

^[5]

Auth pages fail to build due to missing env vars at static prerender time

Next.js App Router attempts to statically prerender pages at build time. If an auth page (login, signup, etc.) calls createClient() or similar Supabase initialization that reads SUPABASE_URL or SUPABASE_ANON_KEY, and those vars aren't available in the Vercel build environment, the build fails or produces broken static output.

Fix: add export const dynamic = 'force-dynamic' to auth page files:

// contentcommand auth pages
export const dynamic = 'force-dynamic';

^[6]

vercel.json functions pattern breaks App Router deployments

The functions key in vercel.json is a Pages Router / legacy API routes pattern. Using it in an App Router project to configure maxDuration causes build failures or silently ignores the config. In client-brain, this required switching to route segment config exports.

// Wrong: vercel.json functions pattern (App Router)
// { "functions": { "app/api/**": { "maxDuration": 60 } } }

// Correct: route segment config in the route file
export const maxDuration = 60;

^[7]

Explicit @vercel/node@3 runtime string breaks API-only project builds

Vercel auto-detects Node.js for .ts files in the api/ directory. Specifying "runtime": "@vercel/node@3" explicitly in vercel.json causes build failures — the runtime string is either invalid in the current Vercel build system or conflicts with auto-detection. Observed in client-brain during initial deployment setup.

Fix: remove the runtime key from vercel.json function configuration entirely.

^[8]

What Works

Vercel native cron jobs for scheduled background tasks

Vercel cron jobs configured in vercel.json are reliable and bypass the Security Checkpoint. Consistent across 4 projects (asymxray, client-brain, eydn-app, stride-v2) with frequencies ranging from every 2 minutes to weekly. The vercel.json syntax is stable:

{
  "crons": [
    {
      "path": "/api/cron/process-automation",
      "schedule": "*/2 * * * *"
    },
    {
      "path": "/api/cron/daily-ingestion",
      "schedule": "0 6 * * *"
    }
  ]
}

^[9]

Making sensitive env vars optional at build time

Service role keys and other credentials that aren't needed during static generation should be typed as string | undefined and guarded, rather than asserted as string. This prevents build failures when Vercel's build environment doesn't have production secrets. Observed in orbitabm for SUPABASE_SERVICE_ROLE_KEY and in client-brain for API credentials.

// Pattern: optional at build time, required at runtime
const serviceRoleKey = process.env.SUPABASE_SERVICE_ROLE_KEY;
if (!serviceRoleKey) {
  if (process.env.NODE_ENV === 'production') {
    throw new Error('SUPABASE_SERVICE_ROLE_KEY required in production');
  }
  return; // skip during build
}

^[10]

UptimeRobot for external health monitoring

UptimeRobot's free tier (50 monitors, 5-minute check intervals) successfully bypasses Vercel's Security Checkpoint where GitHub Actions cannot. In asymxray, switching from GitHub Actions health checks to UptimeRobot eliminated the 429 failures entirely. The free tier is sufficient for most projects.

^[11]

GitHub Actions for CI/CD (not health checks)

GitHub Actions pipelines work correctly for the build/lint/test/deploy workflow — the Security Checkpoint issue only affects outbound HTTP requests to Vercel deployments. Eydn-app and asymxray both run full CI pipelines (lint, type check, tests, build verification, security audit) without issues. Keep CI/CD in GitHub Actions; move health monitoring out.

^[12]

Gotchas and Edge Cases

Turbopack middleware bug persists on Vercel even with Next.js 16.1.1

In asymxray, a Turbopack middleware bug that was fixed locally (Next.js 16.1.1) still manifested on Vercel's build environment. The --webpack flag workaround had to be restored in vercel.json even after the local fix. Vercel's build environment lags behind local Next.js versions in ways that aren't always documented.

^[13]

Stale middleware.js.nft.json cache causes intermittent build failures

Vercel's build cache can retain a stale middleware.js.nft.json file that causes builds to fail in ways that don't reproduce locally. In asymxray, the fix was adding a cleanBuildCommand to vercel.json that removes the file before each build. This is a medium-confidence finding — the root cause may be specific to certain middleware configurations.

^[14]

In-memory caches survive redeployments within the same instance lifecycle

Counterintuitively, in-memory caches don't always clear on redeployment — if Vercel reuses a warm instance, the old cache persists. In labelcheck, a stale ingredient cache required a forced redeployment (not just a new deploy) to clear. This is the inverse of the cold-start problem: sometimes you want the cache cleared and it isn't.

^[15]

API-only projects require non-obvious vercel.json configuration

Projects that are pure API (no Next.js frontend) require explicit buildCommand, output directory, and function glob configuration in vercel.json. The defaults assume a framework project. In client-brain, this required 3 separate iterative fixes: skipping the build step, using empty strings for output config, and using a single glob pattern for function matching.

^[16]

Serverless connection pooling math is multiplicative, not additive

When sizing database connection pools, the relevant number is max_connections_per_instance × max_concurrent_instances, not just max_connections_per_instance. At max: 10 with 20 concurrent invocations, you're opening 200 connections. Most managed Postgres plans (Supabase free/pro, Railway) cap out well below that. Default to max: 1 and increase only with explicit capacity planning.

^[2]

Where Docs Disagree With Practice

vercel.json functions key is documented but breaks App Router

Vercel's documentation includes the functions configuration key for setting maxDuration and memory limits. In practice, this pattern does not work with Next.js App Router — it's a Pages Router artifact. The correct App Router approach is export const maxDuration = N in the route file itself. The docs don't clearly delineate which pattern applies to which router.

^[7]

Explicit runtime strings in vercel.json are documented but break builds

Vercel's documentation shows runtime configuration like "runtime": "@vercel/node@3" as valid vercel.json syntax. In client-brain, this caused build failures. Vercel's auto-detection for .ts files in api/ is more reliable than explicit runtime specification, and the explicit form appears to conflict with current build infrastructure despite being documented.

^[8]

Edge Runtime compatibility is broader in docs than in practice

Vercel's Edge Runtime documentation lists supported APIs but doesn't clearly flag that common Node.js instrumentation patterns (perf_hooks, process.memoryUsage) are entirely absent — not polyfilled, not partially supported, just missing. Code that works in Node.js serverless functions silently fails or throws at the edge. The global performance API is the correct replacement for timing, but the docs don't surface this migration path prominently.

^[4]

Tool and Version Notes

• Next.js 16.1.1 + Turbopack: Turbopack middleware bug present in Vercel's build environment even when fixed locally. Use --webpack flag in vercel.json buildCommand as workaround until Vercel's build environment catches up. Observed in asymxray.

• Next.js App Router (any version): Route segment config (export const maxDuration, export const dynamic) is the correct configuration surface. The vercel.json functions key is Pages Router only.

• Upstash Redis: Required for any shared state in serverless (rate limiting, counters, distributed locks). The @upstash/redis client works in both Node.js and Edge Runtime. Observed in eydn-app.

• Vercel cron jobs: Reliable across all tested projects. Free tier supports cron schedules; Pro tier required for sub-minute intervals. Cron invocations bypass the Security Checkpoint.

• UptimeRobot free tier: 50 monitors, 5-minute minimum interval. Bypasses Vercel Security Checkpoint. Sufficient for health monitoring on most projects.

• Sentry in Vercel deployments: eydn-app uses 20% trace sample rate for production performance monitoring and 10%/100% session replay (normal/error). These rates are a reasonable starting point for production Next.js apps.

Related Topics

Sources

Synthesized from 35 fragments: git commits across 7 projects (asymxray, client-brain, eydn-app, stride-v2, orbitabm, labelcheck, contentcommand). No external sources ingested yet. Date range: unknown.