Fix uptime flapping: stabilize intermittent monitoring failures

What's happening

Uptime flapping is when a monitor reports the site as DOWN, then UP, then DOWN within minutes. Every flip fires an alert. The on-call rotation gets blasted with notifications, real outages get lost in the noise, and confidence in the monitoring system collapses. Pingdom, Better Uptime, UptimeRobot, and self-hosted Healthchecks all surface this as a flapping state.

The root cause is usually one of three things: the monitor's check is too aggressive (5-second timeout against a 6-second p99 endpoint), the infrastructure is intermittently failing (a flaky load balancer, a cold-starting serverless function), or the check region has connectivity issues to the origin region. The fix depends on which.

Diagnosis starts with the monitor's check log: every probe time, latency, and result. If failures cluster around specific times of day or specific check regions, the cause is environmental. If they're random and uniform, the endpoint itself is the bottleneck. CheckFast's /uptime tool exposes the per-region log alongside response-time graphs.

Why it matters

Alert fatigue is the main impact. On-call engineers stop trusting alerts when 80% of pages are flaps. Real outages — when the site is genuinely down for users — get acknowledged late or ignored entirely. MTTR climbs.

Status-page noise damages user trust. If your public status page reports 5 incidents a week, customers assume the product is unreliable, regardless of whether actual user impact occurred. Enterprise customers especially scrutinize the status page during procurement.

Common causes

• Check timeout (e.g., 5s) too aggressive for endpoints with high p99 latency.
• Cold starts on serverless functions causing sporadic 5-second response times.
• Origin server in a single region with intermittent network blips.
• DNS issues causing periodic resolution failures from check probes.
• Database connection pool exhaustion under load returning intermittent 503s.
• CDN edge node caching a 502 from a brief origin failure, serving it for the cache lifetime.
• Monitor checking against a single endpoint that's known to be slow.

Detect this on your site

Run a quick scan with the Uptime Monitor. The tool surfaces this exact issue with the records and context needed to apply the fix below.

How to fix it

1. 1

   Pull the check log and graph latency

   Open the monitor's per-check log for the past 24-48 hours. Plot response time per probe. Failures clustered above a clear threshold (e.g., 4-5 seconds) point to timeout misconfiguration; randomly distributed failures point to infrastructure.

2. 2

   Increase the check timeout to 2x p99

   If your endpoint's p99 is 6 seconds, a 5-second check timeout will flap. Set the monitor timeout to 12 seconds. Fix the endpoint's p99 separately, but don't let the monitor be the bottleneck.

3. 3

   Check from multiple regions and require quorum

   A single check region that fails doesn't mean the site is down — it might just be a regional ISP issue. If your monitoring provider supports multi-region quorum, configure 3-5 regions and alert only when 2+ report DOWN. Better Uptime and Pingdom support this; CheckFast is currently single-region.

4. 4

   Identify cold-start patterns

   If failures cluster at low-traffic times (3am UTC weekdays), serverless cold starts are the likely cause. Use Vercel Fluid Compute, AWS Lambda Provisioned Concurrency, or Cloudflare Workers (which have minimal cold-start latency) to keep instances warm.

5. 5

   Use a dedicated health check endpoint

   Don't monitor the homepage — it has marketing scripts, A/B tests, and other variability. Create /api/health that returns 200 with a simple JSON payload after verifying database connectivity. Monitor that. Fast, deterministic, and meaningful.

6. 6

   Add retry logic in the monitor

   Most monitors support "retry on failure" — re-check after 30 seconds before alerting. A genuine outage persists; a flap recovers. Two consecutive failures (with a retry between) is a more reliable alert signal than a single failure.

7. 7

   Investigate root cause for repeat flappers

   If the same endpoint flaps repeatedly, the underlying infrastructure has an issue. Profile the slow path — check database connection pool size, upstream API latency, GC pauses. The monitor is a symptom; fix the cause.

8. 8

   Mute alerts during known maintenance

   Deploy windows, infrastructure migrations, and planned downtime should suppress alerts to avoid pager noise. Most monitors support maintenance windows. Couple this with a deploy-pipeline integration that auto-mutes during deploy.

Example

// /api/health route — fast, deterministic, meaningful
import { NextResponse } from "next/server";
import { db } from "@checkfast/db";
import { sql } from "drizzle-orm";

export async function GET() {
  const start = Date.now();
  try {
    // Verify DB connectivity with a cheap query
    await db.execute(sql`SELECT 1`);
    return NextResponse.json(
      { ok: true, latency_ms: Date.now() - start },
      { headers: { "Cache-Control": "no-store" } },
    );
  } catch (err) {
    return NextResponse.json(
      { ok: false, error: String(err) },
      { status: 503, headers: { "Cache-Control": "no-store" } },
    );
  }
}

// Monitor configuration (CheckFast / Better Uptime / Pingdom):
// - URL: https://yoursite.com/api/health
// - Timeout: 12s
// - Regions: us-east, eu-west, ap-southeast (require 2/3 fail)
// - Retry: yes, 30s after first failure
// - Expected status: 200
// - Expected body: contains "ok":true

Dedicated health endpoint plus a multi-region quorum configuration for providers that support it; CheckFast is currently single-region.

Related fixes