# Downtime Duration Accuracy Implementation Plan

> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.

**Goal:** Make every reported outage duration reflect the *real* affected period — fix the recovery email under-reporting (~9m 30s constant) and the audit page over-reporting (e.g. 582 min for a 1-minute live outage).

**Architecture:** Three small, independent fixes plus a structural smoke. (1) The recovery-email branch in `monitor-engine.ts` reconstructs the outage start with three indexed `findFirst` queries instead of slicing a fixed ~5-row hysteresis window. (2) The downtime report clamps an *ongoing* outage's upper bound to `now` instead of the filter's end-of-day. (3) The report excludes soft-deleted monitors so a deleted-while-down monitor stops showing a perpetual outage. Pure duration math is extracted into `downtime-calculator.ts` for unit testing.

**Tech Stack:** Next.js 16 API route, Prisma (MySQL), TypeScript, Jest (`npm test`), tsx smoke scripts.

---

## Background — root causes (already diagnosed, do not re-investigate)

- **Bug #1 — recovery email duration too short.** [monitor-engine.ts:291-298](../../../src/lib/monitor-engine.ts#L291-L298) finds `firstDownOfStreak` inside `historyBefore`, which is hard-capped to `requiredDowns + requiredUps` (~5) rows ([monitor-engine.ts:238-246](../../../src/lib/monitor-engine.ts#L238-L246)). For any outage longer than that window the start saturates, so duration is a near-constant ~8–10 min. Proven by Mangochi: DOWN email "Incident Started 13:10:34" vs RECOVERED email "13:18:31 / 9m 30s".
- **Bug #2 — audit ongoing outage too long.** [downtime/route.ts:171-187](../../../src/app/api/reports/downtime/route.ts#L171-L187) and [downtime/route.ts:111-127](../../../src/app/api/reports/downtime/route.ts#L111-L127) compute an ongoing outage's duration as `end - downStart`, where `end` is the filter's end-of-day (`<endDate>T23:59:59`, set in [StatusHistoryTable.tsx:59](../../../src/components/audit/StatusHistoryTable.tsx#L59)). A down detected at 14:17 reads `23:59:59 − 14:17:32 = 582 min`. Resolved outages ([route.ts:147-148](../../../src/app/api/reports/downtime/route.ts#L147-L148)) use the real `up` timestamp and are correct — **do not touch them**.
- **Bug #3 — frozen trio.** Monitors deleted/paused while down show a never-closing outage. Excluding `deletedAt` rows removes the deleted-monitor case. (The `active = false` "paused while down" case is an open product decision — see Task 4.)

**Indexes already present** (no migration needed): `@@index([monitorId, createdAt])` and `@@index([monitorId, status])` on `Heartbeat` ([schema.prisma:575-583](../../../prisma/schema.prisma#L575-L583)). The three `findFirst` queries in Task 2 are `LIMIT 1` lookups served by these indexes.

---

## Pre-flight (run once before Task 1)

- [ ] **Sync the branch** (per project CLAUDE.md — `claude/work` must carry latest main):

```bash
git checkout claude/work
git fetch origin
git merge origin/main
```

- [ ] **Confirm the test runner works:**

Run: `npm test -- src/lib/utils/downtime-calculator.test.ts`
Expected: PASS (existing `formatDuration` / `estimateDowntimeStart` tests green).

---

## Task 1: Clamp ongoing-outage duration to `now` (Bug #2)

**Files:**
- Modify: `src/lib/utils/downtime-calculator.ts` (add `ongoingOutageMinutes`)
- Test: `src/lib/utils/downtime-calculator.test.ts`
- Modify: `src/app/api/reports/downtime/route.ts:111-127` and `:171-187` (use the helper)

- [ ] **Step 1: Write the failing test**

Add to `src/lib/utils/downtime-calculator.test.ts` (inside the top-level `describe('Downtime Calculator', ...)`):

```typescript
describe('ongoingOutageMinutes', () => {
    const now = new Date('2026-06-15T14:18:00Z');
    const endOfDay = new Date('2026-06-15T23:59:59Z');

    it('measures a live outage to now, not the filter end-of-day', () => {
        const downStart = new Date('2026-06-15T14:17:00Z'); // 1 min before now
        expect(ongoingOutageMinutes(downStart, endOfDay, now)).toBe(1);
    });

    it('never returns less than 1 minute for a just-started outage', () => {
        const downStart = new Date('2026-06-15T14:17:55Z'); // 5s before now
        expect(ongoingOutageMinutes(downStart, endOfDay, now)).toBe(1);
    });

    it('caps at the range end when now is past the range (historical query)', () => {
        const pastNow = new Date('2026-06-20T00:00:00Z'); // now is after the range
        const downStart = new Date('2026-06-15T22:00:00Z');
        // upper bound is endOfDay (23:59:59), not pastNow → ~120 min
        expect(ongoingOutageMinutes(downStart, endOfDay, pastNow)).toBe(120);
    });
});
```

- [ ] **Step 2: Run the test to verify it fails**

Run: `npm test -- src/lib/utils/downtime-calculator.test.ts`
Expected: FAIL with "ongoingOutageMinutes is not defined".

- [ ] **Step 3: Implement the helper**

Append to `src/lib/utils/downtime-calculator.ts`:

```typescript
/**
 * Whole-minute duration for an OPEN (still-down) outage. The upper bound is
 * the earlier of the query range end and the current time, so a live outage
 * reads its real elapsed time instead of stretching to the filter's
 * end-of-day. Floored at 1 minute (consistent with the report's resolved-row
 * floor). For a CLOSED outage use the real up-timestamp, not this helper.
 */
export function ongoingOutageMinutes(downStart: Date, rangeEnd: Date, now: Date): number {
    const upper = Math.min(rangeEnd.getTime(), now.getTime());
    const minutes = Math.round((upper - downStart.getTime()) / 60000);
    return minutes > 0 ? minutes : 1;
}
```

- [ ] **Step 4: Run the test to verify it passes**

Run: `npm test -- src/lib/utils/downtime-calculator.test.ts`
Expected: PASS (all `ongoingOutageMinutes` + existing tests green).

- [ ] **Step 5: Wire the helper into the downtime report**

In `src/app/api/reports/downtime/route.ts`, add the import at the top alongside the existing `rawBoundary` import (line 6):

```typescript
import { rawBoundary } from '@/lib/services/retention';
import { ongoingOutageMinutes } from '@/lib/utils/downtime-calculator';
```

Then replace **both** ongoing-outage duration computations.

First occurrence — the new-monitor flush at [route.ts:114](../../../src/app/api/reports/downtime/route.ts#L114):

```typescript
// BEFORE
const durationMinutes = Math.round((end.getTime() - downStart.getTime()) / 60000);
// ... durationMinutes: durationMinutes > 0 ? durationMinutes : 1,

// AFTER
const durationMinutes = ongoingOutageMinutes(downStart, end, new Date());
// ... durationMinutes,   (drop the inline `> 0 ? : 1` — the helper floors it)
```

Second occurrence — the final-record flush at [route.ts:174](../../../src/app/api/reports/downtime/route.ts#L174):

```typescript
// BEFORE
const durationMinutes = Math.round((end.getTime() - downStart.getTime()) / 60000);
// ... durationMinutes: durationMinutes > 0 ? durationMinutes : 1,

// AFTER
const durationMinutes = ongoingOutageMinutes(downStart, end, new Date());
// ... durationMinutes,
```

**Do not** change the resolved-outage block at [route.ts:147-159](../../../src/app/api/reports/downtime/route.ts#L147-L159) — it correctly uses `hb.createdAt` (the real recovery time).

- [ ] **Step 6: Typecheck**

Run: `npx tsc --noEmit`
Expected: no new errors. (Per project memory, `npm run lint` shows ~30 pre-existing react-compiler errors that CI does not — ignore those; only new errors in the touched files matter.)

- [ ] **Step 7: Commit**

```bash
git add src/lib/utils/downtime-calculator.ts src/lib/utils/downtime-calculator.test.ts src/app/api/reports/downtime/route.ts
git commit -m "fix(reports): clamp ongoing outage duration to now, not filter end-of-day"
```

---

## Task 2: Reconstruct true outage start for recovery emails (Bug #1)

**Files:**
- Create: `src/lib/utils/outage-window.ts` (`reconstructDownStart`)
- Test: `src/lib/utils/__tests__/outage-window.test.ts`
- Modify: `src/lib/monitor-engine.ts:291-298` (use the reconstruction)

- [ ] **Step 1: Write the failing test**

Create `src/lib/utils/__tests__/outage-window.test.ts`:

```typescript
jest.mock('@/lib/prisma', () => ({
    prisma: { heartbeat: { findFirst: jest.fn() } },
}));

import { prisma } from '@/lib/prisma';
import { reconstructDownStart } from '../outage-window';

const findFirst = prisma.heartbeat.findFirst as jest.Mock;

describe('reconstructDownStart', () => {
    beforeEach(() => findFirst.mockReset());

    it('returns the first down after the prior up — not a windowed edge', async () => {
        const lastDown = new Date('2026-06-15T13:27:00Z');
        const priorUp = new Date('2026-06-15T13:09:00Z');
        const firstDown = new Date('2026-06-15T13:10:34Z');
        findFirst
            .mockResolvedValueOnce({ createdAt: lastDown })  // most recent down
            .mockResolvedValueOnce({ createdAt: priorUp })   // up before that down's run
            .mockResolvedValueOnce({ createdAt: firstDown }); // first down after the up

        await expect(reconstructDownStart(42)).resolves.toEqual(firstDown);
    });

    it('returns null when the monitor has never been down', async () => {
        findFirst.mockResolvedValueOnce(null); // no down rows at all
        await expect(reconstructDownStart(42)).resolves.toBeNull();
    });

    it('falls back to the last down when there is no prior up (down since first record)', async () => {
        const lastDown = new Date('2026-06-08T14:06:00Z');
        const firstDown = new Date('2026-06-08T14:05:33Z');
        findFirst
            .mockResolvedValueOnce({ createdAt: lastDown }) // most recent down
            .mockResolvedValueOnce(null)                    // no up before it
            .mockResolvedValueOnce({ createdAt: firstDown }); // earliest down overall

        await expect(reconstructDownStart(42)).resolves.toEqual(firstDown);
    });
});
```

- [ ] **Step 2: Run the test to verify it fails**

Run: `npm test -- src/lib/utils/__tests__/outage-window.test.ts`
Expected: FAIL with "Cannot find module '../outage-window'".

- [ ] **Step 3: Implement the reconstruction helper**

Create `src/lib/utils/outage-window.ts`:

```typescript
import { prisma } from '@/lib/prisma';

/**
 * True start of the CURRENT (most-recent) outage for a monitor, reconstructed
 * from raw heartbeats with no row cap.
 *
 * Three LIMIT-1 lookups (all served by Heartbeat's (monitorId, createdAt) and
 * (monitorId, status) indexes):
 *   1. most recent 'down'  → the tail of the outage (before any recovery 'up's)
 *   2. most recent 'up'/'maintenance' BEFORE that down → the outage's left edge
 *   3. first 'down' after that edge → the outage start
 *
 * This deliberately replaces the windowed `firstDownOfStreak` lookup in
 * monitor-engine, which was capped at (requiredDowns + requiredUps) rows and
 * so saturated long outages to a constant ~9m 30s. Matches the per-transition
 * reconstruction the downtime report already does, so emails and the audit
 * page agree on when an outage started.
 *
 * Returns null when the monitor has no 'down' heartbeat at all.
 */
export async function reconstructDownStart(monitorId: number): Promise<Date | null> {
    const lastDown = await prisma.heartbeat.findFirst({
        where: { monitorId, status: 'down' },
        orderBy: { createdAt: 'desc' },
        select: { createdAt: true },
    });
    if (!lastDown) return null;

    const priorUp = await prisma.heartbeat.findFirst({
        where: { monitorId, status: { in: ['up', 'maintenance'] }, createdAt: { lt: lastDown.createdAt } },
        orderBy: { createdAt: 'desc' },
        select: { createdAt: true },
    });

    const firstDown = await prisma.heartbeat.findFirst({
        where: {
            monitorId,
            status: 'down',
            ...(priorUp ? { createdAt: { gt: priorUp.createdAt } } : {}),
        },
        orderBy: { createdAt: 'asc' },
        select: { createdAt: true },
    });

    return firstDown?.createdAt ?? lastDown.createdAt;
}
```

- [ ] **Step 4: Run the test to verify it passes**

Run: `npm test -- src/lib/utils/__tests__/outage-window.test.ts`
Expected: PASS (3 tests green).

- [ ] **Step 5: Wire it into the recovery branch**

In `src/lib/monitor-engine.ts`, replace the windowed lookup at [lines 293-294](../../../src/lib/monitor-engine.ts#L293-L294):

```typescript
// BEFORE
const firstDownOfStreak = [...historyBefore].reverse().find(hb => hb.status === 'down');
const downStart = firstDownOfStreak ? firstDownOfStreak.createdAt : new Date(now.getTime() - (monitor.intervalSeconds * 1000 * requiredDowns));

// AFTER
const { reconstructDownStart } = await import('./utils/outage-window');
const reconstructed = await reconstructDownStart(monitor.id);
const downStart = reconstructed ?? new Date(now.getTime() - (monitor.intervalSeconds * 1000 * requiredDowns));
```

(`historyBefore` is still used by the state evaluator above — leave it. Only this recovery-branch lookup changes.)

- [ ] **Step 6: Run the engine's existing tests to confirm no regression**

Run: `npm test -- src/lib/__tests__/monitor-engine-claim.test.ts src/lib/__tests__/monitor-engine-softdelete.test.ts`
Expected: PASS (claim + soft-delete contracts unchanged).

- [ ] **Step 7: Typecheck**

Run: `npx tsc --noEmit`
Expected: no new errors.

- [ ] **Step 8: Commit**

```bash
git add src/lib/utils/outage-window.ts src/lib/utils/__tests__/outage-window.test.ts src/lib/monitor-engine.ts
git commit -m "fix(monitor-engine): reconstruct true outage start for recovery emails"
```

---

## Task 3: Exclude soft-deleted monitors from the outage report (Bug #3)

**Files:**
- Modify: `src/app/api/reports/downtime/route.ts:53-65` (monitor query `where`)

- [ ] **Step 1: Add the `deletedAt` filter**

In `src/app/api/reports/downtime/route.ts`, the monitor query currently filters only by `search` ([route.ts:53-65](../../../src/app/api/reports/downtime/route.ts#L53-L65)). Add the soft-delete exclusion:

```typescript
// BEFORE
const whereMonitor: Prisma.MonitorWhereInput = {};
if (search) {
    whereMonitor.OR = [ ... ];
}

// AFTER
const whereMonitor: Prisma.MonitorWhereInput = { deletedAt: null };
if (search) {
    whereMonitor.OR = [ ... ];
}
```

This stops a monitor that was deleted while down from showing a perpetual outage. Active-but-paused monitors are intentionally left in scope (see Task 4 open decision).

- [ ] **Step 2: Typecheck**

Run: `npx tsc --noEmit`
Expected: no new errors.

- [ ] **Step 3: Commit**

```bash
git add src/app/api/reports/downtime/route.ts
git commit -m "fix(reports): exclude soft-deleted monitors from downtime report"
```

---

## Task 4: Structural smoke + verification

**Files:**
- Create: `scripts/smoke/downtime-duration-accuracy.ts`
- Modify: `scripts/smoke/run-all.ts` (register it — SAME commit, per project rule #1)

- [ ] **Step 1: Write the smoke script**

Create `scripts/smoke/downtime-duration-accuracy.ts` (offline, static-source assertions — mirrors the existing structural smokes' style):

```typescript
import './_lib';
import { readFileSync } from 'fs';
import path from 'path';
import { ok, fail } from './_lib';

const root = path.resolve(__dirname, '../..');
const read = (p: string) => readFileSync(path.join(root, p), 'utf8');

// Bug #2: report clamps ongoing outages to now via the helper.
const route = read('src/app/api/reports/downtime/route.ts');
if (!route.includes('ongoingOutageMinutes')) {
    fail('downtime-duration-accuracy', 'report no longer uses ongoingOutageMinutes — ongoing outages will stretch to filter end-of-day again');
}
if (/end\.getTime\(\)\s*-\s*downStart\.getTime\(\)/.test(route)) {
    fail('downtime-duration-accuracy', 'report still computes an ongoing duration as end - downStart (Bug #2 regression)');
}

// Bug #3: soft-deleted monitors excluded.
if (!/deletedAt:\s*null/.test(route)) {
    fail('downtime-duration-accuracy', 'report monitor query dropped the deletedAt: null filter (Bug #3 regression)');
}

// Bug #1: recovery branch reconstructs the start instead of slicing the window.
const engine = read('src/lib/monitor-engine.ts');
if (!engine.includes('reconstructDownStart')) {
    fail('downtime-duration-accuracy', 'monitor-engine recovery branch no longer calls reconstructDownStart (Bug #1 regression)');
}
if (engine.includes('firstDownOfStreak')) {
    fail('downtime-duration-accuracy', 'monitor-engine still uses the windowed firstDownOfStreak lookup (Bug #1 regression)');
}

ok('downtime-duration-accuracy', 'ongoing-clamp + soft-delete filter + outage-start reconstruction all wired');
```

- [ ] **Step 2: Register it in `run-all.ts` (same commit)**

Add to the `SCRIPTS` array in `scripts/smoke/run-all.ts`, at the end (after `'auditor-backup-code-race.ts',`):

```typescript
    // 2026-06-15: outage-duration accuracy. Guards three fixes — recovery
    // emails reconstruct the true outage start (not a ~5-row window that
    // saturated to 9m 30s); the downtime report clamps ongoing outages to
    // now (not the filter end-of-day, which inflated a 1-min outage to
    // 582 min); and soft-deleted monitors are excluded from the report.
    'downtime-duration-accuracy.ts',
```

- [ ] **Step 3: Run the new smoke alone**

Run: `npx tsx scripts/smoke/downtime-duration-accuracy.ts`
Expected: prints the `ok(...)` line, exit 0.

- [ ] **Step 4: Run the full regression gate**

Run: `npx tsx scripts/smoke/run-all.ts`
Expected: all scripts pass, final "Phase 0 exit gate" line.

- [ ] **Step 5: Commit**

```bash
git add scripts/smoke/downtime-duration-accuracy.ts scripts/smoke/run-all.ts
git commit -m "test(reports): smoke guard for outage-duration accuracy fixes"
```

- [ ] **Step 6: Manual verification (evidence before claiming done)**

1. On a monitor that is currently DOWN, open the Audit Logs → Status & Events page. Its row's Duration should read a small, growing value (~1 min just after it went down), **not** hundreds of minutes.
2. Trigger or wait for a recovery on a monitor that was down for >5 check intervals. The recovery email's "Incident Started" should match the DOWN alert's incident time (within one interval), and Duration should equal the real outage length — **not** ~9m 30s.
3. Cross-check: the same outage's Duration on the audit page and in the recovery email should now agree.

---

## Open decision for Nelson (resolve before/at Task 3)

**Should an `active = false` (paused) monitor that was down when paused still show an open outage in the report?** Task 3 only excludes `deletedAt` (deleted) monitors. Options:
- **A (plan default):** keep paused monitors in scope — their outage shows, now correctly capped to "now" by Task 1. Honest but the row keeps growing while paused.
- **B:** also exclude `active: false`. Cleaner board, but hides a real "was down when we paused it" fact.

I recommend **A** (don't hide reality; Task 1 already stops the runaway number). Flag if you prefer **B** and I'll adjust the Task 3 `where`.

---

## Self-review notes

- **Spec coverage:** Bug #1 → Task 2; Bug #2 → Task 1; Bug #3 → Task 3; regression protection + manual proof → Task 4. All three diagnosed bugs have a task.
- **Naming consistency:** `ongoingOutageMinutes(downStart, rangeEnd, now)` and `reconstructDownStart(monitorId)` are used identically in their definitions, tests, wiring, and smoke.
- **No schema migration** — relies on existing `Heartbeat` indexes; safe for the cPanel deploy path that doesn't auto-run migrations.
- **Resolved outages untouched** — only OPEN/ongoing duration math changes in the report.