Back to dashboard

Multi-year backtest validation

Does the Phase 1.4 scoring rule flag danger earlier than response actually began?

Illustrative analysis, not a certified backtest. Rainfall values below are reconstructions of the reported shape of each season (steady build, peak, recession), not gauge records. Response timings are approximate readings of published reporting. Nothing here should be used operationally without validation against official KMD / WRA gauge archives.

March–May 2024 long rains

Tana Delta / Tana River basin (with Seven Forks cascade)

Reported outcome: 294 deaths, ~55,000 households displaced, impacts in up to 42 of 47 counties, and 192 dams nationally flagged high-risk, including Seven Forks dams on the Tana spilling into downstream communities.

Rule first flags High
Week 3 (late Mar)
Rule first flags Critical
Week 4 (early Apr)
Illustrative lead time
~3 weeks earlier

Step by step — what the rule would have said each week

  1. Week 1 (mid-Mar)
    8 mm/24h · 26 mm/7d21/100Low
  2. Week 2
    17 mm/24h · 58 mm/7d45/100Moderate
  3. Week 3 (late Mar)
    26 mm/24h · 92 mm/7d71/100High
    EcoTwin would have escalated to authorities here.
  4. Week 4 (early Apr)
    38 mm/24h · 131 mm/7d98/100Critical
  5. Week 5 (mid-Apr)
    52 mm/24h · 178 mm/7d100/100Critical
  6. Week 6 (late Apr)
    61 mm/24h · 214 mm/7d100/100Critical
    Large-scale national response and mass evacuation orders scale up (per cited reporting).
  7. Week 7 (early May)
    44 mm/24h · 196 mm/7d100/100Critical

Breakdown at the first High flag (Week 3 (late Mar))

The same explainable factors responders would have seen in the app.

  • 24-hour rainfall
    26.0 mm in the last 24h vs threshold 40 mm (65% of threshold)
    +33
  • 7-day rainfall
    92.0 mm over the last 7 days vs threshold 120 mm (77% of threshold)
    +38
Total risk score71
Sources referenced
  • 2024 season fact sheet (as cited in this project brief) — 294 deaths, ~55,000 households displaced, up to 42/47 counties affected, 192 high-risk dams.
  • Seven Forks / Tana River dam overflow reporting — Downstream communities inundated by cascade spill during the same window.

March–May 2018 long rains

Tana River / Lake Victoria basin counties

Reported outcome: widespread flooding across more than 30 counties, with roughly 180+ deaths and several hundred thousand people displaced. The Patel Dam breach in Solai (Nakuru) in May 2018 killed dozens, underlining infrastructure — not only rainfall — as a driver.

Rule first flags High
Week 3 (late Mar)
Rule first flags Critical
Week 4 (mid-Apr)
Illustrative lead time
~3 weeks earlier

Step by step — what the rule would have said each week

  1. Week 1 (mid-Mar)
    6 mm/24h · 22 mm/7d17/100Low
  2. Week 2
    14 mm/24h · 51 mm/7d39/100Moderate
  3. Week 3 (late Mar)
    23 mm/24h · 84 mm/7d64/100High
    EcoTwin would have escalated to authorities here.
  4. Week 4 (mid-Apr)
    35 mm/24h · 126 mm/7d94/100Critical
  5. Week 5 (late Apr)
    49 mm/24h · 171 mm/7d100/100Critical
  6. Week 6 (early May)
    58 mm/24h · 205 mm/7d100/100Critical
    National multi-agency response and displacement camps scale up (per cited reporting).
  7. Week 7 (mid-May)
    41 mm/24h · 188 mm/7d100/100Critical

Breakdown at the first High flag (Week 3 (late Mar))

The same explainable factors responders would have seen in the app.

  • 24-hour rainfall
    23.0 mm in the last 24h vs threshold 40 mm (57% of threshold)
    +29
  • 7-day rainfall
    84.0 mm over the last 7 days vs threshold 120 mm (70% of threshold)
    +35
Total risk score64
Sources referenced
  • 2018 long-rains flooding reporting — 30+ counties affected; ~180+ deaths; several hundred thousand displaced.
  • Solai (Patel Dam) breach, May 2018 — Dam failure caused dozens of deaths downstream, independent of that day's rainfall.

What this suggests

In both reconstructed seasons the rule crosses High while weekly totals are still climbing — roughly two weeks before documented large-scale response scaled up — because the 7-day accumulation term reacts to sustained rain rather than a single headline downpour. Both seasons also included an infrastructure failure (Seven Forks spill in 2024, the Solai dam breach in 2018) that rainfall alone would not have caught, which is why the live system compounds dam status into the score.