docs(showcase): add PocketBase D5 history analysis to DEBUGGING.md

Strategy 9: pull all D5 status records from PocketBase in one request,
categorize errors, cross-reference with deploy history, and compute
per-service flapping rates to distinguish real bugs from transient
production issues.

Author: jpr5

showcase/DEBUGGING.md

@@ -382,6 +382,71 @@ RAILWAY_PROJECT_ID=6f8c6bff-a80d-4f8f-b78d-50b32bcf4479 \
 
 Missing `ANTHROPIC_BASE_URL` or `GOOGLE_GEMINI_BASE_URL` means the service bypasses aimock and hits the real API. This produces non-deterministic results that look like flapping. Local docker-compose sets ALL providers to aimock — production must match.
 
+### Strategy 9: Pull PocketBase D5 history to distinguish bugs from flapping
+
+**When**: Production shows blues (D4) but everything passes locally. You need to know if a feature is genuinely broken or just flapping from transient production conditions.
+
+**Do**: Pull the current D5 status records from PocketBase in one request and categorize the errors. PocketBase stores ONE record per `d5:<slug>/<featureType>` key (upsert), with `fail_count` tracking consecutive failures.
+
+```sh
+# Get ALL currently-red D5 records with error details:
+curl -s 'https://showcase-pocketbase-production.up.railway.app/api/collections/status/records?sort=-updated&perPage=200&filter=key~%22d5:%22%20%26%26%20state!=%22green%22' | \
+  python3 -c "
+import json, sys
+data = json.load(sys.stdin)
+items = data.get('items', [])
+print(f'{len(items)} RED D5 records')
+for item in items:
+    key = item.get('key', '?').replace('d5:showcase-','')
+    fc = item.get('fail_count', 0)
+    signal = item.get('signal', {})
+    error = signal.get('errorDesc', '') if isinstance(signal, dict) else ''
+    print(f'{key:<50} fc={fc:<3} {error[:80]}')
+"
+```
+
+**How to read it:**
+- `fail_count=1` → just flipped red on the last cycle (likely flapper)
+- `fail_count>=3` → consistently failing (likely a real bug)
+- `fail_count=0` with `state!=green` → transitional state, check again next cycle
+
+**Categorize errors** to find the root cause pattern:
+- `page.fill: Timeout` → React hydration too slow (probe infrastructure issue)
+- `assistant did not respond within 30000ms` → backend or aimock not returning
+- `chat input not found` → selector cascade failed (page didn't render)
+- `keyword missing` → aimock returned wrong fixture
+- `auth: after clicking sign-out` → auth probe timing race
+
+**Cross-reference with deploy history** to identify deploy churn:
+```sh
+gh run list --branch main --limit 10 -R CopilotKit/CopilotKit --workflow "Showcase: Build & Push"
+```
+
+If a feature flipped red right when a deploy happened and `fail_count=1`, it's deploy churn — the Railway service restarted mid-probe. Wait one cycle and re-check.
+
+**Get per-service flapping rates** from the harness API (last 10 probe runs):
+```sh
+curl -s 'https://showcase-harness-production.up.railway.app/api/probes/probe:e2e-deep' | \
+  python3 -c "
+import json, sys
+data = json.load(sys.stdin)
+runs = [r for r in data.get('runs', []) if r.get('state') == 'completed' and r.get('summary')][:10]
+stats = {}
+for run in runs:
+    for svc in run['summary'].get('services', []):
+        slug = svc.get('slug','?').replace('e2e-deep:showcase-','')
+        result = svc.get('result', '?')
+        stats.setdefault(slug, {'green': 0, 'red': 0})
+        stats[slug]['green' if result == 'green' else 'red'] += 1
+for slug in sorted(stats):
+    s = stats[slug]
+    rate = s['green'] / (s['green'] + s['red']) * 100
+    print(f'{slug:<25} {s[\"green\"]}/{s[\"green\"]+s[\"red\"]} green ({rate:.0f}%)')
+"
+```
+
+**Key insight**: If ALL integrations flap at similar rates (50-70% green), the problem is systemic (probe infrastructure), not per-integration code bugs. If only one integration is consistently red while others are green, it's a real code bug in that integration.
+
 ## Quick Diagnostic Commands
 
 ```sh