Phase 2a deployed — team optimistic

Multi-query pipeline live. 29 Hz goal tracking + 10 Hz scene classification. 58 Hz throughput intact. Annie successfully navigates to kitchen, finds Mom's tea. Internal Slack: "this is working better than expected."

WiFi instability begins — dismissed as transient

Pre-monsoon humidity rises. Neighbors' routers add 2.4 GHz congestion. VLM inference RTT to Panda climbs from 18ms to 35–90ms on roughly 8% of frames. The NavController's 200ms command timeout fires silently — robot freezes mid-corridor, resumes after reconnect. Team notes it in a comment but ships no fix: "it usually recovers." No fallback behavior exists. The fast path was engineered to 1ms precision; the failure path was never designed at all.

Partial mitigation (deployed APR 2026): Hailo-8 L1 safety layer runs YOLOv8n at 430 FPS locally on Pi 5 (zero WiFi dependency). The safety path no longer freezes — Annie still avoids obstacles during brownouts. But L2/L3 semantic queries ("where is the kitchen?", "what room is this?") still degrade silently when VLM RTT spikes. The robot keeps moving; it just stops understanding. Mom experiences this as "Annie is wandering" rather than "Annie is frozen" — a different failure, not a solved one.

Glass door collision — both sensors wrong simultaneously

Mom's bedroom has a floor-to-ceiling glass sliding door left partially open at 45°. Annie approaches at 1 m/s. VLM reports "CLEAR" — the glass is transparent, the camera sees the room beyond. Lidar beam strikes the door at a glancing angle (below the reflectance threshold), returns no return. The "VLM proposes, lidar disposes" safety rule assumes at least one sensor is correct. Both are wrong simultaneously. ESTOP fires at 80mm — too late. Annie hits the door frame at reduced speed, knocking it off its track. Mom is shaken. No injury, but trust is damaged. The temporal smoothing (EMA filter) had 14 consecutive confident "CLEAR" readings — it amplified the error rather than catching it.

IMU REPL crash corrupts SLAM map — localization lost for 3 days

Pico RP2040 drops to REPL during a long navigation session (known failure mode, requires manual Ctrl-D soft-reboot). Without IMU heading, EKF diverges within 90 seconds. slam_toolbox accumulates ghost walls. The occupancy grid — which Phase 2c semantic annotation was being built on top of — becomes unusable. Three days of room-label training data are corrupted. The map must be rebuilt from scratch. Phase 2c rollout is delayed 3 weeks. This is the second time a Pico REPL crash has blocked a milestone; no watchdog or auto-recovery was ever implemented.

Mom stops using Annie — "it just freezes"

Monsoon peak. WiFi drops 15–20% of frames during peak household streaming hours (7–9pm, when Mom most often wants tea or the TV remote). Annie freezes in the hallway, blocking passage. When it resumes, it has lost goal context and asks "Where would you like me to go?" Mom has to repeat herself. After the third freeze in one evening, Mom stops calling Annie. She doesn't complain — she simply stops. The team doesn't notice for two weeks because the dashboard shows 94% nav success rate (computed over all hours, not the 7–9pm window). The metric was right; the window was wrong.

Phase 2c stalls — SLAM prerequisite chain broken

Phase 2c (semantic map annotation) requires Phase 1 SLAM to be stable enough to serve as pose ground truth for labeling. But SLAM is still fragile — the IMU watchdog is unimplemented, map corruption happens roughly monthly, and the Zenoh fix from session 89 was never deployed (the multi-stage Dockerfile buildx build has been "blocked on CI setup" for 3 months). Phase 2c cannot start. Phase 2d (embeddings) cannot start without 2c. Phase 2e (AnyLoc) cannot start without 2d. Three of five Phase 2 sub-phases are gated behind an infrastructure prerequisite that is itself gated behind another prerequisite. The roadmap looked like a DAG; it was actually a single chain.

VRAM ceiling hit — Phase 2d quietly abandoned

SigLIP 2 ViT-SO400M requires ~800MB VRAM on Panda. The E2B VLM already uses ~1.8GB. Panda's GPU has 4GB total. With OS overhead, the two models cannot coexist. The research said "competing with VLM for VRAM" — the competition was never resolved. Phase 2d is deprioritized to "future work." The embedding extraction capability — which would have enabled place recognition, loop closure augmentation, and scene change detection — is shelved. The perception architecture loses its memory layer before it was ever built.

Project pivots — edge thesis abandoned, cloud VLM fallback adopted

"Too many moving parts on Panda." The decision is made to route VLM inference to Titan over the home LAN, treating WiFi as the transport layer rather than the failure mode. This is the exact architectural bet the research identified as the risk: if WiFi is unreliable, cloud inference is worse. The pivot does not solve the glass door problem, the IMU crash problem, or the SLAM prerequisite chain. It trades edge latency (18ms) for LAN latency (35–120ms) and makes the system more fragile to the same failure that already caused Mom to stop using Annie. Six months of edge-first infrastructure work is partially undone in one architectural decision made under time pressure.

Orin NX 16GB module sits unused for 6 months — no carrier board ordered

Optional/speculative scenario. An Orin NX 16GB SoM is purchased mid-2027 as "future upgrade path" to run Isaac Perceptor (nvblox + cuVSLAM) locally. The module ships in a tray; the carrier board is on a separate SKU from a different vendor with a 4–8 week lead time. No one orders it. The module sits in a drawer for six months. By the time the carrier arrives, DGX Spark + Panda already handle the workload the Orin was meant to absorb, and the stereo camera required by cuVSLAM still hasn't been purchased either. The hardware isn't wrong; the bill-of-materials discipline is. One missing $200 part turns a $600 module into a paperweight. Buying into an ecosystem before verifying the full chain works end-to-end is its own failure mode.