The Plateau Is a Skill-Type Discontinuity, Not a Difficulty Increase

The learning staircase for VLM-primary hybrid navigation has a hidden discontinuity between Level 3 (BUILDER) and Level 5 (INTEGRATOR). The research calls Phase 2c "medium-term, requires Phase 1 SLAM" as if SLAM is simply the next item on a homogeneous skill list. It isn't. Levels 1–3 are an ML skills domain: Python, prompting, API calls, EMA filters. You iterate in seconds. Failure is a wrong output token. Level 4 is an infrastructure skills domain: ROS2 lifecycle nodes, Zenoh session configuration, Docker multi-stage builds, sensor TF frame calibration. You iterate in hours. Failure is a silent drop with no error message — MessageFilter discards your lidar scans because the IMU topic timestamp is 300ms ahead, and nobody told you.

What the plateau actually looks like in practice: Sessions 86–92 in this project were spent implementing SLAM (session 88), discovering the Zenoh apt package ships the wrong wire protocol version (session 88–89), building a multi-stage Dockerfile with a Rust toolchain just to compile rmw_zenoh from source (session 89), fixing the IMU frame_id from base_link to base_footprint (one string, six hours of debugging — session 92), writing a periodic_static_tf publisher because slam_toolbox's lifecycle activation requires a TF gate that no documentation mentions (session 92), and tuning EKF frequency from 30 Hz to 50 Hz because MessageFilter's hardcoded C++ queue size of 1 was dropping 13% of scans under load. None of this is "more ML." It's a different field entirely — distributed systems, sensor fusion, robotics middleware — wearing robotics clothing.

The minimum viable knowledge for each level:

Level 1 (CURIOUS): Zero prerequisites. One video. The goal is visceral understanding that a robot can navigate from camera-only VLM inference at 54 Hz without a map.

Level 2 (TINKERER): Python and an API key. Run _ask_vlm(image_b64, prompt) in a loop. The key insight here is that the single-token output format ("LEFT MEDIUM") is what makes 18ms/frame latency possible — you're not parsing a paragraph, you're reading two tokens. Once you see this, the multi-query alternation pattern becomes obvious: you get scene + obstacle + path for free by cycling prompts across frames.

Level 3 (BUILDER): Add hardware: Pi 5 + edge GPU (Panda/Jetson/similar) + USB camera + HC-SR04 sonar. Deploy the NavController. The time investment is 1–3 days of GPIO wiring, Docker setup for the VLM server, and getting the /drive/* endpoints responding. The VLM side is still pure Python prompting — you haven't touched ROS2. Phase 2a and 2b are fully achievable here: multi-query dispatch, EMA filter, confidence-based speed modulation, scene change detection via variance tracking.

Level 4 (PLATEAU) has two sibling rungs, not one. Rung 4a is SLAM deployment, described above: lidar, ROS2 Jazzy, slam_toolbox, rf2o, IMU, Zenoh source build, multi-stage Dockerfile, TF frame archaeology. Rung 4b is the rung most practitioners never see, because it is invisible until it is named: activate the idle NPU on the robot you already built. The Hailo-8 AI HAT+ — 26 TOPS, purchased months ago, physically attached to the Pi 5 — has been sitting idle for the entire VLM build-out. YOLOv8n runs on it at 430 FPS with zero WiFi dependency. The IROS dual-process paper (arXiv 2601.21506) shows that exactly this split — a fast local detector under a slow semantic VLM — cuts end-to-end latency by 66% and lifts task success from 5.83% (VLM-only) to 67.5%. Rung 4b costs ~1–2 engineering sessions per the research doc's assessment. The same skill-type discontinuity applies as 4a: HailoRT + TAPPAS GStreamer pipelines + .hef compilation from ONNX is a new ecosystem to learn, not "more ML." But there is no procurement wait, no hardware dependency chain, no permission to request. The rung is already built into your robot.

The invisible-rung principle. The Learning Staircase lens surfaces a meta-lesson that is normally hidden by how roadmaps are drawn: the staircase has invisible rungs corresponding to dormant hardware already owned. The next step up is not always "buy more compute" — it is often "activate what you bought months ago." In this codebase, the pattern repeats: the Hailo-8 on the Pi 5 is idle; the Beast (second DGX Spark) sits dormant while Titan does the work of both; an Orin NX 16 GB is owned and earmarked for a future robot that has not yet been assembled. Each is a ready-made rung on the Level 4 tier. The reason they stay invisible is that the published research roadmaps list models and algorithms, not idle silicon — so a practitioner reading the roadmap feels stuck between "VLM working" and "buy a better GPU" and misses the fact that the better rung is already mounted to the chassis. Practitioners should audit their hardware inventory every time they feel plateaued: the next staircase step may be physical, not ordered.

Level 5 (INTEGRATOR): Once SLAM is stable and the Hailo-8 is serving YOLOv8n bounding boxes to the nav loop, integration is almost anticlimactic. You already have (x, y, heading) from SLAM pose. You already have scene labels from the VLM. You already have fast reactive obstacle boxes from the NPU. You compose them into the dual-process architecture: Hailo-8 at 30+ Hz as the safety floor (L1), VLM at 15–27 Hz as the semantic layer (L2), SLAM + VLM semantic-map fusion on top. Room annotations accumulate. Annie answers "go to the kitchen" via SLAM path + VLM waypoint confirmation, and keeps avoiding obstacles even when WiFi drops because L1 is purely local. The hard part was getting here, not the code at the top.

Level 6 (EXTENDER): AnyLoc, SigLIP 2, PRISM-TopoMap. Custom embeddings for place recognition. Voice queries against the semantic map. This is where you're doing original work — combining the research's described architecture with hardware-specific constraints (800MB SigLIP 2 competing with 1.8GB E2B VLM for 4GB of Panda VRAM). At this level, you're contributing back to the methodology.

What unsticks people at the plateau: Three things, in order of impact. First, a working Docker Compose that someone else has already debugged — one where the Zenoh version is correct, the healthchecks are real (not exit 0), and the TF supplement node is already included. The research has this in services/ros2-slam/. Second, a sensor validation script that prints a single line: "IMU: OK, Lidar: OK, TF: OK, EKF: OK." Four green lines means you can start. Third, accepting that the SLAM plateau is not a sign you're doing something wrong — it's a domain transition. You're not a bad ML practitioner. You're a good ML practitioner who has just entered robotics middleware, which has a 20-year accumulation of sharp edges.

15-minute demo vs. 3-hour deep dive: The 15-minute demo lives entirely at Level 2. Show a webcam feed. Run the VLM. Print LEFT/CENTER/RIGHT at 54 Hz. Then show the multi-query cycle: frame 0 asks "Where is the mug?", frame 1 asks "What room is this?", frame 2 asks "Nearest obstacle?". Print all three on screen simultaneously. That's the architecture. Nothing else is needed to convey the core insight. The 3-hour deep dive starts at Level 3 and spends roughly 90 minutes at Level 4 — specifically on Zenoh version selection, multi-stage Dockerfile construction, TF frame naming conventions, and EKF parameter tuning. The remaining 90 minutes covers Phase 2c semantic annotation and the VLMaps pattern. The demo-to-deep-dive ratio is 1:12, and almost all the difficulty is concentrated in one transition: the plateau.