I started to notice that most credible robotics companies I could find, from the frontier humanoid labs aal the way to other YC startups, are organised around one thing: manufacturing or exploiting demonstration data from the physical world. The models are public and the bodies they use are commoditising by the month; the bottleneck seems to be the data. This post is my personal robotics primer i made from reading as prep getting into the space.
This post is long - it was my collation I used during the lead time on my own arms. So as they shipped, I read through this countless times. Two Eras and How Robots are Learning give you the larger picture, If We Solve This maps what actually changes if the bets I'm making land, The Analogy with LLMs replays the LLM ecosystem's 2022-26 history and runs it forward as a forecast for robotics, and The Full Stack walks the machine layer by layer from motors to teleop fallback.
Two Eras
For roughly fifty years, "robotics" has meant classical automation, which is still most of the installed base and defines the baseline the new wave is trying to beat.
A classical industrial robot is a precise, dumb machine that repeats an exact motion forever. An engineer hand-teaches it a sequence of coordinates (move to point A, close the gripper, move to point B, open) and the robot replays those coordinates to sub-millimetre accuracy in a structured, predictable environment. The brain running this is often a PLC (programmable logic controller), a ruggedised industrial computer that executes deterministic logic.
This paradigm is fast, cheap-per-cycle, and extraordinarily reliable, but it's brittle: it doesn't "understand" what it's grabbing, so it fails the instant a part is in the wrong place or a new product comes down the line. Reprogramming means an engineer rewriting the routine, which is why classic automation only pays off at scale: you need to push millions of identical units through one fixed cell for years to justify the integration cost.
The hardware of this era is worth knowing by name, because it's the same machinery the embodied AI crowd bolts its models onto: a learned policy outputs "close the gripper here", and something physical still has to grip. Every one of these machines is really two parts, bought and made smart separately. The arm is just a positioner: it moves an empty wrist through space and, on its own, does nothing useful.
The end effector (end-of-arm tooling, or EOAT) is whatever you bolt to that wrist to do the actual work: a gripper, a suction cup, a welding torch, a screwdriver, a hand. Hold onto that split (the tool first, then the arms that carry it), because that's the order the rest of this section follows.
Robotiq, founded in 2008 in Levis, Quebec, is the best-known independent maker of adaptive grippers, and its flagship 2F-85 is as close to a standard part as this field gets. It's a two-finger electric gripper: parallel jaws with an 85 mm stroke squeeze with a programmable 20-235 N of force, and can either pinch a part or wrap around it in an "encompassing" grip. It weighs 925 g, talks over RS-485/USB, costs roughly $4,999-$5,825, and ships with plug-and-play URCap software so one line of code opens or closes it.1
That last bit is the real product: grasping is the hardest unsolved problem in robotics, and a good gripper hides it. Robotiq also sells vacuum grippers (the PowerPick line) and force-torque sensors, raised C$31M (about US$23M) from Battery Ventures, and (unusually for this space) hasn't been reported as acquired as of 2026.2
That's the tool; now the arm that swings it. Almost all of these arms are the same underlying mechanism - a serial chain of motorised joints, the classic articulated arm - and they split on one thing: safety.
The caged kind is the precise, dumb machine from the top of this section, fast and strong but fenced off from people. The kind people actually buy for new work is the cobot (collaborative robot): the same articulated arm made lightweight and force-limited, safe enough to work next to humans without a safety cage because it senses collisions and stops. "Positioner" is what any of these arms does; "cobot" is just the human-safe flavour of it.
Denmark's Universal Robots created the category in 2008 and still dominates it, with roughly 31-39% market share and 75,000+ units installed across 65 countries; Teradyne bought the company in 2015 for $285M.3 The models are named by payload in kilograms: UR5e (5 kg payload, 850 mm reach), UR10e (12.5 kg, 1300 mm), UR16e (16 kg, 900 mm), UR20 (20 kg payload generally, with 25 kg available only when the documented boundary conditions are satisfied, 1750 mm).
No vendor in this world lists true public prices; everything is distributor and integrator quotes that move with tooling, region, and support. As of 2025 the street figures run about $30k-40k for a UR5e, $45k-60k for a UR10e, $57k-65k for a UR16e, and $65k-85k for a UR20 with tooling and integration counted.4
China's AUBO undercuts all of this with 6-axis clones on a nearly identical naming scheme (i3 through i16, built in Changzhou with a US arm in Detroit): the AUBO-i5 gives you 5 kg of payload, 886.5 mm of reach, and ±0.02 mm repeatability, and AUBO was the first Chinese cobot maker to pass EN ISO 13849-1 safety certification.5
Every arm so far has been one serial chain of joints. The delta robot breaks that mould entirely. It's the other mechanism family: three thin arms with parallelogram linkages hang from a ceiling-mounted base down to a small floating platform. Because the heavy motors stay fixed at the top and only lightweight linkages move, it's blindingly fast (over 15 g of acceleration, 150+ picks per minute, over 200 cycles per minute in modern variants) but limited to payloads of a kilogram or two and mostly up-down-sideways motion.
Reymond Clavel invented it at EPFL in 1985 to drop chocolate pralines into boxes, Demaurex licensed it in 1987, and ABB's FlexPicker is the iconic commercial line: the 4-axis IRB 340 moves 1-2 kg at 150 picks per minute, and the 5-axis IRB 365 handles up to 1.5 kg at 120, with an IP54 rating and food-grade lubrication for food and pharma lines.6
All of this is mature, mass-deployed, and boring in the best sense. The cobot market alone was worth roughly $1.4-2.3B in 2025 and is growing at around 19-21% a year, with the cobot end-effector market forecast to reach about $2.66B by 2030.7 None of it is hobbyist-tier - it's $5k-85k professional equipment that lives in real factories - so buying it is a procurement decision, not a research breakthrough. The value at the frontier sits entirely in the software layer that makes these commodity bodies smarter.
That said, if you're a lab or startup building manipulation policies, the canonical dev stack is a UR5e plus a 2F-85: the most documented combo in existence, with plug-and-play URCap software and huge community and ROS support, so you spend your time on the AI rather than on drivers.
Buy through authorised distributors (Electromate in Canada, or UR+ certified partners; Robotiq sells through its own Configurator), and note that used UR arms go for well below new. If budget is tight and you can tolerate less software polish, the AUBO i5 is meaningfully cheaper for the same ~5 kg / ~880 mm class, but treat China-sourced cobots as a cost play: validate support, safety certification, and software-ecosystem maturity before committing to production.
A FlexPicker is a different animal entirely, a line-integration capex purchase in the tens of thousands rather than a benchtop buy. Whatever you pick, get multiple quotes, and for the hobbyist end of the ladder see Getting Hands On.
The new era, which most companies in the YC ecosystem are organised around, is "physical AI" (also called embodied AI or robot learning). The core bet is a direct analogy to what happened in language: GPT-style models got smart by training on the entire internet, so a robot foundation model should get smart by training on massive, diverse data of the physical world.
Instead of hand-coding one program per task, you train a neural network (a "policy") that maps what the robot sees (camera pixels, sensor readings) to what it should do (motor commands). The setiment here is "replace mechanical complexity with intelligence": rather than ever-more-precise, expensive, purpose-built hardware, use cheap general-purpose arms driven by a big learned model, and let software absorb the complexity the way GPT absorbed language.
If it works, it collapses the cost of automation and unlocks the "long tail" of messy, constantly-changing tasks (warehouse packing with shifting SKUs, home chores, hospitality) that classical robots could never touch. (The analogy holds well enough that The Analogy with LLMs later runs it as a forecast.)
The catch, and the thing to internalise above all else, is that the bottleneck is currently data, not algorithms. Unlike text, there's no internet-scale corpus of robot actions to scrape, and that scarcity is the organising fact behind almost every company in the rest of this post.
How Robots are Learning
For decades the real answer was hand-written control code plus reinforcement learning (RL), where a robot learns by trial and error against a reward signal. RL is powerful but slow, unstable, and hard to specify for messy real-world tasks (what's the reward function for "fold this shirt nicely"?).
The dominant training method now is imitation learning, also called behaviour cloning (BC): a human performs a task through the robot dozens or hundreds of times, and a neural network is trained by supervised learning to copy the mapping from observation to action. It's the robotics version of how an LLM is pre-trained on human text.
BC has a famous weakness called compounding error: the policy predicts one action at a time, small mistakes push the robot into states it never saw in training, and errors snowball. Two 2023 breakthroughs fixed this by predicting chunks of future actions at once.
ACT (Action Chunking with Transformers)8 pairs a transformer with a conditional variational autoencoder and debuted on Stanford's low-cost ALOHA rig (roughly $20k of bimanual hardware at debut; ALOHA-2-class stations now run roughly $20k-32k depending on configuration, and later designs are cheaper still). It could thread a needle, open a ziploc bag, and slot a battery.
Diffusion Policy9 generates action sequences by iterative denoising with a CNN or transformer backbone, borrowing the exact maths behind image generators like Stable Diffusion, and is the usual pick over ACT for contact-rich tasks where several trajectories are equally valid and you want smooth, multi-modal output. These two are now standard baselines you'll hear named constantly, and they train from remarkably little data: ACT-class tasks typically start from around 50 teleoperated demonstrations.
The current frontier is the Vision-Language-Action (VLA) model: take a pretrained vision-language model that already understands images and text from the internet, and bolt on an "action head" that outputs robot commands. Google DeepMind's RT-210 proved the concept in 2023 by treating robot actions as just more text tokens, and it demonstrated the emergent semantic reasoning that makes VLAs exciting: told to grab "something to clean a spill," it picks the sponge without being explicitly taught. The scaled-up RT-2-X, trained on the Open X-Embodiment dataset, is a 55-billion-parameter policy that needs a multi-GPU server just to run.
OpenVLA11 (June 2024, from Stanford and Berkeley) made the recipe open and small enough to matter. It's a 7B-parameter model built on the Prismatic VLM - a 600M fused SigLIP+DINOv2 visual encoder feeding a Llama-2 7B backbone - that discretises each action dimension into bins and emits them as vocabulary tokens. Trained on Open X-Embodiment's 970,000 real robot demonstrations spanning 22 different robot bodies, it beats RT-2-X while being roughly 8x smaller.
The flagship line belongs to Physical Intelligence, the San Francisco lab founded in 2024 by Karol Hausman (CEO), Sergey Levine, Chelsea Finn, Brian Ichter, Adnan Esmail and Lachy Groom. The funding curve tells you what investors think this is: a $70M seed in March 2024 at a ~$400M valuation, a $400M Series A that November at ~$2.4B led by Jeff Bezos with OpenAI, Thrive and Lux, a $600M Series B in November 2025, and as of July 2026 reportedly raising ~$1B at an ~$11B valuation, up from $5.6B.12
All of that rests on roughly 80 employees and no commercial product yet: the pitch is "ChatGPT for robots," and the market is pricing the model, not revenue.
Their pi series is the lineage everyone benchmarks against. pi013 (October 2024) bolts a separate 315M-parameter flow-matching "action expert" onto a ~3B PaliGemma vision-language backbone (about 2.3B of it fine-tuned); instead of emitting action tokens one at a time, it flows out chunks of 50 future actions for smooth 50Hz control, demonstrated across seven embodiments - bimanual ALOHA stations, a single-arm Franka Panda, mobile manipulators - folding laundry, assembling boxes, and bussing tables.
pi0.514 (April 2025) added a hierarchical chain of thought: first predict a high-level subtask in plain text ("pick up the cutting board"), then generate the continuous motor commands through a 300M action expert, still in 50-step chunks. It was co-trained on ~400 hours of mobile-manipulation data plus web image-text tasks (VQA, captioning, detection) plus data from other robot bodies, and it generalises to entirely new homes.
pi0.715 (April 2026) is their current frontier and is closed-weights: only pi0, an autoregressive pi0-FAST variant, and pi0.5 are open-sourced through the Apache-2.0 openpi repository, whose base checkpoints were pretrained on more than 10,000 hours of robot data.16
NVIDIA's entry, Isaac GR00T17, is the same idea aimed squarely at humanoids, which matters because a humanoid is only useful if a single brain can generalise across chores, and VLAs are the leading candidate for that brain.
GR00T N1 (March 2025) was billed as the world's first open humanoid foundation model: a dual-system design pairing slower vision-language reasoning ("System 2") with a fast diffusion-transformer action head ("System 1"), its 2B-parameter weights free on Hugging Face. N1.5 swapped the vision-language model for Eagle 2.5 and trained for 250,000 steps on 1,000 H100s at a batch size of 16,384, nearly tripling its score on the DreamGen task suite (13.1% → 38.3%), and N1.7 has since landed on GitHub.18
GR00T targets full humanoid platforms rather than desk arms, so it's the model in this list you download to read rather than run at home.
A sobering set of numbers keeps the generalisation story grounded: pi0.5's own ablations show out-of-distribution task success dropping from 94% to 74% when web co-training data is removed, to 49% without cross-embodiment data, and all the way to 31% without multi-environment data.14 Generalisation is real but fragile, and it's bought almost entirely with data diversity, which is the reason so much money is pouring into data collection.
The one comfort is that none of this is locked inside a single lab: with openpi, OpenVLA, and the GR00T weights all free downloads, the capability is diffusing about as fast as it's being built.
Jim Fan, who runs NVIDIA's GEAR embodied-AI group, spent his 2026 talk at Sequoia's AI Ascent arguing that this whole recipe is mis-weighted: VLAs are really "LVAs", most of the parameter mass sitting in language, so the models come out strong on knowledge and nouns and weak on physics and verbs.19 His exhibit is the original VLA party trick itself, RT-2 moving a coke can onto a photo of Taylor Swift: real generalisation, but generalisation over what things are, not over how motion works.
His alternative starts from video models. Train a network to predict the next blob of pixels at scale, he argues, and it picks up gravity, buoyancy, reflection and refraction with no physics equation anywhere; his demo has Veo 3 solving a maze by running the simulation forward in pixel space. The failure clips are as telling as the wins (his favourite shows a model quietly rewriting geometry the moment the camera looks away, which he files under "physics slop"), and the claim is that next-frame prediction is doing for the physical world what next-token prediction did for language.19
DreamZero, the group's February 2026 result, turns that into a policy. It's what they call a world action model (WAM): a video model that jointly decodes the next few seconds of the world and the actions that go with them, on the observation that high-dimensional continuous motor signals look a lot like pixels, so you can render both at once.20
The paper's explanation is that the physics lives in the video prior: DreamZero zero-shots tasks and verbs it never saw in training (roughly double the generalisation of VLA baselines, plus cross-embodiment transfer), and you can watch what it's dreaming as the robot executes. When the predicted video is right the action works, and when the video hallucinates the action fails.20
Fan's own calibration is that DreamZero is "kind of like GPT-2", getting the shape of the motion right rather than getting tasks reliably done, and his stagecraft ran ahead of his results: he held an actual moment of silence for VLAs on stage ("rest in peace; long live world action models"). File that against the ~$11B being priced onto Physical Intelligence's VLA lineage and you have the field's live architecture argument in a single slide. My read is that the funeral is premature but the direction isn't: I'd expect the next frontier VLAs to absorb world-model pretraining rather than be replaced by it, which is roughly what the Learning roster below (FastWAM, VLA-JEPA, LingBot-VA) already shows happening.
There's one more architecture you may already be reaching for: why train a robot model at all when you could hand a plain LLM some tools - an inverse-kinematics solver, a grasp planner, The Classical Core wrapped in an API? James Wigglesworth, whose Amplibotics farms real-world manipulation data through teleoperated industrial arms (he co-founded Haddington Dynamics, the robot-arm maker Ocado bought in 2020), answers it on a recent podcast with an analogy from the last wave: the companies that made "video" by stitching Midjourney stills together. The statue is dancing, but it's a different statue in every frame.21
He means clips like the one below, which went viral on Instagram in May 2023: statues dancing in a marble hall, generated frame by frame with Stable Diffusion.22 Every frame is a lovely image, and it's never the same statue twice.
You can piece together impressive demos from parts built for something else, but his point is that what shipped video generation was the direct path: a model designed for the job, trained on the right inputs, and he expects the same here. The endgame he describes is the layering pi0.5 and GR00T already sketch: the LLM keeps the high-level planning and the intelligence, and pipes its output into a VLA that owns the motion.21
Inside the Models
The paragraphs above tell you who built what and when. Open the models up and every one is a remix of deep-learning ideas you may already own - CNNs, autoencoders, diffusion, transformers - pointed at a new output type, and all of them except OpenVLA ship as readable source in LeRobot. I keep my own study notes on those building blocks on my wiki, so each subsection below links the robotics idea back to the note I'd revisit to rebuild the intuition.
ACT
ACT is a conditional variational autoencoder whose decoder is DETR, the transformer object detector. Unpack that slowly. A variational autoencoder compresses its input into a small latent vector and reconstructs from it (mine live in the Stable Diffusion note, where the VAE does image compression); "conditional" means the reconstruction is also conditioned on the robot's observation. The VAE earns its place because human demonstrations are inconsistent - you grab the cup slightly differently every time - and the latent variable gives the model somewhere to put that style variation instead of averaging all your demos into mush.
The VAE encoder only exists at training time: it reads the robot state plus the whole ground-truth action sequence and squeezes them into the latent. The main transformer encoder then sees that latent, the state, and image features from a ResNet-18 whose batch norm layers are frozen (robot batches are too small and correlated to trust their statistics, so the ImageNet running averages are kept).
The clever part is the decoder: its input is literally a tensor of zeros. All the information about "which timestep am I predicting" lives in a learned positional embedding of length chunk_size, which is DETR's object-queries trick aimed at time instead of space, and it's what lets ACT emit all 100 actions of a chunk in a single forward pass rather than autoregressively, one token at a time.
Two shipped details are worth knowing because you'll meet them in the config files. n_decoder_layers defaults to 1, faithfully reproducing a bug in the original ACT code where only the first decoder layer was ever used; LeRobot keeps the bug on purpose so pretrained checkpoints stay compatible.
And ACT's optional temporal ensembling runs inference every single tick and averages the overlapping predictions for each timestep with weights , oldest heaviest; the code comments warn that favouring new actions instead would diminish the benefits of chunking, since the newest prediction is the one that has seen the least of the future it's predicting into.
Diffusion Policy
Diffusion Policy is the same denoising maths as the image generators (forward process adds noise, a network learns to reverse it - my walkthrough is in the Stable Diffusion note), except the thing being denoised isn't an image but an action trajectory: a horizon × action_dim array that starts as Gaussian noise and ends as a motion plan.
The vision stack has one lovely substitution. It's a ResNet with the final global-average pool replaced by SpatialSoftmax, which turns each feature channel into the expected pixel coordinate of its activation: a set of learned 2D keypoints, "where things are" rather than "what things are", close cousin to the landmark detection idea from classical CNN.
Observations from the last couple of ticks are encoded, concatenated and flattened into one global conditioning vector, and a 1-D conditional U-Net does the denoising, one-dimensional because it convolves over time, not image space. Conditioning enters through FiLM (feature-wise linear modulation): the conditioning vector predicts a per-channel scale and shift applied inside every U-Net block, which is how "what the camera sees" steers "what trajectory emerges from the noise".
At run time it's a receding-horizon planner: predict a 16-step trajectory, execute the first 8 actions, re-observe, predict again. You'll meet that pattern again in The Action Queue, because it's the seed of how every modern policy gets deployed.
The Pi Recipe
pi0 and pi0.5 share one architecture: a prefix/suffix, two-expert design on top of PaliGemma. The prefix is the vision-language model's territory - SigLIP image tokens plus the language instruction, attending bidirectionally, the same transformer attention you already know. The suffix belongs to a much smaller "action expert" (~300M parameters) whose tokens will become the action chunk, with an attention block boundary so the prefix can never attend to actions. One big model to understand the scene, one small model to move the arm, fused into a single transformer.
The action expert doesn't diffuse; it uses flow matching, diffusion's faster cousin. Instead of learning to undo noise step by step, the network learns a velocity field that transports noise straight toward the data. Training draws a time , blends between noise and the true action chunk , and regresses the network's predicted velocity onto the constant target with plain MSE. Generation is then just integration: start from noise, take ten Euler steps along the predicted velocity, and an action chunk condenses out.
The engineering trick that makes a 3B-parameter model usable is cache reuse. The expensive prefix runs once per chunk and its attention keys and values are cached (the KV cache: a transformer can store the key/value tensors of tokens it's already processed, so new tokens attend to them without recomputing); the ten Euler steps then re-run only the small action expert against that frozen cache. You pay for the VLM once and for the expert ten times, not the other way round.
pi0.5's changes are small enough to enumerate. Time conditioning moves out of the token sequence and into adaptive RMSNorm: the timestep embedding modulates the normalisation layers inside every expert block instead of riding along as an input token.
The robot's state moves into the text prompt: it's digitised into 256 bins and rendered as a literal string, Task: fold the towel, State: 132, 87, 201...; Action:, which is why pi0.5's tokeniser budget is 200 tokens against pi0's 48, and why the model has no state-projection layer at all: proprioception became language. And its normalisation default switches from mean/std to quantiles (q01/q99), which bites in practice: a dataset carrying only mean/std statistics fine-tunes happily under pi0 and errors out under pi0.5.
SmolVLA
SmolVLA23 is the same flow-matching recipe with an efficiency story, and its four size levers are a nice checklist for how to shrink any VLA: keep only the first 16 layers of the SmolVLM2 backbone; make the action expert 0.75x the VLM's width; have the expert cross-attend into the cached VLM keys and values rather than running joint self-attention (falling back to true self-attention every other layer); and freeze the vision encoder, training the expert alone.
The defaults are tuned for fine-tuning on a consumer GPU rather than pretraining, which is the niche it fills for SO-101 owners.
If We Solve This
Suppose the bet pays off: policies generalise, the data gold rush produces its GPT moment, and teaching a robot a new manipulation task becomes roughly as easy as describing it. What actually changes? The demo reel shows off folding laundry and pouring drinks, and if that were the product, none of this would justify the capital flowing in. Three constraints have quietly shaped the physical economy for fifty years, and they all break at once.
The first is the fixed-automation constraint from Two Eras: automation has only ever paid when you could push millions of identical units through one cell for years. Learned policies delete the words "identical" and "for years", and with them the boundary around what is automatable.
Then demographics: the supply of physical labour is capped by working-age population, and that cap is collapsing. China loses roughly 239 million working-age people by 2050, Germany's 20-64 cohort shrinks by more than ten million by 2060, and South Korea's total population is on a path from 52 million toward ~22 million by 2100.24 Physical AI is the one supply-side lever that scales with demand rather than with birth rates.
And finally presence: some work is priced not by skill but by the risk premium of putting a human body near it. Map those three constraints onto industries and the high-value targets fall out one by one.
The Cleanest Kill
Garment manufacturing is a roughly $500 billion industry employing some 60 million people, and it's still almost entirely hand-made for a single reason: nobody has ever built a machine that can handle limp fabric. SoftWear Automation's celebrated 2017 "sewbot" plant (23 million t-shirts a year for adidas at 33 cents of labour each) never delivered at that scale, and thirteen years and tens of millions of dollars in, the company raised another $20M in 2025 still chasing a finished t-shirt machine.25
Every classical "win" in this industry dodged the blocker rather than solving it: Sewbo chemically stiffens the cloth so it behaves like sheet metal, CreateMe glues seams instead of sewing them, FoldiMate made the human do the picking. Physical AI is the first approach that attacks cloth-state estimation directly, which recasts why Physical Intelligence chose laundry folding as pi0's flagship demo.13
That wasn't a chore demo; it was a shot at the last trillion-dollar unautomated industry. And if it lands, trade geography moves with it: ready-made garments are over 80% of Bangladesh's exports and around four million jobs, sewn at ~$113 a month because no machine competes.26
The Biggest Number
The single largest number in this whole exercise is care. Home-health and personal-care aides are already the fastest-growing occupation in the United States (about 766,000 openings every single year, more than any other job27), while the WHO projects a ~10 million global health-worker shortfall by 2030 and Japan alone expects to be 570,000 care workers short by 2040.28
This is the one market where demand is inelastic, demographically guaranteed to explode, and structurally unfillable by human supply. It's also the hardest and slowest target on the list: in-home dexterous assistance is the unsolved frontier The Frontier Map covers later as the dexterity problem, wrapped in safety and liability. Biggest prize, worst first wedge: so the near-term entrants sell monitoring and companionship while the physical-assistance half waits for the hands to work.
The Baumol Reversal
Construction is the strongest candidate for the largest structural change. US construction value-added per worker fell roughly 40% between 1970 and 2020 while economy-wide productivity more than doubled (Goolsbee and Syverson call it "strange and awful"), making construction the textbook case of Baumol's cost disease, where the price of labour-heavy work inflates simply because the labour never gets more productive.29
Labour is about half of direct construction cost, construction is a record 64.4% of a new-home price, and McKinsey prices closing the productivity gap at ~$1.6 trillion a year, against a workforce the industry says needs 349,000 more people in 2026 alone.30 If learned policies crack unstructured site work, housing affordability stops being labour arithmetic. That's why the earthmoving wedge (Off-Highway Autonomy, below) matters beyond its own revenue: fenced sites are the beachhead.
Off-Highway Autonomy
"Off-highway autonomy" means self-driving machines that never touch a public road: haul trucks in open-pit mines, excavators and dozers on jobsites, tractors and sprayers in fields. Fenced sites are the most commercially mature form of vehicle autonomy, years ahead of robotaxis, for a boring technical reason: the environment is bounded.
Routes are fixed haul roads, the operator writes the traffic rules and enforces them on every vehicle, speeds are low (mine haul trucks run 40-60 km/h), there are no pedestrians or unpredictable human drivers, and GPS/RTK positioning works beautifully in an open pit. You geofence a known site and design for it, instead of solving the open-world long tail that makes robotaxis hard.
The business splits two ways. Incumbent OEMs sell integrated autonomy on their own new iron: Caterpillar's Cat Command for Hauling and Komatsu's FrontRunner coordinate whole fleets from a control room, and their numbers are anything but experimental - Caterpillar ran ~827 autonomous trucks in 2025, publicly targets over 2,000 by 2030, and has passed two billion tonnes hauled, while Komatsu's thousandth ultra-class truck and ten-billion-tonne total appear in Already Inverted below.31
The price of admission is mine-scale: an ultra-class truck runs $5-8M new (the electric-drive, ~400-tonne Cat 798 AC shipped commercially in 2025 with autonomous-ready hardware), a FrontRunner retrofit takes about a week on newer trucks and up to three on older ones, and a full autonomous-mine deployment with networks and a control centre has run $100M+, for reported ~30% efficiency gains at Vale and Fortescue.32
Independents retrofit existing mixed-brand fleets instead: ASI (Autonomous Solutions Inc., spun out of Utah State University in 2000) runs its OEM-agnostic Mobius platform across 100+ mixed-brand machines from one control centre, its ASI Mining arm (majority-owned by Epiroc) bolts autonomy onto older Cat and Komatsu trucks, and in March 2026 ASI acquired Scythe Robotics for its Scythe Sight computer-vision stack.33
So the verdict splits too. Autonomous haulage is commodity infrastructure, arguably the most boring and most successful autonomy on Earth, with a decade of operating history. The live frontier is construction and earthmoving, which is far less solved than mine haulage: OEM-agnostic retrofit autonomy for excavators, dozers and mixed construction fleets, where teleoperation-first data flywheels are racing to train policies from real-site data.
There's also a second, stranger moat: getting autonomous machinery CE-marked in Europe at all (Regulatory Friction explains why even an internal EU prototype is legally a machine). Which brings us to the two-person company running this playbook.
Flywheel AI
Flywheel AI (YC Summer 2025, San Francisco) pitches itself as "Waymo for excavators".34 Most excavators are purely hydraulic (the joysticks push pilot hydraulic circuits, so there's no wire for a computer to hack), and roughly six million of them already exist worldwide, so Flywheel retrofits rather than replaces.
Their machine-agnostic kit physically bolts actuators onto the existing joysticks and foot pedals, adds multiple cameras plus telemetry and audio streaming, and hands the machine to a remote operator driving from a laptop and joystick console over the internet. Install takes a few hours and doesn't stop a human climbing back into the cab.
Every teleoperated hour records synchronised camera video and joystick inputs, and that's the point. The staged plan runs remote operation today, partial autonomy of repetitive sub-tasks next (the founders single out the swing motion as work that "should not be manual"), then supervised autonomy with one operator overseeing several machines, with autonomous dig, dump, trench and grade workflows on the roadmap alongside fleet control and supervisor override.
In partnership with Frodobots.ai/BitRobot (FrodoBots Lab raised $8M for BitRobot and previously open-sourced a 2,000-hour driving dataset) they released the "Excavator" dataset from a 38-tonne Volvo EC380, marketed as ~100 hours: the Hugging Face card shows ~96 hours of raw video across four cameras (176 sessions at 1080p, video at 25fps with operator actions at 25Hz), which works out to only ~24.6 hours of unique operation, stored as plain MP4 and CSV rather than LeRobot format.35
Their early bucket pick-and-place model was trained separately on ~6-7 hours collected on a Kubota U17 during the YC batch.
Their YC page frames the problem: a roughly one-million-worker US construction shortage, 1,000+ construction deaths a year, and ~40% operational downtime from inefficiency and blind spots, all borne by a shrinking, ageing pool of skilled operators sitting inside machines that can crush people.
Pulling the operator out of the cab solves safety and lets one operator cover multiple sites; the company's own economics are ~$150K a year in operator savings plus ~$500K in avoided safety costs per machine, set against a $2 trillion global construction market and a claimed ~$180B retrofit TAM.36 Pricing isn't published: estimates floating around put it at ~$20,000 per kit plus ~$6,000 a year of software, and the "$100K+ ARR potential per machine" line is the company's own aspiration, not a list price.
The team is two people. Mahimana ("Mahi") Bhatt (CTO) took a master's at Worcester Polytechnic Institute, passed through IHMC, then spent four and a half years at Motional rising from intern to senior software engineer and building a data/ML pipeline running 50,000+ simulations a day. Jash Mota (CEO) previously built remote-teleoperation tools with enterprise users including Accenture. Crunchbase dates the company to 2023, though coverage frames the effective launch as mid-2025.
The Long Tail Reopens
Manufacturing is where the fixed-automation constraint bites hardest, and the shortage numbers are already brutal: Deloitte projects 1.9 million unfilled US manufacturing jobs by 2033, and the American Welding Society projects a need for over 300,000 new welders by the end of the decade against an average welder age of ~55.37
The revealing part is which tasks stayed manual. Apple's automation lab found robots couldn't feel screw torque or manage glue placement, and iPhone final-assembly automation was repeatedly shelved over defect rates; Foxconn delivered roughly 4% of its famous one-million-Foxbot pledge, and the market's actual answer was moving assembly to India.38
Poultry deboning quantifies the moat as a yield number: automated deboners leave 16-17% of the meat on the bone where humans leave ~13%, in plants running 80-100% annual turnover, which is why the world's biggest meatpacker simply bought the robotics company.39
The early proof that learned policies reopen this long tail is already commercial: Chef Robotics went from one million to more than forty million robot-plated servings in three years by targeting high-mix food manufacturing that fixed automation can't touch40, and Amazon's force-sensing Vulcan now stows ~75% of item types at human-comparable speed, in warehouses with 150% annual turnover and a leaked internal estimate of ~$8 billion a year in attrition cost.41
Agriculture sits just below the profitability line with the same shape: a third of marketed yield measured left in Californian fields for want of pickers, and H-2A guest-worker certifications quadrupling in a decade.42 Below, Vulcan, Amazon's first robot with a sense of touch.
Seed-stage companies are already working this long tail from very different angles, and each one teaches a different part of the stack.
Bucket Robotics
Bucket Robotics (YC S24) attacks the inspection half of the manufacturing story: a CAD file goes in, a defect detector comes out.43 Factory quality inspection is still largely a human squinting at parts, and the AI alternative has been bottlenecked by data for years: defects are rare so examples are scarce, and high-mix manufacturers re-tool so often that any hand-labelled photo dataset is stale before it's useful.
Bucket sidesteps the cold start by ingesting the part's 3D engineering model and synthetically rendering photorealistic images of it with simulated defects (burn marks, bumps, breaks, cracks, warps, finish flaws) under varied lighting, "before the part even hits the line".
A model deploys "in minutes" on existing cameras plus a cheap edge box (an NVIDIA Jetson Orin Nano), needs no manual labelling, classifies how a part failed so the tooling can be fixed, and keeps improving as real production images stream in.
The product is software-first (no new robots required), branded "The Trust Layer for Physical Production", and spans object classification, segmentation and robotic guidance as well as defect detection, the same synthetic-data and edge-AI moves powering humanoid and self-driving training, applied to inspection, with on-device inference chosen for latency and factory data privacy. Customers are enterprise manufacturers across automotive and industrial production plus, per TechCrunch, defence customers on an explicit dual-use path.44
OS3
The same high-mix logic runs past the factory gate into hospitality back-of-house, where OS3 (YC S26) is placing a much earlier and much more vertically integrated bet.45 The two-person San Francisco company (founded 2025) builds "semi-humanoid" robots for physical labour: YC's listing headline is "Intelligence and Machines for Physical Labor", while the company's own site (versioned v0.2, updated May 2026, very much build-in-public) goes with "Robots for the real world" and "Robotics, end to end".
The defining choice is owning both halves: they design the body in their own SF lab ("designed in our SF lab, not sourced") and train the models that drive it, arguing hardware and models must shape each other.
Their learning recipe is human-video-first, real-robots-last: pretrain policies on internet and egocentric first-person human video, add reinforcement learning, and spend scarce, expensive real-robot time only where nothing else works, with a biological framing that after deployment the robot keeps learning and "the body adapts". They're piloting in hospitality (hotels, restaurants, back-of-house) and post weekly deployment updates branded "the drop". There's no named product SKU and no pricing; a robots-as-a-service motion is a reasonable inference but not something the company has stated.
Founder Rishabh Chanana did graduate study in robotics and AI at UCSD, NLP research with the CSG Lab associated with Harvard Medical School and Georgia Tech, trained LLMs at ServiceNow, and holds a patent on an ML system; his YC bio says he decided at 13 to build a company called OS3 and built his first rover at 17.
Co-founder and CTO Chris Hailey is a USC computer-science graduate (Presidential Merit scholarship) and former Coinbase engineer on the asset-addition team, credited with enabling ~$19B in trading volume - and a 3:12 marathoner. Their YC partner is Gustaf Alstromer. They sit inside a visible YC cluster of humanoid and semi-humanoid bets (Piggy Robotics, K-Scale Labs, Twolabs), so read them as one runner in a crowded race rather than a singular signal.
Warehouse Automation
Vulcan is the visible tip of a bigger story, because warehouses are where the "automation only pays at scale" rule is being dismantled first. A warehouse receives goods, stores them, picks items to fill orders, and packs them for shipping - and picking is the expensive part, eating 50-65% of a manual warehouse's operating costs, largely because workers spend roughly half their shift walking to shelves.46
Decades of automation attacked this in layers. The oldest is fixed heavy infrastructure: automated storage-and-retrieval systems (rail-guided "stacker cranes" shuttling pallets and totes into tall racking) and shuttle bots riding rails inside the racking. Fast, but rigid, expensive, and bolted to one building's layout.
The newer layer is autonomous mobile robots (AMRs), free-roaming wheeled robots navigating by cameras and LiDAR. Amazon kicked this off by buying Kiva Systems in March 2012 for $775M, then its second-largest acquisition ever: Kiva "drives" slide under a shelf and carry the whole shelf to a stationary human picker ("goods-to-person"), and Amazon's fleet grew from ~10,000 robots in 2013 to 750,000 in 2023 to over a million by 2025.47
Everyone who can't buy Amazon's stack rents: Locus Robotics runs 2,000+ LocusBots across 350+ sites, passed five billion cumulative picks in 2025, and reached $180M ARR by mid-2026 (up from $165M at end-2025, after a $117M Series F in 2024), while 6 River Systems' Chuck robots, acquired by Shopify in 2019, now operate under Ocado Intelligent Automation.48
The market runs ~$8.2B in 2025, growing ~45% year on year, with 3PLs the single largest buyer segment at ~39% of spend.49
Notice what all of that automates: transport and storage. The hard, still-mostly-manual frontier is manipulation (a robot arm reliably grasping an arbitrary, never-seen SKU from a cluttered bin, or stacking mixed boxes into a dense pallet), because for most picking and packing a human hand is still cheaper than a robot arm.
Third-party logistics providers are the worst-served case: they sign one-to-three-year client contracts against wildly shifting SKU mixes, so a $250K+ fixed system tuned to one client's goods becomes a stranded asset when that contract churns, which is part of why McKinsey finds only ~20% of North American warehouses have adopted any automation at all.50
The economics have crisp thresholds: below ~500 orders a day, stay manual; a robotic pack cell (commonly $30-80K, with $100-150K at the high, integrated end) replacing three to six packers at $55-65K fully burdened each pays back in 8-12 months across two shifts; a ~$250K sorter saving ~$50K a year takes five years, viable only at thousands of orders a day.46
Hence the Robots-as-a-Service shift: vendors bundle hardware, software and maintenance into fixed monthly fees that scale up or down on 30-90 days' notice, cutting total cost of ownership by up to ~30% versus outright purchase.
So the mature part of this market is a procurement decision (Locus, Ocado/6 River, Zebra/Fetch or Geek+ for rented AMR fleets; Dematic, Exotec, AutoStore or Vanderlande if you really do have a five-plus-year stable flow worth $millions of fixed steel and a building redesign).
The frontier is learned manipulation, and its cautionary tale has already happened: Covariant, founded 2017 by Pieter Abbeel, Peter Chen and Rocky Duan, built RFM-1, a multimodal robotics foundation model for warehouse picking, and its best outcome was the Amazon reverse-acqui-hire: roughly $380M plus $20M in licensing for the founders and about a quarter of the staff.51
Amazon's own Sparrow item-picking arm and vendors like RightHand Robotics now push structured-bin pick rates above ~1,000 items an hour, but general grasping remains commercially unproven at scale: the sensible posture is a paid, limited-SKU pilot measuring real pick rate and reliability, never a core dependency, and for a founder or investor the defensible edge is manipulation generalisation and proprietary dexterity data, not another goods-to-person AMR.
The next three companies are frontier bets on that gap.
Ultra
Ultra (YC S24, New York, ~4 people) productises learned manipulation in the least romantic form factor available: the Operator (model OP1), a stationary dual-arm robot that rolls up to an existing workstation on a fixed base with locking casters, plugs into a standard 120V outlet (10A peak) plus a dedicated 10 Mbps Ethernet line, and installs "in hours, not weeks" with no integration work.52
From a 5×5 ft footprint it works a cell of up to 10×10 ft with zero-to-ten feet of vertical reach and up to ~10 lb of payload in a ~14-DoF dual-arm configuration. Nothing is explicitly programmed: RGB cameras feed onboard neural networks that output joint and gripper positions at ~10 Hz, and Ultra cites an autonomous picking policy trained on about two hours of teleoperation data from its research arms.
The Operator targets e-commerce order packing, sorting, kitting and returns for 3PLs and brands running their own fulfilment, runs up to 24 hours a day with no battery to swap, and sells deploy-first through a roughly three-month trial before scaling; Ultra says robots are "running in warehouses across America right now". Pricing isn't disclosed.
Remy AI
Remy AI (YC W26) started one notch more ambitious: a single versatile bimanual mobile robot for the small and mid-sized e-commerce 3PLs that legacy automation ignores, pitched at roughly half the cost of incumbent solutions - which Remy benchmarks at $100-150K per solution including install and servicing, by the company's own arithmetic.53
The go-to-market starts at packing stations and extends to picking, kitting and receiving, claiming generalisation across constantly changing SKUs without per-product reprogramming via pre-trained vision-plus-robotics models, simulation built from photos of the customer's site, domain randomisation and error recovery, rented rather than sold, with ROI "in weeks, not years" and deployment "in days, not months". The market context checks out: roughly 70,000 US 3PLs inside a ~$1.2 trillion global 3PL market growing ~8% a year, with McKinsey's ~20% automation figure above as the gap.
The revealing move came by mid-2026, when Remy's public site pivoted its framing almost entirely to the data layer: the headline is now "We capture how humans move through the world", foregrounding head-mounted egocentric capture and wearable tactile sensors that record synchronised vision, proprioception and contact forces from people doing real work (YC's tagline for them: "Documenting human dexterity"). That's the egocentric bet from The Frontier Map run as a company (the moat is dexterity data, not the metal), in the same current as Apple's EgoDex dataset.
AutoPallet Robotics
AutoPallet Robotics (YC S24, San Francisco, founded 2024, ~6 people) differentiates on form factor instead of learning alone: swarms of small, self-contained battery-powered robots that drive upside-down, held magnetically to an overhead steel panel grid, lowering a downward-facing vacuum gripper with its own cameras and sensors to pick cases and both build and break pallets packed densely beneath the work cell.54
The overhead superstructure is freestanding modular steel bolted to the floor of an existing warehouse (retrofit, not rebuild), against alternatives the company prices at $100-350M for a full automated-building rebuild or $300-500K per large fixed or mobile robot, in a market it sizes at over 30 billion cases moved manually each year in US warehouses at a $10B+ annual labour cost (all figures from AutoPallet's own launch).
Each robot carries separate quick-swap lithium-iron-phosphate batteries for the drive base and the vacuum pump and coordinates over a wireless mesh for instruction-sharing and collision avoidance; the swarm claims throughput of "tens of thousands of cases per shift", though The Robot Report notes no payload, case-weight, pallet-height or model names have been disclosed.55
It sells as Robots-as-a-Service on a design-partner basis with a claimed sub-12-month ROI, made its first public demo at Manifest 2026 in Las Vegas (February 2026), and lists Y Combinator, Aera VC, Long Journey, Pioneer Fund and Xpress Ventures as investors.
Targets are case picking and mixed-SKU palletising for 3PLs, grocery and foodservice distributors: the dense, awkward end of the long tail, attacked by rethinking the form factor rather than polishing another arm.
Already Inverted
For dangerous and inaccessible work, the economics inverted before the learning did. Sellafield is a £136 billion, hundred-year clean-up whose legacy silos the National Audit Office calls an "intolerable risk", a programme that's robotic or nothing.56 Fukushima's reactor core measured 530 Sv/h, enough to kill the purpose-built inspection robot in about two hours; against ~880 tonnes of melted fuel, retrieval so far totals under a gram.57
Mining shows the commercial version: the global autonomous haul-truck fleet grew 84% in a single year to 3,832 trucks, Komatsu commissioned its thousandth and has passed ten billion tonnes hauled autonomously, and Rio Tinto and Caterpillar both report zero injuries across their fleets' entire operating history alongside ~15% lower unit costs, safety and economics pointing the same way.58 Inspection already wins on cost alone: in one vendor case study, scaffolding was 93% of a tank-inspection bill and the robot cut the job by ~96%, which is how Gecko Robotics (YC W16) became a $1.25 billion company.59
The backdrop: ~5,070 US workplace deaths a year and over $100 billion in workers'-compensation costs60, with commercial diving running ~40x the average fatality rate, the shortage behind the hull-cleaning and subsea-inspection fleets (Neptune Robotics, Hullbot) that already sell robotic outcomes per job.61 Below: an autonomous Komatsu 930E hauling at Nevada Gold Mines' Cortez site.
Salem Robotics
The nuclear wedge already has a young specimen. Salem Robotics (YC S26, Austin, two people) builds nuclear-native autonomy software that turns robots a facility already owns (chiefly Boston Dynamics' Spot quadruped) into autonomous radiological surveyors, with the broader tagline that their software turns "any robot we deploy into an everyday independent worker at your facility".62
Today a radiation-protection technician physically walks survey routes carrying a handheld detector, marks readings on paper maps and transcribes them later; legally mandated dose limits cap how much exposure any one person can accumulate, which caps how often surveys can happen at all, while the paper workflow fragments records in an audit-driven industry.
Salem's platform runs a plan-execute-report loop: an operator plans a mission on a dashboard, the robot walks the route carrying a radiation detector (a "frisker") collecting timestamped, spatially linked measurements, and the software packages them into auditable digital compliance records.
The product is deliberately hardware-agnostic, sold either as software on the customer's robot or as a ready-made system, with short paid one-to-two-week validation engagements as the entry point (a 77-second demo video sits on the site). The roadmap runs through oil and gas, CBRN response and space infrastructure operations toward "the autonomous operating system for hazardous site robotics", their phrase verbatim.
Pricing is undisclosed, and the DOE-environment relationships, ties to five commercial nuclear plants and early pilot scoping with Texas facilities are the company's own statements. Size the market soberly too: independent 2026 research puts hazardous-environment robots at roughly $4.39B growing ~9.5% a year, a long way short of the $17B+ figures sometimes attached to the space.63
Urban Mining
Recycling deserves its own entry because it compounds two theses. The labour case is stark - materials-recovery facilities run ~200% annualised sorter turnover in one of America's most dangerous occupations, and AMP's AI sorters pick at 80-120 items a minute against a human's 30-40 (the company's own benchmark). On that arithmetic AMP has pivoted from selling robots to owning and operating entire facilities.64
An AMP vision system scanning the stream in a materials-recovery facility. Photo: AMP But the deeper case is what's in the stream. Circuit boards run 200-350 grams of gold per tonne against 1-10 g/t in mined ore: the waste is often the richer deposit.65
Copper demand is projected to nearly double from 25 to ~50 million tonnes by 2035, and even S&P Global's high-ambition scenario (all-time-high recycling rates included) still leaves a 1.6-million-tonne annual deficit; the IEA reads the current mine pipeline as a ~30% supply shortfall by 2035.66
Battery-anode graphite is barely recovered at all today, a few per cent at most, since recyclers chase the cathode metals and burn the anode; the IEA expects graphite demand to nearly quadruple by 2040.67
The light rare earths neodymium and praseodymium, the NdFeB magnet metals, close a loop that belongs in this post. Adamas Intelligence projects NdPr oxide shortfalls of 68,000 tonnes a year by 2035 and expects robotics itself to become the single largest NdFeB demand driver by 2040, at roughly 4 kg of magnets per humanoid. Robots picking e-waste would be feeding the supply chain that builds robots.68
Platinum-group metals show the mature endpoint: recycling already supplies roughly a fifth to a third of supply depending on the metal, with PEM electrolysers arriving as a new demand vector as catalytic converters decline.69
Solve sorting at human dexterity and waste streams become domestic mines with traceable provenance. The "urban mining" framing stops being a metaphor.
Operators Win
Across every vertical above, the same commercial lesson keeps repeating, and it contradicts the instinct to sell robots. The vendor model kept breaking: Covariant's founders and a quarter of its staff were absorbed into Amazon via licence deal, Berkshire Grey went from a $2.7 billion SPAC to a $375 million take-private, Nimble pivoted from selling picking robots to being a 3PL, and JBS bought its deboning-robot supplier outright.70
The winners run operations and sell outcomes: AMP owns the sorting facility, Chef sells plated servings, Hadrian runs the machine shop, Agility rents labour by the tote at GXO71. Own the operation, bill the outcome, keep a teleoperator in the loop as both insurance policy and training-data source - a flywheel The Frontier Map examines properly, run as a business model with revenue arriving before autonomy does.
There's one more operator business hiding in this section, and it's the one I keep circling. The sim-to-real path coming up in The Full Stack (digital-twin simulators reconstructed from ordinary phone photos) inverts into a data company: walk real construction sites, warehouses, and plant rooms with a scanner, reconstruct them, and sell the reconstructions as training environments.
Robotics gyms built from real clutter and real long-tail mess, the distribution the labs can't fabricate and the diversity pi0.5's ablations say the models starve without. You'd be selling the scarcest input in the field before owning a single robot.
A fair objection: site scanning is old. Matterport has been walking buildings since 2011, construction laser scanning is older still, and photogrammetry apps reached phones over a decade ago. What changed isn't the sensor but the output and the buyer.
Classical photogrammetry produced meshes for humans to measure: geometrically decent, visually dead, hopeless with glass, scaffolding poles and wet concrete - and useless for training camera policies, which is the visual sim-to-real gap that domain randomisation exists to paper over. 3D Gaussian Splatting collapsed that gap in 2023: a few minutes of ordinary phone video becomes a scene whose novel views are near-indistinguishable from real footage, trained in minutes on a consumer GPU and rendered in real time.72
Meanwhile the buyer changed from a project manager paying to measure progress into a training pipeline paying for distribution, and the ingestion path got built: Isaac Sim now renders neural reconstructions natively through NVIDIA's NuRec libraries, and Stanford's digital-cousins pipeline turns a single photo of a real scene into fully interactive, physics-ready simulation variants automatically.73
What still separates a scan from a gym is also what keeps this a business rather than a script. A raw splat is static and semantically dumb: it has no object boundaries and no articulation (the excavator arm is fused to the scene), carries no mass or friction, and offers nothing to grasp. Turning captures into segmented, physics-annotated, resettable training environments is where the defensible pipeline lives - and why incumbents sitting on millions of human-viewing meshes, captured under licences never written for model training, don't automatically win.
Honest Numbers
A calibration note, because the headline TAMs deserve scepticism. Figure's Brett Adcock quotes "$30-40 trillion", that's the global wage bill for physical labour, not a revenue forecast; the sober bank estimates are Citi's ~$7 trillion by 2050 and Morgan Stanley's ~$5 trillion with a billion humanoids.74
And the sharpest critique cuts deeper than sizing: Aghion, Jones and Jones point out that automating a task collapses its price, so the wage bill being replaced is an upper bound that shrinks as you capture it - most of the value may arrive as deflation spread across everyone rather than revenue concentrated in robot companies.75 I genuinely don't know which way that resolves, and neither does anyone selling you a TAM slide.
The Analogy with LLMs
Every claim in the section above leans on one analogy. The core bet of physical AI, back in Two Eras, is explicitly "GPT got smart on the internet, so a robot foundation model should get smart on demonstration data". If the field's entire thesis is an analogy, the source material is evidence: the LLM ecosystem ran its full cycle from capability to tooling boom to consolidation between 2022 and 2026, in public, with dated funding rounds and visible corpses.
The analogy has stopped being subtext. NVIDIA's Jim Fan pitches the company's robotics roadmap as "the great parallel": world modelling standing in for pretraining (simulate), action fine-tuning for supervised fine-tuning (align), physical RL for reasoning RL (surpass) - the LLM playbook copied line by line, on his own account.19
The LLM ecosystem's history is the best forecasting instrument available for what gets built next in robotics, so this section replays it properly on its own terms, then maps it onto the robotics stack. If you built software through those years you lived the first half; skim Four Years of LLMs and jump to Reading the Clock. And hold the whole thing loosely: a naive forecast is only useful with its error bars visible, which is why the section ends with where the analogy breaks.
Four Years of LLMs
The one-line version of the LLM story: capability moved up the stack, and each layer only worked because the one below it had matured first. 2022-23 was about the base model, 2024 about context and modality, 2025 about reasoning and post-training, 2026 about agents doing long-horizon real work. Every capability jump opened a tooling gap, a cohort of startups rushed it, the labs absorbed the easy 80%, and the survivors settled into infrastructure. Here it is era by era, with the tooling arc attached to each, because the tooling arc is what transfers to robotics.
Instruction Following
The transformer was five years old and GPT-3 two, but ChatGPT (30 November 2022) was the product moment: RLHF turned a next-token predictor into something that did what you asked. GPT-4 (14 March 2023) made it feel reliable and added vision. Meanwhile the LLaMA weights leaked to 4chan on 3 March 2023, and within days Stanford's Alpaca had instruction-tuned the 7B model for roughly $600 - the true start of the open-weights movement.76 Meta then made it official with Llama 2 (July 2023), reasoning that the marginal cost of releasing was near zero once the leak had happened.77
The tooling rushed in immediately. LangChain had launched days before ChatGPT and became the default "how do I build with this" repository, taking a $10M seed from Benchmark in April 2023; GPT Index became LlamaIndex.78 The same month, the vector-database funding blitz peaked: Pinecone raised $100M at a $750M valuation, Weaviate $50M, Chroma $18M, all inside April 2023, on the thesis that models had no memory and tiny context windows, so retrieval was the mechanism.79
The era also produced the false start that matters most for reading robotics. AutoGPT and BabyAGI (March-April 2023) wrapped GPT-4 in a self-prompting loop, hit 100,000 GitHub stars within weeks promising autonomous AI employees, and delivered infinite loops, runaway API bills, and no completed tasks.80 The idea was right; the models weren't ready. Keep that one in mind - it comes back when we date the humanoid wave.
Context and Modality
Claude jumped to 100k tokens in May 2023 and 200k by that November; GPT-4 Turbo hit 128k the same month; Gemini 1.5 Pro reached a million in February 2024.81 Long context quietly killed a whole tier of retrieval hacks, the chunking-and-stuffing machinery that existed only because windows were 4k. Multimodality went from bolted-on vision encoders to natively omni models: GPT-4o (May 2024) trained one network across text, vision and audio with ~320ms average voice latency. And Claude 3 (March 2024) established the small/medium/large family tiering that every lab now copies.82
This is also when the correction hit the tooling. The "you don't need LangChain, it's a for-loop and string formatting" backlash hardened through mid-2023, and LangChain answered by pivoting to LangGraph (stateful graph orchestration, January 2024) and monetising LangSmith, its observability product: an implicit admission that the original abstraction didn't fit production.83
Naive RAG matured into hybrid retrieval and reranking pipelines, and the gap was measurable: on Meta's CRAG benchmark, straightforward RAG answered about 44% of factual questions while the best industry pipelines managed about 63% without hallucination; Microsoft open-sourced GraphRAG in July 2024.84 Meanwhile pgvector and MongoDB quietly ate the dedicated vector databases from below - Retool's 2024 State of AI survey put them at ~21% adoption each, ahead of Pinecone.85 The "is it a product or a feature?" question resolved itself. Mostly a feature.
Observability was born in this window: LangSmith, Langfuse (open source, self-hostable), Helicone (YC W23), Arize Phoenix (OpenTelemetry-native) and Braintrust (eval-first, founded 2023) all launched into a gap that barely existed in 2022 and was table stakes by 2025. LLM-as-judge, using a strong model to grade outputs against a rubric, became the standard eval primitive in 2024. The timing matters later: observability arrived after the capability, and took about two years to become mandatory.
The Efficiency Track
In parallel with all of it ran the economics. Mixture-of-Experts became the default architecture for pure inference reasons - grow total parameters, keep active parameters small. Mixtral 8x7B (December 2023) was the open proof, and DeepSeek-V3 (December 2024) was the shock: 671B total parameters, 37B active, trained for a reported $5.6M.86 By 2025 essentially every frontier model was MoE. The local track ran alongside: llama.cpp plus 4-bit quantisation put open models on laptops, Ollama and LM Studio productised it, and Phi-3 and Gemma 2 proved that 2-14B models could punch far above their weight through synthetic-data curation and distillation.
The compound result was a price collapse. GPT-4 launched at ~$60 per million output tokens; by 2026 frontier output sits around $15, the cheapest capable models are ~300x cheaper than GPT-4 at launch, and Epoch AI measured inference prices for a fixed capability level falling a median of roughly 50x per year.87 The collapse didn't shrink the market - it repriced everything above it, and shifted spend from chat (hundreds of tokens per turn) to agents (millions of tokens per completed task).
Serving is the layer that stayed independent through all of this. vLLM (June 2023, out of Berkeley) became universal open-source plumbing.88 Together and Fireworks scaled past $1B in annualised inference revenue by 2026; Groq and Cerebras specialised in speed until NVIDIA neutralised Groq in a ~$20B assets-and-licence deal in December 2025, its largest on record.89 OpenRouter became the neutral routing switchboard, at ~25 trillion tokens a week by 2026 and a $113M Series B at a $1.3B valuation led by Alphabet's CapitalG.90
Why did serving survive? None of the labs wanted to run everyone's open-model inference. Fine-tuning went the other way: better base models plus prompting plus RAG solved most problems without training, the standalone platforms got absorbed (Predibase into Rubrik, OpenPipe into CoreWeave), and fine-tuning lives on as a feature of the serving layer, mostly for distillation.91
Reasoning
o1-preview (12 September 2024) opened a new scaling axis entirely: spend compute at inference time on a hidden chain of thought. DeepSeek-R1 (20 January 2025) open-sourced the recipe under MIT with visible thinking traces, showing that pure reinforcement learning (GRPO) could induce reasoning almost without supervised fine-tuning - and days later Nvidia shed $589B in a single session, the largest one-day single-stock loss in US history, as the market briefly concluded that compute no longer mattered.92
That reading faded within weeks; the US-China open-weights rivalry it symbolised compounded instead. Claude 3.7 Sonnet (February 2025) then made reasoning a dial on one model rather than a separate model, and the hybrid pattern won.93
The deeper shift underneath: post-training became the main event. RLHF gave way to DPO, then to RLVR (reinforcement learning with verifiable rewards, where correctness on maths and code is checked automatically with no human rater), then to agentic RL on real tool-use environments. Most of a 2025 model's usable capability comes from post-training, not pretraining. Remember that when we get to what breaks the robotics data bottleneck.
Agents and Plumbing
The 2023 AutoGPT dream finally worked in 2025, and the reason is the calibration lesson of the whole story: it had been a capability problem, not an architecture problem. Three unlocks fixed the same loop that failed in 2023 - reliable function calling, long context to hold task state, and RL-trained reasoning so models plan, check their own work, and stop looping.
Anthropic's computer use (October 2024) added the screen-operation primitive, Claude Code (February 2025) and OpenAI's Codex turned agentic coding into the reference product category, and Opus-4-class models sustained thousands of autonomous steps.94 Devin (March 2024) was the premature version, viral demo and heavy scepticism attached; by 2026 the category it named was simply real.
The framing shifted with the capability. "Agent framework" gave way to "agent harness": not a graph library but the streaming loop, permission-governed tool dispatch and context management wrapped around the model. Prompt engineering was demoted to a subset of context engineering, a term Andrej Karpathy and Tobi Lütke popularised in summer 2025 and Anthropic formalised that September.95 Memory became its own contested layer (MemGPT/Letta, Zep, mem0), with the harnesses absorbing the common case, and conventions like CLAUDE.md and AGENTS.md became the de-facto way to hand an agent site-specific instructions.
The protocol story is the cleanest arc of all. OpenAI's function-calling spec (June 2023) became the universal input format.96 MCP launched in November 2024 as Anthropic's open standard for connecting models to tools; the inflection was OpenAI adopting it in March 2025, after which monthly downloads went from ~2M to ~97M and the protocol was donated to the Linux Foundation's newly formed Agentic AI Foundation in December 2025.97
MCP is the clearest "became plumbing, not a fad" outcome in the whole history: the industry collectively refusing to let one lab own the interface. Google's A2A agent-to-agent protocol is real but more niche, and agent payments is the standards land-grab still in progress: Coinbase's x402 (May 2025), then Google's AP2 and Stripe and OpenAI's ACP within two weeks of each other in September 2025.98
Around the agents grew a runtime economy: agents that write code need somewhere safe to run it, which is E2B, Modal (a $355M Series C at $4.65B, May 2026) and Daytona for sandboxes, Browserbase for browser infrastructure.99 Another layer the labs would rather not own, so it stays independent.
The Tooling Lifecycle
Run the five eras together and every tooling category follows the same arc. A capability lands, opening a gap between "the model can do X" and "I can ship X in production". Startups and open source rush the gap; a category name gets coined and a whole cohort gets funded inside a month (April 2023, vector databases). The backlash arrives: abstractions turn out leaky, demos don't finish real tasks, and someone asks whether this is a product or a feature.
Then the platform absorbs the easy 80% - function calling killed the output-parsing libraries, harnesses swallowed orchestration and memory, Postgres swallowed vectors, and the labs acqui-hired the eval and fine-tuning talent (Humanloop into Anthropic, August 2025).100 What survives becomes boring infrastructure: the capital-intensive, cross-vendor or otherwise hard parts the labs don't want to run (serving, observability, sandboxes, protocols) get neutral governance, raise real Series Bs and Cs, and stop being optional.
The figure below lets you run that arc category by category. Click through the LLM rows and watch where each run ends (vector databases and fine-tuning never reach step five), then switch to the robotics rows and see how early on the identical track most of them still sit, with one conspicuous exception that skipped the queue.
That gives you the tell this whole section runs on: ask whether the model labs would rather own it or outsource it. If owning it improves their core product and they can build it, they absorb it. If it's undifferentiated heavy lifting they would rather not operate, it survives independently - and eventually gets boring, which is the same thing as winning.
The final ledger, for calibration. Compounded: instruction tuning and RLHF, open weights (LLaMA leak → Llama 2 → DeepSeek, Qwen and Kimi), long context, native multimodality, MoE, local and quantised models, test-time-compute reasoning and RLVR, post-training as the main event, agentic coding, harnesses and context engineering, MCP, the price collapse and the chat-to-agent spend shift, observability and evals, serving infrastructure, sandboxes.
Hyped then faded, or premature: AutoGPT and BabyAGI (right idea, wrong year), the Assistants API, Sora-the-product versus its demos, "compute no longer matters", Llama 4 as an open-weights leader, naive single-shot RAG, the pure vector-database-as-product thesis, fine-tuning platforms, most no-code agent builders, and every "this benchmark equals AGI" claim.
Reading the Clock
Now point that history at robotics and ask where each layer sits on the LLM clock. On the model axis, robotics in mid-2026 is roughly LLMs in 2020-2022, the GPT-2/GPT-3 era. pi0, OpenVLA and GR00T are capable base models with impressive demos and fragile generalisation: pi0.5's own ablations, the 94% → 31% collapse from How Robots are Learning, are the robotics equivalent of GPT-3's brittleness before RLHF.14 The ChatGPT moment, the product that makes the capability legible to everyone, hasn't happened, and every VLA lab is racing to be the one that has it.
The AutoGPT calibration applies directly to the humanoid question too: the 2024-26 humanoid theatre is a mid-cycle false start, the right idea arriving before the capability, and the LLM precedent says the category gets vindicated when the capability lands - mostly not by the companies that named it.
But the clock runs differently per layer, and this is the strange part: robotics tooling is arriving before its model breakthrough, not after. Foxglove, Formant and MCAP all predate any working robot foundation model, because the autonomous-vehicle industry (Cruise, Waymo) pre-built the observability stack a decade early and its diaspora productised it - you met the ex-Cruise founders in The Tooling Companies.
In LLM-land, observability was nascent after ChatGPT; in robotics it's mature before the ChatGPT moment. The tooling layer is early relative to its own gold rush, which the lifecycle says is fine for the categories the labs will never want (observability stayed independent in LLM-land too) and brutal for anything the eventual model labs decide to absorb.
The full mapping, layer by layer. The rest of this section walks the rows that deserve more than a cell.
| LLM-era layer | Robotics analogue | Status, July 2026 | Naive forecast |
|---|---|---|---|
| Pretraining corpus (the free internet) | Demonstration data (teleop, egocentric video) | The master bottleneck; no free corpus exists | The defining disanalogy - see below |
| Scale AI / the RLHF labelling workforce | Teleop operators (XDOF, Aktoria, offshore ops) | Load-bearing, deliberately understated | Persists far longer than in LLMs: it is both safety net and data factory |
| Open weights (Llama → DeepSeek/Qwen) | openpi, OpenVLA, GR00T weights; Unitree/AUBO hardware | Already diffusing | A Chinese open VLA at frontier parity is robotics' DeepSeek moment |
| Observability (LangSmith, Langfuse, Braintrust) | Foxglove/MCAP, Ember, Formant | Mature early (the AV inheritance) | Stays independent per the absorption test; consolidates |
| LLM-as-judge / evals | VLM-as-verifier (Instance Labs), RoboArena | Nascent - the gap LLMs had in 2023 | The next table-stakes layer; "evals are the new unit tests" replays |
| Orchestration frameworks (the LangChain arc) | ROS 2, LeRobot | ROS already ran the full arc a decade early | Expect a LeRobot backlash-and-reposition; harnesses beat frameworks |
| Agent harness (Claude Code) | The autonomy stack: safety supervisor, watchdogs, fallback controllers | Exists as safety engineering, not a named product | "The autonomy harness" becomes a category you buy |
| Protocols (function calling, MCP) | MCAP, the LeRobot dataset format, URDF | Fragmented; MCAP is the closest MCP-like win | A neutral, foundation-governed embodiment interface; owning it loses |
| Serving infrastructure (vLLM, Together, OpenRouter) | On-robot edge inference plus training clouds | Barely exists as a category | Distillation-to-edge toolchains and policy serving become real companies |
| MoE / router economics | System 1 / System 2 splits (GR00T's dual system, pi0.5's hierarchy) | Already the standard architecture | Converges the same way for the same reason: latency and cost |
| Sandboxes (E2B, Modal) plus RL environments | Simulation (Isaac, Genesis, MuJoCo), digital twins, splat gyms | The live frontier | The strongest analogy in the map - see below |
| Context engineering / memory (mem0, Zep) | Maps, semantic scene memory, fleet-shared experience | Mostly unbuilt | A named discipline by ~2028; harnesses absorb the common case |
| Fine-tuning platforms (faded) | Per-site policy-tuning services | Being built now | Fades the same way once few-shot adaptation works; becomes a feature |
| Agent payments / auth | Insurance, liability, compliance records, CE marking | Fragmented, jurisdictional | The trust layer: a standards land-grab, then neutral plumbing |
| The AutoGPT false start → 2025 vindication | The 2024-26 humanoid theatre | Mid false start | Vindicated when capability lands - but mostly not by the 2023-cohort names |
No Free Corpus
Everything else in the map is downstream of this row. LLMs got their pretraining corpus for free and paid humans only at the margin, for RLHF and labelling; robotics has to manufacture its corpus at market rates. That inverts the ecosystem's build order. In LLMs, Scale AI became critical after the models worked; in robotics, the data companies (Asimov, Human Archive, Hub and the rest of The Data Companies) are being funded before the models work.
It also predicts margin compression right where The Data Bottleneck already calls it: a dozen near-identical YC startups selling egocentric hours are the April-2023 vector-database cohort of this wave. Right about the gap, funded simultaneously, mostly commoditised within two years, with one or two survivors repositioning up the stack.
The LLM precedent for what breaks the bottleneck is worth staring at. What eventually broke it was synthetic data plus RLVR, models generating their own training signal against automatic verifiers, rather than any amount of extra collected human text. The robotics equivalent is policies improving against automated success detection, in simulation and in fenced real deployments - which is why evals matter so much here: they're the mechanism by which the data bottleneck eventually dissolves.
And teleop behind the curtain is robotics' RLHF workforce, with one structural difference that changes the forecast: unlike LLM labelling, teleop doubles as the safety fallback that makes deployments commercially viable before autonomy works. Expect it to persist much longer than dignity would suggest.
Gyms and Sandboxes
Two runtime categories stayed independent in LLM-land and got big fast: sandboxes (E2B and Modal, somewhere safe for agent code to run) and, through 2025-26, RL environments, the curated task worlds agents are post-trained against, which became a funding frenzy of their own. Robotics fuses both into a single layer: simulation.
Isaac, Genesis and MuJoCo are the sandbox; digital twins and splat-reconstructed scenes are the environments. The absorption test says the labs won't want to own everyone's environments - NVIDIA wants to sell the substrate, the VLA labs want to consume environments, and the environment content, the diverse real-world long-tail scenes, is what nobody upstream can fabricate.
That lands directly on the scan-to-gym operator thesis from Operators Win. In LLM terms, walking real sites with a scanner and selling reconstructions as training environments is being an RL-environments company in 2024: slightly before the rush, positioned on the one input the labs can't generate internally. The precedent also says how it plays out - the category gets hot, then crowded, and the durable version sells curated, physics-annotated, resettable environments rather than raw captures. The Braintrust of gyms, not the Chroma.
NVIDIA is already publicly betting this layer, and the arithmetic explains why. The LLM labs are spending serious budget on millions of RL coding environments, and Fan argues robotics needs the equivalent; his version of the problem is that an environment made the old way is a real robot in a real cell, so a million environments would mean a million robots.19
His escape ladder starts with RL directly on real hardware for narrow tasks: his lab drives GPU-tray assembly to near-100% success, then lets the robots run for hours. The middle rung is phone-scanned scenes rebuilt as interactive simulation and multiplied as digital cousins, a loop he calls "real-to-sim-to-real" (the sim-real cycling idea has pre-splat history101). The top rung is neural world models as the environment factory itself, DreamDojo from the middleware layer below.19 His compression of the bet is an equation his employer must adore: compute = environments = data.
Evals Replay
Evals and Reliability already names Instance Labs as robotics' analogue to LLM-as-judge; the LLM timeline lets you date the replay. LLM observability went from nascent to table stakes in about 24 months once agents shipped, and the eval-native company (Braintrust) outlasted the general loggers because the thing customers paid for was the score rather than the trace. Robotics evals in 2026 look like LLM evals in 2023: one notable startup, no benchmarks that survive contact between labs, a reproducibility crisis, and human labelling that doesn't scale.
So the forecast has three parts. Per-rollout automated success detection becomes non-negotiable for anyone shipping policies by the time fleet deployments are normal. The curation half (flawed demonstrations actively poison policies, the garbage-in property from Evals and Reliability) becomes its own product, the way dataset-quality tooling did in LLM-land. And the winner is eval-native rather than logging-native: Foxglove owning traces while someone else owns verdicts is the LangSmith/Braintrust split, rerun.
Efficiency Transfers
MoE won LLMs because you can't serve 1.8 trillion dense parameters economically. Robotics has the same constraint tightened by physics: a 55B RT-2-X needs a multi-GPU server, but control loops run at 50Hz on a battery-powered body, and latency rules out the cloud.
The field already converged on the same answer under the same pressure - big slow vision-language reasoning bolted to a small fast action expert (pi0's 315M expert on a ~3B backbone, GR00T's dual system, both from Inside the Models) is the robotics version of the router/MoE pattern. Capacity where you can afford it, active compute where you can't.
The forecast extension is the llama.cpp/Ollama moment: a standard toolchain for quantising and distilling VLAs onto Jetson-class edge hardware. It's unbuilt, everyone currently hand-rolls it, and it's obviously coming; "distillation to edge" becomes a product category the way it became a feature of LLM serving.
Expect the price collapse too. Unitree's ~$16k G1 against $80k+ research arms is already the hardware half;102 the model half arrives when open VLAs are good enough that the policy is free and only data and integration are paid for. The LLM lesson is that the collapse doesn't shrink the market - it moves the bill from cost-per-unit to cost-per-completed-task, which in robotics already has a name: Robots-as-a-Service. The winners-run-operations pattern from Operators Win (AMP, Chef, Agility at GXO) is the robotics version of the chat-to-agent spend shift: the bill attaches to the completed task rather than the machine.
The Scheduled Shock
The US-China dynamic transfers with the roles part-reversed. In LLMs, China caught up on models under export controls and shocked the market from below on cost. In robotics, China already dominates the hardware layer (Unitree, AUBO, the component supply chain, the give-away humanoid promotions) and the open question is the model layer.
The naive forecast writes itself: a Chinese lab releases an open-weights VLA at frontier parity, trained on data from the world's densest manufacturing base, which is where the demonstration data lives, and the Western labs' closed-model valuations reprice overnight the way Nvidia did on 27 January 2025. Physical Intelligence at ~$11B on no commercial product is the position most exposed to that event;12 openpi reads partly as a hedge against it - commoditise your own layer before someone else does, which is the Llama 2 logic.
The ROS Lesson
Robotics ran the LangChain arc before LangChain existed. ROS was the beloved-then-resented glue framework ("you don't need ROS" is a genre of blog post), survived its backlash by becoming boring plumbing, and ROS 2 plus MCAP is now the commodity substrate the middleware layer files under table stakes. LeRobot is currently in its honeymoon phase, holding the Hugging-Face-transformers-of-robotics position, and the precedent predicts its backlash-and-reposition around the time serious production deployments outgrow research ergonomics. I say that as someone who's spent this entire post praising it.
Wigglesworth, the Amplibotics founder from How Robots are Learning, runs the arc one step further: his bet is that VLAs eventually consume ROS's job outright. Collision avoidance, object recognition and trajectory planning fall out of a policy with vision in the loop, so "basically everything ROS can do will just be done by a prompt in a VLA", on a timescale he puts at years and years.21
He isn't waiting to find out. His own stack skips ROS because it bottlenecked their high-frequency data recording, and he notes that today's LLMs write ROS much worse than plain Python, because the corpus holds so much less of it. If he's right, this is the absorption test reaching one layer deeper than this section takes it.
The more valuable transfer is the harness. LLM-land discovered in 2025 that the sellable product was the harness: the loop, the permissions, the context management wrapped around the model. Robotics already builds harnesses without using the name: the safety supervisor in Rust, the 200ms command-staleness watchdog, the fallback controller that balances a humanoid instead of cutting power, all from Teleop Behind the Curtain.103104
The forecast is that this gets named, productised and sold: "the autonomy harness" as a layer you buy rather than rebuild, absorbing policy orchestration, memory and recovery logic the way Claude Code absorbed orchestration and context management. The CLAUDE.md convention even suggests its robotics analogue - a per-site deployment context file, the machine-readable version of what integrators currently encode by hand. Miru's config layer is one slice of it; the full harness is unclaimed.
Neutral Plumbing
Function calling and MCP predict the shape of robotics interop. The pieces exist: URDF for embodiment description (old, creaking), MCAP for logging (already the MCP-like win: created by one company, donated, now the neutral default), and the LeRobot dataset format for demonstrations, the de-facto standard the Flywheel dataset gets criticised for not using.35
But there's no standard interface between a policy and a body: no common way to describe an embodiment's action space, query a skill, or hand a task between systems. The MCP lesson is precise about how this ends. The standard that wins will be launched open by one lab, adopted by its biggest rival within months, and donated to neutral governance within two years - and the attempt to own it proprietarily, the Assistants API of robotics, will lose to it.
The Absorption Test
Run every robotics tooling category through the question the LLM era distilled: would the eventual model labs rather own it or outsource it?
Absorbed, on this reading - sold to a lab or platform, or rebuilt internally: policy orchestration and memory (into the harness); per-site fine-tuning services (into few-shot adaptation, the way LLM fine-tuning platforms went); simple data collection (the labs already run their own teleop floors); and eval-lite, since basic success detection will ship with the model API. Covariant into Amazon was this pattern running early.51
Stays independent: fleet observability and data platforms, cross-vendor and capital-intensive (the Foxglove bet); simulation substrate and environment content, which labs consume but won't produce; edge inference and distillation toolchains (the vLLM/Together slot); teleoperation-as-a-service at scale, because labour operations are the last thing a lab wants to run; protocols, neutral by construction; and compliance and audit infrastructure - jurisdictional, boring, mandatory, which is Salem's lane.
Fades: undifferentiated egocentric-data brokers (the vector-database cohort of this wave); no-code robot-behaviour builders, already appearing and likely to churn the way the no-code agent builders did; and thin policies on rented hardware without a data moat, this field's ChatGPT wrapper.
The Gap List
Read the LLM stack against the robotics stack and the most interesting objects are the slots that are proven categories in one and empty in the other. Eight, by my count:
- The Braintrust of robotics. Eval-native, per-rollout automated success detection plus demonstration-quality curation, sold as the score layer above Foxglove's trace layer. Instance Labs is the seed; the category is barely contested.
- The Ollama of VLAs. A standard quantise-distil-deploy toolchain from lab checkpoint to Jetson-class edge hardware. Everyone currently hand-rolls this.
- The OpenRouter of policies. Once multiple competent open VLAs exist: neutral routing, benchmarking and fallback across policy providers per task and embodiment. Premature today, obvious at DeepSeek-moment-plus-one-year.
- The RL-environments industry. Curated, physics-annotated, resettable training worlds built from real-scene captures - the scan-to-gym thesis from Operators Win, which the LLM timeline dates as slightly early rather than wrong.
- The MCP of embodiment. A neutral skill and action-space interface between policies and bodies. Whoever launches it open and gets one big rival to adopt it defines the field's plumbing.
- The mem0 of fleets. Persistent semantic scene memory and fleet-shared experience - what every robot in the fleet learned last night, portable across bodies. Mostly unbuilt.
- The autonomy harness as product. Safety supervisor, watchdogs, recovery behaviours, per-site context files, sold as the wrapper every policy deployment needs. Miru is one slice (config); the full harness is unclaimed.
- The trust and payments layer. Insurance-grade audit records, liability attribution, and eventually machine-to-machine payment for robotic work: Salem's compliance records generalised, plus the AP2/x402 analogue nobody has started.
Where It Breaks
All of the above is a naive forecast, and it's only worth carrying if the error bars travel with it. Five places the analogy structurally fails.
No free corpus is the big one, covered above: it makes robotics' data economics permanently worse and its data companies structurally more important than Scale AI ever was. Iteration speed compounds it. LLM tooling could ship weekly because failure was a bad completion; robot failure bends metal and hurts people, so every loop in the lifecycle runs slower, and the regulatory layer (CE marking, the 2027 EU Machinery Regulation from Regulatory Friction) is a first-class constraint rather than an afterthought.105
The edge is mandatory too. LLMs could centralise inference in datacentres and amortise; a 50Hz control loop on a battery can't, so the local-and-quantised track that was optional in LLM-land is the main track here. Hardware capex and working capital change who can afford to play at all: a tooling startup in LLM-land needed a laptop, while several robotics layers need fleets, sites and insurance.
And embodiment fragmentation taxes everything. LLMs converged on one interface, text in and text out, which made every tool horizontal; robots have wildly different bodies, an N×M problem that gap five on the list above exists to collapse.
Against those error bars, weigh the bull case at its most quotable. Fan closes his talk with a technology-tree endgame: a physical Turing test (watching a task done, you can't tell human labour from robot labour; he puts it two to three years out), then a "physical API" (fleets configured and orchestrated like any other software), then physical auto-research, robots designing and building their own next iteration.
He claims "95% certainty" the tree completes by 2040, on the arithmetic that 2026 sits midway between AlexNet in 2012 and 2040 and that the second half runs faster.19 Treat it the way Honest Numbers treats TAM slides: as information about the direction of travel, not the arrival time.
The compressed forecast, one sentence: robotics in 2026 is LLMs in 2021 on models, 2023 on data-company hype, and 2025 on observability - and the money in the gap list is mostly in the layers the LLM era proved the labs will never want to own.
The Full Robotics Stack
Those learned policies have to run on something. A modern robot is a computer with a body, and the layers below run from metal up to intelligence.
Actuators & hardware
An actuator is a motor plus (usually) a gearbox that moves a joint. The number of independently-moving joints is the robot's degrees of freedom (DoF) - a human-equivalent arm-plus-wrist is 6-DoF. Cheap "smart servos" like the ROBOTIS Dynamixel and Feetech STS3215 families (motors that report their own position) are what made low-cost research arms possible.106
Counting Degrees of Freedom
Hold your arm out and count what it can move. The standard biomechanical model gives the human arm seven degrees of freedom once you freeze the torso and ignore the fingers: three at the shoulder (flexion/extension, abduction/adduction, internal/external rotation), one at the elbow (flexion/extension), one in the forearm (pronation/supination, the radius rolling over the ulna), and two at the wrist (flexion/extension plus radial/ulnar deviation).107 Sum them and you get .
Seven matters because a rigid body in 3D space needs only six numbers to pin down its pose: three for position, three for orientation. An arm with seven joints has one more than it strictly needs, which makes it kinematically redundant - you can hold a cup perfectly still while swinging your elbow through space, and that spare joint is what lets you reach around obstacles.108
For a robot arm, DoF is simply the count of independently actuated joints (usually revolute or prismatic) that affect the end-effector pose, and the count tells you what the arm can do.109 Three or four DoF buys simple pick-and-place: mostly positioning, with limited control over which way the tool points.
Five is more flexible, managing some orientation changes but not arbitrary tool orientations everywhere. Six is the canonical industrial standard because it's just enough to reach any position and orientation within the workspace (full SE(3) control), typically laid out like a human arm: three joints doing the shoulder-plus-elbow work and a three-axis spherical wrist.110
Seven and up buys back the human-style redundancy: multiple internal configurations for the same hand pose, which improves dexterity and obstacle avoidance. So the human arm is roughly 7-DoF, robot arms in the wild run from about 3 to 10+, six is the industrial default, and seven is what you pick when you want human-like flexibility. A dexterous hand blows the curve entirely at roughly 20-27 DoF.
Backdrivable Actuation
A recurring frontier theme in this layer is backdrivable / quasi-direct-drive actuation. Conventional robot hands use high gear ratios (~288:1) that add friction and make a light finger feel heavy, which is why Origami Robotics builds a 21-DoF hand at a radical 15:1 ratio that can "feel" contact by reading motor current directly (more on the company in The hardware companies below).
Backdrivable means you can grab the robot's finger and push it, and the motor behind it will turn. That sounds trivial, but for most robot hands it's physically impossible - and the reason why unlocks a lot.
Start with why gearboxes exist at all. Electric motors are naturally fast and weak: they love spinning at thousands of rpm but produce tiny torque. A finger joint needs the opposite - slow and strong. So you put a gearbox between them that trades speed for torque: a 288:1 ratio means the motor spins 288 times for one turn of the joint, and in exchange the joint gets roughly 288× the motor's torque. That's how a small, light motor can drive a finger that grips firmly. So far, all upside.
The catch is that the trade also works in reverse - and it works against you. Pushing on the finger from outside means driving the gearbox backwards, and now the 288:1 leverage is inverted: your push arrives at the motor divided by 288, while every bit of friction in those gear stages stacks up against you. Below some ratio you can shove the joint and it yields; at 288:1 the friction wins and the joint is effectively locked solid. That's what non-backdrivable means - force flows out through the gearbox but can't flow back in.
Two specific villains, both named in Origami's "dexterity deadlocks" essay:
- Friction and stiction. Every meshing gear pair leaks a bit of torque to friction. Stiction (static friction) is the nastiest form: a light touch on the fingertip produces a force at the motor that's too small to overcome the gearbox's own internal grip, so the motor feels nothing. The contact is invisible below that threshold.
- Reflected inertia. From the motor's viewpoint, the finger's inertia is divided by the ratio squared - but from the world's viewpoint, the motor's spinning rotor is multiplied by the ratio squared. A tiny rotor behind a 288:1 gearbox feels, to anything the finger touches, like it weighs (288)² ≈ 83,000× its actual rotational inertia. So a physically light finger slams into an egg like a heavy one, because whatever it touches must decelerate that huge apparent mass.
The maths behind reflected inertia. First the cast of symbols, so nothing appears unannounced. Subscript means "measured at the motor's rotor"; subscript means "measured at the joint" - the finger the world actually touches.
- - the gear ratio: the motor spins times for one turn of the joint ( for a conventional hand, for Origami's).
- - the angle of the motor shaft and of the joint. One overdot is angular speed (), two is angular acceleration ().
- - the rotor's own moment of inertia: how hard the motor's spinning core is to accelerate. - the physical finger's inertia about the joint.
- - torque at the motor shaft and torque at the joint.
- - kinetic energy.
The gearbox rigidly links the two shafts, so their angles - and therefore speeds and accelerations - are locked in ratio:
Now ask how much torque must arrive at the joint just to accelerate the rotor. Accelerating the joint at forces the rotor to accelerate times harder, which takes motor torque
and the gearbox multiplies torque by on the way back through, so seen from the joint side:
The joint obeys (Newton's second law for rotation) with an effective inertia . Note where the square comes from: it isn't one effect but two multiplied together. One factor of because the rotor must spin times faster than the joint, and a second factor of because the gearbox levers the resulting torque back up.
The energy argument says the same thing in one line. The kinetic energy stored in the spinning rotor, rewritten in terms of joint speed via :
Whatever pushes on the joint must supply the energy of a body with inertia - there's no way around it, because the energy really is stored in that fast-spinning rotor.
Plug in numbers. The total inertia anything at the fingertip must fight, , is the finger's own inertia plus the reflected term:
At the multiplier is : a fingertip-sized rotor of order kg·m² (about 10 g·cm²) reflects to roughly kg·m² - the inertia of a 1 kg mass swinging 30 cm from the joint, hiding inside a "light" finger.
At Origami's the multiplier is , about less apparent inertia from the same rotor, small enough that the finger's own (actually light) mass is once again what contact feels.
The same equation read backwards also explains why high gearing is so tempting: from the rotor's side the finger's inertia appears divided by the same factor, , so the motor barely notices its load. The price is that the world massively notices the motor.
Now the "feel contact by reading motor current" part. In a motor, torque is proportional to current, and the electronics already measure current, so in principle the motor's current signal is a torque sensor for free. The question is whether external forces at the fingertip ever make it back to the motor to show up in that signal.
Through a 288:1 gearbox, they don't: friction swallows them, so the current tells you about the gearbox, not the world. Through a 15:1 gearbox, the path is "transparent" enough that a gentle fingertip contact produces a measurable current change at the motor. That's why Origami's hand needs no separate tactile sensors on the fingers: the actuators themselves are the sensors. The jargon for this property is torque transparency.
The figure below runs this experiment: the same three 2 N fingertip taps arrive at the motor through a 15:1 and a slider-chosen gearbox. Drag the ratio and watch the taps sink under the detection floor, then read the bottom row of the table, because that row is the training data your policy inherits.
| what the dataset sees | tap 1 | tap 2 | tap 3 | what the policy can learn |
|---|---|---|---|---|
| 15:1 | ✓ felt | ✓ felt | ✓ felt | contact has a signature |
| 288:1 | ✗ missed | ✗ missed | ✗ missed | touch never enters the data |
"Quasi-direct-drive" is just where this sits on a spectrum. True direct drive (1:1, no gearbox) is maximally transparent but needs big, heavy, expensive motors to get usable torque: fine for a MIT lab leg, not for a finger. High ratios (100:1+) give you cheap torque but a numb, rigid limb.
Quasi-direct-drive is the engineering compromise: a low single-stage ratio (roughly 6:1 to 15:1) paired with a torque-dense "pancake" (axial-flux) motor, keeping enough transparency to feel while getting enough torque to grip. Origami puts those pancake motors inside the finger joints at 15:1 - radical because conventional wisdom says you can't get useful finger strength at a ratio that low.
Why this is a frontier theme rather than a hardware nicety: the post's takeaway line is that bad actuators cap what any learning algorithm can achieve. A policy can only learn from signals that exist. If the hardware physically can't sense gentle contact - and slams through it with 83,000× reflected inertia when it happens - then no amount of data or model cleverness produces delicate manipulation, because the information was destroyed before it ever reached the network. Backdrivability is what puts touch into the data stream at all.
A pair of intuitions to keep: a non-backdrivable joint is like a car with the engine seized - you can't push-start it by rolling it, force only flows one way. And you already own a backdrivable actuator: your own arm. Someone nudges your hand and you feel it instantly and yield - muscles and tendons are superbly transparent. That's the property Origami is buying back with the 15:1 ratio.
The Prototyping Bench
Actuators are only half the story of this layer; the other half is the physical toolchain that lets a small team design, build and qualify them. A robot is a physical product, so before you can ship one you have to fabricate parts, build and debug the electronics, prove the hardware survives real-world abuse, and keep the robot connected. None of the gear that does this is exotic. It's the same bench any serious hardware team assembles, and it collapses the fabricate-measure-debug-qualify loop from weeks (outsourced machine shops, contract test labs, slow PCB houses) to hours or days in-house.
Fabrication first, because it's the fastest iteration loop in the building. A Bambu Lab desktop FDM printer melts plastic filament and lays it down layer by layer, so you can redesign a gripper mount in the evening and hold the new part the next morning instead of waiting weeks for a machine shop.111
The enclosed P1S (256mm-cube build volume, up to 500mm/s) at roughly $399-699 street is the workhorse; add a dual-extrusion H2D (~$1,749-1,999, or the ~$3,799 H2D Pro with a 350C nozzle and 65C heated chamber) only if you need engineering filaments like carbon-fibre nylon or PPS. One trap: the famous X1 Carbon ($1,199 list) was discontinued on 31 March 2026, so it's a non-buy - its $649 successor, the X2D, adds a dual nozzle and an actively heated 65C chamber for nearly half the price.
A Revopoint MetroY-class 3D scanner does the reverse of printing: a blue laser plus structured light captures the exact geometry of a real object (a part, a mould, a human hand) as a point cloud, so you can reverse-engineer a shape, design something that mates to an existing surface, or check a printed part against its CAD model to hundredths of a millimetre.112
The MetroY Pro (0.02mm accuracy, 7M points/sec, streams over Wi-Fi 6, ~$1,400-1,600) is the sweet spot for engineering QC; the Ultra (0.01mm single-frame precision, 90fps, $1,899, or $2,499 as the CMM Edition with a certified carbon-fibre ball plate and Revo Measure software) only earns its keep if you need on-site accuracy verification.
One caveat: these scanners aren't ISO-certified metrology instruments - fine for engineering QC, not for final sign-off in regulated industries. Printer plus scanner together close the loop between the physical and digital worlds: print, scan, verify against CAD, repeat.
Electronics next. Anything that moves needs flex PCBs: circuits printed on bendable polyimide film instead of rigid fibreglass, in 1-4 layers on 25 or 50 micron dielectric, so wiring can fold, twist and route through moving joints - rigid boards physically can't follow a wrist.113
You order them from quick-turn fab houses: JLCPCB runs promotional flex boards from about $2 for early spins, and PCBWay (three-shift operation, a claimed 99% on-time rate) handles tighter quick-turn and rigid-flex work. Expect flex to cost several times rigid FR4, because polyimide is dimensionally less stable and the process is more complex.
To debug the boards you want an oscilloscope (voltage over time on a screen, the fundamental instrument for seeing analogue signals and noise), a logic analyser (Saleae or Rigol) to decode digital buses like SPI, I2C and UART between chips, and a current-limited bench power supply sized to your actuators - 70A-class, because robot motors are hungry. Rigol and Siglent are the standard startup bench brands, and $350-900 buys plenty: the Rigol DS1054Z is ~$349, the DHO804 ~$439, the DHO924S ~$899, from vendors like Saelig, TEquipment or Circuit Specialists.114
Then qualification: the "will it survive the field?" tools. An IMU-calibration turntable (rate table) is a precision motor-driven platform that spins an inertial sensor at exactly-known rates and angles so you can calibrate the gyroscopes and accelerometers; two-axis tables handle gyro scale-factor and accelerometer bias via rate-transfer and tumble tests.115 Without this the robot doesn't know which way is up or how fast it's turning - the difference between a platform that knows its own motion and one that drifts, foundational for anything legged or mobile.
Precision tables run from thousands into tens of thousands, so buy used from Ideal Aerosmith or Acutronic-class vendors, or build a low-end one for early calibration. A thermal chamber bakes and freezes hardware to prove it works across temperature; a vibration rig shakes it to simulate transport and operation, exposing loose connectors and fatigue cracks (CSZ's benchtop TCB-1.3 uses pneumatic hammers for 6-DoF random vibration across 1-60 GRMS and can nest inside a temperature chamber for combined temp-plus-vibe screening); a thermal camera spots an overheating motor driver or MOSFET before it fails.116
Combined temp-and-vibration setups from CSZ, Sentek or ETS are the big-ticket item, often $10K-50K+ on direct quote - if budget is tight, rent chamber time or use a contract test lab early and buy once you have recurring qualification cycles. This is the gap between "works on the bench" and "works in a warehouse in July", and it's the gap that kills hardware startups.
Last, connectivity. A mobile robot needs an internet link, and teleoperation (the standard fallback when autonomy fails and a key data-collection method for training policies) dies with the link. The Peplink MAX Transit 5G (~$999, internal CAT-20 modem, dual SIM, about $250 over the CAT-18 version) bonds several cellular modems and SIMs into one resilient pipe via SpeedFusion, which Peplink markets explicitly for teleoperated vehicles; the MAX Transit Duo Pro (1Gbps, dual LTE) even runs Docker and Python edge compute on the router itself.117
The MikroTik Chateau 5G is the budget alternative, and the Chateau 5G ax (Gen 6) adds faster Wi-Fi, a better CPU and 2.5GbE, fine for fixed or base-station links where bonding matters less.
Budget roughly $5-10K to stand up print, scan and EE-debug, and 5-10x that once you add real thermal, vibration and IMU qualification. And be clear-eyed about what this bench is: commodity infrastructure, not frontier. Individually the tools skew prosumer - a well-funded hobbyist owns a Bambu printer, a Rigol scope and a Revopoint scanner.
What separates a serious robotics startup is assembling the full loop and using it with discipline: closed-loop print-then-scan-verify, real IMU calibration on a rate table rather than hand-waving, actual environmental qualification rather than "it worked on my desk". This bench is also why adjacent hardtech founders court robotics teams: the DFM (design-for-manufacturing) rework, the Shenzhen supplier relationships and the QC processes that get a robot built in the thousands are the skills every physical-product company needs. The frontier lives upstream, in the actuators and the learned policies - which brings us to the companies building them.
The Hardware Companies
The 15:1 hand threaded through this section belongs to a real company you can watch. Origami Robotics is a five-person Y Combinator Winter 2026 startup (YC lists San Francisco; other sources say Millbrae - both Bay Area) founded by brothers Quanting (Daniel) and Quanliang (Ryan) Xie.118
Daniel is a CMU Robotics Institute PhD student (expected 2029) in the CLAW Lab under Yonatan Bisk and a former Apple manufacturing intern; he leads the manipulation-model side, and says he founded the company after hitting the same bottleneck this section describes: no existing robot hand was truly designed for learning, and manipulation data was scarce.119
Ryan is the serial-hardware half: an MS in Robotics from Michigan, about 1.5 years as a robotics software engineer at Canvas (the drywall-finishing robot startup), and founder of Ground Robotics (StartX S25), which builds the G1 autonomous turf-care robot for golf courses.
The hand is only half of Origami's bet. The other half is a wearable data-collection glove co-designed with kinematics identical to the hand, so human demonstrations transfer to the robot with minimal embodiment gap and minimal sim-to-real loss. The business stacks in layers: sell the hand, sell the glove, sell or bundle access to the resulting manipulation dataset to physical-AI labs, and longer term deploy gloves across factories and warehouses to build a proprietary data flywheel that trains a general "manipulate anything" model sold back as automation.
They claim to have sold hands to physical-AI labs including Amazon and gloves to multiple companies; no public unit pricing exists. The design choice itself sits on an industry-wide trend line, shared with Tesla Optimus among others, away from high-ratio geared hands toward backdrivable, proprioceptive actuation.
Loading tweet…
If Origami is rethinking the hand, Almond is driving down the price of capable arms. Almond is a three-person San Francisco startup (YC Spring 2025) that designs and assembles its Axol dual-arm robot in its own SF factory. The origin story, per their launch materials: they deployed other people's robots on real grocery and factory shifts, kept hitting hardware limits, and built their own.120
Each Axol carries two independent 7-DoF arms with 860mm reach, full 180-degree wrist pitch and yaw, 4kg sustained and 6.5kg peak payload, two-finger grippers with swappable tips, a 500Hz CAN control loop over USB-C, and a 1500W (24V, 62.5A) supply. The free open-source Python SDK covers bimanual IK, CAN motor control, ZED camera streaming, WebXR VR teleop, gravity-compensated hand-guiding and LeRobot bindings.
The problem Almond attacks is the same data-plus-hardware bottleneck this post keeps circling: the best manipulation AI is trained by imitation, the reference rig is Stanford ALOHA, commercial builds via Trossen run somewhere in the $20k-32k region, and research arms are weak and fragile on real production lines.
Axol undercuts that: the robot (arms, grippers, supply, SDK) at $7,999 launch / $8,999 regular, the height-adjustable mobile Base at $999, and the Axol Kit (robot, Base, three ZED X One S cameras, an NVIDIA Orin NX 16GB compute box and cables, shipping included) at $11,999 launch / $12,999 regular, assembled in San Francisco and marketed as US-made, with first orders shipping in launch week. The model is affordable AI-ready manipulation hardware plus services.
The founders are Saba Khalilnaji, an ex-DoorDash logistics engineer and former CTO of Conduit (YC W23), and Shawn Patel, previously CTO at Camphor (PearX W24, not YC), with hardware time at Google and an EECS degree from Berkeley; their YC partner is David Lieb, co-founder of Google Photos and Bump. Axol's spec sheet doubles as a vocabulary map of the robot-learning stack (bimanual, payload, reach, singularities, end-effector, 500Hz CAN, teleoperation, IK, imitation learning, LeRobot), and one ~$8-12k product bundling sturdy arms, cameras, teleop and a learning framework shows how demonstration data becomes autonomous behaviour.
The Robot Learning Company (TRLC) plays the same commoditisation game one price bracket down, pitched as "the infrastructure that robot intelligence is built and deployed on". It's YC-backed (Spring 2025 / X25), based in Germany, and run by a solo founder, Jannik Grothusen.121 The flagship TRLC-DK1 ($3,999) is a single-arm teleoperation-and-learning kit: a passive "leader" arm you move by hand, a motorised "follower" that mirrors it (700mm reach, 1kg payload), and two cameras, recording demonstrations over USB-C to any Linux, Mac or Windows laptop.
The bimanual TRLC-DK1-X ($6,999) doubles the leader-follower pairs and adds a third camera, keeping a dual-arm system under the $10,000 mark. Roughly 100 demonstrations train a policy via imitation learning (plus RL, per the company) instead of hand-coded programming, everything is Apache-2.0 on GitHub, built on Hugging Face's LeRobot and crediting GELLO, the Low-Cost Robot Arm, SO-100 and OpenArm as influences, and kits ship "Open Source + Made in Germany" on a 4-6 week lead time, sold direct.
TRLC's wedge is the long tail: industrial arms have historically cost tens of thousands and must be hand-programmed by specialists per task, so automation only pencilled out for high-volume factories, and the learning-based alternative needed standardised hardware and data that labs and small businesses couldn't access. A full learn-from-demonstration rig for a few thousand dollars rather than $50k+ targets stationary, repetitive tasks that were never worth automating before, with a business model of kits to researchers and developers now, pilot partners for repetitive manual work next, and a bid to be the default hardware-data-software stack, kept LeRobot-compatible on purpose.
Grothusen has a mechatronics degree and about two years of robot-learning research at UC Berkeley and TU Munich, with earlier stints at GoodBytz robotic kitchens, a DESY internship doing RL for particle accelerators, and Entrepreneur First (W23, London) on his profile. His calling card: in November 2024 he and Kaspar Janssen (ETH Zurich) built a GPT-4o-powered spill-cleaning robot in four days from about $250 of open-source arms ($120 per arm) and ~100 demonstrations, covered by TechCrunch.122 Investors include 468 Capital, Y Combinator, Munich Innovation Ecosystem and TUM Venture Labs, with the round reported at ~$500K.
Finally, HLabs (HLaboratories) sells the parts everyone else assembles robots from. It's a US-based YC Winter 2026 company, founded in 2025 and run solo by Paul Hetherington in San Francisco - a second-time YC founder who previously co-founded and ran Mystic (YC W21, formerly neuro.ai), a low-latency serverless-GPU API for deploying ML models, as CEO for about six years with around 11 employees.123
HLabs' pitch, straight from the YC launch, is "focus on getting your robot to walk instead of turning on reliably". Teams rarely build robots from scratch; they assemble compute, motor drivers, actuators, sensors and a power/comms bus from off-the-shelf parts, and many of those parts - especially cheap high-torque actuators - are China-sourced (CubeMars, Unitree), arrive late, use inconsistent interfaces, and leave engineers hand-wiring mismatched PCBs and debugging why the CAN bus won't talk.
HLabs designs and manufactures in the USA a matched set that snaps together and runs from a plug-and-play Python API:124
- RB1 - a Jetson carrier "main board" combining compute and power distribution (the custom motherboard problem from the Compute layer below, productised).
- WM1 - a 2-channel USB-C wireless radio for video and data that makes the RB1 remotely controllable.
- ACB3 - an FOC motor-control board (STM32G474 microcontroller, 12-30V, up to 40A over XT60, CAN over a 2-pin JST-GH connector at 1 Mbit/s, hall-effect encoder, USB-C for firmware rather than power, Apache-2.0 open-source firmware on the SimpleFOC library).
- M1 - a pancake motor.
- A1/A1m - planetary and cycloidal actuators driven by the ACB3.
(FOC, field-oriented control, is the maths that makes a brushless motor smooth, quiet and torque-accurate; planetary and cycloidal are two gearbox designs trading off size, backlash and torque.)
Pricing isn't public: everything is direct preorder under a one-week launch sale, with electronics stated to ship in summer 2026 and the M1 motor within a month or two of the launch post. Early customers, per YC, span household robots, humanoids, robot arms, quadrupeds and defence - HLabs rides both the onshoring/supply-chain-security tailwind and the standardisation of the robot stack around Jetson, CAN, FOC and Python, betting that robotics is industrialising from bespoke hand-wired builds into purchasable modules.
Hetherington also describes the WM1 mounting to a Steam Deck as a handheld ground station, and an RB1 sensor loadout of two IMUs plus a compass, XT30 power carrying CAN, and a camera module. The obvious reference points if you're comparison-shopping are mjbots' moteus, ODrive, CubeMars, Unitree and Seeed's reComputer.
Four companies, and the same bet underneath: a hand that can feel, an affordable bimanual arm, a teach-by-demonstration kit and a box of matched parts all assume that whoever owns the hardware-and-data loop, not just the algorithm, owns the resulting model.
Compute
The robot needs a computer small and power-efficient enough to ride onboard yet powerful enough to run neural nets in real time. The de-facto standard is the NVIDIA Jetson family of system-on-modules: a single small board fusing an Arm CPU, an NVIDIA GPU and memory into one unit that sips 7 to 130 watts where a desktop GPU burns hundreds. The GPU matters because perception and control policies are neural networks, and the compute has to ride onboard because streaming video to the cloud adds latency and fails the moment connectivity drops.
The mainstream generation is Orin (2022): Orin Nano (up to 67 TOPS at 7-25 W), Orin NX (up to 157 TOPS at 10-40 W), and the flagship AGX Orin (up to 275 TOPS with 64 GB, about $1,999 as a bare module).125 The $249 Orin Nano Super Developer Kit is the best-value entry point, well within hobbyist reach.
The new generation is the Blackwell-based Thor (2025), built for humanoids running large VLAs onboard. Its T5000 module pairs 2,560 CUDA cores and 96 Tensor Cores with a 14-core Arm Neoverse CPU and 128 GB of LPDDR5 (~273 GB/s), delivering up to ~2,070 FP4 TFLOPS in a 40-130 W envelope - NVIDIA cites roughly 7.5x the AI performance of AGX Orin at 3.5x the efficiency.126
The $3,499 AGX Thor Developer Kit (module plus heatsink, reference carrier board, 140 W supply, WiFi 6E, 1 TB NVMe SSD) opened pre-orders in August 2025 through Seeed Studio, Arrow and RS, and a cheaper T4000 module (~1,200 TFLOPS, 64 GB) is planned.126
Crucially, a Jetson module has no ports and can't run alone; it plugs via a 699-pin connector into a carrier board that exposes the real-world I/O: USB, Ethernet, PCIe, camera inputs (CSI/GMSL2), GPIO and power. If you've ever built your own PC, you already have the mental model: the Jetson module is the CPU-and-GPU, and the carrier board is the motherboard, except there's no off-the-shelf ATX standard, so every robotics team designs or buys its own.
NVIDIA's reference carriers inside the dev kits are fine for prototyping, but shipping a real robot means a board sized, shaped and ruggedised for that specific machine, and a whole vendor ecosystem supplies them: Connect Tech (the Forge carrier for AGX Orin, the Gauntlet for Thor), Auvidea (the X242 for Thor, with dual 10GbE, PCIe x16 and CSI-2 camera support), plus Advantech, Aetina, MiiVii, TZTEK and Forecr's DSBOARD line.127
That carrier board is where a hardware team spends real engineering effort. The compute module is maybe 20% of the problem; rugged, correctly shaped, sensor-rich I/O is the unglamorous work that decides whether a prototype ever ships.
Be precise about what's frontier here and what's not. Jetson itself is commodity infrastructure, close to a monopoly on embedded robot GPU compute, and nearly everyone buys it off the shelf; Thor is the one frontier-adjacent piece, new and expensive and existing purely to run the large on-robot models that define the current wave. The real frontier work is what you build around the module: carrier design, sensor fusion, calibration and the policies on top.
Match the brain to the job - the Orin Nano Super for prototyping and education, an Orin NX on a Connect Tech or Forecr carrier for mainstream mobile robots, and AGX Orin 64 GB or Thor for multi-camera perception and onboard VLA workloads, budgeting separately for a production carrier board once you leave the dev kit.
Sensors
Cameras give sight, and in robotics that usually means depth. Stereo cameras like the Stereolabs ZED line use two lenses a fixed distance apart (the "baseline"), mimicking your binocular vision: software compares the two images and computes a depth value for every pixel (a depth map) from the parallax between them, plus the camera's own motion (visual odometry). An arm can't pick an object, and a mobile robot can't dodge an obstacle, without knowing 3D distances; stereo gives dense depth far more cheaply than LiDAR while also feeding the models rich colour imagery. Depth cameras (Intel RealSense) do similar.
The current ZED hardware splits by job. The ZED X ($599) sees 0.3-20 m across up to a 120° field of view on a 12 cm baseline in an IP66 aluminium housing, aimed at agriculture, construction, logistics and last-mile robots; the compact ZED X Mini ($549, 94 x 32 x 37 mm, IP67, built-in IMU) uses dual 1920x1200 global-shutter colour sensors for 0.1-8 m or 0.15-12 m depth depending on lens, built to mount near a robot wrist for manipulation.128
Both use an automotive-grade GMSL2 cable link (up to 15 m) instead of USB so the camera can sit far from the compute, and since GMSL2 isn't USB you must budget for a matching capture interface: Stereolabs sells a ZED Carrier Mini and the ZED Box Mini, an integrated Jetson Orin Nano/NX unit with dual GMSL2 inputs, at roughly $879-$1,335.128 Older USB3 models like the ZED 2i are cheaper and plug-and-play, just less rugged.
The ZED SDK layers GPU-accelerated neural depth, object detection and skeleton tracking on top, integrates with Isaac ROS and ROS 2 Humble (confirm support for your exact Jetson before committing), and reaches ~60 FPS neural depth for one camera on an AGX Orin, ~30 FPS for two.129 A Jetson on a carrier board plus one or two ZEDs is the canonical perception stack inside a large fraction of today's research and commercial robots.
On the manipulation bench the counterpart is the RealSense D-series (spun out of Intel in July 2025), and one model has quietly become the standard: the D405 ($272), a 60 g short-range camera delivering sub-millimetre depth at 7-50 cm - the eye-in-hand wrist camera the imitation-learning world converged on. Trossen's WidowX AI Follower ships with one, and LeRobot's RealSenseCamera driver is its only first-class depth camera, recording the depth stream natively alongside RGB.130 Its siblings split by range - the D435i ($334) covers 0.3-6 m for overhead scene views, the longer-baseline D455 ($419) holds accuracy at 2-4 m - and the buying logic is that the same D405 bolts onto a $120 SO-101 wrist and a $50k UR wrist unchanged, so camera spend survives every arm upgrade.
One spec is worth knowing before you choose. The D435i and D455 are active stereo - an IR dot projector paints texture for the matcher, and direct sunlight washes it out - while the D405 has no projector at all: pure passive stereo from two visible-light imagers. That distinction matters more each year because the matcher itself is becoming a model. NVIDIA's FoundationStereo and its real-time successor Fast-FoundationStereo are zero-shot stereo networks that take the rectified left/right pair from any calibrated stereo camera and beat the onboard ASIC's depth - cleaner edges, fewer holes, no sunlight problem - so the same $272 camera gives you hardware depth on a laptop today and model-grade depth on an RTX box later (librealsense exposes the raw left/right streams; the D455 joins in with its emitter switched off).131
An IMU (inertial measurement unit - accelerometer + gyroscope) tells the robot its orientation and motion ("which way is up"). LiDAR gives laser-based 3D depth at longer range.
And the underrated, still-unsolved sense is touch. Robots see well and know their own joint angles, but they're largely numb, and the decisive physics of a grasp happens at the fingertip contact, invisible to cameras the moment the hand occludes the object. Touch is the only sense that directly measures slip, contact force, friction and local texture: the signals you use unconsciously to modulate grip on a grape or a wet dish. Without them, a robot either drops things or squeezes too hard.
Note the distinction between proprioception (the robot sensing the pose and load of its own body) and tactile sensing (what's happening at the skin-object interface) - they're different signals and both matter. Proprioception buys coordination and stable grasps; tactile reports contact location, micro-slip and surface friction, the information vision loses the instant fingers wrap an object.
Two sensor families dominate. Vision-based optical sensors, pioneered at MIT around 2009, put a tiny camera behind a soft transparent gel coated with reflective paint and lit by coloured LEDs; press the gel against an object and a photometric-stereo algorithm turns the shading of the deformation into a high-resolution 3D map of the contact. It's effectively a microscope for touch - it measures geometry rather than raw force, with force and slip inferred from how printed marker dots shift.
The ready-made commercial version is the GelSight Mini: $499, replaceable $49 gel cartridges rated for about 1,000 coin presses, a 3280x2464 camera behind a 4.25 mm gel, plug-and-play USB with ROS and Python support.132
Meta's DIGIT (2020) is the same optical idea made radically cheap and open source: about $15 in parts at thousand-unit scale, USB-3, small enough for the fingers of a multi-finger hand, and sold assembled through GelSight if you'd rather not build one.133
Its successor Digit 360 (announced October 2024 with GelSight) is a fingertip-shaped research prototype with over 18 sensing features: some 8.3 million taxels of omnidirectional touch, forces down to 1 millinewton, an on-device AI accelerator, plus vibration, heat and even odour sensing.134 It's still a statement of intent rather than a retail product.
The second family skips the camera entirely: thin arrays of capacitive, MEMS or magnetic "taxels" (tactile pixels). Xela Robotics' uSkin reads the field shift of small magnets through Hall-effect sensors: three-axis sensing points around 4 x 4 mm (shrinking to 2.5 x 2.5 mm) at 0.1 gram-force resolution, in skins from 4 mm thick, designed to retrofit existing grippers and hands. Newer versions compensate for magnetic interference (unaffected by a magnet with a 3 kg pull) so they can work around iron and metal parts; pricing is industrial and quote-based, with new capabilities promised at Automate 2026.135
AnySkin from NYU and Meta (successor to ReSkin) is magnetised rubber over a magnetometer whose replaceable skin decouples from the sensing electronics "like putting on a phone case", and it was the first uncalibrated tactile sensor to show zero-shot transfer of learned manipulation policies across sensor instances.136
The trade-off between the families is consistent: optical gives stunning spatial detail (thousands of pixels per contact) but is bulky and slower; magnetic and capacitive arrays are thin, fast and happy on curved surfaces, but coarser.
The hands these sensors live on span an absurd price range. At the bespoke end, Shadow Robot's DEX-EE (2024, developed with Google DeepMind) is a three-fingered hand about 50% larger than a human's, built to survive long-duration RL experiments.
Each tendon-driven finger carries five Maxon brushed DC motors and a 10 kHz local force-control loop, closes in 500 ms and pinches at 10 N; the fingertips are themselves stereo cameras with internal dot-pattern illumination, streaming 640x480 video at 50 FPS and sensing forces from 0.01 N to 18 N, with 36 Hall-effect taxels across the middle and proximal joints.137 It's quote-based: DeepMind-tier budgets only.
At the other end sits CMU's LEAP Hand (Kenneth Shaw, Ananye Agarwal and Deepak Pathak, RSS 2023): 16 degrees of freedom from 16 Dynamixel XC330 servos (the ROBOTIS bundle is about $2,066), anthropomorphic, 3D-printed, assembled in under four hours for around $2,000, and claimed to be roughly 8x cheaper than the Allegro Hand it outperforms.138
The 2025 v2 is a hybrid rigid-soft design that gets 16 DOF from just eight motors via differential joint stiffness, spanning a $200-300 low-cost variant up to $3,000-4,550 for the advanced metal versions. Pathak went on to co-found Skild AI in 2023 with fellow CMU Robotics Institute professor Abhinav Gupta; its $300M Series A at a ~$1.5B valuation (Jeff Bezos, Sequoia, Lightspeed among the backers) is a fair proxy for what investors think solved dexterity is worth.139
Touch counts as unsolved because there's no standardised, durable, cheap, high-bandwidth touch sensor the way there's a standard camera; the hardware, the wear-and-replacement story and the ML to interpret raw tactile signals are all still open; DeepMind, Meta FAIR, CMU and MIT are in it for that reason.
Meanwhile tactile data is becoming training fuel: imitation-learning and VLA policies improve markedly when touch is added, and touch enables recovery behaviours (regrasp on detected slip) that vision-only systems can't express - the frontier is fusing vision, proprioception and touch in a single policy. The sensors are increasingly commodity; making a robot manipulate well with them is still frontier. For a first lab setup, a LEAP Hand plus a couple of GelSight Minis or DIGITs is the highest-value combination.
Middleware
Robot software is many small programs that must talk to each other dozens of times a second. The dominant plumbing is ROS 2 (Robot Operating System 2) - not actually an OS but a publish/subscribe messaging bus where "nodes" exchange "messages" on named "topics" like /camera or /cmd_vel. Under the hood it rides on DDS (Data Distribution Service, an industrial pub/sub standard), and it displaced ROS 1 in production because it added real-time support, security, and multi-robot capability.140 One caveat worth filing: serious teams increasingly run ROS 2 as one layer inside a proprietary stack rather than standalone.
DDS isn't the only wiring on offer, though the alternatives sit outside the verified corpus behind the rest of this post, so take this paragraph as orientation. Zenoh, a lighter transport protocol, is now an officially supported ROS 2 alternative to DDS,141 and the LeRobot crowd has taken to dora-rs, a Rust dataflow framework with Python bindings that plays the ROS role without the ROS baggage142 - the SO-101 URDF mirror cited later in this post lives in a dora project.143
When you record everything a robot sensed for later replay and debugging, you write a bag file; the modern, self-describing container format for these logs is MCAP, created by Foxglove, launched in 2022, and now the default bag format shipped with ROS 2 and NVIDIA Isaac.144 A single timestamped container holds 3D point clouds, video, audio, GNSS and time-series channels together, which is what you want when the question is "what did every sensor see the moment it dropped the mug?"
At the low level, motors and boards talk over a CAN bus (Controller Area Network, borrowed from cars). The naive way to wire a robot is a dedicated cable from the computer to every motor - thirty motors, thirty cables. CAN replaces that with one shared two-wire pair running through the whole body: every device taps into it, broadcasts short ID-tagged messages, and listens for the IDs addressed to it. That's why a humanoid's limbs carry one thin wire bundle instead of a hundred-wire harness, and it's the same trick the smart-servo daisy-chain plays at hobby scale.
Once the robot runs, you inherit every problem an ordinary software team has, except the bugs live in the physical world. That realisation spawned a "picks and shovels" industry on top of the plumbing: observability and logging, fleet management with over-the-air (OTA) updates, simulation, and policy evaluation. Investors love this layer for the same reason they loved Datadog, Snowflake and GitHub in the last software wave - you sell tools to every robot company instead of betting on which robot wins.
Foxglove is the observability standard-bearer: MCAP plus its 3D visualiser is the field's default answer to "record what every sensor saw when something went wrong". The full profile (founders, funding, customers) lives in The Tooling Companies below.
Fleet management is the "push new software to 500 robots at once and watch their health" problem. Formant has raised roughly $45M for fleet management, observability and teleoperation, from investors including BMW i Ventures, Intel Capital, Ericsson Ventures and GS Futures.145 InOrbit raised a $10M Series A in September 2025 (L'ATTITUDE Ventures, Globant; roughly $13.5M total) and has announced OpenRobOps (ORO), a permissively licensed, self-hostable open-source fleet manager due later in 2026.146
Cogniteam's Nimbus platform treats robots as connected devices with one-click OTA updates across a fleet, and on the open-source fringe ROSscope v0.1.0 pitches itself as "Prometheus + Grafana with native ROS 2 awareness".147
Simulation (test policies in a virtual world before risking real hardware) is the third pillar, and the raw speed numbers explain why the Learning layer below leans on it so heavily. DeepMind's MuJoCo hits ~2.7M physics steps per second in its MJX form on an 8-chip TPU v5, and ~650K even on an Apple M3 Max; NVIDIA's Isaac Sim/Lab manages ~150K steps per second on an RTX 4090 while running 4,096 parallel humanoid environments.148
The split between them is real: Isaac carries roughly 20x higher single-environment physics overhead than MuJoCo, so teams pick MuJoCo for contact-rich manipulation accuracy and Isaac Lab for massively parallel humanoid locomotion training.148
Then there's Genesis, an open-source generative physics engine from a CMU-led consortium of 20+ labs, announced December 2024, claiming 430,000x faster-than-real-time simulation (~43M FPS on a single RTX 4090), 10-80x speed-ups over Isaac Gym and MuJoCo MJX, pure Python throughout, and text-prompted generation of 4D training worlds via a vision-language model - the project's own numbers, so hold them loosely.149
The newest entrant isn't a physics engine at all. DreamDojo (NVIDIA GEAR, February 2026) is a video world model trained into a working neural simulator on 44,000 hours of egocentric human video: continuous action signals go in, the next RGB frames and sensor states come back in real time, and the mechanics of each robot are learned from data, with no physics equations or graphics engine anywhere in the pipeline.150
Not a pixel it renders is real, and its pitch is post-training rather than debugging: a massively parallel rollout farm made of a few real robot stations plus a large pile of inference compute running world models. Whether a learned simulator's physics errors compound slower than a classical simulator's reality gap is, as far as I can tell, still an open question.
Evaluation is the hardest of the four, and it gets its own layer two sections below. The market detail to file now: mislabelled successes silently poison training data, evals and RL rewards, which is the core pitch of Instance Labs, the YC-backed verifier startup profiled in The Tooling Companies.151 Around it a benchmark ecosystem is forming: RoboArena for distributed real-world evaluation of generalist policies, RobotArena-infinity and PolaRiS for real-to-sim evaluation via neural reconstruction and Gaussian splatting, and NVIDIA's Isaac Lab-Arena.152
If you're standing up your own stack, most of this layer costs nothing: ROS 2 as middleware, MCAP as the log format, Foxglove's desktop app for visualisation (the paid cloud platform is aimed at teams with serious fleet-data volumes).
Simulation is where you need hardware - an RTX 4090 (~$1,600-2,000) is the reference card behind both the Isaac and Genesis figures above, cloud A6000 or H100 instances substitute if you lack one, and MuJoCo is free and quick even on Apple silicon. Isaac Sim is free too, but locks you to NVIDIA GPUs. Fleet platforms (Formant, InOrbit, Nimbus) are commercial SaaS priced by robot count, so request quotes, or wait for OpenRobOps if you want self-hosted; for policy evals, Instance Labs' automated verifier beats hand-labelling.
Where the frontier sits in all this: the plumbing (ROS 2, MCAP) is commodity table-stakes; observability and fleet tooling is real commercial infrastructure already part-way to commodity; the live frontier is policy evaluation and fast, high-fidelity simulation, classical or learned.
Data collection
Because demonstration data is the bottleneck, an entire industry exists just to produce it. There are two big families. Teleoperation rigs: a human physically moves a small "leader" arm and a larger "follower" robot copies it in real time while cameras record, the leader-follower pattern, canonically Stanford's ALOHA (bimanual, two-arm) and the cheap leader-only GELLO153. A clever variant, UMI (Universal Manipulation Interface)154, removes the robot entirely: a ~$370 handheld gripper with a GoPro lets a person collect "in-the-wild" data anywhere.
The second family is egocentric (first-person) video: because a humanoid shares human body plan, a head-mounted camera watching a person do chores is almost a demonstration for the robot. Meta's Ego4D155 is the academic reference dataset; Meta's Project Aria glasses are the standard capture device.
That's the map. The machinery deserves a closer look, because the economics of each rig decide what data exists at all. Every demonstration is a bundle of synchronised camera video plus the exact joint motion over time, and a policy like ACT or Diffusion Policy needs hundreds to thousands of them per task.
Unlike LLMs, there's no internet-scale corpus of robot actions to scrape, so cost-per-demonstration and demonstrations-per-hour are first-order levers on the whole enterprise. Before 2023, bimanual manipulation research meant industrial arms at $80,000+ each, which priced out most university labs.
The ALOHA lineage is the worked example. Trossen Robotics (founded 2005 in Downers Grove, Illinois by Matt Trossen, with research arms under the "Interbotix" brand since around 2016) had long built cheap arms on one trick: instead of custom industrial actuators costing tens of thousands, daisy-chain ROBOTIS Dynamixel X-series smart servos (the XL430, XM430 and XM540), each of which reports its own position, temperature and load. The smaller arm is the WidowX; the largest is the ViperX 300 (6 DoF, 750 mm reach, 1,500 mm span, 750 g payload, 4,096-count servo resolution, ~$6,743).
In 2023, Stanford's Tony Zhao, Chelsea Finn and colleagues bolted four of those arms to a frame and called it ALOHA, "A Low-cost Open-source Hardware system for Bimanual Teleoperation."156 Two WidowX 250s are the leaders you hold; two ViperX 300s are the followers copying your joint angles with 3-5 ms latency over the local Dynamixel bus (teleop needs <10 ms to feel transparent).
The rig logs all 14 joint values at 50 Hz alongside three cameras (two wrist-mounted, one overhead), and roughly 50 demonstrations are enough for ACT to slot a battery, open a bag or cook shrimp. Total build cost: under $20,000, with the full bill of materials and code open-sourced. That's why datasets from hundreds of labs are comparable, and why Trossen became the field's default supplier.
Below, the ALOHA 2 rig, video from the ALOHA 2 project page
The family grew fast. Mobile ALOHA (2024) put the rig on a wheeled base for whole-body tasks like cooking, at roughly $28,000-35,000 to build157. ALOHA 2, the version in the video above, is Google DeepMind's refined hardware revision and was never sold commercially158. A DIY stationary build runs ~$18,500-19,700 in hardware, or ~$20,500-22,700 once you add an RTX 4090 training box, plus $2,000-5,000 more if you pay for professional assembly.
In 2025 Trossen retired the ALOHA branding ("ALOHA is now Trossen AI") around a purpose-built arm, the WidowX AI: 6 DoF, 1.5 kg payload, 700 mm reach, 1,400 mm span, 1 mm repeatability, 4 kg mass, shipping with a 24V/25A supply and iNerve controller159. Individual arms cost $4,545.95 (Base), $4,685.95 (Leader, with a hand grip and finger pinchers) or $4,995.95 (Follower, with an Intel RealSense D405 depth camera).
Ready-made kits run from the single-pair Solo AI at ~$8,999.95, through the Stationary at $23,995.95 (two leader-follower pairs, four depth cameras, touchscreen and frame), up to the Mobile at $33,695.95 without a laptop or $37,845.95 with one.
Everything speaks LeRobot, MuJoCo, NVIDIA Isaac, Gazebo, Google Colab and ROS 2 natively, lead time is two to three weeks, firmware hit v1.8 in spring 2025, and the original ALOHA Solo kit is discontinued. One very 2026 caveat: Trossen quotes final pricing at ship time to pass through per-batch import tariffs, so confirm cost with their sales team before budgeting. Their own setup advice even specifies a stable ~48x30 inch maple-topped table, because visual consistency across a dataset matters more than you'd think.
None of this is frontier hardware; hobby-servo arms with 1 mm repeatability and modest payloads are nowhere near industrial spec. Think of these rigs as the commodity GPUs of embodied AI: a cheap, standardised, reproducible substrate whose entire point is to let the frontier work (ACT, Diffusion Policy, the VLA models) happen on top of it at scale.
Below Trossen sit the budget routes. I2RT Robotics sells the YAM ("Yet Another Manipulator") line as the buy-it commercial alternative: vertically integrated down to its own motors, 6 DoF on a CAN bus, starting at $2,999 for the base YAM (2 kg payload, 780 mm reach) and stepping up through the Pro ($3,499, 3 kg), Ultra ($4,299, 4 kg) and long-reach BIG YAM ($4,999, 4 kg, 1,100 mm), with a $2,999 YAM Leader as the teleop input arm160.
Each ships with a linear gripper, power supply, CAN-USB cable and table clamps on five-day lead times (seven to ten for the BIG YAM).
I2RT's system kits go from the $6,149 MolmoAct 2 Research Kit (Ai2's reference bimanual setup) and the $11,000 ABC Box for open bimanual behaviour cloning, up to a $15,999 Mobile Station and the $23,999 full bimanual Teleoperation Station, all marketed as the "most attainable hardware with the largest open-source dataset." Founder and funding details weren't publicly disclosed as of mid-2026.
If you want assembled-and-warrantied without the kit price, OpenArm's DK1 is quoted around $12,000, roughly 44% under a DIY ALOHA. And with no capital at all you can lease via SVRC (whom you'll meet properly below): its rental fleet - a leader-follower teleop rig, Mobile ALOHA, OpenArm, TRLC's DK1 - runs on one-month minimums with maintenance included, quote-gated except the DK1 at a published $499 a month, return anytime.161 Or skip owning hardware entirely and buy the collection itself, with pilots from ~$2,500.
Cheaper still, GELLO attacks the input side only153. Philipp Wu, Pieter Abbeel and colleagues at UC Berkeley (2023) realised the leader doesn't need motors at all: you 3D-print a shrunken, kinematically matched copy of an arm you already own (Franka, UR5, xArm) and fit ~6-7 Dynamixel XL-330 servos (about $35 each, with 12-bit absolute encoders) purely as joint sensors, so each miniature joint commands the corresponding real one.
The result costs <$300, assembles in about 30 minutes, and is fully open source; in the paper's 12-participant, 5-task user study it beat both VR controllers and 3D space-mice on completion time. ROBOTIS now sells pre-packaged GELLO bundles for the xArm7 and UR5.
The most original idea of the group is UMI154. Built by Cheng Chi and Zhenjia Xu with colleagues across Stanford, Columbia and Toyota Research Institute (RSS 2024, a Best Systems Paper finalist), it's a handheld parallel-jaw gripper with a fisheye GoPro and IMU on the wrist: ~$370 of parts, split roughly $73 for the 3D-printed gripper and $298 for the camera gear.
You simply do the task with your own hand, at about 30 seconds per demonstration; afterwards, visual-inertial SLAM reconstructs the gripper's full 6-DoF path from the footage, and the visible finger spacing recovers the gripper width. No robot exists anywhere in the collection loop.
The catches are real: your deployment robot needs a matching gripper with a stroke above 85 mm, and camera-only capture means no force or depth signal, where a force-capable ALOHA or YAM rig costs 10-40x more. The paper demonstrated dynamic tossing, cup-on-saucer placement, bimanual cloth folding, sequential dishwashing and in-the-wild cup manipulation, and the concept keeps spawning variants (FastUMI162, MV-UMI, DexWild, Actuated-UMI) plus ready-made commercial clones at $2,000-3,000. Of everything in this layer, UMI's robot-free paradigm is the one piece still doing frontier-flavoured work.
It's also the layer's biggest money. Two of 2026's robotics unicorns are direct descendants of the wear-the-gripper idea: Generalist, co-founded by ex-DeepMind researcher Pete Florence, seeded hand-mimicking gripper devices with contributors worldwide, amassed 500,000+ hours of manipulation data, and raised $400M at a $2B valuation in June 2026 with NVIDIA on the cap table.163
And Sunday, founded by Tony Zhao and Cheng Chi themselves (ALOHA's and UMI's first authors), swapped the $20k teleop rig for a roughly $200 Skill Capture Glove, and hit $1.15B in March 2026 on the strength of its Memo home robot.164 Fan's verdict, which those two valuations price in: UMI is "perhaps one of the greatest papers ever written in robotics data".19
Hands pose their own capture problem: an arm has six or seven joints, but a human hand has roughly 20-27, and dexterous-manipulation data is useless unless you record most of them accurately. That niche belongs to Manus (legally Manus Meta B.V. of Geldrop, Netherlands, founded 2014 as a VR-gaming-glove maker; the "Meta" is their own branding, nothing to do with Facebook).
Their flagship Metagloves Pro tracks 25 DoF per hand using electromagnetic-field sensing, a small emitter plus five fingertip sensors, which unlike camera-based hand tracking neither loses fingers to occlusion nor drifts over time. A clenched fist still tracks cleanly165.
The numbers: 120 Hz sampling, ~30 ms latency wired (50 ms wireless), 15 m wireless range, ~3-hour swappable batteries, about 167 g per glove, shipping with a wireless dongle, optical markers, a travel case and a two-year warranty.
A pair runs roughly $6,315-11,278 depending on glove size and which of six software-licence tiers you pick (from bare "No License" up to Core+SDK), and for robotics you specifically want Core+SDK: its C++ and ROS 2 SDKs unlock the retargeting pipeline, the software step that maps human hand geometry onto a robot hand's different mechanical structure. A Haptic variant adds one vibrotactile buzzer per finger so a teleoperator feels when the robot makes contact.
Retargeting currently supports 8+ robot hands, including the Robotera XHand 1 Pro, Tesollo DG-5F, Sharpa Wave, Psyonic Ability Hand, Alt-Bionics Surge, Agibot Omnihand and ByteDance's 20-DoF ByteDexter. The gloves are the official data glove of NVIDIA's Isaac Teleop framework (general availability at GTC 2026, integrated in Isaac Lab 2.3)166, with named research users like CMU's BiDex bimanual teleop (a reported 95% task success) and Peking University/BAAI's RoboBrain-Dex VLA, plus OpenGraph Labs on egocentric capture; listed partners include NVIDIA, Shadow Robot, Google DeepMind, Amazon Robotics and ByteDance, alongside a long-running Xsens mocap partnership.
Two rumours to ignore: Meta Platforms hasn't acquired Manus, and the <7 ms ROS 2 latency figure that floats around conflicts with the official 30/50 ms spec. And skip the discontinued 2022 Quantum Metagloves (historically ~$4,999-9,000) on the secondary market unless deeply discounted, since MANUS Core 3.0 and the Isaac Teleop plugin target the current Pro line.
There's now a UMI-style way to skip the retargeting problem entirely. DexUMI (Mengda Xu, Han Zhang and colleagues across Columbia, Stanford, CMU and NVIDIA's GEAR lab) is a wearable exoskeleton with a one-to-one kinematic mapping to a specific five-finger robot hand: you do the task in the exoskeleton while its wrist camera records, software inpaints the robot hand over your own in every frame, and the resulting policies averaged 86% task success across two robot-hand platforms with no teleoperation anywhere in the loop.167
The catch mirrors UMI's: the exoskeleton is built per target hand, so what the mapping buys you in fidelity it costs you in generality.
Egocentric video is the scaling bet. Ego4D set the template155: Meta AI plus a 13-university consortium had 931 people across 74 locations in 9 countries wear GoPro Hero 8/9s or early AR glasses for 1-10 hours each, with every university team recruiting 10-100 local participants. The result is 3,670 hours (~7 TB) of unscripted daily life, and the project doubled as R&D groundwork for Meta's own wearable-glasses programme. One licensing trap: Ego4D is CC-BY-NC-4.0, non-commercial only, so you can't build a business reselling it.
The capture hardware has since graduated into Project Aria168: 75-gram research glasses packing an RGB camera, two stereo SLAM cameras, two inward eye-tracking cameras, IMUs, spatial microphones, GNSS, a barometer and a magnetometer, used by 200+ academic and corporate partners. They're not sold to consumers; you apply through the Aria Research Kit, and no public price exists.
Aria Gen 2 (announced 2026) keeps the ~75 g weight, adds foldable arms, runs 6-8 hours continuously, and gains a PPG heart-rate sensor plus a contact microphone in the nosepad, with price and general availability still undisclosed. The budget alternative is the Ego4D approach: a $300-450 GoPro on a head or chest mount, at the cost of losing synchronised gaze, SLAM and calibration.
The transfer recipes now work, which is what makes the capture hardware worth its price. Georgia Tech's EgoMimic (PhD student Simar Kareer) mounts the same Aria glasses on the robot as its eyes, shrinking the human-to-robot "domain gap" so far that 90 minutes of human glasses recordings produced a reported 400% task-performance gain, with generalisation to unseen environments169.
Figure's Project Go-Big partnered with Brookfield (owner of 100,000+ residential units) to film passive human video in real homes, and reports its Helix robot navigating to natural-language targets ("go to the fridge") trained on 100% human video and zero robot demonstrations, a result Figure frames as "YouTube for robot behavior"170.
NVIDIA's EgoScale went further and measured a scaling law: across 20,854 hours of egocentric human video, every doubling of hours produced predictable gains in downstream robot task success on a 22-DoF hand, averaging +54% over a no-pretraining baseline171. Related HumanScale work argues egocentric human video can outperform real-robot data for embodied pretraining.
Behind that result sits a lopsided training mix. EgoScale pretrains almost entirely on that in-the-wild human video, learning to predict hand joints and wrist poses; the action fine-tuning that aligns it to a real robot took about 50 hours of motion-capture glove data plus four hours of teleoperation, set against the 20,854 video hours.171
The teleop share is the number Fan leads with, at under 0.1% of the mix. At test time a single demonstration is enough to pick up a new shirt-folding strategy,171 and he bills the log-linear fit as "a neural scaling law for dexterity", arriving six years after the Kaplan scaling laws did the same for language.19
Hence the demand curve: labs now spec 5,000 to 100,000+ hours per project, at least one robotics consortium is planning 100,000 hours for a single 2026 effort, and commercial collectors advertise thousands of hours per day of capacity. The rigs in the field run from Aria glasses and GoPros on caps to chest mounts, wrist cameras, tactile gloves and full-body mocap suits, chasing the core modality list: synchronised RGB, IMU motion, depth, hand pose and gaze.
Re-cut this section's two big families by how far the capture sits from the robot's body and you get the three rungs of Fan's data strategy. One teleop rig can't beat 24 hours of data a day, and in his telling it's more like three once "tantrums" and resets are counted. Data wearables (the UMI descendants, exoskeletons and gloves above) take the robot out of the collection loop and scale to hundreds of thousands of hours. Egocentric video doesn't even need the wearable, and if an FSD-style ambient flywheel spins up, he puts ten million hours within reach in the next year or so.19
Fan's FSD comparison is the load-bearing part, and it's worth pausing on if you drive a Tesla. Every journey feeds the largest physical-data flywheel anyone has built, and nobody in the car feels the upload; strapping on a headset or a glove to record dishwashing is nowhere near that invisible, which is why his endgame is capture that fades into the background of ordinary work.
He attaches three dated bets: teleoperation's share of training data drops to almost nothing within a year or two, an ensemble of purpose-built wearables (each custom-designed for a hardware family and use case) covers the middle, and egocentric video becomes the main diet. The scale equation then loses its robot: the ceiling is just 24 hours a day, times however many humans you can put sensors on.
Note the half-disagreement with this post's own forecast. No Free Corpus expects teleop to persist far longer than dignity suggests, but that call is about the safety-fallback role, not the training-data mix; both can be true at once, and I'd take both bets.
One reading skill this corner of the field demands: the "ego/video/X" branding is shared between robot-control work and generative-video work, and you have to check which is which. Three recent papers make the point. PointWorld172 (Stanford/NVIDIA: Wenlong Huang, Dieter Fox, Fei-Fei Li; a CVPR 2026 Highlight) is actually robotics: a 3D world model for robot manipulation using point-flow, trained on ~2M trajectories across 500 hours, with real-time inference.
EgoX (KAIST, CVPR 2026, arXiv 2512.08269) converts third-person video into first-person view with video diffusion, generative-video adjacent, not a robot controller. And UniVidX (MMLab/HKUST, SIGGRAPH 2026) is purely generative video, a unified framework across 30 graphics tasks; despite the name, it involves no robotics at all. Read PointWorld's abstract; skip the other two unless you care about video generation.
Now look at the labour underneath. All of this is manual work done by actual humans, and the economics are stark. DeepAware's vendor report has teleoperated collection costs falling from ~$340/hour in 2024 to ~$118/hour173, and the people holding the cameras and leader arms earn far less: the companies profiled below pay collectors anywhere from a $1/hour base (gig workers in India) to $30/hour (app-based collectors after bonuses). TechCrunch's blunt verdict on the sector: "collecting robot training data is dirty, unglamorous work."174
Raw hours are also worthless until labelled. Footage must be annotated with task boundaries, success and failure flags, hand pose and segmentation before a policy can train on it, and imitation learning is unusually sensitive: a small fraction of fumbled demonstrations disproportionately wrecks the learned policy, because the robot faithfully copies the mistakes. Today that curation is done by human annotators or by prompting vision-language models, both slow, expensive and inconsistent, so label-free curation has become its own research niche (CUPID's influence functions175, Demo-SCORE's online self-curation, GiB's "Good in Bad" subtask sifting).
The sharpest result in that niche is the Fujitsu Research and CMU "periodic workflow" paper176 (Fan Yang and colleagues at Fujitsu, with CMU's Yonatan Bisk and Graham Neubig; co-author Quanting Xie is also the Origami Robotics founder), a WACV 2026 oral, a slot given to roughly 87 of 2,458 submissions, about 3.5%.
The method contains no neural network at all: K-means tokenisation of pose and trajectory data (K of 6-14), a 2D FFT to estimate the period of repeating work, Dynamic Time Warping to rerank candidates, and a modified Needleman-Wunsch sequence alignment borrowed from DNA bioinformatics to find cycle boundaries.
On its own first-of-its-kind benchmark (580 multimodal sequences of 3D body and hand pose, indoor RFID/VIO tracks and outdoor GNSS traces, spanning factory production, exercise and shuttle routes at 5-8 repeats each), it beats GPT-4o zero-shot on all three tasks it defines: period detection (error 0.025 vs 0.038, boundary tIoU 0.937 vs 0.882), task-completion tracking (error 0.106 vs 0.388) and procedural-anomaly detection (tIoU 0.423 vs 0.173).
It also crushes classical unsupervised baselines (ClaSP, 1D-FFT), matches or beats supervised anomaly models like ASFormer and FACT with zero labels, and runs ~5x faster than GPT-4o and ~10x faster than Gemini-2.5 Pro, with code, dataset and a demo Space all public. The caveat stays attached, though: the authors frame this as human-activity understanding for factories, exercise and shuttle routes, not robot imitation learning. The demo-curation application is our extrapolation, not their claim.
Speed is a quality axis most of this pipeline doesn't even grade, and Amplibotics founder James Wigglesworth has the bluntest take on it I've heard. The garbage-in property has a second face: pace. Train on demonstrations recorded at human teleop speed and you get a policy capped at human speed, when the bar that matters is the welding arm on a car line that already outruns any person.21
A human is not a good bar, and note that this cuts against Fan's physical Turing test: a policy trained at human pace could pass that test, indistinguishable from human labour, and still lose to the welding arm on cost per part.
Amplibotics' fix is to gamify the operators. Pilots get a score and a grade on task-completion speed, and the best of them fold shirts faster through the rig than with their own hands, because a gripper doesn't have to move the way a human arm does.21 The company recruits accordingly: a May 2026 partnership with SkillWaze trains collegiate esports players as industrial robot pilots.177 Algorithms that retime slow demonstrations are appearing (he names none), and by his own account they're still new.
The Data Companies
So who actually runs the factory? A cluster of young, mostly YC-backed companies now sells collection, labelling or both as a product, all working from the same thesis: LLMs got smart on internet text, robots have no equivalent corpus of physical action, so somebody has to manufacture one. Reading five of them together is the best available map of the layer.
Asimov (YC W26) is the purest expression of the egocentric bet; its site's literal tagline is "Real-world human movement data for humanoid robots."178 It ships a free phone plus lightweight head mount to collectors anywhere in the world (no technical background required), who record themselves cooking, cleaning, organising, doing desk work, gardening, repairs and errands through a mobile app.
A proprietary pipeline auto-blurs faces and PII, records no audio at all, and turns the footage into annotated datasets (3D body pose, depth maps, semantic labels, activity segmentation) sold B2B to frontier robotics labs, with B2B pricing undisclosed. It operates across three continents, claims "thousands of hours a day," and frames its market with the line that 8 billion people generate 16 hours of real-world physical activity every day.
Two details make Asimov worth studying. First, the labour design: collectors earn a $5-15/hour base rising to $30/hour after their first five hours, plus volume bonuses (verified against Asimov's own YC job listing), while restaurants, hotels, cleaning firms and warehouses get pitched "zero-cost labor", meaning Asimov covers worker salaries during normal duties in exchange for the workplace data, with revenue sharing for larger partners. It even runs a captive Bay Area cleaning subsidiary, Asimov Cleaning, doing 100+ weekly cleanings as its own data farm.
Second, the founders really are early-career: Lyem Ningthou and Anshul Verma are both UC Berkeley undergraduates graduating in 2026, running a company of three to five people.
Ningthou built ML and data pipelines for the US Air Force and was a founding engineer at Blume (YC W24), with FLIP, USDA and Numerator also on his LinkedIn; Verma did a ~4-month SWE internship at Scale AI, an Amazon internship, and ML research at BAIR.
Worth stating plainly: those were internships, not senior tenure, however the pitch reads. The only funding on record is a $500K entry dated January 2026, which is the standard YC deal179.
Human Archive (YC W26) pitches itself as "the Common Crawl for human sensorimotor intelligence" and goes after richer signals than video alone180. It designs custom wearable rigs (head-mounted RGB and RGB-D cameras, stereo-IR depth, tactile force gloves, full-body IMU suits, wrist cameras) and puts them on gig and service workers, starting in India, across home cleaning, hotels, restaurants, quick-commerce, construction, factories and agriculture.
Raw capture flows through internal QA, face-blurring and annotation (task labels, hand tracking, object segmentation, SLAM, 3D pose, 3D MANO hand reconstruction, tactile force maps) into two products: HA-Multi, the full fused multimodal stack, and HA-Ego, a lighter RGB-plus-wrist-camera stream.
The numbers are substantial: capacity up to 8,000 hours a day, 1,000+ headsets deployed plus 50+ heavier devices across 7+ custom hardware types, 120-125+ signed partnerships, tens of thousands of hours collected, and a 25-person operations team, with a stated scaling target of 50,000+ contributors181. The consumer mechanics are clever and slightly uncomfortable: at the door, the app offers a discounted service in exchange for recording consent, or full price with no recording, while workers are compensated separately at a $1/hour base (inc42 reports a $1-10/hour average)182.
The $8.2M seed (announced May 2026) was led by Wing Venture Capital with NVP Capital and YC, plus angels from OpenAI, NVIDIA, Google, Meta, Mercor, AfterQuery, BAIR, SAIL and DoorDash AI Research, and Brad Boa. One marketing claim fails the sniff test: the launch billed "the world's largest multimodal robotics dataset," which isn't true on hours: rival Build AI's Egocentric-1M, at roughly a million hours, dwarfs it. (For calibration, Ego4D's academic ~3,700-hour scale is what all of these companies are racing past.)
The four founders are all about 20 years old, UC Berkeley and Stanford dropouts based in the San Francisco Bay Area: CEO Raj Patel (Stanford), Rushil Agarwal (operations and data pipelines), Samay Maini (multimodal datasets and tactile hardware) and Shloke Patel (robotics and mechanical systems, reportedly Raj's cousin).
Hub (YC Spring 2026) sells an "API for real-world training data": a lab submits a brief, say thousands of hours of first-person household video across many countries, and Hub sources it from an advertised network of 500,000+ individual contributors and verified businesses, running collection, automated processing and human-in-the-loop QA with delivery "in days, not months"183.
It spans four modalities (egocentric video, standard video, images, audio) in both catalogued and custom-scoped modes, priced by modality, volume, coverage and complexity, with no public price list. Even the country count is inconsistent between its own site ("100") and its YC page ("150+").
Hub carries the most caveats of the five, and they're worth spelling out because they teach you how to read a data-vendor pitch.
The advertised inventory (~54,000 hours of egocentric footage across 84 environments, ~2.4M image frames across 38 verticals, ~70,000 video clips totalling ~18,000 hours, plus "80,000 hours of audio in 47 languages") is entirely self-reported, and the "under two minutes" delivery and "itemised pricing within the hour" claims appear nowhere in the company's own public materials.
The "paid in USDC stablecoin" story belongs to an earlier incarnation of the company; current field-operator job posts in Brazil advertise plain cash at $100-2,000 a month. And the named customers (Google, Adobe, DeepMind, Rime, LiveKit) trace only to Hub's own LinkedIn posts, with no case studies or customer-side confirmation: plausible at pilot scale, at best. The self-reported traction line is revenue agreements scaling "from 5-figure to 7-figure" on successful delivery.
The strangest part is solid, though: a pivot. Hub raised ~$1.7M (pre-seed plus opening seed, September to October 2025) led by SwissBorg, Europe's ~$2B-AUM digital-wealth platform with over a million KYC'd users - but for a different, DePIN-style business (decentralised physical infrastructure networks: crypto-incentivised crowds contributing a physical resource), a "programmable-bandwidth supernetwork" upcycling users' idle internet connections into data streams184. The human-data-capture identity came later, so any "SwissBorg users as contributors" narrative conflates the old pitch with the new one.
Co-founders are Armin Kiani (CS and robotics background, previously at Freedelity, described as Belgium's largest consumer-data platform, and a 2020 Microsoft Innovation Award winner) and Tim Sprecher (co-founder and COO per the SwissBorg release), leading a team of ~10 founded in 2024 and based in Palo Alto. Their one distinctive claim holds up: because Hub captures data itself, it can attest provenance in a way scraped web data never can.
6thSense (YC S26) bets that touch is the one modality everyone else omits. Its tagline is "a nervous system for physical AI": in-house wearable tactile sensing gloves plus egocentric cameras, worn by a person doing ordinary chores, recording what the hands actually feel185.
The gap is real. Apple's EgoDex dataset offers ~829 hours with full 3D hand and finger tracking but no tactile channel186, EgoScale's 20,854 hours are likewise vision and pose only, and the scarcity persists because tactile capture needs custom pressure hardware, tight multi-sensor synchronisation and labelling labour that robotics teams don't want to build in-house.
Each 6thSense demonstration bundles roughly eight time-synchronised streams: fingertip pressure, egocentric RGB, depth/RGB-D, articulated 3D hand pose, IMU motion, wrist and scene cameras, optional narrated annotations, and success/failure labels. Tasks span dishwashing, laundry folding, vacuuming and drink prep, the company owns the full pipeline (hardware, sync, calibration, QC), and it also moulds custom tactile skin to robot hands as what it calls a "1-to-1 analog" of the arrays going into the next generation of hands.
Pricing isn't public: no dataset sizes, hours, sensor counts or named customers appear on the site, whose only call to action is to reach out if you're training a manipulation policy.
The funding trail is thin but real: a pre-seed of roughly 1 billion KRW (~$500K) around the YC announcement, reported in Korean press187.
The four founders are James Baek (CEO; Georgia Tech background), Matt Wulff, Ronak Agarwal and Alex Hyungwoo Noh (UChicago), and YC's materials say the team comes from Tesla, Mecka, Samsung, DoorDash and Amazon. Mecka is a name worth knowing in its own right: it raised a $60M Series A in June 2026 to train robots from body-sensor and iPhone data188.
DeepAware AI (YC S25) is the odd one out: not a collector but a general store, trading as the Robotics Center of Silicon Valley (SVRC) and pitching the full physical-AI stack from one vendor189.
Line one is hardware: 60+ robot platforms to buy or lease with ~72-hour US delivery or same-day San Francisco pickup, from a Unitree G1 humanoid at $18,400 (a price that as of August 2026 lives only in the store's markup, on a listing still marked "coming soon") and a Go2 quadruped at $3,790, through an OpenArm 1 at $6,000, a Wuji dexterous hand at $16,000, a Wuji teleop glove at $5,400 (tariff and shipping included) and a PaXini Gen3 tactile sensor at $1,299.
Line two is a data marketplace advertising "42 commercial datasets, 20,000+ hours, 11M+ episodes, 30+ countries" under perpetual non-exclusive commercial licences: an Embodied AI Bundle of 5.35M episodes across 7 robots (226,300 hours) for $120,000, a KUAVO humanoid set (1.7M episodes, 25,000 hours) for $45,000, a Titanmi TM-01 medical-teleop set (110,000 episodes, 1,200 hours) for $12,000, and smaller sets at $4,800-8,500.
Keep your calculator out, though. The marketplace's own headline of "20,000+ hours" is smaller than the 226,300 hours claimed by that single Embodied AI Bundle: the numbers are internally inconsistent, so treat every aggregate stat from this vendor with caution.
Lines three and four round out the stack. Custom teleoperation collection runs from a Pilot (20 demonstrations, $2,500, one to two weeks) through a Campaign (100 demonstrations, $8,000, two to six weeks) to enterprise custom work, at $8-35 per episode, delivered in LeRobot, HDF5 or RLDS formats using rigs that include ALOHA-style leader-follower arms, Meta Quest 3 VR, a 6-DoF SpaceMouse and SenseGlove Nova 2 haptic gloves.
RL environments and evals ship in Isaac Sim, MuJoCo, Genesis and Gazebo, bundled with benchmarks (LIBERO, CALVIN, RLBench, Meta-World, Colosseum, Google Robot) and eval harnesses for OpenVLA, Octo, RT-X, pi0, Diffusion Policy and ACT, priced at Starter $5,000 (two weeks), Suite $25,000 (four to six weeks) and Digital Twin custom (six to eight weeks).
Physically, SVRC runs a San Francisco HQ at 90 Welsh St with a showroom, teleop control room, training bay and repair shop, plus a Boston site at 125 Western Ave in Allston near MIT and Harvard. It's an NVIDIA Inception member, founded 2025, and its verbatim mission is to "make physical AI development practical - absorb the complexity between research demos and real-world robot deployments."
Its self-published "State of Robotics 2026" report pegs the global market at ~$38B (a claimed +34% year on year), tracks 12 commercial humanoids, and puts VLA adoption at ~40% of new deployments173; useful numbers, but remember they come from a vendor.
The founders are Jerry Huang (CEO, Harvard) and Stela Tong (Stanford), second-time founders running a four-person team that's hiring across ML, robotics and business development. Their history is winding: roughly three years building a factory energy-optimisation startup together, work on industrial energy systems for Siemens and Sumitomo, and Crunchbase still describing DeepAware as AI-driven automation for data centres. The trajectory reads as factory energy, then data-centre automation, then robotics. The claimed differentiator against data-only rivals such as Encord is being the vendor that also sells the hardware and runs the sim-to-real loop end to end.
These five aren't alone. The biggest broker by funding is XDOF ($70M raised, profiled properly in Deployment)174; Cortex and EgoData work the same lane.
Step back and the shape of the layer is clear: pretraining robots on human demonstration data is frontier work, but the collection business underneath it is commoditising fast, with many near-identical startups competing on scale, price per hour and annotation quality, and margins likely to compress. The durable lesson is that a robot foundation model is only as good as the diversity, volume and quality of the demonstrations behind it, and that a paid human workforce, some of it at $1 an hour, is what actually produces them.
Learning
This is the BC → ACT/Diffusion Policy → VLA pipeline described above, glued together in practice by Hugging Face's open-source LeRobot library. The lazy shorthand is "the PyTorch of robotics", and I used it myself until I read the source: it's right about ambition and wrong about shape. PyTorch is a tensor library; LeRobot is closer to Transformers plus Datasets plus a hardware driver stack, welded together and pointed at one specific workflow - puppet an arm, record what happened, train a policy on it, run the policy back on the arm.
Of the models named above, everything except OpenVLA ships inside it (ACT, Diffusion Policy, pi0, pi0.5 and GR00T are all there), and the other fourteen registered policies double as a census of who's shipping robot policies right now.
From the pi lineage comes pi0-FAST, the autoregressive variant that emits discrete frequency-space action tokens, plus Hugging Face's own SmolVLA (covered in Inside the Models). The pre-VLA generation survives as TD-MPC (model-predictive control in a learned latent space), VQ-BeT (a behaviour transformer over vector-quantised action codes), and the SAC-style Gaussian actor powering HIL-SERL's human-in-the-loop RL.
Then the 2025-26 VLA wave: AllenAI's MolmoAct 2, the multitask diffusion-transformer policy behind Boston Dynamics' Atlas large-behaviour-model work, X-Square's WALL-OSS and EO-Robotics' EO-1 (both Qwen2.5-VL backbones with flow-matching action heads), Shanghai Jiao Tong's Evo-1 (the same recipe on an InternVL3 backbone), VLA-JEPA (a V-JEPA2 world model feeding a flow-matching head), FastWAM, X-VLA (ICLR 2026's soft-prompted cross-embodiment transformer), and LingBot-VA, an autoregressive video-action world model.
The compute bill for joining in is smaller than you might guess. Inference on an open VLA (pi0, OpenVLA, GR00T-N1-2B) fits on one consumer GPU: openpi quotes over 8GB of VRAM for inference and over 22.5GB for LoRA fine-tuning, which makes a 24GB RTX 4090 the community sweet spot, while full fine-tuning wants 70GB+ and therefore an 80GB A100/H100 or a cloud rental at ~$2-4/hr.16
The repository started life in JAX, gained PyTorch support in September 2025, and drives ALOHA, DROID, LIBERO and UR5 robots out of the box. The standard hobbyist path runs through this stack: collect ~50 teleoperated demos on a cheap arm from Getting Hands On, then fine-tune pi0 or ACT.
When real data is too scarce, teams fall back on sim-to-real: train in a physics simulator (NVIDIA Isaac Sim, DeepMind's MuJoCo, the newer Genesis), then apply domain randomisation - randomise textures, lighting, camera pose, mass, friction and latency, and inject sensor noise - so the policy survives the "reality gap." Increasingly the simulator is a photorealistic digital twin reconstructed from just 10-20 phone photos via 3D Gaussian Splatting (a photorealistic 3D-reconstruction technique),72 with randomised backgrounds and generated "digital cousins" of the scene supplying the diversity that real data would otherwise have to.73
Evals / observability
You can't unit-test a neural policy or prove it correct, and labs now generate thousands of robot attempts ("rollouts") per week, far more than humans can watch. Yet every training run, benchmark, and RL step needs a "did it succeed?" label for each attempt, and today that label mostly comes from slow, expensive human review or from paying a frontier vision-language model to act as judge. Robotics compounds this with a reproducibility mess: inconsistent hardware and inconsistent definitions of "success" mean results from different labs often can't be compared at all.
The observability half of the layer is just as awkward. A deployed robot emits multimodal, time-synchronised, high-bandwidth streams (camera video, point clouds, dozens of sensor topics) recorded on a bandwidth-constrained edge device, and debugging a failure means replaying all of it in exact time alignment. Conventional tooling (Datadog, Grafana, Snowflake) assumes small, structured, cloud-native telemetry and simply breaks here, so serious robotics companies like Cruise and Waymo built bespoke internal recording and visualisation stacks long before anything existed off the shelf.
So a new layer sells automated verification and robot-grade observability - the robotics analogue of the LLM ecosystem splitting into models, eval tools, and reward models. Measurement is becoming an AI product in its own right: training the robot brain is only half the problem; knowing whether it worked, at scale and without humans watching, is the other half.
The Tooling Companies
Foxglove is the flagship of the observability side: the "multimodal data platform for Physical AI," or per the live homepage tagline, "Scale Physical AI from prototype to production."190 It captures a robot's data at the edge, syncs it to cloud or on-prem storage, indexes it for large-scale search, and gives engineers a browser or desktop app to scrub through recordings frame-by-frame in 3D.
The site lists six product areas: Visualization (a unified debugging workspace over live and recorded data, evolved from the open-source Foxglove Studio, itself a fork of Cruise's "Webviz"), Data Management (ingest, index, search), Fleet (secure live remote connection to field robots), the open-source MCAP log format from the middleware layer above (ROS 2's default bag format since May 2023191), plus Integrations (SDK/API/webhooks) and Extensibility (custom panels, scripts, converters).
The company exists because founders Adrian Macneil (CEO) and Roman Shtylman (CTO) hit this pain first-hand at Cruise and left in early 2021 to productise it192. The legacy standard, ROS 1 "bag" files, was slow, not self-describing (you needed the exact message definitions to read old data), and poor at mixing serialisation formats.
Macneil had been the first Director of Infrastructure Engineering at Cruise, where he built its first multimodal data-visualisation platform, and before that the first Director of Engineering at Coinbase, having founded and sold the e-commerce startup Exp;resso in 2014; he's a New Zealander (Massey University), now SF-based. Shtylman is also ex-Cruise; the Coinbase chapter is Macneil's alone.
MCAP itself deserves a beginner's minute: an open, self-describing, append-only container that stores many timestamped channels of any format (ROS 1/2, Protobuf, JSON, FlatBuffers) in one file, with LZ4/Zstd compression and time/topic indexing for fast seeking, and libraries in C++, Go, Python, Rust, Swift, and TypeScript. It's the black-box flight recorder of robotics, and the reason you can replay a failure just as the robot experienced it.
The business model is bottom-up SaaS, and the pricing is public (as of July 2026)193: a Free tier ($0; 10 GB storage, 3 users, 5 devices, 1 project), Pro at $20/month plus usage (1 TB storage then usage-based, 3 users then $42/user/month, 5 devices then $20/device/month, 3 projects), custom-priced Enterprise (self-hosted/on-prem, SOC 2, forward-deployed engineers), and a free Academic tier for .edu/.ac addresses (1 TB, unlimited users, 10 devices, 3 projects).
Foxglove raised a $40M Series B in November 2025 led by Bessemer Venture Partners (with Eclipse and Amplify), taking total funding past $58M190194, and the customer list (Amazon, Anduril, NVIDIA, Shield AI, Dexterity, Chef Robotics) reads like the space's who's-who, with Dexterity reporting over 20% of dev time saved and $150,000 a year in tooling costs avoided.194
The wider team includes alumni of Cruise, Aurora, Amazon Robotics, Stripe, and Coinbase; headcount sits somewhere between 50 and 88 depending on which database you believe. Competitors span Rerun (Rust-based open-source visualisation), RViz (the free built-in ROS visualiser), and platform players Formant, Sift, Roboto, Model-Prime, Encord, ReductStore, and Bedrock.
Ember Robotics (YC S24) is the same thesis pushed to the edge, a "Datadog for robots," but engineered around the fact that robots, unlike servers, routinely lose their network195. Its diagnostics run on the robot itself, cache telemetry locally during connectivity outages, and replay and sync automatically on reconnect, because cloud-based monitoring goes blind the moment something breaks in the field.
The platform continuously collects health data from a robot's sensors and computers, flags anomalies in real time, and gives engineers a dashboard to trace failures across logs, telemetry, and system events, layered with release tracking for software and firmware, hardware reservation and scheduling for shared fleets at half-hour precision, and enterprise controls (SSO, audit logs, RBAC, per-device auth).
Ember started narrow, as a plug-and-play camera-diagnostics tool for Intel RealSense D400 cameras over ROS (out-of-the-box tracking of disconnections, dropped frames, and time-sync errors, plus image-quality analysis and alerts, configured via a config file), then broadened into fleet observability covering cameras, IMU, LiDAR, CPU, and memory, with claimed scaling "from 1 to 10,000 robots."
It ships as standard Linux packages with an API-first design and built-in plugins for RealSense, Luxonis, and Orbbec cameras and for ROS and eCAL middleware, targeting robotics/autonomy, aerospace and defence, automotive/ADAS, industrial and energy, and semiconductor equipment. The pain it monetises is structural: a single camera glitch cascades through perception, control, and safety; lean hardware teams treat reliability tooling as secondary, so they either debug reactively or waste months building custom diagnostics because existing tools don't fit.
The founders are two former Tesla Autopilot engineers, both women, who founded the company in 2024 to productise the kind of internal diagnostics tooling that previously only giants like Tesla built in-house (Ember is an NVIDIA Inception member). Shivani Mouleeswaran (CEO) came via Tesla Autopilot, Simbe Robotics' retail-inventory robots, and a NASA internship, with a background in real-time diagnostics, sensor calibration, and field-service alerts; Ritika Shrivastava (CTO) was a computer-vision engineer on Autopilot, took her UC Berkeley EECS BS (2021) and MS (2022) while researching and teaching computer vision and robotics, and interned earlier at Microsoft and GoDaddy196.
Their shared Tesla camera-stack work is the origin story for the camera-first product. The team is about two people in San Francisco, and pricing isn't published; expect an enterprise, sales-led model.
On the evals side proper sits Instance Labs (YC S26), whose pitch is "ground truth for robot learning."197 You give it a task description plus camera footage of a robot attempt; it returns a pass/fail verdict with supporting evidence and a grounded subtask breakdown with live captions, and it can segment one long video into separate rollout attempts.
It ships as an API (POST /v1/verify), a Verification Console, and a public demo that takes drag-and-drop .mp4 uploads (one clip per rollout, multiple allowed) or live phone-camera recording. Labs use it to clean training data, run trustworthy evals, and produce RL reward signals without human labelling.
The engine is a fine-tuned vision model that runs locally on a single GPU at roughly 2.0 seconds per rollout with no per-call cost, and the self-reported benchmarks are striking: macro F1 of 0.76 versus Claude Opus 4.8's 0.53, claimed wins on 8 of 8 datasets at about 2.5x lower latency (2.0s versus 5.2s via API), measured on 10,000+ held-out human-labelled episodes across 8 benchmarks and 7 robot platforms198.
Treat the numbers as vendor-reported, but the shape of the claim (a small purpose-built verifier beating a frontier generalist at judging) is the "VLM-as-a-verifier" pattern: robotics' analogue of LLM-as-judge and reward models. No pricing is published; the local demo is free to run, and the site positions Instance as a trusted "verification layer", which implies a paid product on top.
The founders are Claire Mao (CEO; MIT maths and computer science; previously NASA JPL, BCG, and the MIT Media Lab) and Lucy Cai (CTO; MIT CS and AI; robotics research at MIT's LIS Lab; previously SpaceX and Amazon AWS), MIT computer scientists and friends since middle school, running a team of about four197. No funding round beyond YC backing is disclosed.
The same software-on-other-people's-robots bet extends a layer up the stack. Lightberry (YC Fall 2025, San Francisco) sells the robot's operational brain rather than the measurement of it. Its product is a cloud-backed runtime/SDK that gives humanoid and service robots a personality plus social intelligence, fusing perception, navigation, speech, and dialogue into one configurable "social brain" that decides when to listen, speak, or move autonomously around people199.
The signature feature is voice-programmability: a non-technical user configures behaviour and personality by talking to the robot, and writes no code at all. The company's own tagline is "We give robots personality. Robots now talk, see the world, and learn by interacting with people!"
The argument for Lightberry's existence is that OEMs are shipping capable bodies (Unitree alone ships 1,000+ robots a month) that are "demo-ready, not deployment-ready," and, in the company's own framing, "slapping ChatGPT realtime on it is not enough; every manufacturer tried that already." The robots target people-facing roles: reception, retail and sales, events and conferences, offices, and homes. Publicly named deployed instances include Koko, Synthia, Alfred, Blake, Minnie, and Bradford, and one called BeachBoy co-hosted YC Demo Day as MC (it even has its own X account). There's also a "Robot Store" selling supported robots with Lightberry pre-installed.
A supported-hardware caveat matters here. The YC launch page listed the Unitree G1, Fourier GR-2 and N1, Booster T1 and K1, and High Torque Mini-Pi and Mini-Pi+, but as of mid-2026 the homepage headlines an exclusive Unitree partnership ("All robots ship with Lightberry built-in"), so treat the multi-OEM list as launch-era breadth that may have narrowed.
That partnership, beginning January 2026, is the traction story: investors estimate it could reach roughly $10M ARR on full deployment (per Paulina Szyzdek of HardTech VC, and Singularity Capital)200, and the backers include Y Combinator, Liquid2, a16z Scout, Singularity Capital, and angel operators from NVIDIA, Meta, and Google.
Lightberry doesn't publish its own per-robot or subscription pricing; the price anchors on record are the hardware itself (Unitree G1 from ~$35,000 per the investor note, well above the ~$16k street price quoted elsewhere in this post, and Booster T1 at ~$20,000-$30,000, per Singularity Capital) and a hiring signal: a Senior/Staff Voice AI engineer role at $130K-$280K plus 0.25-1.00% equity on the YC jobs page.
Even the founder count is fuzzy: the YC company page lists three co-founders while the launch page and some coverage name only two.
Ali Attar is a repeat YC founder (SigmaOS, YC S21, a high-performance macOS browser used by 100,000+ people handling ~75,000 LLM requests a day; earlier ventures Loop and Dromadis Health) with a consumer-UX and human-robot-interaction background. Stephan Koenigstorfer is a CERN-trained physicist holding a PhD, MBA, and MSci, with prior robotics and software roles at Bucket Robotics and Stack AV, and a 2024 paper "Bringing CERN to classrooms" in The Physics Teacher. Stephan Wolski (Head of Robotics) was Remote Guidance Tech Lead at Argo AI and an early engineer at Stack AV.
For a beginner, Lightberry is the cleanest possible diagram of the layered stack: the OEM sells the body and the low-level control (balance, locomotion, obstacle avoidance), while Lightberry lives only in high-level cognition, a continuous listen→think→act loop with an "internal monologue" borrowed straight from LLM-agent architectures.
Deployment
Today's "autonomous" robots usually aren't reliable enough to run unsupervised, so a human is kept in the loop as fallback via teleoperation. It comes in three flavours, from most to least human.
The first is full teleop, where an operator wearing a VR headset puppets the robot's every move: this is how Tesla's Optimus poured drinks at its Oct-2024 "We, Robot" event, where one bot cheerfully admitted it was "assisted by a human" and Tesla stock fell ~11% afterwards; a Dec-2025 Optimus demo tumble was again attributed by observers to a teleoperation glitch.201
The second flavour is remote assistance, or "fleet response": the robot drives itself but phones a human when confused. Waymo runs at least ~70 fleet-response agents supervising ~3,000 vehicles across ten US cities, answering questions like "is this cone a lane closure?" and suggesting paths, but they can't take control, because the Waymo Driver stays in command and can reject their input.202 The third is supervisory monitoring: one person watches many mostly-autonomous robots and steps in only on failure.
What makes all of this hard is latency: the round-trip delay from operator input to robot to camera and back to the operator's eyes, known as "glass-to-glass". Below ~40ms feels instant; above ~150ms operator trust erodes; above ~250ms precise control becomes unreliable and head-tracked VR induces motion sickness, because your head moves but the picture lags. Typical teleop stacks run 150-400ms, so sub-200ms over the public internet is the working threshold, and operators sit geographically close or on dedicated links to hit it.103
The numbers in the wild match. Waymo's remote assistance runs at ~150ms domestically and ~250ms with overseas operators, and Feb-2026 reporting revealed some of its remote advisors are as far away as the Philippines, restricted from issuing "special commands" and forced to escalate to trusted domestic centres.203 At the other extreme, Adamo claims a sub-40ms stack (3-5ms encode, 15-25ms transit, 5-8ms decode) and cites 30-40% higher operator throughput and roughly halved error rates from it.204
Teleoperation-as-a-service (TaaS) is the business of renting out this whole human-behind-the-robot capability (trained operators, the low-latency streaming stack, safety kill-switches, fleet-monitoring dashboards) so robot companies don't have to build it themselves. Because operators are a labour cost, the work is offshored to lower-wage regions: reported rates run roughly $22-$55 per hour for operators in India, the Philippines and Eastern Europe, a labour-arbitrage layer sitting quietly beneath the "AI".205 A skilled operator produces ~5-50 demonstration episodes per hour, with quality degrading as they fatigue.
Teleop is also the data pump. Every teleoperated action is recorded as a labelled demonstration through the robot's own body, and production pipelines typically anchor a new robot with 500-2,000 high-quality teleop episodes on the exact target hardware before autonomy improves. So the same human is simultaneously the safety net that lets you deploy unreliable robots today and the factory manufacturing the training data that'll eventually remove them - and whoever owns that teleop-plus-data layer captures value across every robot maker, which is why $70M+ has flowed into pure-play data/teleop startups.
The flagship is XDOF, which exited stealth in June 2026 with $70M from Thrive Capital, Spark Capital, Andreessen Horowitz, Lux Capital and WndrCo, roughly 60 employees, ~20 customers including frontier AI labs, and ABC-130K, the claimed largest open-source bimanual manipulation dataset at ~130K trajectories plus 300 hours of simulation and 100 hours of evals, co-released with Berkeley.206
Its CEO and co-founder is Philipp Wu, first author of the GELLO paper from Data collection: the man who designed the field's favourite cheap teleop leader now runs its best-funded data broker.
Around it sit Scale AI and the data-labelling workforces (Cogito, Shaip, iMerit), Adamo on the latency stack, and Extend Robotics, HaptX and Sanctuary on immersive XR and haptic operator rigs; for DIY research teleop, a ~$500 Meta Quest 3 headset is the de facto operator interface. Teleoperation itself is decades-old commodity tech, and the managed-service wrapper of operators, streaming, safety and the data flywheel is the real 2025-2026 land-grab.
The consumer-facing end of the same pattern is 1X's NEO home humanoid: $20,000 outright or $499/month, pre-orders opened late Oct 2025 with US Early Access shipping in 2026, ~30 kg, up to 70 kg lift and 25 kg carry on a soft tendon-driven body.207 Hard chores are done by 1X employees teleoperating via Meta Quest 3 headsets from the company's California HQ in "Expert Mode" (commentators openly argue that teleop, not autonomy, is the real path for NEO).
Its July-2026 hand upgrade shows where the hardware is heading: 25 force-controlled degrees of freedom (22 in the fingers and palm, 3 in the wrist), backdrivable, with tactile sensing and IP68 waterproofing.208
Understand what you're buying, though: a data-collection device whose remote operators can see through its cameras and hear through its mics. Person-blurring, owner-designated no-go zones and an owner-approval gate on operator take-over are opt-in mitigations, not guarantees, and CEO Bernt Bornich has been refreshingly direct about the incentive: "If we don't have your data, we can't make the product better."209
Teleop therefore drags in questions of candour (which "autonomy" demos are puppeted), safety questions (who holds the kill-switch, over what network), privacy questions (a stranger seeing inside your home through NEO's eyes) and labour questions (gig operators in the Global South as the invisible substrate of "AI robots"). The Frontier Map returns to this layer as a business, including the safety engineering and one instructively unverifiable startup pitching this as a service.
That's the whole stack: actuators → compute → sensors → middleware → data collection → learning → evals → deployment-with-teleop-fallback. Almost every company in this post occupies just one of these layers.
bar = number of companies in this corpus at that layer · click a layer to inspect
The Classical Core
Say you've ordered an SO-101 pair (the ~$100-per-arm open-source hobby arms of Getting Hands On) and a used UR10e (Universal Robots' 12.5 kg-payload industrial cobot from Two Eras), and you plan to fine-tune pi0.5 through openpi (Physical Intelligence's open-source repository for it). You may reasonably ask: if a neural network learns the mapping from cameras to motor commands end-to-end, why learn hundred-year-old geometry and a control law from the 1920s?
This section answers that, then builds the geometry, kinematics, and control from absolute zero (no physics or robotics assumed) up to the point where you can read a URDF, debug a bad pose, understand why the UR10e sometimes lurches near a "singularity," and know what layer your learned policy is sitting on top of.
The single sentence to carry through everything below: a learned policy replaces the hard, brittle middle of classical robotics - designing controllers and hand-coding perception-to-action - but it doesn't replace the coordinate bookkeeping (frames), the joint↔Cartesian bridge (FK/IK), the calibration (transforms), or the safety envelope (limits, singularities, force caps) that wrap it. Those stay hand-built, and getting them wrong is the most common reason a founder's VLA "doesn't work" when the model itself is fine.
Why It Survives
A modern vision-language-action model (pi0.5, and its cousins ACT and SmolVLA - Hugging Face's small open VLA) is the middle slice of a sandwich. It ingests camera images, a language instruction, and the robot's proprioception (its own joint readings), and it emits a tensor of numbers every control tick. The load-bearing fact is that both ends of that tensor live in classical-robotics coordinate systems that you must define, calibrate, wire up, and bound.
Concretely, four things the network can't do for you:
-
The numbers coming out mean something specific. pi0.5's action labels are, per the openpi/pi0.5 documentation, "end-effector deltas OR joint velocity commands, depending on the embodiment."210 The action space is a per-robot configuration decision you make, not a fixed property of the model. (That general label space can be velocity or EE-delta; for the SO-101 concretely, openpi/LeRobot uses joint position targets - see PID and Control.)
If the head speaks end-effector poses, those poses are in the robot's base frame and must be converted to joint angles by inverse kinematics on every tick before any motor can move. If it speaks joint angles, no IK is needed. You have to know which world you're in.
-
The numbers going in must be consistent. The proprioceptive state you feed the policy (joint positions, or an end-effector pose) has to use the same frames and the same orientation representation at training time and deployment time, or the network silently learns garbage. This is probably the most common way robot policies fail in practice, and the failures are usually small: a camera renamed, images in a different channel order, normalisation statistics from the wrong dataset.
LeRobot's answer is worth knowing because it explains a whole subsystem of the codebase - the transformation pipeline between data and model is a first-class serialised object, saved alongside the weights and replayed identically in both directions, so the statistics live in
preprocessor_config.jsonrather than inmodel.safetensors. A checkpoint without its processor files is unusable: a bag of weights that'll drive your arm into the table. -
The camera sees pixels in a camera frame. To relate "the mug is at pixel (u,v)" to "move the gripper to (x,y,z)," you compose the camera's internal geometry with the transform between the camera frame and the robot base frame. That transform comes from hand-eye calibration, which is pure transform algebra (Frames and Transforms, Inverse Kinematics).211
-
A learned policy will happily drive the arm into itself, past a joint limit, through a singularity, or into you. Every guardrail - clamping joint angles, rejecting unreachable poses, damping near singularities, capping contact force via the UR10e's wrist sensor - is hand-written classical code wrapping the policy output. None of it is learned.
There's a pleasant asymmetry between your two arms that makes this concrete. On the SO-101, you can deploy a policy with almost no IK at all. The SO-101 leader and follower are identical arms; teleoperation is a direct joint-by-joint mirror. The leader streams its six values (five arm-joint angles plus the gripper) and the follower copies them one-to-one.
So the recorded demonstration actions are follower joint targets, training predicts joint targets, and deployment sends joint targets to the servos. Demo collection, training, and execution all speak the same six numbers; the classical burden collapses to conventions plus a one-time calibration.212
The UR10e is the opposite. It's an industrial arm you'll typically command in task space ("move the tool to this pose"), so any end-effector-space policy on it needs IK every tick, and you inherit the full singularity and branch-selection story. Understanding both cases, and why they differ, is the payoff of this section.
Bridges From University
Before Frames and Transforms builds frames and transforms "from scratch," a correction is owed: if you did a year of university maths, it won't be from scratch for you. The four pillars this section rests on (frames, kinematics, the Jacobian, and control) each have an algebraic core, and a typical first-year engineering or maths course derives every one of those cores, usually with the same worked examples.
I learnt most of these at university, and my worked notes live on my maths wiki. The "notes" links below point into them, so you can check each derivation in full.
Robotics didn't invent this machinery; it renamed it and bolted geometry onto it. What follows is the receipts: five bridges, each porting a block straight out of first-year maths, then naming the one or two new ideas robotics stacks on top.
Read this as a map of how little is actually new, so that when Frames and Transforms through PID and Control pile on unfamiliar words (, quaternion, manipulability, anti-windup) you can locate each on top of something you probably already own. The unfamiliar part is always small and always nameable.
Orthogonal matrices → rotation matrices
The rotation matrix that Frames and Transforms introduces first among its four rotation representations is, algebraically, a matrix first-year linear algebra characterises completely. A typical matrices chapter defines an orthogonal matrix as a real square matrix satisfying , equivalently , and then does the work of showing what that condition forces on the rows (notes: matrices: definitions and rules):
The conclusion, usually stated in words, is that "the row vectors of A are mutually orthogonal unit vectors," and that says the same of the columns. That sentence is the definition of a rotation matrix given in Frames and Transforms - "three columns are the (unit-length, mutually perpendicular) axes of the rotated frame." The structural fact was already proven; Frames and Transforms only reinterprets the columns as physical axes.
Most courses even do the canonical worked example: take the z-axis rotation matrix and prove it orthogonal row by row (notes, Example 10.2: matrices: definitions and rules):
Rename A to R and that is, verbatim, the elementary rotation about z and its inverse. The headline property Frames and Transforms asks you to "burn in" - , "the inverse of a rotation is just its transpose" - is the definition of orthogonal rather than a new fact to memorise, and that worked example (Example 10.2 in the notes) is a completed proof that inverting this particular rotation is free. When Inverse Kinematics and The Jacobian lean on "never generic-invert a pose," they're cashing a cheque first year already wrote.
The one thing a first-year course usually doesn't say: orthogonality alone is too weak. is satisfied by two families of matrices, those with and those with . The matrices are proper rotations; the ones are reflections (they flip handedness, turning a right-handed frame into a left-handed one).
Robotics wants only the first kind, because a physical rigid body can't be reflected into its mirror image by any motion. So it adds the second condition , and the set of matrices satisfying both and gets a name: , the special orthogonal group, the club of proper rotations. That single extra scalar equation, the determinant selecting +1, is the entire gap between the first-year matrices chapter and the rotation representation in Frames and Transforms. Everything else was already there.
Cross products & orthonormal bases → frames, torque, angular velocity
A coordinate frame, in the telling of Frames and Transforms, is "an origin plus a right-handed set of three mutually perpendicular axes." A first-year vectors course builds that object. It defines the unit vector (notes: vectors: definitions and notation):
the standard basis:
and proves the orthonormality relations that make it a frame rather than just three vectors (notes: vectors: dot product):
Unit length, mutually perpendicular - an orthonormal basis is a coordinate frame; the only thing Frames and Transforms adds is attaching one to each link, camera, and tool and tracking how they relate.
The "right-handed" qualifier that Frames and Transforms stresses - curl the fingers of your right hand from x toward y and the thumb points along z - is the cross product, stated geometrically (notes: vectors: cross product):
where, as the notes put it, points "in the direction given by the right hand rule... start at a and rotate our fingers toward b, our thumb will point in the direction of the cross product." The same chapter usually also proves anticommutativity and the cyclic products that encode right-handedness of the basis itself:
, with the noted sign flip , is "robotics is right-handed everywhere" written in algebra. And you probably met the cross product doing physics already, in the Lorentz force on a charged particle (the notes give it too):
Read that equation's shape, not its physics, and you have the two workhorse cross products of The Jacobian. Torque is - a moment arm crossed with a force, structurally identical to ; the wrench that the UR10e's wrist sensor measures and that converts to joint torques is built from this operation.
Angular velocity is the other one: when Frames and Transforms introduces the skew-symmetric matrix as "the matrix form of taking the cross product with ," it's invoking an object a first-year matrices chapter already defines and exemplifies (notes: matrices: definitions and rules):
Skew-symmetric matrices, defined by with zero diagonal, are the raw material for and, in Lie-group language, for - the space in which Rodrigues' formula and live in Frames and Transforms. First year knew the matrices; robotics gives them a job.
Euler's formula → quaternions
Quaternions are the one representation in Frames and Transforms that a typical engineering or maths syllabus skips entirely, and the thing to learn first because everything stores orientation that way. But nobody with a maths background starts cold: quaternions are the 3D sequel to a 2D story complex numbers already told in full. That story is Euler's formula (notes: complex numbers: Euler's formula):
and, crucially, the fact that multiplying two of these adds their angles:
Look at the expansion on the right. Those two bracketed real and imaginary parts are, entry for entry, the product of two 2×2 rotation matrices - the cosine-addition and sine-addition formulas are the matrix multiplication in disguise. That derivation proves, without naming it, that unit complex numbers compose rotations by multiplication. De Moivre adds that repeating a rotation is raising to a power:
and the same chapter shows these rotations preserve length and invert by conjugation (notes: complex numbers: definitions):
The one sentence most courses never write (the geometric punchline sitting one inch from everything above) is this: multiplying by rotates the plane by . The unit complex numbers () aren't merely numbers; they are the 2D rotation group , and the modulus-1 constraint is the 2D version of "unit length."
Now the sequel. A unit quaternion does for 3D what a unit complex number does for 2D, and the correspondence is line-for-line. Quaternion multiplication composes 3D rotations, the way composed 2D ones. The conjugate inverts: the quaternion identity is promoted one dimension.
The norm is multiplicative: is verbatim, which is why unit quaternions stay unit under composition. And SLERP (smooth interpolation along the shortest arc) is De Moivre's fractional powers made continuous: instead of for integer , you take a fractional power to blend two orientations at constant angular speed.
Three wrinkles appear only in 3D, and they're the entire content of "learning quaternions": rotations no longer commute (, unlike the commutative ), the angle enters as rather than (the half-angle in the Frames and Transforms formula), and that half-angle produces the double cover in which and name the same rotation. Master those three deltas and you have quaternions, because the algebraic skeleton is a complex-number chapter most STEM degrees already finished.
Change-of-variables Jacobian → the manipulator Jacobian
The Jacobian calls it "the one object to internalise beyond frames." If you did integral calculus, you already computed Jacobians and used them there, where the very same determinant appears as the factor that rescales area under a change of variables (notes: integral calculus: area integrals):
The 3×3 version is there too, for triple integrals:
It's the same object, reframed. In those integrals, the matrix of partial derivatives told you how a small patch of area or volume stretches as you pass through the coordinate map.
The manipulator Jacobian of The Jacobian is the matrix of partial derivatives of the forward-kinematics map, and it tells you how a small change in joint angles stretches into a change in tool pose - the "best linear map" , or in rate form the that The Jacobian asks you to burn in. Same matrix of partials; the inputs became joint angles and the outputs became a pose.
The reframing pays off most where the determinant vanishes. Integral calculus also proves the reciprocal relation between the forward and backward change of variables, with the Jacobian of the inverse (backward) map:
You already know, then, what happens as : the inverse factor blows up. In an integral, that was the substitution degenerating - the coordinate map collapsing a region to zero area, so the back-transformation became undefined.
In The Jacobian the identical arithmetic is the physical event called a singularity: the arm loses the ability to move its tool in some direction, , and because inverse kinematics runs the map backward through (or the pseudo-inverse ), the commanded joint velocities explode by exactly the the integrals already exhibited. When The Jacobian says "blows up near singularities" and Inverse Kinematics says IK "explodes," they're describing going to infinity. The damped-least-squares fix in Inverse Kinematics and The Jacobian is simply refusing to let that reciprocal run away.
The damped oscillator & phase plane → PID
This is the richest bridge, and the one a first-year differential-equations course practically gift-wraps. PID and Control builds PID control on a plant it never fully writes down - but a standard harmonic-motion chapter does, and solves it three ways. It derives the damped, driven oscillator from Newton's second law (notes: differential equations: harmonic motion):
where is the mass, the damping coefficient, the spring constant and the displacement, with and . This is the plant. Every robot joint under position control is this equation: a mass, a spring-like restoring term, a damping term, a driving force. The characteristic equation solves it:
and the sign of classifies the motion into overdamped, underdamped, and critically damped - the three behaviours a tuned controller chooses between. The underdamped case, the one that rings, is a decaying oscillation:
with . And the knife-edge between ringing and sluggishness is the critical damping coefficient:
Then, in a systems chapter, the same second-order ODE gets rewritten as a first-order system and analysed for stability through eigenvalues (notes: differential equations: 2D linear systems):
where is the trace of and its determinant; fixed points are then classified by into nodes, spirals, saddles, and centres (notes: differential equations: classification of fixed points). The worked complex-eigenvalue example in the notes, , is a stable spiral (a trajectory decaying inward while it rotates) which is the phase-plane portrait of the underdamped ringing above; same system, drawn in state space instead of against time.
Finally, nonlinear systems yield to linearising: find a fixed point, form the Jacobian of partials, and read off stability from its eigenvalues (notes: differential equations: nonlinear systems in the phase plane):
with the Hartman-Grobman theorem guaranteeing that this linearisation faithfully predicts the local phase portrait at a hyperbolic fixed point. Hold onto that: linearise-then-check-eigenvalues is how one now certifies a controller near an operating point.
Here is the dictionary between first-year ODEs and the three gains of PID and Control, term for term:
| PID term (PID and Control) | First-year ODEs | What it is |
|---|---|---|
| P-gain, "a spring," stiffness | in harmonic motion | proportional restoring term; raising raises the natural frequency |
| D-gain, "a damper," | in harmonic motion | damping term; raising raises toward critical |
| Underdamped ringing / overshoot | stable spiral (, complex ) in classification of fixed points | decaying oscillation, |
| Critically damped, no overshoot | in harmonic motion | ; tuning D to critical is choosing |
| Marginal oscillation | centre (, ) | undamped, , the stability boundary |
Read the tuning advice in PID and Control ("start with P, raise Kp until it just oscillates, add D to damp the ringing") through this table and it stops being folklore. Raising raises , which makes the spring stiffer and the response quicker but pushes the system toward the underdamped (ringing) regime from .
Adding D raises back toward , walking the eigenvalues out of the spiral region and toward the critically damped computed above. Choosing the D-gain that just kills overshoot is solving for . If you took a first-year ODE course, you already did the mathematics of PID tuning; PID and Control supplies the vocabulary and the closed loop.
The one term with no analogue in first-year ODEs is I, the integral. The plant is second-order (two state variables, position and velocity), and P and D act on exactly those two.
The integral gain introduces a third state, the accumulated error , which lives outside the phase plane entirely; it's what drives steady-state error to zero (the residual droop a pure spring leaves under a constant load like gravity), and it's also what can destabilise the loop and cause integrator windup, because adding a state moves the eigenvalues and can push them across the stability boundary you drew on the trace-determinant plane. Everything about P and D you can locate in first-year material; I is the new coordinate.
First-year courses often point ahead to this themselves. A typical first-order autonomous-equations section, introducing stability of fixed points, carries a forward reference to a later control course: these ideas, the notes say, are ones "you will additionally encounter in Process dynamics & control in Year II" (notes: differential equations: autonomous equations and stability). If that course never came the way it was billed, or came without any robots in it, PID and Control is that course - arriving late, by way of two robot arms, and finding every foundation it needs already laid.
Not on the Syllabus
None of this was in my university notes, and most degrees skip it too. Five things, and only five, that these bridges can't cover - each supplied fresh by the section it names:
- Homogeneous 4×4 transforms (SE(3)). Packing rotation and translation into one composable matrix, with the inverse. A first-year matrices chapter has the rotation block; translation-in-a-matrix is new. Supplied by Frames and Transforms.
- The det = +1 condition, SO(3). Orthogonality selected rotations-or-reflections; the determinant picks proper rotations only. Supplied by Frames and Transforms.
- Quaternions. Absent from most syllabuses - but a direct 3D lift of the complex-number chapter, per bridge 3. Supplied by Frames and Transforms.
- The PID law itself. , the feedback/error-signal framing, and the integral state with no analogue in your ODEs. Supplied by PID and Control.
- Laplace transforms and transfer functions. The frequency-domain machinery for pole placement, Bode plots, and stability margins - the analysis language PID and Control gestures at and the real content of that promised Year-II control course. Supplied by PID and Control.
Frames and Transforms
Everything is "in a frame"
A coordinate frame is an origin plus a right-handed set of three mutually perpendicular axes (x, y, z) attached to some object: the floor (the "world" or "base" frame), the robot's base, each rigid link, the last link (the "tool," or TCP = tool centre point), the camera, the workpiece, the table. "Right-handed" means if you curl the fingers of your right hand from x toward y, your thumb points along z; robotics is right-handed everywhere (graphics engines are often left-handed - a real source of sign bugs when you import assets).
A pose is the position and orientation of one frame expressed in another frame. Nothing in robotics has a coordinate without naming the frame it's measured in. Every real task - "put the gripper 5 cm above that cube" - is a chain of poses: world → robot base → each link → tool → object. Manipulation, perception, and control all reduce to converting a pose from one frame into another and chaining those conversions. If you internalise nothing else, internalise "frames all the way down."
A notation that'll save you hours: write a transform as , read "the pose of frame B expressed in frame A" (equivalently "B relative to A," or "the operator that converts a point's B-coordinates into A-coordinates"). A point known in frame B becomes, in frame A, . The indices cancel when you chain: . The inner B's meet and vanish, leaving A→C.
Trap: frame-of-reference confusion. Always name both frames ("pose of tool in base," never just "the tool pose"). Use the index-cancellation rule as a checksum: if the inner indices of two transforms you're about to multiply don't match, you're about to make a mistake - you probably need to invert one of them first.
There are four ways to write down a 3D rotation - matrix, Euler angles, axis-angle, quaternion - and the next four subsections take them in turn.
Rotation Matrices
A 3D rotation is a 3×3 matrix whose three columns are the (unit-length, mutually perpendicular) axes of the rotated frame written in the reference frame. The set of valid rotation matrices is the group ("special orthogonal group in 3D"), defined by two conditions: (the columns are orthonormal, unit length and perpendicular) and (right-handed, no mirror-flip). Properties worth burning in:
- The inverse of a rotation is just its transpose: . This falls straight out of . It's cheap, exact, and needs no numerical matrix inversion.
- Columns tell you where the body's axes point in the reference frame; rows tell you where the reference axes appear from the body.
- Composition is matrix multiplication, and order matters - rotations do not commute. .
- It uses 9 numbers for 3 degrees of freedom (the six orthonormality equations remove six), so it's redundant, but that redundancy buys you singularity-free behaviour and easy composition. This is the "safe" representation to store and compose internally.
Trap: rotating a vector vs. changing its frame. The same either rotates a physical vector to a new physical vector, or re-expresses one fixed vector from one frame into another. These are inverse operations. Decide which you mean every time.
Euler Angles
Euler angles (and the roll-pitch-yaw variant) describe an orientation as three sequential rotations about coordinate axes - e.g. yaw about z, then pitch about y, then roll about x. Three numbers, human-readable: that's their entire appeal. The costs, all of which bite beginners:
- Convention hell. There are twelve valid axis orderings (XYZ, ZYX, ZYZ, …), times intrinsic-vs-extrinsic, times active-vs-passive. "30° of pitch" is meaningless without the full convention. Two libraries with different defaults will silently disagree.
- Intrinsic vs. extrinsic. Intrinsic means each successive rotation is about the new, already-rotated, body-attached axes; extrinsic means all three are about the original fixed world axes. A useful fact: an intrinsic X-Y-Z sequence equals the extrinsic Z-Y-X sequence (reversed order). That reversal is a top source of "my rotation came out backwards" bugs.
- Gimbal lock. At a middle-angle singularity (pitch = ±90° in the ZYX convention), the first and third rotation axes align, you lose a degree of freedom, and the decomposition becomes ambiguous.213 Near it, a tiny change in orientation demands huge, discontinuous changes in the angles - a controller interpolating in Euler space can command joints to swing wildly.
Trap: never control, interpolate, or train on Euler angles. Use rotation matrices or quaternions internally; convert to roll-pitch-yaw only for human-readable display and logging. For a learning system specifically, Euler's wrap-around discontinuities (at ±180°) and gimbal lock give the network two very different numbers for nearly the same orientation - it learns noise.
Axis-Angle
Euler's rotation theorem says any 3D orientation is a single rotation by some angle about some unit axis . Store it as the 3-vector - the "rotation vector" or "exponential coordinates": its length is the angle, its direction is the axis. This is the most physically intuitive representation, and it's the bridge to the screw-theory / Lie-group treatment that Modern Robotics214 uses. The conversion from axis-angle to a rotation matrix is Rodrigues' formula:
Here is the 3×3 skew-symmetric matrix built from (the matrix form of "take the cross product with "). In Lie-group language lives in (the Lie algebra) and - the matrix exponential lifts the axis-angle up to the rotation, and the matrix logarithm brings it back down. You don't need to compute these by hand yet; recognising the picture is enough. Axis-angle is undefined at (no axis) and gets sign/wrap fussy at .
Quaternions
A unit quaternion is four numbers obeying . It encodes a rotation of angle about unit axis as:
Trap: the half-angle. The quaternion uses , not . This catches everyone exactly once.
Why four numbers for three degrees of freedom? The unit-norm constraint removes exactly one, leaving three. That one "extra" number is what buys smooth, gimbal-lock-free coverage of all rotations - it's a topological fact that you cannot cover the rotation group smoothly with only three numbers. The facts that matter in practice:
-
Double cover. and represent the same rotation (adding a full turn - rotation by about the same axis - flips every component's sign via the half-angle, yielding , yet the physical orientation is identical). So the map from unit quaternions to rotations is 2-to-1.215
Trap: the double-cover sign. When comparing or interpolating two quaternions, if their dot product is negative, flip the sign of one first - otherwise you interpolate "the long way around," up to 360°. For training data, canonicalise the sign (e.g. force ) or the network sees two labels for one pose.
-
Composition is quaternion multiplication (the Hamilton product), non-commutative like matrices, but cheaper (~16 multiplies vs. 27) and it doesn't drift toward non-orthogonality the way accumulated matrix products do.
-
The inverse of a unit quaternion is its conjugate - cheap and exact.
-
To rotate a vector , promote it to a "pure" quaternion and compute .
-
Normalisation drift. Repeated multiplication accumulates floating-point error; the norm creeps away from 1, and a non-unit quaternion silently applies an unwanted scaling. Renormalise (divide by the norm) periodically. This is the quaternion gotcha in long-running loops.
Trap: storage order and convention. Libraries disagree on whether w comes first or last. ROS/tf2216, SciPy's
Rotation, and Eigen's internal coefficient order use ; many graphics/maths texts and Eigen's constructorQuaterniond(w,x,y,z)put w first. Always check before copying raw arrays. There's also Hamilton (robotics/ROS) vs. JPL (aerospace) convention, which differ in a multiplication sign.
SLERP (spherical linear interpolation) blends two orientations along the shortest great-circle arc on the 4D unit sphere, at constant angular velocity - smooth, natural rotation:
(flip one sign first if the dot product is negative, per the double cover). Naively interpolating Euler angles component-by-component gives jerky, variable-speed motion that can pass through gimbal lock. This is why quaternions win for interpolation, blending, and smooth controller setpoints.
Practical rule: store, compose, and interpolate in quaternions or rotation matrices; use Euler only for display. Note that Modern Robotics itself does not teach quaternions as its primary tool - it uses rotation matrices and the matrix-exponential/screw formalism - so you'll pick quaternions up separately (3Blue1Brown217). They matter because LeRobot, ROS, URDF, and essentially all pose data store orientation as quaternions.
Homogeneous Transforms
To combine rotation and translation into one composable object, stack them in a 4×4 homogeneous transformation matrix in (special Euclidean group in 3D):
You append a "1" to points (), and a single matrix multiply applies rotation-then-translation: . has exactly 6 degrees of freedom (3 rotation + 3 translation), the pose of a rigid body, or of a robot's tool. Two facts to memorise:
- Compose by multiplying, inner frames cancel, order matters: . This is how you walk a kinematic chain from base to tool - forward kinematics is nothing more than this.
- The inverse has a closed form - never run a generic 4×4 matrix inverse on a pose:
Transpose the rotation block; the new translation is . Exact and cheap.
Trap: pre- vs. post-multiply. Left-multiplying by a new transform acts in the fixed/reference frame; right-multiplying acts in the current/body frame. This mirrors extrinsic (left, world axes) vs. intrinsic (right, body axes). "Rotate then translate" "translate then rotate."
Twists and Screws
Modern Robotics unifies velocity and finite motion through screw theory. You need only the intuition now. A twist is a 6-vector (angular velocity stacked with linear velocity ), the instantaneous velocity of a rigid body, the analogue of angular velocity.
The Chasles-Mozzi theorem says any rigid-body displacement is a rotation about, plus a translation along, a single fixed axis in space, a screw motion, like a bolt threading. The exponential coordinates of a rigid-body motion are (six numbers: a screw axis times how far you travel), and maps them up to ; the matrix log maps back. It's the same exp/log pattern as axis-angle rotation, extended from to .
Why care as a beginner: each revolute or prismatic robot joint is a screw axis, and this is the cleanest, singularity-free way to write forward kinematics (Forward Kinematics). You don't compute these by hand yet.
The TF tree
ROS's tf2 library maintains a live tree of coordinate frames: nodes are frames (world, baselink, camera_link, tool0, …), edges are time-stamped parent→child transforms, each edge storing "pose of child in parent." Because it's a tree (every frame has exactly one parent), tf2 can compute the transform between _any two frames by walking up to the common ancestor and back down, composing and inverting transforms for you. You ask "where is the object in the base frame?" and tf2 chains the maths.
The authoritative conventions (ROS REP 103/105)218: all frames right-handed; standard body/world axes are X forward, Y left, Z up; angles right-handed; SI units (metres, radians); orientation stored as a quaternion in order. Standard mobile frames chain map → odom → base_link; for the UR arm you'll see base/base_link → shoulder → … → wrist_3 → tool0.
Trap: camera frames are the exception. Cameras often use Z forward, X right, Y down - different from the X-forward/Z-up body convention - a frequent source of "my object is rotated 90°" bugs.
The two or three formulas actually worth memorising
- SE(3) inverse: . Never generic-invert a pose.
- Chain/compose: ; and .
- Axis-angle ↔ quaternion: . (A fourth, if you want it: Rodrigues, above.)
Forward Kinematics
Forward kinematics (FK) is the deterministic map from joint values (angles for revolute joints, displacements for sliding/prismatic joints) to the pose of the end-effector in the base frame. It's the easy direction: one input vector of joint values produces exactly one pose, always, in closed form.
A robot arm is a kinematic chain: rigid links connected by joints. Both your arms are open serial chains - a single unbranched sequence base → J1 → link → J2 → … → tool. Each joint gives one degree of freedom. Revolute joints (R) rotate about an axis and are by far the most common; the UR10e is a "6R" arm (six revolute joints). Prismatic joints (P) slide.
The end-effector pose lives in as a 4×4 transform, and FK is the chain product . This is why you were told to nail transforms first: a rotation plus a translation is a pose, and FK is just chaining poses by matrix multiplication.
DH versus PoE
Denavit-Hartenberg (DH), 1955 - the classical method. It attaches a frame to every link by strict rules, then describes each joint-to-joint transform with exactly four numbers: (link length), (link twist), (link offset), (joint angle). One row of a "DH table" per joint; each transform is the fixed sequence . Its one real advantage is minimal parameter count. Its curse:
Trap: standard DH vs. modified (Craig) DH. Two incompatible conventions exist, with different frame placement and multiplication order. Mixing them is the single most common FK bug. Always know which one a table uses.
Product of Exponentials (PoE), Brockett 1984219 - the screw-theory method Modern Robotics teaches and the one LeRobot-adjacent tooling (Pinocchio/Placo) uses in spirit. The space-form formula:
- is the home configuration: the 4×4 tool pose when all joints are at zero. You just read or measure it off the robot at its zero pose.
- is the 6-vector screw axis of joint , expressed in the fixed frame with all other joints at zero, written . For a revolute joint, = the unit vector along the rotation axis and , where is any point on that axis. For a prismatic joint, and = the unit slide direction. (Convention flag: some texts write ; watch the sign.)
- is the matrix exponential of the screw - a 4×4 in computed via Rodrigues for the rotation part.
There's also a body form, , with screw axes expressed in the end-effector frame - same answer, different reference.
Why Modern Robotics prefers PoE: you only ever define two frames (base and tool) instead of DH's n finicky intermediate frames; revolute and prismatic joints use the same formula (just build the screw differently); the screw axes have direct geometric meaning you can eyeball off a drawing; and the parameters vary smoothly, avoiding the artificial singularities DH suffers during calibration. Lynch and Park's blunt verdict: there's no compelling reason to prefer DH over PoE unless minimising parameter count is critical. DH's only edge is 4 parameters per joint vs. PoE's 6.220
Reading URDF
A URDF (Unified Robot Description Format) is an XML file describing a robot as a tree of <link> elements connected by <joint> elements. It's the lingua franca that turns "an arm" into something software can run FK on.221
Each <link> may carry <visual> (mesh + colour, for rendering), <collision> (simplified geometry for fast collision checks), and <inertial> (mass, centre of mass, inertia tensor - needed for dynamics/simulation, not for pure FK). Each <joint> carries the data FK actually needs: type (revolute/prismatic/continuous/fixed/…), <parent> and <child>, <origin> (the fixed xyz translation + rpy rotation from parent to joint), <axis> (the unit vector the joint moves about/along), and <limit> (min/max position, velocity, effort).
A URDF therefore implicitly encodes the entire kinematic chain - the <origin> transforms and <axis> vectors are the raw ingredients for either a DH table or (more directly) the PoE home config and screw axes . A URDF is, in essence, a serialised set of homogeneous transforms plus joint definitions.
Who consumes it: ROS's robot_state_publisher reads the URDF plus live /joint_states and publishes FK to the tf tree; simulators (Isaac Sim/Isaac Lab, MuJoCo, PyBullet, Gazebo) load it to spawn the robot; kinematics libraries Pinocchio and Placo (a Pinocchio wrapper popular in the LeRobot ecosystem) parse it and hand you FK, Jacobians, and IK for free. Being able to read and debug a URDF is a core skill - if your sim and real robot disagree, a wrong <origin> or <axis> is a prime suspect.
FK for your two arms
The UR10e is a 6R e-Series cobot. Universal Robots officially publishes its nominal kinematic parameters222:
| Joint | [m] | [m] | [rad] |
|---|---|---|---|
| 1 | 0 | 0.1807 | |
| 2 | −0.6127 | 0 | 0 |
| 3 | −0.57155 | 0 | 0 |
| 4 | 0 | 0.17415 | |
| 5 | 0 | 0.11985 | |
| 6 | 0 | 0.11655 | 0 |
The two big link lengths (upper arm) and (forearm) set the UR10e's ~1300 mm reach.
Trap: the published table is nominal, not your robot. Every UR ships with per-unit calibrated DH deltas stored on the controller. If you compute FK from the published nominal values you can be off by a few millimetres from the real arm. For precise work, read the robot's own calibrated parameters - this exact gotcha caused real bugs in the ros-industrial
universal_robotdriver.223
The SO-101 is a ~$100-150/arm open-source hobby arm: six servos = five revolute positioning joints (shoulder_pan, shoulder_lift, elbow_flex, wrist_flex, wrist_roll) plus one gripper. The gripper doesn't move the tool frame, so FK drops it. The SO-101 URDF ships with LeRobot224 (community mirrors host it too143), and LeRobot computes FK/IK by feeding that URDF into a Pinocchio-based kinematics library - you never hand-derive it.
What to compute by hand exactly once
Do these once, to build intuition; let software do them forever after:
- Compose two or three rotations as both quaternions and 3×3 matrices, and confirm they agree. Convert a rotation to axis-angle and back.
- Multiply two 4×4 homogeneous transforms by hand and see that "pose composition = matrix product."
- Do FK by hand for a 2-link or 3-link planar arm and verify the classic tip position: , . This one exercise makes FK click.225
- Build one PoE example on that same planar arm: pick the home config M, write the screw axes, evaluate the product of exponentials, and check it matches.
- Compute one 3×3 matrix exponential via Rodrigues for a single revolute joint.
Never hand-code the following in production - always use a library (the modern_robotics package's FKinSpace/FKinBody, or Pinocchio/Placo, or Peter Corke's Robotics Toolbox): full 6-DoF FK from a URDF, Jacobians, IK, and calibrated-DH correction. The hand computation exists purely so that when the SO-101 or UR10e reports a bad pose, you can tell whether it's a frame convention, a degrees-vs-radians units bug, or a calibration problem.
Worked Example
Every bullet above, run on the same toy robot: a planar 2R arm with link lengths , and joint angles , . Each step checks the one before it, so a slip anywhere shows up as a mismatch at the end.
1. Geometric FK. The classic formula gives the tip directly. With :
Keep - the PoE computation below must reproduce it.
2. Rodrigues for one revolute joint. Joint 1 rotates about by . Build the skew matrix and its square:
That's exactly - the sines and cosines land in the right slots by themselves.
3. Quaternion cross-check. The same two rotations as quaternions: for a rotation, so and . The Hamilton product (same axis, so only and survive):
That's : half-angles added, , so the composed rotation is - agreeing with the matrix product .
4. PoE on the same arm. At the zero pose the arm lies along , so the home configuration puts the tip at :
Both screw axes point along ; joint 1 passes through the origin, joint 2 through , so :
For a zero-pitch screw through a point , the exponential is just "rotate about the axis through ":
Joint 1's exponential is the Rodrigues matrix from step 2 with zero translation. Joint 2 uses and :
5. Multiply through - this is the "pose composition = matrix product" bullet happening in front of you. Right to left:
Read the last column: the tip sits at - exactly the geometric FK from step 1. Read the rotation block: , i.e. the tool has turned by , as a planar arm must.
6. And back to axis-angle. From that rotation block, , and the axis falls out of the skew-symmetric part as - closing the loop on the "convert a rotation to axis-angle and back" bullet. If any of these numbers had disagreed, the mismatch pattern tells you what broke: wrong slot signs mean a frame convention, everything off by a factor of ~57 means degrees vs. radians.
Inverse Kinematics
Inverse kinematics (IK) is the reverse and much harder direction: given a desired tool pose , find joint angles that achieve it. This is what you actually want in practice ("put the gripper here"). Three structural reasons it's hard:
- It is nonlinear. FK is built from sines and cosines of joint angles composed through matrix products, so inverting it is a system of coupled transcendental equations with no general algebraic solution.
- Multiple solutions. A reachable pose can be hit by many distinct joint configurations - the human analogy is touching a point on a table with your elbow up or elbow down.
- Zero or infinite solutions. Targets outside the workspace, or blocked by joint limits, have no solution; at singularities there can be infinitely many; a redundant arm ( joints for a 6-DoF pose) has a continuous infinity. So IK isn't a clean function - it's a search over a branched, constrained solution set. That's the biggest conceptual jump from FK.
Analytic (closed-form) IK
For certain geometries you can write as explicit trig from . The classic sufficient condition (Pieper) is a spherical wrist - three consecutive axes meeting at a point - which decouples position (first three joints) from orientation (last three).
The UR arms are an interesting case: they don't have a classic spherical wrist (there are wrist offsets), yet their specific geometry still admits a full closed-form solution, worked out by Hawkins (the ur_kinematics solver) and others.226
The key fact for you: a reachable UR pose has up to 8 distinct solutions = , from three independent binary choices - shoulder left/right, elbow up/down, and wrist flip up/down. Not all 8 are always valid (some violate joint limits or are unreachable). Modern implementations like ur-analytic-ik227 return all branches in microseconds at machine precision. Analytic IK is exact and complete but robot-specific and doesn't natively handle collisions or redundancy.
Numerical IK
When there's no closed form (an arbitrary URDF, a redundant arm, or you want collision/limit constraints), solve iteratively - the general workhorse. This is Modern Robotics Chapter 6.2, and it's Newton-Raphson root-finding on the pose error, driven by the Jacobian (The Jacobian). The loop: start from a guess ; compute the pose error as a twist between the current pose and the target; update ; repeat until the error is below tolerance. The "inverse" (the generalised or pseudo-inverse of the Jacobian) comes in flavours:
- Jacobian transpose (): cheapest, no inversion, reliable but slow - a gradient-descent flavour.
- Pseudoinverse (Moore-Penrose right-inverse, , the minimum-joint-velocity solution valid when has full row rank - for the square 6×6 UR Jacobian it reduces to , and the tall/overdetermined case uses the left form instead): fast, minimises joint-velocity norm, but blows up near singularities - as becomes near-singular, gets huge entries and commands enormous joint velocities.
- Damped least squares (DLS / Levenberg-Marquardt): . The term regularises the inverse so it stays bounded near singularities - you trade a little tracking accuracy for stability. This is the practical default for velocity IK and Cartesian jogging. I assumed it was also what LeRobot's end-effector path runs; reading the source proved me wrong - see What LeRobot Runs below.
Numerical IK needs a seed and converges to the branch nearest that seed - good for trajectory continuity, but it returns one solution, not all, and can fail to converge.
Singularities
A singularity is a configuration where the Jacobian loses rank (for a 6-DoF arm, ). Physically, the arm instantaneously loses the ability to move its tool in some Cartesian direction - a degree of freedom collapses. To keep the tool moving at constant speed near a singularity, the required joint velocities shoot toward infinity. That's the freak-out: a joint (often a wrist or the elbow) suddenly whips around fast. UR's own docs warn that near singularities the arm "can stop moving or have very sudden and fast movements."228
The 6R UR arm has three singularity types (the three vanishing factors of ):
- Wrist singularity: wrist joint 2 reaches 0° or 180°, aligning two wrist axes and losing an orientation DoF.
- Elbow singularity: the arm is fully stretched (or folded) so shoulder, elbow, and wrist-1 are collinear - it's at the edge of its reach and can't move further radially.
- Shoulder singularity: the wrist centre passes directly over the shoulder axis (the line of joint 1).
What it feels like on your UR10e: you'll hit these most often when jogging in tool/Cartesian mode through an extended or awkward pose, and the pendant throws a "close to singularity" protective stop. PolyScope on the e-Series has singularity-avoidance features that slow the arm and can decline a straight-line (MoveL) move that passes through a singularity; for hand-guiding near one, UR caps tool/elbow speed at mm/s to prevent elbow whip.229
The practical rule: MoveJ (joint-space moves) sidestep the problem because they don't demand a Cartesian straight line and need no IK at all; MoveL (Cartesian-linear) moves are where singularity blow-ups and branch problems bite. Your force-controlled or Cartesian-jog code can hit these too; DLS damping is what keeps velocities bounded instead of the arm lurching.
Redundancy and the null space (know the words)
A 6-DoF arm reaching a 6-DoF pose has no redundancy, only discrete solutions. A 7-DoF arm (or a task that constrains fewer than 6 DoF) is redundant: infinitely many give the same tool pose, differing by internal "self-motion." That extra freedom lives in the null space of (joint velocities that produce zero tool motion, ), exploited as , where the second term reshapes the arm's posture (avoid a joint limit or an obstacle) without disturbing the tool.
Your UR10e is the exactly-6 case (discrete solutions, no redundancy); your SO-101, with only five positioning joints, is instead deficient. It constrains at most 5 task DoF and can't reach an arbitrary 6-DoF position-and-orientation at all. Neither is redundant for a full 6-DoF pose, but you'll meet null-space language constantly in the humanoid/VLA literature (Pink, cuRobo), so it's worth recognising.
Solvers in 2026
- KDL (old ROS default): Newton-Raphson pseudoinverse; famously fails when a joint limit is hit.
- TRAC-IK (2015)230: the pragmatic upgrade - runs two solvers in parallel (an improved Newton-Raphson with random restarts, and an SQP solver that treats joint limits as hard constraints) and returns whichever finishes first. The CPU baseline.
- IKFast (OpenRAVE): auto-generates fast, exact analytic C++ IK for a given robot; generation is finicky, and it does no collision checking.
- Pink + Pinocchio231 / Placo: Python differential IK that formulates each control step as a QP - weighted tasks (reach this frame, keep posture) as the objective, joint/velocity/collision limits as inequality constraints. Great for multi-chain systems; common in LeRobot-adjacent code.
- cuRobo (NVIDIA)232: GPU/CUDA-accelerated, runs thousands of IK seeds in parallel - ~37,000 IK/sec at batch 1000 (~23× TRAC-IK), collision-free motion plans in ~30 ms on an RTX 4090. Built on PyTorch+CUDA+Warp.
What LeRobot Runs
The taxonomy sets up an obvious question: which of these does LeRobot actually use? The answer is none of the obvious ones. The only general-purpose kinematics object in the repository is RobotKinematics, a 151-line wrapper around placo, and the kinematic model is the URDF itself:
self.robot = placo.RobotWrapper(urdf_path) # the URDF IS the kinematic model
self.solver = placo.KinematicsSolver(self.robot)
self.solver.mask_fbase(True) # fix the base: world = URDF root link
self.tip_frame = self.solver.add_frame_task(target_frame_name, np.eye(4))There are no DH parameters and no hardcoded link lengths anywhere in the repository, and the public API works in degrees because degrees are the motor's native unit. The IK call itself is two lines:
self.tip_frame.configure(self.target_frame_name, "soft", position_weight, orientation_weight)
self.solver.solve(True) # ONE QP step per call - not iterated to convergenceRead that against the maths above. The frame task is a soft weighted least-squares objective, not a hard constraint, and the URDF's joint limits enter the QP as inequality constraints - the same bounded-step-toward-the-target job DLS does, but with limits as first-class constraints rather than a clamp bolted on afterwards.
It takes one QP step per call: convergence comes from the 30 Hz control loop re-solving against fresh joint readings, which is numerical IK unrolled across time rather than iterated in place. Because each solve is seeded with the current joint vector, branch continuity (the deadliest trap in the list below) comes for free. The cost is silence: no success flag is returned, so the caller can't distinguish "reached" from "out of workspace" from "hit a joint limit", and there's no fallback path.
One default deserves its own paragraph, because it's a hardware fact expressed as a float. The SO-101 end-effector pipeline sets orientation_weight = 0.01 against a position_weight of 1.0. The SO-101 has five positioning joints, so it can't reach an arbitrary 6-DoF pose at all - it's the deficient arm from Redundancy and the null space.
Rather than reject unreachable orientations, LeRobot weights orientation a hundred times weaker than position and lets the QP sacrifice it gracefully: a good engineering answer to a deficient arm, in one line of config. (The Jacobian, meanwhile, never appears in LeRobot's own code; it lives inside placo, which is the honest sense in which you "need" it - to understand what the library is doing on your behalf.)
The repo holds one other IK, and it's a worked example of the optimisation-based end of the spectrum. The Unitree G1 humanoid gets a CasADi nonlinear program solved by IPOPT, minimising a weighted objective of for both arms in a single problem, with the orientation error written as the SO(3) geodesic residual - the same matrix-log error the traps below tell you to use instead of Euler angles.
It's built on Pinocchio rather than placo, and it's the only IK in the repository that logs failure.
Where IK actually lives in the learned stack
The core mental model: the pi0.5 policy does not do classical IK. The network takes images + language + proprioception and directly outputs actions; for the SO-101 those actions are joint positions (six servo targets), learned end-to-end from teleop demonstrations. There's no explicit pose-to-angles solver in the learned loop - the network implicitly learned the mapping from data.
The scale of what that deletes is easy to miss: trace one inference tick through LeRobot on an SO-101 and between the network's output vector and the serial packet to the motors there's a denormalisation and a clamp, nothing else. Nothing runs an IK solver, a Jacobian (grep LeRobot's source for "jacobian" and the single hit is an unrelated log-probability correction in an RL actor), or a PID in Python.
Fifty years of classical robotics sit in the gap between those two lines of code, and this codebase simply doesn't use them. So for the core imitation-learning workflow on the SO-101, you can largely ignore analytic and numerical IK.
IK re-enters at four seams. (1) end-effector action spaces: LeRobot ships an EE controller (e.g. SO101FollowerEndEffector, still maturing; the SO100FollowerEndEffector is the more established one) where the policy outputs a delta tool pose and a one-step placo QP (What LeRobot Runs) converts it to joint commands; some setups train in EE-space because it generalises better across robot instances, which needs FK to label EE poses from recorded joint data and IK to execute EE actions.233
(2) Teleop by moving a target in space (VR controller, SpaceMouse, phone pose) uses IK under the hood, though, again, the SO-101 leader-follower is a pure joint mirror that needs none. (3) Sim and data generation (Isaac/GR00T, cuRobo trajectories, scripted collision-free motion).234 (4) The UR10e whenever you command it in Cartesian/EE space rather than streaming joint targets: ur_rtde235 exposes getForwardKinematics, getInverseKinematics (returning the solution nearest a seed qnear), and servoJ/speedJ for joint-space streaming, and there's "no singularity issue in joint movement" because streaming joint targets needs no IK at all.
Trap: reachability. Commanding a pose outside the ~1300 mm workspace, or an impossible orientation, returns nothing - numerical IK fails to converge, analytic IK returns complex roots. Sanity-check the target is inside the reach envelope before blaming the solver.
Trap: joint limits. A mathematically valid solution may require a joint past its physical limit. KDL silently fails; TRAC-IK/Pink respect limits as constraints; your own DLS loop must clamp to limits or it will "solve" into an illegal configuration.
Trap: solution-branch flips mid-trajectory - the deadliest one. As you move the tool along a Cartesian path, if the solver jumps from one of the 8 branches to another between waypoints (elbow-up → elbow-down), the arm executes a huge, fast, possibly dangerous reconfiguration even though the tool barely moved. Prevent it by seeding numerical IK with the previous timestep's solution to enforce continuity. This is closely tied to singularities - near one, branches merge and you can slip between them.
Trap: never compute IK error in Euler angles (gimbal lock, wrap-around). Use rotation matrices, quaternions, or the twist (matrix-log) error, as Modern Robotics does.
The Jacobian
If you internalise one object from this whole section beyond frames, make it the Jacobian. FK is a nonlinear map from joint angles to pose, . The Jacobian is its derivative - the local linearisation. Take the time derivative of FK and a clean linear relationship falls out:
- = joint velocities (rad/s), one per motor - for the UR10e, for the SO-101 arm.
- = the end-effector twist: a 6-vector stacking angular velocity (3) and linear velocity (3) - how the gripper spins and translates.
- = the Jacobian, a 6×n matrix that changes with configuration . It isn't constant; you recompute it at every instant.
Intuition: the Jacobian is the gearbox from joint space to Cartesian space, but a gearbox whose ratios shift continuously as the arm folds and unfolds. Each column of is the tool twist produced by moving one joint at unit speed with the others frozen; stack the columns and you get how any blend of joint motions maps to gripper motion.
Nearly every classical result (IK, singularity analysis, force control, redundancy, trajectory following) is expressed through . It's the hinge between "what the motors do" and "what the hand does." (Modern Robotics distinguishes the space Jacobian, expressed in the base frame, from the body Jacobian, in the tool frame; same object, different frame, and you don't need the distinction for intuition, just remember a twist is always "in some frame.")
Velocity IK is running this backwards: to make the tool move at a desired twist, solve (or when is not square/invertible). This is "resolved-rate control," the basis of Cartesian jogging - when you drag a UR arm's TCP in a straight line, the controller is inverting thousands of times per second. It's also how numerical IK works (Inverse Kinematics).
If you want to see this maths breathing in a modern learning codebase, the cleanest sighting is LeKiwi, LeRobot's three-wheel holonomic mobile base. Its wheel kinematics builds the matrix mapping the body twist to the three wheel speeds, then runs it backwards with a literal np.linalg.inv(M).
It's a Jacobian that happens to be constant, because wheel geometry, unlike arm geometry, doesn't change as the robot moves - and that inversion is the only matrix inverse used as a kinematic map in LeRobot's ~120,000 lines, which tells you how completely the learned stack has swallowed the rest of this chapter.
Singularities revisited through the Jacobian: at a singularity drops rank, its columns become linearly dependent, and the reachable twists collapse from a full 6D space to a lower-dimensional subspace - the tool is instantaneously stuck along some direction. Near it is ill-conditioned, so blows up: tiny commanded Cartesian velocities demand enormous joint velocities. That's not a bug; it's the maths screaming that the inverse is exploding, and DLS () is the fix.
You detect nearness via the singular values of (its SVD): when the smallest singular value you're at/near a singularity. A cheap scalar proxy is the manipulability measure , which at singularities.
The manipulability ellipsoid is a picture of this: push all unit-norm joint-velocity vectors through and the image in Cartesian velocity space is an ellipsoid - long axes are directions the tool moves easily, short axes are "stiff" directions, a near-sphere is a dexterous well-conditioned pose, and a flat pancake means near-singular.
Force Duality
Transposed, the same Jacobian hands you statics for free:
- = the wrench at the tool: a 6-vector stacking moment (3) and force (3) - the generalised force the hand exerts on, or feels from, the world.
- = the joint torques the motors must produce.
- = the transpose of the same velocity Jacobian.
Why the transpose? Power must balance in both spaces. Mechanical power at the joints, , must equal power at the tool, . For this to hold for every , you need . So velocity maps forward through (joints → hand), and force maps backward through (hand → joints): velocities and forces are duals, and statics falls out of kinematics with zero extra modelling.
Modern Robotics titles this chapter "Velocity Kinematics and Statics" for this reason.236 There's even a pleasing symmetry: the force ellipsoid is the inverse of the velocity ellipsoid, so the direction you move fastest is the direction you push weakest, the robotics version of a gearbox trading speed for force.
Two directions of use, both critical: (1) feel forces without a dedicated sensor - measure joint torques (from motor currents) and estimate the external wrench, the basis of "sensorless" collision detection on torque-transparent arms; (2) command forces - to press with a desired wrench, command , the core of Cartesian force control and impedance control.
Why this plus the UR10e's wrist sensor unlocks contact-rich skills. The UR10e e-Series has a built-in 6-axis force/torque sensor in the tool flange (datasheet: force range ±100 N, torque ±10 N·m)237. It directly measures . The controller reads in real time, knows at every instant from the encoders plus FK, and via it turns "keep 10 N of contact force normal to this surface while sliding freely along it" into precise per-joint torque commands thousands of times per second.
That's hybrid position/force control (some Cartesian axes force-controlled, others position-controlled), and it's what makes peg-in-hole assembly, connector insertion, surface finishing/polishing, wiping, and screw-driving work. Without F/T plus Jacobian force control, a position-controlled arm either misses (too rigid: tiny misalignment, huge force) or needs perfect vision; force control lets it feel its way in.
For your ML stack: this is also why the F/T channel matters as data. pi0.5 can consume the wrench signal as an observation (touch is a modality vision can't see), and the same duality is what lets a learned policy's Cartesian intent be realised safely against contact. The classical Jacobian doesn't disappear under learned control - it lives underneath as the layer that converts "what the hand should do/feel" into "what the six motors command."
If you internalise just two equations from all of this section, make them (velocity forward) and (force backward). Everything else is commentary on those two lines and the fact that the same matrix and its transpose govern both.
PID and Control
The one-sentence model
Feedback control is a thermostat. You have a setpoint (desired value), you measure the actual value from a sensor, you compute the error (), and you drive an actuator in proportion to that error to shrink it. Run this loop tens to tens-of-thousands of times per second and the actual value chases the setpoint. PID is the most-used recipe for turning error into command, and every joint in your robot, from a $12 servo to the UR10e, is a nested stack of these loops.
The three terms, with physical intuition
- P (proportional) = a spring. Output proportional to current error: big error, big push; zero error, zero push. A joint under pure-P position control behaves like a spring pulling toward the target, being the stiffness. Two problems: it's "lazy" near the target (the push shrinks as the error shrinks), so it settles with a permanent steady-state error - e.g. gravity constantly pulling a joint down means the joint droops until the residual error generates just enough command to balance gravity; and if you crank too high it overshoots and oscillates (a stiff spring with no friction rings).
- I (integral) = trim / auto-zeroing. Accumulates error over time; as long as any error persists, the integral grows and adds more command until the error is driven to exactly zero. This is what kills steady-state error - the gravity droop, a constant load, friction. The cost is added phase lag/sluggishness and possible slow overshoot if too aggressive.
- D (derivative) = a damper / shock absorber. Responds to the rate of change of error - how fast you're approaching the target - and pushes against that velocity, braking early. This is why D fights oscillation and lets you run a higher without ringing. Its cost: differentiating amplifies sensor noise (a noisy encoder makes D scream), so in practice D is low-pass filtered, or you take the derivative of the measurement rather than the error to avoid a "derivative kick" when the setpoint jumps.
Summary: P pulls (present error), I remembers (past accumulated error), D anticipates (predicted future error).
Tuning intuition: start with P only, raise until it responds briskly and just begins to oscillate; add D to damp that oscillation (letting you push higher); then add just enough I to erase the residual offset. You trade off rise time, overshoot, settling time, and steady-state error - you can't max all four.
Ziegler-Nichols is the textbook heuristic (raise to sustained oscillation at "ultimate gain" with period , then , and from ); it gets you in the ballpark, then you hand-tune.
Trap: integrator windup - the #1 bug in naive PID code. If the actuator saturates (hits max current/torque or a speed cap) while a large error persists, the integral keeps accumulating a huge value because the error isn't shrinking. When the setpoint is finally reached, that bloated integral takes a long time to "unwind," causing massive overshoot. Fix with anti-windup: clamp the integrator, stop integrating while saturated, or back-calculate from the difference between commanded and delivered output. Anyone hand-writing a control loop must handle this.
Cascade Control
A robot joint isn't one PID - it's a cascade of nested loops, each faster as you go inward, each feeding the setpoint of the loop inside it:
- Innermost - current/torque loop (~10-32 kHz), inside the motor drive. Controls current, which is proportional to torque. Its setpoint comes from the velocity loop.
- Middle - velocity loop (~1-8 kHz). PID on joint speed; its output is the torque setpoint.
- Outer - position loop (~500 Hz-4 kHz). PID on joint angle; its output is the velocity (or torque) setpoint.
Cascade works because the inner loop is much faster than the outer: the position loop can assume its velocity command is achieved essentially instantly, disturbances are caught locally by the fast inner loop before they perturb position, and you tune inner-first-then-outer.
On the SO-101's Feetech STS3215 servos:106 each smart serial-bus servo is a self-contained closed loop - DC motor + magnetic absolute encoder + microcontroller running its own internal PID position controller. You send a goal position over the TTL serial bus and the servo runs the PID locally; the arm is six such servos daisy-chained. The PID gains and torque limit are user-settable registers,238 so you can soften or stiffen a joint. Key point: your host code (LeRobot) never runs the joint PID - it just streams goal positions; the PID lives in the servo.
On the UR10e (e-Series): the torque/velocity/position loops run inside the joint drives and controller at high rate; UR exposes only setpoint-level commands to you.
Gravity compensation
Gravity is a constant, predictable disturbance: each link's weight makes a pose-dependent torque at every joint. Rather than make the feedback loop fight gravity from scratch every cycle, you compute the gravity torque from a model (masses, link lengths, current angles → the term of the dynamics) and add it as a feedforward term. Now the feedback loop only corrects small residuals. This is why a well-modelled cobot can be hand-guided ("freedrive") - it's actively cancelling its own weight so it floats. Feedforward (model-based, predicts the command) + feedback (PID, corrects the error) is the standard pairing.
Three levels of command, and impedance/force mode
- Position control: command a joint angle or tool pose; the loop holds it stiff. Default for pick-and-place; resists disturbances but fights the environment on contact (a tiny position error against a rigid surface makes an enormous force).
- Velocity control: command a rate; good for visual servoing, teleop, conveyor tracking.
- Torque (effort) control: command joint torques directly - the lowest level and the substrate for compliant/force behaviour ("apply this much force" instead of "go to this angle no matter what").
For contact tasks (wiping, sanding, peg insertion) you make the robot compliant: it yields to contact in a controlled way, behaving like a programmable spring-damper, , soft (low stiffness ) in the direction you want to give and stiff where you want precision.
Impedance control measures motion and outputs force (needs backdrivable torque-controlled joints, "force out"). Admittance control is the dual: it measures force from an F/T sensor and outputs motion, then a stiff inner position loop executes it (for stiff industrial arms, "position out").
The UR e-Series exposes admittance-style compliance via force_mode(task_frame, selection_vector, wrench, type, limits)239: you pick a frame, choose via the selection vector which axes are force-controlled vs. position-controlled, and set the target wrench on the compliant axes so the arm moves freely along those to maintain the force while staying rigid on the others. That's the primitive for wiping, polishing, and force-fit insertion.
Uncertainty flag from practitioners: the UR's built-in wrist sensor is fine for "roughly right force" and safety, but its noise floor limits high-bandwidth, high-precision force loops - delicate insertion sometimes wants an external F/T sensor.240
The Layer Cake
This is the crux for your openpi/LeRobot setup. A modern VLA is slow and high-level; the classical loops are fast and low-level; they're decoupled and layered.
- Policy layer (~1-50 Hz): pi0/pi0.5241 uses action chunking242 - one forward pass outputs not a single action but a chunk of ~50 future actions (a short trajectory, ~1 second at 50 Hz), generated by flow matching (a faster cousin of diffusion that generates the action chunk in a few steps) over continuous actions in the deployed low-level controller (no lossy tokenisation at execution - though pi0.5's pre-training does use FAST action tokens). The robot executes the chunk while the next inference computes, hiding latency and producing smooth motion. Actions are typically joint targets or tool poses, not torques.
- Streaming interface (~500 Hz): the chunk's targets are streamed down as setpoints to the fast loops.
- Classical PID loops (500 Hz → 32 kHz): inside the UR joint drives or each Feetech servo, doing the actual tracking and disturbance rejection. The policy says "put the joints roughly here over the next second"; the fast PID makes the physical joints actually get there.
LeRobot explicitly decouples action prediction (inference) from action execution (control): v0.5.0 added Real-Time Chunking (RTC)243 so consecutive chunks join smoothly instead of jerking at the boundary (The Action Queue below unpacks how, because "blend the chunks" undersells it badly).
On the SO-101: LeRobot streams chunked goal joint positions over serial; each servo's internal PID tracks them - "policy → goal position → servo PID → motor." A consumer GPU runs the pi0.5 LoRA inference (LoRA is a cheap fine-tuning method that trains a small adapter instead of the whole model); the servos handle the fast control.
On the UR10e: LeRobot/openpi push targets via RTDE244 at 500 Hz (e-Series rate; legacy CB-series was 125 Hz), using URScript primitives245 as the bridge:
servoj(q, …, gain, lookahead_time)246 - online real-time servoing to a stream of joint positions, fed at up to 500 Hz with a dense trajectory. Itsgainparameter acts like the P-term of a PID (higher = snappier tracking, more prone to vibration);lookahead_timesmooths. This is the primary primitive for streaming a policy's chunk into the UR.speedj/speedl- command joint/Cartesian velocity at 500 Hz (velocity streaming, visual servoing).movej(q)- trapezoidal joint-space move (fast point-to-point, path in joint space);movel(p)- straight line in Cartesian space (controlled tool path, but where singularities bite).
movej/movel are blocking, planned moves (good for scripted waypoints); servoj/speedj are streaming, reactive primitives (what a learned policy almost always uses so it can react every cycle).
The layer-cake to hold in your head: neural policy (pi0.5, ~10-50 Hz, joint-target chunks via flow matching) → streaming interface (servoj/speedj over RTDE @500 Hz on the UR; serial goal-position on the SO-101) → classical position/velocity/current PID loops (500 Hz → 32 kHz, inside the UR drives or each Feetech servo) → motors. Force tasks swap the outer behaviour for impedance/admittance (UR force_mode) so the arm acts like a programmable spring-damper against contact. Gravity-compensation feedforward runs underneath so the arm floats. Windup handling and cascade structure are the two details separating a toy loop from a working one.
The Action Queue
Now do the arithmetic the layer cake implies, because it recasts everything above it. (The chunk maths above used the 50 Hz of pi0's ALOHA demos; your SO-101's loop runs at 30 Hz, so that's the clock from here on.) pi0.5 inference on a consumer GPU runs roughly 100-300 ms. A 30 Hz control loop gives you 33 ms per tick. The model is somewhere between three and ten times too slow for its own control loop, and no amount of engineering closes that gap directly.
So action chunking, which Inside the Models treated as an architecture choice, is really a survival requirement: without it, running a 3B-parameter network on a real arm is impossible, full stop.
The mechanism is a queue, drained from the other side. One forward pass emits chunk_size future joint targets; they go onto a deque; the control loop pops one per tick (every 33 ms at 30 Hz) while the next forward pass computes in the gaps. ACT's entire deployment path in LeRobot is nine lines:
if len(self._action_queue) == 0:
actions = self.predict_action_chunk(batch)[:, : self.config.n_action_steps]
self._action_queue.extend(actions.transpose(0, 1))
return self._action_queue.popleft()Notice the trigger: queue empty. The queue drains all the way to zero, then refills. With pi0.5's defaults (chunk_size=50 at 30 Hz) that means the arm executes 1.67 seconds of motion completely blind, then thinks again - and if thinking takes 300 ms, the arm has nothing to do for those 300 ms. It stalls at the seam. That's fine for ACT, whose inference is ~10 ms; it's visibly not fine for a VLA.
RTC, Physical Intelligence's real-time chunking algorithm247 as ported into LeRobot, fixes the seam with two moves, and the numbers are worth working once. First, the trigger changes from queue-empty to queue-depth: inference fires when 30 actions are still buffered, about a second of motion, so the arm never runs dry.
Second, it accounts for the actions that'll execute while it thinks: at 300 ms of measured latency, delay = ceil(0.300 / 0.0333) = 9 control ticks will have physically happened by the time the new chunk arrives, so the first nine actions of the new chunk are dropped on arrival (those timesteps are already in the past) and 41 fresh actions land. Net effect: the policy runs at roughly 1.5 Hz, the arm consumes at 30 Hz, and the seam is a blend rather than a stall.
The blend is the part worth getting right, because it's not an average of two finished trajectories. The new chunk is generated to agree with the old one. pi0.5 produces its chunk by ten Euler steps of flow matching, and RTC hooks inside that loop: at each denoising step it computes the chunk the model currently implies, measures where it disagrees with the previous chunk's unexecuted tail, backpropagates that error through the network itself, and bends the velocity field toward agreement:
x1_t = x_t - time * v_t # the chunk the model currently implies
err = (prev_chunk_left_over - x1_t) * weights # disagreement with the committed tail
correction = torch.autograd.grad(x1_t, x_t, grad_outputs)[0]
result = v_t - guidance_weight * correction # steer the generation itselfInpainting by autograd, essentially. The weights are the elegant bit: heavy at the start of the overlap, where those actions execute within milliseconds and are effectively already committed, then decaying toward the end, where the fresh observation deserves to win. The constraint fades in step with the policy's right to change its mind.
For contrast, LeRobot's older split-machine gRPC path really does blend numerically in the queue - overlapping timesteps get a literal 0.3 * old + 0.7 * new - which is the cruder mechanism RTC replaced. And ACT's temporal ensembling from Inside the Models is a fourth variant: genuine overlap-and-average, but it needs inference every single tick, which is the one thing a VLA can't do. RTC is that smoothness at 1.5 Hz instead of 30.
One layer sits above all of this. The rollout loop can run an ActionInterpolator that turns each policy action into interpolation_multiplier control ticks, and the observation path splits to match: robot.get_observation() runs every single tick, because the action post-processor needs fresh proprioception, while the costly processor pipeline and engine.notify_observation fire only when the interpolator asks for a new action. Fresh senses at full rate, heavy thinking at a fraction of it: the same trade RTC makes, replayed one layer up.
And because RTC's inference runs in a background thread against the same hardware as the control loop, the robot object gets wrapped in a ThreadSafeRobot whose lock guards just two methods, get_observation and send_action. Everything else is proxied unlocked, because nothing else touches the motors.
One operational footnote from the code that may save you a debugging session: the latency estimate feeding delay is a running maximum that never decays until an explicit reset (the tracker's 100-entry window bounds the percentiles it reports, not the max). One slow inference permanently inflates the assumed delay for the rest of the session, so if a long rollout gets gradually more sluggish, look there first.
The primary sources behind this section - textbook chapters, UR documentation, solver repositories - are footnoted at their point of use throughout, so any claim you want to chase has its receipt attached.
The Frontier Map
That's the fifty-year-old machinery under the hood; now zoom back out. Eight pressing problems account for essentially everything the ecosystem is organised around: what each one is, why it's still open, who's attacking it, and how to say it at a party.
The Data Bottleneck
What: The master problem. This is the catch from Two Eras, now with receipts: robot foundation models need enormous, diverse demonstration data, and there's still no internet-scale corpus of physical actions to scrape. Real-robot teleoperation data is slow and expensive; it captures the same few tasks in the same few labs.
You can watch this problem operating across the whole space: almost everything founders trade, whether gear listings, dataset requests or launch posts, is about acquiring demonstration data, or the hardware to collect it. People are buying, borrowing, and reselling SO-101 pairs, Trossen WidowX arms, UMI grippers, GELLO rigs, YAM-style arms, leader-follower teleop kits, and Manus gloves.
None of these are robots that do jobs; every single one is an instrument for recording human demonstrations to train policies on (the teleop rigs covered earlier). The scarce resource in robotics right now is hours of high-quality manipulation data rather than ideas or compute. The urgency around second-hand rigs, with buyers paying a premium to pick up hardware fast, is the data bottleneck talking.
Why unsolved: collecting diverse, high-quality, richly-annotated physical-world data at scale is hard, labour-intensive work - "dirty, unglamorous work." The industry's current big bet for scaling past teleop is egocentric human video: labs advertise needs of 5k+ hours of RGB+IMU egocentric capture, and brokers answer with off-the-shelf datasets spanning 10+ countries. The logic: teleoperating robots for data is expensive and slow; strapping cameras to humans doing chores is cheap and infinitely parallel. Whether human video actually transfers across the "embodiment gap" to robot bodies is one of the open scientific questions.
Who's attacking it: essentially everyone, but the pure-play data companies are Asimov, Human Archive, and Hub (three YC companies whose entire business is collecting first-person human video), plus 6thSense, Remy AI, XDOF, and DeepAware/SVRC - a whole data-broker layer paying people to wear rigs.
Cocktail-party take: "The winners in robotics will be decided by data, not model architecture. There's a Scale-AI-for-robots land grab happening beneath the flashy humanoid companies, and the margins there will probably compress because a dozen near-identical YC startups are all selling egocentric hours."
Dexterity and Touch
Robots can navigate and see, but reliably grasping and dexterously handling arbitrary objects (a grape without crushing it, a wet dish, a key rotated in-hand) remains the central unsolved problem. Robot vision is good now, but the moment fingers wrap around an object the camera's occluded and the robot's blind. The ecosystem circles this constantly: the hunt for frontier touch sensors, the GelSight/DEX-EE/LEAP canon, and Origami's "dexterity deadlocks" essay all point at the same fact.
It stays unsolved because the decisive physics happens at the fingertip contact, invisible to cameras once the hand occludes the object, and there's no standard, cheap, durable touch sensor the way there's a standard camera - nor any consensus on how to use touch data in learned policies. This is where the hardest open research lives.
The attack comes from both ends. On hardware: GelSight and Meta DIGIT/Digit 360 (optical tactile), Xela uSkin and AnySkin (magnetic arrays), Shadow DEX-EE and LEAP Hand (research hands), and Origami Robotics (backdrivable force-sensing hand). On data, 6thSense sells the tactile layer everyone else skips.
Cocktail-party take: "Touch is the one modality everyone omits, and it's what dexterous manipulation needs. The sensors are becoming commodity (a GelSight Mini is $499), but making a robot actually manipulate well with them is still frontier. That's why DeepMind, Meta FAIR, CMU and MIT are all in it and why Skild raised $300M."
Cheap Hardware
Research-grade manipulation used to require $80,000+ industrial arms, which priced most labs out. The field needs cheap-but-capable, standardised, reproducible bodies so thousands of people can collect data and run the same models, and open hardware has mostly delivered: SO-101/LeRobot at ~$100-250 an arm, GELLO at <$300, UMI at ~$370, then Almond's Axol ($7,999), Trossen ALOHA kits, YAM/I2RT arms and Anvil Robotics kits at $4,730 further up.
Two orders of magnitude came out of the cost in a few years, but building the hardware is still a physical grind with real holes in the US supply chain. The electronics-lab shopping lists (thermal chambers, vibration rigs, IMU-calibration turntables, 70A supplies), the flex-PCB manufacturer hunts, and the way adjacent hardtech companies lean on robotics teams for DFM/Shenzhen/QC contacts all point at the same gap: you can't currently build a robot in the US without either importing parts or building everything yourself, which is the gap The Robot Learning Company and HLabs' US-made modular parts exist to fill.
Cocktail-party take: "The SO-101 is the Raspberry Pi of robot manipulation - $200 of 3D-printed plastic and hobby servos, deliberately not frontier hardware, but it runs the same VLA policies as a humanoid, so it's the best cost-to-learning on-ramp in the field."
Evals and Reliability
The "did it work?" problem: you can't prove a learned policy is correct, and sim success doesn't guarantee real-world success (reported 24-30% performance drops transferring vision policies sim-to-real). Every training/eval/RL step needs a per-attempt success label. And data quality is becoming its own discipline: imitation learning has a garbage-in property - flawed demonstrations actively poison policies - so as collection scales, curation becomes the bottleneck behind the bottleneck.
It stays open because neural policies are non-deterministic, human labelling doesn't scale to millions of rollouts, and success definitions and hardware are inconsistent across labs - a reproducibility crisis. Instance Labs attacks it with automated success detection ("VLM-as-a-verifier"), alongside emerging benchmarks like RoboArena and the Fujitsu/CMU WACV "periodic workflow" paper176 on label-free curation - unsupervised separation of good demonstrations from bad ones.
Cocktail-party take: "Training the brain is only half the job - you also need scalable, trustworthy measurement of whether it worked, and that measurement is itself an AI product. This is robotics' analogue to LLM-as-judge and reward models."
Deployment Tooling
As robots move from demos into deployed fleets, they need the operational tooling the cloud-software boom created - observability, fleet management, OTA updates, config management, middleware. This layer is forming fast, and investors love it: the pattern from the cloud era is repeating, where the tooling companies get funded before the applications get big (The Analogy with LLMs maps this layer's LLM twin in detail). It's still forming because robots break the assumptions of normal DevOps: bandwidth-constrained, intermittently offline, and a bad deploy has physical consequences.
The names to know: Foxglove (data platform + MCAP, $40M Series B), Ember Robotics (edge-first observability that keeps working offline), Miru (config management / OTA, profiled below), plus Formant and InOrbit - and the wider supplier layer takes in Instance (policy evals, see Evals and Reliability), 6thSense, The Robot Learning Company, and HLabs (US-made parts).
Cocktail-party take: "Modern robotics is a data problem as much as a control problem - you improve robots by recording, searching and learning from what they experienced, which is why robot observability and data platforms are now a venture-scale category, investable separately from the robots themselves. If you want a low-risk way into robotics, this layer runs on skills software people already have."
Miru
Miru (YC S24, one-liner "Config Management for Robotics", originally launched as deployment infrastructure for robotics and IoT) attacks the least glamorous part of that last mile: the configuration files. A single robot can carry thousands of parameters (lidar scan rate, camera exposure, speed limits, feature flags, per-customer and per-site settings), many of them custom per hardware revision.
YAML and JSON have no built-in type validation, so a typo can silently break a robot; schemas and software versions must evolve in lockstep across a mixed-version fleet; and deploys happen over unreliable networks, where a wrong config means a crashed robot rather than a failed web request.248
The timing bet (my read, not a company claim) is that robotics is exiting the demo phase into fleet-scale commercial deployment (warehouse AMRs, delivery robots, humanoids, agricultural and industrial machines in the hundreds to thousands of units per company) and that those companies want their scarce engineers on autonomy and ML, not on rebuilding deployment plumbing. It's the vertical-infrastructure opportunity SaaS minted a decade earlier with CI/CD, feature flags and observability, appearing on schedule now that enough units are in the field to make fleet management painful.
Today most teams handle this by SSH-ing into robots one at a time or building fragile internal tools; Miru's YC launch page claims many robotics and IoT companies dedicate "more than half of their software team" to maintaining these homegrown deployment systems.248
Its answer is cloud plus on-robot software: engineers define config values and schemas in the cloud (both CUE and JSON Schema are supported, so a string where a number is expected gets caught before it ever reaches a robot), set override logic, and a lightweight Agent on each device applies the result at fleet scale via what the marketing calls "rendering" - combining a schema with config values into a device-specific output.249
The rest is recognisable DevOps translated to hardware. Changes flow through a Git/GitHub pipeline into releases that are built, staged, deployed over-the-air to targeted device groups, version-tracked, drift-detected and audited, with pre-deployment validation and automatic audit trails; non-engineers change settings through validated UI components instead of SSH.
The product spans the Agent, a CLI, a Device API (with Server-Sent Events for real-time device lifecycle events, plus provisioning tokens for onboarding devices) and a Platform API. Pricing isn't published; expect sales-led enterprise or usage-based. Mind the domain shuffle: miruml.com redirects to mirurobotics.com, the docs, blog and GitHub still live on the miruml namespaces, and YC lists the company simply as "Miru".
The team is three UT Austin alumni who met in their freshman dorm and previously built Clips AI together, an AI video tool that scaled to ~7,000 monthly users in three months and was named a 2023 "Most Disruptive" undergraduate startup by Poets&Quants: Armel Talla (CEO, computer science and maths, a former Google/YouTube intern on internal tools and infrastructure), Vedant Nair (COO, business and CS, with Bain and EY-Parthenon behind him) and Ben Smidt (CTO, electrical and computer engineering, the hardware and robotics focus).250
"The robot" is really a fleet of networked computers governed by configuration, and safely changing settings on thousands of physical machines over flaky networks (where one wrong number can physically break a machine) is why validation, versioning and controlled rollout matter far more here than in ordinary web software.
Teleop Behind the Curtain
What: The autonomy gap - the Optimus/NEO/Waymo pattern from The Full Stack: "autonomous" robots lean heavily on hidden human operators, both as a safety net and as the data-generation workforce. Aktoria's launch says the quiet part loudly: the viral demos all have humans behind them. Teleoperation is simultaneously the fallback that makes deployments viable today and the pump that generates training data for tomorrow's autonomy.
Why unsolved: true unsupervised, dexterous, economically-useful autonomy at scale doesn't yet exist anywhere; latency physics (below ~150ms for comfortable control) forces operators geographically close or offshore.
Two numbers bound that loop's design space. The canonical worst case is a satellite link at roughly 600ms glass-to-glass, far beyond useful direct control; and the standard defence against a dropped or lagging link is a command-staleness watchdog, typically set at 200ms - if no fresh command arrives inside the window, the robot executes a safe-stop, ramping velocity to zero and holding position.103
The watchdog lives in a safety supervisor, a separate and deliberately simple software layer that constrains the robot and halts it when commands go stale or unsafe, often written in a memory-safe language like Rust. For a humanoid the stop itself is non-trivial: unlike a bolted-down factory arm it can't just cut power on E-stop (it would fall and cause secondary collisions), so it must hand over to a predefined fallback controller that balances the robot into a safe pose.104
Who's attacking it / exploiting it: Flywheel AI (teleop excavators), 1X (teleoperated home humanoid), Waymo (fleet response), XDOF ($70M, profiled in The Full Stack), Aktoria (teleop-as-a-service, unproven, profiled below), plus the labour-arbitrage layer of offshore operators.
If you're buying rather than building this layer, the credible vendors to evaluate are XDOF, the data-workforce firms (Scale AI, Cogito, Shaip, iMerit), Adamo on latency, and the XR-and-haptics rig makers (Extend Robotics, HaptX, Sanctuary) from The Full Stack. Whatever you pick, demand contractually: measured glass-to-glass latency (<150ms target, ideally <40ms for fine manipulation), data ownership and licensing terms, operator-location and security disclosure (onshore vs offshore, and who can issue privileged commands), and a hardware-independent E-stop plus a humanoid-safe fallback controller.
Cocktail-party take: "Teleop props up the entire humanoid/robotaxi boom while staying deliberately understated. It's simultaneously the crutch that lets you ship unreliable robots today and the factory that manufactures the data to eventually remove the human - and it means some 'autonomy' demos are partly a magic trick. Understanding that 'autonomous' is a spectrum with humans on call is one of the highest-value insights for reading the industry clearly."
Aktoria Robotics
Aktoria is the horizontal, outsourced version of the in-house capability 1X built for NEO: rather than every robot company staffing its own operators, Aktoria wants to be the Twilio or Stripe of robot teleoperation, a remote-ops layer anyone can rent. Its live homepage (fetched July 2026) brands the company "Human as API for Robots" under the headline "Operate any robot, anywhere in the world", and sells three service lines: data collection ("collect data on YOUR robots, remotely, 24h a day"), fleet management (monitoring plus intervention on deployed fleets), and deployment (a "TRAIN → DEPLOY → MONITOR" loop).251
Nothing is priced anywhere: there are no published SKUs, SLAs or per-hour rates. The sales motion is a "Book a call" button routed to founders@aktoria.com, and "sold as operator-hours" is a reasonable inference rather than a stated fact.
Now the caveats, because Aktoria is the least-verifiable company in this entire post. The "Backed by Y Combinator" badge you may see attached to it appears nowhere on its own site, its YC company page returns a 404, and it's absent from the YC Fall-2025 batch list. The juicy specifics that circulated with its launch (158ms glass-to-glass latency from operators in India to robots in San Francisco, full-time operators in India, a Rust safety supervisor) appear in no public source and remain unverified.
The founders aren't publicly identified anywhere: there's no name, Crunchbase entry or press coverage attached to them as of July 2026, only that plural founders@ address, which implies more than one of them. Even the latency "moat" dissolves on inspection: the teleop literature cites <150ms for comfortable control and <100ms for precise manipulation, and consumer VR-teleop stacks are marketed at 15-40ms end-to-end, so a 158ms edge wouldn't be much of one even if the number were real.
What's verifiably true is a minimal landing-page-stage company selling remote robot operation and training-data collection as a service, with no robots of its own.
So why give an unverified landing page this much space? Because the problem it attacks is real and well-documented (learning-based robots need both demonstration data and edge-case takeover, the gap its messaging targets), and because it's a perfect specimen of a second lesson: an impressive-sounding profile can smuggle in unverifiable specifics (an exact latency, offshore staffing, an accelerator badge), so always distinguish what a company's own site says from what's asserted around it. Treat Aktoria as unproven until you can verify founders and funding; the macro thesis it rides is solid either way.
The contrast case is Amplibotics, Wigglesworth's company: not proof the model works, but at least checkable - a named founder with a track record, a live product site, and third-party coverage of its partnerships. It ships a device that plugs into the industrial arm you already own (UR, KUKA, Omron and FANUC are the integrated brands): you type a prompt, and the robot does the work. A remote pilot is driving it.21
The economics, on his account, are the interesting part. Most of the vendors in The Data Companies pay collectors by the hour and sell datasets on; here the customer pays you while the data accrues, because the robot is doing real machine-tending in their shop (taking custom parts out of machines), work they were already paying to have done. The customer buys the device plus a subscription to keep the robot running; Amplibotics retains rights to the data; and the recordings train the model that's meant to eventually replace the pilot on that exact task.21
This is the Flywheel AI move generalised off the excavator, and cousin to Asimov's captive cleaning subsidiary - except there the data farm is a subsidiary, and here it's a paying customer. Wigglesworth calls the tier "real-world work" and prices it as his premium product, on the argument that a demonstration only proves value if someone was willing to pay for the task itself: whiteboard-drawing demos are cool, but nobody buys them.21 It's the Operators Win flywheel run without owning the operation.
The Humanoid Question
What: Real work vs. hype: the bet that a human-shaped body can slot into human environments (factories, warehouses, eventually homes) without redesigning them. Meanwhile the hardware itself is commoditising: founders casually lend and rent Unitree G1s among themselves, and PNDbotics reportedly offers a humanoid on a 45-day free trial for only shipping costs (a promotion that appears nowhere in PNDbotics' own materials, so treat it as rumour).
When hardware makers give robots away, it tells you (a) Chinese OEMs have driven body prices down brutally, and (b) the binding constraint is what the robot's brain can do, which is why Lightberry, Remy, and every VLA lab sit on top of other people's hardware. The value is moving to software and data.
Why unsolved: locomotion (walking/balancing) is largely solved in demos, but general-purpose manipulation is not; only a handful of companies (Agility, Apptronik, Figure) have robots doing real paid work, and even those are narrow, zoned, low-volume, and often human-supervised.
Who's attacking it: the funding table tells you how seriously investors take the bet. Figure is the outlier at a $39B post-money valuation (Series C, September 2025) on more than $1.9B raised.252 Its Figure 03 launched in October 2025 running the in-house Helix VLA model, and its BotQ factory is scaling toward one robot every 90 minutes. The deployment evidence is real but modest: a Figure 02 pilot at BMW Spartanburg moved 90,000+ components over roughly 1,250 hours in 11 months, and a Figure 03 logistics deployment is expanding through 2026.253
Apptronik raised $520M at a $5B valuation in February 2026, a Series A totalling over $935M and roughly $1B raised overall, from investors including Google, Mercedes-Benz, John Deere and the Qatar Investment Authority.254 Its Apollo robot works designated factory and warehouse zones at Mercedes-Benz, GXO and Jabil, targeting a unit cost around $50k.
Agility Robotics is going public via SPAC (Churchill Capital Corp XI, announced 24 June 2026) at roughly $2.5B,255 and its Digit has the most honest CV in the field: 100,000+ totes moved at GXO under Robots-as-a-Service, with Amazon, Toyota and Mercado Libre as customers.71
The cautionary entries matter just as much. Tesla Optimus paused production mid-2025 over overheating joint motors, short transmission life and weak hands; a Gen 3 low-volume pilot is expected summer to autumn 2026, with real volume unlikely before H1 2027.256 1X (Norway/US, OpenAI-backed, ~$125M+ raised) opened pre-orders for its ~30kg NEO home humanoid in late October 2025 at ~$20,000 or $499/month, with first US early-access deliveries in 2026 - but NEO's heavily teleoperated in practice.256
At the commodity end sits the Unitree G1, a ~$16,000 full humanoid unveiled at ICRA in May 2024 (some 2026 retail listings dip to $13,500), from Unitree Robotics, the Chinese firm Wang Xingxing founded in 2016 and first known for its cheap Go2 and B2 quadruped dogs.102
It stands 127-132cm, weighs ~35-47kg, walks up to 2 m/s, carries 3D LiDAR plus an Intel RealSense D435, runs about two hours on a quick-swap battery, and spans 23-43 DoF depending on configuration. It matters because it turned humanoid research from a six-figure endeavour into something a university lab can afford, roughly 10x cheaper than its rivals.
It did for humanoids what the SO-101 did for arms, and it's now the de-facto standard research platform, with a huge open-source ecosystem (Unitree's own GitHub, LeRobot-compatible pipelines, NVIDIA's Isaac GR00T end-to-end workflow17), ROS 2 support, Jetson Orin compute, teleoperation via Apple Vision Pro, Quest or PICO, and even an open-sourced Unitree VLA model, UnifoLM-VLA-0.
For actual research you want the EDU versions, bought direct from shop.unitree.com: EDU Standard at ~$43,900 with the full Python/C++ SDK, ranging up to $73,900, with EDU Plus adding arm and waist joints (~29 DoF) and optional Dex3-1 three-fingered force-controlled hands.102
Know what you're getting, though: the base G1's fixed grippers make it poor for serious manipulation, so budget extra for Dex3 hands and a teleop rig, and expect fragility and DIY software integration. It's a lab bench, not a product; DeepAware/SVRC lists the G1 at $18,400 (still a waitlist listing as of mid-2026) and leases it on a three-month minimum with lease-to-own, quote-gated, if you'd rather not commit $44k.161
PNDbotics (Beijing) shows where the mid-tier is heading. Its flagship Adam SP is a ~$100,000 prototype shown at CES 2026: 167cm, 62kg, 41 DoF including two five-fingered hands, 6 km/h top speed, 5kg payload, and 360 N·m peak torque from PSA quasi-direct-drive actuators; the company's raised tens of millions and is building a Tianjin plant sized for hundreds of units a year.257
More telling is Adam-U, a stationary 31-DoF data-acquisition and teleoperation rig from $45,000, bundled with a Noitom PNLink motion-capture suit and Inspire RH56E2 tactile hands, with pre-orders open. That's the industry pattern in miniature: humanoid makers increasingly sell data-collection pipelines and Robots-as-a-Service rather than just robots, because a trial or rental gets a customer collecting teleop data on the platform.
Notice what the money is actually chasing: not the bodies so much as the autonomy and manipulation AI (the VLA models) plus the data flywheel to train them, which is where Figure, Apptronik, 1X, Google DeepMind and NVIDIA really compete, for a labour market investors size in the trillions.74 A practical corollary: you can't buy a Figure, an Apollo or a Digit outright - they go out via pilots or Robots-as-a-Service contracts to enterprises. At the frontier the sane moves are renting capability via RaaS or buying a G1 to build on; don't pay six figures for a demo.
Cocktail-party take: "Locomotion and choreographed demos are real and cheap; economically-useful unsupervised dexterous autonomy at scale doesn't yet exist anywhere. Watch for paid deployments as the only honest signal - Agility moved 100,000+ totes at GXO under Robots-as-a-Service71; most home-humanoid marketing is teleoperated theatre."
Regulatory Friction
Where robots can actually be deployed, and under what legal regime - and, just as decisive, which markets to enter first. The commercial beachheads are unglamorous verticals with labour shortages: Remy (3PL warehouses), Flywheel (excavators), Salem (nuclear inspection), AutoPallet (boxes), Ultra and OS3 (industrial/hospitality labour). Even incumbent cobot vendors like AUBO (a Chinese maker of cheap six-axis arms) share the same read on the pain, and the classical-automation world agreeing with its would-be disruptors is what product-market-fit smoke looks like.
It stays unresolved because it diverges sharply by geography - the US allows near-permitless prototype testing on private land via self-certification; the EU's Machinery Regulation 2023/1230 (mandatory 20 Jan 2027) treats even an internal-use autonomous machine as needing a CE mark and third-party notified-body sign-off.105 A live Polish off-highway project shows the pattern: a working prototype, a willing site owner, and months lost to CE-marking questions and stakeholder consultation. The technical frontier and the deployment frontier are in different places.
The regulation's reach surprises people. It's been in force since July 2023 and becomes mandatory on 20 January 2027, when it fully replaces the Machinery Directive 2006/42/EC with no dual-application transition and a ten-year technical-documentation duty, and it bites on any machine "placed on the market" or "put into service" - internal use included.
The new high-risk Annex I categories (autonomous mobile machinery, self-learning systems, AI-based safety components) need a third-party notified body to sign off rather than a self-declaration; the narrow research exemption covers only machinery "specially designed and constructed for research purposes for temporary use in laboratories", not a prototype on a live jobsite; and retrofitting autonomy can count as a "substantial modification" that legally makes you the manufacturer of record, triggering a fresh conformity assessment.
Underground in Poland, add ATEX explosive-atmosphere certification (Directive 2014/34/EU, Group I gear for methane atmospheres) via notified bodies GIG-PIB or KOMAG, with oversight from Wyższy Urząd Górniczy (the State Mining Authority) under the Geological and Mining Law, plus extra flame-resistant and anti-static requirements.258 A European multinational's legal team will block the US "just bolt it on and run" workflow: even an internal pilot is a CE-marked machine on paper. Do the conformity maths before the hardware maths.
The most commercially mature robotics on Earth lives here: off-highway autonomy (Caterpillar, Komatsu, ASI, Flywheel AI), where the regulatory-engineering problem is a real moat.
Cocktail-party take: "Off-highway autonomy (mine haul trucks, excavators on closed sites) is the most boring, most successful autonomy on Earth, years ahead of robotaxis, because a fenced deterministic site removes the open-world long tail. The unsexy frontier there is getting autonomous machinery CE-marked in Europe before 2027."
(Two honourable-mention problems woven through all of the above: the embodiment gap - a human hand isn't a robot gripper, so human video alone isn't sufficient, which several companies attack by co-designing human capture gloves to match their robot hands; and onshoring / supply-chain security - US-made parts and defence dual-use, which HLabs and Bucket Robotics ride.)
The one-sentence summary: robots stopped being programmed and started being trained, and nearly everyone above is buying shovels for that gold rush - data shovels, hardware shovels, or infrastructure shovels.
Getting Hands On
If all of that left you wanting to touch the hardware, buy it in this order. A concrete ladder from $100 to $35k+, with what each rung teaches and where to buy. All prices are as of mid-2026.
Rung 0 - Just software (free). Before buying anything, install Hugging Face LeRobot and download open VLA weights (pi0/openpi and OpenVLA are Apache-2.0; GR00T-N1-2B is on Hugging Face). Inference on an open VLA needs a single ~12-24GB GPU; an RTX 4090 (24GB, ~$1,600-2,000) is the sweet spot and handles LoRA fine-tuning. This teaches the model side of How Robots are Learning (BC, ACT, Diffusion Policy, VLAs) without any robot.
Rung 1 - SO-101 and LeRobot (~$100-500). It's the cheapest fluency in the field; start here. The SO-101 is a deliberately cheap, open-source 6-DOF arm: most of its structure is plastic parts you 3D-print yourself, bolted around six off-the-shelf Feetech STS3215 serial servos (motors that report their own position, so the computer always knows the arm's pose).
It was designed by The Robot Studio in collaboration with Hugging Face and released on 28 April 2025 as the Apache-2.0 successor to October 2024's SO-100, fixing that model's rough edges: cleaner wiring that ends the joint-3 disconnects, assembly without gear removal, and lower-friction leader motors.224259
Buy a leader+follower pair; the pairing is what you're buying, because one arm is the human input device and one is the learner. The leader mixes gear ratios (1/191 base, 1/345 shoulder lift, 1/191 elbow, 1/147 wrist and gripper) so it holds its own weight yet moves with low force under your hand, while the follower runs 1/345 throughout.
You physically puppet the leader, the follower mirrors you while 1-2 cheap USB webcams record (add a wrist cam and an overhead cam; the vision policies require them and kits usually don't include them - the $272 RealSense D405 from Sensors is the standard depth upgrade for the wrist), and 30-50 repetitions of a task give you a demonstration dataset. LeRobot then trains a policy on it (ACT, Diffusion Policy, SmolVLA), and afterwards the follower performs the task autonomously: Lego block into bin, cloth folded, objects sorted.
The official bill of materials is ~$229.88 for the pair (EUR 226.30; a single follower arm is ~$121.94, which is what the "starting at $100" marketing means) if you 3D-print the parts yourself, or ~$35-50 more via a print service.260 If you want less hassle, Seeed Studio sells a ~$220 motor kit, and WowRobo, PartaBot, Hiwonder and ThinkRobotics sell kits up to fully-assembled, US-shipped units at $300-500 after assembly premium and tariffs. Get the standard 7.4V STS3215s (~16.5 kg-cm of torque) unless you specifically need the ~30 kg-cm 12V variant.106
The whole teleoperate → record → train → deploy loop runs from a normal laptop in minutes to hours via LeRobot's CLI tools (lerobot-find-port, lerobot-setup-motors, lerobot-calibrate, plus the Feetech SDK); you don't strictly need a GPU for teleoperation or data collection, though training is much faster with one.
The arm matters less than the ecosystem it plugs into. LeRobot is Hugging Face's Apache-2.0 robot-learning library, less "the PyTorch of robotics" than Transformers, Datasets and a hardware driver stack in one repo (see Learning), led by Remi Cadene (ex-Tesla Optimus). It implements ACT, Diffusion Policy, VQ-BeT, TD-MPC, HIL-SERL, Pi0, SmolVLA and NVIDIA's GR00T policies, and the LeRobot paper was presented at ICLR 2026.224
Hugging Face is explicitly trying to do for robotics what it did for NLP (an open hub of shared datasets, pretrained policies and reproducible hardware), and even acquired the French humanoid startup Pollen Robotics. Community datasets recorded on SO-101s get uploaded to the Hub and reused, so every buyer compounds the collective training corpus. The June 2025 worldwide hackathon alone drew 3,000+ registered participants across 100+ local events with 50+ SO-101s provided, and the events have continued into 2026 (Munich, for one).261
Be clear-eyed about what ~$230 buys: hobbyist-grade hardware (3D-printed plastic, ~$200 of servos, low payload, modest repeatability) that nobody deploys in production. But industrial arms cost $10k-50k+, which priced students and hobbyists out of manipulation research entirely, and the SO-101 collapses a full teleoperation-and-training rig to roughly $100-250. The same LeRobot policies that run on it also target frontier humanoids, so it's the de-facto standard cheap embodiment for collecting demonstration data. It teaches leader-follower teleoperation, imitation learning and the entire LeRobot toolchain; start from the official docs and the community Discord.
Rung 2 - GELLO (<$300) or UMI (~$370). GELLO is a build-it-yourself leader device (~6-7 Dynamixel servos, ~30-min assembly) if you already own a Franka/UR5/xArm.153 UMI (the ~$370 handheld gripper + GoPro from The Full Robotics Stack) covers robot-free "in-the-wild" collection via SLAM.154 This rung teaches the two philosophies of data collection (teleop-input vs. handheld) and why data-collection throughput is a first-order lever.
Rung 3 - Almond Axol ($7,999 / $11,999). The Axol robot alone is $7,999; the full Axol Kit (robot + mobile base + 3x ZED cameras + NVIDIA Orin compute) is $11,999 (launch prices). It teaches real bimanual (dual 7-DoF) manipulation, a 500Hz CAN control loop, VR teleop, IK, and LeRobot: a real step up from hobby arms into capable dual-arm hardware. Buy direct from almond.bot.
Rung 3-alt - Anvil kits ($4,730-15,120) or TRLC ($3,999-6,999). Anvil Robotics is the "you can now train @physical_int style robots in 1 day for only $5k" company, if you caught the five-tweet launch thread posted on 16 March 2026 by co-founder handle @0x796F; it pulled ~1.27M impressions (749 likes, 317 bookmarks, 81 reposts).262 The name-drop is Physical Intelligence, whose funding curve you saw earlier, because Anvil's kits exist to train that class of VLA policy (pi0, pi0.5).
The company behind the thread is a Palo Alto startup founded in July 2025 with manufacturing and ops split between SF and Taiwan, pitching itself as "Legos for robots": founders Mike Xia (CEO; co-founded Lumina Industries, then Chief Product Officer at Voltage Park) and CTO Vijay Pradeep, with $6.5M raised across a $1M pre-seed and a $5.5M seed led by Matter Venture Partners and Humba Ventures, joined by DNX Ventures, Supercharge.vc, Spacecadet, Position Ventures and Superhuman founder Vivek Sodera.263
The pitch lands because assembling a teleop rig normally takes a skilled team months: every lab has been rebuilding the same plumbing (teleop rigs, IK solvers, camera sync, real-time control, data pipelines) from scratch. Anvil sells it ready-made, the robotics analogue of buying an assembled GPU cluster instead of wiring your own data centre, so you start generating training data on day one instead of month six.
A kit bundles the arm(s), a Meta Quest headset as the teleop controller (via a Quest APK), three 60fps camera streams, a control box, a 6-DOF IK solver and a 500Hz (up to ~1,000Hz) real-time control bus, with "no-code" data-collection and training software built on ROS 2, Foxglove and LeRobot.
The hardware itself is productised open source: OpenARM from Tokyo's Enactic (~$6,500 bimanual) and OpenYAM, derived from I2RT Robotics' YAM ("Yet Another Manipulator") desktop arm. That's how Anvil hits its price points, and it means the moat is integration, software and speed-to-ship rather than novel hardware IP.
On pricing, the "$5k" headline means the cheapest single-arm teleop config: the OpenYAM Quest Teleop Kit at $4,730. Budget realistically from the shop.anvil.bot catalogue: the bare OpenYAM arm is $1,900, a Quest 3S teleop kit $350, cameras $180 each (you'll want about three), a customisable gripper $400 and the Anvil Platform + Devbox compute $1,900, so a usable single-arm station lands around $5-7k.264
For imitation-learning collection you typically want the leader-follower (puppet) setup: the OpenARM 2.0 Leader-Follower Kit is $15,120, single-arm OpenARM 2.0 Quest Teleop is $9,370 (on sale from $9,970), the bare OpenARM 2.0 is $5,600, dexterous-hand configs start at $14,600 and a wheeled lift base adds $800. Kits are quoted at 48-hour dispatch and 1-2 day air freight, but many SKUs show sold out, so expect a waitlist (the team was responsive to inbound DMs at launch).
It's not vaporware: 100+ robots shipped, 50+ customers and seven-figure revenue, all inbound and word-of-mouth, with Nvidia's GEAR lab, Path Robotics, Zeon Systems and a Portland chocolate factory among the named customers (the robots are roughly the size of a middle-school child).263
The verdict: "train in 1 day" is marketing. You get a working data-collection-and-training loop on day one, but a reliable policy still needs meaningful demonstration data; Anvil sits one layer below the frontier as infrastructure, well above a $100 toy arm and well short of a research breakthrough. Docs live at docs.anvil.bot, code at github.com/anvil-robotics.
The alternative at this rung is TRLC's DK1 ($3,999) or DK1-X ($6,999), profiled with the other arm makers in the hardware companies. Both routes are ready-made "professional-grade 500Hz control" data-collection rigs teaching the same loop as the Axol with different hardware lineages; pick one if you want the most-integrated, least-assembly option. You can also try before buying: SVRC leases the DK1 at a published $499 a month, return anytime, and resells it turnkey at $8,500 - don't confuse that markup with TRLC's direct prices.161
Rung 4 - Trossen / ALOHA (~$9k-38k). The Trossen AI Solo ($9k) is the single-pair entry; the Stationary ($24k) is the modern calibration-ready successor to the famous ~$20k Stanford ALOHA rig (2 WidowX AI leader-follower pairs + 4 depth cameras); the Mobile tops out near $38k with a laptop, and a bare WidowX AI base arm is ~$4.5k if you only want the actuator platform (exact list prices in Data collection).265
Or DIY ALOHA from Stanford's open BOM (~$18.5-20k). It's the reference bimanual data-collection hardware that underlies a huge fraction of academic imitation-learning papers. Buy from store.trossenrobotics.com (confirm tariff-adjusted price at ship time); Trossen sells outright only, with no rental programme anywhere on its store, but SVRC leases a Mobile ALOHA and a leader-follower teleop rig on quote (one-month minimum; the store's own hidden price markup puts the teleop rig at $1,800 and Mobile ALOHA at $3,200 a month, though no rate is rendered on the page).161
The off-ladder rung - a real cobot, used or rented. Everything above is data-collection hardware; the arm you'd eventually deploy on is the UR5e/UR10e class from Two Eras, and its $30-60k new price is the least interesting way to get one. Universal Robots runs its own leasing programme (UR Financial Services, with vendor-finance firm DLL, through the distributor network),266 SVRC leases UR3e and UR5e arms on one-month minimums with maintenance included (quote-gated, like nearly everything in this market),161 and one 2026 rental guide pegs a UR5e-class arm at roughly $2,500-6,000 a month - treat that as the going band, since no lessor publishes model-specific rates.267
Used is the budget path: complete UR5 systems (arm + controller + teach pendant) ask roughly $5,600-13,500 on eBay, and US surplus dealers like Robots Done Right carry UR3-through-UR16e stock, priced on contact.268 And if you want an hour on a real cobot before committing to any of this, The Construct rents remote bench time on a live UR3e (gripper plus 3D camera, bookable 24/7 over the internet) as part of its ROS-learning platform.269
Rung 5 - Unitree G1 (~$16k base, ~$43.9k EDU). The base G1 (~$13,500-16,000) is for locomotion and education demos; the G1 EDU Standard (~$43,900) is the real research choice, with full SDK, ROS 2, Jetson Orin, and LeRobot/Isaac GR00T pipelines. You'll want to budget extra for Dex3 dexterous hands and a teleop rig (Apple Vision Pro or Quest ~$500). Buy from shop.unitree.com, or lease from SVRC on a three-month minimum (quote-gated, lease-to-own available).161 This rung teaches the full humanoid stack; it's the standard "borrow one" research platform. Don't buy Figure/Apptronik/Agility (not sold outright; RaaS/pilot only), and don't expect true autonomy from 1X NEO ($20k or $499/mo, largely teleoperated).
General rule: at the frontier, rent capability via Robots-as-a-Service or buy a G1 to build your own - do not pay six figures for a demo. And for a technical founder trying to speak the language, Rungs 0-2 (LeRobot + an SO-101 pair + a weekend) buy you more genuine fluency per dollar than anything else in this post.
The Glossary
Terms a robotics founder uses daily, one crisp sentence each.
The language, mapped to the stack. Filter by layer and click a term to read its one-line definition — then flip on QUIZ and try to name the term from its definition alone.
A robot policy built on a pretrained vision-language model with an added "action head," so it inherits web knowledge and can generalise to novel instructions and objects.
- ROS 2 (Robot Operating System 2): The dominant open-source robotics middleware - not an OS but a publish/subscribe messaging bus where software "nodes" exchange "messages" on named "topics."
- MCAP: An open, self-describing container file format (created by Foxglove, foxglove.dev) that stores many timestamped channels of sensor data in one file; now ROS 2's default bag format.
- Bag file: A recording of all a robot's sensor/software topics to a file for later replay and debugging - robotics' black-box flight recorder.
- Teleoperation: A human remotely controlling (or partially assisting) a robot instead of it acting fully autonomously.
- Leader-follower: A teleop setup where a human moves a small "leader" arm and a larger "follower" robot copies its joint angles in real time.
- Imitation learning / behaviour cloning (BC): Training a robot by supervised learning to copy human demonstrations - the robotics version of how LLMs pre-train on text.
- ACT (Action Chunking with Transformers): A 2023 imitation-learning method that predicts chunks of future actions at once (via a transformer + variational autoencoder), fixing behaviour cloning's compounding-error problem.
- Diffusion Policy: A 2023 method that generates smooth action sequences by iterative denoising, borrowing the maths behind image generators, preferred for contact-rich tasks.
- VLA (Vision-Language-Action model): A robot policy built on a pretrained vision-language model with an added "action head," so it inherits web knowledge and can generalise to novel instructions and objects.
- pi0 / openpi: Physical Intelligence's flagship open-source VLA (pi0 and pi0.5), built on a vision-language backbone plus a flow-matching "action expert," released under Apache-2.0.
- OpenVLA: An open-source 7B-parameter VLA from Stanford/Berkeley (2024), trained on the Open X-Embodiment dataset of 970k real robot demos.
- GR00T: NVIDIA's open humanoid foundation model (Isaac GR00T N1/N1.5), a dual-system VLA aimed at humanoids, with weights on Hugging Face.
- Sim-to-real: Training a policy in a physics simulator then transferring it to a real robot, across the "reality gap" (reported 24-30% performance drop for vision policies).
- Domain randomisation: Randomising textures, lighting, friction and physics in simulation so the trained policy is reliable enough to survive the real world.
- Digital twin: A virtual replica of a real workspace used for simulation, increasingly reconstructed from ~10-20 phone photos via 3D Gaussian Splatting.
- Egocentric data: First-person video from a head/chest-mounted camera showing the wearer's viewpoint and hands - a near-demonstration for humanoids because it mirrors a robot's view.
- Ego4D: Meta's canonical academic egocentric dataset - 3,670 hours of first-person daily-life video from 931 wearers across 9 countries (licensed non-commercial).
- Proprioception vs. tactile: Proprioception is the robot sensing its own joint positions and torques (body coordination); tactile sensing is what's happening at the skin-object contact (force, slip, texture).
- GelSight: A vision-based tactile sensor - a camera behind a soft silicone gel photographs the gel's deformation to compute a high-resolution 3D depth map of the contact.
- DIGIT: Meta's low-cost (~$15 BOM) open-source vision-based tactile sensor, small enough to mount on multi-finger robot hands.
- DoF (degrees of freedom): The number of independent joints/axes a robot can move - 6-DoF for an arm-plus-wrist, ~20-27 for a dexterous hand.
- End effector: The tool bolted to a robot arm's wrist (gripper, suction cup, welding torch) - also called end-of-arm tooling (EOAT).
- Parallel gripper: A two-finger gripper whose jaws slide toward each other to pinch or wrap an object (e.g. the Robotiq 2F-85).
- Dexterous hand: A multi-fingered, high-DoF robot hand (LEAP Hand, Shadow DEX-EE) for fine in-hand manipulation, vs. a simple two-finger gripper.
- Cobot (collaborative robot): A lightweight, force-limited arm safe enough to work next to humans without a cage because it senses collisions and stops (Universal Robots, AUBO).
- Delta robot: A fast, ceiling-mounted "spider" of three parallelogram-linkage arms with fixed top motors, used for high-speed light pick-and-place (ABB FlexPicker).
- AMR (Autonomous Mobile Robot): A free-roaming wheeled robot that navigates by cameras/LiDAR (e.g. Locus Robots), as opposed to a fixed arm or rail-guided system.
- Payload / reach: The maximum weight a robot arm can carry, and the maximum distance its gripper can extend - the two headline arm specs.
- IK (inverse kinematics): The maths of computing the joint angles needed to place a gripper at a desired position and orientation.
- CAN bus: A two-wire automotive-style communication bus that lets many motors and boards share one wiring loom on a robot.
- Jetson: NVIDIA's family of onboard AI computers (system-on-modules) - Orin (mainstream) and Thor (humanoid-grade) - the de-facto robot "brain."
- Carrier board: The board a Jetson module plugs into (via a 699-pin connector) to expose real-world ports (USB, Ethernet, PCIe, cameras, power); the module can't run standalone.
- ZED: Stereolabs' line of stereo depth cameras (ZED X, ZED X Mini) that compute per-pixel 3D depth from two lenses, popular robot "eyes."
- IMU (inertial measurement unit): An accelerometer + gyroscope chip that reports a device's orientation and motion - the robot's inner-ear.
- Encoder: A sensor that reports a motor/joint's position, letting the computer always know the arm's pose (smart servos have them built in).
- PLC (programmable logic controller): A rugged industrial computer running deterministic logic - the classical-automation "brain," pre-physical-AI.
- RaaS (Robots-as-a-Service): Renting robots on a monthly fee (hardware + software + maintenance bundled) instead of buying them outright, common for warehouse fleets.
- 3PL (third-party logistics): A company that runs warehouses and ships orders on contract for other brands - the largest buyer segment for warehouse robots, and a hard automation case due to churning SKUs.
- Edge deployment: Running AI/perception on the robot itself rather than in the cloud, for low latency and because robots can't assume a network.
- Evals / policy evaluation: Measuring whether a learned policy actually succeeds at a task - an unsolved, scaling-limited problem now sold as a product (Instance Labs, instancelabs.ai).
- Data flywheel: The loop where deploying robots (or teleop rigs) collects proprietary demonstration data that trains a better model, which enables more deployment.
- Embodiment gap: The mismatch between a human body and a robot body (a hand isn't a gripper) that corrupts human-demonstration data unless corrected.
- MEMS: Microelectromechanical systems - tiny fabricated mechanical/electrical structures, one basis for compact tactile-sensor "taxel" arrays.
- Taxel: A "tactile pixel" - one sensing point in a tactile-sensor array (Meta's Digit 360 claims ~8.3 million).
- FOC (field-oriented control): The motor-control maths that makes a brushless motor smooth, quiet and torque-accurate (HLabs' board uses the open-source SimpleFOC library).
- Backdrivable / quasi-direct-drive: A low-gear-ratio actuator you can push and it pushes back, letting the robot "feel" contact - key for dexterity (Origami's 15:1 hand).
- Rollout / episode: One recorded attempt at a task, the unit of data in robot learning and the thing evals label pass/fail.
Reading List
Ordered roughly by leverage. "Skim" = read abstract + figures; "Read" = worth the full time. (The classical-core maths sources are footnoted at their point of use in The Classical Core.)
Foundational - read these to speak the language:
- Physical Intelligence's pi0.5 blog (pi.website/blog/pi05) - Read. The clearest window into a frontier VLA, including the unflattering ablations showing generalisation collapse (94% → 31%) when diverse data is removed. This single post teaches why data diversity is everything.
- Hugging Face LeRobot docs (huggingface.co/docs/lerobot/so101) - Read/do. The hands-on on-ramp; following the SO-101 tutorial end-to-end will teach you more than any amount of reading.
- OpenVLA project page (openvla.github.io) - Skim. The reference open VLA; understand the architecture (SigLIP+DINOv2 vision encoder + Llama-2 backbone) and the Open X-Embodiment training data.
Hardware and data collection:
- ALOHA paper (arXiv 2304.13705) and Mobile ALOHA (2401.02117) - Skim. The origin of the cheap-bimanual-teleop + ACT recipe that anchors the whole field.
- UMI paper (arXiv 2402.10329, umi-gripper.github.io) - Read. The most original idea in data collection (robot-free handheld capture via SLAM), and still spawning variants.
- GELLO (arXiv 2309.13037, wuphilipp.github.io/gello_site) - Skim. The clever <$300 teleop-leader design.
Tactile - if you go deep on dexterity:
- GelSight demo video (via gelsight.com) - Watch (~5 min). Makes optical touch sensing click visually.
- AnySkin paper (arXiv 2409.08276) - Skim. Replaceable magnetic tactile skin with zero-shot cross-instance policy transfer; the one tactile paper to actually know.
- Meta Digit 360 announcement (businesswire, Oct 2024) - Skim. The bleeding edge (~8.3M taxels, 1mN sensitivity).
Egocentric data - the humanoid data bet:
- Ego4D (ego4d-data.org, arXiv 2110.07058) - Skim. The canonical dataset; understand scale and the non-commercial licence.
- Figure Project Go-Big (figure.ai/news/project-go-big) - Read. The proof point: a humanoid trained to navigate homes on 100% human egocentric video, zero robot demos.
- EgoMimic (Meta AI blog) - Skim. Mount the same Aria glasses on human and robot; 400% task gain from 90 minutes of recordings.
- PointWorld (point-world.github.io, CVPR 2026) - Skim. The one actually robotics paper of the PointWorld/EgoX/UniVidX trio (the other two are generative video - skip unless curious).
Infrastructure & the "picks-and-shovels" thesis:
- Foxglove Series B coverage (therobotreport.com / businesswire, Nov 2025) - Skim. The clearest articulation of "data platform for physical AI" as a venture category, plus MCAP.
- Instance Labs (instancelabs.ai, demo.instancelabs.ai) - Skim / try the demo. The evals-as-a-product frontier; run the public success-detector demo to feel the problem.
Market & reality-check reading:
- "Collecting robot training data is dirty, unglamorous work" (TechCrunch, June 2026, on XDOF) - Read. The unvarnished economics of the data layer.
- Human Archive coverage (TechCrunch/inc42, May 2026) - Skim. The uncomfortable labour economics (gig workers at ~$1/hr) behind "physical AI."
- "Teleop, not autonomy, the path for 1X NEO" (therobotreport.com) + Waymo fleet-response blog (waymo.com, May 2024) - Read. The "human behind the robot" open secret, from both a humanoid and a robotaxi angle.
- Origami Robotics "dexterity deadlocks" (origami-robotics.com/blog/dexterity-deadlocks.html) - Read. The sharpest short argument that hardware (actuator transparency) caps what any learning algorithm can achieve.
- DeepAware/SVRC "State of Robotics 2026" (roboticscenter.ai/state-of-robotics-2026) - Skim. A market snapshot (~$38B market, VLA adoption ~40% of new deployments, teleop cost falling) - useful numbers, but remember it's a vendor's report.
One framing to carry into every founder chat: the field has bifurcated into frontier (VLA models, novel actuators/tactile, the data-collection idea) and rapidly-commoditising infrastructure (cheap arms, ROS 2/MCAP, observability, the data-collection labour). Knowing which side any given company, product, or paper sits on - and that the true bottleneck everywhere is data and reliability, not model cleverness - is most of what "speaking the same language" actually means.
Footnotes
-
Source: "Robotiq 2F-85", Robotics Center; "2F-85 and 2F-140 Instruction Manual", Robotiq, 2019 ↩
-
Source: "Robotiq raises $23M for collaborative robots", The Robot Report, 2018 ↩
-
Source: "Global Collaborative Robot (Cobot) Market", Emergen Research, 2025; "Universal Robots increases payload capacity of UR20, UR30 cobots", The Robot Report, 2025 ↩
-
Source: "How much does a Universal Robots cobot cost?", Standard Bots, 2025; "UR20", Universal Robots ↩
-
Source: "AUBO-i5", AUBO Robotics USA; "AUBO Robotics i5", Unchained Robotics ↩
-
Source: "Delta robot" and "Reymond Clavel", Wikipedia; "IRB 365", ABB; "ABB launches 5-axis high-speed delta robot", The Robot Report ↩
-
Source: "Cobot End Effector Market", Next Move Strategy Consulting; "Collaborative Robot Market", MarketsandMarkets ↩
-
Zhao et al., "Learning Fine-Grained Bimanual Manipulation with Low-Cost Hardware" (2023) - arXiv 2304.13705; ALOHA project page ↩
-
Chi et al., "Diffusion Policy: Visuomotor Policy Learning via Action Diffusion" (2023) - arXiv 2303.04137; project page ↩
-
Source: "RT-2: Vision-Language-Action Models Transfer Web Knowledge to Robotic Control", Google DeepMind, 2023 ↩
-
Kim et al., "OpenVLA: An Open-Source Vision-Language-Action Model" (2024) - arXiv 2406.09246; project page ↩
-
Source: "Physical Intelligence raises $400M for foundation models for robotics", The Robot Report, 2024; "Robotics AI startup Physical Intelligence raises $400M from Jeff Bezos and OpenAI at $2.4 billion valuation", TechStartups, 2024; "AI Robotics Lab in Talks to Raise $1 Billion at $11 Billion Valuation", Bloomberg, 2026 (a reported target, not a closed round) ↩ ↩2
-
Physical Intelligence, "π0: Our First Generalist Policy" (2024) - physicalintelligence.company ↩ ↩2
-
Physical Intelligence, "π0.5: a VLA with Open-World Generalization" (2025) - pi.website ↩ ↩2 ↩3
-
Physical Intelligence, "π0.7: a Steerable Generalist Robotic Foundation Model with Emergent Capabilities" (2026) - pi.website; paper PDF ↩
-
Source: "openpi" (Apache-2.0 repository with model checkpoints and hardware requirements), Physical Intelligence, 2024-2025 ↩ ↩2
-
NVIDIA, "Isaac GR00T" developer platform - developer.nvidia.com; GR00T N1 paper - arXiv 2503.14734 ↩ ↩2
-
Source: "GR00T N1.5", NVIDIA Research, 2025; "Isaac-GR00T" repository, NVIDIA, GitHub ↩
-
Jim Fan: "Robotics' End Game", Sequoia Capital AI Ascent, 2026 ↩ ↩2 ↩3 ↩4 ↩5 ↩6 ↩7 ↩8 ↩9
-
NVIDIA GEAR: "World Action Models are Zero-shot Policies" (DreamZero), arXiv 2602.15922, 2026 ↩ ↩2
-
James Wigglesworth: "The Hidden Opportunity of Robotics Datasets", podcast interview, YouTube, 2026 ↩ ↩2 ↩3 ↩4 ↩5 ↩6 ↩7 ↩8
-
Typical AI (@typical.a.i): dancing-statues AI video, Instagram, 29 May 2023 ↩
-
Mustafa Shukor, Dana Aubakirova, Francesco Capuano et al.: "SmolVLA: A Vision-Language-Action Model for Affordable and Efficient Robotics", Hugging Face, June 2025 ↩
-
UN World Population Prospects 2024; South Korea trajectory via Our World in Data (2024) ↩
-
Sourcing Journal, "Sewbots Making T-Shirts at New Arkansas Factory" (2017); Yahoo Finance, "SoftWear Automation secures $20M advance" (2025) ↩
-
WWD/Sourcing Journal, "Bangladesh's new minimum wage" (2023); IBISWorld, "Global apparel manufacturing market size" ↩
-
BLS Occupational Outlook Handbook, "Home Health and Personal Care Aides" (2024-34 projections) ↩
-
WHO, "Health workforce" (projected ~10M shortfall by 2030); Japan MHLW projection via ISVD (2026) ↩
-
Austan Goolsbee and Chad Syverson, "The Strange and Awful Path of Productivity in the US Construction Sector", NBER Working Paper 30845 (2023) ↩
-
NAHB, "Cost of Constructing a Home in 2024" (64.4%); McKinsey Global Institute, "Reinventing construction through a productivity revolution" (2017); ABC workforce projection via LBM Journal (2025) ↩
-
Source: "Caterpillar sets out to hit over 2,000 autonomous mining trucks by 2030", International Mining, 2025; "Caterpillar achieves two billion tonnes hauled with autonomous trucks system", Caterpillar, 2025 ↩
-
Source: "Autonomous and semi-autonomous haul trucks guide", DirtMatch, 2026; "Autonomous systems in mining and agriculture vehicles", Robotics & Automation News, 2026 ↩
-
Source: "ASI's Scythe acquisition amplifies autonomous capability", Yahoo Finance, 2026; company site asirobots.com ↩
-
Source: "Flywheel AI", Y Combinator directory, 2025; "Launch HN: Flywheel AI", Hacker News, 2025 ↩
-
Source: "Excavator dataset", Hugging Face (FlywheelAI), 2025; "FrodoBots Lab raises $8M to launch BitRobot", The Defiant, 2025 ↩ ↩2
-
Source: "Flywheel", AngelsRound, 2025; company site useflywheel.ai ↩
-
Deloitte / The Manufacturing Institute, "Manufacturers need as many as 3.8 million new employees by 2033" (2024); American Welding Society, "Welding Workforce Data" ↩
-
AppleInsider, "How Apple learned automation can't match human skill" (2020); Quartz, "Foxconn is replacing 60,000 workers with robots" (2016) ↩
-
ASME, "Harnessing AI, Robotics to Debone Chickens"; Scott Technology / JBS, "Global protein deals" (2025) ↩
-
AgFunder News, "Chef Robotics raises $43M Series A to scale AI-enabled robotics in meal assembly" (2025) ↩
-
Amazon, "Vulcan, our first robot with a sense of touch" (2025); attrition estimate from a leaked internal memo via Engadget (2022) ↩
-
Baker et al., "On-farm food loss in northern and central California", Resources, Conservation and Recycling (2019); American Farm Bureau, "H-2A program use continues to soar" (2024) ↩
-
Source: "Bucket Robotics", Y Combinator directory, 2024; company site bucket.bot ↩
-
Source: "How YC-backed Bucket Robotics survived its first CES", TechCrunch, 2026 ↩
-
Source: "OS3", Y Combinator directory, 2026; company site os3robotics.com ↩
-
Source: "Warehouse automation ROI by size", Robotomated, 2025; "The difference between manual and automated pick and pack", Dollan Prep Center ↩ ↩2
-
Source: "Amazon.com to acquire Kiva Systems", Supply & Demand Chain Executive, 2012; "Amazon Robotics", Wikipedia ↩
-
Source: "Locus Robotics surpasses 5 billion pick milestone", PR Newswire, 2025; "Locus Robotics", Sacra, 2026 ↩
-
Source: "Warehouse automation market report", Mordor Intelligence, 2025 ↩
-
Source: "Navigating warehouse automation strategy for the distributor market", McKinsey, 2024 ↩
-
Source: "Amazon hires Covariant founders, inks licensing deal with robotics AI startup", GeekWire, 2024 ↩ ↩2
-
Source: "Ultra: building the next million robots to automate American warehouses", Y Combinator Launch, 2024; company site ultra.tech ↩
-
Source: "Remy AI", Y Combinator directory, 2026; company site tryremy.ai ↩
-
Source: "AutoPallet Robotics: the future of case picking in warehouses", Y Combinator Launch, 2024; company site autopallet.bot ↩
-
Source: "How upside-down AutoPallet robots solve palletizing density", The Robot Report, 2026 ↩
-
National Audit Office, "Decommissioning Sellafield: managing risks from the nuclear legacy" (2024) ↩
-
Safecast, "No, radiation levels at Fukushima Daiichi are not rising" (530 Sv/h reading and robot failure); TEPCO fuel-debris retrieval updates ↩
-
Mining Technology, "China becomes global leader in autonomous surface mining trucks" (2025); Komatsu, "1,000th ultra-class autonomous haul truck" (2026); Rio Tinto autonomous fleet record via Global Mining Review ↩
-
CNBC, "Gecko Robotics raises $125 million, surpassing billion-dollar valuation" (2025) ↩
-
BLS, "Census of Fatal Occupational Injuries, 2024"; workers'-compensation costs via Insurance Information Institute / NASI ↩
-
Source: "Salem Robotics", company site, 2026; UT Austin Nuclear and Applied Robotics Group ↩
-
Source: "Market size analysis: the $4.39 billion hazardous-environment robots market", openPR, 2026 ↩
-
Resource Recycling, "AMP Robotics enters facility operations via RDS deal" (2025) ↩
-
Hagelüken & Corti, "Recycling of gold from electronics", Gold Bulletin (2010) ↩
-
S&P Global, "The Future of Copper" (2022), full report; IEA, "Global Critical Minerals Outlook 2025" ↩
-
IEA, "Global Critical Minerals Outlook 2024" (graphite demand ~4x by 2040, NZE); recovery-rate estimate via Stellarix (2024) - a consultancy figure, treat as indicative ↩
-
Adamas Intelligence NdPr shortfall projections via Canadian Mining Journal (2022) and "Rare Earth Magnet Market Outlook to 2040"; IEA, "Rare Earth Elements" (2026) ↩
-
Johnson Matthey, "PGM Market Report May 2025" - 2024 secondary supply: platinum 19.3%, palladium 30.6%, rhodium 29.1% of total supply ↩
-
TechCrunch on Covariant's Amazon absorption, "Covariant is building ChatGPT for robots" + IEEE Spectrum, "Amazon stowing robots"; Berkshire Grey take-private via Global Venturing (2023) ↩
-
GXO press release, "GXO Signs Industry-First Multi-Year Agreement with Agility Robotics" - gxo.com ↩ ↩2 ↩3
-
Kerbl, Kopanas, Leimkühler and Drettakis, "3D Gaussian Splatting for Real-Time Radiance Field Rendering", SIGGRAPH 2023 ↩ ↩2
-
NVIDIA, "Omniverse NuRec" neural reconstruction libraries (Isaac Sim 5.0+); Dai et al. (Stanford), "Automated Creation of Digital Cousins for Robust Policy Learning", CoRL 2024 ↩ ↩2
-
Citi GPS via Citigroup Insights (2024); Morgan Stanley, "Humanoid robots: a $5 trillion market by 2050" (2025); Brett Adcock wage-bill framing via Sourcery ↩ ↩2
-
Philippe Aghion, Benjamin F. Jones and Charles I. Jones, "Artificial Intelligence and Economic Growth", NBER (2017) ↩
-
Joseph Cox: "Facebook's Powerful Large Language Model Leaks Online", Vice, 2023; Rohan Taori et al.: "Alpaca: A Strong, Replicable Instruction-Following Model", Stanford CRFM, 2023 ↩
-
Meta: "Meta and Microsoft Introduce the Next Generation of Llama", Meta AI, 2023 ↩
-
LangChain: "Announcing our $10M seed round led by Benchmark", LangChain Blog, 2023 ↩
-
Pinecone: "Pinecone Raises $100M in Series B Funding", Pinecone, 2023; Weaviate: "Weaviate Raises $50 Million Series B Funding", PR Newswire, 2023; Chroma: "Chroma raises $18M seed round", Chroma, 2023 ↩
-
Wikipedia contributors: "AutoGPT", Wikipedia; Significant Gravitas: "auto-gpt stuck in a loop of thinking (issue #2726)", GitHub, 2023 ↩
-
Anthropic: "Introducing 100K Context Windows", Anthropic, 2023; OpenAI: "New models and developer products announced at DevDay", OpenAI, 2023; Google: "Our next-generation model: Gemini 1.5", Google, 2024 ↩
-
OpenAI: "Hello GPT-4o", OpenAI, 2024; Anthropic: "Introducing the next generation of Claude", Anthropic, 2024 ↩
-
Max Woolf: "The Problem With LangChain", minimaxir.com, 2023; LangChain: "LangGraph", GitHub, 2024 ↩
-
Xiao Yang et al.: "CRAG - Comprehensive RAG Benchmark", NeurIPS, 2024; Microsoft Research: "GraphRAG: New tool for complex data discovery now on GitHub", Microsoft, 2024 ↩
-
Retool: "The new State of AI report: Real use cases and the AI stack", Retool, 2024 ↩
-
Mistral AI: "Mixtral of experts", Mistral AI, 2023; DeepSeek-AI: "DeepSeek-V3 Technical Report", arXiv, 2024 ↩
-
Epoch AI: "LLM inference prices have fallen rapidly but unequally across tasks", Epoch AI, 2025; Guido Appenzeller: "Welcome to LLMflation - LLM inference cost is going down fast", Andreessen Horowitz, 2024 ↩
-
vLLM Team: "vLLM: Easy, Fast, and Cheap LLM Serving with PagedAttention", vLLM Blog, 2023 ↩
-
Sacra: "Together AI revenue, valuation & funding", Sacra, 2026; Hayden Field: "Nvidia-backed Fireworks hits $17.5 billion valuation as companies pursue cheaper AI models", CNBC, 2026; Kif Leswing: "Nvidia buying AI chip startup Groq's assets for about $20 billion in its largest deal on record", CNBC, 2025 ↩
-
TechCrunch: "OpenRouter more than doubles valuation to $1.3B in a year", TechCrunch, 2026 ↩
-
Rubrik: "Rubrik to Acquire Predibase to Accelerate Agentic AI Adoption", Rubrik, 2025; TechCrunch: "CoreWeave acquires agent-training startup OpenPipe", TechCrunch, 2025 ↩
-
OpenAI: "Introducing OpenAI o1-preview", OpenAI, 2024; DeepSeek-AI: "DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning", arXiv, 2025; Derek Saul: "Biggest Market Loss In History: Nvidia Stock Sheds Nearly $600 Billion As DeepSeek Shakes AI Darling", Forbes, 2025 ↩
-
Anthropic: "Claude 3.7 Sonnet and Claude Code", Anthropic, 2025 ↩
-
Anthropic: "Introducing computer use, a new Claude 3.5 Sonnet, and Claude 3.5 Haiku", Anthropic, 2024; Cognition: "Introducing Devin, the first AI software engineer", Cognition, 2024 ↩
-
"+1 for 'context engineering' over 'prompt engineering'" ~ Andrej Karpathy, X, 2025; Anthropic: "Effective context engineering for AI agents", Anthropic Engineering, 2025 ↩
-
OpenAI: "Function calling and other API updates", OpenAI, 2023 ↩
-
Anthropic: "Introducing the Model Context Protocol", Anthropic, 2024; Linux Foundation: "Formation of the Agentic AI Foundation (AAIF)", Linux Foundation, 2025 ↩
-
Google: "Powering AI commerce with the new Agent Payments Protocol (AP2)", Google Cloud, 2025; Stripe: "Stripe powers Instant Checkout in ChatGPT and releases the Agentic Commerce Protocol", Stripe, 2025; Coinbase: "Introducing x402: a new standard for internet-native payments", Coinbase, 2025 ↩
-
Modal: "Modal's Series C: Raising $355M at a $4.65B valuation", Modal Blog, 2026 ↩
-
Sifted: "Anthropic hires cofounders of AI startup Humanloop", Sifted, 2025 ↩
-
Chang & Padir: "Sim2Real2Sim: Bridging the Gap Between Simulation and Real-World in Flexible Object Manipulation", arXiv 2002.02538, 2020 ↩
-
Source: "Unitree Robotics unveils G1 humanoid for $16K", The Robot Report, 2024; specs and EDU pricing via "Unitree G1 Complete Guide 2026", ZMP Robots, 2026 ↩ ↩2 ↩3
-
Source: "Teleoperation internet delay handling", Robotics Center, 2026 ↩ ↩2 ↩3
-
Source: "Learning Safe-Stoppability Monitors for Humanoid Robots", arXiv, 2026 ↩ ↩2
-
Regulation (EU) 2023/1230 on machinery (the Machinery Regulation) - eur-lex.europa.eu ↩ ↩2
-
Waveshare wiki, "ST3215 Servo" - waveshare.com ↩ ↩2 ↩3
-
Source: "DOF of the Human Arm", Northwestern Mechatronics Wiki ↩
-
Source: "Redundancy resolution of a human arm for controlling a seven DOF wearable robotic system", IEEE EMBC, 2011 ↩
-
Source: "Degrees of Freedom vs Functions of a Robotic Arm", Reach Robotics blog ↩
-
Source: "What are the six degrees of freedom in a robot arm?", Marginally Clever, 2014 ↩
-
Source: "Bambu Lab P1S", Bambu Lab store; "Bambu Lab printer prices", OriginalPricing; X2D specs via "Bambu Lab X2D specs and release date", Makers101, 2026 ↩
-
Source: "MetroY blue-laser 3D scanner" and "MetroY Ultra metrology solution", Revopoint ↩
-
Source: "Flexible PCB fabrication", JLCPCB; "FPC and rigid-flex PCB", PCBWay ↩
-
Source: "Rigol vs Siglent", HobbyistScope; Siglent bench scopes at Saelig ↩
-
Source: "IMU calibration specifications", VectorNav Inertial Navigation Primer ↩
-
Source: "Benchtop vibration table", CSZ Industrial ↩
-
Source: "Robotics and teleoperations solutions" and "MAX Transit Duo Pro", Peplink; "Chateau 5G ax", MikroTik ↩
-
Source: "Origami Robotics", Y Combinator company page; "Origami Robotics: make AI to learn physical skills easier than ever", Y Combinator launch, 2026 ↩
-
Source: "Quanting Xie", CMU Robotics Institute; "quantingxie.github.io"; Ryan Xie background via "Ryan Xie Leads Ground Robotics", CEO Weekly ↩
-
Source: "Axol: a dual-arm robot built for AI", Y Combinator launch, 2025; specs and pricing at almond.bot and docs.almond.bot ↩
-
Source: "The Robot Learning Company", Y Combinator; pricing at robot-learning.co; code at github.com/robot-learning-co/trlc-dk1 ↩
-
Source: "Built in four days, this $120 robot arm cleans a spill with help from GPT-4o", TechCrunch, 2024 ↩
-
Source: "HLabs: US-made robot actuators and electronics", Y Combinator launch, 2026; hlaboratories.com; Mystic via "Mystic", Y Combinator ↩
-
Source: "ACB v2.0", HLabs documentation; firmware at github.com/h-laboratories/acb-v2.0 ↩
-
Source: "Jetson Orin", NVIDIA; "Orin Module Comparison", Connect Tech ↩
-
Source: "Jetson Thor", NVIDIA, 2025; "$3,499 NVIDIA Jetson AGX Thor Developer Kit", CNX Software, 2025 ↩ ↩2
-
Source: "Connect Tech leads market with Jetson Thor carrier", Connect Tech, 2025; "Auvidea X242, an NVIDIA Jetson Thor T5000 industrial carrier board", CNX Software, 2025 ↩
-
Source: "ZED X Stereo Camera" and "ZED X Mini Stereo Camera", Stereolabs; "Stereolabs ZED Box Mini", CNX Software, 2025 ↩ ↩2
-
Source: "ZED X documentation" and "Isaac ROS setup and installation", Stereolabs ↩
-
Source: "RealSense D405" product page and store listing, RealSense, 2026; "Cameras", LeRobot documentation ↩
-
Source: "Fast-FoundationStereo: Real-Time Zero-Shot Stereo Matching", NVIDIA, 2026; the D405's projector-less passive design per "Introducing the RealSense D405", RealSense forum; left/right imager access per librealsense issue #10415 ↩
-
Source: "GelSight Mini", GelSight Inc. ↩
-
Source: "DIGIT", Meta AI, 2020; "DIGIT: A Novel Design for a Low-Cost Compact High-Resolution Tactile Sensor", Meta AI Research ↩
-
Source: "GelSight and Meta AI Introduce Digit 360 Tactile Sensor", Business Wire, 2024 ↩
-
Source: "Tactile Sensors", XELA Robotics; "Xela Robotics adds 2 enhancements to uSkin sensor family", The Robot Report ↩
-
Bhirangi et al. (NYU/Meta), "AnySkin: Plug-and-play Skin Sensing for Robotic Touch" (2024) - arxiv.org ↩
-
Source: "DEX-EE Series", Shadow Robot, 2024; "Shadow Robot DEX-EE hand takes manipulation to next level", The Robot Report, 2024 ↩
-
Source: "LEAP Hand: Low-Cost, Efficient, and Anthropomorphic Hand for Robot Learning", RSS 2023; "LEAP Hand v2", RSS 2025 ↩
-
Source: "Skild AI", Contrary Research; "Partnering with Skild: The Future of Robotic Intelligence", Sequoia Capital ↩
-
Source: "ROS 2: The Next Generation for Robust and Scalable Robotics Applications", Robotics and Automation News, 2026 ↩
-
SO-101 URDF community mirror - huggingface.co ↩ ↩2
-
Source: "Introducing the MCAP file format", Foxglove blog, 2022 ↩
-
Source: "InOrbit announces Series A investment", InOrbit press release, 2025; "InOrbit unveils OpenRobOps open-source fleet manager platform", Robotics 24/7, 2025 ↩
-
Source: "ROSscope: open-source observability platform for ROS 2 fleets, v0.1.0", Open Robotics Discourse, 2026 ↩
-
Source: "MuJoCo vs Isaac Sim", Robotics Center of Silicon Valley; benchmark figures also reported in "arXiv 2506.18123", arXiv, 2025 ↩ ↩2
-
Source: "Meet Genesis: An Open-Source Physics AI Engine Redefining Robotics with Ultra-Fast Simulations and Generative 4D Worlds", MarkTechPost, 2024 ↩
-
NVIDIA GEAR: "DreamDojo: A Generalist Robot World Model from Large-Scale Human Videos", arXiv 2602.06949, 2026 ↩
-
Source: "Instance Labs", instancelabs.ai (self-reported benchmark claims) ↩
-
Source: RoboArena and real-to-sim evaluation papers - "arXiv 2508.11117" and "arXiv 2512.16881", arXiv, 2025; "Simplify Generalist Robot Policy Evaluation in Simulation with NVIDIA Isaac Lab-Arena", NVIDIA Developer Blog, 2025 ↩
-
Wu et al., "GELLO: A General, Low-Cost, and Intuitive Teleoperation Framework for Robot Manipulators" (2023) - arXiv 2309.13037; project page ↩ ↩2 ↩3
-
Chi et al., "Universal Manipulation Interface: In-The-Wild Robot Teaching Without In-The-Wild Robots" (2024) - arXiv 2402.10329; project page ↩ ↩2 ↩3
-
Grauman et al., "Ego4D: Around the World in 3,000 Hours of Egocentric Video" (CVPR 2022) - arXiv 2110.07058; ego4d-data.org ↩ ↩2
-
Source: "Learning Fine-Grained Bimanual Manipulation with Low-Cost Hardware (ALOHA)", Zhao, Kumar, Levine & Finn, RSS 2023 ↩
-
Source: "Mobile ALOHA: Learning Bimanual Mobile Manipulation with Low-Cost Whole-Body Teleoperation", Stanford, 2024 ↩
-
Source: "ALOHA 2: An Enhanced Low-Cost Hardware for Bimanual Teleoperation", Google DeepMind, 2024; project page ↩
-
Source: "ALOHA is now Trossen AI", Trossen Robotics, 2025; WidowX AI product page ↩
-
Source: "Robot Rental & Leasing" and "TRLC DK1", Robotics Center of Silicon Valley (SVRC), fetched August 2026. The $18,400 G1 figure and the $1,800/$3,200-a-month rig rates appear only in the pages' structured markup (schema.org and data attributes), never as rendered prices - every listing shows "Request a Quote" ↩ ↩2 ↩3 ↩4 ↩5 ↩6
-
Wu et al., "FastUMI: A Scalable and Hardware-Independent Universal Manipulation Interface" (2024) - arXiv 2409.19499 ↩
-
Bloomberg: "Nvidia-Backed Robotics Startup Generalist AI Valued at $2 Billion", Bloomberg, 2026; Forbes: "Generalist Is Betting Its Robot-Training Gloves Will Usher In Robotics' ChatGPT Moment", Forbes, 2026 ↩
-
Sunday: "Sunday Raises $165M to Launch First Autonomous Robots by Thanksgiving", GlobeNewswire, 2026; Tech Funding News: "Sunday raises $165M at $1.15B valuation", 2026 ↩
-
Source: "Metagloves Pro", Manus Meta, 2025; pricing via "Manus Metagloves Pro", Knoxlabs ↩
-
Source: "NVIDIA Launches Isaac Teleop at GTC 2026 with Manus as the Official Data Glove", Humanoid Robotics Technology, 2026 ↩
-
Xu et al.: "DexUMI: Using Human Hand as the Universal Manipulation Interface for Dexterous Manipulation", arXiv 2505.21864, 2025 ↩
-
Source: "Project Aria", Meta; "Project Aria Gen 2", Meta blog, 2026 ↩
-
Source: "EgoMimic: scaling imitation learning with Project Aria", Meta AI / Georgia Tech, 2024 ↩
-
Source: "Project Go-Big", Figure, 2026 ↩
-
Source: "EgoScale", NVIDIA, arXiv 2602.16710, 2026; coverage via "NVIDIA's EgoScale unlocks high-dexterity robotics via 20,000 hours of human video", Humanoids Daily, 2026 ↩ ↩2 ↩3
-
Huang, Fox, Fei-Fei Li et al. (Stanford/NVIDIA), "PointWorld" (CVPR 2026 Highlight) - point-world.github.io ↩
-
Source: "State of Robotics 2026", DeepAware/SVRC vendor report, 2026 ↩ ↩2
-
Source: "Collecting robot training data is dirty, unglamorous work. Some AI labs are already paying XDOF to do it", TechCrunch, 2026 ↩ ↩2
-
Source: "CUPID: Curating Data your Robot Loves with Influence Functions", arXiv 2506.19121, 2025 ↩
-
Fujitsu Research + CMU, "Unsupervised Discovery of Long-Term Spatiotemporal Periodic Workflows in Human Activities" (WACV 2026 Oral) - arXiv 2511.14945 ↩ ↩2
-
The AI Insider: "SkillWaze and Amplibotics Partner to Train Collegiate Gamers as 'Robot Pilots'", The AI Insider, 2026 ↩
-
Source: "Asimov", Y Combinator company page, 2026; tryasimov.ai ↩
-
Source: "Asimov company profile", Tracxn, 2026 ↩
-
Source: "Human Archive", Y Combinator company page, 2026 ↩
-
Source: "Human Archive taps into India's services startups to collect data for physical AI", TechCrunch, 2026 ↩
-
Source: "Inside Human Archive, the startup recording Indian workers to train the world's robots", inc42, 2026 ↩
-
Source: "Hub: real-world training data for frontier AI labs and robotics", Y Combinator launch, 2026 ↩
-
Source: "Hub.xyz Closes Major Funding Round with SwissBorg to Scale The First Distributed AI Data Infrastructure Network", Business Wire, 2025 ↩
-
Source: "6thSense", Y Combinator company page, 2026; 6thsense.dev ↩
-
Source: "EgoDex: Learning Dexterous Manipulation from Large-Scale Egocentric Video", Apple ML Research, 2025 ↩
-
Source: "6thSense pre-seed coverage", wowtale (Korean press), 2026; echoed in "Robotics raises", automate.org, 2026 ↩
-
Source: "Mecka AI raises $60 million Series A for robotics training data", Fortune, 2026 ↩
-
Source: "Robotics Center of Silicon Valley", DeepAware AI: About, data services, marketplace; "DeepAware AI", Y Combinator, 2025 ↩
-
Source: "Foxglove Raises $40 Million Series B to Power the Future of Physical AI", Business Wire, 2025 ↩ ↩2
-
Source: "MCAP as the ROS 2 default bag format", Foxglove blog, 2023 ↩
-
Source: "Foxglove raises $16M to build dev infrastructure for robots", TechCrunch, 2022 ↩
-
Source: "Foxglove pricing", foxglove.dev, retrieved July 2026 ↩
-
Source: "Foxglove Raises $40 Million Series B to Power the Future of Physical AI", Business Wire, 2025; "Foxglove raises $40M to scale data platform for roboticists", The Robot Report, 2025 ↩ ↩2
-
Source: "Ember Robotics: Datadog for robots", Y Combinator Launches, 2024; company site at emberrobotics.com ↩
-
Source: "Shivani and Ritika", Women in Automotive Technology podcast, 2025; "Smart Real-Time Robot Diagnostics", Samson Rose podcast ↩
-
Source: "Instance", Y Combinator company directory, 2026; product at instancelabs.ai ↩ ↩2
-
Source: "Instance verification demo", Instance Labs, 2026 (benchmarks are self-reported) ↩
-
Source: "Lightberry: robots with soul", Y Combinator Launches, 2025; company site at lightberry.com ↩
-
Source: "Featured investment: Lightberry", Singularity Capital, 2026; Paulina Szyzdek (HardTech VC) note via Substack ↩
-
Source: "Tesla Optimus bots were controlled by humans during the 'We, Robot' event", TechCrunch, 2024; "Tesla Optimus robot takes suspicious tumble in new demo", Electrek, 2025 ↩
-
Source: "Fleet response: lending a helpful hand to Waymo's autonomously driven vehicles", Waymo blog, 2024 ↩
-
Source: "Waymo overseas human assist wasn't secret, but is it secure?", Forbes (Brad Templeton), 2026 ↩
-
Source: "Sub-40ms latency: how Adamo does it", Adamo blog, 2026 ↩
-
Source: "Embodied AI data collection: teleoperation and sim-to-real in 2026", EVS International, 2026; "Robot training data strategy", Shaip, 2026 ↩
-
Source: "Robotic teleoperation data startup XDOF launches with $70M in funding", SiliconANGLE, 2026; "Collecting robot training data is dirty, unglamorous work", TechCrunch, 2026 ↩
-
Source: "1X NEO is a $20,000 home robot that will learn chores via teleoperation", Engadget, 2025; "Teleop, not autonomy: the path for 1X NEO humanoid", The Robot Report, 2025 ↩
-
Source: "Human-level hands: 1X just gave humanoid robot NEO something close", Forbes, 2026 ↩
-
Source: "Your future robot companion may come with a human stranger in control", BGR, 2026 (carries the WSJ Bernt Bornich quote) ↩
-
Physical Intelligence, "π0.5: a Vision-Language-Action Model with Open-World Generalization" (2025) - pi.website; action-space documentation in the openpi repository - github.com ↩
-
Source: "What Is Hand-Eye Calibration?", MathWorks Computer Vision Toolbox documentation; "Camera Calibration and 3D Reconstruction", OpenCV 4.x calib3d documentation ↩
-
Source: "SO-ARM100/101 Robotic Arm Calibration and Remote Control", Waveshare wiki ↩
-
Source: "Gimbal Lock", Compu-Tools blog, 2026 ↩
-
Kevin M. Lynch and Frank C. Park, "Modern Robotics: Mechanics, Planning, and Control" (2017) - free PDF at hades.mech.northwestern.edu, book site at modernrobotics.org ↩
-
Source: "Quaternions" interactive visual explainer, saeed1262.github.io, 2024 ↩
-
ROS 2 tf2 tutorials, "Quaternion fundamentals" - docs.ros.org ↩
-
Ben Eater and Grant Sanderson, "Visualizing quaternions" interactive explorables - eater.net; 3Blue1Brown, "Quaternions and 3d rotation" - 3blue1brown.com ↩
-
ROS REP 103, "Standard Units of Measure and Coordinate Conventions" - ros.org; ROS REP 105, "Coordinate Frames for Mobile Platforms" - ros.org ↩
-
Wikipedia, "Product of exponentials formula" - en.wikipedia.org ↩
-
Source: "Forward Kinematics: Product of Exponentials", Robotics Unveiled ↩
-
Articulated Robotics, "Making a Mobile Robot" URDF tutorial - articulatedrobotics.xyz ↩
-
Universal Robots, "DH parameters for calculations of kinematics and dynamics" - universal-robots.com ↩
-
ros-industrial/universal_robot, issue #600 on nominal vs. per-unit calibrated kinematics - github.com ↩
-
Hugging Face LeRobot documentation, "SO-101" - huggingface.co/docs/lerobot/so101; LeRobot paper (ICLR 2026) - arXiv 2602.22818 ↩ ↩2 ↩3
-
Source: "Assignment 6: SO-101 Kinematics", Maegan Tucker, ECE 4560 (Georgia Tech) - a worked version of this exercise on the SO-101 itself ↩
-
Source: "Universal Robot Kinematics", Robert L. Williams II, Ohio University; "UR5 Inverse Kinematics", alexanderelias.com ↩
-
Victor-Louis De Gusseme, "ur-analytic-ik" - github.com ↩
-
Universal Robots, "What is a singularity?" - universal-robots.com ↩
-
Source: "Singularity prevention", Universal Robots e-Series user manual (SW 10.7) ↩
-
Patrick Beeson and Barrett Ames, "TRAC-IK: An open-source library for improved solving of generic inverse kinematics" (2015) - researchgate.net ↩
-
Stéphane Caron et al., "Pink" (differential inverse kinematics on Pinocchio) - github.com ↩
-
Balakumar Sundaralingam et al. (NVIDIA), "cuRobo: Parallelized collision-free minimum-jerk robot motion generation" (2023) - report at curobo.org, code at github.com ↩
-
Source: "lerobot-kinematics", Box2AI Robotics, GitHub - a community FK/IK library for the SO-10x arms ↩
-
Source: "Sim-to-Real with the SO-101: LeRobot", NVIDIA Physical AI learning path; "Isaac Teleop: LeRobot devices", NVIDIA IsaacTeleop documentation ↩
-
SDU Robotics, "ur_rtde" API documentation - sdurobotics.gitlab.io ↩
-
Modern Robotics, Chapter 5, "Velocity Kinematics and Statics" - modernrobotics.northwestern.edu ↩
-
Universal Robots, "UR10e e-Series datasheet" - universal-robots.com ↩
-
Source: "STS3215 register documentation and control code", GitHub ↩
-
Universal Robots, "Force mode - performing motion using the force mode, compliant in a specific feature and direction" - universal-robots.com ↩
-
Source: "Force mode UR10e", Universal Robots practitioner forum ↩
-
Hugging Face blog, "Pi0 (π0 and π0-FAST) vision-language-action models" - huggingface.co ↩
-
Maximilian Igl, "Action chunking" explainer - maximilianigl.com ↩
-
Hugging Face blog, "LeRobot v0.5.0 release" (Real-Time Chunking) - huggingface.co ↩
-
SDU Robotics, "ur_rtde examples" - sdurobotics.gitlab.io ↩
-
Universal Robots, "URScript manual" (SW 5.24) - universal-robots.com ↩
-
Universal Robots, "Trajectory improvements for servoj" - universal-robots.com ↩
-
Kevin Black, Manuel Y. Galliker, Sergey Levine: "Real-Time Execution of Action Chunking Flow Policies", Physical Intelligence, June 2025 ↩
-
Source: "Miru: Config Management for Robotics", Y Combinator company directory; "Miru: deployment infrastructure for robotics and IoT", YC Launches, 2024 ↩ ↩2
-
Source: "Miru documentation", Miru, 2026; "Robotics config management is a mess", Miru blog ↩
-
Source: "2023 Most Disruptive business school startups: Clips AI, University of Texas (McCombs)", Poets&Quants, 2023; founder roles verified via YC, LinkedIn and "FounderTrace" ↩
-
Source: "Aktoria homepage", fetched via text proxy, July 2026; negative checks: the YC company page for Aktoria returns HTTP 404, it is absent from the "YC Fall-2025 batch list" (Extruct) and from the "2026 VR-teleoperation vendor comparison", Robotics Center, 2026 ↩
-
Source: "BMW to Use Figure AI's Figure 03 for Logistics Work at US Factory", The AI Insider, 2026 ↩
-
Source: "Apptronik raises $520 million at $5 billion valuation for Apollo robot", CNBC, 2026 ↩
-
Source: "This humanoid robotics company is going public, but its CEO isn't promising a robot in your home anytime soon", TechCrunch, 2026 ↩
-
Source: "Humanoid Robots in 2026: Figure, Apptronik, 1X and Tesla Optimus Compared", Value Add VC, 2026 ↩ ↩2
-
Source: "PNDbotics humanoid robot wiki", PNDbotics; Adam SP specs via "Adam SP", Humanoid Guide, 2026 ↩
-
Source: "Poland mining ATEX requirements", Danem, 2026; "EU Machinery Regulation 2023/1230", Danem, 2026 ↩
-
Source: "Hugging Face releases a 3D-printed robotic arm starting at $100", TechCrunch, 2025 ↩
-
Source: "TheRobotStudio/SO-ARM100" (official bill of materials and CAD), GitHub ↩
-
Source: "LeRobot Worldwide Hackathon", Hugging Face, 2025 ↩
-
Source: "Anvil Robotics launch thread", @0x796F on X, 2026 ↩
-
Source: "Physical AI custom robot builder Anvil raises seed funding", Crunchbase News, 2026; "Anvil Robotics raises $6.5M for composable-modules physical AI platform", The AI Insider, 2026 ↩ ↩2
-
Source: "Anvil shop catalogue", Anvil Robotics, 2026 ↩
-
Source: "Trossen Robotics store" catalogue prices, Trossen Robotics, 2026 ↩
-
Source: "Universal Robots launches Financial Services leasing program in collaboration with DLL", Universal Robots, 2020 (page still live mid-2026; no fresher confirmation under that branding) ↩
-
Source: "Cobot Rental 2026: How to Rent a Collaborative Robot", Sharebot, 2026 ↩
-
Source: "Used Universal Robots", Robots Done Right; eBay UR5 listings, asking (not sale) prices as of August 2026 ↩
-
Source: "Remote Warehouse Robot Lab", The Construct ↩