Memories.ai is paying people to show robots how to do their jobs. The San Francisco startup supplies first-person video data for robot training, with cameras capturing the hands-on details of preparing food, stocking shelves and repairing equipment.
The company is launching Memories Upload, a global gig-work platform where contributors take paid video-recording tasks to help train robots. Its wearable LUCI Pin camera, developed by Memories.ai, weighs under 45 grams and attaches magnetically to mounts worn on a person’s head, chest or wrist. It captures POV video of the contributors doing various tasks. Two cameras can be paired for stereo capture, providing views that help estimate depth.
Memories.ai says its network recorded more than 1 million hours of first-person video in the platform’s first month in beta and that it aims to collect more than 10 million hours by March 2027. The company was founded by former Meta researchers Shawn Shen and Ben Zhou. The company announced an $8 million seed round in July 2025 led by Susa Ventures, with investors including Samsung Next.
Humanoid robotics startup Figure and Physical Intelligence, a startup that builds AI software “brains” that can control a variety of different types of robots and embodied AI devices, are both exploring how robots can learn from POV video footage. Figure’s Index platform pays people to record everyday tasks for its Helix AI models. Physical Intelligence has tested combining human video with teleoperation data, which is recorded as people control robots remotely, demonstrating how to complete tasks or handle unfamiliar objects and environments. Physical Intelligence co-founder Sergey Levine wrote in a blog post that human video and other supplementary data could provide “a source of knowledge about what can happen in the real world,” complementing data collected by robots themselves.
For a robot learning to make coffee, those recordings could show how someone grips a cup, positions it beneath a grinder and moves it without spilling. Footage from different kitchens, tools and ways of working can expose models to a wider variety of tasks and environments. Developers still work out how to translate those observed movements into actions suited to the robot’s own body.
“Recognizing an activity is different from understanding exactly how a hand grips a tool, moves an object or completes a step, especially when the camera is moving and objects are partly hidden,” Rick Zhuang, director of product and engineering for Cosmos at Nvidia, which works with Memories.ai, told Fortune.
Finding those details in a growing collection of footage is part of the problem Memories.ai is trying to solve. The company uses Nvidia’s Metropolis Video Search and Summarization tools to optimize video streaming and processing. Its video-captioning model is fine-tuned from Cosmos, Nvidia’s family of models for understanding the physical world, and it also uses Cosmos-based models to label actions. The company further trains Cosmos on curated video for customers’ specific tasks, a process known as post-training.
The resulting descriptions are tied to specific moments, letting developers search for an action and retrieve relevant examples without watching hours of recordings. “What stands out is the focus on making everyday human experience searchable and useful for robot learning,” Zhuang said of Memories.ai.
The company says its customers include three of the top five frontier AI labs, alongside robotics developers buying its first-person training data, although the company declined to name these customers citing non-disclosure agreements. Co-founder and CEO Shawn Shen told Fortune that customers increasingly want footage covering a wider range of work environments, tasks and camera angles. Two people making coffee in different kitchens can supply useful variation, he said, even when they are completing the same task. Developers also request wrist-level footage to see interactions that a camera on someone’s head might miss, such as exactly when a hand touches an object.
Adam Scraba, Nvidia’s head of physical AI ecosystem for vision agents, told Fortune that the wide variety of specialized tasks in heavy industry, coupled with the need for highly accurate models, presents one of the field’s hardest challenges.
“The training dataset for Vision AI and video analysis becomes critically important for basically all Physical AI solutions. Developers need to find the right corner-case examples, trust their labels and trace each annotation to the moment it describes,” he said.
Producing labels developers can trust is particularly difficult when hands make subtle movements, objects are partly hidden or the boundary between two actions is unclear, Zhuang said. Those cases make human review and evaluation essential. Human video is most useful alongside robot demonstrations and simulation, he added, helping developers connect observations of human work to actions a robot can perform.
Paying human contractors to provide video in this way can raise privacy concerns. Recordings can potentially capture coworkers, passersby and sensitive information. Memories.ai said camera wearers and coworkers who regularly appear must sign consent forms before recording. Before footage leaves the country where it was captured or reaches customers, the company’s teams use automated tools to obscure other people in the frame and mask identifying details such as badges, screens and license plates. It also manually reviews a sample of frames from each batch to check that these sensitive details have been blurred or obscured.
This story was originally featured on Fortune.com
