Artificial intelligence companies are turning to video and motion data as they develop systems designed to operate in the physical world.
• Text helped train today’s chatbots; recorded actions could shape tomorrow’s robots.
• Robotics researchers are collecting footage of people and machines performing tasks such as grasping objects, navigating spaces and using tools.
• The data may help robots learn through observation rather than relying solely on manually programmed instructions.
• A growing market is emerging around specialized datasets that capture movement, environments and real-world interactions.
• The approach also raises questions about privacy, copyright and whether online videos can be used to train commercial systems.
The push reflects a broader shift in AI development. While language models learn from words, robotic systems must understand timing, force, balance and physical consequences. Building those capabilities requires vast amounts of carefully labeled video and motion data—resources that remain limited compared with the text available online. Labs are now racing to close that gap, betting that better training data will be as important as more powerful algorithms in bringing useful robots into homes, workplaces and public spaces.