LabOS: An integrated AI-driven laboratory platform for automating, guiding, and optimizing experimental research and analysis

Background

Modern laboratory research, spanning fields such as biology, chemistry, and biomedical engineering, increasingly relies on both computational analysis and complex physical experimentation. As scientific protocols become more sophisticated and interdisciplinary, there is a growing need for seamless integration between digital planning (dry lab) and hands-on execution (wet lab). This integration is essential for improving reproducibility, accelerating discovery, and facilitating knowledge transfer among researchers of varying expertise. However, the traditional separation between computational and physical workflows often leads to inefficiencies, fragmented documentation, and challenges in scaling best practices across different laboratories and teams.

Current approaches to laboratory management and automation are hampered by several critical limitations. Most labs rely on a patchwork of disconnected software tools for data analysis, experiment planning, and protocol documentation, which rarely interface directly with the physical execution of experiments. Error detection and correction are typically manual, leading to increased risk of mistakes and irreproducible results. Furthermore, knowledge transfer is hindered by incomplete or inconsistent records, making it difficult for new researchers to learn complex procedures or for teams to replicate findings across sites. Existing systems also lack robust support for real-time feedback, hands-free guidance, and high-fidelity digital representations of the lab environment, all of which are crucial for ensuring safety, efficiency, and collaborative research in modern scientific settings.

Technology

LabOS is an integrated laboratory intelligence platform that unifies computational (dry lab) and physical (wet lab) research through advanced AI and extended reality (XR) technologies. Its dry lab component is powered by STELLA, a self-evolving agentic AI framework composed of Manager, Developer, Critic, and Tool Creation agents. This framework autonomously plans research goals, generates and executes bioinformatics code, refines workflows, and expands a dynamic repository of computational tools—known as the “Tool Ocean”—while validated workflows are stored for future use. The wet lab side employs STELLA-V, a domain-adapted visual language model that processes real-time, multi-modal sensor data from XR glasses, offering stepwise procedural guidance, error detection, and comprehensive logging for reproducibility. High-fidelity 3D/4D digital twins of the laboratory, constructed from multi-camera video using Gaussian splatting, enable object tracking, semantic querying, and continuous system improvement through feedback loops.

What differentiates LabOS is its seamless, closed-loop integration of AI-driven reasoning, real-time execution support, and spatial intelligence within a single, modular platform. Unlike fragmented traditional systems, LabOS bridges the gap between computational planning and hands-on experimentation, enabling autonomous workflow refinement, error correction, and rapid skill transfer. Its agentic AI outperforms leading language models in biomedical reasoning and tool composition, while the domain-specialized VLM achieves superior accuracy in protocol guidance and error detection. The system’s ability to generate and leverage digital twins for object-centric tracking and simulation-based training, combined with robust safety, privacy, and multi-user collaboration features, positions LabOS as a transformative solution for accelerating discovery, enhancing reproducibility, and democratizing advanced laboratory capabilities across diverse research environments.

Benefits

• Enhances reproducibility and efficiency by integrating computational planning with physical lab execution in a unified platform.
• Employs a self-evolving agentic AI framework (STELLA) that autonomously plans, executes, refines, and expands bioinformatics workflows and tools.
• Provides real-time, context-aware guidance and error detection in wet labs through domain-adapted vision-language models (STELLA-V) and XR glasses.
• Generates high-fidelity 3D/4D digital twins for object tracking, semantic querying, safety monitoring, and simulation-based training.
• Supports automation and semi-automatic robotic execution of lab actions using reusable motion primitives.
• Enables multi-user, multi-site collaboration with real-time teamwork, supervision, and remote access capabilities.
• Ensures data privacy and compliance with biosafety through local-first computing, audit logs, and modular integration with lab systems.
• Validated in real-world applications, accelerating discovery, improving skill transfer, and enhancing experimental outcomes.

Applications

• Autonomous biomedical research workflows
• Real-time XR-guided lab training
• Automated protocol error detection
• Remote multi-user lab collaboration
• 3D/4D digital twin lab simulation

Additional info

This integrated AI platform unifies computational and physical laboratory operations. It employs an agentic AI for research planning, code generation, and expanding computational resources. A visual language model, utilizing XR glasses, provides real-time guidance, error detection, and data logging in the physical lab. High-fidelity digital twins enable object tracking and semantic querying, with continuous feedback loops enhancing overall capabilities.