CosmOS: An operating system for the AI era.

As the team works on shipping updates quickly and transparently, we want to dig a little deeper into some of the key components of Ai Pin and CosmOS over the coming weeks. Through blogs, videos and other content, the aim is to give you greater context on what we’re building and what the future has in store. First up: a small overview on our operating system, CosmOS.

From the start, our north star was to create the world's simplest, most intuitive computer—one that harnesses the power of AI to understand your needs and proactively assist you in your daily life. We believe that AI shouldn't be just another feature but rather the very foundation of modern computing, and that means building an all-new operating system to begin to deliver on its promise.

By integrating AI at the deepest levels, CosmOS enables:

Multi-stage reasoning: CosmOS goes beyond simple queries to understand the context and intent behind your requests. Web search and current assistants are excellent at doing the exact thing you want, but often fail to use that information to provide helpful next steps, or to simply do multiple things at the same time.

Contextual understanding: Who are you? What are you doing? What have you done? By learning from your interactions and history, CosmOS builds a deep understanding of your preferences and needs. This is essential to enable personalized, context-aware assistance.

Focused, streamlined interactions: Current mobile computing is driven by the attention economy. CosmOS cuts through the noise and distractions of the digital world, surfacing the information and actions you need right when you need them. The aim is for less distractions and less friction. 

What makes it an operating system?

Our system spans from device internals to LLMs to any API in between. The goal is to help you with arbitrarily complex requests so you can get back to the business of living. One silly but illustrative example could be: “Look at my battery level, add it to the number of countries in South America, and set a timer for that number of minutes." Fulfilling this request requires you to:

• Get the battery level using the device.
• Get the number of countries in South America from the web.
• Add the numbers.
• Use the device to set a timer.

TAO: The framework powering CosmOS

Under the hood, CosmOS is powered by the latest breakthroughs in large language models (LLMs) and AI architectures, seamlessly integrated to deliver a fluid, intuitive experience. At the core of CosmOS is the TAO framework: Thought, Action, Observation. TAO enables many backend resources on the Ai Bus to connect for use on Ai Pin.

This advanced yet simple architecture enables our AI to navigate complex tasks and deliver on your intent. Here's how it works:

• The TAO loop begins with an input event, such as a voice request to Ai Pin through Ai Mic. Ai Mic is built as a service directly in the on-device OS as part of our custom fork of AOSP (Android). The microphone stream is fed directly to on-device transcription, and forwarded to the supervisor over a secure channel.
• The supervisor agent collects all necessary context (such as the current conversation and device signals), and decomposes the request into the first set of tasks to be delegated to one or more specialized agents. Each agent is focused on a particular domain or service.
• The agents complete their assigned actions and return observations to the supervisor.
• The supervisor assesses the observations to determine if the original request has been satisfied or if further actions are necessary.
• Once all tasks are complete and the request is satisfied, the result is communicated back to you via voice or the Laser Ink display.

Let's walk through an example. Suppose you ask CosmOS: "When will the restaurant my friend just mentioned open?" TAO would execute a multi-stage plan: 

• Search your recent messages to identify the restaurant. 
• Look up the restaurant's hours online. 
• Return the opening time to the user. 

Specialized agents would handle each step, with the supervisor orchestrating the process to quickly deliver the information you asked for. This TAO architecture allows CosmOS to improve at the pace of all LLMs and enables remarkable capabilities.

Let’s go through a few of them:

Multi-step reasoning and action: CosmOS can break down complex queries, pursue multiple lines of reasoning, and take actions across various services to deliver on your intent.

Parallel processing: Multiple requests can be handled simultaneously, with the supervisor delegating and coordinating tasks across agents.

Intelligent error handling: If an agent encounters an issue or service outage, the supervisor can handle it by retrying or seeking alternative resolutions.

Continuous personalization: Every interaction helps CosmOS build a richer understanding of your needs and preferences for more targeted assistance over time.

And this is just the start. We're continually evolving the TAO framework to support even more sophisticated communication between you and our AI. The goal is to increasingly break down barriers between you and the appropriate AI services, enabling you to fluidly accomplish your tasks without friction.

Privacy first, trust always

Maintaining your trust is our highest priority. What is key with CosmOS, is that you are always in control of your data. We will never sell your information, use it for advertising, or feed it into our AI models without your explicit permission. Your privacy and security are protected in a number of ways:

Built on a core foundation layer:  we leverage the state-of-the-art security functionality provided by AOSP such as SELinux, file-based encryption, and software integrity checks.

Per-user encryption: Your personal data is always encrypted with unique keys that you own. This ensures that your data remains secure as it moves through and is stored on our cloud servers. Your Ai Pin is the ultimate backup of these keys, enabling you to decide what, when, and how cloud services can access your data. This is the foundation of our industry-leading privacy goal, giving you full control and transparency over your data.

Hardware-enforced transparency: Ai Pin features a dedicated privacy chip that visually indicates when sensors are active, so you always know when it is observing or in action.

No "always on": There are no wake words or continuous recording. Sensors only activate when you intentionally engage the device.

Tamper-resistant design: The Trust Light is controlled by a dedicated processor that detects if the Trust Light electronics have been tampered with, and if this happens, it will disable the optical and audio sensors.

We are committed to being fully transparent about our data collection and use and you will always have the ability to access, manage and delete your information as you see fit.

Building the future together

To us, CosmOS provides the foundation for which AI-powered personal computing can be built upon—for Ai Pin and for other future devices, too. As we expand with our SDK, we’re eager to work closely with developers, partners, and users like you to expand the capabilities and reach of this OS. And as new AI breakthroughs emerge, CosmOS is architected to swiftly integrate them, ensuring you have access to the most advanced capabilities. 

With CosmOS, we're bringing to life a long-held dream of computing: a truly personalized, intelligent assistant that understands your needs and proactively supports you in accomplishing your goals. It's a vision of technology that adapts to people, rather than asking people to adapt to it. We are at the foothills of what is possible with AI experiences and we’re excited to continue the climb.