VisionClaw is an open-source project that turns Meta Ray-Ban smart glasses into a real-time AI agent. It streams what you see and hear to Google’s Gemini Live API, which can reason about your surroundings and take actions on your behalf through 56 tool integrations. The project hit 1,800 GitHub stars within weeks of its February 2026 launch, making it the first popular open-source attempt at giving AI agents a wearable form factor with actual eyes.
The concept is straightforward: instead of pulling out your phone to ask an AI something, you just speak. The glasses capture what you’re looking at, Gemini processes the visual and audio context together, and if you ask it to do something (send a message, search the web, control your smart home), it routes that action through OpenClaw, the open-source tool integration layer that Nvidia compared to what GPT was to chatbots.
How VisionClaw Works: Four Layers From Lens to Action
The architecture breaks down into four distinct layers, each handling a different part of the pipeline from raw sensor input to executed action.
Layer 1: Hardware Input
The Meta Ray-Ban glasses stream video at roughly 1 frame per second as JPEG images through Meta’s Device Access Toolkit (DAT) SDK. Audio is bidirectional: your voice goes in at 16kHz PCM mono in 100ms chunks, and Gemini’s spoken responses come back through the glasses’ built-in speakers at 24kHz. The glasses need Developer Mode enabled through the Meta AI companion app, though VisionClaw also supports an iPhone camera fallback for testing without the hardware.
One frame per second sounds slow, and it is. You will not track a tennis ball or read fast-scrolling text. But for the use cases VisionClaw targets (identifying objects, reading labels, scanning environments), 1fps turns out to be enough.
Layer 2: The Gemini Brain
The core connection runs over WebSocket to Google’s Gemini Live API using the gemini-2.5-flash-native-audio-preview model. This matters because it processes audio natively rather than converting speech to text first, which means lower latency, better understanding of tone and emphasis, and more natural conversation. A rolling 20-message context window keeps the conversation coherent across multiple exchanges.
Layer 3: OpenClaw Agentic Bridge
When Gemini recognizes that you want something done (not just answered), it issues an execute function call. VisionClaw’s OpenClawBridge intercepts this and routes it via POST request to a locally running OpenClaw instance. OpenClaw currently supports 56+ tool integrations: WhatsApp, Telegram, Signal, iMessage, email, smart home control, web search, calendar, shopping lists, and dozens more.
This is where VisionClaw stops being a voice assistant and starts being an agent. A voice assistant answers questions. VisionClaw can look at your calendar, see that your next meeting is in 15 minutes, notice from the camera that you’re still at a coffee shop, and proactively message your colleague that you’ll be late.
Layer 4: Live Streaming via WebRTC
An underrated feature: WebRTC enables real-time POV sharing through 6-character room codes. A remote colleague can see exactly what you see, capped at 2.5 Mbps and 24fps. Think remote support, field inspections, or collaborative troubleshooting where “Can you see what I’m looking at?” has a literal answer.
What You Can Actually Do With VisionClaw Today
The marketing around wearable AI tends toward the aspirational. Here is what actually works right now, based on documented use cases and user reports.
Hands-Free Communication
Say “send a message to Sarah saying I’ll be 10 minutes late” while walking, and VisionClaw routes it through OpenClaw to WhatsApp, Telegram, or iMessage. No phone, no typing, no breaking stride. This is the most immediately practical use case and the one that works most reliably.
Visual Product Search
Point your glasses at a product in a store and ask “How much is this on Amazon?” Gemini processes the visual input, identifies the product, and OpenClaw runs the search. The results come back as spoken audio through the glasses. Early users on Reddit report this works reasonably well for branded products with clear labels, less so for generic items.
Scene Understanding and Navigation
Ask “What building is that?” or “Can you read that menu for me?” and Gemini describes what the camera captures. This is genuinely useful for accessibility scenarios. Sean Liu, VisionClaw’s creator and an NYU Mixed Reality researcher at Intent Lab, also built GlassFlow, a companion project focused on real-time transcription for hearing-impaired users.
Smart Home Voice Control
“Turn off the living room lights” works through OpenClaw’s smart home integrations while you’re moving around the house. The contextual advantage over a phone assistant: you can reference what you see. “Turn on the light in this room” becomes meaningful when the system knows which room you’re in.
The Wearable AI Agent Landscape in 2026
VisionClaw is not operating in a vacuum. The wearable AI agent space went from niche to contested in early 2026.
Open-source competitors:
| Project | Hardware | Key Differentiator |
|---|---|---|
| Clawglasses | Custom hardware ($99-$599) | Purpose-built, 12h battery, 70K+ units sold |
| Brilliant Labs Halo | Custom AI glasses ($349) | Privacy-first, on-device processing, OLED display |
| OpenGlass | DIY kit (~$25) | ESP32-based, ultra-cheap |
| Omi | Wearable pendant | Audio-focused, plugin ecosystem |
Corporate players entering the race:
Samsung announced AI smart glasses with agentic AI features launching in 2026. Google committed to a 2026 debut of AI-powered smart glasses through partnerships with Samsung, Gentle Monster, and Warby Parker. Apple is ramping up work on glasses, a pendant, and camera-equipped AirPods.
What separates VisionClaw from these corporate efforts is the same thing that separates Linux from macOS: it is open, composable, and hackable. You choose which LLM brain to use, which tools to connect, and what data leaves your device.
Why the Glasses Form Factor Changes the Agent Equation
Every AI agent built so far operates in a digital environment: browser agents see web pages, coding agents see code, workflow agents see API responses. VisionClaw represents something structurally different. It gives agents access to the physical world in real time, through a device that looks like a normal pair of sunglasses.
This matters for three reasons.
Context without friction. Pulling out your phone to photograph something, open an app, and type a question takes 15-30 seconds. Speaking a question while looking at the thing takes 2 seconds. The cognitive overhead drops to near zero, which changes how often people interact with AI. Early VisionClaw users report using it 20-40 times per day, compared to a handful of phone-based AI queries.
Ambient awareness. A phone-based agent only knows what you tell it. A glasses-based agent continuously sees your environment. This enables proactive behavior: noticing a product recall on a shelf, flagging that a parking meter is about to expire, or recognizing a colleague approaching before a meeting. None of these work without persistent visual context.
Hands-free operation. For field workers, surgeons, warehouse operators, or anyone whose hands are occupied, a glasses-based agent is not a luxury. It is the only form factor that works. The smart glasses market is projected to exceed $30 billion by 2030, and enterprise use cases are the primary driver.
Setting Up VisionClaw: What You Need
If you want to try VisionClaw yourself, here are the requirements:
- Meta Ray-Ban smart glasses (Gen 2, $299+) with Developer Mode enabled. Or skip the glasses and test with your iPhone camera.
- A free Gemini API key from Google AI Studio.
- Xcode 15.0+ (iOS) or Android Studio Ladybug+ (Android 14+ / API 34+).
- OpenClaw running locally on the same Wi-Fi network (optional, for tool integrations beyond conversation).
The iOS app is more mature than Android. Battery life on the glasses runs 3-4 hours with continuous streaming, which is the biggest practical limitation. Meta’s DAT SDK is still evolving, and breaking API changes between versions are common.
Security note: OpenClaw requires API keys, passwords, and personal information for its integrations. Third-party skills can be written by anyone. If you connect your email, messaging, and smart home through OpenClaw, you are trusting that codebase with significant access. Review what you connect carefully.
Frequently Asked Questions
What is VisionClaw and how does it work?
VisionClaw is an open-source AI agent that connects Meta Ray-Ban smart glasses to Google’s Gemini Live API. It streams video at 1fps and bidirectional audio from the glasses to Gemini, which can see what you see, hear what you say, and take actions through 56+ tool integrations via OpenClaw. It runs as an iOS or Android app.
Do I need Meta Ray-Ban glasses to use VisionClaw?
No. VisionClaw includes an iPhone camera fallback mode that lets you test the AI agent functionality without owning Meta Ray-Ban glasses. You will miss the hands-free wearable experience, but the voice and vision capabilities work through the phone camera.
Is VisionClaw a jailbreak for Meta Ray-Ban glasses?
No. VisionClaw uses Meta’s official Device Access Toolkit (DAT) SDK, not a jailbreak or hack. It does require enabling Developer Mode on the glasses, which bypasses Meta’s default AI experience in favor of Gemini and OpenClaw, but this uses officially supported APIs.
What can VisionClaw actually do in practice?
Documented use cases include hands-free messaging (WhatsApp, Telegram, iMessage), visual product search, scene description, smart home control, real-time language translation, calendar management, and remote collaboration through WebRTC POV sharing. Communication and visual search are the most reliable features currently.
What are the alternatives to VisionClaw for AI smart glasses?
Alternatives include Clawglasses (purpose-built hardware, $99-$599, 70K+ units sold), Brilliant Labs Halo ($349, privacy-first with on-device processing), OpenGlass (DIY $25 kit), and Omi (audio-focused wearable pendant). Samsung, Google, and Apple are all launching competing smart glasses with AI features in 2026.