Claude Code Can Finally See Your Screen (Full Setup) transcript

You're building something in Claude code, but something on the page looks wrong. So, you crop a screenshot, you save it, you switch windows, you paste that in, and then you write a paragraph explaining which button you actually want to fix. You send that message and it changes the wrong thing, and it also breaks something else entirely. You've done that today and you'll do it again tomorrow. Claude code can read your entire code base.

It can reach your email, your slack, your notion, but it can't see the thing you're looking at right now, and it can't be reached from anywhere except a chat box. What if I told you that there's a way to solve both of those problems with one tool? This is exactly what I'm going to show you in this video. We're going to give Claude code eyes and ears across our entire workspace. I'll also cover what the tool is, where it falls apart, and how you can start using it for free.

So, let's get started. Okay, so let's be precise about what Claude code is actually missing. Claude code is not trapped in your folder. If you use its connectors, aka its MCPs, it can reach your inbox, your calendar, your notion, and your database. So, reach isn't the problem here. The gap is much smaller than that and a little more annoying. It has no eyes. It cannot see what's on your screen right now. So, anything that's visual, you have to translate into words, and words are often a terrible format for that box, the third one.

No, the text under that. And it's got no doorbell. You can't reach for it whilst you're on another window, for example. You can summon regular Claude from anywhere, but not the world's most powerful coding agent. Instead, you have to open the Claude desktop app, go to Claude code, be in the right folder, and ask for the thing you want. That problem right here is exactly what I've been looking to solve for months. I get a lot of emails from companies asking me to look at their product.

And honestly, I end up saying no to almost all of them. But, I recently got an email for the tool I have literally been looking for for a while now, and here's why that is. The Claude app on your desktop has voice. You can talk to it, but I barely use the Claude app. I live inside of Claude code, and Claude code has no voice or no screen access, nothing that remotely looks like that. So, I've had the thing I wanted sitting one app away from the thing I actually work in for months.

So, I agreed to take a look at the product, and I have been testing it out for the last 2 weeks now. So, yeah, this video is sponsored by Voice OS. They gave me complete access to the tool for free and told me I could be as honest as I want about it, which is great because you're going to see me cover this tool's best features, but also its downsides. The catch is that this is a paid closed cloud tool. You can get started with the 7-day free trial and decide if you like it from there.

It is $30 a month if you pay monthly or $12 a month if you go yearly. And if you cancel after the 7-day trial, then you don't lose access completely to the tool. You drop down to 100 dictation sessions and 25 agent sessions a week. That's about three agent runs a day, which is enough to keep using it, just not enough to work the way I do in this video. I did include free alternatives at the end of this video in case that's what you want.

However, each have their own limitations, which I will also cover there as well. But, with that out of the way, let's actually install it and test it out together. Installing it is simple. You go to their site, you download it for your operating system, you open it, and you make an account. That is the whole install. Then there are two things to turn on before any of this works. The first is the apps tab. That's where every connector lives, and you turn them on one at a time.

Each one opens an OAuth window, you approve that, and it's connected. You turn on Claude Code, then you click into it, and you connect that. The connector list covers the most common apps like mail, Slack, Notion, Drive, Calendar, but if your workflow lives in something a little niche, or if the app you need is not already built into that list, you may need to add or build an MCP integration, which we will get into how to do that later in this video.

And with Claude Code connected, here's the feature I actually made this video for. Now, this is the part that changed how I work with Claude Code. You hold the hot key like this. see it's activated, and while you're talking, you can circle something on your screen that it will give as context to your prompt. And that circle that we drew, as you can see, just became context, and just became the thing that it's pointing to visually inside of your prompt.

So, instead of saying something like, "Change the color from red to the one below that text where it says you can," what you can literally do is say, "Hey, Claude, could you change this red text to yellow for me?" >> Changing the red text to yellow. >> And that's it. Just like that, it sent it its request, it opened a Claude instance, a Claude Code instance, and it's currently doing it. This is a big improvement from taking a screenshot, opening the desktop app or the terminal, pasting the screenshot inside, and describing the specific thing you want to change.

Here's what I absolutely love about this feature. The hot key works everywhere. You're on on website, and you notice a bug, for example, over here it's not showing the thumbnail for the next articles coming up. I can literally just hold the hot keys, circle around the problem, and give it my prompt. Hey Claude, could we add thumbnails next to each title here on my website? >> We're working on that. >> There you go. It just sent its request, and it doesn't just fire and forget.

There's a panel on the left where everyone lives. There's another panel at the top where you see your current run. You can see which model it's on. I can click into it. I can see which model it's on, which intensity it has. I can modify that. I can ask it to stop if I don't want it, or I don't know if it misunderstood me. I can look at the prompt and reread it. And you can even hold the hot key mid-run to ask it to change something.

Actually, never mind. I wanted to design a specific thumbnail instead for each video, just like that. So, Claude codes stops being a place you go and sit, and it becomes something you throw work at while you carry on doing something else entirely. And once you've got that going, you can then go one level further and build integrations for any tool on your desktop. Let me show you how. Another thing I love about this thing is that if you can't find an integration for a tool that you use, you can just create one.

For example, I love Hermes Agent, but there isn't an integration for it there. So, I went to custom integrations, and I built one myself. I clicked on create, and then I just explain what it is I wanted to do. Hey, I want to create a Hermes integration. And then I had a little back and forth with it until this was done right here. This means you can build integrations for this tool to be able to reach any tool with a usable API or MCP path so that you can use that tool with Voice OS from anywhere on your desktop.

Now, I can just run Hermes and ask it something like, "Uh Hermes, what tasks did you complete today?" >> One sec, checking task. I have sent the question to Hermes agent, and the summary will appear in your panel as soon as it finishes. >> There you go. And it will just answer me from anywhere. I don't need to go to Telegram or the Hermes desktop app anymore when I'm busy doing something else. >> Hermes completed email audits and partner buys, updated Notion metrics, resourced X posts, checked YouTube competitors, and verified all 10 jobs while demo assets remain outstanding. >> There you go.

It's that simple. Building custom integrations for more complicated apps isn't one shot, though. For example, I built a calculator integration, and that was literally as simple as asking it to do that. But creating a custom Hermes integration took a bit of back and forth given my Hermes instance runs on a separate device, and I want to be able to control it from an entirely different device instead. By the way, I will include the custom Hermes integration I built for free in the description along with everything else I cover in this video.

And with that, let's talk about the places where this tool can fall a little short. Every tool has a breaking point, and after I tested this tool for the past 2 weeks, here are mine. The first and the biggest one, in my opinion, is that it's cloud-only. There's no local inference option. If you've built your whole setup around running things locally, this might bother you as well. The second is the connector list. It's good on the obvious apps, but it's thin outside of these.

The good thing is I literally only needed one more integration, which is Hermes agent, and then I was able to just build a custom one for myself in under an hour. The third is that it can sometimes mishear technical words like I would say Hermes and it would hear her mess, but luckily there's an easy fix for that. You just go to the app, go to dictation, and add all the keywords you typically use so that it familiarizes itself with them and hears those more accurately.

And this brings us to the question I'd be asking if I were watching this video. What are the free alternatives? There are free ways to get some of this and I thought I'd do some research and dig into each one. I do have to warn you though, every one of these alternatives has their own limitations which I'll be covering as well. The first one is ChatGPT's desktop app which can also use voice mode for CodeX. If you already use ChatGPT or CodeX for your coding, you can enable the voice chat hotkey.

It can even see your screen while you're talking to it. The only downside of that is you can't point at specifics on your screen using that and it doesn't really solve the problem if Claude Code is the coding agent that you use. The second one is already sitting inside Claude Code. You just click on the microphone button and say what you want to say, but you have to already be inside of Claude Code. You can't activate it when you're on another window and you can't show it your screen at all unless you manually screenshot and paste that into the chat window.

The third is Voice Mode. It's an MCP server that gives Claude Code voice conversation. It is MIT licensed and it can run speech-to-text locally so your audio never leaves your machine, but it is voice only. There's no screen watching or screen sharing and getting it set up takes a little bit more work. So, here is the honest comparison. The free options are strong on voice. What none of them can do though is hand that specific piece straight from your screen onto a Claude Code run that's already working inside your project.

So, after using this tool for the last two weeks, here's what I think about it. I have noticed myself reaching out for it a lot more frequently. I mean, it's just so convenient, and I understand the price is not going to be convenient for everyone watching this video, but at my scale, with the business that I'm running, it's a drop in the ocean compared to what it lets me actually do and how convenient it makes working for me.

I've even started using it beyond Cloth code and Hermes. I've noticed myself reach for it and ask it to send me emails whilst I'm editing a video or set me a task on Notion whilst I'm on a meeting. I'm even using the dictation mode whilst I'm on random tabs, because it almost feels like a real-life Jarvis to me. So, if you're in Cloth code every day and a lot of what you do is visual, then take the free trial and try it out yourself.

Tell me in the comments, by the way, what's the one thing you wish your setup could do that it currently can't? Maybe I'll turn that into a video next. Also, as I mentioned, the custom Hermes integration and everything else that I covered in this video are free inside the description below. And if you enjoyed this video, then leave a like, or if you're new to my channel, then subscribe, because I have got a ton more content like this coming your way.

Oh, and would you look at that? The algorithm gods have decided you're very likely to enjoy this next video, as well. So, click it, and maybe I'll see you there.