Introducing WebMCP: Agents in the Browser — RL Nabors transcript

[music] >> And even though I know some people are still coming in, I've also been informed to start directly on time. So much apologies to anyone who was hustling up here from one of the talks downstairs. You should have known this was the best track and hung out here from hour zero, but uh by all means come in, have fun. Hi there. I am Rachel Lee, Rachel Lee Neighbors, Neighbors like Nearest Neighbors, but not spelled that way.

I have worked on the standards that powers today's web today's web with Mozilla on Firefox DevTools, the W3C on web standards like the Web Animations API. I've even been a PM on Microsoft's Edge browser, and you may know me from my pandemic babies on the React team, react.dev and reactnative.dev. All right, so I've spent the last 3 years consulting with AI startups and LLM / big browser companies, some of whose logos you may recognize, most of whom you will not, on all things web, AI, and UI.

Recently, I've just become newly minted principal developer experience engineer at a rise. Uh have you ever had your CTO ruin your agentic flow with a slight change of prompt or an LLM migration? Have you ever been that CTO? Anyway, if that is you, you need to check out a rise. Evals are your friends, check them out. Now, before my big tech career, I made web comics for teenage girls on a well, it was a it was a part of the iVillage Network website.

I don't think anyone remembers iVillage, but it was kind of a big deal back in the day. I had 400,000 weekly readers. It was powered by ads, and I learned how to build for the web so I could reach those people where they were. It was 2006, and it was a mood, let me tell you. Everyone online was edgy. We were on MySpace where we had 14-year-olds talking about music with 42-year-olds, and as you can see, I was really into the goth and emo scene.

I was the only goth in my village. >> [clears throat] >> When I got into the web, I believed that the browser is not just a document reader, despite what the CSS wonks will try to get you to believe. It is an infinite canvas that can render anything, not just documents, but video, audio, whatever you need, there's an API for it, and I have been working to that end on the web ever since. I worked on that Web Animation API with the W3C creating such insane demos as the interactive Alice in Wonderland, uh and generally made the web do things that it was not supposed to do.

Uh And now I write about how agents and AI are changing how we browse and consume content on my little newsletter, uh The Agentic Web. And I do a fair bit of move shenanigans with agents myself. This is uh ASA, it stands for Anti-Social Social Agent. It's an MCP app that lets you use Twitter and BlueSky next to each other without actually having to visit the sites or be subjected to other people's algorithms. Anyway, check it out if you like it.

Uh I'm here today to show you how, with the power of the browser's primitives that already exist and have already been specked out right now, you can turn your agent into an infinite canvas, or at least get your website into agents, one way or the other. So, I have this habit. Every year when I'm thinking, "What do I want to make talks about? What do I want to do?" I look at the one thing on my laundry list that I keep putting off.

In this case, my web comics site, the archive site that I put together at like 2010 said, "Okay, you're good there. I got to go now." And it's starting to get 404s, starting to get some broken images. Maybe I shut down some clusters here and there, and it's like, "Oh dear, the CDN has failed." Anyway, it needed some repair, and I was like, "Well, don't just repair it, make it future-proof." If everyone is working in agencies these days, let's meet them where they are.

I'm always meeting my audience where they are, and I still get fan mail from the women who grew up with my comics, so maybe I can get them to try out agents, too. All right, so just as a cringe warning before you go to rachelagreat.com, 2000s were a vibe. Old comics, younger Rachel Lee, just a little bit cringe, okay? I want to honor the fans, and also I'm working a lot with copyrighted images here, and I don't want to get sued by Marvel, so I'm going to use my own stuff.

The nice thing about having a former career as an artist is I can do whatever I want with my art, no questions asked. Of course, I did have to hose them down with glazing to make sure nobody else would, but that's another story. Check out the repo at the end of this if you want to know more about protecting your content from being used for training purposes. Now, I wanted to make my comics available and navigable, one server, but three different kinds of clients, humans in browsers, humans in agents, and agents in browsers using the web on behalf of their humans.

I know, it's complicated. They've got like this polycule thing going on. So, what we're going [clears throat] to cover today. MCP tools and transports, how to host them on the web just like you would a website. MCP apps, how do they work? You've already been told a lot about this. I'm just going to show you a really cool example and point out the foot guns. And Web MCP, coming to a browser near you. Before we go, how many of you currently host an MCP server?

Like on the web, not locally. Okay, well, I love you guys. Everyone else, you're going to learn. And how many of you have made an MCP app? All right, how many of you want to make one? Uh yeah, this is going to be the next web pages. And lastly, has anyone here played with the with Web MCP? Okay, well, we'll see if we can change that. So, you're here. I assume you know what an MCP server is. I'm I'm not going to go into them too much.

Um So, we're going to get started with transports. Now, if you've built for the web, I guess I should ask, how many of you have built websites and hosted them in your lifetimes? All right. So, you're familiar you're probably familiar with HTTP, but you might not be familiar with the concept of transports. It turns out there's more than one protocol you can use to communicate from one surface to another. Transports are how MCP servers communicate with agents.

And there's a important difference between when you should use either of these. So, STDIO stands God, it sounds terrible when you read it out. Standard input output, you can call it >> [laughter] >> studio. Like the Phil Collins song, cuz we're in England, okay? Studio. We're going to call it that for the rest of this talk. Unlike my German proofreader who was like, "I don't think you call it studio." I was like, "Dude, you want me to call it anything else?

I think we're going to have problems." All right, so the server, it runs as a local process, it's spawned by the client. This is why when you're wiring up studio-based MCP app, it's always like a string of JSON with a bunch of command-line inputs that most of your users don't want to use, don't want to configure. Uh communication, it happens via as I just said, standard input and output, and the server stays alive throughout the session.

HTTP is a little different cuz we're running over the web. We are not touching with a 10-ft pole, by the way, any of the security or privacy concerns with either of these layers. That is beyond the scope of today's conversation. But HTTP, the server runs as a web service, it listens at an HTTP endpoint which you may be familiar with from API development, and communication happens via our best friend HTTP post requests, and it works well with serverless setups.

You know, Vercel keeps and and Cloudflare keep trying to sell you those edge functions. Well, they got something to put in them now. All right. >> [clears throat] >> With studio, you have to update a config file with a command to spin up a local server, but with HTTP, your user's expe- Oh my god. Your user experience is as simple as going over to your own settings. Okay, it is buried a little bit, but trust me, when you get Oh my god.

Please, if anyone from Anthropic is watching this, help. Give it a name, and put in a URL, and it is now Come on, hurry. Come on, come on, come on, come on. Past Rachel Lee, but you can see I can actually type the damn thing out. It's very It That's not how you spell it. There you go. It's [clears throat] very There you go, I put it at /mcp. You hit add, and now there it is. It's a connector, and I go ahead and go in, and I I give it all the permissions to always allow because I know this one.

But it's not easy if you're using an HTTP server for your MCP tools for the user to get them into their in this case, Claude. All right, so these are the tools I wanted to make available. They look a lot like what's available when they visit the website. They're very similar to that navigation, and they all return structured data in the form of JSON, except for get transcripts here, and we'll talk a little bit more about this guy in a little bit.

But point is, list the comics, list the storylines because the comics include different story arcs and endings and beginnings. List the characters, search said comics, which is actually searching the transcripts, that's a different talk, and search [clears throat] by character because I love the cat, I want to see all the cat. Uh and the transcript is different, it returns markdown. Boom. All right, let's take a look at these babies in action.

These are just the tools. We haven't gotten to the interactive part, bear with me. I have a narrative arc. All right. It is a bit wordy. Everyone who's like, "Chat's the future." I'm like, "It is not the future." This is like, "Thank you for having a long discussion with me, Claude. Yeah, you did find that I make the comics good, Claude. Good baby. I would like to make a I would like to make the full transcripts and text available to the context for meta-analysis of themes and character arcs, etc. instead of asking Claude to like get transcript from 533 different comics.

It does It is not an efficient use of context to do that. I would really love to use MCP resources, which are the perfect vehicle for this and skills and so many other things, but the spec is loosely defined, it is loosely implemented, and I cannot find a client where I can actively get to these resources. They're on the server. They just ain't visible in your UI. So, please if anyone here is listening who builds an agent harness, go and even bare-bones, I'll take it.

Give me something to work with here. All right. So, I would love for one or more clients to implement resources. Agents really don't like using MCP tools to go add markdown to their context. If you've ever maintained a documentation site and you thought, I will add the MCP tools to the documentation site, that just solves the problems with the coding agent knowing how to No. No, I would rather die in a fire than go call your documentation by MCP tools.

So, let That is not how they were intended to be used by God. >> [clears throat] >> So, resources could also allow harnesses to decide when to pre-prime that context with docs. Like you're switching over to react mode, good. Go get all the react resources from react's MCP server. Thank you. Goodbye. All right, little tirade over. That was your public service announcement. Uh tools, they can return plain data, JSON, text, markdown, but they can also return an interactive app app app app app >> [laughter] >> So, I wanted to put a comic reader in your agent.

So, while you're reading, you can read some things with some pictures instead of a long wall of text. It's like, Claude, you are not my my best friend in crisis. Why are you wall of texting me right now? >> [clears throat] >> It's been said that chat is the lowest common denominator of the user experience. That it is to the future of agentic experiences what the CLI was to software. My mother programmed in COBOL, and when at school they were making us click double-click on icons to access video games, she was like, we do not do that in this household.

We open DOS and we do CD blah blah blah.exe. That is how programmers do things. But we all know that you don't do that on an iPhone, you tap and you point and grunt, right? So, this is probably a phase for us developers. There's a current what I call the chatbox landing page or starfish design because it sits there and it lets the user do all of the work. And you have to have a lot of context in your head when you land.

You have to kind of know what this thing can do. Like if you're on linear, you're like, I know this is a ticketing system. I'm pretty sure I want to see XY or Z. It's great if you're already familiar with the system, but otherwise it can be a bit challenging not to have any visual cues or guidance. It's sort of like these text games were really popular back in the day, but I wouldn't want to have to do my taxes by being like, check mailbox.

Is there mail? Whose house is this? That's not great. It's asking asking your poor customer to do a little bit too much discovery. So, fortunately, we have MCP apps inline in agent rich media experiences created by bundling of HTML, CSS, and JavaScript into a single file. It sounds so familiar. I'm having flashbacks to like DHTML. Huh. So, this is get page. Get page is my MCP tool baby, and I love it very much. This is it in action.

It returns an MCP app. I say, "Yo, I want to read Crow Princess the comic to the starfish um chatbox of everything." And of course, Claude's getting chatting. It's like, "I'll help you read. Oh, let me just check all of my MCP things. Wow, I found it in the other comics collection. Let me go get page that." Here we go. Okay, thank you. And check it out. I can show the commentary. I can show the comments. That's pretty cool.

Those are available on the website. It's everything the website can do. It looks just as good as the website. You can navigate forward and backward. But the fun part is you go up here, you can click on text mode, and boom, it shows you the transcript. So, you could just read it if you wanted to. That'll come back later. So, yeah, it's like a little mini website. Comic reader, but it's using all the resources from the website.

Okay. Now, ha, >> [sighs] >> how do you do that? Well, I define the tool as get page, give it the description. There's an input schema. Of course, I'm going to have to take the comics ID, the storyline ID, the page number, so I can really hone in on where to get this. To be honest, it just needs a storyline ID and the page number. I'm going to refactor that. And of course, all of this stuff is required. But most importantly, down here where the magic happens is you pass this little meta attribute, give it UI, and you point it you point it at a universal resource locator.

Boom, right there. Love it. Okay. Important thing to keep in mind is that MCP apps are a little special. They're not like websites. They're They're an island. They're single HTML files. So, everything that you need there has to be embedded base64. If you're going to link to something external like a cached CDN font file or something, you got to add that to essentially a course policy. So, this means you're not going to be dynamically pulling in stuff from other people's servers unless you really are certain about that and enabling it in your app.

It's sandboxed in an iframe. You don't get local storage with this. So, you're not going to be tracking state across iframes. You're stuck with it. Um async tool calling. Okay. And no network access. Ask You have to ask server to do things for you with the call server tool. This gets into a real mother may I situation, which I'll cover in the gotchas. So, what does this mean? It means that becomes super useful to build your MCP app with your design system.

In this case, I'm sharing fonts, CSS, and everything from the same server. Got the course policy set up and everything. And the HTML and JavaScript components can be bundled. I'm using V single file for this. So, if you already have a design system, put your paw up if you do. Okay, good. All the rest of you, God, I'm sorry for you. But a design system is a thing of beauty. All right, bonus round. Ha. >> [sighs] >> Links require host permission.

Remember I said mother may I? You can't just window open or, you know, href. No, no. No, no. You got to be like, appref current open link pretty pretty please. Okay. Remember I said external resources needed course? Well, they got this thing called the CSP or content security policy. Fonts, images, other external resources, they're blocked by default. So, without that CSP config, the browser would just not load those things.

So, if you're getting a blank look and you're wondering what's jacked up, check your CSP. And unlike MCP UI, which you may be familiar with, MCP apps can call tools. So, I could do that forward and backward navigation with a tool call. But if you're doing that, remember some tools were not meant to be viewed as a big fat pile of JSON. You want to set visibility app there just to make sure that the model's not going to try to use that and return text.

It's not going to go, "Oh, look at this lovely forward and backward. I will use that." No, sucker arm. Don't do that. All right. So, people are still using browsers, right? There's smarter, more accessible browsers coming out all the time. Big and small browsers are shipping with agents built right into them. Sadly, they still struggle with navigating websites because they rely on taking screenshots and guessing how to use the navigation or traversing the DOM, which burns tokens chewing up XML, which is really not a great use.

Both of these are compute-intensive cuz you're using visual models or you're just eating a bunch of HTML to find out where the next URL is. Wouldn't it be great if they just had a way to call the same functions and links you already have in your JavaScript directly from the page that they're visiting? Well, with Web MCP, they can do just that. It makes every HTML page a mini MCP tools server. However, I just want to point out that Web MCP is to MCP as a JavaScript is to Java.

Inspired by riding the hype wave of But if you actually look under the hood, it's taking inspiration. It is not a one-to-one compliance with the MCP spec. It's really just like tools for agents in your browser, yo. All right. So, the specs might diverge. Keep that in mind. It comes in two flavors. Since my comics are not forms except for search, I use mostly the imperative model. But with declarative, you can just add tool name and tool description attributes to a form.

Boom. If you're using JavaScript or functions to process that form, you just basically told you exposed that form as a tool to the agent. Great for sites with lots of forms already. If you're one of those people whose site is a dashboard, using the imperative model, it pass navigator.modelContext.registerTools for with parameters. It's great for API calls, workflows, data transformation. I used a lot of those. All right.

So, >> [sighs] >> Now, if you if it's available, if you find model context in navigator, you can say, "I'm going to register a tool." You give it a name, give it a description, give it an input schema. My god, it's so similar to MCP. Amazing. How can this not be the same thing? But you also pass it an execution a callback. And in this case, this is a What do we do? Next page? This one basically grabs the link rail next from the page and passes it as the thing that you're supposed to navigate to next.

So, you could call this to navigate forward and backward on the page with a browser without having to click around. Woo. This is new, so I can only help you imagine how it works because the standards bodies are still fighting back and forth on it. So, I'm going to show you how to use it using the MCP B extension. This is kind of like a debugging extension. You can see all of the little uh tools are exposed over there.

I got a little chat window. This kind of imitates what people might see in Gemini or some future in-browser agent. In this case, I believe I am saying uh read this to me. It returns the transcript. And then I might say I might type in I uh thank you. Love that transcript. I might type in like go to the next page. Come on, go to the next page. Oh, I was speaking into the microphone at this point. That's fine. Anyway, >> [clears throat] >> and when I hit when I hit go, it takes me to the next page.

That's just the model calling that function on its own. Yay! That's the wow. Uh don't applaud yet. So, best place to get started to check this out is actually with this interview I did with the creator of Web MCP. It actually started at Amazon. This was something he made to get around a bunch of auth issues and I love that I screen captured a window.navigator.registerTool. Uh there's standards community group, all very lovely.

Um it's not fully supported yet, but you find all the resources in the commentary on that article. So, I started my journey with the agentic web with this talk like 2 years ago when I was like, "The browser's going to die." But what people didn't take away from the clickbait was that the web isn't dead, y'all. Like, it's just changing. It's evolving. CSS and JavaScript aren't just the language of the web. They're the language of interactive experiences on agents.

All right. Here it is in action. I want to show you that there is a version of this that talks. Of course. Brenda looks skeptical while Rachel continues explaining. Rachel, "Ha, Gore left out some important tidbits." Rachel, "What about how plant life will respond to the extra carbon dioxide?" Comma panel two illustration shows tall plants and vegetation growing vigorously. I could keep going, but that's just the Web Speech API there.

It sounds terrible. I'm sure you can do something great with 11 Labs. It's a browser's built-in text-to-speech capability. Zero-dependency solution. No no inference required for that. It's right there waiting for you to use it. So, are all the other amazing uh APIs available on the web today. So, all these APIs, Web Speech, Animation, Audio, Canvas, WASM, CSS, and more. They're already there waiting for you to make something with them.

And I just can't wait to see what you do with this infinite canvas. >> [applause] [music]