Architecting Context-Aware Agents with the Microsoft IQ Stack transcript

Hi everyone. Thank you for joining us for our next live stream. I'll be your producer for this session. Before we start, I do have some quick housekeeping. Please take a moment to read our code of conduct. We seek to provide a respectful environment for both our audience and presenters. While we absolutely encourage engagement in the chat, we ask that you please be mindful of your commentary, remain professional and on topic.

Keep an eye on that chat. We'll be dropping helpful links and checking for questions for our presenters to answer live. Our session is being recorded. It will be available to view on demand right here on the Reactor channel. With that, I'd love to turn it over to our speakers for today. Thank you for joining. >> Welcome everyone. Uh this is Microsoft IQ Live and we're in the very first episode of this series. I'm Acha Bush, a cloud developer advocate and today I'm joined by Marco.

Hi Marco. >> Hello. Hi everybody. I'm Marco Castelline. I'm VP products of Core AI at Microsoft. >> Awesome. So today we have a really exciting topic. We are diving into the Microsoft IQ stack and we are going to talk about architecting contextaw aware agents with Microsoft IQ uh and we will talk about all uh bits and pieces of how we work with four IQs inside the autopilots and if you're new to the autopilot concept we will also briefly talk about what uh agent 365 autopilots are and we will basically brief you uh anything and everything with the demo.

So we will have really less slides and more hands-on demos throughout this session. So okay let's switch the slides and see the problem why we have poor IQ's in the first place. So I think the problem is really clear uh when we don't have any context given to the agent agent is really hesitant to talk about it just like human um when we don't have any information about let's say a museum we go to we cannot talk about uh anything we see over there we need a bit of context so agent is again as good as the context uh it can reach so um across that if you don't give any context to the agent And it will try to fill up information.

Maybe there will be some hallucinations and we will never know if data is correct or not. And also agent will never know the access levels and the permissions across the documents and everything. And because of that reason uh today uh we are talking about the Microsoft IQ one IQ stack and Marco if you can switch to the next slide. Yeah. So um four IQ concepts each are bringing something different into our agent architecture.

Work IQ is responsible of uh bringing the workspace context chats, emails, messages, all of the Microsoft 365 uh tools we are using. We can actually utilize that data in our agent architecture. And then foundry IQ um we can build knowledge bases that are as specialized around our policies uh documents and all of the structured and unstructured data can live behind the Microsoft uh foundry IQ space as well and fabric IQ is the uh space where we can bring the uh knowledge basically all of the operational information uh from uh let's say the system records entities all of the business that is operating and all the information we're keeping behind the scenes, we can connect all of that to our agent through fabric IQ.

And finally, the web IQ uh which is very much needed for pretty much all of the agents is something that can uh offer more than just doing web grounding and we will talk about it briefly too. uh it brings all of the information from outside of our organization and yeah with that I think let's get started with web IQ first and then we will jump into uh the cool demo Marco will introduce us after this so a brief introduction to web IQ web IQ grounds your agent uh information outside of your organization and it's not only um browsing data we can bring news images uh web So all of these uh information can be uh brought uh through web IQ.

So what we're offering here is very different than the web grounding. The quality is definitely different compared to the normal web grounding and also the speed because web IQ is designed for um multi- aent uh architecture. the speed and the information we're grounding behind the scene h can definitely make the difference if you're using multiq structure and also the token efficiency it's definitely lower uh compared to any other web uh service you're using and with that okay let's get started with the demo and then see the web IQ portion of it first >> all right so I'm going to start with web IQ and I'm going to use this raw and I'm using it raw for a good reason in a minute certainly we're going to go into the foundry service and all the crazy stuff in there.

I'm using Web IQ RAW just so you can see how fast it is. Now, I did put Aisha's name in here. We're going to search on you, Aisha. So, let's see. Let's see what what we get. >> Let's try. >> But the reason I'm using this raw is exactly this. Before I could even get the words out of my mouth, the results come back. You see, if I had captured this in an agent, then other stuff would have happened and it would have been we would have been waiting for it.

But when I use it raw like this and this is the playground that uh you know if you have access to this and right now it's still unlimited access but you can find it at webiq.microsoft.ai. Uh in this case I'm using the playground. So when you use the playground like this boy you really see how fast this thing is. It is ridiculous how fast this comes back. I mean almost always subsecond that we get here. And so you know it's got me all of this information about Aicha about her LinkedIn.

And I think I you know I saw these other things but all this stuff came back super quickly. Now there are as I mentioned there are a number of different endpoints oops that you can use here and I guess my my zoom and this don't want to play nicely with each other. So uh you know there is the web endpoint but there are also videos browse news uh images and stuff like that. There is uh some stuff in beta finance places and I don't really know what Sonic is.

I've been meaning to figure that out. What? I mean, I assume maybe it's searching sounds or something. I honestly don't know what that is. I've been meaning to Let's see. Let's see what happens if I try. I'm gonna try this. Actually, I'm just just I've never done this before. >> Again, searching me and >> I'm going to search you. Many songs have been written about Aisha >> about me. [laughter] >> No, I gave you the same stuff. >> The results are in Turkish mostly, by the way. >> They are.

Well, in Turkish. Well, you are Turkish, so I mean I think that's that's reasonable, but In any case, okay. Well, Sonic in this case didn't give me any songs about you. I don't really know what that does. But regardless, the point remains that and if I had done a video search on the other hand, I'm sure I mean there are reams of videos of you on YouTube as you are in fact a developer advocate. You do this stuff all the time.

Anyway, so my point about web IQ and your point about web IQ is exactly that this is a way that you can ground your agents to the real world uh in a very fast and headless way. This is not Bing, by the way. I mean, this is actually a newer indexing strategy than Bing and it's specifically made to work with agents. It's made to have no face. Uh, it's made to be super quick. So, that is the quick version of uh Web IQ. >> Awesome.

So, I'll be quick on um talking about the rest of the IQs, but the next one I'm going to talk about is uh Foundry IQ, which is a very popular one. Um, foundry IQ is the knowledge retrieval for our agentic system. So, it is powered by Azure AI search which which was out there for a really long time providing all of the retrieval augmented generation services for our applications. Now, Foundry IQ is designed for our agentic retrieval where not only you can bring indexes directly to your agent, but if you use Foundry IQ, you can actually design knowledge bases which already has context about your data behind the scene.

So with the foundry IQ, it's not only the indexes coming through Azure AI source to Foundry IQ. You can also create uh let's say connect your structured and unstructured data to foundry IQ such as one leg sharepoint you can connect web you can connect a custom MCP and also you can connect the work IQ fabric IQ you can bring all of the IQs into your knowledge base if you want to build one single unified knowledge base and then manage all of your knowledge across uh the board through a single node space and this is something that really looks like you're designing a mini agent because you can bring a model in and you can basically define which knowledge source is for which information and again you can define the knowledge bases purposes and manage all of that data layer in a different space.

So Foundry IQ is really amazing service when it comes to agentic knowledge retrieval. And next I'm going to talk about fabric IQ briefly which is also amazing uh one of the amazing IQ services. And here you can pretty much share your business's language with your agent. So it is nothing um like Foundry IQ is providing fabric IQ is there for all of your business knowledge. So you can have your organizational roles, entities, all of the rules, let's say all of the information about your customers, let's say account information, shipment information or um basically all of the things that makes your business uh a purpose basically is the language of your business and then you can build ontologies on top of that and share that data through fabric IQ.

So um it's grounded on on top of one leg and here you can build and again design your data agent according to your needs and you need to give all of that um let's say business language to your data agent so that it will be able to understand what we mean when we use let's say three letter acronyms and all of those uh crazy threeletter things we use again also inside Microsoft and so that your agent is aware what we are really talking about and with that those two cool IQs, Murka is going to show you how we can use both of them inside our agent. >> All right, here we go.

So now now we're getting into the real meat of this presentation. Now, first of all, I'm going to start with the fact that in Foundry, uh there are two types of agents. Now, there are hosted agents and there are prompt agents. So, for those of you who have been using the Foundry Agent Service for a little while, you might be accustomed to these kinds of prompt agents where prompt agents, I mean, the instructions are all right here.

The tools are declared right here. This is a different prompt agent than uh different agent than the one I'm going to show in a moment. But basically, prompt agents are for relatively simple use cases where everything can be specified declaratively. But now we have the notion of posted agents. And this one, this is going to be my hotel servicing agent. And this is indeed a hosted agent. So this is using the new capability of the Foundry agent service.

This just went G a week ago I think or maybe a week and a half ago. But uh this basically allows me to use any orchestration agent. So you can have a much more complex agent orchestration. You can have multiple agents inside of here and that kind of thing. Uh but uh that's what I'm using here. I'm using a hosted agent because as you're going to see in a moment, uh, this hotel servicing agent does a lot. And the way it's doing a lot is it is using all of the things.

It's using Foundry IQ, Fabric IQ, and as you're going to see in a little while later, work IQ in a big way. Now, I'm going to ask this again. Actually, what are Now, I'm going to leave this on the screen, but I'm going to ask it again. I'm going to And this is going to spin up over here. Uh, if it lets me. Come on. Let's fire this guy up and ask about Maria Garcia's current reservations. And it's going to take a little while.

It's going to take about, I don't know, 45 seconds or so because, well, the truth is, although this is a demo instance, I have loaded this thing up with 75,000 reservations, 15 hotel properties. I have all this stuff in here, and it is >> the data stating. >> Yeah. And you can actually see it zipping along on the right hand side here, uh, querying all of the things. So right now it's querying fabric and it's using fabric IQ to query fabric and if I had asked it to do something it's going to call Foundry IQ.

So I have this connected to first of all Foundry IQ. So let's look at the Foundry IQ while we're waiting for it to come back. Let's look at the Foundry IQ that I have here. Now what is Foundry IQ? Now you mentioned IA that this is agentic search. Well what does that mean? Now I have these search indices and they don't have to be search indices. I mean, if I look at the sources of this stuff, I mean, there's all kinds of sources that I can add to this thing to Foundry IQ, including arbitrary MCP sources, but uh Foundry IQ in general is meant to be a knowledge store, but kind of an agentic knowledge store.

So, in this case, I've connected it to four different search indices. I have my properties, I have my policies, I have the dining, like the menus and all this stuff. And yes, there are real like menus in here or not real, real fake menus uh in here. And I have my operations. I made this as much as I could to approximate a real hotel chain and what their data would be. And so now if you look at this thing, this is not just your average knowledge base.

It's not just straight up searching this stuff. There is a model here and there's a reasoning effort here. There are also these instructions here. Why is that? Because Foundry IQ is actually a separate agent. And this is something that you'll find really about all of the IQs is that in in reality each one of these things is actually a sub agent. They are a separate agent. And as a separate agent, they have one job. And this guy foundry IQ, its one job is to search across all of these sources according to its input query and to find whatever it is that the agent who called it is asking for.

And so it's applying this agentic reasoning. But that means that it may also go back and forth. It might try one source, not get what it's looking for, try another source. So, it can be recurrent that it's it can go back and forth into the various knowledge sources. Part of the reason why we do this is the notion of reuse. I mean, if I open up, let's see if I if I open up uh Foundry Control Plane over here. Let me let me I didn't have that open before, but let's load this guy up now.

Foundry control plane. This is kind of going to be when this loads. I hope it loads. The front door of Foundry control plane. Let me reload that because maybe it doesn't want to play with me right now. Here it comes. Um, I have a lot of agents running. In a moment, this is going to start tabulating all the agents that I have running. Me and Aisha both actually are using this instance, and we're cracking away making agents in here.

We have a bunch of agents running in here. And in many cases, we want to reuse uh the same documents, the same grounding sources. Now, in this case, I have four, but what if I had 50? Like in any real environment, you're going to have a whole bunch of these things. And those could be search indexes, blob stores, elastic search, it could be all kinds of stuff that are spread all across your business. Now, what you don't want to do necessarily is define this again and again on every single agent that you build.

That is not what you want because these things, each one of these data sources requires some context about what the data source is, what it contains, and all that kind of stuff. And so you'll be cluttering this guy up with tools and with context or what I'm doing here really or what I also did earlier in the prompt agent I connected to a single foundry IQ source and with that I get all of these different knowledge sources all at the same time.

So this way with just that single tool call or what looks like a single tool call from the perspective of the calling agent I am connected to multiple different data sources. So that's what this is all about. That's what Foundry IQ is all about. So it's about reuse and it's also about what I call context delegation. Now, now we do see that our query has come back here and we found Maria Garcia has six current and upcoming reservations on file and here they are at our various Lyenza uh hotels around the world.

She's a she's a world traveler I guess. Now to get this stuff, well actually this particular query that I made was very fabric intensive because this data is all stored in fabric and this is a great place to show what uh fabric IQ is all about because I mean we as humans well this is how we deal with data. So here I am, you know, if I'm the hotel, you know, operator, I might go to this report, this PowerBI report, and we have millions, probably billions of PowerBI reports in operation right now.

This is how we as humans work with data, but this this is not so good for agents. This is not really how agents function. So not great for agents. Now, behind these reports, behind these PowerBI reports are these things. Let's see if I can zoom this thing out. I don't know that I can. Uh, this is a semantic model. And this semantic model, uh, so you know, my properties are in here and my bookings are in here and my maintenance are in here.

I have all this stuff about my my hotels in this database that underlies this thing. And on top of this database is this semantic model. And this semantic model defines things like the relationships between tables. And so here in my semantic model it's it shows indeed that properties are linked to requests. And here also I have these kinds of derived things like the number of cons confirmed requests fulfilled requests these kinds of sums and products and things that I'm calculating over here.

Now there is one thing that I did wrong when I made this semantic model by the way or really when I made the underlying database which is that when you look at this with your eyes you can read it and that's no real semantic model anywhere is like this especially if you're using data from some old school system like some Oracle or SAP where they limited table and column names to eight characters and so everything in here looks like q_rstr but it's for that reason that a semantic model by itself is insufficient for AI.

So when you have an agent, you're connecting it to your data. You you kind of can connect it to a semantic model, but that doesn't really tell it enough. I mean, that tells it the relationships between the tables, but it doesn't under most circumstances doesn't really tell it what those tables are, what they contain, what the meaning of those things are to your business. Which is why there is this button here, which is generate an ontology.

So I did generate an ontology from this semantic model and this is what it looks like or at least this is part of it. I mean there's you can see there's there's entities over here for all manner of things and if I zoom in here a little bit. Now the ontology is a little bit different and an ontology is a metadata metadata layer on top of uh your data. And this is you know one piece of my ontology anyway. And in this piece this is the reservation.

Now, most of the time these boxes in the ontology, they map to tables in the underlying system. That's true. But what these boxes really are, they're not tables. They are business entities. They are things that have a meaning to your business. So, in my business, I have guests and I have reservations and I have maintenance requests and I have whatever they whatever else they might be that happens in a hotel. And so, that's what these things represent.

And when you open these things up, which I'm not going to do because it's not letting me do it. Okay, here we go. When I open these things up, I can look at this thing and I can see that in here I can have about this. I can have descriptions about it. What is it? What is this thing? It's a reservation for a room. What are some synonyms about it? Well, you might call a reservation a booking. You might call it a stay. There's various different things that you might call this thing.

But back to where we were. If this thing lets me get back to it. Uh, there we go. Now these linkages between this stuff, so between a reservation and a special request, between a reservation and a guest, these linkages are not exactly table linkages in an ontology. They are verbs. And so a guest books a reservation. A reservation results in a stay. As a hotel, I hope that you actually do show up for your reservation.

A reservation has a special request. And so this ontology can describe to the agent what this data actually is, what it means to my business and the relationships between this data not just at the table level but at the business level. So I could connect my agent directly to an ontology. If I have an ontology and an ontology generally covers a single data source. When I do that, uh the ontology itself is an MCP server and so I can connect my agent directly to the ontology as an MCP server and it will the MCP server will use the ontology to write the queries.

So it's a sort of a it's NL2 ontology is what it's actually called. Natural language to ontology. You give it natural language, it uses the ontology, it writes the query, it runs the query and gives it back to the agent. can do it that way, but that's not what I did here. That is not the thing that I connected my agent to. And of course, my my demo failed here for some reason, but I will fix it anyway. Uh, so, okay, now I'm in a weird space here because where's my little sidebar?

Okay, let me just open this agent again. Come on, agent. Come on back. Wake up. There we go. All right. So, what I actually did Oh, and I think the problem is I didn't give it all of the tables. That's the mistake I made. Okay. Anyway, what I actually connected my downstream agent to is this. So, you can connect it to an ontology. However, that's just one data source. Now, in this case, I actually have multiple different data sources over here.

Well, I got the symmetric model, but I also have two different lakehouses here. And in any real business, this is going to be the case, right? You're going to have multiple different places where data is stored, especially if you've grown by acquisition or anything like that. In this case, I have two. I have my guest services lakehouse. But then separately somewhere else I have my hotel operations uh uh lakehouse. So I want to connect my agent to multiple different and disparate data sources.

And I can do that with the fabric data agent which is also part of fabric IQ. Now this fabric data agent it's not just that it's connecting it to these different data sources. If we actually look at what's in here now each one of these data sources will have a description. It'll have generally uh some instructions. The instructions often have like a data dictionary in it. Well, that's the ontology, so that's relatively simplistic and stuff like that.

So, it'll have all this stuff in it and then we'll have these kinds of agent instructions. So, there's one thing to like for each data source, it'll tell it what each data source is, what it contains, and that sort of thing. The ontology is a little bit self-describing. We'll have example queries in here as well. But then the agent instructions will tell it moreover kind of like when should it use what like so now you got all these data sources what should it use this lakehouse for what should it use that lakehouse for what should use a semantic model for and that kind of thing and generally too uh we'll have this stuff like let's say uh mice I didn't even know this until I started doing these things with some hotel chains uh like these kinds of acronyms that people and other agents might use in interacting with this data.

So, MICE, I guess, stands for meetings, incentives, conferences, and exhibitions. And this is apparently, I guess, a relatively common acronym in the in the hotel industry. I never heard it before, but my point is like if if if you don't tell this to the agent and you start asking about mice, it might think that one of your hotels is infested. But no, actually, what we mean is uh these are meetings, incentives, conferences, and exhibitions.

And so your agent instructions are a place where you can give it the kind of terminology that you are going to be using in your business and the way that people are going to be interacting with this data. And that's pretty important. That's that's an important step to making all of this stuff work. And once again, I mean, if taken together, like if you look at all this stuff, I got the instructions. I got data source instructions for each one of these things.

I got example queries. And that's a lot of context. That's a bunch of stuff. And so once again, what I don't want to do is load this into my agent. I mean, if I ask something of this agent that doesn't need it to go look up the data for some reason, then I don't need to load up all this context. That's a lot of tokens that it doesn't necessarily need. So, the idea is that when it needs data, when it needs to access the data from any of these sources, it phones a friend. that phones this hotel operations agent.

This hotel operations agent contains all the context, all the data source linkages that it needs uh to be able to get this data and it can do this very effectively because it's all packaged together in one place which is the fabric data agent and these are the various components of fabric IQ. Now back to you AA. >> Thanks Marco. Okay, so let's jump into the last two concepts I want to talk in the slides and we can really dive into the agent we are building.

Uh, okay. So the next topic is work IQ and work IQ as you all know is um one of the most used IQs as well. It helps us reach the Microsoft 365 data behind the scenes. All of the workspace information uh chats uh emails let's say our or charts etc. All of that can be reached through Microsoft uh work IQ. And the reason why we uh come up with work IQ because one of the most common questions we get from developers is is work IQ the replacement of Microsoft Graph and the simple answer is no.

Microsoft graph is still there. It's an API uh set where we still use but work IQ is specifically built for agentic consumption instead of throwing the maybe 200 300 I don't remember the numbers but m Microsoft graph has more than 200 endpoints instead of throwing all of that information to an agent what work IQ does behind the scene is actually providing 10 dynamic tools to an agent and whenever agent need uh whenever agent needs to reach out any Microsoft 365 information, it goes and plans out how to use Microsoft 365 data, it first comes up with a schema and list of APIs it can utilize and then it uses let's say fetch tool to retrieve all of that information and after that if you if it needs to do any action it uses do action uh tool again and there are a bunch of other really cool tools tools.

For example, there's an ask tool which you can directly go and ask a question if you use the ask tool, ask question to entry 65 copilot. So all of those dynamic tools are built for agentic consumption. If your architecture requires M365 data and if you build a agentic architecture then you would need work IQ and this is definitely the optimized version of Microsoft 365 data set where you can bring this data set using three different endpoints.

You can either use work rest APIs, you can use A2A or you can use MCP. Um and in this demo I think Marco is using the MCP version but you in addition to all that you can also bring work IQ as a part of the foundry IQ too. If you don't want to let's say do any actions take any actions with your agent you can just use work IQ as a data source. So you can use M365 data as a single data source. And on top of all of these IQ goodness, there is one extra topic we want to tap into today, which is the autopilot concept.

This is a very it's getting more and more popular. We are getting a lot of questions about this. Uh and agent 365 autopilot works really well and natively with all of the IQs as well. That's why we wanted to cover this as a part of uh our uh slide deck. But also on top of all that, Marco's demo also includes autopilots which is also another benefit to it. Um, so what is agent 365 autopilot in the first place and how are they really different than any other agent?

First of all, when you build an agent, let's say you want to use that agent on teams. When you actually publish that agent to teams, and in fact there's a button on Foundry, you can publish a teams. what you actually do behind the scenes is actually wrapping your agent to a bot service and then publishing an app on teams platform. Uh but in agent 365 which is a really new service what we're providing here is instead of bringing agents as apps why not bringing agents into our tenants as their own identity.

So basically recruiting an agent into a tenant which let's say agent can report to Marco agent can also have its own mailbox its own calendar team's presence it can join in meetings it can take notes it can send emails on behalf of itself not on behalf of you. So that changes a lot of dynamics in the entire tenant because in that case agent can have its own space its own collaborations and you can even see uh who agents is working most uh the most in the entire tenant etc.

So when you create hosted agents, it already comes with a agent 365 blueprint. And when I say blueprint, I'm talking about the identity ID. Um and you can just like how you collaborate with any other colleague, you can at mention the autopilot, you can collaborate with it. When you ask uh that autopilot to send an email, it will use its own email address to send the email and take all of the actions on behalf of itself.

And on top of all that, it works pretty natively as I mentioned earlier with Microsoft IQ stack. So when you bring um IQ's into the autopilot then it really has all of the capabilities and knowledge that another colleague can have because what you really build over there is not just let's say retrieving a policy or retrieving let's say a shipment information but agent now has if you bring work IQ the knowledge of let's say what is talked in that recent meeting if that customer needs to be prioritized or let's say there's some new changes in the finance and let's say there's a new hiring freeze etc all of that company internal knowledge is also part of the agent so that's why connecting autopilots with the IQ is a really power uh solution power dynamic and one last slide I want to show you before we switch to Marco's autopilot demo with four IQs what we are trying to show you today is this very very simple diagram program we have the autopilot with its own identity.

So agent we build in the foundry uh connecting to all of the IQs is becoming an agent uh 365 autopilot and connecting across the IQs to be able to bring all of those data set into the workspace so that agent is capable of knowing what is in the policies in the company what is in the company like what what do we really do in the company business and in the same time access to the web IQ and get retrieve information from the web.

On top of all that, also has access to the work IQ for all of the information that we all usually have in the work tenant. And with all that, let's see this awesome demo. >> All right, folks. Here we go. All right. Now, now we're going to get to the real the real stuff. So, let's check this out now. Um, okay. So, first of all, now let's talk about Work IQ. I mentioned uh my agents here, my my hotel agent is indeed using work IQ to access emails and things like that.

Well, but you know, work IQ is actually pretty useful in lots of different ways. I mean, here uh you'll notice that if you're using M365 Copilot, it is using Work IQ. Look at that. And so when I ask her, do I have any emails from Maria Garcia? It finds my emails from Maria Garcia. How did it find them? Well, it was using work IQ. Believe it or not, behind the scenes, even our own tools use work IQ. Another way that you can use work IQ uh is well, here I am in the GitHub copilot app.

This has become my primary coding agent now. I disagree on this. She likes the CLI. Well, I should speak for you, but I like this app. I think this app is cool. um because I've never been a CLI person, but you know, I had to do some I I administer an Azure domain for a bunch of people actually and in part of my job is I administer this Azure domain and I got a little bit of a nasty gram from our security that people had created storage resources in this Azure instance that uh had open authentication.

That is to say, you could you know access them from the web and in this case we generally don't allow that without special approval. So, I had to go through them all and find them and determine who owned them, like who created them, who owned them, uh, do they still work here, do we even still need the security resource, all that stuff. And what did I use for that? Or really, what did it use for that? Work IQ right there.

So, you I actually have connected my uh my GitHub copilot app to work IQ as an MCP server. And this is super handy because what I had to do, if you look on the right hand side, and I've kind of hidden part of it so that it doesn't show a lot of PII here, but uh I made it actually write emails to each one of these people, 27 of them, mind you, and said, "Hey, I found these storage accounts for you uh that have open authentication.

I'm going to shut that off. Let me know if you need, you know, the special approval, like if you have something depending on this thing." And then I scheduled a task for nine o'clock this morning actually to turn that all off uh for everybody. So uh so you can use work IQ even for coding tasks and even like actually even when I wrote that hotel uh demonstration that I was showing earlier. I had a problem with it that I know Aisha had solved before and she sent it to me in a team's message or an email or something and I said somewhere in there in the hotel agent I'm like go look at the teams messages from Aisha. she solved this already, whatever the problem is.

And it did it. It found it. Uh it found it in my teams and and there it was. Now, it's also used here in Scout. Uh and so here here I'm like, where's that slide deck that I just sent me? What did it do? It totally used work IQ to find this very slide deck that we're showing here today. So if you haven't seen Microsoft Scout, we used to call it claw pilot, but basically it's a little thing that runs and it has lots of uh claw like capabilities.

Even though it has no code in common with openclaw, it uses work IQ as well, but all of these things all of these things have uh one thing in common which is that all of these things are running on behalf of me. So work IQ has a mode where it can run on behalf of you. And so when you use work IQ, uh it is logging in as you and that means that it has all the permissions that you have for better or worse, right? It can search my emails, it can search my teams, but it could also delete files from one drive and sharepoints and things like that.

Well, now this hotel agent though, the way I have actually deployed this hotel agent earlier, we we used it uh where where did it go? There it is. We used it like this and in this case it was, you know, use it was on behalf of me. So, it's using my ID, but that's not how I how I deployed this. Actually, how this is deployed is like this. This is the real hotel guest services agent. And even looking at it this way, you're like, "Oh, okay.

Well, it's a chatbot. That's nice." No, not exactly. Because if I look at this in the org chart, you see, well, it reports to me. This thing has its own user. And that means, too, that it has its own email inbox. which means that it itself can email Maria Garcia who cancelled her reservation in Scottsdale. So, uh, so this thing has its own identity. But when we say it has its own identity in the Microsoft world, we don't just mean an intra ID.

We mean, well, it has an entra ID certainly, but we also mean that we've provisioned it a full M365 instance. So it has its own email, its own one drive space, all of those things. And it's using work IQ as the portal to all of this stuff. So how did it send this email? How is it checking its teams messages and all of these things? It's doing that using work IQ because what work IQ really is, it's the headless version of all of the Microsoft apps of Outlook and Teams, of Word and SharePoint, of even Dynamics.

Now uh that's the idea of work IQ the agentf facing version of this. So underneath all of this down here we got this agent running in foundry but I've defined it up here in agent 365 and I defined it in a very special way. I defined it as an agent template. When you define something as an agent template what that means is that now I can go or really anybody can go in here. I should you could do this too. We look at built for your org.

And here's my one of these is the right one. I' I've actually made a couple of these as I noodled with it. But >> I talk to your agents time to time. >> Yes, you can. Well, you can make one of your own, right? So, here's my here's my hotel guest services agent. And in a moment here, this should light up. There we go. Now, basically what this means is now you Aisha, you can create an instance of this agent of your own.

When you create this instance, then you will have your hotel guest services instance that reports to you in fact. And so there can be more than one of these agents. Now, underneath the scenes, they all share the same Foundry agent codes. They're all going to be running in the same Foundry agent, but their context is different. They're going to be running in different user contexts. Yours will be running in your context, your agents users context.

Mine will be running in my agents users context. But speaking of context, uh another cool thing about this is that these things run in their own security context. These agents, now you can define this at the template level. And so I can decide at the template level what permissions I want this agent to be able to have. Now let's say for example that I did not want this agent to have rewrite all to files. Now there's another agent that I've been working on which is more a bank agent.

This is another one that I've been building right now. So this is also an agent template. And in this bank agent now banks have you know a rather stricter security requirement. So for this one aha I have set it only to file read all. And so this type of user can't possibly delete a file. In fact in this case it can't even write a file. Uh which is maybe a bridge too far. But I mean this cannot possibly do it. this user just can't do it.

It is banned. There's nothing that this user can do to mess with my files. So now, like I said, it's it's logged in as a different user than me. That does have some nuances, though, you know, like so for example, like let's say that I made a video of our Lorenza Scottdale Sonoran Resort. I made this video as me and I and and now I say to my hotel guest services uh agent here, hey, share Maria Garcia on email. Email her a link to my my video that is stored in one drive, right?

It can't do it. It can't do it unless I have previously shared it to the hotel guest services anymore than a can do it. A is a different user for me and so is this. And so unless I share that file to it upfront, it can't even see it. So these are some considerations you have to take into account when you make these users that are separate users. You make these agents that are separate users. Well, they are running in a different space.

And so if you want them to be able to work with or manipulate files in one drive or shareepoint for example, you have to explicitly share that to them. You know, now you could share folders to them or whatever like that, but regardless you do have to take that into account. Furthermore, to make this all work, uh there is a cocktail of permissions. So in reality when you do this stuff and uh this bank agent I was showing you a moment ago is actually my seventh one.

So I've been I've been doing this bit by bit as this technology has matured but this user this hotel guest services user and again it is literally a user it needs to have permissions to the underlying Azure thing. So it has to have I forget what I think it's founder user permission uh because it is a user so you need to give it that arbback role or add it to a security group in Azure that gives it that role. It needs to have permissions to the underlying fabric as well.

So it is a separate user. It needs specifically to be granted these roles in Azure in fabric in M365 so that it can see what it needs to see. And that's part of getting these things set up is not just setting up the security context, but setting up the roles on the underlying resources so that this type of user can see all the things that it needs to see to do its job. So that is pretty key to making all this work. But the fact is back to work IQ.

Work IQ is the headless version of the Microsoft apps. It can be used on your behalf as here or it can be used on the agent's behalf as here on behalf of the agents own standalone user which yes at Microsoft we have been calling them autopilot agents for better or worse because it doesn't mean they're really autopilot but I mean this hotel guest services thing I can make it such that like it emails me or teams me for approval for something like oh you know Maria wants to change a reservation like do you approve this?

So, it's not really an autopilot in that sense of the word. It's all how you set it up, but that's what we're calling it. So, there it is. >> There is a really um related question in chat. Uh Anton is asking when autopilot agent is using work IQ, is it using its own permissions as a user agent to identify sites and resources it has access to? And what you were just explaining is that if you have a file but agent identity doesn't have access to that file then agent cannot do anything about it.

And I actually learned this in a really interesting way when we were building one of these autopilots for builds. Um I set everything up in fabric and everything in the fabric IQ was working and it was also working in Foundry because when you log into Foundry uh it's using your own credentials. So it uses your own token and agent works in the foundry. But when I switch to teams, teams agent has its own identity. It has its own blueprint.

It's not working at all. And the one little thing I forgot is adding the agent in the fabric workspace where agent needs to be a part of that fabric workspace so that it can access to the data through fabric IQ and it can work with the data. In fact, if you go to your fabric workspace, I'm sure this hotel agent uh the hotel autopilot is a part of that workspace. So, >> yes. Now, there's another question on the chat here and I am unable to reply to the chat for some reason, for some technical reason I can't figure out, but that's okay.

I'm going to say it live. So, John James Marsh, who I believe you said you're coming from Riad, which is cool. I'm going to be there myself in a couple of weeks for the lead conference. Uh, John James Marsh asks, "What type of license do you need for work IQ?" And uh, well, so what type of license do you user need? You need, I think, a copilot license. Am I right about that, Ara? >> Yes, correct. You need M365 co-pilot license. >> But what type of license does an agent need if it's using its own identity with the work IQ?

I believe this is all it needs now. Am I right about that? >> Frontier. Yes, you would need Microsoft 365 Frontier uh license to be able to work with Agent 365 in the first place. >> Right now I cannot zoom out. It won't let me zoom out. There we go. [laughter] Whoa. Um anyway, okay. Uh now, uh one of the thing I and and you remember that I was saying earlier that one of the things and I'm going to get I'm going to I'm going to show you a quick gotcha over here.

Now, let me see if I can find this one over here. Now, you remember actually you're the one who taught me this. Uh do you remember that I said earlier that there was when I was building this hotel agent, there was some problem that you had solved that I didn't remember what it was, but I made it go look it up. I made the co the co-pilot go look it up for me. And it was, if I remember correctly, it was a co-pilot setting over here.

There are these policies in agent 365. And somewhere in here, uh you do have to enable, which one was it? Do you remember what it was that we had to enable in here to make this thing work? one of these policies. >> I think it was Frontier. Yeah. >> Frontier. >> Yeah. So, >> anyway, I'm just saying like this one got me for like it was like I mean it was a whole night that this got me until you came online. I was in a different time zone than I am.

You came online and you told me straight away what the problem is. I turned it on and everything worked. So, so that doesn't happen to all of you. Just so you know. Please turn that on. Don't don't mess with your own self like this. All right, let's take some other uh comments and questions that are in here. >> Yeah, also one other thing is that uh I don't know if you remember but figuring out if agent is sending an email or not took us a lot of time.

We actually weren't to exchange admin gave us access to agents mailbox because we we were not able to see it at all. I don't know if you have the Outlook open, but you should really need to grant yourself access to able to see if agent is really sending email or not. >> Right. So, this is indeed that's what I did here. That's what that's what Aisha is referring to. So, I have my hotel guest services. This is the this is the actual agents email box.

And what I did here is at the exchange admin level, uh I delegated its box to me. Now one of the challenges by the way of doing this in a real corporate environment see the thing is I am the super admin of all of the things and AA is too right she and I are sharing this instance and and both of us can do everything everywhere in uh in Azure in fabric in outlook and exchange and entra we have all the things over here now most of you in a real world environment are not going to be that way right you're going to be the Azure administrator maybe but not the entra administrator you're going to you know whatever you'll be able to administer part of But not the other part.

This does require some collaboration between administrators. You're going to have to make some friends. Maybe buy some boxes of chocolate or something to make this work because everybody likes chocolate. Maybe not you, Aisha. You're not a big chocolate person, are you? I can't remember. [laughter] >> Everybody except likes chocolate. So, don't buy her chocolate. But in any case, uh, but you know, my point is like there is a little bit of a collaborative effort that's going to happen here because you may build the underpinnings of this stuff in Foundry and you may have the data in fabric, but you're going to need your exchange administrator to help you out with some of this stuff, including, for example, delegating the uh the mailbox here because the agent really has no power to do this. the agent itself, hotel guest services agent can't delegate its own mailbox to me.

There's nothing I could do to make it do that. I don't think there is anyway. Uh so that kind of has to be done at an admin level uh when this agent is created and it is a best practice actually as as Aisha mentions so that you can see what it's doing. >> Yeah, there there is also a question in the chat. Pablo is asking, "What shall we do if we want an agent autopilot um to be able to access all the contexts of the members of a team except email perhaps?

Um but I'm not really sure if it's uh something we can do with autopilots. We cannot uh like autopilot cannot access to um users information like basically it cannot act on behalf of me or you. It acts on behalf of itself." Well, unless you unless you share it, right? So, like I mean here for example, I could go here and say go to my calendar and I could share my calendar. Let's say I'll pick this calendar. I can I can share it with uh with my hotel guest services agent or I can share it with my my bank agent here which is called Marcos Teller like so.

Uh well, I have two of them. So, anyway, so you can share stuff with these agents just like you would with another user. You can delegate stuff to these agents. And so I could I could, you know, choose how I share my calendar. Does it have the ability to literally edit events on there and make events? Does it have the ability to make events as me? So you can choose uh by sharing files, by sharing emails or whatever or calendars.

You can choose what you want this thing to be able to see. You and other users can choose these things, but you do have to make those choices. >> Yeah. And in fact what I would really do when you uh gave this advice just came to me I would really add the agent into into the team's chat or channel so that agent can see everything we are sharing over there h if there's a conversation happening in the team's meeting agent being in the chat will have will give the agent all the context probably and as much as data is shared with the agent just like another colleague then agent will be able to access to it. >> All right, other questions.

Uh, what do you what do you see going on in the in the chat over here? I >> um there is one additional question I've seen. It's a little bit uh on top. Miguel was asking um basically saying that it's the first time hearing about Microsoft IQ stack. Interesting in learning more. Is there a a beginner course introductory material? We actually have Microsoft IQ series where we have beginner friendly videos and cookbooks uh available there and actually that really last slide is very relevant to this question.

Um so uh the IQ series is definitely the place if you want to dig deeper into individual IQs, learn about you know how you can use them in agents or in different concepts and u we have cookbooks videos and if you complete all of that then you will be also earning the digital badge of individual IQs. We currently have foundry IQ and work IQ. Fabric IQ is also in the works. We will be releasing that soon. Um and you can also access the slides we just shared.

Um and you can see um all the you know you can reuse the slides if you want to. And also just to give you a brief information about the series. This is the very first episode of the Microsoft IQ live and we really wanted to start this uh series bi-weekly series because we really wanted to have a space to talk about what is new with the IQs and dive deeper into individual IQs and different multiq concepts. Uh so if you continue um watching the IQ live series you will be learning more in every episode. uh go ahead and check out AKMSicrosoft IQ live link you will see all of the scheduled sessions in the agenda we have starting from today until Microsoft Ignite we have bi-weekly sessions scheduled and each one of them is covering something uh that we basically slightly tap into today but going deeper in the every concept in every uh possible way and for example the next session is going to be about Foundry IQ serverless which we didn't get a chance to talk about today, but it's also a very cool um uh service we just released at build and uh I'm sure you will also get a lot a lot of information about it in the this session uh happening in two weeks.

Another really cool session is web IQ. We will dive deeper into the web IQ with the product team being here and showing you all of the goodness that can come from the web IQ, so on so forth. Yeah. So, uh, Marco, which session are you most excited about in the IQ live? >> Well, you know, I mean, I'm a big fan of Fabric IQ, actually. I mean, and in part because of how kind of chunky it is. Well, you know, I mean, I love my work IQ and I use it every day, but I mean, I think uh that Fabric IQ has a lot of aspects to it that are hard to replicate in other ways or with other companies.

And so, that I would say is one that I would be more excited about than some of the others. Yeah, I guess so. And there's one question uh and I'm going to direct it to you. Are fab fabric IQ and Foundry IQ can work together. If yes, then how can we do it? Any document reference? >> Okay, so Foundry IQ and Fabric IQ can work together actually. And there's a couple of different ways that you can go about that. You actually can connect Foundry IQ directly to Fabric IQ if you want to or you could just have them side by side and that's what I did in in my case in the case of this agent that we were looking at today and really most of the agents that I built.

I have them side by side and there there are reasons why you might want to do both. Basically, uh, if you might need, um, an agent to be the arbiter between different sources of information, one of which is fabric IQ, and there are others that, uh, you know, other structured data stores that you might want to sort of arbitrate between, then you might want to do the connection via Foundry IQ. It seems to me that under most circumstances, these things will be more like sidebyside that you won't call Fabric IQ from Foundry IQ, but rather you'll call, you know, you'll have your agent connected as one tool to Foundry IQ as a second tool to Fabric IQ as I did it here.

But you have both options and you can kind of choose uh which one works for you. And there's also a question as to whether Fabric IQ uh uses Purview. I'm actually not sure whether fabric IQ itself uses perview and that might be a good thing to uh cover on the fabric IQ dedicated session but the fact is that when you make these agent users uh they are equally subject to perview. I didn't really go into that but there's a whole purview settings for that uh and all that kind of stuff.

And so uh perview applies to autopilot style users just as it does to any other user in your organization. And it applies to the same data sets as well. uh so uh you know I mean maybe we can go deeper into that portion of it into the data protection portion of it in the fabric IQ session but on the surface I'll say yes purview applies to those things just as it applies to you as a user >> uh and and and there's a question on licensing too will we be able to use work IQ for custom agents without requiring an M365 co-pilot license in the future uh well I believe if I'm not mistaken a now it all of that stuff is rolled into that M365 Frontier license, the one that we looked at earlier.

So, you don't have to give it all the different licenses anymore. Is that right? Yeah. >> There might be some additional changes uh coming in the future, but currently we need the M365 copilot license to be able to utilize work IQ. The Frontier license is required for agent 365. Um but yeah, so uh we will see what what comes next with the work IQ. Uh I'm not really sure if there will be any changes. This is something we can really ask in the work IQ uh live stream we have in September. >> Indeed.

Indeed. And it's maturing quickly. I mean a lot of these things are changing very quickly. And so even from when I started working with this stuff what it was only a few months ago, right? April I think something like that to now. Well, it's gotten a lot easier for one thing and the licensing has gotten cleaned up and those kinds of things. And so we are making continued improvements uh to to make this easier and easier to work with. >> Yeah, I think we can take one last question which is about purview.

You just talked about is fabric IQ used purview internally. >> Yeah. Well, like I said, I mean I I don't really have a great answer for that one. So I mean I think that might be a a better topic for the fabric IQ deep dive that you have coming up uh in the near future. Yeah, in fact we have two sessions for fabric IQ. One is talking about fabric IQ all up and the next session is going to be about ontologies deep dive where we talk specifically about ontologies in fabric.

So you should definitely catch all of those uh if you're interested in fabric more. And with that we just have one minute left. So I'm going to close the session out. Thank thank you so much Marco. It was an amazing session and if you can share the demo uh that you have I'm sure you have GitHub repo of that uh in the chat that would be also really amazing if anyone wants to dig deeper into that and yeah thank you so much for joining in this session and speaking with me. >> All right thank you AA for having me see you next time.

Thank green or in the chat and we'll see you at the next one. Thank you all for joining and thanks again to our speakers. We're always looking to improve our sessions and your experience. If you have any feedback for us, we would love to hear what you have to say. You can find our survey link on the screen or in the chat and