Privacy-Preserving Intelligence — Steve Korshakov, Bee (acq. Amazon) transcript

[music] Hello everyone. I hope this talk will be shorter. Uh I'm from Amazon. Uh our company was acquired about 8 months ago and we built the uh AI wearable which is on my hand. Uh which is essentially a microphone that records everything and builds your personal agent, personal AI and uh on top of that you can extract all the data that you record and plug it to your systems or agents uh and do whatever you want. Uh just to be to get in perspective how confidential how like private data we capturing uh a single person usually like captures about 10 million tokens per year.

So this um and even within like a first week of recording uh uh people usually tell uh extremely sensitive stuff to their friends to their family. uh you can learn virtually everything about the person within the just like one week of wearing the B device which is extremely sensitive. I think we one of the most sensitive uh capture device on the market now. So and uh because of this we had to encrypt everything and our mission was to not have access to any of this data and not being able to look at it anyone at Amazon and it became a little bit challenging for us at Amazon because uh Amazon itself provides strong security and privacy guarantees but if you Amazon and you using Amazon stuff there is like much more uh serious security stuff you need to do.

Uh first of all we defined like few core principles what we needed to do for our specific agent. First of all we believe the agent should be like working all the time non-stop for for your good. Um it should be doing stuff on your behalf. Uh and um we should not consume uh customer resources such as batteries and stuff. So this way uh we uh so this leads us to one specific design uh of the um of the our system. Current system usually builds the uh on state uh on request response uh system where you send uh request to a l from your say iPhone uh calculate something and the back end that gives you back.

Unfortunately, we already see that this is not enough uh that we need to uh that we need to run stuff continuously and sometimes for days we can see uh we can see this like as a like glimpse into the future how cloud code works. So like just few months ago it was like more like request response stuff like change this, change that and now it can works for like hours for us. We think the same will happen to all your like personal agents anyway.

So uh because of this we built uh a stateful runtime with persistent memory um that we still don't have access to. Uh it can connect to different tools. It can connect to any like third party services if we if we program it to um and we don't require the user device to be online. So it's fully autonomous but at the same time it's fully controlled by the user. Um so encryption system is built um uh on few on four like core ideas that we need to follow.

First of all the key leaves and manage it only on customer device. So it's users iPhone or Android device itself. We don't have the key ourselves. We don't persist it anywhere. So keys is is um is stored only on the customer phone. Everything is encrypted. We don't have any opt out there. No way to disable it. There no way to bypass it. Uh at the same time we uh to protect ourselves from um like internal threats uh we do fully transparent and audit uh of our all our workloads and we on top of that we try to minimize the dependencies on the uh on what we can trust really.

Um so uh any security system if you do end to end encryption or any kind of encryption there is a huge problem is key management. So the first step is like I want to tell you how we manage the key. So we start with the uh the key as I mentioned before starts on the phone and it leaves persisted on the phone. Then the phone connects to our back end and runs very sophisticated attestation uh pipeline uh that verifies both integrity and that the specific workload is inside of public uh transparency log.

We use six store for our transparency log and anyone can go there and try to look and verify that this workload is genuine. uh the the method is too complicated to include in this uh talk but we will publish details uh at some point. Uh once the uh once uh at the station was finished we the the client shares with the our main front end back end and back end then replicates this key with uh the similar nodes uh that runs within our confidation compute because we can't leave the unencrypted data out of our perimeter uh all our we run our own inference too.

So this puts us a little bit uh more complicated task uh than typical AI company. We run like all kind of models, all kind of inference uh uh software and uh so um yeah and we don't replicate code to this inference node. We like replicate only on specific ones limiting the scope of what we can do. And on top of that we introduced everywhere where we have the keys in the memory uh uh the forced expiration of seven days.

Uh we picked the seven days because we think it's like how much realistically the time horizon horizon for the like something useful can be done for the user. uh 24 hours will be too low because you can like not open your phone for like 24 hours something will be missed and like so we pick like about seven days. Um then we need you know we need to ship some something to the production and then the the biggest question like how we can uh well not ship something that will compromise anything.

Our goal was to uh build a system that no one inside of uh Amazon will be able to ship anything unnoticed. Obviously the software has bugs have problems but we shouldn't be well being able to ship anything. So we solve this by two uh two tire system essentially. So there is a dedicated team inside of organization inside of uh Amazon and maybe maybe not even one I would say uh that um manages the privacy part of this the transparency log our team when we ship the software we can influence them we can't control them and we hardcode their in their signing keys inside of our client apps and our back ends so we can't really uh um uh we as the team we can do this and like it's very very hard and very high level uh um employees need to sign off to any kind of change so it's like at the big company like Amazon it's virtually impossible really um and uh and we do this in two parts because it's this process too slow so we split it in two parts so like the first one is to build the base image which we put some kind of base software for uh that is needed for our own team like the tools that measure the boot, measure the manifest, measure workloads and data that we need to uh put to the node.

Um and um and then when we want to deploy we do the process the same similar time. uh we got the base image and then we deploy the uh to transparency log specific manifest that anyone and uh this setup helps us to be able to security audit companies and inside outside to anyone to well we're not doing this public but like we two like very um high-profile audit companies we work with them all the time um we can provide any image any any data that was deployed ever so we can like trace any possible uh weak spots uh if we like deploy something wrong uh which we do not um then after selfverification of VM it's uh issues a certificate that embeds all encryption pro uh all transparency proofs attestation documents into certificate itself we are using private CA because you can't do this in public certificates because it will populate the public uh transparency log so we had to use the private on um we probably will introduce the uh extra proxy that will do a normal TLS with attestation with like lighter mode.

Uh but we don't have this yet. Um yeah, that's essentially what we built. Thank you. Any questions? [applause] >> Any questions? Another going inside as far as Can you repeat? >> Yeah. Sorry. Were there things that had to change your own existing? >> Oh, what changed with like when we joined Amazon? >> Yeah. >> Well, the the big change is that we uh before like you run on Amazon and Amazon gives you pre like guarantees as a customer that they can see your data.

But once you inside this changes a lot because you Amazon like so that's why you need like to provide more protection on top of this. So we need to protect from our internal threats too. Uh so that's was a big change. So um before that it was um just kind of easier I would say to configure everything. Um not sure I can tell much honestly. Yes, it's just normal EC2 instances. Yeah. >> Yeah. Yeah. We almost we not using like Yeah.

Almost everything we build from scratch. Well, um we tried to use like the existing stuff like that is more like common like popular software. We built uh try we try to minimize amount of code that we produce. So it's I I calculated before this talk it's just like about 20k lines on memory safe language. So it was very small scope that we were able to audit and verify that all this kind of stuff and most of this code is just verifying at the station really and everything else can be like reused and like very we like you know it's very trustworthy I would say software so we didn't try to don't invent uh um yeah we don't try to invent like when when I was a telegram like we reintroduced like build our own crypto and that was like questionable way of doing stuff.

So, I try not to do the same at Amazon, obviously. Yeah, that's what we do. >> Any other questions for Steve >> up in the back? Just think about Well, I I just prefer them not to put to the computers uh to personal one. Uh so we we did several experiments how to tame them, not to do bad things. H say I think nothing works except like sandboxing and just not giving them a way to hurt themselves. It's like you know our brains they can't stop the heart at will right so otherwise you know they will be we have much more problems so I think the same we shouldn't give them away to do any harm that's the only way honestly and um yeah and put something in between if they want to change something unfort yeah >> uh I surprised they're not representing this uh right like they're screaming so much about security but they didn't came to this one.

Uh uh I'm not I tried open claw. It's like it was once they started to try to tighten this down it became much less use useful. Um so I think their approach is not really that good. So I would love to have wild agent that's that's our goal too but we try to just deploy the sandboxer for specific agent and it will just they just can't do much of the stuff. Everything else fails unfortunately. >> Cool. All right. Well, thank you so much, Steve, for the presentation.

That was amazing.