Impl Trait aka Look ma’, no generics! by Jon Gjengset transcript

All right, let's see if my end of this works then. Yay! All right, uh my name is John and today I'm going to run you through almost quite literally uh through impl trait um or otherwise known as look ma no generics. Uh impl trait is this weird chameleon type where it looks like you can put it in a bunch of different places and that it kind of does the same thing, but in reality it's actually quite different depending on which position you put it in.

It means in fact completely different things in mm let's say half of these contexts and then the other half it's it's kind of similar. Um and so today I'm going to go through some of those. There's a decent amount to cover so I'll be speaking relatively quickly. Uh hopefully you'll have questions at the end about what did you say during those, you know, 30 minutes in the middle there. Um All right, so let's start. Let's start with uh the one that we've all come to love which is also the first one that landed in Rust 1.26 which is uh impl trait in return position.

So what problem is this trying to solve? Um well, return types that have weird names is really where this is coming from. So imagine you have a function like the one on the right where you want to return some function um no a some type, but the the type there is kind of hard to name. In fact, it's impossible to name cuz it includes a closure. Enclosures do not have nameable types in Rust. This gets weird. You can return instead something like a boxed din in this case iterator item equals whatever, um but this includes overhead which makes us sad, right?

Now you have a heap allocation, you have dynamic dispatch. Um this also means you get fewer optimizations because you lose out on monomorphization. That's also sad. Um but beyond the fact that it is, you know, overhead which which as Rust programmers we're kind of allergic to, the other thing is that it requires object safety for the trait that you want to return here. It also requires that it actually implements a trait, but let's take that for granted.

It requires that trait is object safe. I'm not going to go too much into object safety, but it just places some requirements on that trait, some constraints that you have to that trait has to meet, and that also makes us sad. The other thing that makes us sad about having to go through box din trait here is that it doesn't nest nicely. You have to pay this cost at every layer of nesting of the types. So, in this case, I have an outer type, which is an async block, and async blocks are not namable.

And then inside of it, I have an iterator, and whose name is also not namable. And so, I end up having to write something like box din future output equals box din the iterator item equals, and it just gets really nasty, but also you pay this overhead, this this allocation, this dynamic dispatch at each level. So, we're sad about this. And so, in Rust 1.26, we got return position impl trait, also known as arpit, or armpit, if you wish.

These are also known as existential types. So, the idea here is basically you're telling the compiler there is some type, like I as a human know there's some type you can put here. Please just put it there for me, compiler. And so, in this case, you can just write as the return type here impl iterator. And what that means is the compiler knows that there's some type in here that it can infer, and it should just make that type be there.

And then the only thing it's going to promise about that type is that it implements the following trait. So, I don't need to name the closure or the async block or whatever. I just say that it implements a particular trait. This also then has really the reality that there are two types in play. These are called the opaque type and the hidden type. The hidden type is the real type under the hood, the one that the compiler knows about.

In this case, it'll be something like stood iter filter with a type that has a closer as a name and then some other bits on it. Like this weird hairy type that we might not be able to name. That is the hidden type. The opaque type is the type that we show to callers of this API. It is the one that's written in the source code. And the opaque type is opaque because you do not get to see through it. So, you don't get to look at the hidden type.

So, just keep those terms in mind as I go forward. Um what's nice is that this also nests. All right, so you can write in this case impl future output equals impl iterator. Uh so, you can keep writing them down like this and unlike the box dyn case, um you don't have the same downsides to doing so. You don't end up with multiple layers of dynamic dispatch and you also don't have the requirement to around things like object safety.

So, that makes us happy. Um impl trait one of its sort of core features in this position is that it lets you promise less about your APIs. So, if you compare the two functions on the right here, F1 and F2. F1 promises that it returns the into iter type that you get back from vectors. So, it promises that if for all eternity in this API, I will return that type. The second one might actually return the same type, but the only thing it promises is that the thing that is returned return implements iterator over U32s.

This means that if in the future you wanted to change the implementation of this function to let's say iterate over a B-tree set or something, you could do that without it being a breaking change. You could not as easily do that with F1. You can hide implementation details that you want the ability to change later. Um now, there are a couple of caveats with return position impl trait. The first of them is that as I said, there has to be some type, some single type that the compiler knows about that it can use for the hidden type.

This means that if you have branches like this and the branches return different types, there is no one hidden type that the compiler could choose here. And so, you don't get to use impl trait in these cases. The the would just tell you there is no single hidden type. And so instead here you can either uh uh use boxed in so you can box them um or you could use an enum that implements the trait and then you know delegates to the appropriate variant.

The nice thing though is you can still use the syntax. You can still say returns impl iterator and then internally you do like a boxing like a boxed in or something and that boxed in is still going to implement the trait. Um as a result you still get to hide the the details of the implementation. So if you later on refactor this so that there is actually only one type uh you can then get rid of that stuff internally and it's still not a backwards incompatible change.

Um now one thing that's important to know about impl trait is that auto traits are preserved. So auto traits are things like send sync um uh pin unpin sorry. Uh so send sync unpin sized the the traits that the compiler automatically implements for types. Um those you're allowed to sort of p or the compiler rather is allowed to peek through. So here you have the the gen function at the top that promises only that it returns an impl send impl sized but it really returns the unit type which implements all sorts of traits including a lot of auto traits.

And then you have the what function which calls gen and just returns the same thing gen returned. But the what function promises that the thing it returns implements sized and send and unpin. And you might assume that this shouldn't compile but it does because send and unpin are auto traits and the compiler is allowed to look through this opaque type into the hidden type and realize that this hidden type actually implements these traits.

This see might seem a little weird it might seem a little leaky. It's because it makes impl traits a lot nicer to work with. But it's worth knowing about because this can be a backwards compatibility hazard. So you might write a function where you return something that has impl trait in it but you're not thinking of the fact that it also includes a sort of additional guarantees of plus send because the function you've currently written the type you've currently put there implements send.

And so in the future if you just change the body, you haven't changed the signature at all, it might turn out to be a breaking change because whatever that real type, the hidden type, no longer implements send. So this is something to be on the watch for. Um The other thing I want to clarify with return position impl trait is that it is not the same as being generic over some R and then returning R. So F1 and F2 here are not the same.

Even though they might seem like they are the same. For F1, the caller chooses the return type. For F2, the implementation dictates the the return type. So in F2, whatever I put inside the body of the function is what dictates what the hidden type is. In the top one, the caller gets to say, "I want you to return a vec." Or "I want you to return a B-tree set." That top one, we do have some instances of in the standard library, but they're pretty rare.

Uh one of the instances you might have seen is the um the collect method on iterators, right? So that one is generic over its return type. So you can tell it produce the following type as long as that type implements from iterator. That's the requirement. And this this first kind of function is is kind of uncommon because it requires that the function has some way to generate R. Right? Because it's supposed to return an R.

So it needs to know how to make one. Um in the bottom one, there's no such requirement because the type is just dictated by the body of the function. So the body of the function obviously knows how to make one. Okay. And then we get into uh a topic that is especially weird around impl trait, which is lifetimes. And warning here be dragons. Um so the current rules that I'm about to tell you about return position impl trait are probably going to change in the next edition.

For the better, but just as a caveat. I will point out why that is as we go along. So uh impl trait does not capture lifetime parameters to a function call. So, if you have a thing like gen here on the right, the return type here, the opaque type, does not name tick A, and the hidden type also does not include tick A. And the reason being here I'm returning unit, so I'm not actually returning anything that includes tick A here.

I'm just returning some other type that is actually static. And so, I can write an fn foo that calls gen with some reference that's only valid within the same function, and the compiler will not complain. The compiler will say this is totally fine because the this type doesn't say anything about the type being constrained to a lifetime. And so, it's sort of implied that it doesn't capture anything. So, this compiles.

Um Now, it does capture type parameters. So, if you write that exact same thing, but it's now generic over T, and then it captures the it takes as argument the T, and it still only returns unit, so it still that doesn't actually use the T in any way in the hidden type, then now if you try to write fn foo returns impl size plus static, and you call it and you give it a reference, so the foo here is basically exactly the same, this will not compile because the compiler says this T leaks into the hidden type, even though there's no actual contents that leak, but the the type itself leaks into this thing, and the type T here includes a lifetime, specifically the lifetime of this borrow here, and that lifetime is not static, and therefore this does not compile.

Now, this is weird. You should think this is weird because it is. It captures type parameters, but not lifetime parameters. This is the part that's going to change. So, in particular, in the future, the compiler is going to capture both lifetime and type parameters just to make the thing consistent. Um things get really weird when you start playing around with lifetime parameters like this. And I'm telling you this because you will run into this trap, and it will be confusing, and I want you to have heard at least once why it's weird.

Um so, let's say that we change gen, so this time it no longer returns unit. It actually returns the T that's passed in as the argument. So, now the the hidden type that's returned truly does have a lifetime bound. It cannot live longer than the lifetime of this borrow, right? Because it includes a borrow of that length. Well, this doesn't compile because the compiler tells you the hidden type captures tick A. And there's no nothing that tells me there's nothing in the opaque type that says that it captures tick A.

And so, the compiler actually requires you to say something here that that opaque type has captured a lifetime. It's tied to some lifetime, but you haven't written that down. And the most obvious way to fix this problem is you write this, you write plus tick A at the end. That's what you will see a bunch of just examples in the wild to do this. Unfortunately, it is wrong. Um and the reason why it is wrong is because what this is really saying is that this function returns a type that implements size and lives at least as long as tick A.

But that is not actually what this is. This lives at most as long as tick A. Right? It can't the the tick A here has to outlive the hidden type because it's capturing that T. So, if the tick A got dropped earlier, the hidden type is no longer valid. So, putting the bound here is actually backwards. This is again one of the reasons why this is going to change in the next edition because the obvious fix is wrong. It turns out that the way that you're supposed to do this Actually, let to add to this, this often works and this is the reason why it's particularly confusing.

If you put this here, it will compile. And often times you can get away with doing the this throughout your program, it'll just be fine. But it is wrong and the way that it hits you is in super specific cases, like if you have multiple lifetimes, um or sometimes if you have weird variants in in types, then this suddenly doesn't work with very weird compiler errors. And it's because this is subtly wrong. Now, the correct way to do this, apparently, I learned this when making this presentation, is this.

Now, I don't want you to ever have to write this code. It's gross. But that is the way that you're supposed to do it. Because this, for some reason, tells the compiler the appropriate thing, which is this lives at most as long as ticket. Um so, this is sad. This is why it's going to change. And if you're confused, everyone is. Um so, one of the things that's been partially holding up the stabilization of async functions and traits is the discussion around these semantics.

Uh especially because there are also compiler bugs in this inference. So, sometimes you get it right and the compiler still doesn't accept your code. And that's unfortunate. Um there's a really good summary of this problem at this link. I'll put the slides up afterwards so you can find it. Um it's a really interesting read about this. Uh and you'll find that like even some of the seasoned people working on the Rust type system are confused by this and have only recently realized some of the weird implications.

And again, one of the reasons it's going to change in the next edition. Okay, so that's return position impl trait. So, now let's look at this one where impl trait is an argument position. Oh boy, we're in for a long journey here, right? Like if that was return position impl trait, surely this is also painful. But no, this is just sugar for T trait. This is actually just the same as that. And so, it means something completely different.

It means this function is generic over any type that implements this trait. So, this is not there exists some single type. This is for any type, for all types. It is not an existential. It is just a generic type parameter. Um and it's called A pit because it's a pit, I don't know. Um argument position impl trait. And it's just truly so much simpler, but also completely different. It was stabilized in the same version of Rust, and I think it was mostly just for sort of symmetry with having it in return position.

It feels like you should be able to put it in argument position as well. But it's not the same thing at all. Um it is a less syntax heavy um shorthand that that's easier to write. Like it is truly just easier to write this often than generics, um but it has no actual expressive power beyond normal generics. In fact, it's it's slightly worse. It's slightly weaker because you can't name generic types that are specified in this way.

So if you have here function one and function two, um and function one has a normal generic expression, and function two has the same expression just using impl trait, function one you can call and then specifically say, "I want your argument to be a U32." With the second one, you don't have anything you can put here cuz there aren't generic type parameters. You don't have a way to specify that this integer argument should be treated as a U32 in in the generic type parameters.

You you can put it after the literal, but let's ignore that for now. Um and this extends to the fact that even if F2 did have a generic type parameter in addition for some other argument, then now you're not able to name that argument either. So if you added an impl trait argument to F1, you would no longer be able to name D. And so this is something that's worth thinking about whenever you add impl trait arguments to a function is that it can actually be a backwards incompatible change because you can no longer call it and explicitly give generic argument types.

So a little bit of a subtlety there. Um and that that's all about argument position impl trait because that it's just a syntax for generics. So now we move into things that have not stabilized yet. These are still nightly only, although hopefully many of them are pretty close to stabilization, uh but this is more future-looking. Um so the first of these is impl trait in associated type position in traits. Um and I let's let's do the same exercise and go through why is this useful.

Um and you might recognize this pretty immediately, which is it's useful in almost exactly the same way as return position in full trait is when it's an associated type on a trait instead of being a return position in a function. So here, for example, I want to implement into iterator for odd uh and the into iterator trait requires that I name the type of the iterator that it produces, but the name of the iterator that I produce includes the name of a closure and so I cannot name it.

And so my option then is well, I can put box din iterator item equals. Sure, I can do that. But then it comes with all the same costs as we just talked about. And this is actually surprisingly common. So it comes up in things like into iterator. It also comes up a lot with futures. So you'll see this in things like the service trait from tower, which also has an associated type that is a future of the thing that you know the the request that's going to be handled.

Um And so you know, if you if you have Yeah, so this makes us happy, right? We can write in full iterator there and we get item equals U32 back. Great. Solves the problem. Now this just works. So that makes us happy, but it's nightly only, so it makes us sad. Um it's been on nightly for a while. Like this has been on nightly for I think 3 years or something. Um but part of it is because it's actually a pretty complicated feature to get right.

Um you'll see some of why later. So here's an example of the the tower service trait where you want to be able to just have an async block in this call method, um but async blocks are not nameable. So what do you put in as the associated type future here? Well, with in full future it's easy. Otherwise, you would have to do something like box din and then you you pay the prices of doing so. And yeah, this this is correct.

Like one is an odd number. It's fine. Um Great. So what about capturing? I talked about like how lifetimes are really complicated for return position impl trait. Um for associated uh associated type position impl trait, so uh I think it's like at pit or at pit or something. Uh these all just grow. It's as you'll see soon. Um so, these use the new capturing rules, so the ones that will be the case for return position impl trait in the future.

Um so, here it captures everything that's in scope. So, what that means is if you have an associated type like this, and this is also you see using uh generic associated types. So, if you have some generic types up here and you have some generic types down here, all of those are considered captured into here, and you don't need to name them. The opaque type implicitly captures all type arguments, all type parameters, and all lifetime parameters uh that are in scope wherever you define it.

Uh so, the rules are easier to reason about. The implications, not always so, but but even so, the the rules are at least the same for lifetimes and types. Um the real question, and one of the reasons why this hasn't stabilized yet, is because the there's some ambiguity here about which hidden type are we talking about? With return position impl trait, this is straightforward. There's only one definition of the function, and it only has one hidden type.

But, once you have an associated type, you can name that associated type in multiple places, right? So, here the into iter um method here returns a self into iter, so it should get to dictate the type. But, the eat method also takes a self into iter. And so, here if you look at it, like there's an implicit assumption here that the um the into iter that uh we get back from a vector is the same as the type of the argument here.

And so, this the body of this function implies that into it the self into iter here should be vec into iter. But, the argue the body of this function implies the self into iter should be this filter thing. Which definition wins? Which one does the compiler use? It's not entirely clear. And you might go, well, it should just reject this code. Okay, rejected with what error message? It kind of has to pick one. And the sometimes it might even be that it could compile.

This is not an example of this, but sometimes types are like fungible. Like the compiler can infer different types. And so maybe it can like infer by looking at this hidden type and that hidden type that they're actually the same if I make like this a U32 and that a bool or something. So the compiler needs to be able to know how to reconcile multiple definition locations for a particular existential type like this. And that's hard.

Um Now, the other reason why this is tricky is for ID completion. Um so ID completion is already pretty hard, but let's imagine that we have this code from before, but I haven't written eat yet. So I'm in the body of eat and I write iter. What completions do I get? So iter here is of type self into iter, but we don't know what type that really is. So and the only way for the compiler to figure that out is by parsing the body of this function and running type checking on it.

And you don't really want your IDE to have to implement full type checking, then it gets really really slow. And we don't like slow IDEs. Um and so this is one of those like maybe it should only give you completions for the opaque type and not the hidden type, but then what do you do about auto traits which as I talked about peek through? So can like if you're if you're um editor if you like hover over iter, does it tell you that that type implement send?

How? Right? So so there are some open questions here and again, this is one of the reasons why this has not been stabilized yet. Um it is hopefully though coming soon. Like many of these questions were we're getting towards resolution on. Um there's some good follow-up links here that you can watch if you really want this feature to land. Um so I'll you can find those later if you're curious. Yeah. What is the solution for that?

Yes. Yep. Okay. Um okay, so then we have type alias impl trait. So, this notice is not part of a trait. I can answer in a little bit more detail later. Um so, type alias impl trait happens not in the context of a trait. So, it's very similar to what we saw before. But, the question is like really why constrain this only to traits? Why should only traits get to use this like type name equals and then impl trait? Why not let it everywhere?

Also also known as existential types. So, this is type type alias impl trait and it lets you give names to arbitrary types. And this gets really handy when you combine it again with iterators or futures. So, for example, in the standard library, there is a method called standard future ready. Uh it is a function that you give a value to and it gives you back a future that will yield that value immediately. So, if you try to await it, you immediately get that value back.

Um and one of the reasons why it is in the standard library not as an async function, but as a function that returns a type that implements future is so that you can name that type. That type is called ready in the standard library. And you want the ability to name this function so that, for example, you can store it in a struct. You can't name it, it's kind of hard to store it in a struct because you need to when you write the struct definition, you need to write something to the right of the colon.

And so, it's actually really useful to be able to name these types with an actual name you can write in source code and not just with like an impl that's determined by the compiler and not written anywhere because it's needed for struct definitions. Um so, what type alias impl trait lets you do is you can define a type basically anywhere in your file, anywhere in your module. So, in this case, type ready takes a T is impl future output equals T.

And then you can define a function that you dictate that it returns that type alias. And then now this becomes your opaque type and this becomes your hidden type. And so now from now on in your program, whenever you refer to the type ready, what you're really referring to is that hidden type from the contents of this function. And so this lets you start to name um these uh these hidden types, these types that are hard or impossible to name throughout your code base.

Um And here too, what about lifetimes, right? We always have to think about lifetimes. Uh and here this also follows the new rules, which is capture everything that's in scope. But because these are written at the module level, there is nothing in scope except for any generic arguments to the type alias. And so this actually becomes a way to opt out of generic type parameters. So imagine that you have a an associated type in a trait, and you don't want it to capture some generic type of that impl block.

You could go through one of these and just not name it out here, and that way it would not appear it would not be a part of the the the hidden type. I can give an example of that later on if you want. Um but basically this sort of completes the picture, right? Of you can now use this type alias construct both in traits and not in traits. Um unfortunately though, there's even more Yeah, go ahead. Hey, can you go back one slide?

Yeah. Isn't this kind of opposite of the general Rust mentality of being explicit? Now you kind of explicitly have to opt out. Uh explicitly have to opt out in what sense? Oh, of captures. Yeah. Yeah, so this rule around captures is unfortunate. Um but it is even worse to have to name all the captures because we don't have a good syntax for doing so, right? We would need to invent new syntax to say this only captures these things.

And this is where people write just plus tick A and assume that it is that syntax, but it means the wrong thing. And so we would need to invent new syntax and point people to the other syntax. Right. And we can't change the old syntax. We can't change the old syntax cuz the old syntax also has a meaning that is just different from the meaning that we want in this context. And so, what you often want is this capture semantics that that's generally what you expect when you write the code.

And I agree it would be nice if it could be explicit, but there's a there's a trade-off space here, unfortunately. Um and and like yeah. Uh don't you have like this kind of explicit captures in that C++ lambda or do you? Um So, that's slightly different. So, that's about capturing um values and variables from the surrounding scope. It's not about capturing type parameters and lifetime parameters. Right. So, so there has been proposals about adding additional syntax for saying, "I want to capture this type in the in the opaque type." But but then we get back to the same problem of people still just write plus tick A and it does the wrong thing.

Yeah. Um I I I do, too. Until I realized that it was wrong. Um Okay. So, so I mentioned that there's ambiguity in resolving the hidden type when you have associated type position in the trait. Um this gets even worse because who can where can the defining use be when you just have a module level type alias? Does it have to be in the same file? In the same module? In the same crate? Like for example, if you have a function in your crate that returns some type that I can't name.

Can I declare a type alias in my crate that calls your function and assigns it into that type? And now I have a name for a type you didn't want me to name. It turns out this is actually a problem. Like there are some crates that rely on the the secrecy or the unnameability of a type to give safety guarantees. And so, we can't allow that. Uh and so, there's a there's an interesting trade-off space here of like where are you allowed to have the defining use the the hidden type for an opaque type?

Um and I I can dig into this um further as well afterwards if you if you want to go deep down the rabbit hole with me. Um Okay. Next one, moving on. Uh so, this is return position impl trait in traits or I ar pit it. Uh So, the motivation here is that we we have this, right? We're associated type position impl trait, uh and that's nice, but it gets really bad when you have more functions. Like, imagine that this trait had like 10 functions that all wanted to return futures.

Then, that means you also now need 10 associated types. That gets really annoying to type out, and you get tired of it, right? Um this particularly happens when you have things like async functions because all of them return impl future. And then, if we assume that we don't have async functions in traits yet, then that means 10 associated types. That makes us really sad. Um so, how can we do better? Well, we can have just return impl trait, right?

Return position impl trait in traits makes us happy. Uh unfortunately, it's nightly only so it makes us sad. Uh I you might see a pattern here. Um this has only been on nightly for about a year, uh and only recently have we gotten to the point where it it's kind of close to stabilization. Um in fact, it was proposed to be stabilized very recently, and then there's been some debate about some of the things that I'm about to talk about.

Um but, this would be really nice because now you don't need an associated type for every function because it's sort of this like implied somehow um that there's this hidden type in here. Of course, uh the real question here is where do the wheres go? So, if I call a function in a trait that has an associated type, then I can say, you know, where I want to take something that implements this trait, where this associated type implements, for example, send.

But, here I just proposed that we get rid of the associated type. So, what does the where bound? So, here I have a function that is generic over it or iterator or whatever you want to call it, right? And it wants to require that the type that's returned by the iter function implements clone. But we don't have anything we can put here. There's no name that goes there because there's no associated type anymore. So how can I have this bound?

And this gets particularly weird when you have asynchronous functions or futures because you really often want to say where the return type is send so that you can use it with async executors that are multi-threaded like Tokyo, right? If you can't say that I require that it's send, what do you do? Um now you you can change this in the trait definition. So I could write here impl iterator plus send, right? But then I'm requiring that every implementer of the trait guarantees that their implementation is send or in this case is clone.

And I might not want that either. I don't actually want everyone to implement this trait to also give back cloneable iterators. I want to be able to just say when I now call it, I want it to be clone. Um not to mention if you added on the trait, then now it becomes a backwards incompatible change to remove it later when we do have a name uh way to name these associated types. Because if the trait guarantees that the return type is always clone, then removing that guarantee and now saying that people have to specify themselves as callers is backwards incompatible, requires them the callers to change their code.

Um the really bleeding edge here is uh RTN, return type notation, which basically tries to solve this naming problem. Uh and the currently proposed syntax is this. So you can say I'm generic over it where iter parentheses colon clone. So this is really saying the return type of the iter method is clone. Is what that syntax is for. They haven't figured out what this should look like in where bounds yet, but at least you can now it's at least possible to do.

Um this is not necessarily the final syntax, um but we need something like this uh to solve this problem. Now we don't want to just a just a second. Uh we don't want to block return position impl trait in traits on getting this because this is fairly new. It's fairly far from stabilization. Um but that still means that we have to figure out what to do about the send bound problem because it will hit the ecosystem. Uh and without this, it's a little bit unclear uh what we stabilize.

Maybe we maybe there's some lint that should go in there that tells people, "Hey, you put an impl trait in here, but just so you're aware, there's no way for people to set bounds on it. But if we put a lint everywhere, then everyone gets sad cuz they get warnings everywhere. So, it's it's a tricky uh trade-off space. Um it's also tricky to or these are tricky because again, moving from having the bounds in here in the trait definition to requiring bounds here is a breaking change.

And so, if we tell people to go use return position impl trait in traits, um then they might then put themselves in a sort of backwards incompatibility trap for the future. Yes. Yeah, but this also gets even more complicated if the iter function is generic, right? Uh if the iter function is generic, uh yes. So, so So, some of the problems here is like I want to say that there's a generic type parameter to iter, and when iter is called with that generic type, then it should be cloned.

Yes. So, so that Hence, this even the syntax here isn't quite figured out because you need something that's expressive enough and actually can support nesting in a sense. Um and again, lifetime captures because of course we got to capture lifetimes. Um the rule for return position impl trait in traits um is not the same as return position impl trait. Uh and that is because return position impl trait in traits uses the thing that return position impl trait will change to.

So, everything is use Everything that's nightly only is using the the new proposed capture rules, and only the thing that was stabilized, so return position impl trait, uses the old and broken rules. So, this one will also, like the other ones, capture all associated um all sort of incoming uh type parameters and all incoming lifetime parameters. That means that it will also capture the reference to self. So, if you write a thing like this in a trait, this opaque type also captures self.

Even if it doesn't move self into the thing that you return, the type will considered will be considered to uh be bounded by the lifetime of self. Um and the way that you would sort of stipulate that that's not the case is that you would use an associated type instead that did not uh capture the lifetime that we got in here. So, by not naming it an associated type, you are also not capturing it. Because remember, for associated types, you or for type aliases, you only capture the things that are explicitly named.

Okay. Uh this is also coming soon. Please, please, please, I really want it to land soon. Um here are some more discussions you can follow on this. Uh this one we really want to land soon because it is the path to asynchronous functions and traits, which is what everyone really wants. Because if you think about it, asynchronous functions and traits are really just sugar for impl trait and return position impl traits, right?

Like this thing at the top is the same as this thing in the bottom just with more syntax. They're actually the same thing. Um and so, the path to getting here is to have this. They are in fact so much the same that it the the way it's currently looking, you'll be able to write a trait definition using impl futures like the this syntax, and you will be able to implement that trait using this syntax, and vice versa. Because they are truly that interchangeable.

It really is just sugar. Um and it is actually pretty reasonable to have your trait definition right like return position impl trait um so that you can add additional bounds for example just lets you write a little bit more of an expressive uh type signature um and then have all your callers implemented using async functions because it's easy and then you get auto trait inference and all of that. Um, the the same question remains though which is how do you set bounds on the return type?

Right? So this this return type notation problem still exists for async functions. How can I say I want an implementation of you know the service trait where the thing that I get back from call is send? We don't know yet. Okay, that's a run through impl trait as far as we know it today. Uh I will add before I take questions, I do have some bonus slides for other things other places where you can put impl trait where they also don't mean the same thing uh but I'm not going to dig into them unless you want me to.

I'll take questions on this stuff first and then we can go down the rabbit hole together if you want. Thank you. Okay, hopefully there are questions because I had a lot of questions making this. Yes. Um, so if uh async fn trait stabilizes somewhat soon. Yeah. As a as myself as a maintainer of a library that has a lot of async fn traits, I'm going to get so many feature requests from people like hey can you please use this?

Yeah. But having seen your talk now, I'm like there's so many gotchas around send bounds and things that don't work in what you don't expect. So I'm almost like no. Yeah, so This is it's a What like what's your advice? It's a it's a perfectly reasonable response actually. Um, so so one of the things that's being debated now around the stabilization of return position impl trait in traits and async just in traits is um how do we communicate exactly what we're stabilizing?

So, if they stabilize it the way that things are today, so without the ability to name things like return bounds, um we can't really put out a And I say we, if that they uh they can't write up a blog post. We as Rustaceans can't put out a blog post that says async just functions and traits are now stable because in some sense it's not true. Right? Because, for example, Tide couldn't adopt it today. Axum couldn't adopt it today because it would be too hard to deal with these send bounds.

If you did, you'd have to cut another backwards incompatible release down the line. Um and that may be a cost you're willing to pay, but it might also not be. And so, the question then becomes, if we don't say it's stable, what do we say? Because there are a bunch of valid uses for return position impl trait and traits and async just functions and traits where this would be useful today, even without the ability to name return bounds.

So, for example, we've heard a lot from uh people in the embedded space, for example, who don't have multi-threaded executors. They just want this expressive power, and they don't need to add any additional bounds. This alone would make their code a lot easier to write. Imagine this if you're in um a no std environment, for example. So, you cannot use box. Okay, so you can't use the box din trick. So, what do you do?

And it's actually really hard and annoying, and to some extent, many of them are just forced to use nightly in order to be able to use this feature because it hasn't stabilized. Um and so, there is a pretty strong argument for stabilizing some of this. And then the question becomes, how do you how do you make sure people don't fall into the sort of trap of adopting it without realizing the consequences? Um and how do you communicate it in a way where people understand if, for example, Tide or Axum say, "No, not yet." Yeah.

So, um a question about your aspect example, the one with the eat function. Aspects, aspect appear. This one? This one? Yeah. So, you say which definition wins? Yeah. But, if I understood you correctly, then the the impel in the type into either, that was that's just an existential type. Yeah. And the impel the sort of the implicit impel you have in the eat Yeah. then that would be a for all type. Um no, so so notice that this does not say impel iterator.

So, this is not using impel trait argument. This is naming the type. So, this is not impel trait an argument position. It is not a generic type parameter to eat. It is naming a concrete type. It is specifically naming the opaque type here. So, so this does not make eat generic. It just makes it say that it must take an argument of exactly the type that into it or returns. Okay, so that's how you would So, they're not independent. existential in in in argument position.

Uh in some sense, yes. This is an existential in argument position, yeah. Yeah. Okay. Yeah. This plus static thing, the subtle bug The plus static thing, yeah. Uh plus static is over here. Well, the subtle bug that you mentioned that it doesn't actually do what we think it does. Oh, you mean the the like plus A thing? Yeah. Will that Will we Will the compiler complain or will it say hello? Uh so so in the in the next edition, again, this is something where the the decision hasn't been made yet for exactly what will happen.

So, so take this with a grain of salt. But, the hope is that in the next edition, I don't know whether they're going to make this a compiler error, but it will probably become a warning because this is very rarely what you want. Right? It almost certainly does not mean what you intended it to mean. And in fact, in the next edition, the correct thing to do will be to remove the bound. Right? Because in the next edition, it will automatically capture all lifetime arguments.

Which is sort of what you would expect. Like this gets back to the point of like, shouldn't this be explicit? But very often what you expect when you write this is that it does capture the incoming lifetime. Um so, uh what that sort of looks like is uh me I can't find it. It doesn't really matter. Uh is that the compiler would probably warn if you do this. Uh and it will just compile fine if you remove it. Whereas currently, if you write this, it just won't compile.

And so that forces people into this wrong pattern. Yeah. If the type alias can we have one of ready um Type alias in full trait. So, this one. Yes. Um So, this uh so this type ready, for example, Yeah. is generic. It could be used in by two completely separate functions in the program right now, right? Would they have to return the exact same type? Like is there still a Okay. So, so the question here is let's imagine that I had a ready one and ready two.

And ready one returned uh ready U32 and ready two returned a ready bool. Is it a requirement that the hidden types for those two are the same? Um I don't know. I think so. So, I I think the the impl future here does not apply to the It doesn't get to vary by generic. So, so I think there has to be one type ready, one single type um a single hidden type and that single hidden type is generic. At least that's my understanding.

So you would not get to write Yeah, so you would not you would not get to diverge to the hidden type based on the type parameter. I I don't think you will at least. Yeah. Could you just repeat what that plus tick A actually means? Yes. Yes, absolutely. Okay, so plus tick A here. So the question is what does the plus tick A actually mean if it doesn't mean what I expect it to mean. What this means is that the type that you return uh lives longer than tick A.

It outlives tick A. The type. The return type outlives tick A. But that's not true. It must live shorter than tick A. Right? Because if it captures tick A, that means that if tick A no longer exists, it doesn't get to exist. Yeah. So how does this compile if if you're returning something that lives longer? Ah, it's so the reason it compiles today, it's a great question. The reason it compiles today is because of covariance.

Uh and I'm not going to try to explain variance on the fly here, but but basically the reason we get away with it today is because it doesn't matter whether it's less than or equal or more than or equal because they're equal. So currently today it works because of that. Um but but in practice it gets really weird the moment you have multiple lifetimes for example or if you have um contravariance in the arguments because then suddenly this no longer it gets really weird.

Yes. Uh what if you would replace the tick A's by tick static? Uh okay, so if you return this by tick if you put stick tick static here, uh this is another uh weird thing that often works but is wrong. Right? So, uh I put an example of this here uh that I hope no one would notice and no one did, so that's great. Um which is here, I did put plus tick static, right? Which is I just told you it's wrong to put plus tick lifetimes.

Um and and it is wrong here, but it will work. Um and so here, this works because the return type here uh capture sorry, this one, which does compile. Um it returns uh unit. It does not capture tick A. So, the type that's returned, the compiler knows is static. Right? The compiler knows that the hidden type here is static because currently, if you capture something, you have to say. And it doesn't say, so therefore it doesn't capture anything, so therefore it's static.

So, the compiler here knows that this type is static. Um but when I here say implicitize plus tick static, again it works because the lifetimes are equal. But again, this is promising that it lives for at least as long as static, which happens to be true because longer than static is static. Yes. But this expression is of how long something lives with the lifetime. Yeah. You only can ever say something live longer than another lifetime, right?

Don't we lack this possibility to say the opposite? Think is So, think of it more as what this currently says is tick B is longer than tick A. Like tick B colon tick A, and it should say tick A colon tick B. Yes. So, we have the when you have actual where bounds, you can express both directions. But but with this syntax, you can't. Uh if you could name your own return type, then you could. So, if you had return type notation, you could say where and then the the the way to name your own return type and then the lifetime could go in either order, but we don't have that.

Uh yeah. Um so if gen the function gen here is Yeah. is defined in one crate Yeah. uh food and so on Yes. and crate gen upgrades to edition Yep. that would be a breaking change. Um no. So, um wait, you're saying if gen upgrades or if foo upgrades? If gen upgrades, then when gen upgrades to edition 2024, it has to make sure that the semantics that it converts to match its old semantics. But this is always true across edition boundaries, right?

Like when the whole point of edition boundaries is that the semantics and of syntax might change. And so that this is an example of you need to be aware of how they changed so that in your change you know don't make a breaking change. Uh so yes, you have to be careful, but it is not it's not if the if you kept the syntax the same, then yes, it would be a breaking change. So you would have to change it. Yeah. Yeah, I have another question related to the type and yes, stuff that you mentioned previously.

Uh and it's related to the question that you This one? Yes, exactly. Yeah. So what happens if you define another function here where the intro seconds Yeah. which also returns a T, but the async block inside returns say I don't know, it waits 2 seconds the output future of the same type. Uh what's your answer to that question before was that this is not going to work because the side type is not the same. Yeah. Or did I get that wrong?

Okay, so so the question is um if you have if you let's say have two functions here, uh one called what one that's like this and one that's ready in 2 seconds. Um and so that one will have a an async move and then a you you a timer sleep and then returns to the T. Um will that be allowed? And no, it will not be allowed. So, for the same reason as before, the what you would end up with is you the the hidden method the hidden type here, the async block, would be two different hidden types.

Even though they're both async blocks, two different async blocks, even if they have the same contents, are different types. I'm not sure I understand the question. Yeah, so so if if I have crate A and crate B and crate A defines an existential type like this, it has to also be crate A that has the hidden type. Crate A crate B that depends on crate A does not get to choose the hidden type for crate A. It's just disallowed.

The the the defining use So, this gets back to like what is the defining use, the thing that defines the hidden type. The defining use for an opaque type has to be within the same crate. And in fact, there's even stricter rules. Like I think I think where they're currently landing is that it has to be in the same module. Um but I'm not the they're still figuring out the rules here. So, this gets back to your earlier question, Simon, about like how do we solve this problem?

Like how do we figure out where the defining use is? One of the proposals is to have explicit syntax to say, this is the defining use. So, if you have multiple functions that all say return the same uh type alias, one of them you mark with like square bracket defines. And that will be the defining use for that named um type alias. But again, this is just one of the proposals and it it's not clear that it will be the final proposal.

Uh but that's that's one one the things that maybe we should be explicit here, rather than rely on some heuristic. Uh yes. Because it means all the like when you're typing your code like and you say you type pretty T here then like until you make the function your code won't compile, right? Because you're not defining the function. Right, yeah. So, if you define the the opaque type here, but you don't have a defining use for it yet, you haven't written a function that returns that type.

Um I don't know that it won't compile. That depends on whether you use it somewhere else. Uh but it like if you if you had a file that only contained in this line and nothing else, I it would probably complain there's no defining use. Yeah, probably just a warning because it's not used then. Yeah, I don't know if it's unused whether it would then be downgraded to a warning or whether they would just error. Uh I'm not sure.

Uh yes. And so, um isn't it inconsistent then with the associated types? Like for this type structure there's only allowed to be one in type for the type, but but you mentioned earlier with the associated types actually allowing to have multiple Ah. So, you have to filter that and like Um okay. So, so this gets back to this thing um where there are two defining uses for the opaque type. Um I'm not saying that it's allowed here, but not allowed in type alias simple trait.

I'm saying that we don't know what to do here. And it's the same thing for um for type alias simple trait, where we don't if there are multiple defining uses, we don't know which one to pick. So, it's not as though this code is going to be accepted. Oh, I see. This code will probably not compile because the two types are not reconcilable. But one of the questions, right, is imagine that you have a uh let's say you define a type alias equals impl copy.

So, basically it can be almost like any primitive type. So, you have a type alias that's impl copy. You have two functions. Um one of them returns that hidden type and just has a literal one in it. And another function has a literal one and then U32 on it. Then now the picking U32 as the hidden type is actually legal because you through inference on the first type, you can infer that that literal one is actually U32.

Uh and so it's not necessarily the case that having multiple defining locations is conflicting. Um but whether we even allow that is unclear. It could be that we should just disallow it cuz it's going to be confusing. Um and then again, there there's no final decision on this yet. I saw an arm up there earlier. Yes. Um that input type definition in the function body that takes the input type Yeah. as an argument, how could that work?

Um so how how can it work that this takes an associated type? The So as far as Since that if we forget about the into iter, Okay, forget about into iter, yes. Then you have a function that takes an existential type that will have a very definite concrete type that will be determined by the own body of the function itself. Yeah, so it's weird, right? Like can only ever be returned by itself. Yes. So it it's a it's a very weird construct.

So having existential in argument position is a very weird construct, right? So if we ignore the into iter here and we say that this takes self into iter, but it does not have impl trait in argument positions. It's not generic. It just has an associated type of this, then that means that the defining use for this associated type is now the body of this function. But that's okay because that's the same as here. The associate the the hidden type is dictated by the body of the function.

It's just that in one, it's just whatever the final expression is, and in the other, it's just whichever expression makes use of the type. But in the second case, you will have a function that can only ever be called with an argument that can only get from itself. Uh in the second case, you have a function that can only take an argument that can be generated by itself. Um Uh You mean the input, right? Like the into iter itself.

Yes. So so it I agree it would be weird because um this only the contents of this function knows what that type is. And everyone else only knows that it has to that it is some specific type that implements iterator. So no one would be able to call it. That's true. However, it is fine in the context of other associated methods because they are allowed to may to name the same type. And I know it's too much best for my compiler, but that dictates that into iter is required which one is the real type for these two functions.

I I agree actually. In this case, I think it's somewhat obvious that into iter should be the defining use. Um however, I don't I think this is a somewhat more obvious case. Uh and it's probably better to not have sophisticated heuristics here and instead just say if you have multiple defining uses, it's an error. Or you need an explicit definition. Uh and and I think they're probably going to go down one of those two paths.

But like if you have strong opinions on the matter, like jump into this thread if you as long as you feel like you also have well-reasoned opinions. It is a very very long thread. Uh yes. Um a question about Rust syntax. Am I allowed to return this function name? Like even if it's not like can I capture it in a variable or return the variable? Ah. Okay, so let me let me let me go to one of my bonus slides. Um Okay, so this this is an example of something that uh is also um ha there's an RFC for this that I I has a partial implementation in the standard library nightly, um which is you can write this.

Um which lets you use impl trait for the type of a variable. Now, the motivating use for this in the RFC is actually not let statements. It's for statics. Uh so imagine that you want to have a static and the only thing you want to promise about the static to the rest of the program is that it implements a particular trait. And you want to hide the the the actual type of that static. And you only like for example, you might have a pub static in your crate in your library.

So you want to expose that static to outside callers, which might be a terrible idea, but you you want to. But the only thing you want to promise is that it implements a particular trait. Well, then you would want uh impl trait in like in um in left position, if you will. So so there is a use case for this. Uh but it's not something that's currently on nightly. And if say the compiler couldn't look like peek through the hidden bit to see like these auto traits, would this still be an issue?

Like would we if we could like only guarantee if only can it can't see through kind of it doesn't know it has a clone. Um well well so the in general the the auto traits bleeding through is just always going to be true cuz it's too painful if you're not. But remember that clone is not an auto trait. There are very very few auto traits. It's send, sync, unpin, uh unwind safe, and coerce unsized and sized. Grant my mind saying copy.

Oh, I don't even want to. But yes, yes. So so the the relatively few and they're very specific. They never have any methods. Uh that's one of the requirements is that they're a marker trait. So it it shouldn't cause you too much of a trouble, but but it is a backwards compatibility hazard. Yeah. Can you go back to the previous slides? What we were before. Where was before? Uh that was oh Uh, was it uh this one? Yes. What would the implication be that if instead of the associated type into either, now it's naming the same type for both of them.

But what if it basically just copy-pasted the impl either into the two positions of use? Okay, so if you put impl either here and impl either here. Would that be a solution to this problem if the Um, no, it Well, so that has different semantic meanings. So, if you if you didn't have the associated type and instead you put impl iterator here and impl iterator here, then there's no ambiguity anymore, but it means something completely different.

It means that into either is fine, like it just it does what it does. Um, there's no associated type to name anymore. Um, and eat is now a generic function that takes any iterator. Which is not the same semantics as are implied here. The implied meaning here is that these two are the same type. Correct, but is that actually desirable if this is the outcome? No, this is probably going to be an error. And and I don't think you want to convert it into the other meaning because that meaning is sufficiently different.

Yeah. Yeah, so I mean it's an error here. Yeah. But it's definitely not an error that that you would have sort of something that returns an existential type and another function that takes that exact instance of the existential type. Yes. Yes, so so I want to I want to write this probably with a better implementation of eat, this implementation. Yes. So so there are some implementations here, like the body of eat could be written in such a way that it doesn't dictate a defining use.

That like for example, if the implementation of eat only ever makes use of the fact that it is an impl iterator and nothing else, there's no ambiguity here. So this this pattern is not inherently problematic. Yes. Uh, you have a desk question. I'll you I'll get you later. Uh you mentioned that there are also some potential problems with performing IDE support. Yeah. Do you know if there's like any current thoughts on how that might be resolved or is that just going to be something we have to live with?

We lost the projector. That's fine. Um so for IDE support, the the way in which this gets weird is mostly around performance. Um it's not so much Well, there are two parts to it. One is the IDE does not currently do checking of the types for bodies. And so it would need to start doing that. And that's obviously a major lift. Uh but also even if they magically just had this feature, it would be super slow cuz it would need to do type checking in order to give you completion.

Um I think in practice the the direction this is going to go is that when you have when you have something like this, um you're probably just going to get completions from the opaque type and not from the defining use. And it will just tell you this is an existential type. There might be auto traits implemented for this. That that this is just going to be something where your IDE just cannot give you good or cannot give you exhaustive completions.

Yeah. Uh so you say this is a return uh decision equal trait thing. That that always has to use the auto traits. Is this it? Uh No, this one. Uh you're talking about that one. So it doesn't always have to say like it implicitly has to be Yeah, so so the rule for auto traits is that if you have a return position impl trait um that uh that propagates a hidden type, um it will also propagate auto traits for that hidden type, even though they're not named in the opaque type.

So, basically this this gen here, it says impl sized, but it actually means or the compiler gets to know that it means impl sized plus pin plus co- I'm wondering if we ever get after Do you think we'll ever get custom like self-defined auto traits, and how would that work with this? Okay, so there um there is an RFC for self-defined uh auto traits. It is dependent, I think, on the RFC for um self-defined marker traits, which are also a big question.

Um and I think it also depends on the RFC for uh negative implementations, which they kind of want to allow for marker traits, but not for any other. There's a lot of stuff here, um but the um the the basic answer to your question is that I I think if we get self-defined auto traits, they will have to interact with this. Um now Also, do you do you think that it would also propagate your self-defined I I think it has to propagate all auto traits, um because otherwise it's too annoying to work with impl trait.

Because the auto traits you never think to name them. Uh and if you name them your signatures can become really verbose, um and you might not even know all of the traits that you should propagate the moment you have self-defined ones. Whereas here the compiler gets to just infer them because they are auto traits. Too slow. Because like if if you have async bang, yeah, and that is sugar for impl future, how do you name the auto traits in async bang?

You can you kind of There you want the compiler to say figure it out. Uh yeah, so async defend is a good example where uh if your future if your the body of your async defend is send, then you don't have a place to put the fact that it is send in the signature because a signature is about the type that they sync blocker yields and not about the type of the block itself. You could get around that by turning it into a return position of a trait, but But yes, the the system this is how it has some kind of implications that we might never get self-defined auto traits.

Um I don't think it necessarily implies that we'll never get self-defined auto traits, but it is uh it might complicate them. I know that I've been talking for a while. Are we Can I take another question or two or You you have one. One more question. Was it one? One more. Someone who hasn't asked a question yet. Mm I don't know if you count. Yes. Will you be signing books for beers? Uh I will happily sign books with or without beers.

With beers is better. It feels like a very real question, though. Uh yes, but either of you. You can fight it out. Okay. I'll answer yours afterwards over beer or something. Yes, so I'm actually curious as to when you signed my book you said it was number 55. So is this like you sold 55 copies in the world and and if it went to the Danes do I and I meet that guy or how could you Okay, so the Okay, so Okay, so I I don't know whether if this counts as freebie question as well, but because of books But basically so I I number all my signatures.

I don't number all the sold books. Uh so you have the 55th signature on a book. I'll take your question then. So just to get back to the subject Back to subject, I like it. Even when I write Rust code I write functions that return input iterator and I have to give it the old plus tick for the underscore Mhm. Is that also wrong? Okay, so when the question is I currently write return position and will trade and I often have to write plus take a or plus take underscore or whatever.

Is that wrong? And the answer is yes. Now it is like as I've said before it's it's wrong with an asterisk which is it's wrong in the sense that it still often works. So if your code works fine, you don't have to go change all of it, right? Because it's it's okay. It's just it doesn't mean what you think it means, but if the compiler still accepts your code, then it's not wrong. It doesn't have like undefined behavior in it or something.

It just means that your interface is more constrained that you intended it to be. Yeah, but aren't we all the all of us that have to write either functions like you get a reference to some container Yeah. Any every function that takes something like a reference to self and returns the return position and will trade where you have to put plus you know, take underscore or whatever. Those bounds are all technically incorrect which again is why the rules are going to change for the next edition because it's insane, right?

But if you want the correct way to do it if I can find it here then you can just write that. So it's fine. The the construct exists such that you can do it, but I also don't think that you should go and rewrite all your code to do this in part because this is going to get better and in part because it works today. It's just your API is not quite the same that you thought you promised. We will end it there and I'll take more questions after.

Thank you very much.