Languages WG 26/09/17: Discussions on callbacks and extra state transcript

Good. Okay. >> All right. Cool. Um, events. I think this is what we want to look at. [sighs] The tools and interface is another one of those dark corners that >> maybe I can also try if there are some if I can turn on some AI features that maybe make a transcript. Not sure if that works here. >> Let me check. >> Okay. So, so um the MPIT event register call back also takes extra state but no release. But um attributes do actually. >> Well, yeah, they have a delete call back. >> Yeah. >> Right. >> Yes.

And I think this is where it is really required for the attributes because uh like all the >> Yeah. Right. All the use cases for attributes that I was thinking about mean that I have some state which comes from a higher level language library that I attach to like I could have it handled by my wrapping interface but it would be a lot cleaner if I could attach it actually to the lifetime of of the of the object that this is attached as an attribute.

So we definitely need uh the delete function there. Well, it makes sense because you're providing your own memory for that attribute, right? Um, >> yes. No, you're you're not like you're passing just a I think you're just passing a value that gets stored, right? So uh it's like you you it takes a void pointer but you would like to that attribute you like or no do you get a pointer like I [clears throat] don't even know what you can store in an attribute I never used them because it's because I never yeah I wanted to >> reason about yeah >> uh yes attributes are type typically which will be appointed to a structure that contains further information.

Okay. So, what you do is you Yeah. Right. It's it's a it's a multi-step process. So, what you do is you first request a key for an attribute. So, you reserve a slot for it and then you can actually set it to some pointer that you created beforehand, >> right? But then you need to free the data that this pointer points to at some point. And this has to happen in the delete function. >> Correct? So, so the delete function is meant for the user provided attribute, not the extra state that goes into the >> let me yeah let me let you scroll down a bit to where these are actually define and how the interfaces look right.

We have create key value which is which just gives you the key value back and there you register the actual functions. Right. >> Right. Right. And then I think if you scroll down a bit more then you actually have uh the function which >> which does no this is free set attribute right and this is the one where you >> where you pass in the attribute value which is this void pointer and this one >> right but we have no we have no data type so MPI can't copy the value right it's only it's only a pointer that um >> right it's only a pointer and this is what you have to do with the copy call back true >> right >> so for attributes I think for attributes it all makes sense >> right and there is no extra state here it's it's uh it's uh it only gets provided and and what is what is the interface of the delete and uh and copy functions again those should get this value right um >> get They get um let's see.

Yeah, the the the key val >> Oh, but you also have extra state. >> You get extra state and you get the two pointers for the for the attribute value. >> Yes. Where does this extra state come from? >> Oh, that comes from here, right? You >> Oh, right. So, what do you do with that extra state there? Uh >> um >> no. Oh yeah, maybe how how is it? So you register some extra state which might be so may maybe we should play this through with an example like like you have some like the prime use case would be that you have something like logically you allocate something on the heap and you want to attach it. you as a user allocated and you want to attach it to a communicator and have it freed uh when the communicator gets freed.

Right. >> Right. >> Okay. So what do you do? You first do the allocation in user space. Take the pointer pass it to the pass it as the as the value. So you can get it get it so you can query it back from the attribute and yeah and I think the extra state is is for example if you want to capture something else like for example if you you've been like this is what you would do like if you imagine this from a C++ site and you would pass in a a lambda that captures some other variables for deletion and copying and whatever.

This is where the extra state comes into play, >> right? It's just weird that we only have that on attributes, right? Um >> yeah, you would need it everywhere where you have a callback. This is what Jeff was arguing about. >> Right. Right. Right. Right. But I'm not sure what they use like the we didn't design this I'm sure we didn't design this with third party languages in mind where you would have where you extra state right >> so rationale for >> I see >> the extra state in C >> where everything is maloc and free >> yeah it makes sense like for example if you have uh if if you want to return something to like if you have a more complex text allocator which maintains a free list somewhere where you want to return it to when uh the communicator gets freed.

Then you would have to uh you would have to somehow capture not only the like data that you allocated but also the context where you want to return it or you could imagine it from a C++ point of view. uh if you have like [snorts] like even even with plain C++ with just new and new and delete you wouldn't need the extra state but you would need it if you would have a custom allocator that you construct at some point in which you want to like wire through >> right so it makes sense uh if you have some more like more complex stuff going on like for example yeah you could imagine that you um that you have uh yeah some some free list that you want to return it to uh when the communicator gets destroyed. try to think.

So, where else do we Oh, up, right in the up. And in the up. Okay. In the up, we need it. Um because I mean technically you don't need the extra state for attributes because you can put whatever the you want in >> yes for for the attribute right you could put more complex stuff in the attribute that is true um >> and the same well no for data rep you can't uh there you need the extra state >> right but for example for the op it >> it makes it it makes sense uh >> for for example one one one one act actually use case that we that I have in our C++ interface is when to actually register ops and when to free them and uh so what you would want to do there is like if you want to have this happen somewhat automatically and uh and what you do is you construct the op you add it to some list of operations and currently this is something that a user has to control fully.

And this way you could tie it to the to the communicator. No, wait. No, never mind. No, the OP if the OP gets freed. No, this is something different, right? Oh, but still I What is this useful for having the associate for ops? This is true. Like I'm I'm think I'm trying to think of an example. Um I think I think it is useful for um I think in Python you always need some extra state to invoke uh whatever Python thingy you have.

All right, we I think we have Oh, yeah. Maybe maybe this is this is it true because we have like for example for C++ it's the same. It's like the the callback that you actually invoked that you actually invoke for the implementation of the operation has to be stored somewhere uh since it is since it is registered >> lambda right your your lambda is essentially the extra state and then you have a function that knows how to invoke that lambda that's your >> call yeah right and we currently in the C++ world we do this with some static initialization tricks to just have it there um But uh yeah it's uh right this is is the correct use case. >> Yeah.

So um yeah and then I guess I guess we need Yeah, for that we need and I guess for the data rep we would also need a release call back right. Um >> yeah I think everywhere where you where you can provide some extra state you need to uh wait no do you need release do you actually need do you actually need release because you're you're passing in the extra state you're just saying this is state that you can use but it's like you pass the state at construction time and you you could day or is this now again about uh order when actually some things?

Oh yeah, for example, I think when we have a com going on, we just register it for uh for freeing, right? And we could have some operations still going on. So we need to keep the state around. Is that the reason why we even need that deleter? Because otherwise >> Okay. >> Right. because the uh in this case the ex well >> again for attributes we don't really need it which is really funny but for ops ops are com ops are actually nasty on this um you when you create an op and then you use it in an operation you can free the op before the operations are complete >> right >> uh so we cannot Uh you you cannot >> can you actually can you actually I think I just I just read up on that and I'm not sure like like it's or it's at least it's unclear.

I think it says your arguments to the to a non-blocking call or collectives have to live as long as uh and it's unclear if these are arguments even even uh even even mean the op as well. Isn't it like that? Sure. >> Your location to so it marks it for >> this. This sounds like a signal for Yeah. >> Yeah. And I think if you go to let's say um we need we need links in the examples that I think you need to really go to the beginning of the non-blocking section because it's >> Yeah, there initiate a call registering.

Oh, okay. It's not there. God wing somebody's game. Maybe >> and I think there's also some slightly contradicting things on in in both like the collective section and because non-blocking collectives are somewhere else and some >> in for for collectives it's different I think because we didn't want to c um like if you have an all to all V for example or W we didn't want to cache all the data types and >> so yeah it the there the buffers need to stay around I think for set >> it didn't have to be that way >> okay let me go back here.

It's got to be written somewhere. Jesus. >> Right. So, what it says in the non-blocking section is once initiated, all associated send buffers and buffers associated with input arguments should not be modified. Um, it's it says nothing about the but this contradicts what you what you just said, right? Uh >> yeah I I I thought you could >> as as far as I understand this it only says that for non-blocking collectives you have to keep the buffers alive.

So this means that you could also just delete your op or your data type but this then would mean that you have to >> I think the ops the com and data type and op don't count as buffers. Yeah, >> they their arguments they're captured when you when you pass them in. And that's that's the difference with uh V and W collectives because there we have buffers that you pass in because you're passing a pointer or two >> as array of arguments.

So it is >> okay but but still but still if you if you copy them at this point you like there is some internal like if you pass an op to this then it so you still need the deleter. This is this is like a crucial thing because you could like for example in I reduce you could pass in the op. Um the op gets uh gets stored somehow the the reference counter gets incremented and it is kept alive until this operation is at some later point actually carried out. >> Right. >> So this is why we need the delete there.

Right. >> Yeah. Yeah. >> Right. That's why that's why Yeah. We need we definitely need to call back. That's what I was getting. [laughter] >> Okay. Yeah. Yeah. Good. >> Right. >> One page right now. [laughter] >> Okay. Um and then and then of course another question that came up was what if something fails, right? I think there was a discussion the ticket that um where uh what's his name? Huie and Alessandro discussed what happens with nonsuccess returns.

Um right now for us I think even the date the the op has to return success >> does it have to sort of >> attribute it has to otherwise it's it's erroneous which is not a thing but um let me see collective reduce sketch broadcast area reduction user defined reduction see use the function okay does it say anything about the return oh it returns void so you not return anything. >> Oh. >> Uh, >> okay. Well, that's clear. >> Yeah.

Which I don't know. I've been struggling with that. Um, whether that's a good idea or not. Let's say you >> But you can it it actually says you can call NPI abort inside >> call NPI, [laughter] but that's not that's not a great way to propagate arrows, right? >> Yeah. >> And in we do have a return value. No, that's not that's not true. We I want to have a return value because if you do things like uh CUDA kernel launches you and that fails for some reason, you want to have an a graceful way of handling that and with void you don't really have that.

Um so I think in that case the the collective should fail um but only the collective not everything else. Um so we should rethink whether the new function that takes uh the extra state should also be allowed to return something other than success. >> Um and then have it bubble up. Um, I think those were the the the topics that I had in mind um that were open while I was working on the text. Um, see, so I didn't touch the sessions here.

Um, oh, right. Okay. I think those are maybe those are two different things, right? One is attributes for everything and one is sessions. >> Sessions uh >> uh sorry um >> extra states >> callbacks callbacks >> callbacks [laughter] >> extra state. >> Yeah, >> I didn't touch the I didn't touch the attributes everywhere. >> Right. Right. So yeah with the callbacks but still yeah here we have the extra state that was what this is what I was mentioning there is a chapter on extra state actually which defines extra state >> oh >> uh it's somewhere in the language bindings language interoperability chapter >> maybe it's something that >> yeah language bindings 20 or for me it was 19 uh what was it 19 for me and then it's in language interoperability I think yeah and there is extra state the next there's a [laughter] >> okay extra state should not be modified by the copy of the callback function oh well that stinks >> no it's not about like the extra state.

Yeah, I I don't even understand this. Like the extra state should not be modified. All right. You cannot if your extra state is a pointer to some or is your extra state then a pointer to a pointer? Like yeah, I think it makes sense to not modify the extra state if the extra state like what if what you pass in is an address to the state and you should no wait but what happens if you do free in the delete function. Um so actually you should what what do you even pass in as excess state then?

Is it is it an address to an address? >> It can only be a pointer, right? You can only pass a pointer. >> So, there's some data behind that pointer. And I'm surprised that it says that you can't delete that um in >> well it makes it makes sense because the like the only place where we currently have extra state apart from that data red thing is attributes where you where you provide the extra state um at attribute construction time and not uh um when you like you have have this two-stage phase.

You you provide the extra state when you allocate a particular attribute key and then no wait no it should not make a difference right >> no it shouldn't >> it shouldn't right you the thing is that uh well what I what I see there is well this this this pointer that we have there that we pass in this extra state. The implementation needs to store it internally. And if we now call free on that then it gets uh then it gets reset to a numpinter and uh like we diverge from what uh what is internally stored as the extra state and what like the extra state doesn't point to anything useful anymore.

But that's not a problem because the delete function is only called once. Right. >> Right. So >> this should be fine. >> Somewhere in the attribute handling is a case where uh you pass a void star for what is conceptually a void star. Um I forget what function that is. Um maybe this is because of forrren where everything is a single star. So we can still adapt the convention that everything is passed as a single star but because it's an opaque handle anyway. um either the calling or the call side can then interpret it as a star star. >> Yeah, I think this is for attributes.

But isn't it for the I think there it is for the value that you pass in and not for the >> Yeah, when when you query the attribute, you pass a void pointer and it will return >> right. But there but there it makes sense because you pass in a void pointer and in this which should be a pointer to a pointer because because you get as output the pointer to what is actually stored. >> What is the value? Yeah. >> Uh by the implementation. >> Right.

Right. >> Right. Anyway, petty the poor fart programmer. Yeah. Yeah. But but I think like is there any where where is this this whole chapter 19 is or these additions are part of the AI stuff, right? Are they >> I think like it's it's it's together with language interoperability >> which is mostly the size of types and all that. >> AI has its own uh chapter. >> Okay. But maybe it was still maybe I can look in the change log where this came from.

So it's section and feel. Okay, I can find any references to that to that section also not in the change log. Uh but yeah, it might be old, quite old because like the surrounding sections, they are from uh they're related to MPI 2.1, I think. Or this is where they came up. Yeah. Oh, is that because Oh. Um, so in forran you're basically getting a pointer to an address to an integer of address kind, right? So maybe what it means is that you're not supposed to change what But it doesn't understand.

You're not supposed to change what the internal pointer stored in NPI points to. >> Yeah, but you don't get that one back because you're passing like I would expect. Or is it like is is the extra state that you pass in um is that maybe we're we're mistaking something there. Is it uh is it an input value or an output value when you do attribute create? Maybe that's the that's a thing that you like have the same. Yeah.

Create key there. It's no it's >> it's in. >> It's in. Okay. then it doesn't really make sense to me >> because there I would expect I can just pass a pointer that pointer gets gets stored verbbitimally and I can then do whatever I want with that address. I think I think so even taking the risk that I'm making a complete fool of myself but I seem to remember that in fortran everything is passed as a reference or a pointer right so I could change what extra state points to like >> okay I see >> I could change the extra state no >> no you you should you could you could change the internal value that is stored for extra >> right because somewhere in the implementation it's calling this function passing in the extra state that it has internally stored but this is only by reference so it's uh so you could but but this is the problem that we would have with everything that we're >> right oh right but maybe this is this is the thing because all the other callbacks are just getting back uh either Right?

Because for example the normal operation callback gets um there you have uh have a buffer which is something user provided which you control and you have well you have the data type which is another handle that is guaranteed to be copied. You have an integer which is copied. So there you don't have that problem. So maybe this is actually the reason here because we have some kind of we have uh like can can you maybe search if if anybody anything else takes an MPI address kind um yeah I think there's many places where we do that.

Um >> yeah it should be all >> everything that is a pointer basically. >> Right. Right. Ah yeah yeah yeah but but but never in a call back. This is maybe this is a difference because we're now having having an address that is coming from the implementation gets passed to >> like we have something that is not uh there's not a plain value that that gets passed from the implementation to some user code. So leak implementation state.

So, >> but but this still like uh maybe maybe there's something in the language or ABI chapter I don't know address But it's an integer. So it should not be >> an integer that is big enough to hold an address. >> Yeah, but this means that well I don't know anything about forrren but I would expect uh if it is just declared an input parameter then it should not be modifiable right? I >> think that is a question for Victor.

Maybe Victor you can enlighten us on this. right there. Okay. Not here right now. Um, but I think that that was my understanding of of Fortress and it's all a little weird. Um so let's assume that that is the case. So um >> then it makes sense but then you we would also have to change the whole usage of the extra then we then it should be made clearer first of all what the extra state actually is like also in the attribute stuff because there therefore we argue arguably don't need it but uh still it's uh so it must be clear that the extra state that you pass in there is is a but it's a pointer to an address but then it would be pointered to an address which you somehow had to store so that doesn't doesn't work as well right or do you have to dreference uh or maybe or or is or what what is no >> I think that's just the way fortune passes arguments >> yes But but then this means that I cannot modify the like just the argument I cannot modify it.

That's the >> right >> that's the >> I think it's it's not written in a in a concise way >> right. So but but like for example in if I would do it in uh how would I do it if I would do it in C I would have to given this this rule I would have to copy this pointer that I get as extra state into a new variable called free. >> Oh no no no don't don't don't even think about it. Um it it doesn't [laughter] it doesn't [clears throat] make sense in C.

Yeah, >> which is >> Yeah, I think this extra state chapter really has to be >> which is why it says here this is obvious from the C bindings but not obvious from the for >> ah right okay Mhm. I see. So this is >> so yeah we need to we need to revisit that. >> Yeah. >> Yeah. Okay. Um All right. Um, the other thing I did was I only had ambigant version of MPI of of the new callback because I don't care about the non-imbigant one.

Uh, I don't think it makes sense to I don't know drag that >> dual thingy through. Um, >> yeah. The the question is how like how is the state of big and support in in OpenMPI at the moment? Wouldn't that mean that >> we we have it complete? It's merged. It's there. >> Okay. >> And it will be released in six soon. >> Okay. Then then it makes sense like and and apart from it, yeah, it's not like it's if if it will be released and and and well, it's part of the standard.

So, it's fine to build on the existence of uh of uh of big ends, right? if we do a new standard version. >> And the other thing is let me see I called it NPI op function not user function. I think that is cleaner and eventually I think we could even think about uh deprecating the old version and just use the new one. Um >> you mean the MPI op create? But this would also mean that you would have to deprecate the non begin reduce function, >> correct?

No, no, no, no. It's just op creation. True. Sorry. >> No, even the even the reduce function because then you don't need it anymore. The old one npi user function um you just use npi op function and you take the extra state as an argument. So that the the transition is almost trivial. >> Okay. Um but now yeah we would only have one one place again to um do that and it anyway um but that's that's future um let's see did I write anything?

Oh yeah, I think I think I still have the no no return value here somehow and we should maybe that's something for the forum to decide. >> Yeah. >> Uh that doesn't just hit us uh whether we want this to be able to return something meaningful and propagated. Um because I think that is essentially what Alando was arguing for that we need to be able to propagate errors. Mhm. >> across callbacks. Um yeah, other than that, I don't know.

And I think the rest is I think it's more about nailing the extra state semantics clearly and then just like there there are not too many places where it is used. it's the attributes and then the next question is if is like like this would be would be twofold like first adding it to op and reasoning about extra state given the current design of attributes and how to define extra state and and the and the constraints for it and then when this is done we can think about adding attributes to more stuff which which in some sense contradicts the session attributes maybe but But I don't know like I think the session attributes are just modeled the way that all other attributes are modeled and if we make some changes there they would have to propagate to all attributes that are already there or will be added in the future doesn't make a difference I think.

Well, I think yeah, >> but I think but I also think uh the the session attribute proposal is quite there already. So, >> look at it. Um I would send an email announcement. >> Yeah. Okay. >> Check if I can find it. Yeah. All right. It's first vote. >> It's the first vote already. >> Yeah. Maybe we should drop into the session working group on Monday and see. I think it would we shouldn't just come out of the blue and vote.

No. >> Yeah. This is true. Um, it's probably not even like not even a lot of um, it's not like it's I think it's mostly uh mostly um copying over uh the other what we have for the other attributes. So it doesn't make a difference >> deciding on deciding on the semantics that we want to have. Um so for example I had this change that basically said that um here it used to say if an attribute copy function or attribute delete function returns other than npi success then the call that caused it to be invoked is erroneous like on a face of it that is just a stupid thing to say because you don't control what someone did right when [clears throat] >> I don't know how how they got through with this.

This doesn't make sense. I think what they wanted to say is that the operation is will raise an error. It's not what they said. Um so I would rephrase it as we'll raise the error returned by the callback function. um instead of is erroneous. So the call the call that caused it to be invoked will raise the error returned by the cover function. But is this is this right? Yeah maybe it makes sense. uh h like I'm I'm wondering if it if it then makes it even possible to to uh to figure out where the error came from. >> No, there's no way you know that.

I mean there's the only way to know is that you know the application sort of has a protocol with itself, right? If I erase this arrow then this is true sure. Yeah. And we could add another call back error or another error class that that is something along the lines of um user call back fail error or something that the user can use if if it wants to. And anything beyond that, right, like details on why it failed needs to be stored externally in some variable that the co can access extra.

Um >> yeah this just makes sense. >> I'm I'm not sure about the what happens after that. So I wrote that in that case the attribute will be deleted without invoking the delete function because basically assuming that if you had an error then we your your state machine is broken and we shouldn't come back to you. Um but maybe that's too strict. Maybe we should say that if the copy function raises an error, we will still invoke the delete function.

But that is something that we need input from the people who care about this like Jeff and Land. So um be good to get some back from them. Yeah, I think I really need to if if I have the time I will read into the discussion what the problem there is actually with uh which would like brought up this discussion with NPI for PI because I can I can maybe come up with some uh related problem in the C++ interface that would uh also need that and they're like a bit more fluent in understanding what is going on. >> Yeah, I think that well it is a little different I think.

Or maybe not. I don't know. >> No, it's it's not. You have you have some state that is uh flying around uh at least like not not not in the in the like main implementation part. So >> yeah. Anyway, um yeah, that would be good if you can look at that. I tried to to read through it. I didn't read all of it. Um just to get an idea. >> Yeah. >> So yeah, let's do that and meet again. Uh the question is should we do we want to ask for a plenary to um >> I think that definitely makes sense like uh >> like like even even if it's just for pitching the idea and saying hey uh we don't want to uh intercept the session attribute proposal for now but we may come back to attributes in general >> or the semantics of attributes Right. >> Um >> can can you send an email um for a mailing list asking for >> I can do that. >> Okay. >> Yeah.

Uh the question is uh should we do another meeting before that or should we just like >> would be the the Thursday before the forum meeting? >> Yeah. Right. That makes sense. Okay. Yeah, >> that will work. So, we can prepare for the January >> then. Let's do that. >> Okay, >> good. >> Sounds good. Talk to you. >> All right. Yep. Thank you. >> Bye.