Can Oncology Workflows Run Without Human Touch? - Anant Shankhdhar, Risa Labs transcript

Hi everyone. My name is Anant Shankar. I am an AI engineer at Trisca and I'll be talking about oncology automating oncology workflows from end to end. So at Trisca, we are automating various workflows in oncology. One of such workflows is prior authorizations where we file for authorizations for drugs for cancer patients. So I'll give a brief overview about the workflow before we move further in the call. So the first step is that we intake the orders that we get on a daily basis.

The second step is we verify whether the patient is actually eligible for getting the drugs based on the amount that they have in the insurance available with them. This is called eligibility and benefits verification. Next, we determine what all drugs in the for the patient require authorization. So drug can basically fall into one of the following pathways. So one is NAR which is no auth required. Basically the drug does not require authorization.

Another can be that the authorization has already happened for the drug for a time period which is called auth on file. And the third is auth required which means that the authorization needs to be performed for this drug. So yeah. This is the workflow. I was So although our bots run and perform these steps, finally a human review is required in before submitting all these orders. I was tasked with the with the problem to run some of these orders without any sort of human touch directly towards submission.

Which means I need to confidently identify which all orders can be proceeded without any human verification and then build the entire flow for them as well. So, confidence is a key metric that we were working towards. So, yeah. How we did this was using four agents, namely EV agent, auth agent, necessity agent, and submission agent. EV agent is for eligibility and business verification. So, basically fetching the patient data and determining whether it is uh fine for moving forward.

Auth agent determines the key type of the status of the drug, whether authorization is required or not. Uh and the necessity agent is the clinical brain of system of the system. So, for the cases where authorization is required, we determine whether it has to be done on a whether it is right for the patient based on his vitals or not. And finally, we move towards submission. So, uh starting with the first step, how do we do this without any human in the loop?

So, one of the problems that we have here is that insurance details, insurance documents, everything is a scattered across dozens of portals, APIs, and documents. And it is difficult to find that information at one place. So, in order to in order to start with our pipeline, we need to get information from various portals as well as APIs. In order to do so, we built a unified service that connects to different payer sources and gives the output in a normalized uniform format, which we can use for processing further.

And we also added a deterministic decision engine to flag the cases which would not move forward, thus deterministically uh fixing some of the orders without any human in the loop. So, here's how it works. So, we have a coverage orchestrator which determines whether the patient will go for the API or the RPA path. RPA is basically the automation. Uh we perform the actions for the RPA or call the API and get the output in a fixed coverage result format which we then pass to our deterministic engine to determine whether the coverage is active or not.

If it is, we move it further in the PHN, else we stop that right there. Now, one of the problems we saw here was that uh if we have to build this, then we will need to make custom integrations for different sorts of portals, which is not not a very scalable uh process. But, how we tackled it was that we used LLMs in the loop loop. So, first of all, we made a huge repository of custom actions as well as popular actions that are required in to build RPAs.

Then, we built a LLM-based config generation which performs these actions and builds the config for for a portal to run on. Uh which uh fix which reduces our development time significantly. And finally, uh these automations are fragile, so it may happen that they break during the run time. For this, we have a self-healing loop which identifies these cases during production hours and then mitigates them. So, that thus preventing any sort of failures.

So, yeah, that is the first step. Uh once we have done this, we move further in our chain. So, as we can see our diagram the graph, uh the patient data is fetched. We perform the eligibility verification. We see if it is fine or not. If it failed, we stop there, else we move forward. So, yeah. Moving forward, uh the next step was that data uh was to see uh what all drugs require authorization. So, if I have to uh remove humans from the loop, one simple case that I saw was that uh if a drug has already been authorized, that is the authorization is on file, or if the authorization is not required, then we can partly solve the order.

That is that we do not need human oversight on that part of the order. Uh so, yeah. Uh so, as the first step, we built a simple LLM extraction pipeline, which takes in patient's notes, performs LLM's extraction, and categorizes the drugs into these types. The default being that authorization is required. However, uh yeah. So, we thought that they should would work, but we faced some issues here as well. So, one popular issue was that uh the notes that we were using did not have enough data, which means that it was performing some some errors.

Uh Also, LLM uh extraction is sort of an indis- uh indeter- deterministic process. That means that whatever outputs we get from here cannot be blindly tested. So, we still need a human to review all these things. It might improve the efficiency, but it will not eliminate the human. So, in order to do so, uh we thought that what if I add more evidence to it. So, if the LLM is saying that a drug is not uh no auth required, then I have some information backing that.

I can say that confidently or for the auth and file cases. You already uh yeah. So, in order to do so, we we leverage two other data sources. So, one was the authorization letters. So, this was basically the previous information. So, across the previous runs that a given patient has, they have authorization letters available, which shows that these drugs were actually authorized. So, now, instead of just having one source, I have two sources that give me the same information and wherever they conquer, I can say with confidence that this drug is already been authorized.

Second is the NAR case, the no auth required case. So, here uh we found some other resources wherein from the portals or from some documents, we could find out that on a monthly basis, which are the drugs that the payer does not work for. The insurance does not work for, basically. So, uh we use that information to build a payer rule knowledge base, which is basically a SQL database uh which was made from portal checks as well as LLM extractions.

So, we use all this information, we reconcile the evidence, and we only extract the auth auth statuses with a higher confidence. So, if from this pipeline, we get whether a drug is not authorized not does not require authorization or is already authorized, we have a solid proof behind it. Uh and a higher confidence uh to say that this is the case. And of course, all of this is configurable. So, uh in order if some mishap happens, we always have a plug over the cases.

So, yeah. Uh another a good thing that happened from here is that we noticed that certain orders were completely eliminate We could eliminate certain orders completely from these two type of drugs. Because it may happen that an order does not actually require require authorization at all. The drugs are either are not requiring authorization or are actually authorized from before. So, this enabled us no touch on a certain set of orders that we had.

Yeah. Uh this slide gives a big uh small overview on how we build the payroll knowledge base for the NAR cases. So, uh we had the documents which had this information on a time-wise scale. We built an LLM extraction which uses LLMs uh which uh is configurable. So, we can configure it on different types of documents and we can perform the extraction and we can get the drug-wise constraint that for this pair this is the this is how it will be treated for a period of time.

Another thing is that we leverage some historical information. That is that we know that a certain organization uh treats this drug in a certain sort of way. So, we stored that information and use that as well. Apart from this, we also use some regular portal checks to get this information. So, yeah. Uh performing this movement moved our needle forward a bit. So, now we had these deterministic checks along with our initial ineligibility verification.

So, whatever were not verifiable were flagged out before and whatever moved forward, we could determine these statuses for the drugs and we were able to eliminate some of the orders from here. So, now that we solved for two types of drugs, let's move forward. So, the next problem that we faced here was that some decisions do actually need clinical reasoning. So, so far the drugs that we were we shifted to no touch were the ones where the information was directly available or indirectly available via another source.

But, we did not need to perform any sort of reasoning over here or any sort of question answer. However, for the drugs that are actually auth required, we need to uh check whether the patient is actually eligible for them and also give sort of supportive evidence from where we are answering this we are giving this information. So, in order to do this, we built our third agent, which is the medical necessity agent. So, the medical necessity agent answers simple and complex clinical questions per patient uh and attaches confidence score to any answer.

So, we escalate only the ones that actually need a clinician. Uh so, we did a lot of work on this space. We actually have another have a publication here as well, which I've mentioned. So, how this agent works is that we have a new order and for every order, we read three things. So, we read the patient notes. We read the policy criteria. So, the policy criteria is basically the criteria that needs to be met in order for this drug to be processed further.

Uh we from this criteria, we query the patient medical graph, which is a graph of biomarkers that are extracted for a patient. And we determine what all biomarkers exist and what is the condition of this patient from here. And then we use this thing these this information and pass it to an LLM to get an answered questioner with all the supportive and contradictory facts. So, yeah. And if we have the relevant information or we are able to determine this with a higher confidence, we move this forward.

And for the cases we do not have this enough information, we move keep that for human escalation. So, yeah. This is the next step. So, after this medical necessity validation, we move from the first step where we fetch the data, we did the eligibility verification, we flagged the orders that were not eligible, moved the rest forward, extracted the not needed and already authorized letters. Uh got the got the cases where we can move that without any human intervention.

And then we did the medical necessity validation. Whatever was not met was also moved to review. So, yeah. Finally, we assemble all this information and then we submit it back to the payers. So, this is where our submission agent comes into place. The submission agent is pretty much similar as the EVBV agent. We have customized integrations for every sort of payer that we have. Which is built using an LLM driven config as well as our repository of tools.

So, yeah. This is the entire graph that we got through the entire journey. Uh So, the green part shows the cases that we are going without human touch. So, whenever eligibility verification passes, then enough information is found to not to prevent the drug from going for further medical necessity evaluation. Then this medical necessity evaluation and the submission. So, yeah. Finally, I'd like to mention that these drugs are also all although built initially for this case are being used across multiple workflows.

We have extended the functionality to a more general one where the medical necessity agent can answer questions specific to any sort of workflow for the patient. Similarly for the other agents as well. So, yeah. As the conclusion, I would say the no touch is growing on the share of every order. So, we started with deterministic checks. Agents only for the rules that where what rules can't decide. We used multi-source of evidence to beat the single source of evidence to add confidence to our cases.

We added itself healing in order to scale our RPS. And finally, we added the reasoning layer in order to deal with the cases where actually authentic authorization is required. So, yeah. Thanks for the yeah, thanks for your time. Yeah. Bye-bye.