3Blue1Brown Talks Machine Learning with Jane Street transcript

Okay, one of the things I I want to do, thank you for taking the time, is just try to get a feel for what the Jane Street culture even is and also just technically what do you actually do? What are the salient features? Right, what comes out is like you want to distill a sense of what Jane Street culture actually is as it maybe differs from other places, what pops to mind? >> Jane Street is really weird, but in like the best possible way. >> Okay. >> Like truly when people are um having conversations, like the goal isn't to like have ego or like to prove a point or kind of show people how smart you are or something.

That'd be very exhausting. It's really approaching like a topic that's kind of interesting uh and trying to arrive at a truth at the truth by kind of like talking through things and um I think people are very just like receptive to things that they didn't know before and um I don't know, it's like pretty cool if you can like approach any conversation and expect to have your view change um at the end of it. >> Is there a time that I guess any of you like had your mind changed in one of these interactions or vice versa where you're like having an interaction and you're like change someone's mind that felt very low ego uh that like exemplifies what you're talking about here? >> I feel like it would more it would be like more anomalous to find a time when that didn't happen.

Like I think like a lot of meetings like you know, you know, people like come in with some some thoughts and often we'll ask people to like share their thoughts before they hear everyone else's just to like see what everyone is starting with and talk about it and then maybe at the end we'll like share what our like after thoughts are based on hearing everyone's and usually they've converged. It's like very It's like very nice to see actually.

It's like wow. I think it's like a pretty unique um way of conversing um and I feel like I know that it's unique from just like going back and talking to people from college and they're like wow the way that you talk is pretty strange. >> [laughter] >> Like the way you talk about interactions here or you've like adopted a certain Bayesian habit that like >> Oh, there's like a certain Bayesian habit. There's also like Jane Streetisms, you know, people will say things like oh, that's not unreasonable.

Um >> [laughter] >> I don't know. I I think there's just like phrases you just hear a lot here and it's just absolutely wild outside of Jane Street. >> My impression is that we have much more of like there's kind of just one company. >> Mhm. >> Um and it doesn't matter so much like which group you're in or something. So, people talk across groups a lot without going through uh like their manager talks to somebody else's manager kind of things to set it up.

You just kind of walk over to another group and you say, "Hey, I was thinking about something. Does anybody have a minute to just spitball on it?" Like it seems like something that you guys would know something about and you know, so it's it's lots of very organic >> That seems hugely important, but it also seems very hard to scale, right? I you know, if I try to picture why is it that other organizations evolve kind of hierarchically and it is you're going to manage manage manager is that and maybe I'm naive here, it just seems like chaos otherwise. >> It is a little bit. >> Okay.

Okay. And so is there a kind of an embracing the chaos of that because of the the benefits of cross expertise communication? >> Yeah, a little bit, but I mean there are like you know, people have managers, right? And there are like kind of team leads or project leads or you know who to ask about certain kinds of topics. Um so, there is some structure and like organization to it. But I think that that kicks in where, you know, if you're going to start like a month-long project with multiple people, now okay, let's be organized about it, right?

Um but if it's just kind of like you're going to walk over to try to come up with some ideas or something, that's at the like a small enough scope where just the chaos seems like the better way to do it, I guess. >> Got you. Got you. Um man, I have many more questions there, but I want to say >> on to that. >> Yeah, yeah. I want >> really emblematic of this chaos is we run like a real-time auction for our compute. Uh it's like our compute cluster is available for everybody and you know, like your task would be you know, a task of realistically thinking about how to value kind of like this job um that you're running versus all the other things and um there's not really any guardrails. >> Wait, that's so interesting.

So as whereas like other company large compute, there's a whole like bureaucratic like shifting around chips kind of game to decide like what allocation they get, it's just straight up auction on and like anyone can kind of bid for the compute timer. And it's funny, I almost would have said that that's a case of organization. We've got an auction system that somebody implemented. >> I was I mean I do think there are like advantages cuz I think like people like really feel collectively responsible like in a good way feel like everything everyone is doing.

Like I feel like if I really don't leave my desk or you know some job running and then it's like doing something horrible like bumping into errors like it's not on me to be like checking every like one minute like I do feel like I do feel more responsible for my jobs than everyone else's but like someone else might see it and like you know be like oh I noticed there's an error let me just like pause it for you until you come back and I think that's really nice and people feel like that sense like we're like one you know one team even outside the local team I'm on. >> Yeah, that's so fascinating. >> I mean you'll you'll kind of like the people who are actually I don't know running these jobs like people have access to kind of the jobs that are running and if you kind of see something that's a little off you can just go and talk to them and say hey I see that you know like you're running this job and this MFU is just terrible like what's going on and you can have a conversation and a lot of like things can be sorted out without it like going up the ranks or something and I think it just allows things to run much faster. >> I think it's globally read right too right?

If like you can just nuke somebody else's job anytime. >> Really? >> [laughter] >> Yeah, there there was an instance where a unwitting researcher ran a command in a Jupiter notebook um uh of adjusting the bids on his job. He accidentally ran the command without any arguments which meant that he changed the bids of every single job running on the hive. Um >> And he had access to do that? >> that was not great. I think we fixed that but like it is kind of like a you know we'll build a system as it kind of makes sense and kind of change it organically as it fits rather than coming up with like a structure um necessarily that's restrictive from the get-go. >> curious how you describe to outsiders what Jane Street does, cuz you've got the quantitative finance just like hand-wave sort of description, then you've got a level of specificity that is like trade secrets and kind of obviously can't talk about, but there's a lot of room in between for actually clarifying for you know, people outside of finance like me, what what do you do?

What is the actual uh organization pointing towards? >> Yeah, I guess there was a a phrase that I think Sandor came up with uh which is kind of a mouthful of uh Jane Street being a uh generalist financial services liquidity provider. >> I will be honest, that has so far not added clarity. >> When we have like college students in the office, I give a lot of intro talks and one thing I say a lot is like, you know, I think one of Jane Street's core businesses that was really really really our core business, you know, many years ago, but I think still drives like our philosophy around trading is being a market making firm. >> Okay. >> Which is, you know, what I what I tell folks is it's like a grocery store for stocks.

Um not literally, but you know, I feel like, you know, grocery stores are pretty magical. Like if you had to like go, you know, buy all your groceries and you had to like go to a farm to buy your apples and go to a dairy farm to buy, you know, milk, you'd be pretty sad. And the grocery store has to think about how to price things fairly, right? They can like charge like, you know, $10 for an apple if they want, but no one is going to buy their apples.

And I think that that's kind of like a pretty similar principle. Like you know, we do do the business of like kind of pricing ETFs relative to stocks and kind of like, you know, if they trade out of line, but I think the different way to like conceptualize that is that like, you know, if I am like, you know, sitting in the United States and I want to like invest in, you know, some companies in like China or I want to invest in some companies in, you know, some foreign market, the best way that I can do that is by buying an ETF. >> So that's I mean, that's one category of answering the like what Jane Street does, which I'm sure doesn't scratch the surface like on just like the trading half.

But a lot of the tone that I also hear much more recently is more it's a a more similar to if I'm talking like an AI lab. >> Yeah, I think um maybe the gap between AI lab and grocery store as metaphors is like a little bit hard to maybe one thing that would help there is um if you think about it with a grocery store, they need to know to have a lot of jack-o'-lantern pumpkins for, you know, um Halloween and and these kinds of things, right?

So, there's a lot of like anticipating supply and demand and then you can kind of see how that's a prediction problem that could be somewhat complicated, right? Um so that of where, you know, the machine learning and stuff like that kind of comes in. You want to have sort of the inventory to provide liquidity to you know, what's upcoming and what people will want. Um >> This tracks with I think a a conception that because, you know, in theory if there's value to be added from these instruments like ETFs just as an example, you could have like a management firm around it that like is deliberately trying to keep keep things in line.

But then if it's in a market-driven way, it's some other participant in the market that does that. Is there anything that's misunderstood? Like if if you see in in so far as there's, I don't know, discussion in popular press or just like uh interviews that you have with candidates coming in, are there misconceptions where someone thought about it one way and you realize, oh, actually, that's just that just doesn't match with what you see on the inside? >> I think there's a misconception not just about the scale of machine learning that we're doing, but the novelty of it.

Like I think it's like kind of easy to imagine that mostly what we're doing is we're taking some like models off the shelf and maybe we like bought a bunch of GPUs and then we're like running it. And that like basically doesn't work at all. Um in in financial domains, I think the problems that we're working on are much much much noisier. And so we've had to do a there's a lot more like really I think it surprised me even a lot more like basic science that we're doing.

Um like, you know, where we're really trying to understand fundamentals and working on the frontier of what is known, you know, in the research community at large. >> Mhm. >> that surprising that the misconceptions exist just because um things have ramped up really really quickly um over like the last couple years. Mhm. Um but yeah, I don't know. It's just been kind of a incredible ride. >> So, you felt the meaningful shift just in the last couple of years for like what the what the actual work feels like? >> Honestly, just kind of like the amount of success that we've seen with um some of the ML efforts.

It kind of feels like we are just kind of like at the beginning of the S curve. And um but it's also just like uh very meaningful input to our trading. Um and I would say it's like, you know, the single largest driver of um the systematic trading that we have. >> Yeah, I think there's a lot of misconceptions, but they're kind of like different coming from different directions, you know? People at other trading firms or like banks or stuff like that will generally assume it's going to be more similar to these other institutions than it actually is.

Um so, often if we hire somebody from one of these places, they'll be surprised that it's maybe a little bit more like tech or academia in some ways or some something kind of like that um compared to what they were used to. >> People see that we have like a lot of traders and then like a research group that's meaningfully smaller and assume that means like the majority of our trading is being done by people manually entering orders into like a a Bloomberg or something.

But like mostly this is historical accident in naming and like the the role that we call trader is like in many ways largely a research one. Um and like the firm is way more systematic than I think people tend to assume. >> Right. Yeah, pop quiz, which of the people here appear in the corporate directory as a researcher or a trader, right? Like >> [laughter] >> I'm not sure. >> I was like it's just generally like you're like thinking of your coworker like, "I'm actually not sure what their title is." >> Yeah, there are sort of like, you know, teams and some kind of structure to this even though not everyone's exactly sure which team they're on or you spend what some time on one and some time on the other, that kind of thing.

Um but yeah, broadly there's kind of like specialization at the level of um people who mostly code but understand, you know, the trading and stuff pretty well. Then there's people who uh code but mostly with other coders as their customers, right? And so they're like a little bit of a step further removed from the trading and stuff like that. >> This is like internal tools. >> Yeah, internal tools for like managing all of our I guess branches, you know, and who's code reviewed what and is it ready to be released and various kinds of infrastructure for dealing with all the market data.

It turns out there's actually a lot of data. Um petabytes, so that's a big task. Um And then there's kind of like a a loose um spectrum between sort of like traders and researchers with a lot of in between. Sort of having to do with how much you think about like particular securities, what is happening today in particular trading systems as opposed you know, that on the trading end as opposed to a little bit more like you know, launch off some compute job to do some machine learning stuff to train some model or build a tool for doing that, you know, that would eventually like feed its way into these trading systems on kind of like a longer time cycle. >> Got you.

Now, this is maybe while I'm talking to you Craig, like you've been around for 20 years. In that time, how are things different compared to then? And I have some suspicions for what the answer might be, but as you look over the last 20 years or so, like what is the difference? And let's maybe start like technically and then I'll ask culturally. Like technically for what the actual work looks like back then compared to now. >> Yeah, it's it's very very different.

I mean, the the work um when I started, you felt pretty fancy pants if it was like, oh, we've actually logged data, you know, and we can now like train a linear model on a compute cluster, you know, like, wow. Uh and so that was kind of considered you know, the more advanced stuff we were doing as opposed to um in a way, a lot of it back then was stuff that you could just kind of look at two securities and if you knew how they work and thought about it a bit, you would be like, "Oh, that's out of line.

We should trade them to be in line." Um and so, there were literally uh trading systems that uh would run in an Excel spreadsheet, you know, to like do these kinds of things and you could just look at the trades and be like, "Oh, yeah, that was a good trade." You didn't need any like statistics or things like that, you know. Um and so, the challenges for that kind of stuff were totally different. Um at this point, uh you know, most of the modeling type of work really is machine learning kind of stuff.

Um neural nets increasingly recently, you know, GBTs and stuff also, just all kinds of things. And we hit many, you know, points in between. So, I would say like every five years or so, really the techniques have changed like quite a lot. And yet kind of strangely, it feels um similar as far as like kind of the culture and stuff like that. Like a lot of the original people are still around. >> Why is it that people stick around so long, right?

Like what what is it that makes people stay that seems to not happen at other places? >> I think like earlier you'd asked like them about like, you know, what makes Jane Street Jane Street. And I think one thing that I've really felt is like the people like like there are lots of workplace settings that are super nice, that are super friendly, where like your colleagues are super friendly to you. Um I mean, Jane Street is also one of those.

But I think more than that, like a lot of the my colleagues, like especially even new ones I've met and on teams I'm on too recently, are like my real life friends. Um I think that like people help each other out and collaborate not just cuz they're like nice people and they're friendly, which is, you know, really great, but because like they feel like really invested in your success. Like, you know, when you were in college and you like had a roommate or a friend and like they wanted you to succeed on the problem set like almost maybe sometimes more than you did.

And I think that like I really feel that from a lot lot people around me that they are like as invested in my success as I am. Um and they really feel that like for me as a person and not just cuz like they're a really nice person or that's like kind of a norm at the company. I think it has become a norm at the company for people to help each other out. But I think that also means just like everything feels like kind of like really low friction.

It feels really like fun to go to go to work. I'm like, "Oh, I'm going to go to work. I'm going to meet my friends. I'm going to work on really cool problems." Like if, you know, things are really tough and challenging and high friction, especially as they can be sometimes when you are moving into new areas, there's going to be a lot of support and people are going to be like with me doing it. And I think that really, I don't know, for me that's that that that makes a big difference.

I think for a lot of people I talked to it seems like a really important input. >> I mean, from a personal point like I feel like it's not like I came into I came into Jane Street thinking, "Oh, like I'm going to work here for 10 years." Um it's kind of been like an everyday reevaluation of like do I feel like I'm learning a lot here? Am I having fun? Like um you know, are the conversations that I'm having, like the amount of agency that I have, like is it, you know, better um versus like um comparable other opportunities and I just kind of feel like like the pace at which I feel like I've learned new things um has kind of like increased over the years, which is kind of um shocking.

And I feel like that's like a pretty nice feeling to have. >> Mhm. What one of the things I was a little bit curious to ask was some of the story around joining Jane Street. And I might start actually, Nithya, with you. Are you able to tell me like what were you doing before you joined Jane Street and kind of what was the trajectory to come in? >> Yeah, like I think I personally had like a I don't know, maybe a normal trajectory for a person here, but it felt like pretty unique.

Like I really thought I wanted to be a mathematician. I was like really like set. And my family and my friends were like, "You know, this like math thing is super great. We would really love for you to be a mathematician. Like maybe you should like try like once like working at like a job that's like not teaching math just like once. Like see how you like it. You never know." And I applied for an internship and I got it um at Jane Street and I came and I like really really loved it.

I was like super surprised. Then I came back the next summer and then I came back full-time. >> Okay, that's So the the there kind of speaks volumes. >> [laughter] >> But your path was even less inertial than that, right? Cuz you kind of went back to academia and then came >> Yeah, so I after I came, I worked at Jane Street for a couple of years, and then I did go on leave and get my PhD and then come back. And that was like a super lovely experience. >> Do you like how normal is that?

You you took a little bit of a leave to do the PhD. Is that a little bit of an exception or are there other Jane Streeters that have a similar um leave for PhD return? >> I think I'm one of the very few people who did the particular thing that I did, but I think there are like lots of people at Jane Street who've had their own unique particular experiences, and I think I don't know, I feel like people at Jane Street have been super flexible about making, you know, people's individual circumstances possible.

I I think I'm one of the very few people who like went on like pseudo leave for a few years to get a PhD in the middle of my career. >> What was the What was the interview process like? >> Oh, the interview process was like super fun. Uh I was asked like a mix of like probability questions and like kind of more like miscellaneous problem-solving questions. I think the phone interviews are really fun. When I came to the onsite interviews, I think it was like very different than what I was expecting.

I think I was like kind of expecting to do like more like even more advanced math. And then I think that onsite interview was like playing a bunch of games, um which I found, you know, very fun, but also in the moment very stressful. I think I was like not really prepared. I didn't have a lot of friends who'd gone through the interview process. I think it was like I don't know if people call it type two fun like in hindsight like, "Wow, that was such an interesting interview process." In the moment, I don't think I would've told you that. >> Yeah, I got you. >> I I really remember that I had like a lot of interviews scheduled during my day, and my final interview was with with Shander, and we were like throwing chips at the wall.

Um and uh during that interview midway through, um two recruiters ran in and said, "She's going to miss She's going to miss her flight. And she's going to miss her flight." And then I was like shoved out of the hallway >> [laughter] >> onto an airplane. And I was like, "Wow, I really messed up and failed the interview." Um but it seemed to go okay for me. >> Is there anything when you're like interviewing people that you feel has changed uh kind of the the things that you were looking for, the ways that you would interview, you know, 5 years ago versus now? >> The bulk of our strategy has always been to hire people who are just like smart and hardworking and reasonable and adaptable and like to learn new things and then you can just do the new thing.

Like now for sure like machine learning skills are um a plus. But I think mostly it hasn't changed a super ton just because we're mostly looking for these very general kind of skills. So we deliberately try to like kind of have like math puzzles but not like something where if you took the right class you would have the tool and then it's easy but more just something to talk about and think through that's kind of maybe a little bit novel to them or something like that. >> Which is not easy to do, right?

Like I think it's very easy to like hire the person who's smart and generalist and like can adapt to a new thing but way easier said than done. >> Yeah, and I think also it relates to your previous question about like turnover and how much people tend to stay here. It wouldn't be worth your time if the median time until somebody left was in 3 years to spend a year retooling them on ML. You'd rather just go and like hire somebody, right?

Um but if you're pretty sure that most of these people are going to be here for a long time then it like makes more sense to invest in that kind of thing. >> Yeah, a really tricky thing about interviewing um is uh especially interviewing for a job where you don't come in with the full skills that needed to do the job is that you're basically trying to evaluate like what will a person's like skills and output look like a year from now or 5 years from now.

Um and yeah, this is really hard to do in like an hour or like multiple 1-hour sessions or something like that. Like how do you how do you tell if you can like, you know, train a person over the course of months to be highly effective at something. Um and there's no there's no punchline here. I don't have like oh yeah, we we did it. We did it with this and no, we didn't. Oh, we like we like try we try to do the sticks.

We like try to, you know, the strategy of like find people who can like learn to do a bunch of things and um basically find like generalists who are really adaptable has worked really well and I think it's also one of the ways that we've managed to adapt to like new demands of the job quickly because like if someone was good at learning and applying new skills 5 years ago, even if the skill set that we, you know, are demanding from them has changed quite a bit now, they're probably still pretty good at that. >> And it makes sense that that would kind of line up with the kind of people who just want to learn.

The that seems to be like what what makes the career a little bit more fulfilling. Alok, uh kind of same question for you on the origin stories. Like where were you before Jane Street? What was the trajectory into it? What was the interview process like? >> Yeah, so I I also came in through a relatively like typical pipeline. I guess I applied to the the research internship for summer of 2018. At the time I was studying math and computer science, but it was always kind of like, I don't know, my general assumption was like computer science is the one you do to like get a job and then math is like quite interesting, but like I don't know if it uses that in like >> Was that the motive for like joining CS with it?

Was like kind of a vocational one? >> Not exactly. Like I also did just enjoy CS for its own sake, but I had kind of imagined in my mind like, "Okay, like the jobs I'm considering are the jobs that will like use the CS part of my background. Like if I get to do any math, that's like a fun bonus, but like not really a a criteria I can set." Um but like I I guess even at the time it was kind of clear in my mind that there were roughly two fields that cared about like both these things and one of them was like broadly machine learning and data science and the other was like whatever they were doing over in finance.

Um wasn't really sure at the time. My expectation was like, "I don't know. There's like surely there's a bunch of people like sitting around with their Gaussian brain models that like, you know, explain everything." Um or at least explain everything in like this very specific domain. Um I guess after after spending a summer there I realized there was like in many it's more similar to data science elsewhere that I give it credit for.

Like I was actually kind of shocked at how well a lot of the other stuff that I've been studying kind of transferred or like ideas I had learned in college classes that seemed pretty abstract and like theoretical actually found like pretty concrete applications in just like analyzing the modeling we were doing and things like that. So that was that was a really interesting surprise. I was really kind of glad to to learn that. >> Examples are always hard, but does an example pop into mind of that abstract way of thinking that you as the young math and CS student didn't think would pop in and suddenly it did? >> Sure.

So one thing that I had studied just like the semester before in a in a stats class I was taking was like the theory of N-estimators, which is this idea from a guy named Huber in like the 1900s and it seemed like I don't know the kind of thing where the theoretical knots researchers like come up with something that is like asymptotically correct or only if you're working at a galactic scale and things. Like I don't know, there are a lot of cases like this, but uh turns out this is actually just like a pretty concretely useful thing when you're trying to deal with like malformed data or things like that.

Like it it just sometimes it just is is better. Like it just using these this kind of outer class of like robust estimators. So this is like what I I think I worked on during the internship actually. Like try to apply this theory to a concrete like modeling problem. >> Oh, that's good. That's good. Now Craig, I feel like you and I have crossed paths a number of times in the past. And so this is not just a cookie-cutter question.

It's actually sincere. I I don't know. How did you come to Jane Street? What was that? Cuz I think this >> [clears throat] >> might be a slightly different answer from >> Yeah, it is a little bit different [laughter] because the company was pretty small and like people sort of joked that I was raised by wolves or something like that because [laughter] you know, it seems like you know, most people these days have come from some decent school and like you know, Uh, they knew each other from math camp or or you know something like that.

For me it was pretty um, random. In fact, I think or you could call this a very not random in a way. I only applied to one job ever basically. I just I saw a posting on the OCaml mailing list and I had made a resume just kind of for practice but mostly I was planning to start my PhD and then you know, I just thought, well, I'll send it in, you know, and the process was very different back then. Like I don't think I would get hired now.

Um, they only did like one phone interview, you know, and then I I came in in person because free trip to New York, why not, you know? And then I just end up liking the people I met and stuff like that and so I took the job and that was in like 2005 and here I am. >> Let me back up a little bit. So you're you're following OCaml a little bit. Are you able to just describe one, what that is and then uh, like why were you into it?

Why are you like looking at the jobs postings on this kind of obscure language? >> Oh yeah, I mean just nerd broadly, right? That's a baseline. So whenever I >> it for? Like what were the first projects >> projects, you know? Like I want to like graph a fractal or something or like, you know, when you just kind of like use a computer to do fun things, I would by default use OCaml, you know, made a little uh, a fellow bot, you know, if you made that if you know that board game like >> Oh yeah, yeah. >> just to kind of see how do you make bots for games, you know?

Um, so it's just stuff like that, nothing serious, you know? >> What do y'all do for fun? >> I would call your your physics lessons in the office hanging out. Like like either the power of graphs or something or what is all this stuff? >> Okay, yeah, every so often I give like an ad hoc like math or physics talk in a in a side room. >> Uh, >> What was the most recent ad hoc Alec talk? >> Uh, something about system mechanics and like we're talking about some partition functions and then we ended up on the topic of like how you could appreciate anything basically uh, as long as you you know, expand your mind enough. >> It was Feynman diagrams, wasn't it? >> Um, yeah, so we ended up so that then we were talking about how Feynman diagrams kind of relate to partition functions in statistical mechanics and like they're kind of they're very, very closely linked to things.

Um but but now you get into things like Yeah, e to the power of graphs and it starts getting a little a little odd. >> Yeah. Yeah, no. That's just such classic like Three Blue and Brown clickbait on like [laughter] e to the power of a graph >> Yeah, you should do that one. >> No, no, that would be fun. That actually would be fun. I mean, honestly, that was extremely enlightening, I think. Just I've I've chatted with you know, like Kenji I guess crossed your path before and Nithin we've talked before, but even then I feel like there's a lot of um question marks I have about Jane Street.

I think this helped clarify things. I think it'll also help just make it a lot more real for whoever might be watching this. So, let me just say thank you so much for taking the time. I really enjoyed this. >> [music]