Lecture 1 Part I Fall 2025: Introduction transcript

I wanted to just give a little bit of context about what we are doing here with machine learning and why the mathematical foundations are are so vital. And so I wanted to start with a a story actually when I was a graduate student um I was at Rice University and I was studying medical imaging. And so I went across the street to the medical school to talk with a collaborator and then in the elevator they had this sign that said if you would sign up for their fMRI study where you would learn how your b brain responds to different stimuli then you would get a CD with all of your brain data.

So I thought that sounded really fun and I immediately signed up. Um and it was super interesting. Um they had us lie in the scanner and then they would um have us drink either water or Kool-Aid and see how we responded whether it was coming in a you know regular cadence or whether it was coming at random. Um so I learned a ton through the whole experience. But a few weeks later, the professor who was running the study called me and he said, "When we were looking at uh the data for this experiment and we came to yours, we found um a menioma, a brain tumor, and he said, "You got to go get that clinically looked at." And I did, and they removed it, and it was totally benign, and I never had any kind of symptoms or complications.

So, I was extremely lucky. Um but I think this is an interesting story to think about today um more than 20 years later because AI and machine learning are transforming almost every aspect of that story. Uh, it's changing the way neuroscientists design their experiments, the way that we design scanners and how they acquire data, the way that we take that data and transform it into an image that a radiologist can assess, and the tools that the radiologist uses to to do their assessment.

It's also changing the way we decide whether a patient is a good um choice for surgery or whether surgery is a good choice for the patient. um and even the drugs and pharmaceuticals that are prescribed afterwards. And healthcare is not the only area where we are seeing um enormous work being done in AI and machine learning. People are exploring ways in which it might transform scientific discovery more broadly um predicting the structure of proteins, trying to understand the laws of nature and to make them some of them more quantitative than they are currently. um or understand the early seeds of structure in the universe.

We also seeing it being used we see it being used outside of the natural sciences. Things like trying to use these tools to understand culture and history supporting datadriven policy. I know many of you here are from the policy from the Harris School of Public Policy. um trying to support new legal paradigms and influencing um markets and uh economies. And along the way, we also face significant challenges as we think about AI being used increasingly or an increasing number of facets of our lives.

And that's where things like this class come into play. Really understanding the foundations of machine learning, including the mathematical foundations that we're going to be studying in this class is is vital. And trying to just deploy machine learning tools or build machine learning tools and deploy them without understanding those foundations is sort of like trying to build up biotech without understanding biology.

You might have some hits here and there, but you'll be nowhere near as efficient or as principled or um as efficacious as you will without that understanding. So, let's start with a really simple example to understand what machine learning is is doing. And I think at its its foundation, it's very similar to the way uh we learn things as a kid from examples. So imagine trying to train or teach a small child to tell the difference between cats and dogs.

So you would show them examples in picture books at the park. Um and you would point out which ones are cats and which ones are dogs. And you might also point out different features of the animals, things like how big it is or how pointy their ears are. And after a while, hopefully the child builds a mental model of what helps them distinguish a cat from a dog so that then when they're presented with a new animal, they can accurately classify it.

So with machine learning, we're doing something that's very, very similar at its heart. we are taking um a large number of example images and for each one we have features that we're now quantifying things like size and ear pointiness but we're assigning numbers to those features and we've also assigned a label to each one of the images say indicating whether it's a cat or a dog and then what we want to do is using all of that data build up a mathematical model inside the computer that will help us distinguish cats from dogs so that then when we are presented with a new animal we can make an accurate prediction.

Okay. So where does this come from? So in this really simple example that I'm showing you here we can think about it as the following. We could say I'm going to assign variable names to these different features. So for instance X1 could correspond to the size and X2 could correspond to the ear pointiness. And then our predicted label for an animal we could think of as being a weighted sum of those two different features after applying a threshold.

So we compute a weighted sum. We see if it's above a threshold or below a threshold and based on that determine what our predicted label would be. Now this sounds extremely simple. We're going to spend some time talking about how this works in a little bit of detail. It's much more it's much simpler than a lot of the modern machine learning tools, but it really is at the heart of almost everything that's being done in machine learning.

So, just to give you an example of how important this is, you know, when we look at this very simple weighted sum sort of example, it's only allowing me to draw linear boundaries, lines that separate the two classes. But if we're clever about how we treat our features, we can do things that are are more complex. So for example, imagine that I had um features that looked like this where the data corresponded to these two concentric circles.

This looks like a really poor fit for the linear classifier I showed you, right? No matter where I draw a line, I'm really not separating the two different classes corresponding to the two different dots. But if I do a simple transformation to the data. So for instance instead of having two features ear pointiness and size I've got three features now where the third feature is earp pointiness squared plus size squared.

Then all of a sudden it's possible to draw a nice straight linear boundary that separates the two classes. And so this basic building block that I'm describing can actually result in very sophisticated tools. And what we're going to do in this class is explore how you can actually do these things. And of course, this is related to things that all of us have experienced in recent years, neural networks. So we started with this very simple building block, but of course you can have many more than two features.

And instead of just having a simple threshold, you could have other kinds of nonlinear functions that we use to to process those weighted sums. And we'll explore what some of those are later in the quarter. But once you have that basic building block, then you can basically daisy chain these things together to form much more sophisticated machine learning tools. So if you've ever seen a diagram of a neural network that looks like this with all of these edges and weights and nodes, you can think about subsets of this is corresponding to these very simple weighted sums of features uh and then the application of some sort of nonlinear activation.

So what happens in practice? So, I've got a little uh demo here or animation from uh a website that or yeah, I guess website. They all their videos are on YouTube as well uh called three blue one brown. They've got a lot of really cool animations, especially for linear algebra that I highly recommend throughout the quarter here. Um, but in this case, here's a neural network, and what it's trying to do is to take in an image of a digit, a handwritten digit, in this case a seven, and output um an indication of which digit it actually corresponds to.

And so the different input nodes in this network just correspond to the pixel values of this image. So say zero for all the black images, one for all the white pixels and something in between for all the gray level pixels. So when we feed this in again, subsets of these nodes correspond to um uh the this very simple building block that I was describing. And the composition of all those building blocks then results in subsets of these different nodes outputting larger values as indicated by the white nodes.

So that at the end we're getting a white node corresponding to the digit 7. Okay, I know this is at a really high level but this is the kind of thing that we will be you will be able to understand in mathematical detail by the end of this quarter. And of course, the key question in all of this is how do we know which weights to use? Right? What I've been saying so far is that there are weights that can help us make good predictions.

And of course, training data is essential to figuring out what those weights might be. And so in particular, what we would do is we would have a collection of these training images, in this case, handwritten digits. And for each one have a label, in this case, digits from 0 to 9. And we would establish a a loss function which would tell us which collections of weights give us more accuracy on this training data versus lower accuracy.

And we would try to find the weight vectors that give us the highest possible accuracy or the lowest possible loss. So in this diagram, we're imagining there's only one weight we're trying to find. And the blue line is telling us which values of that weight give us lower loss versus higher loss. And our goal is to find the point with the lowest possible loss. And so we use optimization tools that we will talk about in this class.

Um, and the core idea is sort of akin to thinking about balls rolling down the side of a bowl that's maybe a little bit ripply. And as they roll down, they are going to land in different valleys of this curve. And so we're going to talk about how we can mathematically formulate that idea and how we use it to solve variety of machine learning problems. And we'll be doing it for more than one weight at a time. It's just harder to draw.

Now, I mentioned before that once we have this simple building block, we can daisy chain them to together to build much more sophisticated tools. Um and and this is true and one of the things that we see happening in fact is that when we train for instance neural networks to recognize images to be able to identify whether an image has a cat or a dog or the digit seven. Then eventually we end up with these networks that have different nodes, these colored circles that will um light up or have large positive values for different types of features that arise in the images.

And in this in this diagram what we can see what's happening for one example trained network is that in early layers of the network um we have different nodes lighting up or becoming active when an image has got certain kinds of edges or lines present in it. And then when we get to middle layers we see that those nodes are representing somewhat higher level features. So things like um eyes and noses and lips in the context of facial recognition or wheels or doors in the context of vehicle recognition etc.

And then when we finally get to almost the final layers closer to the output where we're having to correctly identify what the class of the object is, we see very high level features like full faces or full vehicles etc. And this is what enables us to do much fancier machine learning tasks than simply classifying each image as either a cat or a dog, but to more in a more nuanced way identify the content of of images.

And these same kind of tools are being used uh in in different ways in order to do things like um generative AI. So for instance, generating images. So if you've played around with things like um Deli or other commercial products, many of them are using things like diffusion models. And the idea is the following. Given a nice Oh, yeah. >> I I'm really sorry. I just can't hear you. Can you say it louder? >> Yeah. computers. >> Yeah.

So the question I think tell me if I'm wrong is well what happens when the image that we feed in is is corrupted and so um or is something that we haven't seen before. So I think those are two different problems to some extent. So if we were to train a network and then wanted to be able to say recognize images when the images are suddenly very noisy, right? Like maybe it was taken in very low light conditions or No, that's not what you mean. like sometimes. >> Oh, I see.

Okay. So the question is uh more about um adversarial settings where maybe I can feed in an image of a dog and it correctly the network correctly identifies it as a dog and then I add a small perturbation to the image maybe just like a tiny little change to each pixel value that's imperceptible to the human eye but all of a sudden I feed it into the network and I get it classified as something totally different like a house instead of a Um, so this is um, yeah, considered adversarial because I'm definitely trying to perturb the image in a way designed to fool the neural network.

Um, that's not something that we are going to be studying in this class. But people are using optimization tools to do that. They're trying to try and find the optimal uh, perturbation of the pixels in the image that will most fool the neural network. Um and so some of the kind of core mathematical tools we'll cover are used for that. And then at the same time we have people who are trying to figure out better ways to train neural networks to make them more robust to those adversarial attacks.

And so um one way you can think about it is I talked about how when we're selecting the weights of a neural network, we're trying to minimize a loss function. And so people will think very carefully about how they can choose that loss function in order to provide that robustness to attacks. Um again that's not something that we'll talk about in detail in this class but again the kind of core mathematical tools that we'll be covering are are very relevant here.

Yeah fantastic question. >> Yeah. Could you go back to the image where you show the networks of >> how was how was it able to continue to tell what are what are the nodes or what are this part of the network how is this >> how is this image generated >> how do you understand that this part of the network is showing >> okay that's a Great question. The question is, how do we know what parts of the networks are showing what parts of the the features?

Um, this is maybe a little bit hard to explain without any of the material of this class. Um, so I'd rather talk about that offline. Um, but I am going to post the slides and there is um a link or a reference that goes into some excruciating detail on it as well. Um, yeah. So, thanks for the question. It's a great question but I think it's a little bit hard to explain without having any of the background of this class yet.

Yeah. Okay. So, um I had mentioned that similar tools are also used within the context of generative AI. Um and so one class of image generation tools are called diffusion models. And the basic idea is that if I were to take a nice clean image, say of a dog, I could gradually corrupt it more and more by adding noise or random perturbations to the pixels. Um, this is a very easy thing to do. It's always easy to take a a good image and make it worse.

Um, but then what we could do um with a neural network is train that network to take a noisy image and remove a little bit of that noise. And so the reversed process is kind of gradually removing noise or removing this this distortion in order to produce an image. And so with um image generation with diffusion models, we would start with a random image just full of random pixel values and then we would feed it through a dnoising neural network over and over again hundreds of times in order to produce an image that looks clean and like an image that's similar to the images that were in our training set.

So here's a little illustration of this in action. So we've got a collection of different images that are all going to be generated in parallel here. So each little square corresponds to a different random starting point and then we're just going to be feeding each of them into to this network. And you can see as it proceeds it gradually removes noise and yields images that look like reasonable kinds of candid images you might find in different places.

The other form of generative AI that I'm sure you all have encountered are things like large language models. And so one way you can think about this is that we start with a prompt. Say the cute dog beg begged for a and then we would like the large language model to provide the next word in this sentence. And so what we do is we use neural networks to effectively tell us or predict the probability of each possible next word based on the past words in the prompt and then draw words from this distribution.

So if our dictionary contained words like bone or spaceship or ball or martini or necklace, then with high probability the next word is bone or possibly ball, but the probability of it being spaceship is very low. And so once we know what these probabilities are, then we can sample from this distribution. And so with high likelihood, we're going to select bone. But there is some small nonzero likelihood that we would generate or choose the word spaceship.

And this is what allows these models to produce unique and different text every time you might run it. Okay. So these are some of the ways in which machine learning are used that um you may have already encountered. But I think we really need to also think about some of the challenges presented by AI. And I think this highlights the importance of really understanding um the mathematical foundations. So in particular um when we train AI models or machine learning models it requires huge quantities of data for the most part just billions of images trillions of text snippets and this can be challenging to even just choose a good data set.

So in this little toy example I've got a bunch of images of cats and dogs. All of the cats in this data set are indoors. All of the dogs are in the grass. And so this is not a good data set because there's this strong correlation between the class I care about dog and grass even though in reality grass doesn't tell me anything about the type of animal. And so I could train a neural network or other machine learning tool on a data set like this and then when presented with a new image make a completely wrong prediction.

Now, this feels a little bit trivial because we can immediately look at this data set and see that it is not really reflecting the full diversity of situations in which we might encounter cats or dogs. But it can also be much more subtle. So, for instance, imagine that we wanted to build a machine learning tool to distinguish between chest radioraphs representing patients with pneumonia versus non-pneumonia. um we can have similar kinds of spurious correlations represented in these data sets that can result in false diagnosis or or classifications.

And this has been observed. People have seen where neural networks will home in on features like you know what was the data acquisition protocol. So maybe if a doctor thinks you have pneumonia they put you in scanner A and if they think you don't have pneumonia they put you in scanner B. and you don't realize that, you just take all of the test images and then you have these spurious correlations um that can really impaired your machine learning system.

Um and so a key question that we're constantly having to grapple with is whether our data is sufficiently diverse for making important high stakes decisions in healthcare, in finance, in in defense, in a number of different regimes. And I also want to emphasize that you know uh sometimes when we face these sorts of challenges we tend to um talk about whether the data is sufficiently diverse. But often we have to take a step back and think about the whether the way we even set up the problem in the first place is well thought out and welldesigned.

Um and so one of um my former Booth colleagues did a study um a few years ago where he found a major healthcare risk algorithm um exhibited some significant racial bias. And the team that designed it was was surprised by this because they had made real efforts when they were choosing their training data to make sure that there were um I think roughly equal numbers of uh in this case black and white people. Um so they were trying deliberately to avoid the kinds of data um lack of data diversity issues that I was describing in the previous slides.

The problem came in with how they even set up the problem because we not only need the training samp um features but we also need the labels. So they were trying to predict healthcare risk but they didn't have good labels for risk. So instead they used as a proxy healthcare expenditures but those are not the same and the way that different communities access the health care system are very different. And so by focusing on health care expenses instead of health care risks, they ended up with a system that was was deeply biased.

And so this is something that's a little outside the scope of of this class. Often we're going to assume we're presented with the data. But I think it's just worth emphasizing upfront that we should always be questioning where that data came from, how it was set up, and what we're trying to do with it. Um another way in which mathematical foundations are can play a key role is in providing privacy guarantees. So um in early days people would try to protect the privacy of people represented by a data set by aggregating samples.

So instead of having a database with each person's data, they would maybe aggregate groups of 10 people and just show the average for each group of 10. This sounded good, but it turned out that unless you did had more conditions, it was very easy to break this and violate people's privacy. But using mathematical tools and statistical tools, methods like differential privacy came into play where we could randomly perturb data in a way that would give us guarantees on the amount of privacy that we were getting.

Um, of course there's an inherent trade-off between how much privacy we are prefer preserving and how much um accuracy we can have in our machine learning tools, but at least with understanding the mathematics, we were able to really understand those tradeoffs and make sure that we weren't deploying tools that were easily breakable. I also want to emphasize that um statistics plays a major role in a lot of machine learning.

Again, this is a little outside the scope of what we'll be able to cover just in this quarter, but it's worth mentioning. We see statistics being used in the way we think about sampling words or tokens in large language models in trying to think about fairness and different types of fairness or measures of fairness and how we can deploy those in machine learning methods. Um trying to quantify uncertainties associated with our predictions. um preventing overfitting which I have illustrated in my my graphic and also trying to uh understand distinctions between correlation and and causation.

Okay. So with that um I I I guess just in closing here want to emphasize that I've talked a lot about neural networks. I think that's because you've all been experiencing the neural network tools in your daily lives more and more. But we're also going to talk about other methods that arise in machine learning. One for instance is recommener systems. So this is used for instance by Netflix to try to predict how much you are going to like different movies.

So they've got a whole collection of different movies and a whole collection of different users and each user or customer has watched a subset of the movies and indicated whether they liked it or not or back in the early days even giving it a star rating indicating how much they liked it. And so what they would like to then be able to do is to take this this table and fill in all of the missing entries. And this is where machine learning tools come into play.

And in particular, what we're going to actually study in some detail in this class is that the the core idea behind uh their predictions is that there are certain kinds of representative movie watchers. And we after the fact can attribute properties to these movie watchers, though this is not explicitly what's happening in the algorithms, but we could imagine there being a movie watcher who really loves romance movies and another one who loves comedy, another who loves action and another who loves horror.

And so we could imagine that we know for each of those movie lovers which movies they prefer and don't prefer. And then we can think of each individual customer as being a weighted combination of those quintessential movie lovers, right? So maybe maybe I really like romance and comedy and have no use for action and horror and somebody else really loves horror and comedy for instance. And so if we have this model then we can say well our task is to figure out what these representative movie watchers look like, what kind of movies they prefer.

And then for each customer which ones which representative movie watchers they most resemble and we use our observed data to to do that. And once we do that then for a new movie we can make uh a prediction. And this is all based on something called a subspace model. and we're going to talk about subspaces right in the first couple of weeks of class here. Now, the final thing I'd like to emphasize before we get to to the syllabus here is that um when we talk about machine learning systems, there's a lot involved.

I think a common misperception is that machine learning is really about being a good coder and and coding can certainly help. But it also involves a lot of domain data and expertise figuring out what data we should use, how we want to set up the problem, how we want to choose our losses and objectives. And it depends heavily on the mathematical and statistical foundations which you're going to start to get a taste of um in this class and trying to really transform AI or make AI have all these transformative outcomes that we highlighted at the beginning of this lecture without that domain expertise and without those foundations is is frankly impossible.

So let me just give you a couple of examples. Um so how many of you have heard of Alphafold? Anyone? Okay, a good fraction of you. So, this is an AI tool that can predict the structure of proteins. And this is really important for instance for designing different pharmaceuticals. Um, and you know, this was a problem that people have been studying for decades. Um the protein datab bank had a huge collection of different protein structures that scientists had assembled over decades and that combined with different mathematical tools for machine learning led to alpha hold which has had enormous impact.

Um they're thinking that it's just really going to transform the drug discovery industry. Um and in fact the lead one of the lead researchers for Alphafold is a Chicago alumnist. So we're very very proud of him and and his work. But we see similar things arising in a variety of settings. So we see people coming together with computer vision systems together with mathematical and statistical foundations to advance things like precision agriculture or autonomous vehicles.

And again these foundations that we're talking about played a key role. Um people have been studying autonomous vehicles for a long time. There was this very famous DARPA Grand Challenge 20 years ago now where different teams were competing to design autonomous vehicles that could win um a a race on a track. Um and we've come a long way in that 20 years because of these these sustained investments. Um I mentioned medical imaging earlier.

Again, mathematical and statistical foundations are playing a critical role in designing new tools. When we think about um protecting our GPS systems from space debris, this is requiring not only domain expertise but a variety of different mathematical tools. And when you see AI being used to advance weather forecasting, again, core mathematical principles as well as decades of research in numerical weather prediction are really coming together here to make this possible.

So what you the things that you are seeing are way more than sophisticated coding.