696: Brain-Computer Interfaces and Neural Decoding, with Prof. Bob Knight

This is episode number 696 with Dr. Bob Knight, Professor of Neuroscience at UC Berkeley. Welcome back to the Super Data Science Podcast. Today we are graced with the presence of a world-leading cognitive computing researcher, Dr. Bob Knight. Dr. Knight is Professor of Neuroscience and Psychology at UC Berkeley, he's also an adjunct professor of neurology and neurosurgery at UC San Francisco. Over his career, he's amassed tens of millions of dollars in research funding, 75 patents, and countless international awards for neuroscience and cognitive computing research. His hundreds of papers have together been cited over 70,000 times. In this episode, Bob details why the prefrontal cortex region of our brains makes us uniquely intelligent relative to all the other species on this planet. He talks about the invaluable data that can be gathered by putting recording electrodes through our skulls and directly into our brains. He talks about how brain-computer interfaces, BCIs, are life-changing today for a broad range of illnesses, and he talks about the extraordinary ways that advances in hardware and machine learning could revolutionize medical care with BCIs in the coming years. Alright, let's jump right into our conversation. Bob, welcome to the Super Data Science podcast. I understand that I am in a really distinguished company to have you as a guest on my show. You've done two podcast appearances before. One of them was with someone that probably most people in America know, Ira Flatow, who does National Public Radio here, and then the other one is a lesser-known podcast host, some Joe Rogan. I don't know how you pronounce that. Rogan. Okay. Yeah, yes, Joe Rogan, yes. So, yeah, so amazing to have you on. You were referred by our recent guest, Mattara Holler. She was an episode number 683. She was a PhD student under you. We've actually had other former PhD students of yours, Bradley Voitech, who was an episode number 253. And I think he's going to... He's so fascinating that he's someone that will probably have to have on the show again soon. But yes, so thank you very much for coming on the show, Bob. Where are you calling in from? I'm calling in from El Sredo, California. The Little Hill. And so, you are renowned for lots of different threads of research. A lot of its centers around the prefrontal cortex, which is a part of our brain. So, maybe if you could start us off by explaining the different brain regions that we have, the kind of the main areas that we have that we can separate out in the brain. Sure. I mean, I think the first broad past is you separated out the neocortex, which is basically infolded with gyri and sulci. And if you took it out and unfolded it, it would be about the size of a medium pizza. And that's loaded with cells about the medium, not large, medium. Medium, medium, and that has broken down the cerebral cortex into multiple areas, many of which have a direct link to animals, all your sensory cortex, touch, hearing, vision, motor. Those are all your primary areas. And then you have areas that are higher order. Where you do a little bit more sophisticated analysis. So, for instance, you, something hits your retina visually, and then you want to decide whether it's a car or a face, or that gets done in even higher order cortices. And then you have, of course, language organization, which is quite critical, that 98% of right handers have language in the left hemisphere. Males, females, it's a little less, it's about 92%. They tend to have more bilateral representation of many things. That's why you get, many people get damaged in the left hemisphere language areas, you have problems either understanding or speaking. When you continue to move up the food chain into what I think is, well, I don't think it's the most highly evolved area in mammals, and that's the prefrontal cortex. That's a very large part of your cortex. It's roughly 35% of your cortical mantle. The next closest is gorilla, which is much less. And the frontal cortex also has many more two-in-froke connections to other brain areas. So, there's extensive connectivity. So, it's constantly receiving information, doing something with it to make a decision, or maybe not, and then influence other brain areas. And the frontal cortex is actually importantly divided into two broad areas. One is the lateral frontal cortex right here. And the lateral frontal cortex, if you want to just take a first pass, you say it's cold cognition, thinking, planning, attention, working memory. But then you have the orbital frontal cortex, which sits above your eyes. It's a very large chunk of change. And that's more related to social and emotional processing. And of course, what you really want in the perfect world is you want to have to interact, right? You want to be able to take your social and emotional cues, and use them to some degree to influence your decisions. Now, obviously, if you're doing a math exam, you don't need social and emotional. But there are many other things in life that need to have a blended interaction between these areas. Things move around real quick, you know, in the brain. You get information from the back to the front of the brain in 15 or 20 milliseconds, in these large, you know, pathways. You have to know with that, then you have a whole other area that all animals have, which is that the basal ganglia, the lower areas that, for instance, are impaired in Parkinson's disease, and still lower, you have the brain stem, or you have more physiological things related to eye movements and things like that. And other areas in the hypothalamus related to home body, homeostasis. So there's this gradient, and it just goes up and gets more expansive in the cerebral cortex, 100 million neurons, untold numbers of connections, maybe 10,000 neurons, you're up into the trillion range of connections in the human brain, which allows us to be fast really quick, right? Moving stuff around really, really quick. Yeah, and so we, so that the brain stem, the older structures of the brain, further down, these are kind of something that we share with much more of the animal kingdom, amphibians reptiles, but then it's not get out, et cetera. When we get out to the cortex, you were mostly describing there the neocortex, the medium pizza-sized piece, the outside. That is something that, you know, only relatively recently evolved animals have, and it's interesting that humans, as you mentioned there, are prefrontal cortex that you specialize in is the part of the brain that we have proportionally more of than any other animal in the planet. And so at least from a, in a gross, neuroanatomical sense, is what makes us human. I mean, you could make that inference, I think, in many, in many, I actually believe that. I don't know if I could prove it to you, but, you know, for instance, but it's logical. For instance, there's tremendous research in mice, wonderful stuff on this, that, and the other thing, opogenetics, et cetera. But the cerebral cortex of the mouse is 0.8% of the cortex, and humans, it's 35. So there's a big, as I mentioned before, you don't, you see a lot of research on gambling in mice, but I've never seen a mouse in a casino. See a lot of functional humans in casinos. So I do think, I mean, understanding what the prefrontal cortex does, how it orchestrates behavior. It's very salient to how we're interacting. We're actually forming a, you know, inferring what you're going to do. And, you know, we have a, we have a relationship, but it also goes awry and untold numbers of neurological disorders. You just, you name a traumatic brain injury, oral frontal cortex, impaired social regulation, lateral prefrontal cortex, brain tumors, you know, you name it degenerative diseases. So understanding it, the neurodevelopmental disorders, I think the more we learn about anything, the more likely we may be able to not only understand it from the person's perspective that they're being affected by, but maybe even lead to remediation or therapies or things that could help them. So you got it. And then if you forget about the neural part, tumors, strokes, degenerative, trauma, switch to psychiatric disorders, lateral prefrontal cortex, schizophrenia, oral prefrontal cortex, OCD, right, anxiety disorders, impaired regulation of the amygdala. So you could basically link suicides, been linked to even genetic deficits and lateral frontal functions. So the understanding it extends outside of the pure neurological and neurosurgical to psychiatric and developmental problems. Right. So if people have brain damage in an auditory part of their cortex, they're going to have difficulty hearing. If they have damage to a visual part of their cortex, they're going to have difficulty seeing. If they have damage to the prefrontal cortex, it's going to manifest, I guess, as typically, much more complex effects. Like, you know, they'll be able to see, they'll be able to hear those, you know, those senses won't be affected, but their ability to attend effectively or, yeah, that's probably lots of, depending on exactly where in the prefrontal cortex the damage is, there's lots of these different kinds of complex behavioral effects. Yeah, absolutely. There's a cordal, rostral gradient in the lateral frontal cortex, from simple sensory motor to a more, as you mentioned, attention working memory, and in the most frontal, most anterior rostral parts, problems with, or, you know, higher level reasoning, basically, and abstract thinking. So even it's not a homogenous structure, there's there's subregions. Yeah, I would think, you know, I think you should, this is a, we should, you mentioned, you know, the sensory areas, I think you should think of the frontal lobes, the orchestrator of activity. Right, right, right. Basically, taking in things and making decisions about A, are we going to keep it? Are we going to remember it? B, do I have to act on it? If so, what's my plan? And how do I implement it? And I think the other thing that to me is, in some ways, the most fascinating and important, it allows you to go offline. So what do I mean by going offline? So John, I'm pretty sure you could go back to your last birthday party that people have for you. You could time travel back in space, reconstruct something that happened. You could come back to the present where we're having a discussion, or you could go to the future and be thinking, gee, in three days, I have another important interview with somebody. And this is real, this ability to time travel, and it's really unique to humans. I mean, you can go offline back and forth. The other thing you can do, and maybe animals have this, you can go from external attention, us, or you can basically go to internal attention in the popular vernacular, you can mind wander. So who knows? Maybe some of your listeners now have had it. And they're just zapped out and they're into their internal space. All these things are depend on the medium pizza they're going to order. They'll never, they probably won't be able to look at a pizza the same way ever. And by the way, I'll give you the plug for that. How did I learn it was a medium pizza? The brilliant Marion Diamond, one of the most wonderful neuroanatomist ever. There's a movie on her, my love affair with the brain. I mean, just a wonderful woman. But anyway, she pointed, I used to do brain dissection in her graduate class, and she tuned me into pizza. So I mean, I guess you could literally like in an onic cadaver, you'd just you'd take it out of the skull and lay it out on a table and try to get it as far as you can. Oh, you couldn't, you couldn't, you couldn't, you can only flatten it now with with imaging techniques where you can take because it's included. But you, you know, you flat map, you can easily flatten map anything now with, with, you know, machine learning tools that have been, you know, fabulous ways to manipulate data. But I do have, I mean, I have in my office, I have to be on the more less digital side. I have 75 post-mortem brain specimens of diseases that I use for teaching. Exactly. It's exactly the same number of patents you have, but every time you get a patent issued, yeah, you go out and get another brain. No, I haven't been collecting it for 20 years now, but they're really good because you actually see what a stroke looks like. You see it physically and there's something about, you know, it's a little different than seeing a picture. Anyway, yeah. So, so yeah, so we were talking about the prefrontal cortex. It sounds like it's, yeah, this orchestrator of the brain. It allows amazing human capabilities like the time traveling that you're describing, the mind wandering. It allows us to decide what kinds of things we want to remember or focus our attention on. And you alluded to there one way already that we can be studying how different parts of the brain work, including how the prefrontal cortex works, which is studying brain damage patients. So we call that neuropsychology and that's something that you spend a lot of time on. And so that's, and you've already given some examples of how damage to different particular areas of the prefrontal cortex cause specific effects. Another area that you specialize in, is intracranial recording. So we're, you know, you're not studying from brain damage, you're not studying from recordings from outside the brain. So we have various non-invasive methods, electroencephalography EEG, magnetoencephalography, MEG, MRIs, functional MRIs to get a sense of what a lot is being used in real time. But all of those non-invasive ways, they're quite coarse because you can't record from individual brain cells from outside the skull. But you do intracranial recordings. You specialize in that. So what's special about these intracranial recordings, what different kinds of intracranial recordings are there, and what unique insights do they allow? Yeah, the, well, why do we even do intracranial recording? It's because the kind of work we do, it's all disease related. So if you just think of epilepsy in the United States, there are about 1% of the population has epilepsy. I'm sure if you talk to your readers, your people, your people who listen to your podcast, many of them know somebody who has epilepsy. Mercifully, most of them are treated with medications, but a lot can't. And the ones who can, which are in the range of, you know, if you think we've got about three and a half million epileptics, about 15% are not medication responsive, intractable seizures. Those patients, unless it's a really bad genetic disease with a lot of bad brain abnormalities, you can bring them in, and then you target putative areas that are seizure onset, and you insert the electrodes. Now we insert them under robotic control with precise coordinates. And now we take them off meds, and we want them to have three stereotype seizures. Once they've had three stereotype seizures originally at the same spot, boom, they're going to go to surgery. But that may take some time, and the patients are really incredibly cooperative in terms of wanting to participate in research, although they know it's not doing them any good, but you know, probably unlike a lot of our politicians, they actually understand what value of research. I know my mother understood it because of the SOC vaccine when, but I was a kid and, you know, SOC was a God because, you know, they got it. So anyway, they're in the hospital for maybe a week or so, and then we'll do experiments with them where we'll have them to attention experiments or social experiments or language. There are many many things that we've done over the years, and the beauty of it is you're right there. You're not outside the head, you're precisely picking up the electric field in a small brain area that are associated with the behavior. And because you have typically the typical patient has about a hundred to 130 contacts in distributed around the brain, we can now look at information flow, how the networks are connected, how they might support different different aspects of cognition, and more recently in the last few years, we've developed, when I say we, not me, but the field is developed techniques to record actual single cell responses, in other words, the precise firing of one cell, which provides a very nice link to the wonderful world of basic science in non-human mammals. So that's what we do, and it's exciting because you're touching the brain, basically, and you're getting real-time data. It's not, you know, FMRI's wonderful, but it's got a lag. Blood flow, five seconds. When I clap, the auditory cortex, 12 milliseconds. So it's a different temple domain. It's not as elegant as FMRI in the sense we don't have whole brain coverage. We have specific targets, and the researchers might live in the other research, we don't pick the targets. We have nothing to do with, hey, could you put an electrode in area A because I care about it, we just go wherever the docs, the epilepsy docs decide they want to monitor. But you get a lot of data. We're now over 450 patients we've monitored, and each patient has probably done five, six experiments. So there's a lot we've collected. A lot of data has, as have many other great labs. It's kind of exploded. If you can do it, you're going to do it. It's just too exciting, basically. And it really, also because the signal on one trial is very strong because there's a thing, there's a frequency response in the brain called high frequency activity. It's 70 to 150 hertz. You can't pick it up on the skull with EEG, but it's reliable to single trial. So if I say dog, you get it. I say tree, you get it. I say cat, you get it. Because it's reliable to single trial, it's a powerful signal for neuroprostetics. I don't have to average 20 stimuli to know the patient wants to move their left arm. I could do it on a maybe single trial basis. So it really is a nice window into neuroprostetics. Wow. Okay. So you have to get these 130 contacts, these 130 recording electrons to the brain. You have to, there have to be holes in the skull. Yes, you have to. But then because of the robotic placement that you just described, I guess you don't need 130 holes. You need some. No, no, no, you don't have 130 holes. Well, the two thing, about a 20% of the cases, we do, we do what's called electric corticography. The electrodes are only on the medium pizza, basically the cerebral cortex. But probably 80% of the cases, there's a good chance that we think the seizures are coming from a deep structure, the hippocampus or the amygdala or the overall frontal cortex or the insulin, some deep brain structure, and they get stereotactic electrodes. Now, when you do the ECOG, that's, that's a little rough on the patient in the sense we have to do a craniotomy, take a piece of skull off and put the electrodes on and then re-close it. The, this sounds a little bit odd, but the, in the stereo EJ, where we put in typically, let's say 12 electrodes targeting different areas, each electrode with 10 or 15 contacts, they only have a two millimeter hole in the skull. Nobody wants a home skull, but two millimeters is not big, and then we stir it practically in plant, and they have very little post-op problems. Most people don't have any post-op problems. They're really pretty, you know, whereas in the, the people we have to put the big grids on their surface, you know, there are a lot of the data can be, we can't really do well with because they need narcotics for pain. But anyway, yes, so we don't do 130, we do maybe 10 or 12 electrode tracks. Right. And what does that mean exactly, stereotactically? A stereotactic just means it's like anything where you're kind of picking a, let's say wanting stereotactic radiation. You know, someone has a pituitary tumor in this, your MRI would localize it, and then you put your radiation beam in to target. So we're not putting anything in, but we're stereotactic in places and then pulling out data. The other big area which we do some work in while we really collaborate with people who do the hard work is people with movement disorders, you know, Parkinson's disease. They also get electrodes implanted, but in a limited area brain, I get it into a couple areas of the brain that you know if you shut it off, you improve movement. And that's substantial, Niagara, I guess. Not that substantial, Niagara, but in the sub-tlanamic nucleosoluis, it's a very small structure. It's only about that big. And the globus pallidus internal, it's an interesting story because neurologically, if you get a little teeny infarct in the sub-tlanamic nucleosoluis on the right side, the patient gets flinging movements of the contralateral body side. They become hyper-canetic. And someone put two and two together and say, we want to study this nucleosolagically. It must be inhibitory, you know, to the motor system. Turns out the animal researchers figured it out and then a group in France came up with, okay, we're going to put an electrode in and they call it deep brain stimulation, but it's not stimulation in the way making it better. You actually put a high frequency, 130 hertz stim in or 50 to 130 and it shuts off the area. So shutting off the sub-tlanamic nucleosoluis improves tone, just like putting a stroke in it makes you hyper-hyper-canetic. It's an interesting story, a clinical observation to animal physiology, to a device which now, I don't know how many in the US, but I'm sure it's in the tens of thousands of people benefit from treatment for Parkinson's and there'll be more, they're getting better, you know, they're using all kinds of fancy, you know, techniques and machine learning and, you know, used to be that you'd come in and get adjusted by the dock every so often and then they let the patient adjust it. But now we've come up with algorithms where you, you can actually monitor in the brain your motor state and then basically control your stimulator whether it should be on or offered to what degree. So again, this is this, you know, you see these papers that every, what's the right way to say this? We're just an infancy of BCI, brain computer interface, assisted devices. Same thing can be said for the, I'm sorry, go ahead John. Oh yeah, no, that's well, I was just going to say like let's get into, let's get into that. So we talked about how you can insert these contacts, you might have in the stereotactic case that you're providing there, you might have 10 or 12 of these two millimeter holes in the brain, you insert around 130 contacts, you can do recordings and I imagine some of those recordings that you're doing when you're, and tell me if I'm wrong here before we quickly, before we get to the prosthetics and the BCIs, which we'll get to right next. Sure. When these people are, I guess you're waiting around for them to have seizures and so I guess in that time it provides you with a window as long as they have the energy to be running different experiments. Maybe, does that kind of thing happen? That's exactly correct. And you know, I mean, as I said before that, you know, people are smart. They actually, they know research matters and they cooperate. But sometimes they can't because they, they've got pain and they need narcotics or something or maybe they get really lucky and we admit them, we take, and we got the electrodes in in the first 24 hours. They have three or four seizures that perfectly localized. That's it. We're not doing any research. We're not going to have them hang around. They're going to be exploited and go to surgery. I mean, so it's, and it's very effective. It will eliminate seizures in many patients and get control with medications in just a very large percentage. So it's heavily underutilized right now. It's going to be more utilized. I'm sure there's always been this fear of, oh, brain surgery, you know, but, but it works. It works really well. And so I guess in your case, with a lot of interest in the prefrontal cortex and these higher-order functions, these orchestration functions, I imagine you might be running tests that evaluate, you know, some kinds of attention tasks and then you're recording from these 130 contacts and monitoring how the tasks impact the activity of different contacts. How the information flows, how area A sends information to be. There's all kinds of, you know, fantastic new developments in machine learning and, you know, information flow, metrics, directed range of causality. I could go down a long list of things. But yeah, you're, you're trying to, not very few things happen in a spot, right? You know, your, your motor system, yeah, it's pretty clear. There you move. But higher-order things tend to be more distributed and they engage many brain areas. The beauty of it is most of the things happen so fast you don't know what's happening. So you don't get a conscious perception for about 300 milliseconds. These things are done. You don't even know what's happening. You don't know that when I put this glass up, you've figured it's out of glass. It's a glass. And you've done it in about 170 milliseconds. Boom, done. So a lot of these processes, thank God, are, if you had to think about them, you'd get, you'd go crazy. Probably, you'd be superior. They just move. Yeah, so the example there is that like the subconscious processing that allowed my brain to identify that there was a glass. So I might even, I might react to a threat of like somebody in the street pulling out a knife before I consciously see that somebody's pulled out a knife. You very well made because there are visual inputs to your amygdala that I get there in 80 milliseconds. And the amygdala is as I'm sure you probably know in many of your guest snow is really, really likes to respond to fearful stimuli. So yeah. And it's regulated by the frontal lobe, but it'll go on its own just to give you an alerting signal. Yeah, actually the frontal amygdala network is often people with anxiety disorder. Another normally something happens that it's going to make you a little anxious, right? But most people can't suppress it and get control. What happens is you get, you don't get proper down regulation of the amygdala in people with anxiety disorders. Really fascinating. And we can no doubt talk about these kinds of cognitive flows, these studies at the single cell recordings, just the insights that we get from them for hours and hours. But I want to jump to the prosthetics, the brain computer interfaces. So tell me about what these are. You know, we hear things like neural link that Elon Musk has set up as is this private company for having brain computer interfaces. So the idea here is that you're using your naturally occurring neural activity to be able to control some kind of external device or computer. Yeah, listen. One of the most widely used and most important BCI devices is basically cochlear implants. So maybe we could start there. Okay, so a cochlear implant, right? Think about it for a little bit. You know, you got 64 contacts that are going in now, not 25,000. And when it first goes in, people here are like murky. And somehow your brain amazingly takes those limited number of contacts and is able to turn it into understandable speech. What a remarkable plasticity. Yeah, just crazy. So you're saying it's like the natural brain has 20, naturally, you're born in a thousand years. 25,000. Yeah. And we then, but just with 64 contacts. It's unbelievable, actually. But then, you know, the, and then we already discussed so I don't want to go back to it, movement disorders and, you know, the treatment of movement disorders, the other, you know, the, the other field is in general neuro prosthetics to replace a lost function. Right. So the first and people, really people shot for was motor because there's something we can do to restore function and people who are paralyzed, right, that have had a high spinal cord injury or some other, you know, misery and a lot of work has been done on trying to decode signals from your motor cortex. And, and the, the key thing in most of these devices, it's, it's really pretty simple. The area that's active when you do something. So I'm going to squeeze my right hand. Okay. As I squeeze my right hand, my sensory motor cortex is active. Now, if I imagine squeezing my right hand, my sensory motor cortex is active. Similarly, if I show you our, our old buddy, the glass, you'd see the glass. And if I take it away, you can imagine the glass. Right. Yeah. I feel pretty confident now that you've taken an off screen or, you know, our, our viewers who aren't watching on YouTube, they want to have actually seen the glass. But I feel like, you know, you've taken it away from my field of view and I feel very confident. I know what it still looks like. Yeah, you can, you can reimagine it. The same thing you can reimagine the smell of a rose. You can, you can, you can imagine a sensory input tree, baseball. So this is a key principle of neuro prosthetics. The fact that imagining something produces a signal that parallels the actual signal that you get from driving the system with the auditory input or the output of the motor or the smell or the vision. That's a key underlying principle. So, you know, many people are going for motor system. I don't know the details of Elon Musk's operation. I know they've made some nice advances in high tech implantation of high density electrodes in the motor system. And that's great. I'm not sure about the other goals of changing cognition. I think that might be, might be a different issue. But I do, you know, but versus in the air, I'll just pick an area that I'm somewhat familiar with, which is trying to decode people's thoughts, not so much their high level thoughts, but their speaking thoughts. So you have, you've had your Stephen, Stephen Hawking, you got a myotrophic lateral sclerosis. Your brain is a gold mine, but you can't communicate. You have what you want to say, or someone who's had a stroke and can't speak, or they're just a whole range of miseries. If I can decode from your brain, your thought that you want to say, I'm hungry. I love you. Think about that. It doesn't seem like a lot, but to a person, it's huge, basically. Just like in the motor control prosthetic literature, with a brain signal, put it to a robotic arm and you can now pick up a can. You say, oh, gee, that's, that's pretty cool, I guess. But guess what? To that patient, that's control of their life. So it's very, very cool. It's just like the end of a prison sentence. Yeah, I can't, I can't really emphasize how important this research is. Because in the end, you know, you want to do cool things and you have cool papers and you have cool findings and you do all kinds of fancy math. But in the end, I think you want to be able to do something that's going to help somebody. I mean, that's, so what we've done, and I don't want, most of our work has now been, we've moved out of this area and moved into mainly cognitive decision making research, but we started. So just quickly there, so moved out of the prosthetics stuff, so more than, you know, controlling external and going into more cognitive as opposed to physical. Yeah, because I, I mean, I'm not getting any younger and it's quite clear that you're frontal load slowly deteriorates with aging. So I got to figure out a way to tune that baby up. And a lot of our prosthetic work that started in the lab has been, been moved to UCSF and Eddie Chang's lab, which is doing spectacular work. But we started here. And the first thing we did was put, have electrodes over areas in the brain that we know that if you damage it, you can't understand. You have a one of these aphasia, you can't understand what someone's saying to you. It's a very classic syndrome. And basically, we presented words to 100 words to patients with these electrodes. We recorded their EEG and we came up with different models to try to fit the EEG signal from each electrode to what they hurt. So think about it this way. You're watching someone playing the piano. And you're, you understand the piano, you know what the keys are and they're hitting the piano keys, but you don't, there's no sound coming out. But you know what each key represents. You can reconstruct in your mind what that person's playing. That's what we're trying to do. We're trying to assign each electrode to a specific frequency or whatever component of audition and then have all those electrodes combined to produce the sound. So we did that. We held out data, you know, standard hold out and we found out that we could correctly classify words at a 92% rating, which we were just semi-dumfounded. And I remember saying to the postdoc, make a sound file because we had the word to person heard in the word that was reconstructed. He said, well, that's not science. And I said, I remember I said, you want to get a job? So that started it. But that's cool. It's nice, it's powerful, but it doesn't help the patient, right? So you're saying just so I can kind of recap, make sure I've understood properly. He's saying a patient, they've got recording electrodes in their head. Yes. And you figured out how to map electrical like the neural impulses, the brain impulses that they have in their brain, kind of following your piano key analogy, you're able to figure out which electrical impulses relate to specific sounds. And so you can tell either what sound they are listening to, or just like the glass example you provided earlier, just what sound they're imagining. Yes, that was the second phase. The first phase was basically, can we decode the sensory representation of a sound? The answer is yes. And as we briefly discussed before we started, we have a paper now in press where we were able to decode music, basically in the brain. And we decoded pink, pink, pink, another brick in the wall. We didn't, when I show a slide on it, I don't use the album cover of another brick in the wall. Use the album cover with the prism. Of course you do. The light and spectrum, you know, from the darker side of the moon. But you say, was it just done to be clever? No. Music has emotional, effective components that speech doesn't have. So I think understanding the structure of music, because you'd like to have an output device that didn't just say, I love you. You'd like it to say I love you, right? You'd like to have. So hopefully that will help. But then we went to imagine speech where we basically at that, doing imagine anything is hard, because you don't know the timing of when the person precisely starts imagining. Whereas what you're doing sensory, I drive your system. I know you heard baseball. I know you heard tree at this time. I know you saw the light. So imagine is a little tricky. You have to go back to math and do these things that are called dynamic time warping where you have to adjust your signal to match the physiology to match the acoustics and then control for errors. You know, you want to be careful. But it works. And we were able to successfully do that. A talented violin engineering grad student did some really wonderful work. She's now in the tech world doing very well. And that's about where we pretty much stopped. We figured out we could decode stuff. We figured out we could decode what you heard. And we figured out we could at least at a first pass level decode what you're imagining. And there's a couple roadblocks in the field. The what's what is really fueled this field is the fusion of biology, neuroscience and computation, right? And all the elegant machine learning and new algorithms. I mean, they're so, you know, incredibly powerful. They keep, they keep evolving. And that's very important. But there's there's actually a kind of boring thing that's that needs to be cracked. And that is higher density electrodes. And what does that mean? The typical electrodes that are on the surface of the brain, most of them are separated by a centimeter. Some of them are separated by four millimeters. But if you go on the actual cortex, you get independent activity at one millimeter. So in the ideal, the ideal prosthetic device for speech would have a higher density electrode, which is not a, we don't get those for use clinically. We don't need them. And we're not going to put them in just because they're cool. But it is, it's where the field needs to go. Higher density electrodes combined with continued advances in signal analysis and extracting, you know, neural networks and the various developments that have been, you know, just flooding the field basically. So, you know, artificial neural networks. Yeah. I mean, be careful about them because, you know, a lot of people don't understand exactly how they work, you know, like some of the new big, you know, massive, extra former architectures. Yeah. Yeah. But, but we use what we can. And we just want to be, you know, we're, we're, we're agnostic to how perfect they are. We want to know if they give us signal that helps us do something that can move the field ahead in terms of neuro prosthetics. So if you have these higher density electrodes and then, you know, I feel like we can take it's a given that machine learning techniques will continue to evolve rapidly, like they have used the last years. For sure. What are the new kinds of advances that you think we could have in the next five, 10 years? Well, I think having an in a process, implantable prosthetics speech devices within, is within reach with some, instantly real time. Yeah. So somebody who can't speak, they could have recording electrodes and they could be using, yeah, you're just recording from their brain and decoding that signal into their speech. Wow. Yeah, with a speech app, but the vice, we, I think we might discuss it later, but I have this kids general frontiers for young minds. And there's a young postdoc from Switzerland, Stephanie Martin. We have these live reviews where people present five minutes of their research to a panel of kids who then quiz them. And they've already, the kids have already seen their paper and been mentored by a PhD or postdoc. They're ready. And she gave a, you know, she's, you know, she just, you know, Swiss, everything's, you know, perfect. And she's given her talk and she's done. And a 10-year-old kid, he was, she was, she was great. I haven't actually filmed. He said, so Stephanie, you know, if I have one of those things in my head and I'm looking at you and I think, boy, your hair is weird. Would you hear me? Jesus. He's going to mirror, please. Anyway, yeah, it's, it, it raises all kinds of problems right in control. But I think the other area, you know, we explode and we're just tickling those areas. You know, we know we can perturb a network for motor control, right? That's for sure. We can help Parkinson patients. Well, have it a network for emotional control. Can we break or drive or entrain a network for emotional problems? Certainly depression is not, it's not like it's a fixed your brain is deteriorating because it comes and goes. So it's got to be a network oscillatory dysfunction. So if we can figure out how to understand it, can we actually develop techniques, akin to what's happened in the world of motor dysfunction and particularly Parkinson's disease? I think that's a big up in coming very, very important area of research. It's way, it's not way behind, but it's not as advanced as the motor control, but you know, it will be there. And then of course, things like decision-making in frontal lobe function, I was, you know, not kidding. I mean, if you look at aging, the number one biggest change, this is absent a degenerative disorder. Absent, you know, Alzheimer's disease or frontal temporal dementia is basically, you're not quite as sharp in terms of some cognitive functions. Can we ameliorate that? I think the answer I think is yes. And we don't need to go into the brain for that because you can actually, the beautiful, the speech we need to be in the brain period. The DBS akin to Parkinson's for emotional disorders, I think we need to be in the brain. But frontal lobe, I'm not sure we need to be in the brain because one of the dominant rhythms in your frontal cortex is called the Theta rhythm, which is a relatively slow rhythm. It's an oscillation at maybe four to six, seven cycles, you know, per second. That can be, that can be controlled by extra cranial, transcranial, alternating current stimulation, in fact, a paper. I'm blocking on the first authors name and I apologize for that, but just came out with a paper in nature neuroscience, you know, really solid journal showing that TACS of the frontal lobe improved attention and memory of performance in general in older, in older subjects. So and you don't need, you wouldn't need to have it on all time. Maybe you can put it on for 30 minutes when you're reading the sports page or listening to your podcast or whatever. I think there is, those are all coming, coming areas and I want to really emphasize to your reader, you know, we're just scratching the surface. There's so many, you know, things to be understood in terms of how the brain works that will only feed into the this really wonderful world of neuroprostatics. Fantastic. Those were all very exciting applications, thought to speech, notwithstanding, of course, yeah, the issues of your private thoughts. You got to turn your speaker off. It's not common. And yeah, treating anxieties similar to the way that we can treat some motor issues today, like Parkinson's. And yeah, being able to help people with decision making as they age cognitive function, exciting that that can potentially be done without needing to get in the brain inside the skull. Very cool. So and last topic area I want to get to is just consciousness more generally. So I know that you recently had a paper. I think it was with the Hebrew University and it was around the physiology of the brain supporting visual perception. So we've talked a fair bit recently in the episode about auditory perception. What's different about this and visual perception? Well, I think I'll just show you an example and what the key question is. Let's go back to our favorite object in this podcast. You ready? Here it comes. Okay, so when that glass came on, it had a big burst of brain activity in the visual areas. That was over by 200 milliseconds. Has the glass changed in how it looks? Does it look the same? No. So what's going on? What maintains the conscious visual perception? Basically, it's a pretty simple question. But no one knew the answer basically. And my colleagues at Hebrew University and particularly the grad student, Gaul, Hedishna, who did it was a spectacular job showed that yes, the activity in the very early sensory areas does drop like everybody knows by 200 milliseconds. But the more extended areas that tell you whether this is a shape and a color and all these higher level. Remember, we talked about how early you're on in this talk, how you have areas that are specialized for color and shape and form, et cetera, faces. That continues to fire, but it's you need to do multivariate analysis. And you pick it out. It's a very weak signal, but it's a very powerful signal and it predicts these precise duration. If I put this thing on for half a second or a second or 1.5 seconds, it will precisely track the duration as soon as I take it away, it shuts off. So we also showed in that paper that when the stimulus first comes on, not so surprisingly, my favorite area, the front the lobe is activated. And it comes on immediately and it's only on for about 200 milliseconds, at least in the way we can measure it now. I think we need to be there's, I don't think it just goes off. I think it's there's things that we can do to extract, signal it's below the resolution of our current measuring techniques, but there's a burst of activity in the front the lobe and a continued perception in the back of the brain, basically. And that's that's basically what this paper, what this paper shows. It's a nice paper. Very cool. It actually is. I mean, it is, I mean, you know, consciousness is a word. So we just pick it up. It's like memory. Yes, the room of a hundred memory researchers get nine different definitions of what's memory. Consciousness is a little gooey, but it is certainly salient to not just interest to people, but I think it's, you know, obviously to clinicians, I mean, minimally conscious state, coma, you know, just sleep. I mean, there's all kinds of areas that are important that the more we understand about, the more we understand about the brain mechanisms of anything, the better we are, the better we are we are offices society, I think. For sure. Well, I mean, you know, that I did in neuroscience, neuroscience PhD. So I'm, I've certainly drunk the coulade. I think it's the most fascinating thing. You know, how molecules, how chemistry allows us to have a conscious experience to make decisions, to have sensory perception. I think it's the most fascinating topic around, even though I guess I'm hosting the data science podcast, but part of what got me into this was, you know, these artificial neural networks. Yeah. And, you know, also, there's, there's also, there's really fascinating things about data science and the speed with which it moves that we can't get in in biology because you can't get this perfect picture of what all the neurons are doing at once, like we can in an artificial neuroscience system. Well, I, the brain is the penultimate data science machine. It's just built. And, you know, the interesting thing, if we run into each other a year from now, boom, everything's going to come back. We're going to have what kinds of memories of this podcast and where we were in tune. And it's amazing. It's just amazing. Human behavior is crazy. Crazy. It's wild. Yeah. And so you alluded to this earlier, just to start to wrap up the episode. I usually, at the end of episodes, I ask for a book recommendation. And I think this gives us the perfect opportunity to talk about your front ears for young minds, journal, which you already alluded to with the, with the boy who asked about the rear. And so, so what is the front ears for young minds? And maybe some of our listeners, children would be interested in this. Oh, yeah. Absolutely. I do have a book recommendation. Oh, great. I would recommend The Working Brain by Alexander Romanovic Lurios, published in, I was going to sound ancient, 1973. Penguin books, I think. In my opinion, this is the most brilliant neuropsychologist who's ever populated the planet Earth. And based on clinical observations, many of them, just from where the bullet entered the skull and wore injuries from World War II, he came up with this idea of how the frontal lobe does planning, how it checks your behavior, how it adjusts your behavior. We got a zillion very expensive techniques, which actually have confirmed pretty much what Luria said in terms of his clinical observation. So I think it's a good read. I mean, it got me. That's what made me go into neurology, basically. Was that that book? When I read it when I was a med student. So frontiers for young minds, it's been around for 10 years. It's part of the Frontiers Journal series, which has lots of different areas. The Kids Journal is a little bit different. Kids are the reviewers of articles. So what does that mean? You are scientists. You publish something and you have an option now to submit it to Frontiers for Young Mines in a way that a target audience of 12-year-olds will understand it. The age range we have, our kids are 8 to 15. We look at the submission. First, we make sure it's at least in that range and then we'll break it down and we'll either then send it to kids. One kid or a group of kids 8 to 11 or 12 or maybe it's a little more advanced to a slightly older group of kids. Each kid or group of kids is paired with a PhD student or a postdoc who now mentors the kid and they go through the article and what we want them to do is to understand the scientific method. What's the hypothesis? What experiment did you do? What results did you get? How did you analyze it? What's your conclusions? So it really is just the core of what you did for your PhD, what all good scientists do. It's a little bit different general in the sense that first, there's no page costs, zero. It's open access. I have about, I think, last 480 or 500 editors, associate editors handling this. Take a guess what, to some total, we pay them for all their services. Well, it's probably going to be an easy number to guess, so maybe zero. It's marked by, no, and then we have 8 sections that we have neuroscience because that's how it started. It was like crazy idea. And then we now have psychology. We have health. We have biodiversity. Earth and its resources, astronomy, physics. I think I mentioned math and we're just adding for you for your readers a brand new section on AI, machine learning and robotics. And probably the next section based on what the kids want, it's going to sound a little strange because it's like the opposite of robotics. It's paleontology. They like bones, kids like bones. So that's what the journals about. It's been really very successful. We have 11 million users. We have roughly 35 million views and downloads. We have per article, close to 30,000 views and downloads per article, which is more than most journals. So it's out there. We started, of course, in English. Then we got a donor who gave us money and put it all in Hebrew. Of course, it was in Hebrew. What sort are you going to do? They got to have it in Arabic. So they funded it in Arabic. Now we have it. We're just releasing the first 100 articles in Chinese. And we're working with a very generous donor in India to get it into Hindi for kids there. If anybody out there knows anybody who's got some dough, who can help in Spanish. Sorry, can I do that? You leave for sure. Please? No, I want to get to Spanish. You know, our biggest obstacle, it's the perfect thing for STEM. Because it's not what happened 10 years ago. It's what's happening now, right? And in fact, we have 30 articles already 15 published and 15 more in process from Nobel laureates who've submitted articles, who have been reviewed by the kids. So it's really been wonderful. But our roadblock has been the school system because the school system says, what's your teaching plan? And we want to say, well, the teaching plan is to understand the scientific method that doesn't quite resonate yet. But we're working on trying to have an option that the kids will review the article and plus the teacher will get some form of a teaching plan. But that's what our goal is to have it in schools, basically. And any kid out there who want your listeners, the kids can go to the website and they can sign up to be a mentor. You know, I don't know how long it'll take them to get a paper, but they'll eventually get a paper. The parents have to sign a consent because it's, you know, because it's a kid. Your kid's information is never revealed. If you go to the website, the kid gets to make their own little picture of whoever they are. We give them various tools to make avatars, et cetera. They have their bio, what excites them. And just their first name, that's it. So that's the journal. Frontiers for young minds. Sounds fantastic. I'll be sure to have only to frontiers for young minds in the show notes. Bob, this has been an amazing episode. And thank you so much for making your third podcast appearance with us here at Super Data Science. I know you're not huge into social media, but if there's anybody after the show that wants to be able to follow your thoughts, I guess maybe your Google Scholar page is going to be a good place to follow. Well, that's where the papers are. I mean, if, you know, I mean, someone had a, yeah, I think my social media is getting, doing research and getting the pup, the papers out there. So that's probably the best, the best thing. I was going to say, send me an email, but that could get crazy. If you've got a number of people you have, you could also look at Brad Voitex. He's got, well, he's missed your social media. So you might get some good stuff there. And Earl Miller's. Yeah. Brad's great. Earl Miller is a brilliant frontal lobe researcher. He's very active on social media. I'm basically, as I mentioned to you, a semi-nuckled director, but I've got an 11-year-old granddaughter who can help me out with the more technical aspects. Very nice. Well, thank you very much, Bob. This was a fantastic episode. And yeah, I really appreciate all the insights you've had for us. And maybe in a few years we'll be able to dig into the latest in your research and see how things are coming along. Yeah, that would be great. I mean, again, I want to thank you for doing this because in, you know, an educated populace is an empowered populace. And an educated kids are a double empowered because they're the future. Well, in an extraordinary individual, I feel super fortunate to have been able to pick Dr. Knight's brain. And I hope you took a lot from our conversation. In today's episode, Bob detailed how the prefrontal region makes up 35% of our brains out or layer the cortex, a proportion that separates us from the other species on our planet and imbues us with advanced cognitive orchestration capabilities, including time traveling. He talked about how machine learning models are integral to controlling Parkinson's disease and modern deep brain stimulation treatments, how dynamic time-warping algorithms allow him to decode imagined sounds, even musical melodies through recording electrodes implanted into the brain. And he talked about how in the coming years, higher density brain computer interfaces paired with machine learning advances could enable real-time thought-to-speech synthesis, game-changing anxiety treatments, and perhaps without any invasiveness or reversal of aging-related cognitive decline. All right, that's it for today's episode. If you enjoyed it, subscribe to ensure you don't miss any of our exceptional upcoming episodes. And until the next time, keep on rocking it out there, folks, I'm looking forward to enjoying another round of the Super Data Science Podcast with you very soon.