Myth Busting Science Lecture Series - Eduardo Blanco: Are computers as smart as you think?

Myth Busting Science Lecture Series - Eduardo Blanco: Are computers as smart as you think?

Show Video

foreign good evening everyone thank you for joining us tonight for the second presentation of myth busting science I'm Carmi garzion and I'm the dean of the College of science so please join me in giving a big round of applause for tonight's performers Saul Millen who I believe has stepped behind the stage before we begin I just would like to take a moment to recognize those who have helped the College of science bring these lectures to the Tucson community and Beyond we are very grateful for Arizona Arts live and their partnership in bringing this year's lecture series to Centennial Hall and also for helping to arrange the music that you are enjoying every evening I'd also like to recognize the Galileo Circle members many of whom are in the audience the Galileo circle is a valued community of individuals whose support is vital to the College of Science in continuing our Excellence supporting our students faculty and staff so thank you for your continued supportive of our community and thank you for being here with us tonight um and of course I'd like to thank our series sponsors um and and our Media Partners and in particular our presenting series sponsor holu Aloha companies yeah this support helps us continue uh continue to engage the local Tucson community so we're very grateful to be able to bring the lecture series to you um I hope all of you had the opportunity to enjoy last week's opening night presentation by Dr Lee Ryan on the Aging brain and if you didn't get to see that it has been posted on the College of science YouTube channel so it's there for your viewing if you missed it I'd also like to invite you to the two upcoming lectures on myth busting science uh the next after this one will be on February 15th and that's Jessica Tierney who will be talking about global climate change and in particular some of the impacts on our local community here and Mike Warby will be presenting on I think that's March 1st and he's going to be talking with you about his discoveries on the origins of coven 19. um now in partnership with the University of Arizona's poetry Center we are very pleased to introduce a new element to the Series this year all four of our lectures are uh accompanied by a representative from The Poetry Center who has created an original poem based on each night's lecture and so at this time I'd like to welcome to the stage Gabriel dozal who's going to present tonight's poem [Applause] so Gabriel's a proud graduate of the University of Arizona's masters of fine arts program in creative writing his work appears in poetry magazine the Iowa review Guernica the Brooklyn Braille and the live the literary review his first collection of poems the Border simulator will be published by one world Random House this summer so Gabe the stage is yours thank you hello uh good evening my name is Gabriel dosal and through a partnership between the College of Science and the Poetry Center I'm excited to share a poem I've written to welcome you to this evening's presentation uh this is also a little bit of a collaboration with Eduardo the lecturer for today he sent me his notes beforehand and I sort of wrote a poem that Rift off of some of those ideas my book and this poem are about the border of Mexico and the United States but it's also about how difficult it is to parse the news social media and the flood of information that we sift through every day so I'm a big fan of this theme of myth busting Okay so yeah there's two characters uh one is named Primitivo and he has a sister named primativa um and there's also uh voices of those characters that weave in and out of Customs agents and you'll hear me say the word plus and that designates different sections and kind of different voices throughout this poem um okay here we go the algorithm warns Primitivo the zombie truth walks Among Us the facts of the Border are zombies and we the algorithm return crossers to the South but they keep returning craving the Flesh of the Border Plus the algorithm didn't know it but we the buried zombies took the border with us under this sentence come find us the bodies of those who have crossed before us are also buried under this sentence start digging up the words and eventually you'll find our bodies Tangled a new kind of fence ground together opulent flesh of border Plus strains how offense can spell j-o-b for the zombie crossers or jobs for the zombie crossers and jobs equal worth in the Border simulator this jungle job turned its back on Primitivo and ran for the Border Primitivo Now spells this elusive job spell my name Primitivo to learn more about it that's what I'm doing with your name p r i m i Plus we've loved the fence for so long we wouldn't know what to do without it we haunt this fence algorithm and at the same time we help it catch more crossers just like us unwittingly us zombie crossers helped catch the catcher of catchers because US Customs massage the fence algorithm we already have the answers to the questions where did Primitivo and his sister primitiva come from when is Primitivo a past version of himself when is Primitivo a future version where's the past future border this is what primativo is searching for The elusive present moment a moment covered in snow cap lard did you make this snow-capped moment Primitivo you've never even seen snow how could you or did did US Customs lard up the moment you tell us Primitivo that's the quiz at the Port of Entry he's not sure what's on the other side but he knows it's better than his real job the algorithm is saying your house is worth a face but a symmetrical one you have a face only a fence could love but for you will make an exception primi the algorithm says your home is worth a computer that oracles if you didn't already know algorithms are oracles plus myths get ground into facts by the news they're grinding away I want an algorithm to tell me the best way forward I can't trust the primitivos of the World Wide Web of ethics and facts mixed together before the mixing happens myths are ground into fact at the myth-making factory and factories love the Border let me tell you the algorithm says your house is worth your sight gouge out your eyes Primitivo don't see what you are seeing computers are oracles and they know where you work but do you know do you work outside the home is iron denser than cotton Primitivo look at this photo it's a photo of you crossing the border you are already here don't you recognize yourself that's the quiz at the Gate of Entry we set loose Decades of Yearning sorry yawning at the Port of Entry the line never ends and thus our jobs will never end we'll be in heaven looking down at our jobs admiring their efficiency and Order get on the right side of the border the side with jobs you're on the wrong side of the border and the wrong side of History Primitivo thank you [Applause] well let's thank Gabe for that very thought-provoking animotive uh poem really nice connection with tonight's topic so I'd like to introduce tonight's feature presenter Dr Eduardo Blanco come on up Eduardo um so Eduardo is an associate professor in our department of computer science in the College of Science and he's a first generation college graduate um and um oh I lost track there and he received his PhD from the University of Texas at Dallas Eduardo joined the College of science last fall after holding faculty appointments at the University of North Texas for seven years and at Arizona State University for one year and if you ask Eduardo he will tell you that he has found his place here in Tucson where the community and the university as you can see tonight mingle in meaningful ways so welcome Eduardo [Applause] Eduardo conducts research primarily in natural language processing processing the subfield of artificial intelligence that enables computers to understand language his work has been supported by the National Science Foundation the National geospatial Intelligence Agency the Office of Naval Research the patient-centered outcomes Research Institute and generous gifts from industry Eduardo's research has been recognized with the bluebird with a Bloomberg data science research Grant and the National Science foundation's career award which is a a an award that is given to a very small number of early career scientists so please join me in welcoming Eduardo to the stage [Applause] all right good evening everybody thank you Carmi for a very kind introduction I want to thank you all for being here at Centennial Hall tonight also those of you watching online hoping that we all learned tonight and I'm going to talk to you about a couple things the first one I want to discuss with you what kind of problems computers can and cannot solve today and also I want to have a discussion about how smart computers are and how smart they might get in the near future and I want to start stating perhaps the obvious but chances are that today most of us have either done something or perhaps perceived something because a computer suggested we do so if you subscribe to Netflix when you turn on your TV you see a bunch of suggestions those suggestions are customized to you Netflix knows that you are very likely to watch whatever they suggest and they also know that you are actually very likely to like it after you watch it most of us when we want to drive somewhere today we don't even think twice we just type whatever we want to go in the GPS and we just follow whatever the computer says and for the most part that is safe if I want to drive from work to the biosphere just going to type their biosphere not too familiar with the highways here in Arizona yet I'm just going to do whatever the computer says for the most part that's safe now I want to share with you a little personal story about house prices and let me tell you some of you may live in a wonderful modern house with the saguaros and a really nice patio here in Tucson some of you may live in a more traditional home but regardless of what kind of home you may own if you are a homeowner you have to pay taxes and there is no way to not pay those taxes I move to Arizona from Texas and over there the property tax rate is roughly two percent so I'm going to do some quick math for you if the county says that your house is worth a hundred thousand dollars you have to write a check for two thousand dollars every year if after three years now the county says that your house is worth two hundred thousand you're gonna have to write a check for four thousand dollars around 2016 I got my tax statement and it literally said with the county think that your house is worth this much and therefore this is how much you have to pay us in property taxes well I didn't really like the number and I am a Scientist and I kind of like to ask questions right so very politely and very calmly I went to the tax office and I asked the question where does this number come from why do you think my house now is worth blah blah and I had a meeting with the tax assessor conversation was very civil and very polite and he explained to me well look here houses this big has this many bedrooms the lot is this big and this other house in the same neighborhood kind of sold for this prize after we do some adjustments this is how much your house is worth I was not very convinced and I would basically say things like well you know my house is all there the kitchen is not updated we went on and on and at some point he got tired of me and when he got tired of me he said three words and the three words were the algorithm says and let me tell you there is almost zero chances of finishing that sentence and make any sense but he told me with a straight face was the algorithm says that your house is worth blah blah blah and when he said that it was just very very frustrated I could see it in his eyes I not kidding I could say it in his eyes he had shown to me refutable evidence that's how much my house is worth because the algorithm says so that was very frustrating why well it's not quite the case just because an algorithm says something we should not take for granted that the algorithm is correct but honestly the part that he really thought that he was showing me a refutable evidence it was really really frustrating now if you are wondering that this is just some count in Texas that they don't know how to assess house prices bottom line is that the private sector has tried Redfin tried to sell and buy houses depending on whatever an algorithm said they lost a lot of money it did not work it's Redfin also try and again by buying and selling houses based on whatever the computer said they actually end up losing a lot of money the bottom line and what I really really want you to get from today's talk is that computers are not oracles I'm not saying they are completely dumb all I'm saying is like any other sort of information out there we have to question them computers are not Oracle right now this is the mythbusting lecture series so I have to do some myth busting I have three statements some of them are true some of them are not true and we are going to discuss this right now first statement I have I believe it's kind of easy and the statement says modern computers are faster than older computers most of us today change our cell phone every two three four years and yes it is true the newer cell phone is faster than the older one let me give you one example that relates to research IBM did some Pioneer work in the late 80s early 90s on machine translation and that was very very nice work and they did it in what they back then called a super computer that super computer is actually less powerful that the cell phone you may have in your pocket right now so new computers are much much faster that is true now let's look at that slightly perhaps harder statement and this statement is modern computers are as smart as you think now I'm here tonight to tell you that this is a myth I cannot read your mind so excuse me if you thought that computers were not really smart but if you thought like my tax assessor or the executives are Redfin or the ones at Zillow you certainly put in too much Faith to computers uh again I'm not saying they are completely dumb but probably they are not as smart as a lot of people think let's look at the first statement the first statement is modern computers can solve harder problems than all their computers I believe this statement is going to help us understand why computers maybe they are not that smart and believe it or not this is actually a myth the computer you can buy today at Best Buys and the computers that run the latest and greatest technology are actually not any smarter than the older ones you could dust a 20 year old computer in your garage that you've been storing for some reason if you load modern software into the older computer that computer is going to get you exactly the same answer than the new one that's what the theory tells us and it's actually true but because they are so so slow it would take forever for them to give us an answer but there is nothing that an all computer cannot do than anyone can do and when I say it will take forever I actually mean it I don't mean just overnight or two years I mean longer that I'm Gonna Leave but if we were patient enough they will get the job done now just in case there is any computer scientist in the room I want to make clear that I'm talking here about traditional computers the ones you can buy at Best Buy you might have heard of quantum Computing those are under development whenever they are ready they will solve different problems but we are not there yet all right so what kind of problems can computers solve the answer to this question is really straightforward they can they get to solve the easy problems what is an easy problem well that's really where the secret sauce is two examples of easy problems are adding and sorting those problems are easy to ask humans and they are also easy to computers some problems may appear overwhelming to humans and computers are really really good at solving for example complex root finding if you have to plan a bunch of deliveries and for example you have five drivers available and each driver is available to drive for so long and you also want to minimize left turns because left turns are more dangerous than right turns and you also want to minimize the amount of Miles because wasting gas is bad it's a problem that requires certain resources but the computer will give you if not an optimal a very close to Optimal solution much better than what a human could do especially if the number of deliveries very very high let's look at what kind of problems computers cannot solve well the hard ones right what is a hard problem and I'm using to work hard in a very specific meaning here and all I'm really trying to say is a problem that the computer cannot quite solve and perhaps surprisingly these problems tend to be hard to computers but extremely easy to ask humans there are two characteristics that make a problem very very hard for a computer and those are requiring common sense and requiring reasoning what do I mean by Common Sense and reasoning they are kind of scary words right so let me exemplify what I mean by introducing to you the latest and greatest piece of software it's a tool it's called chat GPT you might have heard about it on the news it's all over the place and when I say latest and greatest I actually mean it I could not have dreamed of having a tool such as cha GPT five years ago had you asked me I would have told you no way you're going to be able to have a conversation about anything with a computer well here we are 2023 company by the name of openai offers anybody really this store called chap GPT and all you really need to know to follow is that it is the latest and greatest the smartest we have and you can actually have a conversation with Chad GPD you can ask anything and it will give you gaps it will give you back an answer well let's see whether charger PT has common sense and reasoning that's me and I asked Chad GPT a seemingly simple question hey Chad gbd what is longer 10 meters of rubber or 10 meters of plastic first sentence it is not possible to answer this question well it it is bad but personally I'll be like well refusing to answer I'm like okay nice at least it's not too bad but Chaturbate really thinks it is really really smart so it's gonna try to justify why it is impossible to answer the question and here he goes it says well the length of a material depends on the material and the size no the material does not determine the size and if I tell you the size I've told you the length so not quite it knows how to make an argument so now it gives us an example it is yeah I know Wonderful common sense right it's factually brown right 10 meters of rubber could be longer or shorter than 10 meters of plastic nope and then it keeps going and it throws some information yeah but I think the last one is kind of perhaps my favorite it says well I'm gonna throw in something that's true it's distantly related to the question but it's really irrelevant kind of like you know the length depends on temperature and humidity and blah blah blah this is what I mean by Common Sense and reasoning I don't think any of us will ever come up with that right perhaps the scary part or what scares me a little bit is that if you don't pay attention if you don't know what lens is you know that's perfect English yes perfect grammar perfect diction the structure of the argument is actually pretty good I'm going to tell you the answer then I'm going to tell you why then I'm going to tell you an example and then I'm going to tell you some other information well if this is the best and the latest perhaps we have some work to do right all right so what is the plan for today I want to cover three things the first one is language computers and problems I will admit that's a weird combination of words but it's actually a good description of my job I am a computer scientist I work to get computer solve problems and I am particularly passionate about problems that have to do with language we're going to be discussing how good or how bad they are that's all computers are at solving problems that have to do with language and then I'm also going to tell you why why sometimes they seem to do silly stuff and finally I want to share with you the challenges before us some of the successes we have had and what we have left because trust me we have plenty of work left to make computers smarter so let's get started with language computers and problems and I want to make again kind of like an obvious statement but I think it's important understanding language is complicated it takes children years to be able to make sense when they is fake and that's through this fight they're typically surrounded by adults who are talking all the time if you're a computer this is way more complicated and part of the problem is that we people don't always mean what we say and we understand each other just fine despite sometimes we deny and we contradict I'm gonna try really hard today not to contradict myself but chances are that last week or maybe next week you come across some information that contradicts something I say today and you're not going to have a serious problem figuring out what you want to believe in you have no issues coming across contradictory information nobody in this room ever misrepresents and lies obviously but there are people out there that do exaggerate and minimize or they just don't say the truth and finally we have in language this thing called irony and iron is hey why don't we just say what we don't mean because it makes perfect sense right well trying to get a computer to figure out irony is actually hard the bottom line is the languages ambiguous and I want to exemplify this by introducing you to my friend Santiago he goes by Sunday very good friend of mine he likes to talk and he also likes to be listened to and one day he went in kind of like a monologue and he said I know that no matter where I go or who I build a life with I will never had with anyone what I had with you if I were to stop right here and I ask you is my friend Santi happy or sad I believe most of you will say clearly he's sad he's missing something well he has a particular sense of humor he's so glad that whatever he used to have it's not there anymore and he doesn't need to put up with it stuff like this is why language is complicated and if you are a computer you're just gonna face a lot of challenges Computer Sciences are relatively large field some of us work in a subfield of computer science which is called artificial intelligence and then some of us like language enough that we actually work on natural language processing and all that is is we really want to make a computer understand language in practice this looks like so we get a large chunk of text for example Wikipedia we feed it to a computer and now if the computer has natural language processing capabilities it can do something that we will call intelligent for example you will be able to answer questions who is the president or who was the president in 1962 the computer will figure out where the answer is in Wikipedia computer will also be able for example to simplify given the article for the United States it's going to be able to generate a different article that roughly has the same content but now it can be understood for example by a six-year-old computers today can also translate and so on and so forth we solve these problems with computers and computers the only thing they can do is follow instructions and nothing else they don't have a life of their own they don't get to do whatever they want they follow instructions and sometimes computer scientists come up with perhaps weird terms and a bunch of instructions that a computer can execute is called an algorithm it's not a fancy word it's just a bunch of instructions at a very high level we have two kinds of algorithms the ones that solve problems and the ones that learn to solve problems from examples I'm going to illustrate the first kind let's say that I want to buy hiking boots step number one go online type hiking boats you end up with a long list of hiking boots for sale I am kind of frugal so step number two is sort the boots by price from the cheapest to the most expensive we have an algorithm that solves the Sorting problem I'm going to go over every single line it will take a little while but you just kind of have to trust me for a second and every single line in that code can be run by a computer and given a list you're going to end up with the sorted list that algorithm is not very efficient but it gets the job done let's look at the other kind of algorithms the ones that learn to solve problems from examples and now let's talk about perhaps a more interesting problem that's sorting and this problem is called sentiment analysis centive analysis is about if I give you a review of a hiking board can you tell me whether it's positive or negative whether the customer liked the board or whether the customer does not like the boat well if I want to learn from examples the first step is to collect examples and examples are actually simple they are very intuitive I need a bunch of positive reviews and I also need a bunch of negative reviews for example the weatherproof leather is smooth the ballistic nylon is strong and so on that's positive if a wood doesn't breathe or stays tight that's bad so those are negative in the real world we will need thousands of these examples not just two we need thousands of them but as the weird dog collecting them we feed them to what is called a learning algorithm and this learning algorithm is going to be able to extract patterns from the examples and those patterns are going to sit in a computer now here is where the important part comes this model that the learning algorithm learn using patterns is able to tell us given a review well the new review is positive or negative nobody told it how to tell whether a review is positive or negative the computer the learning algorithm figured it out from the examples now what is better the first kind of algorithms the one that solve problems of the ones that learn from examples answer to this question in science is some most always it depends but if we look at it from the user's point of view I believe what users care is does the computer does the algorithm get it right and if we go with this criteria if you use an algorithm that solves a problem the computer is always going to get it right and that is wonderful whenever you use even if you are not sure how it runs but whenever you use a computer that is running an algorithm that learn from examples it is not going to get it right all the time and the bottom line is we have no clue when the computer is going to get it right and when is it going to get it wrong even if we try really hard it's like going to the casino when you go to the casino you know most days you're going to lose money and you know before you go that every now and then you'll get lucky and you will make some money your ads using a computer are way way better most of the time the computer is going to give you the right answer but by no means all the time it's just doesn't happen now I told you I really care about making computers understand language the only way we can do that is by learning from examples and if we learn from examples we will make mistakes and if somebody tells you that they can understand language with a computer without making mistakes they are misleading you how do these mistakes look like well let me tell you a pattern then a learning algorithm I learn for example the world greatest and the world healthier indicate a positive review it makes sense at face value this is a valid pattern now let me introduce you to a very particular product that you could buy today on Amazon and it's a banana slicer it's made by hot Slayer seven sensor reviews 4.5 average star this thing is Magic right a banana slicer looks like so let's read some reviews together I actually have to read this but somebody said what can I say about the banana slicer that hasn't already been said about the will penicillin or the iPhone this is one of the greatest inventions of all time people love the banana slicer right now this product has saved me countless hours of slicing bananas and I'm not sure how I ever live without one not bad you can buy it for five bucks I mean it's a deal and my favorite our marriage has never been healthier [Applause] a computer is not gonna get that these people are using irony it's just gonna tell you buy the banana slicer it's going to make your life better all right so how bad are computers at understanding language well I'm gonna share with you three skills I don't think they are too difficult and we are going to figure out whether a state-of-the-art computer can get it right or not first one is counting counting is easy right I told you computers could add and subtract and it's actually true if you have your money in the bank and this morning you have a hundred bucks in your account and then you withdrew 50 the bank will say now you have 450 and that's correct you do not have to triple check your bank statements computers can come money just fine but we humans can do more sophisticated content for example we can count objects in an image this is more complicated now I'm expecting the computer to understand what's in the image I'm expecting the computer to understand the question and I'm expecting the computer to come up with the right answer if you ask a state-of-the-art computer how many bears are there in that picture the computer is going to tell you one and if I stop here we all could be very happy and think that computers can count objects in pictures however let's see what happens if I ask the same computer is there one word in that picture and the computer it's a 50 chance right this either yes or no computer says no are there two words yes are there any Birds no that's the scent right what's going on here well what is really going on is that the computer knows how to answer questions that start with how many but doesn't know how to count and that is way too many ways to ask a question that fundamentally requires Counting let's look at another skill misspellings and typos every time I write an email I have to prove really ten thousand times and even then I keep making typos and misspellings let's give a computer a short piece of text the engine had a duty of about 7 million but most were closer to 5 million and now we ask a simple question to the computer what is the ideal duty of the engine state-of-the-art computer says 7 million which is correct wonderful now I asked the same question but I mistyped Duty and instead I say what is the ATI of the engine you and I know that I made a typo you'll be nice and you'll give me the right answer computer doesn't really know what to do with ATI it's never seen it before and it doesn't guesses randomly and it ends up with 5 million how bad is it well around 11 to 12 percent failure rate is that horrible maybe it's okay we are okay with whenever we make a typo we're gonna get the right answer less often but we certainly need to be aware that the computer cannot really deal with this type of misspellings third one is negation negation is something that I'm really passionate about and just to kind of like justify why perhaps we should spend more time trying to figure out how to make computers understand negation let me share with you that in English roughly one out of four statements has a negation so it is very frequent what happens when I give a computer a review that says something like I thought the plane would be awful but it wasn't so what I'm really doing is I am negating something bad now not be enough all we could argue whether it's positive or neutral but it's certainly not negative as of 2020 the three leading companies will tell you almost a hundred percent of the time that that review was negative negating something bad according to these three companies was still bad we obviously have a problem here right why does this happen well it happens because remember all these problems are being solved by learning from examples we get examples we learn from the examples we end up with a computer that can solve the problem based on the patterns that the computer learned from the examples it doesn't matter how much time I have there is going to be a finite set of examples I get to work with we're going to use some of those examples to train and validate the model and that's basically what we call learning stage the learning algorithm consists or is used to train and validate the model after we are done we use after the model sits in the computer we use the other examples we were with but we did not use for training and validating to test the model it is based on the results on the test examples that scientists make statements such as my model my algorithm my computer is 70 accurate 80 accurate 90 percent accurate or kind of my favorite the computer now is better than humans let me tell you why this is misleading it is misleading because it doesn't matter how many resources I have even if I work for the largest and richest company on Earth there is a finite set of examples I'm gonna get to work with and guess what there is many more examples out there and the illustration is actually way worse than this the bottom line is the examples we get to work with are a fairly small chunk of all possible examples the examples we ignore are very very hard to get right and that's why the model was failing when we use a different phrasing to check whether the computer can count if the model had never seen misspellings and typos because they were not present in the examples we were with chances are the model is not going to do very well same thing with negation and the list actually keeps going on and on now if you are just kind of thinking you know Eduardo I have the solution to your problem just collect more examples just work harder that doesn't work and let me just tell you why language changes people create new words the dictionary in September 22 added 317 Awards in 2021 almost a thousand there's no way I can predict what new words are going to come up next year let me give you some examples in case you are thinking I'm making this app and no new words are being created these are new words as of 2022 according to the merriam-wester dictionary what is a damp phone a dumb phone is a cell phone that is not a smartphone it is now in the dictionary I don't really know what the metaverse is but now it's in dictionary whatever it is this virtual thing there is an entry in the dictionary for that five years ago that was not really a common word right the platform kind of common and then kind of my favorite acronyms I guess at some point people keep using them and they make it to the dictionary TVH stands for to be honest new worlds are being created all the time I will never be able to get all the examples that I will find in the future and the bottom line is that all those examples we ignore it is unfair to expect the computer to know anything about them the reality is that the computer does know something those learning algorithms have the capacity to generalize from the examples we were with to the examples we ignore but the reality is that the capacity is not that good computer knows very very little about the examples we ignore let me share with you some of the challenges before us and I want to start with something positive I want to share with you some of the work we have done here at the University of Arizona in my group and this work is about yes no questions what is a yes no question piece of cake yes no question is a question that expects a yes or a no for an answer why should we care about yes no questions well when we humans talk we ask yes no questions all the time I kind of challenge you next time you have a conversation with anybody just audit your own speech and you'll see that you ask yes no questions all the time to the other person and perhaps even more importantly we humans have a very particular way to answer these yes no questions which is with a rather long sentence that almost never includes yes or no it's just explanation so here is an example this is me asking my friend if my friend works outside of the home and she did not tell me yes or no instead she told me man last month I was laid off you and I have common sense we know that being laid off means that my friend unfortunately doesn't work anymore and therefore she does not work outside of the home so that answered ought to be interpreted as no now sometimes the answers are longer and friends say things like meh last month I was laid off for now I work for the marketing firm and I travel a lot you and I have common sense you and I have reasoning we know that when people travel for work they are actually working while they travel or they are supposed to at least and we also know that in this case my friend works for a marketing firm she travels a lot and therefore yes she works outside of the home and that's why that answer ought to be interpreted as yes while this problem of interpreting answers to yes no questions is actually complicated when we started the latest and greatest computer had no clue it was basically as bad as guessing randomly we all know that guessing randomly is a bad strategy to take a test and it's also about strategy to solve any problem here is where we started we started with the simplest possible way to solve this problem and without getting into details of what the number means think of it as a percentage they can go from zero to a hundred the starting point was 25. what have we gotten done to make computers smarter and be able to interpret these questions better well the first step is to recruit a wonderful set of students that do a lot of the work these are some of the students I've had the pleasure to work with over the years there is a few PhD students there are a few Master students there are also undergraduate students and there is even one high school student that did research in my group together we went from 25 to 46. are we making the computer smarter well I think so I mean you know a little bit smarter almost twice as smart we're still far from a hundred and you know we could have decided let's try to work harder to bump that number from 46 to closer to 100 but this is what we did we said let's see if we can answer these questions in other domains let me tell you what I mean by that by going back to the examples we were with when we started every single example we were working with came from informal conversations what happens if we test our model if we test our computer on yes no questions that come from meetings athlete interviews or social media such as tweets well the computer has ignored that the learning algorithm never saw it so what happened is that the computer was guessing randomly not a good idea what have we done well we came out with ingenious strategies to quickly adapt we have a model that is pretty good at least decent at interpreting these questions and answers from informal conversations into other domains and the key here is that we are not just replicating the same work collating examples learning getting a model and claiming the computer now is smart we are actually transferring the knowledge we learn from informal conversations into other domains without requiring extensive manual effort now there are human beings making the computer smarter but the manual effort is relatively low what else have we done well I didn't even bother telling you but whoops I skipped that how good are we in other domains well it depends in social media these questions happen to be really hard because people talk about anything and everything we get around 47.

in customer calls when people call for example an airline the questions are really not that difficult and we are as good as 86 it depends on the domain again we did get to make the computer smarter but it is now like okay we are done and now we can answer these questions what else have we done I didn't quite tell you this but we all speak English here and this is how we did some work with multilingual question answering and basically I didn't even tell you but every single example we were working with initially was in English there is absolutely nothing wrong interpreting the answers to this kind of questions in English but there are many other languages out there right these are languages such as Chinese and Spanish I also other languages that get less attention in the research Community languages such as Hindi Bengali Turkish and so on and so forth fundamental question is can we make the computer smarter by now interpreting answers to yes no questions not only in English but also in other languages the reality is yes we can we can again quickly adapt and transfer the model that can interpret answers to yes no questions from English into other languages now we are working on this I cannot tell you all the numbers today but I can tell you that in Chinese we observe an 18 Improvement is the computer smarter sure it's 18 better but there's still room to grow room to make the computer smarter now let's talk about what we have left what problems are kind of hard and again these problems tend to be actually trivial to most of us humans and I want to start talking about multiple languages I was very fortunate to work in a bilingual environment I grew up speaking Spanish at home speaking Catalan with some of my friends and most of my classes were actually in Catalan I'm pretty sure I'm not the only one in the audience that speaks more than one language and I'm willing to vet that if I ask you the same question exactly the same question in one language that you speak or the second language that you speak you will come up with the same answer well let's go back to chap GPT and let's see what charging Beauty says when I ask in English which is heavier 10 kilograms of cotton or 10 kilograms of iron we know it doesn't know length very well maybe you'll learn weight well it did not 10 kilograms of iron will be clearly heavier than 10 kilograms of cotton and that's because iron is denser than cotton not true makes no sense what happens if I ask the same question in Spanish there is no trick in there it's just same question just in Spanish well now Chad GPT gets it right and it gets it beautifully it says both weighed the same and then the justification is also just beautiful it basically says you try to trick me but I called you your tallest 10 killers of something and then kill us of something I don't care what the something is we're talking about weight here they are both equally heavy how is this possible well I learned all this stuff in Spanish when I say this stuff I mean Mass kilos weight and so on and uh great teacher taught me the physics of what is mass and a kilo and so on and so forth and I store that somewhere in my brain later as a teenager I learned English and as an adult I got better at speaking English but the English teacher did not have to teach me again what is mass what is weight and what is a kilo does not have that concept it knows something about English it can generate wonderful English it can also generate wonderful Spanish but they cannot possibly understand what mass is because it tells me different answers same computer different languages different answers it really makes no sense let's go back to reasoning and I'm gonna go ahead and confess that this example is a little bit tricky I was really trying to and let's see if I succeeded I'm basically saying hey chat gbd there are two teams playing football is 50 000 people watching and one person in the standing room only section is injured and has to live in an ambulance it's a little bit weird but that's what I told chatubity how many players continue playing the game and first let's give some credit to charge EBT it understood the question as how many people or how many players are playing the game in the field not everybody who is with the team and I was shocked charging said there will be 11 players from each team continuing to play the game and then I was kind of like well let me read the justification because in an exam every single question asks justify your answer right we want to know how you got 11 players for each team well let's see it says there are actually 22 players playing at any time football game which is true you lost one player so now you have 21 players not true but you know I guess it doesn't have common sense let's look at the Second Step this is the better one it says you know clearly you have 21 players and two teams therefore you end up with 10.5 players per team you cannot split people in half and then you're like this half goes with team one and this other hard goes to team two it does not work like that I have to give credit to charge GPT though because it was nice enough to say if you run dab right 10.5 players is actually 11 and that's how it ends up with the right answer but clearly it cannot possibly understand that you don't get to slice people in half we humans don't do that we have common sense we have reasoning so let's just write the lecture app and discuss a little bit about how computers are how smart are computers well let me be clear here computers are great machines I believe they solve very interesting problems and they make our life easier let me just give you one example in addition to everything we are used to today our Radiologists spends less time writing reports and more times with patients because a computer assists the radiologist in writing these reports they do make our life easier now I personally think that computers will get smarter and they might get us to this estate where they are like super smart whatever that might mean however I can tell you that we are not there yet there are a lot of people working to make these computers smarter so are computers Really Gonna Get Smart and what kind of role are they gonna play well computers will play a role there is no question about it but I don't think they are necessarily gonna play the primary role there are people behind computers making them smarter they are computer scientists working really hard to write the instructions that made the computers smarter and even that is not enough this hard problem of making computers smarter is not only about computer scientists they will play a role but not the only role we need other disciplines if I want to make computers smart well maybe I should take a look at the human brain maybe I should look take a look at cognitive science the research field that studies how the brain works I care particularly about language well there is a whole field of scholarly work that is devoted to understanding language they are the linguist linguists are playing a very important role I collaborate with them all the time so I get inspiration and I get the computer smarter I get the computer to do harder problems that have to do with language based on the inspiration I get from linguist and finally I just want to say math is also very important algorithms at the end of the day are very heavy on math and if there is any young people in the audience today you want to make computers smarter you're gonna have to be good at math so start today what are we doing well together we are exploring the current limitations of computers I think it's very important that we are aware of what computers cannot do today only if we know what they cannot do we're going to be able to explore that endless potential and make them smarter exploring the current limitations is fun I've done a lot of that today but exploring the endless potential is even more fun because that's basically when we observe the computer doing things that it couldn't do two months ago a year ago and so on and so forth so will it get to a point when these computers are superhuman or smarter than us well maybe I don't know but what I can tell you is that the computer itself in superhuman it doesn't really matter that much what matters is people and let me just share the way I look at this the way I look at this is that we have to put people at the very center of making computers smarter because computers are gonna make humans smarter computers are going to give people superhuman abilities this is all I have for tonight thank you very much for listening I had a lot of fun well I hope you aren't disappointed that computers aren't as smart as you think they are but you should be very excited to know that there are many many smart people in computer science working with many smart people across a lot of areas of science to help make computers smarter and thereby enrich our lives and help us be super humans so thank you Eduardo that was a phenomenal talk let's thank Eduardo again yeah and I I just want to say I think Eduardo really captured the passion Behind The Sciences many of us are fueled by those questions that we think are important and many of us uh really enjoy tackling problems that actually do change our lives and computers and their intelligence certainly uh impact the lives of everyone in this room so thank you again all right um now I would just also like to invite you to next Wednesday's lecture series again that's uh Professor Jessica Tierney who's going to be talking with us about uh global climate change and Jessica's going to give us a little teaser uh on what she's going to talk about in this video hi my name is Jessica Tierney and I'm a climate scientist and professor of geosciences here at the University of Arizona I hope you enjoyed tonight's lecture by my colleague Eduardo Blanco I invite you to come back next week on February 15th here in Centennial Hall where I will be giving my presentation about climate change in particular I will be debunking a myth that you have probably seen maybe on the Internet or in the media which is the climate has always changed so why then is climate change such a problem hope to see you there thank you so if you enjoyed tonight's lecture and want to hear more about other engagement opportunities with the College of science please consider joining our Galileo Circle and learning about the many opportunities to support scientists in our community our Circle uh Galloway of circle team is near the main entrance where you came in tonight so please do stop by and introduce yourself thank you again for joining us tonight we're so pleased to have you here please get home safely and we look forward to seeing you next week [Applause] foreign [Music] [Music] foreign

2023-02-18 14:28

Show Video

Other news