Dr. Latanya Sweeney: How Technology Will Dictate Our Civic Future
serving as Executive Director of the Cornell Center for Social Sciences and it is my distinct pleasure and honor to get to introduce to you, Professor Latanya Sweeney. She's a professor of the practice of government and technology at the Harvard Kennedy School and in the Harvard Faculty of Arts and Sciences, editor-in-chief of Technology Science, director and founder of the Data Privacy Lab, former chief technology officer at the U.S. Federal Trade Commission, and distinguished career professor of computer science, technology and policy at the Carnegie Mellon University. Dr. Latanya Sweeney has three patents, more than 100 academic publications, pioneered the field known as data privacy, launched the emerging area known as algorithmic fairness, and her work is explicitly cited in two U.S. regulations, including the U.S. federal medical privacy regulation
known as HIPAA to all of us. She is a recipient of the prestigious Louis D. Brandeis Privacy Award, the American Psychiatric Association's Privacy Advocacy Award, an elected fellow of the American College of Medical Informatics, and has testified before government bodies worldwide. She earned her Ph.D. in computer science from MIT in 2001, being the first Black woman to do so, and her undergraduate degree in computer science from Harvard University. Dr. Sweeney creates and uses technology to assess and solve societal problems, political and governance problems, and teaches others how to do the same.
So happy to welcome you, Dr. Sweeney and really excited to hear from you. Thank you so much for coming. Dr. Latanya Sweeney: Well, thank you. It's fantastic to be here and what's really amazing is even though I'm actually on the East Coast connecting with you right now, I can feel the warmth of the West Coast already, but I really thank you Claudia and I thank Deirdre as well for this great invitation and the opportunity to be here today. Let me share my slides with you
and here we go. So if you're not seeing it, somebody will tell me fast quickly, I'll count on Jeremy to do that. So what I want to talk to you today about is how technology design is the new policymaker. Many of us don't know these people, we don't elect them to office, but yet the arbitrary decisions they make and design, dictate the rules we live by. Let me give you some examples. This is what a camera looked like at the time American jurisprudence was going to decide what were going to be the rules for taking people's photographs in public and the decision was made that you can take the photograph of people in public without their permission. And this is what a telephone looked like when American jurisprudence was going to decide what were going to be the rules for recording people's conversations. It fights through
issues fighted through multiple courts and so forth, and finally, the decision after a decade of wrangling was that you need to consent to record people's conversation. Take a photograph no consent. I do need consent to record conversations and then in 1983, this is what came along and what's really interesting if I showed this picture to undergraduates, they pretty much don't know what that is. So let me explain, that is the first camcorder. It was 1983, Sony brought the Betamovie and it was the first device that recorded both video and sound in a portable device sold on the consumer market. It's kind of a big thing and so right away the manager at Sony, Minoru Morio, pulls in a designer, Yuji Morimiya, and says look this thing is too big, I need something that will fit in the palm of my hand and Sony had developed, they were sort of the Google or the leading tech company if you will in many ways at that time and they had a reputation for building really solid, high quality things, but in a smaller package and so he says yes we're off to a good start, but I need something that'll fit in my hand. In 18 months, almost amazing feat,
they actually built something that literally fit in his hand. At the time, he's making this proclamation, everything about the camcorder had to be redesigned. They didn't even have a microphone that would be that small. If they just miniaturize the pieces, they didn't have a head and the heat alone would make it too hot to touch, but they do achieve the goal and in fact, their advertising campaign shows that it fit in the palm of their hand.
If this were a business school, story's over, end of story. They clearly hit financial success. We couldn't keep them on the shelves anywhere in the world. Not anywhere here in the United States either and that's usually how we tell the story of technology, but in the space of public interest, that's really just where the story starts. You see the design of the camcorder had no mute button and there was no mute button, with them being delivered here in the United States, there was no way to distinguish between whether I was recording people's conversations or whether I was recording photographs and so right away, people went to jail, so there was one story of a woman who taped that device to her child, puts him on a school bus, and the camcorder records all types of atrocities happening to the child from the school bus driver and one of the teachers. She takes the recording down to the local police department and they arrest her for illegal wiretapping.
There was a story of a protester in Boston who has one of the camcorders. He's on the sidewalk recording a policeman as he's arresting a protester. The policeman stops arresting the protester, turns and arrests the guy with the camcorder, and he faced seven years in prison. Now let's fast forward to today, the American Civil Liberties Union fought the case of the protesters and a few others through the judicial system and has won us all the right to record the police in the service of their duty in public and Pennsylvania passed a law that actually school buses now have a video camera on them that record both video and sound and so what's interesting about that is the fact that the camcorder itself had no mute button. We ended up changing our laws and one of the things is true about technology successes is that the design gets replicated as closely as a competitor can. So today, most of us shoot video using a phone and even today, even though it's just a software program, this actually would be the difference, it has no mute button, even though a software update could make that difference. I use this example to show you how technology designers really are the new policy makers. The arbitrary decisions they make can dictate the rules we live by. They can ignite technology society clashes
and what we've seen over the last few decades has just been a barrage of these clashes for the most part, without resolution. I would claim that we now live in a new kind of government, a new kind of technocracy, this is a new definition of the term, and what I mean by it, it's a government or the control of society or people through the design of the technology itself. So now that I claim that we live in a technocracy, let me give you some examples of what that has come to mean. I've just had the uncanny history
of being early to a clash, seeing an unforeseen problem with the technology, doing a scientific experiment that puts a fact on the table, and then watching a thousand great minds to follow. The first time this was to happen to me was I was a graduate student at MIT in computer science and a discussion ensued between me and an ethicist around whether or not a data set, a health data set, was anonymous so it's sort of like the circle on the left, it had visits and diagnosis and procedures, but it also had a demographics about individuals. Their month day and year of birth, their gender, and their five digit zip code, but it didn't have their name or their address or anything like that on it and so then it was believed that this data were anonymous and therefore, could be shared widely without concern. And so, I began doing a simple back of the envelope calculation and said look 365 days in a year, let's say people live 100 years, that's already 36,500 and the data had two genders at 73,000 possible combinations, but I happen to know that the typical five digit zip code in the United States at that time only had 25,000 people. That meant that that combination would generally be unique. The health data wasn't an arbitrary health data. It was health data on state employees, their families, and retirees here in Massachusetts and one of those people whose data was in there was William Weld. He lived in Cambridge, Massachusetts so for 20 bucks, I went to City Hall, I get the voter list and it came on two five and a quarter inch floppy disks.
When I say that to undergraduates, they don't quite know what I'm talking about, but I think some of us remember those black floppy disks. It came on two. Only six people had his date of birth, only three of them were men, and he was the only one in that five digit zip code. Those demographics were unique in the voter list and they therefore were also unique in the health data and it allowed us to put his name to his record. When I went on and looked at well how many others in the United States would this be, we used 1990 census data, I was able to estimate that 87% of the population of the United States is unique based on date of birth, gender, and zip code.
One day I'm a graduate student and literally within a month, I'm testifying before Congress. Why? Because that wasn't the way it was done just with that one data set, that was the best practice around the world and laws around the world changed. Many of them cited me like HIPAA for example and it really began launching the field that we now think of as data privacy and if we look at data sharing today, we haven't really made a lot of progress. This is sort of a map of all the places a typical patient's information might go and the ones that you see in color, in the dark, our flows that aren't even covered by any regulation here in the United States. It's whatever the contract or the arrangement allows, so even though it was the first time that I was to do an experiment and sort of expose it and we've got thousands of great minds working on it, data privacy continues to be up for grabs. The next time I was to have this kind of experience was about 10 years ago, when I became a faculty member here at Harvard.
I was in an interview with a reporter and I wanted to show him a paper, so I type my name into the Google search bar and links to the paper showed up, but so did this ad implying I had an arrest record and the reporter says forget about that paper, tell me about the time you were arrested. I said well I've never been arrested. He says then why does your computer say you have and we battled back and forth for a bit and we began investigating and we see that every time we type in a full person's name and if it begins with Latanya, you get an ad implying that that person had an arrest record and if you typed in Tanya a full person's name, you got a neutral ad to say want to find more information. So the question was why was this happening. It's not like you can just type in a first name. It's not like you could just make up a name. It had to be that it was the name of known people and so, he says oh I get it it's because you got one of those Black sounding first names and he was an Italian American, so what do you mean by Black sounding first name? But when I, oh my slide's messed up, oh there you go, but when I typed in Latanya into the Google search bar, I see all of these Black faces or Black and brown faces staring back and when I type in Tanya into the Google image search, I see these White faces staring back and so we realize that right away, it really is true that there are some names given more often to Black babies than White babies and for reasons we couldn't understand at that moment. Why was this algorithm so distinguishing? So I pay the money, I go into the account, I show that there's only one Latanya Sweeney in their database and there's no criminal record.
Similarly, for Latanya Farrell, even though both names got arrest ads and Kristen Lynch, which always got neutral ads, I find more information, even though there was a Kristen Lynch in their database for which there was a criminal record. Now discrimination is not illegal, but what is illegal is for some groups of people in certain situations to be disadvantaged and one of those groups of people are Blacks and one of those situations is employment. If Ebony and Jill are both in competition for the same job and a potential employer types in their names into the Google search bar to see what kind of information is online about them, Ebony is put to a disadvantage that Jill is not and the U.S. Department of Justice has an 80:20 split. It's not just it has to be a little, it has to be large, so I do a study. I do over 100,000 look ups across VPNs around the United States
and what I learned is that if your name is given more often to a Black baby than a White baby, you are 80% more likely to get an arrest ad than not and so, this became the first example of where an algorithm was sort of subject to a civil rights investigation and of course it launched the area of study known as algorithmic fairness. Let's kind of look at that example a little bit to think about where this problem could be coming from so if we think of the technology as the issue not itself into the technology, we can just put that as sort of this peach colored box. So one thing that the algorithm had is the bias could be coming in the setup data like in other words, what ads would the advertiser place, were the ads themselves biased.
The other thing that's interesting is that the bias could be coming from the feedback of clicks so the Google algorithm is trying to optimize its income. It only makes income when people clicked on the ads and so could it be the case, is this really a reflection of society? Society was clicking more often on ads given more often whose name of first names were given more often to Black babies than White babies. Either way, we were able to measure the output and see that there was an adverse impact. Later, I worked as a chief technology officer at the Federal Trade Commission and there were some discrepancies with respect to the advertiser.
But for our purpose in this talk, the real issue is that the Civil Rights Act is up for grabs. Our ability to know that that had happened and be able to investigate it and of course what made us possible for us to investigate it was the fact that we could look at the adverse impact and we could do that test. This is a slide. I'll give you another example, Alvaro Bedoya and I used to go around and kind of do a tag team talk
around algorithmic bias years ago and we had this and Alvaro had made up this example and in this example, he said suppose you have a company that has a lot of resumes and they want to use AI and build an AI, offer an AI service to employers, so the employer gives a job description and they make recommendations as to who the employer might hire so the company says these are some people that you might hire. The company says we'll take the ones in green. We don't want the ones that are sort of shown here with the x's and over time, as the algorithm learns the preferences of the employer, it will only detect the employer's bias. In this case, only young people. Now we have a law against that too, but the interesting thing here is it's really hard to determine, to measure that adverse impact. Even though we have a law against it, what's our ability to actually be able to know that it's happening. We don't have access to the selections that the algorithm gave to an employer and we're not able to really learn that the employer is biased and the technology reinforces it so that means that something like the Equal Employment Act is up for grabs. Many of you have heard about these recidivism algorithms. These kinds of predictive algorithms are used throughout criminal justice. This particular example was made popular by ProPublica and Julia Angwin and what is happening here, the idea of a recidivism algorithm is it's going to give a score, a risk score as to how and try to advise the judge as to whether it's a good idea to let a person has been arrested out on bail or not.
And so what we have here is Brisha Borden. At the time this photograph was taken, she's 16 years old. She and another girl are walking down the street and they see a bicycle and they clearly intend to steal this bicycle. As they're walking, a neighbor comes running out yelling so they dropped the bicycle and the two girls run. About two blocks away, the police stop Brisha because another neighbor had called the police so she gets arrested for the theft of the bicycle. When she goes before the judge to decide whether or not she's going to get out on bail or what that bail might be, she gets an eight. A really high score, you know she's a real danger to society
and she had a few juvenile misdemeanors before, but you know the longer one stays incarcerated, the more difficult and the outcomes become much harder for that individual. Meanwhile, Vernon got a low risk score. A three meaning yeah you could just let him out on no bail or very little bail if any at all and even though Vernon had armed robberies and so forth before. So here we have an example in the criminal justice system where the algorithm, which is packaged to be incredibly neutral like you know it's a number, it has these charts, it looks like it's a very independent kind of calculation. But in reality, this is a reflection of the decisions that have been made historically in this part of Florida. That is the training data itself represented this bias. This bias is being replicated and packaged in a kind of mathematical way, but even though, it still has all this adverse impact.
But in our society, it means that criminal justice is all up for grabs by how these technologies work. I'll give a couple more examples. This is Joy Buolamwini when she was a graduate student at MIT and what's happening on the left is she's looking into the camera, she's looking into the camera screen of a computer, and the problem is the computer is looking back, but it doesn't see her and it doesn't see her until the screen on the right when she puts a White mask in front of her and then, we see the active appearance model triangulating to show her eyes, nose, and mouth.
And what Joy's work, of course, shows us is that face detection and face recognition algorithms don't work very well on darker skinned people and women and there's a reason for that. During my time at Carnegie Mellon, we did a lot of work on how to de-identify facial images caught on security cameras and things like that and during that time, there were lots of contest and the contest was for face recognition. You know if you won the contest, you would win a larger amount of money and more rewards and so forth, but the data in almost all of these contests were of pale male faces and so as a result, as face recognition is growing as a field and and learning more about its craft, it's primarily being trained on data that was not diverse and so as a result, we end up with this adverse impact.
I could go on and on through various constitutional rights like free speech and freedom of assembly. We could go through lots of other national and federal laws. We could even go into state laws and just a huge swath of them are being redefined by what technology allows or doesn't allow. I would argue that every democratic value is up for grabs. The third time I was to have this sort of uncanny experience was in 2016. In 2016, a group of students and I set out to follow the election of 2016 and asking questions about how technology could make it all go wrong.
One of the things that we did, we were one of the first to find these what we call persona bots on Twitter and what made these persona bots so interesting is that the people tended to have real sounding names and a realistic photo and their stats weren't that they were just flooding their followers all the time you know, so you don't see a huge number of tweets and they don't have a huge number of followers. They have some realistic number of followers, but the interesting thing about them is that they're a bot delivering misinformation and their followers are all human. The other thing is they did identify a political person. They weren't retweeting. Almost all of their content was firsthand tweets with almost always a link to a misinformation site. And so, as we began to learn more and more about these persona bots, we realized how important a role they were playing in the perception of individuals around what was true. As we got closer into 2016, we also found a lot of problems with voter registration websites and with polling places and how easy it would be to go in, impersonate a voter online, and change. By changing their voter file, one could literally make it where their vote won't count because they'll show up at their normal polling place and if you had changed their address, they start yelling and screaming because they're not on the voter role and they would get a provisional ballot and in many places, the provisional ballot wouldn't count.
We modeled that experiment and found out how easy it was for just only about $9,000 to shave a couple of percentage points off of the election, something that would be hard to notice when you're shaving it across the state because a version of this did occur in Riverside County in California where Republicans came to a closed primary and instead of getting the Republican ballot to vote in that primary, they got everything but and when we looked at it, hundreds of registrations had been changed. I use these examples to say hey polling place, misinformation, disinformation, propaganda, all of our values and our democratic state itself is up for grabs by what technology design allows or does not allow. Now, at this point, everybody's kind of depressed if you've been following me so now what I want to say is what do we do about it, how do we move forward. When I left the Federal Trade Commission as the chief technology officer, you could see the tidal wave of these clashes coming and I thought back on the experiments that I had done and even though these experiments had started lots of great minds thinking, they weren't really big experiments. It wasn't really rocket science. But also along the way, I learned some tools that were quite effective across the lifecycle of technology, how do you intervene. And so when I went back to Harvard, I began teaching these models to students, stakeholder design as a way during design to ensure, to represent a broader spectrum of stakeholder issues, to help issue spot early for a problem that's easy to solve during design, or scientific experimentation. A lot of what you saw later when technologies in the marketplace, what do you do to demonstrate to the world that there really is a problem.
And then, some of the supporting ways that we could get help to those who would normally help us whether that be civil society, regulators, journalists, and others. And we set up and students love this course and we started, you know the students call it the Save the World courses and the students did amazing things. Over the years, the work from those students have changed laws, have made new regulations and modified regulations, changed business practice in a lot of the high tech companies. We have a journal where we publish a lot of it in tech science and let me give you an example of just a few of them. So one of the this is an example of price discrimination on the Princeton Review. These were undergraduates who did this work
and many of them had recently taken the SAT's and they remembered that they had to give their zip code before they could get a price. So they mind the prices out of all the 32,000 zip codes around the United States and they found that in general, people on the East Coast pay more for the same online tutoring service say then people in the Midwest do, but what's really interesting is if you were to focus in a little closer, you see that it's not everyone in the Northeast paying more. If you're Asian, you're almost twice as likely to pay the higher price. This is an example from Aran Khanna. So for years, the industry media had pointed out a bug, well Facebook called it a feature, others of us considered it a bug, but basically every time you would send a message using Facebook Messenger, it would record your GPS location and embed that with the message sent. And so the industry media had not been able to get Facebook to change this so Aran just had a plug-in that you downloaded that goes into Chrome and then you could see where your friends and friends of friends were, plot them on a map as they went throughout their day using Facebook Messenger. In nine days, Facebook changed that and so one kudos for Aran for making it happen and kudos Facebook for finally correcting it.
But I think what really made this story go viral was they had hired him as a summer intern and the day before he's supposed to show up, they call him up and they canceled his job. They said oh we're sorry because you did that work, we just don't think you're fit to work at Facebook. And so you can imagine he calls me up, he says I took your class and now I don't have a summer job, but not to worry, he went on to get a much higher paying job within a week. He ended up with a Ted Talk, a Time Magazine article, in fact articles around the world were written about him, and so it really underscores to us also the way that companies respond when they're faced with these issues. And I would put that in high contrast to Airbnb so students were able Researchers had shown before that Black hosts versus White hosts in New York City for the same comparable properties in the same neighborhood made about 12% less. And so, our students had shown that Asian hosts versus White hosts made about 20% less in Oakland and Berkeley California. Airbnb sent a group of people to meet with us and then changed their platform so now, if you go to Airbnb, they actually set the price to make sure that this type of behavior doesn't happen, a radically different response.
You know, even when you're building technology for good, it still doesn't exempt you from having problems and so, even in something like public comment servers, this is work by Max Weiss was able to show that AI has gotten so good that if we put in 1000 comments on a public web comment server that the government can't tell the difference between whether a human wrote it or whether a bot wrote it and what do we do about that. I'll just give you this last one and then we'll kind of click on to get to question and answer. So, this is a paper that just came out last week from us and so last year, there was a big uproar about Facebook around the advertising and HUD about being able to target by race and ethnicity you're at and sometimes that can create a direct conflict with a federal law we have and in other times, it could just fuel disinformation. So Facebook said oh we'll get rid of those, we're not gonna do that anymore, we're going to do something different and so they introduced an alternative and what this study shows is that the alternative is even more identifying by race and ethnicity than the thing that they used before and how easy it is to target individual groups with disinformation or for credit or housing or other types of discriminatory effects. I don't want to leave you, in addition to sort of building experiments that try to expose problems, we also are offering new technologies and new solutions, so I'll just give you two of them and we'll close out. One of them is VoteFlare so we set up VoteFlare for the Georgia runoff election and the idea of VoteFlare is aimed at monitoring your voter registration and mail-in ballot in real time and contacting you if something needs your assistance or if something changes in a way that could make a difference for you.
And when we set it up in Georgia, we just received replies from thousands of people who said that it was a great comfort to them and so we are continuing VoteFlare and rolling it out in 2022. We also have an alternative privacy platform, so you still use apps and web services like you do now. The only difference is that in MyDataCan, a copy of the information goes into your private storage where you can then use it yourself, do things that you want to do with it, and also some apps will just only ask for your permission to use the data you've brought into MyDataCan so that it can help you live your life better. You can also double encrypt the data so that even us as the holder of the sponsor of the platform, we can't even see that data. What I hope you've taken away from this talk and I look forward to the Q&A is that I claim we live in a technocracy that every value that we may hold dear is up for grabs by technology design and at first when you think about this talk, you might say oh this is a talk about Latanya or me or this is a talk about my students, but really this talk isn't about me or my students. This talk is really about you and all the ways that we want you to be able to save the world. Thank you.
Jeremy Johnson: All right, thank you so much Dr. Sweeney. For those of you in the audience, if you would like to ask questions, you can feel free either to use the Q&A feature or to raise your hand and I can allow you to speak. Dr. Latanya Sweeney: Let them speechless? How could that be?
That's impossible. Hannah Pak: Hi, thank you so much for giving your talk. I think the one question that I have just like taking away from all of this is for individuals who are not say students studying data science, students becoming product designers, you know people that feel like they're very much on the user end, how do you feel that we can make a difference as well, I mean, would you say that mainly it is just in being educated and aware of digital technology, digital literacy or do you think there are other things that people can do as well? Dr. Latanya Sweeney: So the first thing I would say is the students who take the courses that have had this kind of impact you know have come from all kinds of disciplines, so you should not think that this is a data science thing or even a computer science thing. In fact, some of the most like I think only one of the examples that I showed you in those examples was from a computer science student. All of the other students came from other disciplines whether that was
statistics or even we have a fantastic finding from a student in psychology for example, she became interested about sort of about the use of wearables crossed with game controls and what would that mean for just-in-time purchasing on video games and so forth, if you had wearables that could give biometric feedback to what the user was experiencing. And so we've had some fantastic work across disciplines and so that part of your question you know and in fact, we have just launched a new website I should probably put it in the chat somewhere, but it's called techstudies.net and the idea of techstudies.net is that we offer lots of experiments that we haven't done that hold the same kind of promise and we're offering those to students throughout the
Public Interest Technology University Network to do any of those experiments and we'll help support you by whatever means you need. The other part of your question though was what does an individual do? Right so maybe I'm not going to engage in research, maybe I'm not going to engage in developing something alternative, but what can I do as an individual. And on the one hand, there's not a lot alone you can do because the technologies have become so large and so much a part of our society, so that so many of the examples that I gave is irregardless of my independent behavior. You know even if I didn't ever use Google, those ads still showed up under my name and so there's nothing I could do to stop that right, not as an individual. And so I do think we're at this moment in time where it is about collective action and a broader understanding and empowering those who are aimed to help us. Hannah Pak: Thank you so much.
Jeremy Johnson: All right, great. We've got a couple questions from the chat. I know one was from well earlier, but I remember it and so if I could just ask on behalf of that person, with the work that you had with the browsers and with Google searches, they're just asking you which browsers you tested it on and if that was just on Google search that you had tested. Dr. Latanya Sweeney: So we no well it was on Google's delivered ads and and both on Google's website itself, but also on lots of websites where Google delivered ads like reuters.com and you know the Chicago Tribune and a lot of news websites because news websites also had a search feature that when I typed in my name, might come up with articles related to a name but then will trigger these ads and so those were the websites that we used. We used Python so it's not that we were using a particular browser. We use VPNs around the country and flush the cash and so it wasn't about cookies either. Jeremy Johnson: Thank you. I've got three more here that I'll go ahead and read through. Anybody else if you want to ask again, feel free to raise your hand.
We have one question from Judith Mallory who asks what are your concerns about how technology can be used for surveillance purposes, which we know has been historically used against IPOC in harmful ways. What kinds of regulations do you think should be implemented on a local level? Dr. Latanya Sweeney: Well, I mean so surveillance technologies it just continued to progress in terms of you know everyone's worst nightmare. So in the 1990s, which is where this talk started. It was 1997 when I did the Weld experiment. In 1997, the group who was most captured in data that can be most easily re-identified were people who couldn't pay for privacy. You're in health data because you couldn't afford to pay out of pocket for your health insurance. You were in consumer loyalty data because you wanted these discounts and if you received the public service, then we knew lots more data about you.
And so at first glance, we could immediately begin realizing that people of color and BIPOC are even from the very beginning already vulnerable, but we also realize that it's also like the canary in the mind because today you can't pay. It doesn't matter, even if you pay for your health insurance out of pocket, your data still goes in the health data right. Loyalty cards really aren't a thing anymore so and your credit card, most of us pay by credit card. If we have a credit card, we'll pay by credit card and those records about what we bought these groceries and so forth are all sort of in a secondary market. But I think the question goes beyond that so that's 1998. It sort of becomes this preamble to what everyone else was going to experience and I think that's true today too.
I mean you know every 10 seconds, your Android so when I was at the FTC, we did this study and every 10 seconds, Android phones are dropping your GPS location and who gets this data and who's selling this data? Well, there's a whole marketplace on your GPS data. Right, there are a lot of buyers and sellers of it and a lot of reasons that that's happening, which in some ways makes us all already a part of the surveillance. Or if we look at Clearview, the ability to say I'm going to take social media posts and images that are out there and I'm going to use them for face recognition. At first glance, in almost every instance, BIPOC is far more at a disadvantage than others. But it's not just a BIPOC thing and that would be a mistake that everyone has to understand. It's all of us and it's everyone in terms of the surveillance state. Undergraduates are amazing because they just show you how time has changed. So this year we did a survey. We repeated a survey had done 20 years ago because that's how old I am.
And what was one of the most amazing things about that survey was that undergraduates today you know actually trust the government more than they trust private companies. Now, the reason I find that such an interesting finding is because if we go back to the 1970s, so much of our privacy laws and our privacy infrastructure was around not trusting the government. And in fact, the closer you get to Nazi Germany, the more the stronger the rules are of not trusting the government and now we get this interesting flip so and the reason that's important is because when you trust the government, the harms of surveillance can be the most dramatic when the surveillance is being empowered by government.
So it's not to say that the harms are not important and dramatic whether it's employment or housing or disinformation campaigns that are meant to disenfranchise a group. Those are incredibly important, but when it gets to government, it could be your liberty as well. Jeremy Johnson: Great, thank you so much. I'm going to keep going down the line. Dana asks what are some suggestions to regulate some of the most pertinent data risks that you mentioned. What controls are necessary and how can these be disputed by corporations that have used our data? Dr. Latanya Sweeney: So
the worst data risk, I'm not even sure I articulated the worst, I just articulated some, we could have a two hour conversation about all of the inappropriate uses of data that may be harmful to people, which is really the the crux of this and the problem here though and of course it's not just stated, it's the data from particular technologies. To answer the question of how do I unpack the harm You know the question kind of presumes that there's a policy that if we were just putting this policy in place, it would all clear up and I mean Deirdre Mulligan might come up with a policy, but I think for the rest of the world, we're still struggling for what that silver bullet might be and let me explain why it's such a difficult thing to come by. First of all, notice that in this talk, the issue isn't even one issue. It's not like we're talking about only the criminal justice system or only recidivism. We were talking about employment. We're talking about unfairness and credit cards, ads. We're talking about all kinds of different things, all aspects of our life.
So one, we don't you know how do we get our head around just a simple scope seems to deny us the ability for it to be one day to answer. Even just if we go back to the original work and health privacy, even in something like health privacy, we see the difficulty itself because we don't even have a law that controls all health data. At least half the flows that I showed earlier aren't even covered by any of these policies right. And even if we made them all covered by policies, we've been able to show that the current policy has problems too with re-identification. There's another dimension at play too and the other dimension at play is the speed at which technology can go from an idea to the marketplace and this is really and so therefore, a lot of the issues have to happen and play out in the marketplace. And so, I don't know that we've ever been at that point in our history before so for example, we've had amazing technological industrial revolutions. We've had two industrial revolutions that have come before this era,
but when we look at something like cars, it took them a long time you know you could come up with a car and you know cars didn't have good regulators and so people would like push the gas a little bit, the car would take off and hit somebody and kill them. So, there were all these kinds of problems with cars, but cars needed roads and the roads had to be negotiated with cities and towns and it was in those moments that we see the regulations and the responsibility of what an automobile had to have and do began to really take hold. We don't have that in today's technology, you and I could sit here, come up with a technology, stay up a few nights. If we get people to adopt it, we get a million people to follow us on TikTok and and click on the ad and click on the website or the download the app, we're off and running and we can keep changing it as we go and there's no opportunity for society to weigh in and really even understand. So, I don't think there's a magic answer to how to resolve these data questions. We can think of lots of band-aids in particular parts of this problem, we don't have one and then of course the last part of it is and how do you get a new policy. Policy moves as a function of months. Technology is moving as a function of days.
We have a Congress who over the last decades you know just will not respond to slow burning problems whether that's climate change, whether you know we could just have a long list of all the slow burning problems to which we have not really responded and I think we put technology in there. And so we need a much more comprehensive approach and that comprehensive approach you know I have ideas for every part of the technology lifecycle, but it really also requires all of you to look for the unforeseen, put facts on the table, pushing the agenda. Jeremy Johnson: Great, thank you so much. One more question I think we probably have time for comes from An Little who asks
what are your thoughts on how a team developing learning analytics software can avoid falling into these types of pitfalls. I'm thinking some kind of review process to look for ways in which our work might perpetuate pious. Dr. Latanya Sweeney: So, if you're just developing your technology, then what I would suggest is that you bring in a whole group of stakeholders and tell them and you ask them each or you do a study around stakeholder assessment of the technology you're by and you're going to find a zillion issues like you're not going to find one or two, you're going to find way more than you could possibly account for in your design so then, you got to do some kind of risk assessment, that is you want to know of all the possible issues that could happen, which ones are the most important, what's the one, two or three one that's the most important that I need to address and then you answer those in design.
Not after you have your minimal viable product, but in design itself, you make it a part of the solution and part of the guarantee when your technology comes out. That allows you to come out with a technology, one that can proactively respond to society about well it's not going to have these problems and two it ensures more viability for your product with not likely to have these kinds of adverse impacts downstream. Jeremy Johnson: Actually, I think we probably just have time for one more real quick. I'm gonna allow somebody to talk real quick. Is Godwin here? Dr. Latanya Sweeney: I see Godwin's name up there.
Jeremy Johnson: Godwin or do you want me to read it from the Q&A. Dr. Latanya Sweeney: I'll take either one, but it's good to see you Godwin. I hope all is well in Canada. Jeremy Johnson: Okay, I'll read out loud what was in the chat. Now that face recognition technology is no longer used by certain police forces and Facebook, what would you think about the future of face identification technology that has been demonstrated with bias? Dr. Latanya Sweeney: Well, so the first part of that is now that it isn't used is a question not a statement.
So you know I called out Clearview earlier, so you know I would argue they are using it. I don't see them not using it, maybe the rules are different in Canada I have to admit I'm not up on that, but in the United States, it might not be the case that they're buying the face recognition software directly, but their ability to use a face recognition service like through a service like Clearview is certainly true and I think you can look at the sales of Clearview to get a good sense of how well that's going for them. And then the question of you know what do you do I think Godwin's the second part of the question is what do you do about the bias found there. Or do I think that is now free of bias and there's two problems with law enforcement and face recognition. One is the kind of training data problem that we talked about before and then, even if I go back and I retrain the systems with a more representative set of data that's more globally in-tuned, I still have a problem in some places of face recognition where the data samples I'm going to use from the gallery I'm going to draw from has already been oversampled because of over-policing so there's still some complexities that have to be answered when it comes to the police use of face recognition.
Did I lose you guys? Jeremy Johnson: Thank you so much. I think that's all the time we have for questions. I'm going pass it off to Deirdre. Deirdre Mulligan: Dr. Sweeney, I couldn't have been more delighted that you were able to join us. I appreciate not just your research, but the way in which you continue to empower your students and grow the community of people who follow in your footsteps. I think you know your impact on the public interest technology field has been incredibly outsized. There's like a whole team of people who you've trained who are affecting our lives in positive ways and clearly meeting your goal of helping people understand how they can change the world.
And I am really grateful on behalf of Cal NERDS and the D-Lab and AFOG and the Fiat Justice Scholars program that we've been able to put together here at Berkeley to have you join us. I hope you got some sense of the enthusiasm of the community and the students we have here. I'm sure they're going to be pushing us for a lab like yours, but we will be encouraging them to look at your website for projects they might pick up and to pitch things for your journal, which I think has been an incredibly important way for students' work to get out in the world and have influence and you will continue to be a kind of guiding North Star for many of us.
Dr. Latanya Sweeney: Well, it is mutual. I just look forward next time to be in there in person. Deirdre Mulligan: Yes, we will have you back out when we can all actually share a meal. Dr. Latanya Sweeney: Thank you very much. Deirdre Mulligan: Thank you so much Latanya, Dr. Sweeney. Dr. Latanya Sweeney: Thank you, thank you very much.
Deirdre Mulligan: Bye.
2022-01-19 20:22
