Technological Revolutions and Art History, Part Three: Alonzo C. Addison
- All right. Good morning. It's our pleasure today to welcome all of you to the third session of the four-part symposium, Technological Revolutions and Art History, co-organized by the Frick Digital Art History Lab and the Museum of Modern Art. I'm Robert Kastler, director of imaging and visual resources at MoMA. Some quick housekeeping notes. On your screen, you can see a list of the titles in the series and a link to sign up for part four which is the last session of the series taking place on March 11th. In the interest of time, we're not gonna be giving formal introductions for each speaker, but you can find the speakers' full bio is in the chat box.
Also, we're gonna hold the question and answer period at the end of the presentations today. So as questions arise, please feel free to type them in the Q&A box. The Q&A will be run today by Luciano Johnson of the Frick and Lou, I think I'll be helping you out with that. Today we bring you part three, Cultural Heritage and the Ethics of Digitization which we'll explore how issues regarding access and discoverability impact research in art history. As digitizing becomes a necessary process to ensure preservation and access to the world's cultural heritage, places, objects and artistic expressions, we rely on technological advances to meet these needs, but in what context do we apply the current knowledge of our tools to capture these arts and artifacts. With respect to other cultures, how do we approach the process and results in a socially responsible and ethical way without introducing bias and with an eye towards understanding the ideas surrounding ownership and authenticity.
For those participating in this work, questions arise in the ways resources can be provided and applied. Knowledge sharing of best practices, practical applications and delivery platforms are important, but we must also avoid digital colonialism in regards to many under-resourced regions and communities. Advancements in the digital realm, AI and computational imaging, have led to many new possibilities for representation, such as 3D imaging and application of technology that is currently able to create very realistic virtual models, but how do we quantify the accuracy of a digital surrogate and what supporting data, metadata and methods of aggregation are needed for research, awareness appreciation, and share-ability in order to meet the needs of today, as well as understanding those of the future.
We're happy to have four presentations today from experts in various parts of the field that at firsthand understanding of these issues and will be sharing their experiences and perspectives. So now I'd like to turn it over to Alonzo Addison for the keynote, Digitizing a Disappearing World, Crowds, Clouds and Culture. Alonzo is an international leader in the application of digital technology in the service of society. For more than a decade, Alonzo has served special advisor for the World Heritage at UNESCO and author of the popular book, Disappearing World. Welcome Alonzo.
- Thank you very much. Thank you to Luciano, Louisa Wood Ruby and all the rest of the group at the Frick who've gone to great efforts to organize this today. It's a pleasure to be here. I wanna talk today about Digitizing a Disappearing World and Crowds, Clouds and Culture. So hopefully my slides are coming up.
I can't see from my own screen, but, I wanna start with crowds and clouds and at The Frick Collection, there's a nice Turner painting, "The Harbor of Dieppe" in 1825, a crowded Harbor. The world's more crowded than it was then. So I wanna reference that a little bit. I'm not an art historian. So for many in the audience if you are from the art historians, it's not my field, but I thought I'd include a little bit.
I also wanna talk about clouds and we'll come back to both the crowds and the clouds later, Constable's cloud study, also from the same period is in the collection at the Frick and crowds in clouds as metaphors for some of the challenges in the digital world. So today sort of going through it quickly, I'm gonna talk about heritage at risk in a disappearing world. Talk about the shared responsibility that puts on all of us to safeguard our past and then move a little bit to digital's promise, sensing hope. And finally, some of the challenges and solutions that crowds and clouds bring and give. So starting with a disappearing world, heritage is at risk today.
And I think we all know this, but let me go through some examples to show you how and why that's going on. So Stonehenge, it's a world heritage site, world renowned and yet it's crowded and it continues to be crowded. People love it as they do many great places, great works of art, great books and our culture is getting a little overused. Once a year, Stonehenge opens its gates to allow a festival, but overcrowding is also hurting the natural sites, world heritage coverage cultural and natural treasures. This is the Belize Barrier Reef, so pollution and the effects of man and it's even affected great works of art.
The caves that Lascaux, what was called the Sistine chapel of prehistory. Too many visitors, required an upgrade of air conditioning systems and with that came workers that brought in a fungal infection and slowly Lascaux, which you can no longer go in, the original has been overcome by a fungus affecting the works of art. So Lascaux itself, 20,000 years old, rediscovered in the 1940s, closed in the sixties. Lascaux II, down the hill, opened in the eighties and the most recent, a $66 million a year, 66 million Euro virtual replica opened in like 2016, using digital technologies in part to help replicate the original, but a full scale replica down the hill from the original site. So you don't go in the original site anymore.
We'd loved it, sort of to death. So crowds are effecting heritage, but clouds are too literally in the sense of climate change and flooding here in Venice. You may have seen in the news and in recent time the situation that's been going on in Venice. It's not just Venice, it's Prague, it's many places around the world, a changing climate. In Beijing, acid rain and pollution and winds off the desert are affecting the monuments, but also the clouds in a more metaphoric sense, the clouds of war at Preah Vihear, on the border of Cambodia and Thailand, there's been an ongoing conflict over who owns heritage and in Palmyra, in Syria, many remember ISIS and sort of the tragedy there. Some of the monuments you can see here on the left before, on the right, after or Bamiyan in Afghanistan, March 11th, six months to the day before September 11th, 2001 Bamiyan was destroyed by the Taliban.
The Buddhas were. So if you wanna learn more about that, I've written extensively over the years. I think Robert mentioned at the beginning, but moving on from a past in danger to sort of our shared responsibility to safeguard and protect it.
In 1864 in Yosemite, the US created sort of the precursor of national parks. Yosemite was protected as a place for the enjoyment of all. There you can see it in a Bierstadt painting. A few years later in 1872, the world's first national park, here in a painting by Moran, was created.
1872, remember that date. As we move forward, 1945 UNESCO was born out of the ashes of the war with that founding statement about building peace in the minds of all. Shortly after the founding of UNESCO, one of the great, sort of first great cultural challenges it faced came upon it. In upper Egypt, the Aswan High Dam was proposed and being built and slowly flooding the sites there. So a big international campaign involving school children from around the world was organized. At the time a huge international effort and funds were raised and the monuments were moved out of the rising waters onto a higher ground and an Island.
The way it was done at that time, probably would not be the way conservationists would do it today, but at the time it was sort of a groundbreaking effort. A little hard with the virtual. My Zoom layers are overlapping my screen. So moving on, in 1965 at the White House, there was a proposal to create a World Heritage Trust to sort of help protect the great treasures of the world. And in 1972, a hundred years after the birth of the first, world's first national park at Yellowstone, World Heritage Treaty was created in Paris with the idea that our cultural past is the shared legacy and the shared responsibility of all humankind.
And so that key idea that heritage is in danger and yet that it's our shared responsibility to protect it sort of underlies a lot of efforts around the world today. There are quite a few UNESCO conventions or treaties, some dealing more with art, some the trafficking of stolen art and objects, some dealing with cultural sites and more recently dealing with cultural expressions and heritage. So now I wanna move to the digital and talk a little bit about the promise that digital technology is bringing and then from there to some of the challenges. So in this picture, you can see a digital model of Notre Dom. Sadly just a few years ago, as many may remember, on the 15th of April, Notre-Dame caught fire.
There was construction work and an accident on the rooftop where scaffolding was, a fire started and much of the building was destroyed although the great front window survived. Luckily there was a digital model, which is proving useful to a huge team that's working on how to restore and bring back and strengthen and stabilize the monument. So that was a 3D digital model, a 3D scan and we've had a half century of progress, in fact, working to digitize our heritage, our cultural heritage, sort of begins in the 1950s. So in 1953, the phrase to digitize first came into use. It wasn't really till the sixties that we began having devices or interfaces into computers where we could use pens to get vectors onto a screen. In the 1970s, 2D scanning largely used in the art world and then in the eighties, film and motion and then in the nineties 3D and position with the global positioning system location data started to be able to be brought into the machine.
I mean, looking back, it's only a 35 years since, you know, the mobile phone came into being, but the digital devices, there's quite a suite and so I tend to group it into three groups. Primarily today, I wanna talk about the devices to capture or digitize, but once you bring something into a computer, you also need to manipulate it, you need processing power and then you need a way to get it back out. But talking about sort of capturing data, this is a picture Of the Colosseum, digitization program we did many years ago.
But really sort of in the cultural world, it begins in 1976 with Ray Kurzweil, famous early computer innovator and inventor, came up with the first flatbed scanner, in fact, for working with the blind, but the first CCD flatbed. In the late 1970s at MIT, Michael Naimark and his colleagues put a mounted video camera on an old, there you see, an old station, early station wagon, to film in 360 degrees the world around him and this project was sort of laughed at in the federal government. It had been funded by DARPA as being One of the biggest wastes of money at the time, but today we all use that same early technology to get around with Google street view and Google maps and now the technology is being applied to heritage. We began a project about a decade ago with Google to take that technology into cultural sites around the world and record them. The Google Cultural Institute, the World Wonders Project, a lot of that's online today.
In the seventies and eighties, photogrammetry taking sort of stereo pairs of objects and getting three dimensional data took off. By the eighties, you know, you are either with a kite, there a student with a kite or a very expensive other option aircraft. We started doing a lot of detailed photography, stereo photography of heritage.
Some of it happened as early as the world wars, but today of course, drones and other things are commonly used to capture data. 3D scanning in turn by the early nineties was coming into play. This is the original prototype of the Sirah laser I was involved with. And that device we used on tests early on, 3000 points a second, today, depending on the device, 300,000 to millions of points a second, three dimensional coordinates one can digitize or capture about the world around us. In addition to sort of 3D scanning and digitizing, locational data's become important and the advent of the GPS system sort of in civilian use in the nineties, made it possible to get accurate coordinates, but very expensive devices and services then, today, you know, more power is, more locational power is in the typical mobile handheld device.
Hand in hand with digitizing the world comes processing that data. Again, here is the second image from that project at the Colosseum in Rome. So there's the 3D data set, what we call a point cloud. Again there, clouds are coming into play, but computing power, millions of dollars of 3D graphic computing power in the labs, Berkeley in the early nineties on the left and today the equivalent is in a handheld device. And in turn the ability to create alternate realities, virtual reality, AR, MR, virtual models of cultural heritage has taken off, from early videos several decades ago, to vast virtual fly throughs with billions of data points, it's now possible.
Virtual reality, actually virtual reality in a way stereo imaging goes back to the 1830s, but virtual reality, the idea of sort of projecting images into our eyes took off in the 19, late sixties with Ivan Sutherland Sword of Damocles there, by the 1990s took a lot of compute power and equipment, but it was common in labs. And today they make foldable cardboard devices that you can slide a phone into to see sort of stereo virtual tours of places. Immersive visualizations, 25 and 30 years ago. There's a picture of a VR room, built, custom built in Gifu Japan.
A huge facility with a room that one entered into and the floor, the ceiling, and all the walls lit up with sort of a virtual presentation of wherever you wanted to travel. Millions and millions of dollars to produce the systems then and today, as in Sarah Kenderdine the right image of the Dunhuang caves in China with their fantastic paintings, you can take a virtual visit with just a tablet position oneself in space or as in this case, you can go into a virtual voyage to the site in an immersive world. Let me see, I've lost my screen. So AR and VR is now sort of widely available.
The technology is built in the next generation of phones. So one can expect it to be used in museums and cultural institutions much more in the coming years. So we've talked about disappearing world, heritage under threat. We've talked a bit about our shared responsibility and the need to protect it and the ability to digitize and capture and record it. Now, I wanna really step into some of the challenges, the ethics and the, some of the hurdles one faces with digitizing the past. Crowds and clouds again, will come into play.
So, I mean, the first question is why record? Why digitize? Why do we want to make a digital representation of a painting or a book? Is it for entertainment? Is it for education? Is it for greater access? So those that can't visit the place where the object or artifact is can participate, or is it for conservation? And all are valid purposes, but I think often we tend to lose sight of why we are digitizing the world and dangerous mistakes can happen if we digitize something to conserve it and we do it at too low a resolution, it's really not very useful. And DFS, in the early days of slide libraries, sadly with large format slides, example in Berkeley, they were digitized in the early days, sun workstations were the norm, sort of pre personal desktop computers. Images large format slides were digitized at the rather odd screen resolution of the sun device and many of them were disposed of.
And today, of course those on a high resolution screen, those images just feel a little corner. So enormous amounts of data were lost. So if we're digitizing for access, is very different than conservation.
For whom are we digitizing? What's the audience? What are we digitizing? Are we digitizing the object? Are we digitizing its surface only? Are we digitizing the color, the textural data? How are we digitizing it? With what device? If we don't record and keep track of exactly what was used, that usefulness of that data later on can be questioned. What moment in history of the device and of the object. And is our effort to digitize the past sustainable? Is it, will the data last as long as the original artifact. In many cases that isn't true.
I'll get into some examples. And then finally, I wanna talk a little bit about who owns the past and what rights do we have when we digitize. So starting with those, why, why capture and for whom? What is the audience? This was a project in Cairo. In the main square Ramses the second, you can see him standing, had been placed. He'd been found many years earlier, 3200 years old, 83 tons, 11 meters tall. He was found on his side outside Cairo in the desert and moved and reinforced with iron rebar and cemented in place.
At the time, of course, that was state of the art, but you know, many mistakes were made cementing him in place, putting rebar in of course did rust and cracking. Being placed in a main square led to problems of pollution and vibration and degradation. So eventually it was decided that the statue needed to be moved and a digitization project was launched to capture the data of the monument, to build essentially a scaffolding and a rig to transport Ramses out to the new grand Egyptian museum where he rests out by the pyramids today. And so this effort was successful and Ramses was moved in a very elaborately designed cradle, but in fact, a lot of the specifications of the digitization were off and we were given sort of wrong information. the digitization was really for engineering, but a lot of the specifications that were asked for were not correct. So perhaps we didn't use the right devices or we should have done more, but often why we're digitizing and for what purpose gets lost in translation and we could do more.
What are we capturing? What are we recording? What are we digitizing? The data, this is the one of the temples at Bagan, in what was known today as Myanmar, but often known as Burma. Vast site on the upper Irrawaddy river, there are over 3000 temples that scattered the plane. Many, many were destroyed or damaged in the seventies in a major earthquake. And there's been extensive reconstruction and renovation and creative enhancement of the temples.
Well-meaning, but often in many of the cases, the stupa, the tops collapsed or fell off. They'd been poorly restored prior to the seventies with cement, which was much harder than the line mortar that they were originally constructed with and that caused the earthquake damage to be amplified and greater cracking. Some of the tops came down. In the restoration efforts, existing monuments were copied and their turrets placed on, replicas of their tourists placed on others where, you know, they didn't fit.
So what we ended up with, is a bit of a hodgepodge. And so our digital data, what we, this is a one of many virtual representations. This one is out in a Sketchfab and I think later on, you'll hear about Sketchfab and some of the possibilities there, but these libraries of data that people are building are only as good as the source.
So if we don't sort of know the provenance and the history of what we are digitizing, the data itself is then often questionable. And then moving on, the where and the when. This is an example from UNESCO of data from world heritage sites across the globe, all positioned in Google earth. If we zoom in though on some of the data, this is the site of Chavín de Huántar in Peru.
Chavínis in the high mountains of Peru. It's in a valley and UNESCO placed the data, the data of the site and sort of imagery and Google earth, but quickly discovered that there was another site of Chavín de Huántar and people were placing data and photographs and other things that they were contributing to a site which actually is there on the left, on the top of a mountain, not in a valley, not at Chavín, several miles away across the ridge. And users, well-meaning users, were, as in this case, you know, saying that they had taken pictures and were contributing data, but the metadata, the data about where Chavín is, here, the user says that they've GPS located the site, but it's in fact six kilometers from the real site.
And so a huge problem in the digital world and digitization is provenance. How do we ensure that the providence of an object is correct? How do we slowly, folks started moving their data once UNESCO had placed some coordinates in Google earth to the right place, but how much of the data is made up? How much is measured? How much has accurate? We know the data was right. We flew in Marcia Forte an archeologist and did some measurements on site. Second example.
So, which moment in time, when we digitize something, how can we be sure it's from the period we want, be it a painting, a monument, a book, there often are a different additions. With the book it's more obvious, but with monuments, one would think it's quite obvious, but here, for example, in St. Marks in Venice, I don't know how many know the story, but as you can see in these photos which were widely circulated in 1902, when St. Marks as we know it collapsed. Over the course of several days, cracks appeared and then the whole tower came down, but postcards and other prints were made and showing the collapse, the one on the left we know is a fake, the one on the right is probably a fake, but the actual rebel, of course, wasn't a fake. The monument did come down, so if we make a digital replica, a digital recording, a 3D scan, a photogrammetric capture using some of the advanced techniques Carla and Mark will talk about, regardless of how we record it or digitize it, we need to make sure that what we're digitizing is in fact what we want to be digitizing. So what moment in time, here's a, for the art historians, Canaletto, made many images in the 1720s, but which one do we digitize, or which one do we build a model of.
The 1486 St. Mark's in Venice, the 1500 version, the 1745 version, or the current version, which of course is a rebuild after the 1902 collapse. And then once we digitize something, how do we know the data is of that object? We can use locational data if we have it, but here are two other St Marks which are virtually identical to today's version except in scale, but neither are in Venice, Italy. One is in China, one is in Las Vegas. Or these variant replica's built at cities around the world.
One of our challenges now is people are digitizing and uploading to shared libraries and databases all of these images from around, all of these images from their projects around the world, but without provenance, without metadata and some way of sort of tracking, users are increasingly becoming confused and using the wrong data sets. Finally, well, not finally, but in addition, is our effort to document and digitize and preserve the past sustainable. Dollars are scarce in the cultural world and it's important that we do things well and right and yet often efforts are duplicated.
This is one of the very first examples of 3D scanning. The earliest days, I took a team into, I took a team in to try and digitize at Angkor and here our scanner we thought was the first one there and the first effort, but upon arrival at the site, we did learn that from the DHL carrier, of all things, that already other teams had brought the same early prototype devices in to digitize the same thing. So we weren't sharing and coordinating our efforts around the world even in the early days of 3D scanning. I knew where all the devices and I knew who had all the devices in the world and yet we were duplicating efforts without knowing it. Dollars were very scarce, the cost of taking equipment in very high. So we need to better coordinate our efforts, but also make sure that they're sustainable.
Ben Davis has written and talked about, you know, the scarce dollars in the cultural world and is it appropriate that we're using the money for digitization when perhaps we should just be preserving the objects or the paintings or the documents themselves? And why, because often our digital records have a much shorter lifespan than the monuments themselves. So if one thinks about it, stone and monuments, thousands of years of life. Span, paper and wood and perhaps microfilm, silver halide microfilm, and they say several hundred years of life, assuming it's acid free paper in that case there. But as we get into the digital world, the magnetic and optical storage devices are lasting only tens of years. A real challenge.
If you go one step lower, even the encoding, the format, the way you're saving your data or your files, your databases, your imagery, is changing much, much more rapidly. Anyone in the audience who's tried to open a Microsoft word file that's more than a decade or so old, will notice that there are big problems. Microsoft doesn't support its early formats.
You have to use a third party tools to recover your data or WordPerfect for that matter. So the early days of the computer revolution are in a way lost, but it's a real problem in cultural heritage. In all these digitization efforts, there've been many sort of efforts in the museum and the library and archive worlds early on to digitize things and some of those efforts had to be repeated. In other cases, data has been, you know, put aside and the systems that it was recorded with have broken down or become obsolete, no longer accessible. One of the very first sort of virtual reality experiences that the Getty are using early pioneer laser disc to display a Greek vase, became so hard to maintain.
The Getty bought every remaining pioneer player in the world to try and keep the exhibit running. So next, authenticity. And in addition to sustainability of the digital, is what we are doing authentic? Is it real? And how do you know? We saw this in the example of St Marks. Here, you know, the data that was just on the screen, a 3D point cloud of the Sphinx, one could say is highly accurate down to millimeters, it's real data, but of course it wasn't the Sphinx in Egypt.
As you probably noticed, the nose was intact. It's 3D data of the Sphinx in Las Vegas. So is that worth saving or preserving? I guess it depends who you ask, but knowing what our data is and where it is from is really crucial.
And so digital reliability really has multiple errors and biases. And here's sort of a list from the object to the recording device, to environmental conditions when we digitize, to human error or bias or prejudice, to its provenance. And finally, even ownership, issues of copyright, be it works of art, be it books and our right to digitize are often still murky and need to be worked out in certain places in the world, here in Italy, for example, also in Greece, also in an extent in Australia, monuments and the images of them. Interestingly, the 2D not the 3D usually, because the laws predate 3D digitization, but they deal with photo photography. Photographs of monuments in Italy are property of the state not of the individual, but of course that's a law that's sort of almost impossible to enforce with tourism. So unless you publish yourself a photograph of the Colosseum and make a lot of money, I don't think the law is going to come after you, but our laws haven't kept pace with the digital world and need to be rethought.
So just to wrap up now as we come to a close, we're a half century into digitizing in cultural heritage. 1953 digitization as the word came into play, but we have an alphabet to those three formats, guidance is missing or often off or still being developed. In Washington, there's the Federal Agencies Digital Guidelines, but a lot of the guidance doesn't even mention 3D. There's the IIIF Initiative, there many, many projects going on and efforts all great, but more needs to be done. In essence, we need to sort of make a consortium of the museums, of the archives and of the monument and site holders to really think through the next generation of digitization. Everybody is sort of desperate, everybody's doing great works, but they're largely working on their own and figuring it out as they go and there are some defacto standards, but there's nothing like the Exif standard in photography for just where metadata goes with an image.
Sensors are everywhere today. They're embedded throughout our world in devices and cameras, on cars. More and more 3D data is being captured especially by the car efforts, building massive datasets of the world around us.
Trillions and trillions of data points. Mass digitization. Huge effort is going on, especially in the natural and the sciences. This is a project to try and digitize every living species in 3D and in cultural heritage, enormous work has been done. Google books, 16 years in, 40 million titles have been digitized. The Smithsonian.
I mean, my data isn't current, but as of a few years ago, 19 million objects of 155 million had been prioritized for digitization. 4.9 million digitized, a huge number, but of course, a fraction of what needs to be done. In archival material and library volumes, the Smithsonian has even further to go, but mass digitization, scanner beds and platforms are now available. Ways of digitizing you know, objects for under a dollar an item very rapidly with conveyor belts, but still given the number of objects, we're decades away from sort of digitizing everything. So what do we digitize and how, and how can we speed up the effort? So going back to the beginning, the Turner and the Constable paintings showing crowds and clouds, can we use them? I believe we can and we have some challenges of working together to figure out standards, but those billions of mobile devices with digitizers built into them, the global cloud computing network, AI, all of that processing power is there and cultural heritage can tap it.
We haven't, to the extent a lot of other fields have, the World Conservation Monitoring Center did an early project to crowdsource cultural or natural site, natural heritage data. There've been projects, many have probably heard of SETI@home and some of the other sort of crowdsourced projects to allow people to participate, either donating their machines or in this case donating your time to help database cameras, data from cameras in the field identifying what the animals are for scientists, naturalists. But we haven't really done that in the cultural world.
There have been a few projects. Marc Pollefeys at ETH Zurich demonstrated a very impressive system, "Rebuilding Rome in a Day." He used millions of crowd-sourced images he mined off the internet of the Coliseum, positioned them in space and from that data synthetically built up a point cloud, built a data set in the cloud using massive online compute power to process all of those clusters of data to basically build a photogrammetric, a 3D model of the Colosseum. That was a few years ago, so even more is possible, but all of this, using a crowd and a cloud, I think is only possible if we learn to share and to open up our heritage.
This has some challenges with digital ownership and rights with institutions and fear of releasing data, but in the old world, the pre-digital era, we worried a lot about copyright. In academia, everyone puts a copyright on their paper, on your publication, on your data. We need to rethink and move towards sort of the creative commons model or what usually called copy left. The idea that instead of locking up our data, we need to unlock it and share it.
And if we put our data out there for everyone to share and build off of as some of the projects like Sketchfab and others that you'll see have done, people can contribute and people can build on data and that's in fact, sort of the only way to get around that issue of digital obsolescence, of the data expiring before the monument. If it's free and publicly available and it's useful, somebody will probably grab that data and update it for the latest digital device and keep it alive and active. But if we still live in the world of keeping all our data locked and a copy-written so no one can can use it, the chances are it won't be updated and its lifespan will be limited. So we need metadata. We need standards or guidelines or ways to mark our data, but we also need to share it.
We need to utilize the crowd, we need to utilize the cloud and then there is some hope for a disappearing world and it'll take all of us. Thank you.
2021-02-03 23:08
