2021 Wild Ocean Science: Technology
Welcome everyone and thanks for joining us for Wild Ocean Science 2021 today we will catch everyone up to speed on new technology that the Wild Dolphin Project has been implementing and what the future holds for the project but first let's go back to the beginning to remind everyone how it all started Dr. Denise Herzing founded the Wild Dolphin Project in 1985 and her inspiration came from some very well-known women scientists when i was in graduate school Jane Goodall was out studying chimpanzees Diane Fosse was looking at gorillas and Cynthia Moss was looking at elephants this was my inspiration for finding a place in the wild to study dolphins long term what these three researchers really had in common is they found a place in the wild they planted themselves there and committed to at least a couple of decades of observations to try and understand an animal society and that's really what I wanted to do with dolphins in the wild I wanted to start the wild dolphin project because I was really curious about what was going on in the mind of non-human animals and at the time Atlantic spotted dolphins were fairly unstudied and they are intelligent and complex social mammals i chose the Bahamas as my study site because I wanted to study these animals underwater. The Bahamas has water accessible dolphins and this species has spots which helps us determine their age classes. Dr. Herzing has been studying this specific community of Atlantic spotted dolphins for over three decades in this video we will cover some of what has been learned over the years for more details about the science and discoveries made by the Wild Dolphin Project please feel free to check out our scientific publications online or schedule a talk with us to learn more.
Over the years Dr. Herzing has learned a lot about the life history of this species she discovered that Atlantic spotted dolphins are born without spots and gain these spots with age. As a result she was able to split them up into four age classes. Two tones are the calves with no spots, speckled are the juveniles with a sprinkling of black spots on their bellies, Mottleds are young adults and this is when we start seeing white spots form on their dorsal side, the old adults are the Fused age class they have spots that start to coalesce and their bodies are black and white all over. Still focusing on the life history data Dr. Herzing also found out that the females are pregnant for about a year and through non-invasive genetics we have discovered that the older males are the ones actually stirring the offspring.
Dr. Herzing and her graduate students have also explored a lot about the ecology of the Atlantic spotted dolphin we have learned that they prefer to feed on fish such as flounder and razor fish but more importantly we've discovered that the spotted dolphins will actually go off the deep ledge at night to feed on flying fish and squid something the resident bottlenose dolphins have never been observed doing. Dr. Herzing and the Wild Dolphin Project have also learned a lot about the social behavior of this species she has found that females will often associate based on their reproductive stage for example groups of moms with calves will often stick together even if these females never associated with one another prior to having calves. Lastly one of Dr. Herzing's main focuses of study with this species has been acoustics and vocalizations. Here she'll explain in some detail how we record the sounds the dolphins
make and the vocalizations she has learned my main goal was to study dolphin communication To do this I had to lay down a framework to understand their own society this included knowing the individuals their relationships with each other and how they communicated with each other to correlate sound and behavior we've always used an underwater video with a hydrophone input with many years of data collected while recording sounds the hydrophone and simultaneously recording video she has been able to make correlations between vocalizations and behaviors dolphins make three types of vocalizations dolphins make frequency modulated whistles and in some cases they have a specific whistle called a signature whistle which is basically a name Dolphins also make sonar clicks for echolocation this is the sound they use primarily for navigation and hunting Dolphins also make a sound called a burst pulse these are the most unstudied sounds because they are hard to categorize they are close proximity sounds and often found during aggression or play fighting Now that we have caught everyone up to speed with a brief recap of who we are here at the wild dolphin project let's look into the future. And the future is now. Greetings dolphin nation I'm Miles O'Brien today in Santa Monica California we are live from all over the world from Florida from Singapore from Georgia uh the list goes on to bring you all together for our common love for wild dolphins and most importantly the woman of the hour the founder of the Wild Dolphin Project Denise Herzing. Denise, it's good to be sort of together this is our third Wild Ocean Science event the first time we've tried it virtually each year we've done it we've always raised a little more money for the the good work that you and your team do and that's our goal here today we're going to learn about dolphins we're going to learn about how you learn about dolphins and we're going to spend a few moments supporting the cause one way or another but we want you to participate help the cause and most important i want you to think about this as a conversation we have some people here who know an awful lot about dolphins unsurpassed knowledge uh frankly and Denise just a couple thoughts on you know last year of course right about this time couldn't even contemplate a virtual event we're right in the thick of it as far as the pandemic goes this year things are looking a little more up and the good news is I hear your season is already sold out every voyage of the Stenella is fully subscribed so that's you have an exciting season ahead tell us a little bit about what you have planned yeah thanks Miles and it's great to have you back even though it's virtually but this is our life right now um yeah we have a full season we are gonna take our many new toys out that you're gonna hear about and yeah last year we got cut short completely so yeah we're uh encouraging you to ask questions after every section of technology and we'd love to hear from you yeah I think one of the questions on my mind is do you think I'm Mottled or Fused at this stage of my life I don't know I'm feeling kind of fused some days anyway we'll get to all those questions as the evening day whatever it is wherever you may be moves on we want to begin with um you know Denise has been spending now since 1985 following basically one pod of dolphins in the Bahamas and you know people always say well how did she find them and it actually a couple years ago got difficult because we think climate change caused a crash in the food supply and Denise went out and they were gone right and it was actually a real challenge trying to figure out where they went and one of the things they do is come up with new ways to sort of keep their their their antenna out for where the pod may be so why don't we without further ado we're gonna roll a video with Cassie Volker Rusche who is the able research assistant as part of the wild dolphin project is going to tell you about a couple of the new pieces of technology they're using to try to keep track of where the dolphins may be climate change is an issue for many species and even the dolphins over in the Bahamas have been unable to escape the impacts in 2013 50 percent of our resident group from little Bahama bank moved over 100 miles south down to Bimini and after reviewing oceanographic data we decided this move was likely due to a food crash in the fish population on little Bahama bank as you can see from these graphs up on little Bahama bank we saw a clear drop in chlorophyll where the dolphins normally fed on great Bahama bank although there was some variation there was no significant change in the chlorophyll levels since chlorophyll is a proxy for plankton production our conclusion is that there was a major food crash on little Bahama bank the lack of food forced part of this dolphin population to find another area in which to feed leading them to the shallow sandbanks surrounding the island of Bimini as a result of this food crash and the displacement of 50 of the dolphin population the wild dolphin project has had to resort to new technology to help locate some of our more elusive dolphins up on little Bahama bank One of the new technologies we are now implementing here at the Wild Dolphin Project is passive acoustic monitoring. Recently we deployed some passive acoustic
monitoring equipment to gather data when we are not physically in these locations the hope is that this technology will help us find some of our more elusive individuals who remained on little Bahama bank after the displacement in 2013. This particular device that we are using is named an E.A.R. it's basically a lot of batteries and a computer that is programmed to go on and off at certain intervals and collect acoustic data over a period of a couple months. While securing the E.A.R. to the sea floor we had some curious dolphins come and check out what we were doing as you can see we picked the perfect spot to passively listen to any dolphins swimming by if you look on the sea floor behind the dolphins you can see the ear that remained on the bottom until we retrieved it a few months later another way that the Wild Dolphin Project is trying to find some of its missing dolphins is by enlisting the help of citizens through acquiring and id'ing citizen photos all around the world researchers today are depending on citizens to upload their photos to sites such as fluke books.
If you go to our project's website you will find a link for fluke books under the our research tab this will take you to a page on our website describing the process of how fluke books works it takes the photos you submit of dolphins you saw in the Bahamas and using their spot patterns it matches them to the dolphins in our catalog basically it's facial recognition but using spots from this page you can get to the fluke books homepage where you can fill out a form and submit your photos if the computer finds a match it will notify you and let you know who you saw. We hope that the next time you're in the Bahamas and you come across any dolphins that you remember to upload your photos here so that we can let you know which dolphins you encountered. And joining us now along with Denise is Cassie Volker Rusche. Cassie, it's a big ocean how how big of an area can a device like that cover and ideally how many would you like to have so you can you know find out where these dolphins might be. So ideally we would like to have quite a few more we have a couple different places in our research area that we would like to put these down on the bottom always the more the better especially because we can't be out there 24 7. so any device that can help us collect data every day all day long would be absolutely perfect so you you have to leave them there and obviously there's no real-time telemetry underwater and so it records a big long two-month recording uh that's a little bit of a data retrieval nightmare isn't it how do you go about listening to two months worth of um noises of boats and everything else in addition to dolphin noises potentially yeah so we actually send the data off-site Denise I'm drawing a blank on who we send it to i know it's we're teamed up with Dr. Mark Lammers but I'm drawing a blank on who
the site is or this uh people are yeah his group is Oceanwide Science yeah so we actually hired them to do the data analysis primarily because we needed it quick because we wanted to find our dolphins from the data it really you know it's just so stunning that they would just kind of you know disappear on you after all those years. Catherine has a question really both of you with the food supply going down are there efforts to help sustain the food supply to keep the dolphins around rather than having to advance technologies to find them this brings up a very important point about um the wild dolphin project philosophy is you're not there to really disturb nature right you're there to observe it but um is there anything we can you know this is obviously a concern is there anything we can or should do to help these dolphins I mean one way is always to be to support sustainable fisheries and just when you go out to restaurants try and do your research you know try to make sure that they are accurately cultivating the fish that they gather that's always a big thing and just doing your part and researching as best as you can when you go out on some type of tourist boat when you go if they say that they're swimming with certain animals just always make sure that they have the proper permits is a big thing and make sure that they are being sustainable environmentally cautious when they're out there as well so it can be anything from you just as simple as maybe not ordering and eating a lot of fish or cutting down on the types of fish that you are eating especially ones that maybe are declining in their populations out in the wild so just kind of doing your part and being educated on the topic is very important is also a very important step Denise is there any evidence you know dolphins are smart and they're very mobile uh they they know how to find uh their next lunch pretty well but is there any evidence that this crash in the food and the move that they made has had any adverse impact on the pond well i mean just the fact that they had to move 100 miles to find fish and now they're interacting with a local group that causes challenges so absolutely and we don't know the long-term report reproduction uh impact for example but that's why we study long-term do you uh roughly how many dolphins are still missing in action so to speak well uh there were about 52 that moved south now more are trickling down from up north where we previously were so yeah probably probably another 50 are kind of missing in action I would say this is a reminder you know when we think of climate change we focus on specific things like sea level rise and that sort of thing but there's there's so many consequences to our ecosystem that it's almost impossible to think of them all all the unintended consequences of the climate change but here's one thing uh a slight change in temperature in the water or maybe the salinity do we know why that crash occurred but one way or another this has a huge ripple effect on species in the oceans doesn't it yeah it does we what we actually saw was from NOAA information that there was chlorophyll which is at the surface of the water actually decreased up on little bahama bank and that's a proxy for phytoplankton which is kind of the base of the food chain so if that crashes all the little fish that eat that and then the bigger fish that eat the little fish all of that just cascades and declines and also where the chlorophyll decreased on little Bahama bank we did not see that down on great Bahama bank which is where Bimini is situated and that's where they move to so we need to start tracking the chlorophyll levels down by Bimini and hopefully they stay well enough so that the resident population and the new population that went down can continue to be continue to feed on those reefs. So Karen would like to know how the dolphins made this change of location for example would some have scouted and then tried to communicate back to the others that's that's an interesting question I'm sure that's one that maybe we can't answer what do you think Denise do you want to weigh in on that one or is Cassie either one of you i i can weigh in Denise if you want me to so for us we are very non-invasive so we do not tag the dolphins again we don't feed the dolphins we don't touch the dolphins we try to be a fly on the wall for them if there's a synonym for that in the water we try to just observe as natural their behaviors as we can so for us unfortunately we don't know if any scouts were sent out to see but it's possible we do get movement all across the Bahamas all the different islands because every once in a while we'll come for one year and there's a couple uh fused individuals so those are the older individuals that we have not seen before so where were these dolphins so we do get that immigration into our population so we do think there's definitely some traveling going on between the islands. So Mary would like to know how the dolphins fared during hurricane dorian that was a particularly bad one dolphins generally speaking uh they know how to ride out a storm don't they yeah so unfortunately we're not over there as well when the storms are happening so well maybe fortunately yeah that's probably more fortunate that we're not there but um again we don't tag them so we don't have locations of do they go to shallow water or do they actually go to deeper water which might be safer for them so we don't know where they go during those storms but when we do go back to see them we see that they're doing pretty well when after dorian happened that we actually had a relief effort that we took over to dorian for the people in the Bahamas we gave some supplies over there and we had about a day to search for some dolphins and we were able to find four of them in our population and one was actually a calf so that kind of said to us that they seem to be able to fight off that storm and do pretty well in 2004 we did have uh some big hurricanes that wiped out a pretty decent amount of the population and for a couple of years they were still recovering and they weren't doing their normal behaviors so that those storms can definitely impact their population and their cohesion and all that kind of stuff as well interesting Shannon wants to know if you guys have seen any evidence of increased immunosuppression related to climate change now that that that's a tricky one to get at the way you do your work because you probably have to draw some blood or something and you don't do that right we don't do that um the only genetic uh or body sample that we get is actually their fecal samples which we can actually tell paternity based on that i don't know if there's some way that we can decipher the immunosuppressants and that kind of thing that's something we would have to look into but it's definitely a good idea that it's something that we can possibly look into in the future and maybe with our genetic samples we can get even more information but yeah because we're not invasive we don't like to pull them out of the water and get any kind of blood samples because we want to keep that mutual respect and allow them to keep us watching them in the water so yeah as of right now we don't have a way to say to see that but again with that genetic fecal sample we might be able to get some information all right i have a suggestion for you Shannon if you really want to find out the answer on this why don't you fund a little bit of research Cassie's I think we've spurred an idea here Cassie's kind of improved and uh you know science costs money so we're going to invite you to uh pony up if you like and and we'll make sure you're maybe we can make you a co-author of the paper or something at least in the acknowledgements. Jeff is curious if we know what caused the decline in the concentration of chlorophyll on little Bahama bank is there a root cause is it temperature acidification or other issues I think it could possibly we don't know exactly for sure but Denise if I remember correctly it has something to do with the wind change that we had in that area over the summers yeah that's still a real open question we honestly don't know yeah the wind direction has changed a lot so that could decrease nutrients potentially hard to say again we don't you know we don't even know which knobs we're turning anymore when it comes to climate change and the consequences are difficult to prevent. All right well um let's
let's move on everybody's staying so if you have any um questions further that come to mind for Cassie Denise they're not going anywhere uh but uh i gotta tell you you know I've had the good fortune thanks to my cousin Ruth Petzold board member on the Wild Dolphin Project to have had a cruise on Stenella with some of these wonderful people and one of the highlights of the mission was being with the person you're about to meet drew is to say he has an infectious smile and demeanor is an understatement and on top of that I've never seen a guy and I'm in the tv business I've never seen a guy who knows gopros better frankly and his ability to capture the dolphins in their environment is extraordinary and he's here to tell you a little bit about what the next piece of technology is which is 360. So let's watch Drew and then we'll bring him into the conversation so take it away Drew and tell us about 360 degree viewing of dolphins. My name is Drew Mayer and this is my video editing workstation where I sometimes edit videos for Dr. Herzing and our researchers so that they can analyze the behaviors and vocalizations
after we come out of the water imagine for a moment that you yourself are a dolphin researcher what you want to do is concentrate on behaviors so you would point your camera here and record what looks like the activity but then look what happens where do you point your camera now what if you didn't have to point your camera at all Now you don't have to. This is a 360 camera it has two lenses and it's able to record in a complete circle 360 degrees you take this and you put it in an underwater housing And you attach it to what I call drift cam 360. I designed this for Dr. Herzing so that it has adjustable ballast so that at about 20 feet down it hangs in the water column and it records all the behavior of all the dolphins within visual range you can adjust the ballast so that it can hang at 15 feet or hang it 30 feet releasing the drift cam 360 below the surface gives the researcher a near limitless range of perspective radiating from the center point of the camera meaning that it allows for viewpoints into the dolphins lives that we could be hard-pressed to get any other way we can track behaviors and interactions that we might otherwise have missed here's an example of how that works yeah so I mean when i was in there there was a bunch of aggression going on so squawking head-to-head so this is like this is replaying that big group of males yeah so I was in the water following them so they're kind of just traveling this direction now but you hear that buzzing which isn't normally heard in aggression aggression is like squawks and all that kind of stuff so that buzzing is courtship so that's i mean we would just not we would have heard it but then we would have been like why is that going on in aggression so now with this 360 we can just go back and zoom in on these individuals and what they're doing I wonder what else we missed yeah well we didn't play a lot no but we didn't because we can go back and look so let's we'll go back the footage from drift cam 360 proved to be so valuable for research that i added the 360 camera to an apparatus i designed which we call manta here's Cassie and I discussing the benefits of it now this was great yes but this is really interesting This is a 360 camera and this is it's underwater housing and this is a happy researcher once a very happy researcher now Cassie tell us what this does for you so now we have 360 view while we're in the water with the dolphins so again I'm pointing at the main action with the gopro and the high frequency but now if there's dolphins playing behind me or doing aggression behind me then I have the 360 which now captures that footage dolphin research has come a long way from underwater still film cameras and underwater riding slates and our understanding of the dynamics of these amazing dolphin lives and their environment is advancing every step it's exciting to see the dynamics of the whole pod and then to go back and to look at groups of dolphins and alliances even individual dolphins and track them throughout the environment that's what we have with 360 cameras it's very exciting and we're really looking forward to uncovering many new things that we weren't able to see before because we had a limited vision now we capture everything now here's a little treat this is the first time that the dolphins saw drift cam 360. watch what happens Hopefully I've made you as curious about 360 cameras as these dolphins seem to be text us your questions now and we'll do our best to answer them. That was fantastic. Joining us now live on the Wild Ocean Science part three edition is the legendary Drew Mayer along with Cassie and Drew the problem with you is you don't have enough fun in what you do clearly you got to pick up the energy level will ya a little bit and by the way you could probably do late night you know call-in ads you know for products it's really good it's very good stuff makes me want to buy one um so i you know the question which immediately came to mind as i watched this is if you if you attached this video and ran this video into uh an oculus rift headset or something you could have a virtual dolphin experience have you tried that and is that something that people could have access to do somehow actually my son and I have one of those oculus headsets and it's wonderful it's if we can only have everyone have one you it's mind blowing maybe Cassie you could weigh in on the value that they give you all so first of all how much are they well these cameras are more expensive than most of the prosumer cameras they the cameras themselves cost about five hundred dollars the housings cost like a hundred then you need extra batteries then you need sd cards then you need a workstation like you saw that I have now that's that's a bit of a workstation I have four 4k monitors it's pretty heavy duty one of the things we need and Denise you and I were talking about this just last week is here in the lab where we are right now we need a video workstation here we need to have a large screen and maybe a couple of large screens a powerful computer because we're dealing with some pretty heavy-duty data streams from two cameras inside one camera is stitching there's all kind of things zooming in zooming out but anyway we need power and um that costs money and I would say to properly outfit us would probably cost about fifteen thousand dollars but being the wild dolphin project we can make do with less and I would say probably with about eight thousand dollars or so we can be up and running and be able to handle some things here's the problem miles is that i'm not always here uh i do some traveling like you do and um I'm actually this summer gonna be away for a lot of the time I'll be on two trips I think with matthias and and thad I think might be on one of them um and since I'm not here all the time I can't leave uh my equipment around and those two cameras are mine personal equipment too there's uh here's something that's really interesting and Denise and I have been talking with the company that makes these 360 cameras now they make and you may have seen this Miles because you're in broadcast they make a camera that is incredible and the price isn't that much it's about five thousand dollars and then the housing which is incredible also is about another eight thousand dollars this camera can do 3d 360 in incredible resolutions we can add hydrophones to it um it's just amazing you can tell I get excited about this stuff a little bit well you might you might have touched on an answer to Alex's question he wants to know if there'll be advancements made to obtain audio better audio on the 360 camera it sounds like integrating that with a hydrophone and we're going to hear a little bit more about uh you know triangulating audio with Matthias in a moment but you know this could have a tremendous value for research if you had the ability to not only record sounds but find out where the sounds were coming from and in a 360 environment sort of that you're not missing anything right yes and Alex that's an excellent question with this super high end unit that we'd love to get please get it for us whoever's listening you can hook up a hydrophone to it it's it's a good rig but beyond that we do use uh high frequency hydrophones that we have and we'll show that hopefully in a little bit um if we have time um and just recently Dr.
Herzing could you talk about the um the ultra HD high frequency unit that we just recently got oh well we have a new yeah it's a tascam unit um yeah I wouldn't say it's ultra high frequency but it's more high frequency that we synchronize with our video and yeah you know the ideal unit is Matthias's we're going to hear about unit with 360 really that's the ultimate there's the holy grail there and the thing you started out I mean what were you you were shooting tape initially right back oh yeah old vhs video big camera yeah but you know that's science right absolutely yeah I'd say I'd say the resolution has increased a little bit should we say. Larry actually wants to know, Drew, Larry would like to know the make and model of the camera that you're referring to it's made by insta360 and it is the pro the the they have two the pro and the pro two now this this is the Larry that I think it is he makes some incredible dome footage and that he has around the world that is just incredible if you get a chance to go to one of his shows where he has a dome set up and he he plays the 360 or the 180 video you got to go but but Larry if you want to give me a call and I'd be happy to to give you the you know the contacts so all right what's Larry's last name for those of us who aren't familiar so we can find him if it's Larry Curtis that's the guy he has a long-term history of cinematography in underwater and with especially cetaceans he's an incredible guy look him up so liam thinks the oculus and 360 video would be awesome for school presentations it would could really immerse the kids in the research you can imagine this really exciting kids you know for those who can't afford or can't get to the Stenella this would be a pretty cool substitute wouldn't it it would it'd be fantastic if we could do that and with the advancements of technology the prices keep coming down uh the the user experience it really is maybe sometime in the future we can I don't know we can somehow show it but the user experience is incredible um because you're right in the center of the action that's what's that 360 cameras put you right in the center of the action everywhere you look is the dolphins because quite frankly Cassie when when we get in the water sometimes we're swarmed yeah and we and it's confusing we don't even know what to look at um because there are dolphins swirling all around us and they're everywhere so what this does and especially you know I'll bring the oculus in so you can look at it yeah i'd love to it allows Cassie to be right in the middle of the action now she can see everything and she can now isolate different behaviors with different dolphins the things we're going to learn with this is really outstanding and like Denise said pair something like this with Matthias's... and I have to yeah Matthias's A.S.P.O.D. it it really brings us to a whole new level and I have to give props to Matthias he's the one that actually introduced me to 360 cameras we went mountain biking and he's he's talking to me about 360 camera and he whips it out and he puts it on the handlebar of the bike that I'm loaning him and we go for a ride and afterwards it blew my mind and so right then I bought the latest ones they had and yeah I'm looking forward to collaborating with Matthias more on it Matthias I'm sure we will be happy to throw a 360 on your A.S.P.O.D. if you don't have one on there already so I just want to add one thing back to Liam who chimed in in the chat coming from I was born in Ohio so landlocked state so if we could get to students that are in landlocked states and be able to show them the marine world and some kind of virtual the 360 for the dolphins I think that would just be such an awesome experience and really connect with the kids who can't visit the ocean or have never seen it before so i just think that's a really cool tool to bring up and and it's sure a lot more humane than capturing dolphins and putting them in a tank to show kids right this is the see them where they belong all right all right excellent we we have a few more questions we're going to save them to the end because we want to move it along here and talk a little bit you know that experience of being swarmed by dolphins is something I've had the good fortune to live through and it's it's hard to describe how thrilling it is but one of the things you learn very quickly when you're in the water with these animals is the acoustics are such that you really don't know where the noise is coming from and that's it's obviously very important because Denise and her team are trying to connect behaviors with vocalizations and maybe in so doing kind of crack the code and figure out how they communicate which would be a huge breakthrough of course but in order to do that you need to sort of figure out where that where is that noise coming from and our next speaker has spent a ton of time doing just that coming up with the the kind of technology that makes that possible Matthias Hoffman-Kunt is a senior research fellow at the National University of Singapore so it's about four in the morning where he is right now I assume he's drinking a lot of strong coffee to stay with us we do appreciate it he'll be with us live in just a moment but first let's let's see exactly what his technology is all about hi my name is matthias monkhunt and i work for the national university of singapore at the acoustic research laboratory and i've been collaborating with Denise and the Wild Dolphin Project for quite some years now and today I'd like to talk a little bit about what equipment we're using and how we're using it and what's new and interesting about that if you generally look at sound underwater it is the main important method of communication for animals that live in the ocean light as you all know doesn't travel very far maybe maximum 50 meters if you have really really good visibility but most of the time it's much less work sound can travel for thousands of kilometers if the frequencies are low so a lot of animals use sound now for us as observers of marine mammals in the wild this is a bit tricky because our head and our hearing is not made for underwater hearing it's made for in-ear hearing the speed of sound underwater is four and a half times faster than it is in air so for us to distinguish where sound is coming from is almost impossible people have of course recorded over time a lot of video behavior underwater and even with a microphone or a single hydrophone in there but the problem really is that with one hydrophone you can't distinguish where the sun's coming from and then people have also put in arrays of hydrophones so several four or five headphones or a linear array but the problem with that is that it's not synchronized with video so what we have done with this device in the back here which i'll explain in a moment is we have combined three high frequency hydrophones or up to four and a very wide angle video camera which you see up front and it's all integrated into one system which you could then use to operate underwater in there and let me just show you the device and all okay what we have here is we've got a regular underwater housing from an old sony video camera with a dome lens up front and a white very wide angle 180 degree camera and then we've got the rest of the equipment all sealed in there and attached to this we have three hydrophones so one on the left one on the right and one on the bottom and what that allows us is this is all fed into the system and what it allows us is to synchronize that with the video exactly so for each frame if we're hearing sounds we know exactly where it came from we can calculate because with two hydrophones I can calculate an angle with three item hydrophones I'm getting two angles which then gives me a direction of whereas the sounds coming from and I can then overlay that in post-processing onto the video so that for the first time really allows us to identify which animal is vocalizing and what and so for the study of behavior it's really important that we can identify who is vocalizing at what time who is responding to whom so we get this back and forth of communication between the animals so that's the system in short and obviously this is all fitted into an underwater housing and inside is also a a full full-size style acquisition cart a computer amplifiers and everything else is necessary and you can take this whole system and swim with it underwater so when we analyze the recorded data and we put this together then we get the following processing window we have the main screen that shows us the video that was recorded on the camera on the right side we have a waterfall display that shows us the spectrogram of whatever sounds we have recorded in the bottom we have a time series uh so what will what this will do is it will process the computer will process each individual image each individual frame and if there was a sound detected it would put a dot on the location of that whistle let's watch for a second how that happens You see three dolphins and they're all three are echo locating coming towards us and it's detecting the clicks of these animals Again it detects the clicks of these animals we go three animals are coming and that's the end of that sequence Overall I think this is a great new tool that opens the door to a whole new way of investigating the social behavior and vocal communication of marine mammals and maybe someday in the future we'll be able to really understand what they're talking about and then we might be able to talk to them in their language on their terms wow that is so darn cool fantastic technology what a breakthrough this is congratulations on your efforts there I think this is you know potentially a really it's it's a huge breakthrough for the team to be able to you know see and correlate uh where do you see this going from here this this technology um well we were talking a few moments ago about how cool it would be to integrate that with a 360 image is there any technical reason that that couldn't happen or is that just a matter of just applying it no it obviously it can happen well part of that was simply the development of that camera the very first one that we had in the first 360 that we tested out on the bike jail with drew was it had okay resolution but if you start zooming into smaller behaviors of course then it gets a little bit more fuzzy so the problem with the 360 camera is that you need to have the resolution and so only a really really high end 360 can handle this then another problem is when you're zooming in the direction calculation of the of where you put the dots will change because you're not you know let's say your system is calibrated to exactly the whole view that you have on your camera right and now all of a sudden uh you're zooming into something so then then that pointing doesn't work anymore so you need to know how much you zoom I mean it's all doable with software in theory right but you need to that needs to be practiced and we haven't done that so normally the system is has a fixed lens right and we're not zooming in uh but of course we could you know first do the detection and then swim into the behavior and say ah okay now that one was talking oh and here this one responded so I think Mary we have found yet another avenue of research that you can fund if you like because Matthias needs to figure out how to make this work in 360. this can be done it's just a matter of time but it is extraordinary we've got we do have a question and this does occur to me especially when you consider 35 years of data that we're talking about. Shannon says holy cats that seems like so many terabytes of data data would be generated in a very short amount of time uh with the wolf acoustic data we capture i guess she studies rules it quickly becomes difficult to store and manage and we aren't even collecting video at the same time Matthias is the uh the post as we would call it the post-production nightmare uh as as as big a deal as it sounds there uh yeah you end up with a lot of data and downloading and processing and you know just writing the code to you know take each video frame look for the corresponding audio and then put it back together and save it and you're running this you know for lots lots of sections so it's it is quite time consuming and does take a lot of data storage and computing processing power so Denise before you had the uh technology that Matthias has helped you develop how did you how were you able to correlate behaviors and uh vocalizations was it just you know um a little bit of guesswork or how would you do it well you know we would just correlate group behaviors with group sounds that's still the problem right so this is technology I've been waiting for for 35 years really and like Matthias and I have talked we can start asking really simple biological questions like does a mother make a whistle and the calf joins her or does the calf make a whistle and the mother joins her so you can start refining your questions to you know avoid the mass data overwhelming problem really and that's what graduate students are for right ask a specific question that's part of it right yeah Dean would like to know Matthias how long it takes during post-processing to get the squares to line up with the video is that a big uh crunch for a big computer uh well at the beginning it was and now we're we had a lot of software overhang so now we're trying to compress this down so that you really just take the sections that you actually want to look at and say uh if we had a 10 minute video or 10 minute section that we're saying okay now we want to look what's happening that would probably take uh well maybe 10 hours to process somewhere around that so before it was a lot more now 10 minutes it depends on the size of the video so the higher the resolution of course uh it it becomes more data intense we haven't really done really really high resolution stuff yet but the acoustics always at least for the dolphins stay the same so we've we're sampling at uh 400 kilo samples per second per channel uh so we're we're getting in a lot of acoustic data on three or four hydrophones so where does the computing horsepower you need is that something you can use for your uh uh the university or how do you do it well actually processing normally when we let's say we're recording we're out in the boat we've just done some recordings then we download the data and then at night i just let it run and hopefully by the morning it spits out some nice results and we can then look at it sometimes we if it's short sections you can look at it right away for you know say seconds or so if you just want to check what you have but you you run it on the laptop um you know now processing powers on uh you know laptop computers have gone so so far that you can actually do this on a laptop but of course if you're doing it on the biggest system and you have a station for this then it goes faster actually you know it's it's really extraordinary you know moore's law is a wonderful thing isn't it gives us the ability to do things we would couldn't even contemplate it just even a few years ago with the ability to manage all this data I love the idea of applying that technology somehow to a 360 image that would be fantastic.
All right let's we've got a couple other here's another one from Alex he wants to know if there's any ability for live streaming from the research vessel when we're in the Bahamas it's funny to mention that Alex I when I went on my voyage I brought an in-marsat satellite device and it didn't go so well but technology is improving and I did also discover that a lot of the work they do is within cell tower reach in the Bahamas I discovered that because I got some crazy cell phone bills because it was just pinging the Bahamian cell towers all the time but I wonder Denise and and Matthias too if there's if you guys have thought a little more about perhaps engaging students or others in real time while you're on the Stenella at some location yeah I mean we've thought about it but yeah it's all about the technology right and you know what it's like we're looking for dolphins for five hours and then we find them so yeah I don't know it would take a lot to do it but you know you could do a semi-live streaming right maybe at night and to show what you did that would be probably more realistic absolutely yeah with the right satellite connection I think you you could do that but technology is getting better so maybe maybe if Elon Musk chips in and lets us use his Starlink connection and if Elon Musk is a dolphin lover and he's listening right now give us some free starlink time will you please that's what we that's what the Wild Dolphin Project needs right now all right let's let's move on we've been talking about vocalizations and behaviors correlating the two trying to figure out what you know if the names are of the animals the animals have distinct names and all of this is step by step getting us closer to you know really a big breakthrough in understanding how they communicate with each other clearly something's going on there a lot is going on there and we're only able to hear with our ears a certain range of it there's a lot of stuff we can't even hear Denise has had a wonderful she's had wonderful collaborations with a lot of people including Matthias but also with Dr. Thad Starner at the Georgia Institute of Technology he's a heck of an engineer and he's been working on a device a submersible computer which has the capability of serving as more or less a real-time transit so let's um let's watch Thad's video and then we'll bring him into the conversation so take it away then early on Denise and her team used a rudimentary underwater keyboard to interact with the dolphins after about four years we thought well we should probably wait till there's better technology because we're not going to go very far with this so in 2010 I met a group of computer scientists up at Georgia Tech Thad Starner's group and turns out he builds wearable computers and so it was like well I need a wearable underwater computer so he grabbed the job and put some of his students on it this is our C.H.A.T. box chat stands for Cetacean Hearing and Augmented Telemetry and basically what it is is a system of computers amplifiers inside this aluminum casing the computer is programmed with a number of artificially created whistles for different toys the dolphins might like to play with dolphins have a lot of natural toys sarcasm seagrass sea cucumbers so we've been trying to label as many of those natural toys as we can they use a scarf primarily because they like to drag things and they're very good at it that's what they do with sargassum and it's something they have to ask us for they can't go down to the local dolphin boutique and buy a scarf so it kind of becomes oh i need the human to get a scarf therefore maybe I'll be motivated to communicate that word So the way it works is we're in the water I can push a sound for example this is the whistle for scarf this headset just said scarf in english so I know that's the sound I played now if the dolphins decide to mimic this whistle they'll mimic it the computer will recognize it in pretty close to real time and I'll hear the word scarf in my headset dolphins when they greet each other use their signage whistles so we thought it'd be pretty cool to give ourselves a name So that's my name, Denise. Now we also have some of their signature whistles on the computer
that's Brat he's a little brat and he's one of the players in the system so we can greet him in his own name so we thought that would be a start to trying to communicate with the dolphins well the idea is to empower the dolphins to communicate back I wanted a tool where they could access us and ask us to do things or request things from us you have a couple researchers in the water we're both wearing these underwater computers and you're actually modeling the communication system for the dolphins really requires not only good technology but regular extended time with the same individual dolphins so that they get exposed to the system and start understanding the functionality of it it's one thing to mimic a whistle it's another thing to understand what the whistle can get you the team discovered that juvenile dolphins showed the most interest in the interaction this is an age where they're kind of away from mom they're not full adult responsible dolphins yet so they have a lot of play time and so we have about a four year window with individuals when they're in that age (inaudible background noise) research assistants from Georgia Tech joined the scientists at sea to fine-tune and troubleshoot the C.H.A.T. boxes that are built by the students at the University making new interface devices that are user friendly for marine biologists is kind of challenging from the beginning so all of our hardware is custom designed they use 3d printers much of the time we also have machine shops in Georgia Tech so they they're able to mill the aluminum housings and then laser cut the other plastic parts on a software side dolphins present a very interesting challenge because their range of vocalizations is so large in terms of frequency so you have to sample at a very high rate in terms of audio on the computer so it requires a very fast processing and efficient software on battery power with something that has no internet or external connectivity to the outside world so all of your processing is on board whereas you know typical voice recognition things like google now or siri or doing some processing on the phone or the platform and then sending it off to the internet to be analyzed on a much more powerful computer we have to do everything on the system all right and that was a clip by the way from dolphins breaking the code PBS' Changing Sea's used with their permission and we thank them for that as a matter of fact we thank PBS for being thank God for PBS and joining us now to talk a little bit more about his research is Thad Starner who is from Georgia Tech as we said and has worked with Denise for a long time and who's one of the lead researchers on Google Glass correct and as you can see right and Thad tell us a little bit you know when you really start thinking about the challenges of trying to get a computer that is powerful enough to do what you needed to do in real time and you know sort of provide a translation if you will and making it submersible and then on top of that have enough power to project out enough acoustic enough of an acoustic wave for the dolphin to even interpret it's a huge challenge isn't it it was it was monumental you know when Denise first told me about this problem I was like you know I think we can do that we'll take a look at and the first summer was horrible um we killed so much equipment from just trying to make it waterproof well enough to handle the sea water uh we found out that our hydrophones were not waterproof you know so we're trying to put an equivalent of a supercomputer in a box on the niece and not have it blow up while she's interacting with the dolphins and so far we've not had any major fires at least when she's been playing with them so that's good uh but it turns out the the really the hardest thing was physics when the Navy does this sort of work they have these very large speakers and very powerful batteries and and transducers and amplifiers that they do and most of the amplifiers are on deck so what we do is shrink everything down to a very small form factor and very powerful get enough power in the water to make that that sound go forward and and we've recently had a breakthrough in this this is something called "C.H.A.T. LITE" this is the the modern version of the C.H.A.T. system you saw and I don't think you'll be able
to hear this but this can do the same sorry thad that you've shrunk it down to that size well this is this doesn't have the recording or the recognition side this just has a playing size but inside is a custom-made amplifier we basically took the equivalent of your stereo amplifier amped it up around the pun quite a bit and mashed it with a speaker that can work well in water which is surprisingly hard to do and put in the small form factor and now this whole thing can actually sit on Denise's wrist um and it's something that she can take in on every time she's with the dolphins as opposed to you know requiring one of my phd students to be with her uh every trip she wants to do it so we're going from like a couple encounters a year with the dolphins to something where we hope the knees can just have this continuously there and expose them to the the whistles the names we want to assign things like the scarves or the pieces of rope or the seaweed and so we're very excited about that this is number five and we had our first successful test of this thing this month um and whereas you know the chat systems you saw on on stenella with nice in that video clip they each cost about uh the equivalent of a new car to make um this costs about a tenth of that in parts um it's still a lot of still a lot of labor but it is something where we can make a lot more of these and get them out there and have replacement ones for Denise and you know it also kind of has a sort of you know superhero aesthetic to it and you know since Denise is our superhero I think we should start giving her you know good bracelets for it right totally totally she is possessed with superpowers there's no question well let me ask you this though if you need it you know if the navy I'm sure the navy does this for bad reasons we don't like but if the navy idea is to put all the horsepower on the surface and connect it with the cable why not do that in this case or is that too much of a constraint on their efforts well first of all constrains the the researcher to the boat too much and the dolphins really are moving around way too much the other thing that we found out is the dolphins really do not like these tethers because they're they're worried about being snared and that sort of thing so one of the things that is our design constraint is we don't have wires floating in the water um just often those will just stay away if that if that happens so it's all gonna be self-contained and we also need to make it so that you know when when the nieces is having researchers in the water calling for a scarf that you know when one researcher calls for it they can hand it to that researcher then another researcher calls for it they can can up that researcher i really want to be interactive and socially modeling to the dolphins you know here's the name for this object the scarf if you want the scarf make the whistle and what was really um astonishing is uh one of our trips um a couple years ago we really did have something that looked like you know actually several times where it looked like uh the dolphins mimicking our our word for i think was for sargassum um but the thing is it was at a too higher frequency for us to hear easily and also for the computer to hear and so now we've been upping our horsepower on our computers uh so that we can actually listen to that whistle being made at any frequency doesn't matter if it's in our hearing range or above so that raises an important question there's a fair amount of dolphin uh communication uh uh vocalization whatever you want to call it that's way out of our ability to hear so how how can researchers um possibly get a handle on that well i'm glad you asked that I think we have a couple slides that show some dolphin vocalizations can we pull up the first slide please one of the things we see here I don't know if you can see the scale um but uh this is only to 20 kilohertz oftentimes we see dolphin vocalizations going up even into the 200 kilohertz range um and so it's very hard for us to analyze in the water but what we've been doing is collecting lots of examples of dolphin vocalizations in socially meaningful scenarios from Denise's work for the past 33 years and looking for things that are similar so if you look at these vocalizations here's these three separate examples you can start seeing some similar patterns in here but it's very hard for us as humans to understand what these patterns mean or if there's even some repetition here so what we've done is created a a artificial intelligence algorithm that looks at every little chunk compared to every other little chunk and clusters them together and says hey does this look like you know the equivalent of the letter a or the equivalent of letter b does it look like something that is the the the same self self similarity so if we have next slide please this is worked by Daniel Kolsdorf who is was my Ph.D. student who's now currently working with Wild Dolphin Project as a side gig so what we're doing is just looking at all this vocalization and figuring out little tiny features which is what you see in the bottom left here little tiny features that seem to be you know uh throughout the the vocalization throughout the spectrogram and then we put those together into the equivalent of letters a b c d e f g etcetera and that's what you see on the bottom right um the uh we're clustering all the things look similar so these are a b c d f this is up to g now if we have the next slide please what we do from there is take any given vocalization here and we try to put them together explain them in terms of the subs subpart so in this case it looks like this vocalization was was a combination of a i and e now of course these are just our human labels to it we have no idea what this means the dolphins yet but one of the things we start doing though is look to see are these patterns repeating and are they patterns repeating in certain things like when they're doing aggression or play fighting or when you have a mother calf reunion where do these patterns are these are these um letters or these patterns consistent so if i have next slide if you would uh what we're actually look uh did and what daniel dick cosdorf did for his phd dissertation he's looked for a large amount of vocalizations and clustered them and saw if there was a grammar sort of a language structure to this and what we discovered much my surprise is that the more strict what's called a regular s expression in computer science was the one that best explained what we were seeing and so in other words the dolphins really are doing stuff that is consistent that has some sort of consistent structure to it and now we're trying to get to the stage we can say hey can we actually um get finer and finer understanding for those meetings now i'm not saying we're that far yet what we've found is that for four categories i think it was aggression play mother caffery union and foraging we discovered very consistent vocalizations that were common across those
2021-09-01 00:06
