Graeme Cox talks about understanding human emotions in immersive (VR) environments.

Graeme Cox talks about understanding human emotions in immersive (VR) environments.

Show Video

uh everything from skin flush to facial expression  to the posture we read these subconsciously and   understand the uh the emotional response  of the of the person we're interacting with   uh we're looking at ways that we can objectively  measure these and turn these back into   medical grade biomarkers of literally how we feel   i am very excited today to be joined by graeme cox  from emteq labs graeme it's very nice to have you   on today uh please do tell me a bit more about  yourself introduce yourself and what you do at   emteq thanks scott it's great to be here um so  uh as you say i'm the ceo and co-founder of the   company emteq labs uh personally i'm uh um my  background is in artificial intelligence my   degrees in ai from back in the 1990s and i've  been working with big data machine learning   and deep tech for my entire career both in large  consultancies and um in several of my own startups   along the way um like the first startup i was  involved with was actually uh whilst i was still   in uh in the university and that's been a kind of  a feature of my my life ever since um i built a   cyber security startup um around the turn of the  century and ran that through until i sold it to   uh to dell which now forms a core part of dell  secure their site the global cyber security   division um provides some of the core monitoring  expertise and algorithms into their solutions   and um yes the whole area of uh of of of human  performance and behavior and uh effective   computing has been a has been a kind of major  thread in my life uh from from the the start of   my career through to now um i uh either working  as a consultant in understanding how to improve   processes uh people and technology in in situ or  in uh trying to build and operate teams of people   in the companies that i've created myself  and indeed in my uh in my sport i've been   personally doing triathlon for about 20 uh 20  years now and i've been a triathlon coach for   uh nine years and uh in the whole  process of improving performance changing   behaviors for the positive et cetera has been  a it's kind of like that central thread of my   interest in uh in both work  and personal investigation   and so m-tec comes around really as i  think the culmination of all of that   um a fortuitous meeting with a a brilliant chap  called charles Nduka who is a facial surgeon who   i met in the school playground as you do about six  seven years ago down in brighton where i now live   led me to realize that we were talking about new  ways of reading expressivity on the human face   and how important that is to the fundamental  understanding of human behavior and therefore   how you can modify it um what we like and what  we dislike fundamentally drives our behavior so   that gut reaction we have to whatever it is the  things that we do the the the food that we eat   the things that we experience this drives how  we're going to operate in the future and that's   fundamentally important to understanding  the human condition and how to   um how to both improve people's lives and welfare  but how to actually also get the best out of them   as well on a purely health basis 70 7 out of  10 of the leading causes of avoidable death   are driven fundamentally by our behaviors um and  though by those behaviors i mean things like what   we eat and the drugs that we take into our bodies  whether we smoke or not whether we do exercise or   not the choices that we make the little ones  day-to-day that lead up to though to your   to your long-term health and determine whether  you are likely to die of stroke or heart attack or   emphysema or drug abuse or you know whatever it is  it's the vast overwhelming uh load on the nhs and   the burden on human health is fundamentally driven  by the choices that we make as individuals and by   understanding better how to nudge behaviors in the  right direction to help people help themselves to   help them be the best version of themselves is  the basic mission that we set out with emteq to   achieve wow okay that's fantastic um you touched  on health there and the choices we make and   actually there's been a couple of interviews  i've done recently about metabolic health   and how we can be better at making decisions about  how we manage that and the impact of that on our   long-term health as well um so maybe we'll come on  to that in a minute because i'd love to get your   kind of professional and your triathlete opinion  on that um one of them a company called levels is   very much focused in in your kind of persona the  high performing athlete so we'll talk about that   um but tell me more about what is um what is the  technology that you do at emteq and how does that   how would people see that yes absolutely so uh  and obviously i've given you a very high level   view and it's like well what is it what is it  we're actually doing so fundamentally the core   technology that we have been uh developing which  is which has a lot of novelty around it and has   led us to creating a lot of new intellectual  property along the way is in uh is in improved   ways of reading the physical signals of human  emotional response we like to do that with sensors   primarily rather than with cameras uh and we are  reading things like the electrical muscle activity   inside the face to understand the  tiny um reactions that we have the   the micro expression reactions that we have that  indicate our actual gut response to whatever the   stimulus is we're being faced with um features of  uh changes in our heart rate and the variability   of that heart rate and the heart rate itself  and the subtle features within it that indicate   um the level of excitement or arousal that we  have when we're faced with whatever stimulus is   uh and uh and and movement the body movement  of our particularly of our upper body and head   that indicates things like whether we're attracted  into something or whether we're repelled from it   and understanding those different features  the the kind of things that a somebody with   a high emotional iq subconsciously reads and  interprets extremely well and at the other end of   the scale somebody with a degree of autism reads  exceptionally badly this is these cues have been   developed so a digital biomarker that indicates  emotional response that can be used at a clinical   level to to to help drive therapies and behavior  change programs that sounds very important very   important especially in today's in today's  world where i think we're all suffering from   conditions and anxieties and issues that we  probably didn't even know we we could be succumbed   to a year ago but in terms of just trying to frame  it there's some there's a little chart that i use   um all the time it's the united nations sdgs and  here it is here's one i printed out earlier on   um i can imagine you know good health is a is an  obvious um it's an obvious bucket here that you   would fit into are there any others in here that  you think you know emteq could fit into maybe   quality education or reduced inequalities how how  do you feel about that um yes or well certainly um   the ability to take that next level of human  communication and quantify it so to go beyond um   speech analysis which is currently state of the  art with alexa siri etc and move that through   into an emotional understanding of the of that  next layer of communication the body language   and um subtle vocal and physical cues that we give  out that indicate how we really feel about things   that does that that allows us to better influence  behavior at all levels and i see i see the   uh that there's a as a spectrum where at one end  at the one end of the spectrum you are dealing   with clinical issues so the vast quantity of  uh anxiety issues that exist in the world today   um in 2018 it was recognized that  over 300 million people worldwide had   a clinical anxiety problem aside from just the  general day-to-day anxieties and issues we deal   with that are at a non-medical level that's before  coronavirus hit us you know the level of anxiety   uh and tension in the population today as we  speak you know we can all see it it's uh it's   risen massively over the last year and that  is a real uh problem of how we deal with that   but but we're on a spectrum and as you say in  those stg's the spectrum of behavior change goes   from dealing with actual clinical issues at one  end to improving performance of otherwise normal   people at the other end so and those performance  improvements include improved educational outcomes   by tailoring educational content to understand  your audience to actually play to your audience   the ability of content delivered to adapt to  whether the specific audience you're playing   it to is actually enjoying and engaged with  your content or not provides a potential future   generation of educational delivery in in immersive  technologies virtual reality augmented reality etc   or 2d content that allows an adaptation and uh  personalization of delivery that that we that we   don't we're not able to see today so is that where  we would find your technology in the immersive   world integrated into virtual and augmented  reality systems so so our first uh products to   market are in as exactly as you say scotland are  integrated into virtual reality and that's for a   number of reasons so the first is that um a lot of  the commercial use of our technology today is in   is in research whether that is academic research  um clinical research in uh drug and therapy trials   or or indeed in commercial research in you  know market research around content testing etc   and immersive environments like virtual reality  provide the ultimate experimental environment   so for our clinical work where we're dealing  with things like post-traumatic stress disorder   it's extraordinarily difficult and expensive to  put somebody back into a situation that gave them   a trauma in the first place you know you can't you  know you wouldn't want to put somebody in a war   zone in order to help treat their uh their their  war zone related trauma you wouldn't want to put   them back in an accident scenario in order to do  that immersive technologies allow you to provide   access to those traumatic experiences in a  safe and controlled manner similarly when   you're dealing with um anxieties or phobias so  social anxiety for example uh mixing mixing with   crowds social interactions etc you can do that in  a safe controlled and cheap and effective manner   in virtual reality that um that physically doing  that with therapists in in live scenarios is is   just logistically very hard and ultimately we see  ways for people to start doing self-treatment for   anxieties phobias uh and traumatic um episodes  using home-based systems that allow them to to   run through protocols at their own pace in their  own time and to uh to gain the benefits without   constant oversight from uh from a therapist  that is our end goal and similarly again on   that spectrum at the other end of that self-paced  educational performance improvement uh tools like   training people for better public speaking for  example training um frontline personnel to be   able to deal uh to be more resilient in dealing  with emotionally difficult uh environments so   how do firefighters cope with a uh with a  with an accident scene the moment they arrive   being able to train uh in simulations gives huge  benefits to both the organizations that are paying   for that training and indeed to the individual  themselves in the the number of times they can   work through that the flexibility they have to  access that content uh and uh you know there's   there's a huge amount of evidence to show that  simulations do evoke those same gut emotional   responses as the as the real thing so we're a fan  yeah and i've seen some pretty astonishing figures   about the efficacy of virtual reality training  compared to even face-to-face classroom training   um and the numbers are quite you know quite  amazing i think it's um 50 reduction in costs   four times better learning outcome faster learning  the ability to translate all of these um actually   to carry over from the virtual world into the  real world these training is quite astonishing   with your technology does that make it even more  effective because you can tune it to exactly   my response which may not be the same as yours  or the person next door well primarily in the   work we're doing today we're we're adding we're  adding a layer to that uh training or therapy so   rather than providing uh behavioral training for  purely um physical process oriented tasks we're   able to provide behavioral training to improve uh  improve emotional response to reduce your stress   to reduce your anxiety to improve your ability to  react accurately and appropriately in the moment   bringing in that extra dimension but  you're right as well that in the future   the ability to understand the level of  engagement that an individual has with   the subject allows the opportunity for the  content to automatically adapt to that person   so for example if you want to put somebody through  a series of more complex and demanding tasks   being able to estimate the cognitive load of  that individual as they go through these tasks   allows the adaptive content to deliver the  right level of complexity the right level   of stress the right number of distractors  to the individual to suit a personalized   model for them and that's absolutely a goal  that we have with us with our technology   so on the therapy side um i'm a mild sufferer  of um fear of heights and i think if you get to   ask my family they may say it's slightly more mild  slightly more than mild um but yeah i know people   who are absolutely terrified of going up a ladder  i can do a ladder as long as it's not too high   but when you get me climbing up castle ruins  and things like that then i get a bit edgy   now if i was to um want to overcome those fears  i understand that that kind of exposure you   have to just keep nudging yourself a little bit  further my nudge would be completely different to   somebody else's nudge who may be better or worse  of that so is that the kind of benefits that you   you're seeing that ability to personalize that  treatment for an individual so they can be more   effective at overcoming a fear or a phobia yes  absolutely so if you think about it so so we'd   call this virtual reality exposure therapy  so the principle there is in the same way as  some people with milder allergies might train  themselves to deal with a with a mild peanut   allergy for example that that slow and gradual  exposure to small quantities of the stressor   allows your system to build up resistance to  that to that stress or over time it's exactly   the same it's a desensitization process whether  it's fear of heights for flying fear of crowds   whatever it might be that by gradually introducing  yourself to the thing that causes the anxiety you   can learn to overcome and manage that anxiety  particularly if you're given some mental tools   um to to to to keep yourself calm and to handle it  when you find yourself in that scenario the trick   however for success is to be given the right level  of stimulus so if you over stimulate somebody you   know so i suddenly put you standing one foot on  the top of the grand canyon from from scratch   you freak out your your fear of heights is much  worse than it's ever been before there's no way   you're ever coming back to that therapy again on  the other end of the spectrum if i you know i put   you on top of a you know a wooden wooden crate  in the simulation you're a half a foot off the   ground you're standing there thinking well and  what's the point of this so under stimulation   and over stimulation both lead to people not  completing the protocols not getting the benefits   that today um skilled human interaction is  required in order to gauge that level of   uh exposure so a behavioral therapist cognitive  behavioral therapist a psychotherapist etc   would lead you through a clinical process  in helping you treat that phobia and they   would guide saying right i can see that you  need a bit more stimulation a bit less etc   and today and in the near future our  technologies will support and augment   that therapist by providing the therapist with  more objective information our potential end   goal here is that we can take that therapist  intelligence into the ai of our system itself   and the system can automatically read you  and understand the level of stimulus that's   appropriate to you so that you can go through  self-guided therapy in a safe but effective manner   ah i'm glad you came back to ai that's one of my  questions all of this sounds like sensory fusion   and augmentation again something i studied  at university around the same time as you   um i was wondering how the ai piece comes  into this and what benefit that adds to it   absolutely so what we're doing here is developing  a a personalized model of uh of of of emotional   response which to the best of my knowledge and the  in the research that we look at around the world   has not been effectively delivered before so um in  the field of uh emotion ai as it might get called   generally um the world is is is mostly focused  on a single approach a unified approach to   classifying everybody's emotional response  regardless of your age gender nationality   racial background etc and whilst things like um  the major expressions like a smile for example   they are universal features a smile is a smile  uh in its basic form wherever you go in the world   the way that we use our expressivity the  the quantity we express and the uh and the   the way that that is subtly delivered it's  it's not only different in japan as it is   from america but it is also different in men and  women it's different in it's different every level   it's different between you and me scott you know  so you might smile three times more in a day than   i do but that doesn't necessarily mean that you're  a happier person than me that just might be that   your base level of smileyness is set different  to mine and and that and that that personalized   approach to understanding um individual  expressivity and emotional state i believe is   absolutely key to success so we are building  uh deep learning models that not only provide   us with classification across demographics and  allow us to understand broad uh sweeps of of of   response to a given stimulus but actually also  a calibrated learning model for an individual   that that improves its understanding of  you the more that you have exposure to it   and that's a that's it's it's a it's a  machine learning approach uh where we've   moved we're moving into the deep learning space  now which is very exciting as uh for me as an um   with my ai background it's wherever we've been  looking to get to for the last few years and   it's starting to show real uh benefits in  uh in delivery in practical terms i haven't   really described you know physically what we're  dealing with here so so you know to to put him   in in the viewers mind the technology we're  dealing with here we have vr headsets that   have a set of sensors integrated into the facial  interface the bit that actually touches the skin   around the face and that allows us to read  the electrical muscle activity of your face   reads heart rate information reads  your body movement skin flush etc and   uses a combination of local processing inside  the device itself and also cloud deep learning   processing to interpret sensor data from  the human body through into physiological   data and from that physiological data into  a personalized model of emotional response   when we add into that the data stream that comes  out of the virtual reality environment which   provides us with context we're able to really  understand your response to different scenarios so   you know an emotional response is only valid if  you understand what it is you're responding to   you know so so if i just look at you and i don't  see what it is that you're smiling at frowning at   um shying away from whatever it might be  i can't really understand you and your   your your your range of emotional responses and  the wonder of virtual reality is that you have   detailed access to all of the data on   what the stimulus was as well as all about  biometric data and what the response is   that together gives us our machine learning  models that drive the company at its heart   wow um now that raises a lot of questions in my  mind one about are the developers ready to um   to take this kind of information and build these  personalized environments adaptive environments   and then very much linked to that and everything  you just said just then there's a lot of very   personal and intimate data that's coming out  of this right um and we don't live in a world   where personal and intimate data is um very well  managed so what are your thoughts on on kind of   the responsibilities around around managing that  data that's a that's a fantastic question scott   and something that's that's very close to my heart  um with my combination of my my my focus and my   passion in in helping people through helping them  understand their own behaviors with my background   in cyber security as well uh the whole issue  of data privacy and indeed data ownership is   is very close to my heart so the approach that we  uh that we take with our data is that all the data   remains owned by either the individual who is  self-managing using our our clinical end system   or belongs to the researcher if it's a research  system who and they are collecting data from a   number of participants and although our data flows  through our cloud infrastructure we do not retain   any of the um user recognizable data features  that uh that so that none of the raw data from   uh collection ever hits emteq um accessible  property so it goes through our uh through our   deep learning engine and provides insights in  terms of um affect emotional response back to   the to the to the recipient but uh we only  retain the higher level insights that are   generated from that so the uh the ability to  improve our personalized models is of course   fundamental to the success of the company but  the data itself remains owned by the people   who collect it and that's a that's a that's a key  um fundamental requirement for us we're actually   members of the um uh the i'm gonna get  this wrong the x x rsi which is a um   a group fundamentally built for maintaining  and developing the security and privacy of   data collected using immersive environments  because as you say these technologies provide a   huge amount of personal information and even even  in all an ordinary vr headset if you take out the   biometrics so it's just a standard vr headset the  quantity of personal information that could be   captured by a developer is huge to the extent of  things like um you know understanding gender and   age so demographic information gleaned from how  people consume vr content and sexual preferences   educational background  these things can be inferred   from vr scenarios quite effectively and there's  been some interesting research done on that so   um who gets to own your data is fundamentally  important i completely agree that's that's   particularly eye-opening i think some of those  things um inferring sexual orientation from what   you look at i guess within a virtual environment  absolutely well i mean this is a very crude   example but you go into vr i present you with a  woman in a bikini and the the way the the way your   eyes linger and for how long gives a pretty strong  indication of your um of your sexual preferences   and uh and some information about your background  instantly and there's been some solid research   uh on exactly that interesting  experimental setup but there you go  you know and i guess as virtual reality in  these immersive technologies becomes more   and more available and more and more pervasive in  our lives this is a question that's not going to   get any smaller is it it's it's going to grow  totally i mean i i it does amaze me actually   how accepting we are of our social media and the  quantity of data that is being collected there i   mean my understanding of the um of the algorithms  that go behind tick tock for example to determine   uh what the video feed is that you're given  uh there's some there's some startlingly um   insightful algorithms going into every movement  that you make on screen how long you watch every   every video for where your fingers hits on the  screen to scroll um every part of it is analyzed   in order to tailor that stream to you and and it's  incredibly effective but you know it's if you uh   uh if you know any tick tock users  they get hooked pretty quickly   yeah i have heard my oldest also keeps  telling me exactly yes that is the demographic   yeah so um i think lastly to touch on some of  the challenges that many of us have faced in   the last 12 months um and a broader conversation  really um as we find ourselves many of us find   ourselves working from home a lot more than we  ever thought we would if we ever thought we did   um there's a lot of challenges that we have to  deal with there um isolation anxiety um separation   anxiety from your team all of these big things  that are that are kind of building up in in us   um and then there's the role of the employer  so i in some of these conversations i've seen   we've had a lot of conversation about how  far does that role of responsibility that   moral and even legal responsibility of employer  stretch into a working from home environment um   so be good to get your idea of what how  that landscape lies and then the role of   the kind of technologies we're talking about um  today does that fit in in that picture at all   okay yeah really interesting stuff so my first  thing my my personal philosophy on building   and managing teams is that um empowerment and  self-determination is the way forwards now i say   that um in the full knowledge that the teams  that i work with are educated to the highest   possible level they're all career individuals  they're all highly paid and therefore you expect   them all to be highly motivated and individual uh  very individualistic in their approach anyway but   nevertheless for within that context i'm a very  strong believer in personal determination about   flexibility in working hours approach etc and as  such i'm massively against employee monitoring   so that to me that is uh that is anathema as a  business owner and uh and a manager myself in   terms of our technology and how that how it could  be applied because clearly anything that can read   emotions could be applied to both uh for both  human good and also for nefarious purposes as well   um i focus specifically on sensors rather than  cameras because they the depth of information   that we can find is so much greater which  allows us to build our personalized models   at a level that we just couldn't do i believe  using features from um from from video footage   however there's a second kind of um network effect  benefit of of using sensors which is that there is   a very strong sense of um personal acceptance  in using our technology in the first place   the fact that you physically have to put something  on your face whether that is our virtual reality   headset or the or the glasses that we have in  prototype for real world estimation of emotional   response means that you are physically accepting a  level of computer monitoring when you put them on   and hopefully doing it because you are you  accept the benefit that comes with doing that you   that you you put your donning your personal  assistant who is working purely for your benefit   to understand your responses and to give you cues  that will help improve you it's very different   from an employer deploying software to your work  computer with the camera here and using that   camera data to understand whether you're paying  attention to your screen whether you're actually   working whether you're looking grumpy or so  cameras are intrusive fundamentally that's part of   the reason why google glass failed uh when when it  came around it wasn't that you know the glasshole   approach it wasn't that the glasses looked weird i  believe you know in silicon valley that's not such   a big deal at all it's the fact that there was a  camera pointing at the person you were talking to   and that's the level of social intrusion that even  in in the 21st century i don't think we're able to   uh to accept um so um sensor-based technology  focused on the wearer is actually much less   intrusive because there involves a there's  a there's a level of personal responsibility   and acceptance in donning it and using it in  the first place and i believe that's why we're   actually really well suited to succeed in this  field where a lot of camera technologies may   fail because cameras are do not come with personal  acceptance anyone can point one at you anytime and   and they don't have that that ability to deliver  personalized information it's much more generic   um i've got a list of other things we could  talk about but in the interest of time   um let's kind of wrap it up and uh   if any of the uh viewers are interested in finding  out more about what you're doing at emteq or even   xrsi the work that you're contributing to there  where should they go what they what can they find   um well please come and find us on our website  first of all emteqlabs.com um anybody who is   interested in contacting me please do email  me graham emteqlabs.com i'd love to start a   conversation with anybody's interested in finding  out more about our technology or we can um we can   help in any way um the xrsi is indeed xrsi.org and  my co-founder charles sits as sits as the medical  

representative on that on that committee as i  contribute to from my cyber security background   um and um yes you know please start the dialogue  it'd be lovely to talk brilliant well you can give   me some links and i'll put them down here like  magic videos they should be down here for you now   um and um yeah so it just it's just um thank you  so much for coming on and uh sharing your thoughts   and ideas um i i found it fascinating and like i  said i hope we can do another one later on to talk   about the rest of these the rest of these things  i'd love to scott brilliant brilliant um for   the viewers please do reach out to graham using  the links below um do like and subscribe to the   youtube channel and there's a lot of other  very interesting interviews coming online   soon so once again thank you so much for your  time today it's been one thank you very much

2021-10-11 15:54

Show Video

Other news