OHMIC.AI Ear Detection Touch Control & Biometric Feedback Using Existing HW In Earbuds & Headphones
foreign Berg in Hong Kong with another episode of THD podcasts today we have a company joining us from Montreal that if I understand correctly they're using the feedback loop to detect sensing in an in-ear for things like biometric feedback and in your detection when people are wearing their earbud uh but before we get into that let's not forget about our sponsor the ulti association so audio loudspeaker Technology International they'll be hosting a meeting suite at CES in the Venetian this coming year so a new uh little mini Expo for embedded Technologies coming up so we encourage everybody to watch this space and make sure to come see us all at uh CES so without delay Simon's in from Japan again this morning good morning Simon morning all and I guess evening for you Muhammad yeah so Muhammad Ashraf is that how we pronounce it yep okay okay so so oh Mike uh there's been a lot of stuff going on in audio in Montreal lately so so yeah we found out about oh Mike and we wanted to find out about what you guys are up to so uh maybe introduce yourself uh Muhammad sure um thank you uh David and Simon for having me on here it's uh it's exciting um so omecus is um is a startup in Montreal uh that's uh that's developing this technology uh to actually use the existing infrastructure and headphones to kind of realize the features we've always wanted to um so we've seen a lot of headphone Technologies recently in the past couple of years have heart rate or insertion detection or um I mean now buttons are taken for granted so once you're freezing it's expected in headphones but uh the underlying notion is that all of these things require hardware for every feature you need you need a component but doesn't really have to be that way and that's that's our entire methodology it's that you have everything you need in in the driver and in the headphone and the hardware is already there and as long as you leverage that data in in a clever way you know you can get all these features so that's what we're trying to do okay cool so maybe just to visualize things for people let's uh jump into a presentation to to introduce technology so this is Olmec um we are based in Montreal as David said uh we we secured funding from tandem launch um last year and um our team has grown up to uh eight Engineers so far and we um we're working really hard day and night to develop a technology that I'm going to talk about in the future to kind of revolutionize uh headphones and and that's ultimately what uh the entire company is about headphone technology and how we can we can make it smarter okay so maybe we can start from the beginning uh our team members consist of the core team which is the product key I uh who handles Business Development things like like that and Dr Carlos Mendes Jr he's a technical lead he has a PHD in integrated circuit design and and Emily potros Talent launches uh General partner and someone who's been supporting us every step of the way of this startup because startups aren't easy as you you know we're also lucky enough to have an engineering team of biomedical Engineers analog circuit designers and Dr Danilo Pena who has a PHD in Signal processing for Echo cancellation and and noise so right upper Ali he's actually from Portugal uh working remotely but the work he's been doing has been pivotal in actually denoising and removing interference mitigation okay so maybe we can start with the landscape that we're working with yeah um there's a lot of competition around in in headphones it's it's kind of a competitive landscape again as you know and right now there's a lot of intersection between headphones as a wearable and some Industries like wellness for example and health and these are larger Industries you know they're not small they're they're in the buildings and um the way they intersect is through integrating different features like for example gesture recognition insertion detection user identification and heart rate uh but as I said before these features come with drawbacks you know you don't get a free launch of engineering and some of these drawbacks are Slimmer profit margins expensive products a far reduced battery in life and an increase in e-waste and what that means is that we essentially through omic uh have an opportunity from this problem uh because we can provide the low cost uh low power uh way to enable all these features without being added weight of these components right I think just just for people watching they should understand that there were some people using PPG technology I guess three four years ago Jabra Bose and Phillips that launch products and I I know that the Bose product may be shipped a half a million units total and for Bose that's basically a failure um so a lot of it was that people just didn't see the functionality and and the added cost benefit to add that to their earphones so I think I think for people to understand the summary here what you guys are doing is trying to to battle that bomb cost implications so that people kind of have these features with the existing Hardware that's in the device yeah I completely agree so so the thing is you know um when you add that cost someone has to pay for it and when the users pay for it they expect to see something in return um and having heart rate sensing it's it's it's great but that data doesn't really mean much without um insights to come with it so when you have a headphone they can deliver that data without a companion companion platform or without insights or algorithms that can that can tell you exactly um why that data is important to you then you ask yourself why am I paying extra for this feature uh but that's one part of the problem the other part of the problem is a technical in nature because um when you add these PPG sensors for heart rate sensing or proximity sensors for insertion detection for example um you you run into a design problem where you have to make more space for the batteries because these are LEDs you know they're not they're not really passive components um not exactly they take up a lot of power and so you have to have a trade-off between either battery life or size which can lead to an uncomfortable headphone um especially when you talk about tws so it was kind of like a race where tws was gaining a lot of share and there was an expectation of different features in the wearable ecosystem because nowadays wearables are are the values in the platform it's in the ecosystem itself rather than a single device so so all these different Dynamics all these different interplaying forces meant that um the headphone manufacturers had to really think part about whether or not this component was going to add enough value to justify the cost that they're incurring and the cost that they're passing on to the user um and you can see that uh you know that period is a bit over I mean not that many headphones right now exist with PPG sensors because the answer was it wasn't worth it yeah and and there was a lot of incumbent stuff like the the watches and such yeah so and I mean I'm not a hardcore athletic guy but the people that were serious Runners that I spoke to at that time said that yeah they're they're all comfortable with their watch uh for their heart rate and and that kind of covered that space yeah exactly I agree and that's not really stopping I mean you see now um apple apple just got FDA cleared for um for a normal uh heart rhythms and Fitbit just got appeared for um I think it was um arrhythmia detection uh and yeah it was it was an uphold battle I mean you had to get a large sample size I think it was close to 450 000 uh per study um for five months but um but these these These wearables are becoming more and more accurate the question now is what place or what position the the headphones have um with respect to these to these devices because this is a foothold for sure um and and in omic especially uh we ran into that problem early on where we talked about uh okay so we bring in this feature but this feature already exists what are we going to add to the user and we really struggled with this question of what what new thing are we adding and um I think I think it really really clicked with us when we found out that we weren't really competing with wearables rather we were adding to the entire ecosystem as a whole um it's not really that we want to take away from the market share of the smartwatch it's more that we want to increase the use of headphones in general because not only can we do hydrate something we can do a lot more than that and so so you get you get into these interesting different possibilities when you think about um different data points uh when you think about media when you think about stress and think about uh the fact that not everyone has a smart watch um and so you know it's it's really interesting space and it's not really adding anything to the manufacturer which is what takes it over the edge yeah yeah and when you say gesture recognition that's like functional control like the touch control uh concept yeah so I yeah you know all the airpods have that insert detections on all the airpods and they're using they're using a like a proc sensor like a capacitive sensor for most of the touch for the insert detection they're using some kind of uh maybe an infrared sensor something like this so there's a lot of Hardware in real estate because like you said space constraints so if you're able to do that with the existing uh transducer as the sensor I think you guys got a good match here that's the idea that's the idea David all right so here's what happens in in the usual sense um the headphone is always seen as an output device where the membrane or the transducer like you said moves back and forth because of an electrical signal and that creates special waves which we perceive as sound but in omic what we do is we actually see the system as a whole when you put on the headphone there's a coupling between the ear canal and the membrane itself and what that means is the quality functions like heart rate can actually move the membrane back and forth as much as the membrane moves the ear and so we capture that and we translate that into signals like heart rate like insertion detection the cartridge variability like tapping and sliding so all these cool features that I mentioned before okay uh when you say you capture that how is that captured well the same way that music is delivered into the ear uh the feedback is delivered back into the device so our technology is able to actually isolate that that signal um that's always been there but generally has been considered feedback has been thrown out or filters yep okay so it's actually uh the speaker voltage yeah exactly there is the primary signal okay fantastic sure so this is the phenomenon as it is right now and uh the speaker vibrates and inside the ear canal uh there's a lot of different phenomena that coincide with signals that we find like Movement Like speech like heartbeat and when we look at the signal it's uh it's very very rich in a lot of different things and we've only just scratched the surface when we say heart rate this is to be honest what we're comfortable telling people that we can do but there's a lot of different profiles here that we can actually capture um we um we are only throttled right now by the resources that we have and by the team because um we think that we can get hydrated variability we also think we can get breathing rate from there and so putting these into models that can infer for example uh engagement movement things like that can lead to a lot of opportunities in a lot of different verticals but right now what we're focused on is like I said before the basic functions that users take for granted right now when integrating into headphones so this is essentially our technology stack so um uh can I ask a couple of things so uh you've got to get you've got to get the signal off the uh off the speaker uh and then are you proposing to use like the Bluetooth chip as a signal processor for this signal or uh somebody needs to add another chip yeah yeah no Simon you're exactly right uh so right now tws is is taking over really um which means that we have to find a way to integrate with the Bluetooth socs um so we're talking about the qualcomms the broadcoms um the the guys that really put the entire system with a PCB around them so what omic wants to do is and wants to actually miniaturize its signal right now it's technology right now into an IP block for example that can integrate within that chip and so that way we can actually transmit the data back into a device and run all these cool algorithms we have on our software step yeah okay so primarily uh software-based solution it's Hardware enabled but yes primarily a software basically yeah and um do you need to do some kind of signal conditioning on the uh on your speaker signal essentially it's gonna have a pretty tiny yeah oh yeah oh yeah um the pro the primary Direction our efforts are going now in terms of research and development is mitigating uh motion artifacts so um in terms of every wearable out there motion artifacts are really the pain of our existence when you talk about Optical signals for the PPG sensors a sunny day can ruin your measurements uh when you when you are for example in a spinning class it's the ideal scenario to Showcase just how effective heart rate measurements are but that's simply because you're holding on for your life with a very stiff wrist however when we move into sprinting running um things that involve flexing the wrist and relative motion between the actual sensor and the wrists then things get interesting because then uh there's a lot of of interpolation involved there's a lot of guesswork involved there and this is for everyone across the board um there's a lot of techniques to get better um phase lock Loops multiple LEDs but they all come at the cost Hardware or software based and we're no different uh the difference here is that our signal is acoustic in nature so that means that no one's done work on it uh it's it's it's new ground and we we are really in terms of the commercial space some of the first people to actually understand it and explain it and research it and mitigate these motion artifacts okay I wonder if the sensitivity of the diaphragm material would help like that that I guess by it's kind of bi-directional right so we have Acoustics going one way and then this reflection coming back yeah yeah so uh some other research is going into actually uh characterizing the diaphragm itself and understanding the electromechanical Dynamics in play um we're learning a lot about because we're eight people you know so everyone's got to do something so we're learning a lot about um mechanical simulation and um and um yeah there's there's a lot of modeling involved we're very lucky that there's been work done in in the past of all the examples academic and otherwise I mean some of the people on my team have actually uh done this before uh so um well done something like it before so so we're lucky enough to actually have a body of work that we can build on uh but uh that being said yeah definitely the games changed uh it's it's not the same parameters that other variables have to worry about um we will have to worry about transfer functions the size of the diaphragm it's it's um its range of motion even and yeah like you said the sensitivity so um there's a lot of startups out there that actually do research on developing better membranes better membrane materials they're all material scientists but the further adapt research goes the more effective our our technology becomes because then we can hear our signal much more coherently right cool okay foreign technology stack this is basically a high level overview of how the tech Works um but now we can talk a bit more about the business model itself uh so one of the first things we did and I'll make is we did a very very small very rudimentary survey on what exactly everyone has so we actually went on uh this website called enter Mechanical Turk I think and we we got 200 people it's very very basic but it gave us a starting point as to who we should talk first and what we found was uh there are actually still some people that have wired headphones and not just line headphones but over ear headphones it's not a lot but it's enough um what that meant was that we could spend some of our resources in the beginning developing a wired toggle uh which is something that we actually have a few prototypes of and what this does is that this actually plugs into the computer and you plug the 3.5 millimeter Jack into it and it unlocks all these features and it's really cool because it just means that you can get the regular headphones that you've always had and you can transform them into something smart into a device that can read all this data and we just thought about all the different verticals that we can attack just using this PCB we we never thought about you know selling it or being direct to Consumer but what we thought of is well we don't have to make new headphones we can just unlock these features and existing headphones because we have what we need and so you think about for example someone playing a game and whether you're streaming or whether you're just playing you can actually plug this in and the game can interact with your heartbeat with your heartbeat uh which leads to a lot of different possibilities in personalized gaming or even effective gaming where for example uh the boss um gets stronger and stronger the more stressed out you are or you yeah or your stamina for example when you're sprinting is related to your heart rate if they see that your resting heart rate is lower they think you're healthier and so you know you can avoid more stamina but vice versa okay but then yeah uh that's actually one of my favorites uh but then we thought about other things like for example Telehealth uh like when you put on the headphone and you're talking with your doctor and effectively the doctor you know they can see um your heart you could they can see a live feed of your of your heart beating while you're talking to them so and this is your this is your doctor you know this isn't just anyone uh so um and obviously you'll be validated against something else but it's it's a good start you know it's it's um it's a good way to actually get people at home to to be more conscious about their health and to actually get that transparency and and have it far easier a lot of times um people feel like patients when they have things devices trap to their body and this isn't that you know so I think it's a step in the right direction um same thing with employee Wellness uh right now I have headphones on uh but I had these headphones on the whole time uh all day when I was in meetings for example and I was listening to music and focusing uh these headphones could capture my heart rate this whole time if they could have captured my hardship variability and my stress levels uh then I would have known a lot of a lot of different things and a lot about myself and how I react to different structures like if I got an email 450 or something um it's it's uh it was interesting it was interesting for us um at least but but ultimately this is just a start and what we really want to do is we want to capture the wireless Market and for the wireless Market this looks a bit different based on our survey but what we found is that we can actually access a lot more uh Market just because of like I said before tws and uh the mobility aspect involved there people like Wireless especially when they're on the go especially when they're running especially when they're when they're out of their house for example they need to move a lot and so you think about the exercise market and think about virtual reality or augmented reality even when you're on the street or or even if you're watching a movie for example outside right now there's a lot of money being spent on uh screen testing if you for example are studio and you're producing a movie you want to know the audience's reaction or the audience's response to that but right now with streaming models being what they are and everyone you know more people staying at home and watching releases at home um you know you can think of something basic like a beta tier or a screen test here where you're getting access to exclusive content only if you get to watch it with these headphones and so that way there's a lot of money spent in analytics and audience reactions there's a lot more data at scale it's aggregated automatically and the people get exclusive content which is something a lot of people I know personally I would have loved to see this neither cut before came out before an entire Twitter campaign um same thing with that girl um I have a few friends who have seen bad girl and you know no one else can see it now so that's the most opportunity for sure all right just quickly that um that wire dongle is that commercially available or is that just a proof of concept not yet uh we're working on it now and and we're seeing which model fits best uh with with our trajectory um but but ultimately this dongle is something that uh right now we're kind of testing the waters for in businesses in Montreal um but it's it's really straight right now um we can't say anything for certain but uh yeah yeah not the first time I had to answer that question okay so we talked a bit about the use cases but maybe we can just drill on them uh just a bit more uh like I said before remote patient monitoring uh responsive training is one that's really interesting to a lot of different people especially human performance and immersive Wellness so we're seeing a lot of different meditation apps right now uh give you soundscapes and guided uh guided audio files and things like that and you know we thought of we thought of what if omic was to partner with some of these people and actually give feedback as to whether this content is right or not but not just give feedback to the business but give validation to the user give them something like a meditation score give them an an incremental Improvement on how well they they're doing that day um this is this is an objective um numbers based approach to Oneness that doesn't exist now unless you have to buy an expensive device that really impede uh just your your experience in general I think and um speaking of experiences uh there's a there's a huge wave coming along now with with augmented reality and virtual reality and the benefit there that we like is that you know the infrastructure is already there you don't need any extra devices you don't even need a smartwatch in this case you can just put the headphones on and it can essentially give you Telemetry it can give you um that effective experience but it's not just for you it can give the creators and the curators of these experiences um feedback so you think of a recommendation engine for games for example um this is data at the granular level that just doesn't exist right now the closest you can get to it is a like or a dislike and not that many people rate it but more importantly it doesn't tell you exactly what elements of the experience people liked or not and that last sentence that um that that I said we actually you know we were talking about it with the team and and we found that we can actually expand on that so it doesn't just have to be VR and arguments it can be any experience um if you listen to a song on Spotify through this two song on Soundcloud if you're watching a YouTube video uh you have a second by second idea of what this person's heart rate heart rate variability and stress is and what that means is that you can actually understand what parts of the song what parts of the content they liked and what parts they didn't engage with um yeah and and for the and at scale uh so this is just through the headphones which you already need if you're watching it publicly or if you want to listen intently if you want to concentrate on it and so that response analysis or that engagement feedback again isn't something that exists right now and would be really interesting um to to have without all the overheads it could be kind of interesting for online poker so you can pay an extra fee to know if your uh your opponents are like uh their heart rate well they're they're they got a certain hand yeah yeah but only if your opponent knows that you can see their heart rate yeah I'm being silly but but yeah that extended uh data set from people's Biometrics is yeah all kinds of use cases especially for the commercial stuff um to to to get more money out of us all that's the dream right yeah but um yeah so all of this is is you know it has to be realized and um and right now what we're doing is um like I said we're working on this wire dongle we have our evaluation opportunities um with a few businesses and and uh we're looking for a pilot project to kind of showcase um this this type of data and how how what it affects people because it's great that we sit down and dream about other stuff but you know we actually have to put it to the test and this is this is our way to do it um and and since you know we are startup we're looking to to raise um within the next year um and the reason why we want to is because we want to explore the wireless implementation of this technology we want to develop a wireless prototype in and have evocates that we can send out to businesses and actually have them test this out for themselves because a they have more equipment that they can do and B they think about more things in us we can't really think of everything we'd love to be great but having an external business actually rate this and understand this and test it for us and give us feedback on it will give us either validation or the direction for further development and finally um yeah and finally uh we we plan on raising uh for our series a uh that will give us the resources we need to actually have this implemented in an earbud uh through either a licensing approach or making a chip ourselves But ultimately the idea is is the same uh proliferation of this technology and just passing on the value to the manufacturers to the users to to every every Point within the value stream okay question time then fundamentally it is about sensing vibration related data yeah um and so that relates to heart rate and what else so uh hydrate is is the fundamental one um but through heart rate because the data is granular we can get heart rate variability which is simply the interval between one Peak and the other and the change between those intervals a collection of those intervals within a specific window um we can also get through the vibration um and notice if you tap or if you slide up and down left and right whichever one you need and um we know when you have the headphone on or not the reason why we know that is um because of the noise floor so we know exactly whether uh the signal is a signal of someone wearing the headphone or not it's a very simple classification algorithm um we run some signal processing and you know we collect some data and that's that okay so uh yeah you've got the vibrations I think you've got weird detection and attack detection this kind of thing yeah we also have it's actually quite a nice one possibly a little bit easier on the signal to noise issue too I hope so I hope so uh we actually have uh identity verification too but um so so that was actually pretty clever um we when you put it on because we have word detection what happens is that we're able to essentially isolate well we know we know we have it on so the next step would be for example to play a tune and the echo from that tune gives us uh something like a transfer function of the ear canal and the geometry of a person's ear canal from one person to the other is quite unique um so that means that we can actually get a unique signal for a user at least and verify whether that user is device owner or not and um if someone wants to implement this type of technology is it likely that they would have to do a characterization of their specific headphones some calibration procedures yeah yeah um exactly so um it's it's not it's not unlike the calibration of a face ID on the iPhone for example when you first take it out of the box or when you reset it so um similar to a fingerprint as well so all of these things are data collection methods for calibration just like you said Simon and um and you know we're no different uh if you're verifying someone you have to get to know them right and uh and I think that's that's the core of the technology now I do have to say though that identity verification is uh a bit further down the road just because uh hey we want to make sure that this this signal is is quite coherent enough for us but but B uh for a device to adopt this for a mobile device especially to adapters it takes a lot of validation testing um it takes and it's not just accuracy it's more the f-score uh so we want to make sure that not only no errors but there's no false positives or false negatives false positives are the problem they're what gets you because you don't unlock the phone for someone that isn't the five cylinder so we need to make sure that the recall and the Precision are up to standards and we're talking one in every maybe 10 or 100 000. so it's a long ways away all right yeah it's just thinking so this um uh implementation of uh wearing detection just by using this uh you know this feedback system actually striked me as being something that is uh relatively simple especially compared to detective heart rate and uh extremely valuable um it's very interesting do you have IP protection around what you've got yeah okay yes yes we do it's excellent cool yeah no I think yeah definitely the adding the in-ear detection with existing Hardware um there's yeah like I said there's the capacitive touch guys doing in-ear detection there's infrared doing inner detection and they're all taking real estate all taking power consumption and everybody knows you got this little tiny battery trying to do all these functions in your ear plus play your play your songs and take your phone calls um anything of the uh in terms of power consumption how that might differ with the sensor because you still need to be running some kind of an a deck or something the whole time right yeah you know I mean so I mean you're absolutely right um so so the solution really wouldn't make sense if it continues more power than LEDs right um maybe maybe not because you still save a lot of space by not having a sensor sure but um we really don't wanna could like so right now what we did was we bought we bought uh headphones or earbuds that sense your your heart rate from a shoe vendors a lot of them didn't work some of them are discontinued and the ones that did lasted for maybe half an hour to 45 minutes so so you can imagine the kind of the things have burned up the battery in that time yeah exactly yeah yeah exactly it's a bright light you know it's a really bright green light and uh that's what it takes I guess that kind of illustrates how long I exercise because I have some heart rate um I never went more than half an hour apparently yeah yeah oh nice which uh which if you don't mind me asking which one oh uh it was it was a a I I don't know I think I heart or something like this I'm trying to look across the room I got a stack of sample boxes over there but I could put it in the I could put it in the description here um after the video yeah but it's no longer in production but at any rate all right so you got any more questions um no I think we've covered it all right so people can what's the what's the URL to find out more about your company Muhammad sure it's uh I'll make dot a i o h m i c dot AI okay and uh yeah we'll put some information down below further to that for people to follow up and find out more information as this this uh I guess development project looks towards commercialization as kind of where you're at right now it sounds like sure uh thank you I really appreciate this and uh yeah uh like David said if anyone wants to know more uh just visit the website uh there's contact information there and yeah all right okay so thanks everybody for joining us today and thanks for watching everybody and please like subscribe share notification all those good things um so yeah thanks we'll see you guys next time bye bye thank you for your time see you next time
2022-09-29 20:50
