AWS re:Invent 2020: Accelerating the future of connected vehicles with AWS

hello everyone welcome to this session at reinvent 2020 super thrilled to have all of you here i am neil mitra principal solutions architect had been with aws for four years now based out of new jersey i help customers like you in architecting iot solutions using cloud native technologies very excited to have shazaan brand walla joining me today from denso to share a glimpse of their innovation in the automotive space now you are here i assume you are connected with the automotive industry in some way either you are part of an oem tier one partner mobility startup or a builder passionate about automotive technologies so by the end of this session you will learn how you can accelerate building automotive grade solutions using a connected vehicle framework that will help your organization to innovate faster and have an improved top line or bottom line so what's in the menu we will discuss what amazon been cooking in the automotive space specifically you will learn about a framework referred to as aws connected mobility solution and how it helps you to build automotive grid solutions then you will hear from our guest shazaan how denso have extended this framework to build custom use cases that matters to their business finally we'll show you a demo and provide actionable next steps please note this is a 300 level session so we will get technical at amazon we are continuously listening to you our customers and evolving our ecosystem of connected services across e-commerce streaming personal assistance with alexa transportation and logistics etc and all this experience gave us insight into the entire automotive value chain and emerging use cases as andy jassy says there is no compression algorithm for experience we had the opportunity to innovate on behalf of some of the most disruptive customers on the planet that you see here in their transformative journey and while working with all these customers we realized a set of common capabilities that every customer is trying to build on their own related to fleet provisioning fleet telematics fleet management which are referred to as mobility flywheel this is better thought of a cycle as your vehicle can continue to learn evolve be intelligent sheer insights throughout its life cycle just like our smartphones and does we build a framework referred to as connected mobility solution or cms i will use these terms interchangeably from here cms is open source modular customizable incorporates different design patterns and best practices from the age to the cloud our goal is to democratize adoption of connected vehicle technologies for anyone cms has built-in capabilities mentioned in the mobility flywheel it uses serverless technologies such as aws iot services for age and injection data lake services such as s3 kinesis application services such as lambda the platform apis are built on top of the services to help customers like you with a foundation and accelerate your time to market with different shooting capabilities for example you might be building an usage-based insurance app which differentiates your business so in the rest of the session i will explain different design patterns using this technology stack that is used as building blocks for the cms framework but before we get started there are two terms you should be familiar with first is electronic control unit or ecu an embedded system in automotive electronics that controls electrical systems or subsystems in a vehicle examples powertrain or brake control module the second term is telemedic control unit or tcu it's a type of ecu that deals with connectivity like gps wi-fi 5g etc a connected vehicle will have a tco that will collect from data from different ecu's via canvas and publish to cloud to help generate insights the other way to access canvas data is to use an after-market device which i refer to as on-board unit or obu the device can connect to odb2 port on the vehicle collect canvas data and publish to cloud in either ways you need to register the tcu or the obu with the cloud endpoint activate it associated to the end users so the patterns i refer to from here are valid for both this kind of devices so let's get started how it works in the assembly line the tcus which are embedded systems are flashed with the iot sdk such as free addons green graphs device sdks as the device wakes up for the first time it will bootstrap typically by providing a csr or certificate signing request generated by the tpm or the secure element on the device registration can happen in two ways device can talk to an api gateway over https invoke your lambda functions to register the device or through iot core over mqtt you might be thinking why do we need this lambda functions facade provisioning that is because there are different provisioning methods supported by iot core such as just in time bulk registration fleet provisioning and this micro services hosted on lambda makes the framework customizable and reduces device site changes so as an outcome of this process you have a thing registered with aws iit core representing the tcu the low privileged certificate that allows tls mutual art and iot policy that authorizes the tcu for different functions such as mqtt connect or publish the asset library function allows the metadata information of the tcu to be stored in iot device registry or in a graph database hosted in neptune so at this point your tcu thing is created in the cloud it has a serial number imei and it's tested in the assembly line to be working with the cloud endpoint now the fund begins tcu need to be installed on a vehicle so it goes to the required system testing quality testing and activated additional vehicle information is captured such as the different ecu's or domain controllers it's connected to and can be stored in the asset library as you can see in the picture so if you already have existing data models such as this you can leverage the templating capabilities of this framework and store the hierarchical structure in neptune iot device registry doesn't support similar migrations today at this stage tcu is capable of publishing telemetry from the canvas using its sim but broadcasting of telemetry is still disabled now an user let's say the vehicle owner purchases a vehicle from a dealership registers through an oem mobile app the workflow uses api gateway and cognito for the owner to complete the pairing process with the vehicle which in turn updates the asset library schema with the new owner vehicle association information an activation process kicks off asynchronously to update the tcu status in device shadow device shadow is a virtual state of the device on the cloud within iit core upon the vehicle detecting the state chain it generates a new csr to obtain a high privileged certificate from iot core once the vehicle gets a certificate the old one is deactivated at this stage the tcu has obtained a high privilege certificate and is fully functional for telemetry and command control the telemetry data can be processed through different workflows built within cms telemetry data collected from canvas can be decoded or deserialized at the age and published over mqtt are published as it is and transformed on the cloud we recommend mqtt since it's lightweight supports pop-up mechanism but if your device only supports http remember iot core do support https and web sockets so now the tcu is pushing telemetrics to the cloud iot core uses rules engine to route different data types to different backend services raw data stored in s3 which is your data lake your source of truth for analytics and long-term retention animal detection performed in near real time through random cut forest with kinesis analytics telemetry data dtc driver's core are pushed to an elastic search index for consumption through apis or user interfaces now the most hated word used by architects is probably well it depends and that's because there are always tradeoffs depending on the use case for example if you're processing millions of records and concerned about performance use kinesis for batching all kinds of data such as telemetry or ddcs prior to invoking lambda lambda can be transformed can be trans can transform the data within the stream if you invoke lambda directly you need to monitor concurrency issues or latency with cold starts if you're concerned about costs consider using time to leave options in dynamodb or clean your indexes periodically in elasticsearch after taking snapshots or use reserved instances for lambda plastic surge dynamodb and other services so the point i'm trying to make is customize the framework based on your specific needs for even processing the built-in workflow uses lambda and sms so events from different sources example dynamodb streams can generate alerts based on a subscriber's notification settings now trade-offs if you're concerned around performance or extensibility use a robust event orchestration service such as iit events no code approach integrates with iot core lets you build powerful state machine for processing millions of events by now you have provisioned your vehicles which is cool and collected free telemedics but how do you manage them at scale for example how do you find out all the vehicles requiring maintenance for a specific model in a geography that's where this framework has a built-in web application that is hosted on s3 and rendered through cloudfront the web application invokes apis to fetch the data from elasticsearch and enables different personals such as fleet manager with the entire fleet view data can be filtered based on location anomaly detection software version etc cool thing is the apis are also built with multi-tenancy in mind so different personnels can have different access control to the ui which are managed to accommodate a user pool trade-offs if you care about integrating your existing ui to this framework just use the apis and map your data models if you need to bring your own identity and access management system cognito supports different mechanisms such as watts saml etc or you can leverage the custom auth mechanism supported by different aws services like aws iit core visualization is cool but as a fleet manager you need to perform actions say all the vehicles with firmware v1 need to be upgraded to v2 so from this interface you can filter the vehicles to perform over-the-air upgrade over the ear upgrade uses iot jobs under the hood firmware is pushed from a trusted source as a pre-signed url concurrently to many vehicles iot jobs helps you with configuring deployment velocity in java boards retries timers notifications out of the box trade-offs again if you plan to bring your own ots solution we offer all these apis and microservices just test it out if your ots solution is interoperable bringing all these patterns together this is how the framework looks at the bottom you have all the core microservices which deals with the device life cycle activities we give it a name aws connected device framework or cdf and on top of that we built the automotive facade layer with mobility specific use cases and together they become connected mobility solutions ems we have built a solution in a decoupled way to reuse the core microservices aka cdf for different industrial solutions beyond automotive there is also an optional simulation engine which is not in the picture to help you quickly test out this solution with a virtual fleet but we don't want to give you just a reference architecture the entire solution will be available publicly very soon as a one-click cloud formation template how cool is that along with the source code to help you customize the solution with that please allow me to invite shazam to share with you what denso is doing with cms today shazam the stage is all yours thanks neil hi my name is shazam bahrain and i'm a senior manager at denso at our seattle innovation lab i'm here today to talk about how denso is building a new iot connected vehicle platform that'll allow the vehicle to run applications and process its own sensor and camera data on the edge this will allow you as a developer to innovate with new product ideas and models and add new vehicle capabilities without requiring a large amount of cloud storage connectivity or remote processing so let me talk to you today about the future of connected vehicles and make an analogy to the cell phone industry when you got an upgrade from a regular cell phone to a smartphone your world completely changed you went from a cell phone that was simple and not very intelligent to a multi-function super computer in your pocket smartphones allow you to add new functionality via apps and give developers and consumers a full computing platform that unlocks new innovation that's the change we want to make a denser with your vehicle we are building the platform that will enable the vehicle to run applications as innovative as the applications that run on your phone our platform will unlock the data available on your vehicle to applications that can be installed on your vehicle will give you the ability to process that data with a cpu and gpu give you ability to run machine learning inference in a flexible mobile architecture with this kind of platform capability we want to unlock rapid innovation in the mobility industry before i start let me give you a little background about denso and who we are denso was established in 1949 in japan we are the second largest global automotive parts company in the world current products you may be familiar with include thermal powertrain mobility electrification electronic systems last year we had 47.6 billion dollars in net sales and globally 170 000 people spread across 35 global regions for the future denso looks very closely at four core technologies to create new value and influence the future of ability areas such as electrification automated driving connected vehicles and factory automation it's through our work in connected vehicles that we've identified the need to allow edge computing in connected vehicles so let me give you a few real role examples of edge computing and what that means we look at applications such as car security usage-based insurance for insurance services location-based services for car security we want to give the vehicle the ability to detect dangerous events inside and outside the vehicle using the vehicle camera systems let you proactively alert users before something happens and capture a video proof of security events occurring this can all happen directly on the vehicle without having to upload camera data to the cloud for insurance purposes we want to give insurance providers the ability to base insurance risk on real-time driver behavior location usage and vehicle telemetry edge computing gives them the platform to do that this allows insurance companies to get to more precise risk measurement also gives them ability to get better pricing for their customers location-based services like real-time mapping or ad targeting logistics and last mile delivery services will gain the ability to actually run their applications directly on the vehicle unlocking business innovation for them so edge computing with a denser mobility iot solution has a couple of advantages number one speed and resiliency allows for applications that are very low latency and they don't require constant connectivity to the cloud edge computing applications are extremely cost effective they don't require transfer of large amounts of data to the cloud for storage or processing these applications are privacy focused private data never needs to leave the vehicle and be processed directly on the vehicle applications are updatable and can be modified over the life cycle of the vehicle all this together will unlock business innovation it allows developers and operators to grow new mobility business models so let me talk about our product mobility iit core and our mobility services score which is built with aws cms the mobility iit core is a hardware product and hardware reference design that can connect to the vehicle camera systems and the vehicle network called the vehicle canvas we give you the ability to easily read vehicle data vehicle camera data and give you a flexible architecture to run your applications directly on the vehicle we simplify the process of running applications and getting access to data for you this in turn works with our backend called the mobility services core we've built this on aws cms aws cms gives us out of the block access to things like identity access management platform apis application services and data services we then go ahead and add additional capabilities on top of that including digital twin capabilities mobility gateway as well as anomaly detection video on demand and fleet management all this is based with a mobile architecture and flexible architecture this allows us to build an automotive grade mobility solution that can be powered for our customers it lets them build a custom solution on top of that let me give you a few examples of what our hardware platform actually looks like right now like i mentioned it's a hardware reference design it's completely extensible this platform is modifiable based on customer requirements we're making it available first as an aftermarket oem product and we're working to integrate this into new and existing vehicle ecu's the system will be able to connect to the vehicle camera systems and the vehicle can bus we'll make available a cpu and gpu give you the ability to run machine learning inference and give you the ability to update your apps over time so next i'm going to show you a real example of an application we've built running on our platform showing how we can do use edge computing to actually detect security camera events and combine that with the power of the cms system in the demo i'm going to show you a very common scenario you have a vehicle parked in the parking lot and somebody's backing up and they're now going to cause a collision with your vehicle this happens all the time in this scenario a collision occurs the person driving the offending vehicle gets out to go check what happened they still have caused some damage decide they don't want to deal with this they get back in their vehicle and drive away causing hit and run today's world this happens all the time you're out of luck most of the time with the mobility iit core platform a security camera application can constantly continuously record camera and watch the camera it can detect when a vehicle is backing up and about to cause a collision it can detect when a collision occurs it now captures its event data packages up information about the event including a video clip about the event and automatically uploads this to the cloud this is all happening on the edge and only when an event occurs is actually transfer data off the vehicle so now i'll show you what this looks like on our back end we see the event occurring we see when it occurred where it occurred and what the cause of the event was we then also show you a video clip of the event actually occurring so now we hear it's a short little clip the vehicle is backing up the vehicle will cause a collision and then we have proof of all that's happening our architecture is completely event driven so from here we can take it actually one step further and complete the scenario when an event occurs the system can actually activate a live video stream from the vehicle this happens on demand only when needed so we're not streaming continuously when you activate a live video stream we can now actually see that the event has occurred we can see somebody getting out of the vehicle seeing that they are seeing the damage and not doing anything about it getting back into their vehicle and driving away so this is a real example of how we can use edge computing to detect the event capture relevant data about the event actually occurring and add additional capabilities on top of that to see additional capabilities so i'd like to talk about how we actually built an application like this using our platform so we have a mobility iit core device in the vehicle right now and this again capable of reading the vehicle camera systems we take that vehicle camera stream and break it up into three distinct video streams to allow us to do multiple services directly on top of that we take one video stream that we've broken up using gstreamer and make that available to a security camera application running on the platform this application is capable of taking a video stream and combining that data with vehicle telemetry coming up the vehicle canvas this application will detect meaningful event data when a collision is about to occur or when a collision has occurred and then take this event data and send it over mqtt to an iot rule in our mobility services core this iot rule will then write this event data to an event database since it's all event driven the same application can take that event and cause an alert on the vehicle to actually have the vehicle honk its hormones detecting a preemptive collision and try and stop the collision even occurring you'll also send an event now to a video click capture service another video stream is continuously capturing video clips when requested it will capture a video clip when requested and upload that video clip straight into an s3 video bucket storage we then use an api gateway to combine s3 video storage with event data and make that all available in our user interface so to complete the scenario we also want to be able to enable a live stream when an event actually occurs so the first thing we do here now is we activate kinesis video and create a webrtc signaling channel on our back end next we use iot shadow to send information about the signaling channel back down to our devices and we also use that same signaling channel to activate the webrtc sdk on our device it's really important for us to not continuously stream data when if it's not requested so we activate webrtc sdk we get signaling channel information and then the user interface will use the signaling channel to negotiate a video stream and make a peer-to-peer connection and enable a live video stream from the vehicle so here's a full example of how we're taking we'll take a video stream from the vehicle break it up into detections capture video clip information and also enable live streaming all the processing and data collection is happening on the vehicle not in the cloud and only meaningful event data is getting uploaded to the cloud so with that i'd like to talk a little bit about lessons learned as we went through this process utilizing the cms reference architecture significantly accelerated our development time we got to speed the first prototype very quickly using aws tools allow for easy trial and experimentation our developers learned by building and most importantly a close collaboration with aws allow for quick problem solving and help build new solutions for us very very quickly and with that i'd like to hand it back to neil thank you shazam very insightful so you learned a lot in last 30 minutes let's summarize we started with what amazon been cooking for the auto industry and how we learned about mobility flywheel a set of common capabilities in the life cycle of a vehicle related to fleet provisioning fleet telemedics and treatment management and how cms can support those out of the box cms is pay as you go open source modular customizable going to be available as a one-click deployment to cloud formation and then you heard from shazam how denso accelerated their time to market using cms as a foundation and on top of that they built advanced video use cases to deliver an automotive great solution that matters to their customers there is one more thing i'm very happy to share with you we are bringing live video capabilities to cms in 2021 stay tuned if you're hungry to learn more about other automotive journeys on aws check out the sessions from bmw toyota accenture with that i greatly appreciate you joining us today remember the journey of thousand miles begins with a single step and we are here to help you take that step to help you reinvent the future of connected vehicles thank you

2021-02-10 19:18

Other news