DeviceScript - TypeScript for Microelectronics with Peli de Halleux

hey folks welcome back to niky live I'm your host Nick Taylor and today I'm hanging out with P the uh P Hi how are you doing today hello how you doing Nick and you did meure my name that's that's really good yeah well I I I hadn't spoken to you obviously until just a few minutes ago but uh I'm assuming you're from France uh I'm guessing uh I'm from the French part of Belgium yeah oh Belgium okay yeah yeah I can tell I I'm from Montreal so yeah yeah no I'm in Montreal so I'm bilingual so I've been speaking French since uh Primary School um so but but I could you can always tell uh CU people in Quebec they have a very kind of coarse accent and more like abrasive not I don't mean that in a negative way but like when I speak to somebody from France or Belgium it's it's like a softer accent you know uh so uh plus your I think your your last name kind of gave it away and uh so but anyways um for folks who might not know who you are would you mind just kind of giving a little explainer of who uh Pelle is all right so yeah so my name is p that's my nickname my name is Jonathan but um I uh I am a developer in Microsoft research I've been there for a long time I have to do the math to figure out um and I've been my team has been working for a number of years in U simplified programming environments for kids uh if you heard about make code make code is a graphical slash typescript environment for kids to learn coding we're talking K12 here um and um so we we spearhead that and we work with the product um to do that um so before that we had a little stin where we did uh coding on the phone this what's called touch develop you know back in the early days of of smartphones like yeah everybody's gonna want a code on their phone you know so we've been kind of dabbling into and our group has been kind of working in in this space and obviously we build a lot we use a lot of web Technologies and we love TP uh we use a lot um yeah all the stuff we build is typically web based no it's super cool it's um there's uh I got one of my daughters a microbit controller a few years ago um and correct me if I'm wrong but I I remember hearing about this because the UK education system gave these out to I think every student in it was either primary or secondary school I can't remember yeah there you go and yeah they they gave it to year 11 uh year seven I think which was uh 11 year olds and they yeah they they tried to find them all and give them all yeah I thought it was this was the early launch and they've they've sold quite a bit since then and in fact it's the second Hardware it's a second revision of the hardware okay um so they added also a microphone and a speaker so overall it packs it packs a lot of goodies um yeah we could we could talk for a long time about the microbit um it's really you know for people to look at it this this thing it's credit card half a credit card and fully programmable in a number of languages no these are LEDs yeah this guy's powered you can see it has yeah okay you got a few there um yeah and that one of the things I found really neat and approachable about this and I haven't we're going to talk about device script uh in a bit obviously um all these all these iot or or or little micro Electronics most of them seem to have emulators like uh which which I think is super cool because you can I mean in terms of being accessible because it's on you were saying everything's web based so like I remember from microbit they had their online editor so like even if you didn't have your device you could still have fun and you could see the little emulator light up or do whatever it's supposed to do I mean I think the only limitations for some of these things is maybe like I don't know like obviously detecting weather or something because it's it's an emulator perhaps but maybe even it does that I don't know but I I just found it really really well done and and just just a really great experience like even if not just for a child but like I think in general like anybody that just is interested in it so it's super cool and and I guess I'm curious like for device script uh is is the experience similar or or how did you go about approaching device script yeah I simulation is key and and it's definitely an important piece that we're bringing with thece script um you know think about schools uh they might not have the the budget to buy a device per kid or yeah uh there's always some quirks about you know connecting Hardware you know just just teaching kids how to put a USB cable into the right uh into the right spot with without destroying things so the simulation gives you you know the web based simulator gives you kind of this instant it just works effect uh okay blocker um and schools you know we went for the web historically because it's all over the place when you think about schools it's really hard to do a native app um you know think about the worst it school is like an it where every user is trying to destroy the system uh yeah yeah so so web base is it just works and it's easy to install the teacher doesn't have to ask you know and wait for a year for an IT admin to install this the stuff so so simulation is absolutely key we see that in indication that's why the viscript has full simulation so when you think about and in fact this is actually we can talk about it super cool we can dive into the details but we have an A firmware that's you know embed it firw where these days it's C or C++ you know Russ is showing its neck but it's still C C++ so we have a csdk uh firmware and we recompiled that uh to web assembly okay and then we run that in browser y it's cool I I have a question about that are you using assembly script which is typescript for wasum no we're using um mcrip them or okay yeah yeah the the standard well whatever standard is yeah okay uh yeah and there's uh my friend Andy's in the chat uh they're from the UK they're a big fan of iot and all these kinds of things we're talking about and uh they were just mentioning that like we talked about previously the micro bits were handed out in the UK but they've been handed out several times it wasn't just like a one-time thing so I yeah yeah it's I think it's such a great initiative and and to point about things being web based I think the web levels the playing Ground to make things so much more accessible to everybody because like you know I'm obviously very privileged I'm working full-time as a programmer I've got like a souped up MacBook at home uh you know but you know I've I've been tinkering on my iPad you know using Code spaces and so on you know just to see how it hurts my brain a bit because I still I I don't know what you call this effect but I I still enjoy the local development environment and there's I'm not sure what it is that's one day is just going to make me be like it's totally fine in the browser but there's still there's some kind of hesitation that doesn't keep me in there and I don't know what it is honestly I think it's the color scheme of GitHub Dev you know versus it's really they have a slightly different color scheme just your eyes are used to versus running it locally yeah I mean the boundary is just shrinking you know as we yeah and all these text like stack Blitz and code sand box and all these things make it uh especially if you start moving between machines I mean these and in fact we've done quite a bit of work in device scrip to support this containerized U development environment which which leads to some interesting challenges because you do have to talk to Hardware it's like you can yeah your container is somewhere you know in a cloud but your Hardware is on your desk um yeah no yeah no it's it's super cool uh it like just like I've used I've definitely used uh stack blits and code sandbox and and GitHub code spaces there's git pod which is a similar experience and it's I found it really amazing as well because I used to work at uh I don't know if you know the website Dev 2 but I used to work there and it's a open source project and it's a rubyan rails monolith it's it's got reddis you got a postgress databas there's a bunch of things running and I used to live stream with people that were interested in contributing to the project but I used to have to spend like a few hours with them even though we had really great instructions I had to pair with them to get it all set up and then we got to a point where uh it was using git pod but uh somebody had contributed git pod uh as an environment to the the repository and it wasn't really working perfectly but I resuscitated it and basically I went from pairing with somebody for about three hours to they just spined it up in 10 minutes and they were ready to go in and that's that's huge like and that taking a coffee while everything was downloading and yeah exactly we do internally too I mean you know Microsoft obviously we have huge projects and whenever we can at least in in our team you know we'll we'll drop a diff container file in there yeah so that you can spin up a code sandbox and on board a colleague who needs to do a small change oh yeah I mean it's a game changer everywhere yeah yeah and and and like I said we're definitely going to dig into device scripts soon but I'm kind of curious with the emulator so so like obviously you can have the editor environment and then you can have like an image with like interactivity on it like lights loading up or maybe writing text on it and stuff but do you like I'm assuming you map one to one the actual instructions that go into the actual device like or or are you just kind of stubbing all the calls and just like uh I'm just kind of curious how the emulator Works to some degree I mean high level I I get it um but kind of curious about that let's see see if again I can do a live demo here um yeah all right so so you can see here so the way this works in device script there is a service client architecture so okay assume that every every sensor is a server it serves sensor data okay just like you're connecting to a web API um you're G to have you know a server of temperature data a server of humidity data or a server that actually controls your LEDs these are LEDs here for example you know or at the barebone a button is a server yeah and it emits events saying I'm down I'm up I'm down and up okay so we have a we have a protocol with you know a a set of a way to encode this into packets so a binary blob that goes over on the hardware side it goes over a data line so it's encoded using it's you on top of Serial so there is some Hardware that takes care of sending the packet U but because it's a packet we can also send it all over the place through websockets and uh whatnot so once it gets into the web we have a library that lets us take this binary blob and decode it and and look at a pack oh you know the the button just told us that um we the button just told us that we are we are down so now this is what this enables so you have here my Hardware uh and you know I have a slider here and you can see that the slider change is being reflected on the on the simulator here on the left right you can see the slider here so what's going on is this slider is sending packets on the jack do uh bus so it's saying it's sending basically when the position change it updates the position then the microbit which is connected to my computer sends that over USB to uh to this web page and I'm connected to the device using web USB and we can talk about web USB and web serial which are okay super awesome uh but basically web USB gives me an Access to this Hardware device and then there's a react library that you know then there's a device twin framework that's a that kind of maintains the state of the slider and so this is a real device but then talking about simulation I didn't have that slider yeah spin one up uh now I have this virtual slider and the difference is this guy has a you know I can click on the slider and then it moves left and right yeah so that's um the simulation and usually it's called Hardware abstraction layer in in embeda devices it's like how do you separate your application code from the real Hardware like how do you talk to Hardware um and our abstraction layer is this uh system called Jack that that we build for the microbit that exposes everything as Services you'll see when we code you know you're not talking to a specific instance of the of a temperature sensor okay you're talking to a temperature sensor in general yeah okay gotcha and then there's an abstraction layer then there's a driver that kind of works for every specific Hardware so that's okay in fact this view you'll see is the view that actually is loaded in device code okay we do a web View and then we load this View and then we just do some some message passing okay so that's yeah oh sorry I was gonna say in in regards to the hardware abstraction layer I guess that allows you to like obviously talk to the actual Hardware when it's the actual device but because it's an abstraction that's where you can mock or simulate uh like whether that's testing or or just like using the the simulator here I imagine so that that's a that's a really nice uh architecture so here you see that my simulator is talking to the hardware see how it's changing the LEDs yeah because the message is sent across the bus so it um it opens a whole bunch of fun uh cool kind of mixed simulation SL Hardware scenarios uh yeah you know this is reality so if you have to test if your system is going to behave in in a minus 10 degree weather you probably want to simulate it I you want to test it for real but at the early stage when you're prototyping you probably don't want to you know so there's uh simulation is is really really important um in our in our prototyping phases and and I guess also because you have the hardware abstraction layer you can you can test out new apis and calls before you actually physically implement it or or is that something you do too or you can actually write the code before you even have a driver um yeah okay which you know is questionable but uh definitely running against a simulator is much safer you're not you're not going to send your Servo crashing into something else so yeah there's uh We've we've dabbled with testing as well where you can basically some do some record and replay so maybe record uh a sensor in the while and get the data and then inject that back into your system um cool cool the in fact we use the kind of abuse it because some some system becomes interesting we have a debugging service so the debugger is implemented a service so the the debugger you know break points command and all this stuff are shuffled so the whole Chrome debugger protocol is shuffled through the protocol the the flashing you you can Flash the the Ser you know once you have Services things get easy because you basically have a binary format to send small packets um just as a you know as a there is an interesting thing about this protocol is that it has to be compatible with very cheap chips that run 8bit mcus okay um you know they cost the button for example we see here runs a pedal which costs around 4 Cent I think uh but it only has 64 bytes of usable memory uh so everything to be small yeah yeah and uh just GNA mention in the chat there's the the some of the folks are saying yeah this is really cool and uh such a great Rabbit Hole uh bumpy lumps loves microcontrollers and microprocessors and it's I have to admit myself I haven't dug into microcontrollers too much yet or iot stuff it's something I it's something I want to do I just get I get sidetracked with you know like life kids and everything else but but it's uh a Andy who's in the chat uh they were on the stream I think it was earlier this year uh yeah I think it was summertime and just they love all these things so they had uh you know like weather sensors just fun even fun little projects you know like it doesn't necessarily have to be something that saves the world from something right um yeah he Andy says I get sidetracked by every new micro microcontroller that comes out so uh okay Shameless plug if you really want to have something uh weird BR your way this is this is actually full coded it's running on the microbit oh no way you know kind of can get it so like a little like a not a Game Boy but a a very very tiny game yeah my my image is not zoomed enough but yeah it has a full tile editor and you can program things on it I think I programmed it to play to show a smiley when um which this one is actually written in make code arcade uh okay yeah I mean the microcontroller world is just you know you're trying to keep up with the web Technologies of the typescript the backand Technologies and that's just another it's just another fire hose you would have to swallow uh because obviously things are changing and you know changing fast there yeah um so yeah I don't I don't from not having 72 hours per day look at everything uh traditionally embedded has been more of a c C++ you know electrical engineer World um definitely over the you know starting with microbit actually micro python started uh they brought in a full python interpreter on embedded and there's also there's also an effort by Ada fruit called circuit python that's that's become very popular in terms of you know I think you've heard about Arduino which is kind of the yeah very popular way to to kind of democratize doing embedded and then micropython kind of came in and really brought in this kind of script experience and python is experience but U with Device GP we really wanted to optimize for people who do type typescript day in day out and work in yeah yeah no I think that's it's it's super cool that way because like like I mean obviously JavaScript started on the client side well actually I was working in because I used to do a lot of C asp.net and all that like earlier on in my well not earlier on in my career but like the first two-thirds of my career so far were in net and and people talk about node.js but I remember doing serverside JavaScript in ASP back in with jsharp uh or jscript sorry back in like 2002 so it's and the only reason why I bring it up is because uh like it's a l javasript a language like regardless of what people say I've it's just always fascinated me like uh I I remember the first time I built a web page and notepad you know literally typing the HTML the font tag and all that stuff and and seeing it for the first time on the internet like you know you ftpd it to your ISP where you had storage and stuff and and that that's kind of what kickstarted like the fascination with web for me and and I love that so many things you know like like javascript's obviously used on the server side now there's like all kinds of other like typescript took off um and and just just everything being I mean obviously not everything has to be web based but it it gets back to what I was saying before where it just makes things so accessible to so many people potentially because it's just the web you know uh like you can literally still go in notepad and if you can FTP or or or however you want to push up your code you can still do that you know like and it it that's just always fascinated me um but I did have a question about so you were talking about debug because like uh for debugging with typescripts so like like if you're in vs code well if we talk about nodejs for example which which typescript would be running if you're doing uh debugging on in node.js but like you have the V8 debugger and there's like the like are you using the Chrome Remote debugging protocol or how are you well we Implement a debugger in vs code so okay okay it's a language server uh uh yeah okay okay gotcha okay you know we t into the contribution point there and then we um we just implemented from the sample the debugger okay it happens that we Shuffle the messages through our service architecture okay okay yeah it you know debugging in embedded is weird because it's very heavy on timings timings are important when you measure physical things like you know you measure a car between two I don't know infrared nodes and well breaking in debugger kind of destroys that so yeah so but it does have this you know when you when you crash and you can get a stack trace and you can look at the stack trace and look at the you know variables that's that's a lot of uh that's really valuable it so what I mean is stepping will get you somewhere but it might get you place where it's far from reality it debugging a lot of bugging is still you know good console log um yeah I feel that like because even in web development you know when it because it's event driven you know like you click or you hover and then the hover fires something and if you debug that you know there there's things in Chrome where you can like set it on Hover and stuff but it but like it's like you said the event has happened or or like it everything gets out of sync but uh but it's still helpful like you said cuz if you if you hit the debugger you can look at the stack trace and and the the stack Trace is pretty cool I don't know when they changed this in V8 but like you can go back in time and replay now like you they did that at some point like because you not you weren't able to do that before uh like obviously like if you've written to a database or something that then it's that's something but you it's still kind of neat that you can go kind of time travel to some degree you know yeah I think that's probably the the name of of that feature it's been it's been working on time travel debugging as being kind of you know a very cool topic you know and what you touch on events I think one thing to to think about embedded is it's just a very small app like if you're building websites uh just think about embedded are being smaller way smaller in terms of resource but they are highly reactive systems so just like the website are you know driven by the users clicking and things happening and embedded system is a reactive system it it has sensors that trigger it has you know I don't know a window open detector or a motion detector that triggers and that starts your your code so it has the same you you could say oh it's just a small app you know if you're building desktop apps with you know with typescript today and doing electron or you know whatnot or react native it's just the same much smaller and yeah I mean you care about battery and things like that but uh um and also I think these days you don't have to be you know a hardware wizard a lot of sensors you can get a long way in terms of prototyping with of the Shelf breakouts that you just connect with cables there's not even soldering needed uh there's a whole different story around productization and you know production at scale and all this stuff which yeah then things get serious and uh there's a different skill set involved but from the protyping P phase you know you come in with your typescript knowledge uh the confidence that you can build a small up uh and then then you can do some cool things iot apps actually in many ways very simple you know think about a thermostat it's like yeah it's cold turn on the furnace it's okay pod turn up the furnace uh yeah no yeah no because you think these things are like you know some super complicated device but you are right it's just really like yeah okay hot turn it down or turn it up you know there's some de balancing to make sure you know you're not turning on and off the fre the the furnace at a high frequency and so forth but yeah in general it it might be might surprise you that things are a bit easier than they look yeah so yeah um how do we do this yeah so well what we can do is um do you like you want you could kind of guide me like do you want me to load up code spaces or vs code and okay so what we can do is because I know you had camera problems what I'm going to do is I'm G to copy your share screen if you want to just go back to your uh just yeah the view you have right now I'm just going to move you to where I usually have the guest camera so just give me one second I was thinking I would do the demo because I have the hardware I'm trying to figure out if I can do tab okay uh uh you should be able to um I believe um if not do you want to share your whole screen view oh yeah it has split view yeah there we go okay uh yeah start a repo are you in as well oh no oh edges has split view too now okay I didn't know that my my my my little web page on the side doesn't like that let's do some uh if I can get some camera no it's really not happy about that okay yeah there's if I go in this mode no oh it's confused now well well if you want I can still you go ahead and share your screen and I'll pop you into the side and uh I'll I'll figure something out um but yeah just go ahead and share your whole screen or or your browser like you were doing yeah just like that you gonna my my image I guess yeah I could do that or I'll figure something out worst case people have already seen your face but yeah you've seen my face just create a little repo I'll go find an image of you while you're doing that right so I've got a new repo in fact I mean this this is why we've done all this work around um being able to work remote um yeah I'm gonna start codespace here so I just created a new repo and I'm starting the code space me zoom in maybe yeah if you zoom in like U yeah maybe two two or three times yeah that should good yeah that's perfect so I've got this what what happened what did I do I dragged the wrong thing okay I was trying to drag this one uh what I'm gonna do is I'm gonna install device script device script is uh is published as a V Code extension and it also then takes care of installing the tools and so forth so once I have that you can see that there's a new view here and so if I click on The View there's this convenient create new project and I'm just going to say go let's use npm so what this does is I already messed up my my layout oh it's oh uh oh yeah that's weird I've never seen the output on the side it's usually go back go back here yes okay tonight I learned I could do that so what it did it generated a bunch of files and um you know kind of a the usual node thing where you have a million files you know but the one that's interesting is main.ts um and it kind of looks like typescript it is typescript you can see here that it started a server so what it does it it's running on the codespace the tooling okay tooling is running on codespace you can see here there's a server that serves that simulation view so if I want to run this you know I'm going to go here and click run we have our own little run button here um it is going to spin up it is going to show two views so one is a simulation view so the simulators is the same view you saw previously where I can spin things up so you know I can do slider here for example if I one um that's that's the view of this kind of virtual server it has there's a lot of buttons and you know all web design but basically there's there's a WiFi driver here there's a virtual Wi-Fi so because the the typical device you know accesses the web you can see here this this tree has a view of the service and we talk about services so these are all the services running on the brain of your iot solution so it has this controls this internal service role manager assigns what you know we can talk about roles roles two servers so you're going to say oh this button is going to be a in my program and I'm going to talk to it so there's you know if you think about having multiple keyboards or mouses you always have the issue who's who okay it's the same here you know they have five different sensors who's the sensor in in my room and and not then there's these device script manager and debugger that's actually supporting the developer experience settings stores secrets on the device and then we have our Cloud kind of virtualization but um what we can do is oh yeah the last thing is it's running so when say VM this is the web assemblied so our firmware in cile with mcrip and running as a web assembly web worker um on the Cod space this is all running on the codespace okay and you can see here you know if I do instead of this I'm going to do control dot no I'm going to do Millies M and because we do typescript we leverage the whole beautiful develop experience so we we actually sent we compile locally in well I don't know where the let me think I think the extension runs also remote side anyway we compil typescript we compil to a a compact B code the reason is we don't have the space on the device to run typescript so we compile to a bite code but it's still pretty much a VM for it's a JavaScript VM it's really device script we love typescript but actually we run JavaScript on device you know when typescript is done with the compilation there's no more typ check it's yeah yeah exactly yeah yeah that's yeah that it's it's always uh the the old way of saying it was transpiled but I guess compiled is what people say these days but but is is all the places where like as like we're talking about this project too but I'm just curious in general with typescript because like even like for example Dino which is a a a new JavaScript run time it's still running V8 under the hood so JavaScript at the end of the day they strip the types and so on is there is there any that runs typ script that doesn't compile down to JavaScript I I would guess no but but just um I don't know if Bund that's something different which is the latest the lat I don't know if you [Music] heard I have no idea so this we have our own interpreter here so we have a we have our own U JavaScript interpreter that is part of our runtime that runs the embedded device that is a subset of it doesn't have the same optimization as V8 because we're not fighting for the same things we have you know on on an iot machine you basically have pretty good compute these chips are fast but you absolutely have no we have limited code space you know starting like maybe I think we need at least 120k with the web stack and then you have very limited Heap because the TLs stack will already eat it all and then and then you're left with with crumbles but you know think about tens of kilobytes yeah I think we need 64 minimum um now the good news is you run you can run forever on batteries right so that's the tradeoff that you're hitting with with these with iot is that you want to run on smaller chips which are CH cheaper and use less batteries because batter is is a big deal if you put a sensor that's supposed to last five years in a wall you know better be careful with the battery yeah uh just uh I'll interrupt you every now and then just because some folks are in the chat uh they they're saying this is a super fascinating use of typescript uh NX 618 not sure who that is but um but yeah I definitely agree and and to your point about the battery power I totally feel that because like I have a eyewatch and uh I have I got this one it's older it's like a eyewatch five but I have to charge it like every day and a half which is terrible it's it's an amazing device but you know it's like I I'm waiting for the day when they say like okay we we updated iOS uh iatch Os or whatever it's called uh you know and now your battery will run six days you know like it's a huge thing for sure if yeah the GPS stack I mean they're already pretty efficient powerwise but the watch does so much stuff that it's not your it's not your typical iot device that that is very yeah I know for sure yeah yeah so you know typescript is so interesting because again this is similar to writing a web page we have you know when when I say toggle what this really means is that I am sending I am talking to Hardware ultimately so I'm either going to be setting U A Serial I'm me talking via serial line or iore C i2c line or SPI these are all protocols that are common in Hardware or you know toggle toggle pins directly but it is a synchronous most of the time yeah so everything you do in in um in embedded it's always kind of a synchronous and you have to deal with interrupts and it's highly concurrent so in general writing embedded is very hard because you've got concurrency interrupts and then you can't break as soon as you break in the debugger you've changed everything the well change uh so you've got highs and bugs and so forth so this is where for example the threading model you know the execution runtime model of JavaScript where you have you know hey you got one one thread running yeah uh and we actually enforce a weight like we enforce a way because we don't have promises we have something lower level but it looks like it looks like modern tcri the only thing is you cannot do then there's no then okay okay no that's that's super interesting because yeah you've got async A8 syntax there and I did assume that was a promise but and also you can't not away that makes total sense because you want everything to be asynchronous so yeah and I mean spinning uh there's a way to spin uh workers so basically to leak promises you just have to careful this allocates memory you might run out of um but yeah so so weight is kind of but a weight is this amazing thing because it fits 100% with embedded you know when you do that in web pages you have to think oh you know I'm doing a database a request it's going to take a while then come back you know if you do some sensors will take I don't know 20 seconds before they give you a reading so yeah definitely a Big W there um so so the language is actually very very well suited for for writing embedded code you know at l l all right so to continue with the demo so I've I've spinned up a light bulbs uh client and you can see it's not running anything right now because it's basically waiting for a light bulb there's no light bulb in the devices so we've detected we've detected that you actually need so the light bulb is a rooll and think you know think this game and we need a light bulb to run the program so I spin up a light bulb and now suddenly it's running and in fact it's toggling that's amazing and this is all running so the simulator is running in this web page this guy is running on the I guess on the codespace VM uh and I'm still in my browser at home yeah running all this all right so let's do Hardware yeah uh there was uh just a quick question from the chat uh uh I I don't know how often you live stream but uh there's always questions coming in so um uh bumpy lumps is is Alex is wondering uh they'd love to see more about how they modify promises to get that behavior with Hardware uh he's going to be looking up this himself but but I am curious like so like like you said there's no there's nothing there's no venables in here so I I think one of the cool things is because like normally when you've got typescript like for for webdev you've got like the lib DOD uh types and stuff so I imagine you just you change what those base types are so you could like you'll just yeah we don't we don't load we don't load Li up D because we don't have a yeah that's yeah yeah but I mean like what I mean is uh you're you're just defining you're you're overriding some types or removing some types from the existing types that you would usually think would be available we're mostly starting from zero okay uh you know this is our cor lib let me zoom in you guys can see that uh in in fact there's a huge amount of of libraries that are missing you think about JavaScript uh just because we haven't had the time to do it but this is you know if you go and review here you'll see you know we've got kind of the what we thought were Essentials arrays objects you know strings yeah so that's uh and then a lot of them are implemented in C C++ okay okay um and some of them are implemented in device script itself okay so when they're lowlevel and they need to be optimized they're basically part of the crun time and that's hard to implement but for example let's say you're missing the a URL or a form Creator class or something typically you should be able to write that directly in device script now here's here's why it's so it's better to write things in device script is tree shaking now remember space matters so everything that's in your cun time because the way we compile it it gets added to your firmware and you're losing space but the nice thing about the typescript typescript does tree shaking and tree shaking means it gets rid of all the code you don't use yeah so that's a nice nice uh benefit of using the language and and that's very interesting for embedded because you have limited space so let's you load all kind of libraries but then you run out of space well start using this features and then um but yeah so so our library set is is not complete but then again we're not building web pages so so there are things that just don't exist don't make sense I'd love to talk about the Dom and we can talk also about uh react and all this stuff because there's there's some really you know people want to lose weekends geeking out there there's there's opportunities um yeah no I'm assuming you're referring to the simulator uh with all the react and yeah well okay we do support TSX so at the end of the day react TSX you know I think about TSX files it's just a compiler trick that that takes these kind of XML syntax in the middle of your JavaScript and produces JavaScript that uh that a few functions and then reacts is one main use of that um but this is a r hole so we can talk about later um what did I want to mention yeah let's Okay so this demo um this is where it gets actually quite fun um if I want to connect to a device remember this is running in my browser so who do I have okay I'm gonna connect do you mean a actual device now not yeah okay you don't have my camera which is sad I'm connecting Riser P I might go back to my camera we have to figure that out uh but we can look at the mechanics so there's this thing that has been added and it's been around now for a while and it's kind of everywhere mostly it's web USB recently web serial yeah uh these are Game Changer to do Hardware uh development online Hardware development because in the past you would not have access to Hardware yeah and web USB gives you this beautiful crossplatform way to talk to Hardware so what I mean is if I do connect and in this case I'm going to go ahead and connect did I do serial uh let's see uh yeah let's do connect again I have a page that page is talking to the web server running on my codespace I'm going to say connect via serial and I'm connecting to a raspberry piip Pico you can see found one I think if uh you have your OBS uh you have your share screen now but if you want to put your camera on to show your device I can have it on side so uh if you want um oh wait I need to unplug that I need to unplug that camera it's on the oh oh actually yeah that's the right camera yeah okay okay okay okay okay um what was I connecting yes uh the uh yeah something like let's go let's go something more iot is uh this is a ESP 32 here C3 so it's aiao from from seed Studios and then there's a shield but don't worry about that uh I'm going to go and connect this using Cal this is super cool it is I'm glad we can get the I'm glad we can show the hardware on the side here so it is it is not connecting for some um of course it doesn't connect when I'm doing a demo yeah yeah well that's all good that's that's one thing I love about uh streaming is because it's not a perfect YouTube video so it's always expected that things won't be perfect but hold on um Let me let close anything that's that's that's eating up the connection all right it's one more time otherwise we're gonna have to oh there we go so it is connected and now you can see that this is my guy it has a little video oh okay so there's this thing about tabs so I have to untab the connection because you know how the browser kills the other tabs yeah I also encodes I think what what one of the side note when I was talking about how I I still kind of default to local development environment is so many times I've pressed command W or control W to clo to close a tab and it closes the browser yeah so I I've un tabed the thing so it's running uh okay I'm looking at the real device so this device is here it has a this is a BME 680 it's a temperature humidity sensor pretty good one you can see here I'm just reading the temperature in my room you know 20 degre C I'm in my garage yeah I I just want to pop pause for one sec I just want people who are checking out the stream to appreciate this p is in in code spaces I.E a web

browser has a Dei physical device connected that's being read through web USB and everything's working like this is this is like blowing my mind yeah yeah and this is I mean the web is amazing and again this is enabled by what we got webs so we've got a you know is server running on the codespace websockets going to this page which is connected via web USB a web serial yeah the reason I do have to do it here because as a this feature is not really enabled yet by um by vs code there's there's some experimental stuff for web serial web USB but it's not a mainstream API so it's uh so this one talks with about websocket to my server and then the packets so Hardware talks to this web do you see that web page oh no I'm not sharing it okay so anyway so the hardware talks to my web page that has a connection to web serial which is then sent by a websocket to the server then redistribute it to all everybody's listening including my add-on in vs code so that's how I see these and you can see this is live so yeah I don't know where this is just super cool temperature here yeah temperature I know if I blow on this imunity changes see it's real thing so so we've confirmed p is not cold blooded yeah kind of a toasty toasty 20 degree temperature nice all right so I can program this device now so if I go back to main um let's forget the light bulb for now uh so what do I have I have a temperature so what I'm going to do is because I'm I'm using this ciao I'm going to configure my device you can see it already knows what's connected so it added it added the pin mapping so it it basically bloated the hardware configuration for this device I can still simulate uh and now the next thing is I'm gonna do a I'm gonna need a temperature sensor and can see here this is the one I need it's a bmeu 680 um so oops s ke so let's let's keep this so I've got I started my driver and now what can I do well I can you know temperature is a client so there's a reading register and I can read it now it's not happy because read of course is a promise so I have to wait it now I can oh by the way we added because we can because we don't know we added data as a as a built-in to um just because it makes our life easier so I'm going to hit run really hope that deploy it deployed now I did change a hardware stuff so I think my device restarted so I need to reconnect but it's a onetime thing okay it's back let's get rid of that um come on okay let's deploy one more time it's got to be super cool working on all this stuff not just the iot stuff but like like VSS code in the cloud like it just just it's just got to be super neat the way all this stuff is interconnected and while you're doing that uh I did have a question about that device that you're destructuring the properties from when people I I guess because you have a certain amount of devices that are supported so far do you create do you create the types for each of those devices I guess or do the manufacturers of those do that example like or users right yeah and that's that's where in fact that's where things get interesting is that yeah there's a lot of devices and yeah you have to write drivers for them I these drivers are fun and of course it'd be great if the demo worked but let me let me power cycle this guy and let's see if we can get something going but yeah there's and these uh these drivers can be written in device script okay okay that's super cool so again just getting back to web Technologies you know like people leveraging their existing skills so that's because like I think in the past you the drivers would have to be in C or C++ typically yes and for some of them they still have to yeah okay my My Demo is something something happen when I demo it's not happy anymore um maybe I switch back to to another Hardware so before my demo didn't work yeah so we're trying to get all the drivers that use i2c or SPI to be in device SCP because it's easier however some some some sensors still use some older sensors still use you have to do real time communication with them which case you have to go down to see because does JavaScript VM is not real time just like you know when you do set time out in the browser yeah you get it'll trigger sometimes you don't know exactly when yeah you're not guaranteed yeah you're not guaranteed so so same kind of constraints here um what's going on did I put the wrong device oh yeah and no no need to rush yeah if if feel free to just keep tinkering with it but I got to say I'm still Blown Away by everything I've seen so far so like I mean if if the Devo ends up working that's amazing but already I'm no we're we're trying another one which is theat fruit cutie pie okay uh so I'm gonna get rid of that uh let's see and I'm gonna go and actually it's I'm connecting there hold on so I'm connecting to the CutiePie now so that it knows what it's see it's seeing it now yeah that's a cutipie uh it added it and then I'm going to add so this one it's the shtc three same gig we're gonna read the temp temperatures serve get the reading you read it and then and you know beauty of typescript those all the most of the deploying aha it's working so this is the this is the device we're seeing here so yeah one says 17 degrees so one of them is lying um now now what I should see here when I hit run I'm just going to drop a list of the supported devices while you're doing that it is somehow decided that it doesn't want to run oh wait because I should see here my output yeah going on maybe we're we're we're pushing the with the with the browser stuff okay let's see taret let's go back to to that good old log we're trying to be too too fancy all right so we're going to L the temperature here all the goodness of Pat strip is still there you got your literals so right now you're okay so what I can do is I can actually blink the LED on the hardware so I'm gonna I'm gonna go here and uh set the status slide to status slide to something like I need some kind of in uh I need a variable to to toggle so if you know if we're odd or something we're gonna go for there we go something like that and then I need to increment I somewhere okay now when I run this yeah so it it it only accepts hex codes for the status light is that what it is oh well that's just it it's it takes the typical HTML color or 24-bit RGB color so so that's a easy way to kind of write it in typescript oh okay okay surprisingly oh yeah so if you look at the hardware okay oh yeah I see it blinking green and then yeah we have a little bug is that in the in the hardware layer green and and red are the hardware driver takes brg instead of RGB let me go let me go to Blue I didn't I didn't know brg existed well yeah we discovered it recently somebody's always got to change a standard deploy come on start simulators yes don't show this again okay so it's it's blinking it's getting the temperature um somehow now it decided the same bug that doesn't want doesn't want to run my my Hardware device okay now now I'm oh I wasn't connected okay so so now if I if I run this linking blue is blinking blue yeah yeah the the chat's excited about the the I know it's just a blink but we're excited about it yeah and LEDs are kind of hard to to uh to film but true um so i' I basically pushed this program to uh to the hardware that didn't have to download any tooling it's pretty high level you know once you have the driver if somebody wrote it uh I can also step debug it which you know which is is pretty [Music] cool yeah it's the and again just getting back to the existing web Tech so like you're leveraging the debugger and vs code just super nice experience and boom and then I can you know step through and then I can look at my temp so I've got a full de buying experience with with stack tra and everything um that's amazing we we tend not to use it we just we just run things um so that's device script I mean device script is definitely a research project you know there's a lot of there's a lot of ways to contribute um and do more funky stuff the but it's typescript so yeah a lot of it can be written in typescript when you have to go down to see things get more complicated because you need you need a Dev setup and yeah and containers don't work you need to be on your box okay yeah but but it sounds like the majority of the time you can get pretty far in just in in the browser like I mean unless you're creating a a driver like you were mentioning um that's that's super cool and it's like I'll keep reiterating this but like it's just amazing how this is super accessible to like pretty much anybody that has a web browser and like you know and if you don't own the the device uh as you mentioned we have the simulator so like you can still program the whole thing and then like once you do get a device and so like I don't know I I'm pretty Blown Away by the whole experience even though I haven't really done iot stuff it's just the whole developer experience from this and and maybe a lot of it was and we didn't see the whole happy path right you had a few bumps but like uh you know it's a demo that that just happens when you're live but this is super cool and are you telling me you can uh okay so you can do fetch as well so okay okay I guess if you have access to Wi-Fi yeah that makes sense yeah and so this particular one has Wi-Fi right and it's it's connected to I have a MAC address you know it's connected to my to my Wi-Fi here in my garage so if I wanted it to push this to website I could do a Fetch and you know of course doesn't like whatever fetch format but this is the same fetch signature as a browser so we have fetch we also have mqtt which is this very popular protocol for iot devices yeah that's the uh that's for messaging right yeah and you know ultimately we have websockets you can so you can open a client socket we don't support uh spinning up a server right now so you cannot serve Pages uh but fetch is you know if you understand Fetch and you've got um there's there's provisioning in in device script for keys so if you have secrets we support m files so you can say script oh is it settings add device settings and then it kind of you know it dropped dropped this file here so this is your so the local you see it's grayed out it won't be checked in yeah and then this is uh whatever you know Common settings okay yeah yeah and then so like this get switched into your device okay yeah so then like once it gets flashed to the firmware you know like uh if you need to make API calls with keys and stuff that's all there all available yeah and you know it's it's that M file so you know you just install the extension it works uh and then there's there's a way here to to read a setting Okay so we've also done a lot of work around hey you know what let's not encourage you to put your keys in your source code uh the key should be somewhere else and you phas that every time you do an iot service because you're going to need some kind of secret um we don't do we don't do um certificates yet but okay again you know device script is is still a prototyping uh device it is not not ready to be deployed in in industrial applications um there's there's still work to be done but um you know if you if you're a full stack developer you could be a a full metal stack developer yeah yeah yeah um now what what's interesting if you look at this thing here you know if you look at this guy here and hopefully the demo will work you've got a screen you know as an impressive as it is this little thing here is an OLED 128 by by 64 monochrome screen you know so you can go to town uh in fact uh this is this is uh you know we've been trying to motivate people to kind of build their own uh react for this we don't have a reactive UI framework and if you're thinking about building yet another clone of react don't do it for the browser do it for embeded um because we have the GSX kind of you know boiler plate is and we have we have a toy UI framework here by I mean chice that this is not meant to be and if you look at how react works and how this works it's basically immediate there is no tree we just we just draw straight to a a canvas context rendering okay okay so you don't even need the so you mean there's no virtual Dom well there's no Dom obviously but what I mean is there's no virtual representation of what you're I mean whatever it's it's a toy example and for the example to be small you know this is your rendering context and we just fill text for you know the text element is just fill text think about it okay uh is that a good idea probably who knows this is this is how it's used we have this render on image so okay this is your screen driver and here when the when the potential meter has a reading we we have observable so we we pipe that and then whenever we get a new reading we render the whole thing right yeah it's very similar to what react does but react has you know it's it's a mature industrial framework it's more complicated and remember Size Matters memory matters um and do you we don't have a dumb so a lot of things that are made in a existing UI framework made for a browser make no sense on an embedded device but yeah for sure but if you want to write a UR framework that runs on super cool mini screens or or even better so where was this guy so you have also have screens of LEDs like this guy here it has 25 LEDs these are colored they get super bright let me see something running on on this guy oh yeah it's turned on now let's see okay yeah okay you only see you know this you need a real specific lights yeah I can see it all all lit up but but yeah I see your to your point but it's hard to see the LED and uh this you could have a react framework for those uh you know why not I mean you here's a ring of those and you can buy them in strip of hundreds hundreds do so so that's where also kind of the react uh the typescript fun is is like oh I can you know I can write my own UI rendering framework for these things and I'm looking at the total different optimization problem there's no caching like do not cash because you want to reclaim memory as much as possible so there's no such as there's no used memo don't do that yeah yeah okay gotcha gotcha and so it's so it's it's a like you said it's a toy react framework you haven't created a custom renderer from react because like like I know like um like for example uh I think it's Netflix they have like uh like television so they have a a UI for television and they have a custom renderer but it's still using like the react core but this is this is like literally not react right you you've you've kind of mimicked the API you you've taken a lot of the API but it's it's very minute in size like you said because you have 64k on your on your device this is the plumbing that typescript needs to compile a g a TSX GSX file right right roughly and it's actually very well documented in the typescript docs and then it doesn't care what framew

2023-10-16 04:53

Other news