DevCon23 - Enabling Secure Boot (V2) on ESP32 Platforms in Development and Production
Hello, my name is Ning. I work at Thistle Technologies. Today, I'm going to talk about Enabling Secure Boot (V2) on ESP32 Platforms in Development and in Production. Before we start talking about Secure Boot, Let me talk about myself briefly. I run the Engineering team at a company called Thistle Technologies.
For my background, I was trained in Mathematics and Cryptography. I have many years of experiences in building, attacking and securing the hardware devices and key management systems in production systems. Currently, I work at Thistle to make hardware security easy for IoT device makers. Now let's talk about Secure Boot. Some of you may know the Secure Boot is the boot sequence of software images on hardware device in which each software image is loaded and executed only when it's authorized by a previously authorized software image. So in the end, the authorization boils down to a root of trust (RoT) in hardware.
The Secure Boot needs chipset support. Our awesome ESP32 platforms support it. So once Secure Boot is enabled, it cannot be disabled on the same device. So it ensures that only signs of firmware images can run on the hardware device.
Therefore, it prevents malware persistence at boot time. So depending on the design, you're doing on the ESP32 platforms. So for example, if you are doing a crypto hardware wallet, using ESP32 and you sell this product to your customers, you'll probably want to get Secure Boot enabled so that nobody can flash arbitrary firmware on your device.
So to enable Secure Boot on the hardware device, usually the process is consisting of the following steps. So first of all, we need to create and manage the firmware signing keys or a public key infrastructure. Second, we need to connect the just created keys to the hardware root-of-trust. So we need to do a field programming to configure security parameters and the root-of-trust into hardware. Then we need to properly sign firmware images because once the Secure Boot is enabled, the audio images need to be signed in order to run on the device.
So to sign a firmware image, we require signing infrastructures usually. So typically a secure and easy to use, signing key management, it's not very easy to do, it's a challenge. We're going to talk about how we can address these challenges in the rest of the presentation. So now let's talk about ESP32. The ESP32 supports Secure Boot. So it's great that it supports this awesome hardware security feature.
The preferred Secure Boot scheme for ESP32 platforms is the Secure Boot V2 (SBV2). Secure Boot V2 is available from ESP32 revision 3 and later. Also, it's available in other, the newer ESP32 platforms, including ESP32-S2, S3, etc. Secure Boot V2 is based on public cryptography.
It uses the 3K-bit RSA with the PSS padding to sign the bootloader image and the app image. And they will be verified at boot time by the boot ROM. ESP32 SBV2 supports at least two Boot Flows, which I call ESP-IDF and MCUBoot and ZephyrOS. Many of you are probably more familiar with the ESP-IDF Boot Flow, is based on ESP-IDF framework. So you can see the two boot flow in the slide.
In terms of the booting process, it's very similar. In the ESP-IDF Boot Flow, the IDF bootloader is used. The boot ROM will load the IDF bootloader. The IDF bootloader will load the app image. In the MCUboot and ZephyrOS based Boot Flow, the boot ROM will load the MCUboot bootloader and MCUboot bootloader will load the ZephyrOS-based app image. In terms of Secure Boot, there are some differences in implementation.
In the ESP-IDF case, the boot ROM verifies the IDF bootloader using a public key hash burned into eFuse. And the IDF bootloader will verify the FreeRTOS-based app image also using the public key hash and burned into eFuse. In the ESP32 MCUboot and ZephyrOS case, the boot ROM will verify the MCUboot bootloader using the public hash in eFuse. However, when the MCUboot bootloader verifies the ZephyrOS-based app image, that uses a public key embedded into the bootloader, then see a bootloader itself to do the verification. This two conceptually, these two boot flows are very similar, but the implementations are slightly different. So, how do we enable Secure Boot V2 on ESP32 device? In general, there are a few things we need to do.
First of all, we need to generate firmware signing keys. This is usually a one-time effort. Once the key is generated, it will be used to sign firmware that runs on the same device over and over again.
Then we need to determine an eFuse configuration and blow the eFuses to get Secure Boot enabled. Again, this is a one-time effort. Once we know how to do it, we will use the same configuration to fuse the same type of devices in the same product. Third, we need to sign firmware in order to have it run on the Secure Boot-enabled device. So whenever we make a change in the firmware, we need to sign it again so that it can run on the device.
So therefore it's an ongoing effort to do firmware signing. Now let's talk about the signing key generation for Secure Boot V2 enablement. So for local development and testing, we can use OpenSSL or the ESP-IDF tool, the espsecure.py tool to generate a key pair. And the key pair will be written into files.
For production releases, the file-based keys are usually not the best choices. We recommend to use the proper key management system, such as AWS KMS, GCP KMS, and Hardware Security Module-based key management systems. In order to enable the Secure boot on the ESP32 device, we need to burn eFuses. We need to know which eFuses to burn. There are at least two ways of doing this.
The first way is not the recommended way. We can burn the individual eFuses based on the values computed from the public key and the handpicked values to enable Secure Boot. The using the ESP eFuse tool. We don't recommend it because it's easy to make a mistake if we burn individual fuses. If we mess up with one bit, we may break a device. The recommended way for enabling Secure Boot is to sign the bootloader and the app image bundle with the your choice of the private key.
Then we flash the signed images and the valid partition table to an unfused ESP32 device. On the first boot, if everything works, if the images and the partition table are valid, the eFuses will be blown, by the bootloader, to get a Secure Boot V2 enabled. So this is taken care of by the bootloader, in one shot.
And it only requires the flashing of the images. So that's a recommended way. So as I mentioned before, once Secure Boot V2 is enabled, only signed images can boot on the device.
So we need signing continuously. Now you probably have realized that the Secure Boot V2 enablement is largely a firmware signing problem and the key management problem. Because once the firmware images are properly signed, it's a matter of flashing onto the device to get Secure Boot enabled and to get the firmware run. So there are two cases we want to distinguish. The first one is the signing in development.
The usually in development, we only play with a handful devices. The signing key confidentiality is usually not very critical because these devices will not be used in the production scenario will not be in the hands of the customers. So the signing process should be very easy to allow faster user iteration.
So if there is the bug in the firmware, want to be able to easily sign the firmware and get it working and test it on the development unit. For production signing, the situation can be a little different. If you are producing a design and you're selling the finished devices using ESP32 in many units, if you are selling thousands of devices to a customer, you probably don't want to get the signing key leaked so that the attacker can sign your firmware. They can sign any firmware that runs on the device that you sell. So we should not expose the signing key in the production signing scenario. But still, the signing process should be easy to allow fast releases.
If there is a bug in the firmware, or if there's a new feature available in the firmware, you want to be able to quickly release the firmware. In both cases, it's desirable to have the production firmware very close to the development firmware so that what is tested in development can be reflected in production. So therefore, in terms of production images and the development images, ideally, they should only differ, only on public keys and signatures for Secure Boot enabled devices. Okay, so in the rest of the presentation, we are going to talk about how to sign firmware in development and in production to enable Secure Boot V2 for ESP32 platforms. So signing in development can be done in this way. So first, we generate file-based private signing keys using software tools like OpenSSL or the IDF tool.
Then we use the menu config to enable Secure Boot V2. We will configure the signing key file in "menuconfig" and also save the "sdkconfig" file for later use. Now, we can use the IDF build command to build and sign an application with the saved "sdkconfig", so we'll be signing a void app.
I'm going to talk about the void app in the following slide. So once everything is signed, we can just flash the bootloader and the app image onto the device to get Secure Boot enabled on the first boot. So the device and signing keys are usually not very security critical. And sometimes it's OK to check them into a version control system for easier access and for better backup. So here we are showing the most important configuration for enabling Secure Boot from "menuconfig".
Usually, on Secure Boot-enabled devices, we also recommend to disable JTAG, but enable secure ROM download mode on ESP32-S2 or S3 when they are available so that we can also flash the signed firmware onto the devices. Yeah, so with menuconfig, this is used to enable Secure Boot. We saved sdkconfig. Now we can use sdkconfig to build a void app.
So the void app is an app that I wrote that, you know, Thistle is open sourcing as well. So it's an app that does not do much. So you can see like there's only a few lines in the source code of the app. Essentially the prints out a message to the console. It does not do anything, but once it's signed, it can be flashed on an ESP32 chip to blow the security relevant bits to enable Secure Boot and other security features.
A minimum "fuseblower" void app is very important in the production scenario, because by doing nothing, it exposes no attack service to a production device. So that's how you can sign firmware in development manually, and this process can be further automated into a one-click setup. So my company, Thistle, has released the developer tools, including bunch of Docker files, the pre-built Docker images, and utility apps, including the void app to allow a one click firmware signing in development. So here is the GitHub repo that we open source to allow this to happen.
So firmware signing essentially becomes one command, one Docker build command, which you can see here. So you can overwrite the sdkconfig and your private key as build arguments in Docker build. Now I'm going to show a video demo of development firmware signing using the Docker image.
So first of all, like that we will clone the GitHub repo I just talked about. So once the repo is cloned, we change into the ESP32 directory and run the Docker build command. It'll take a few minutes.
The Docker image is cached, so it's really fast in the demo. So these are the two build parameters that can be overwritten to allow you to insert your own sdkconfig and your private key. So once the Docker image is built, the bootloader and the app images are signed automatically inside it. So we will activate the IDF environment so that we can flash the images onto the device. Now the IDF environment is activated. And this is the device connected to the host machine running Docker.
So that's ESP32. So now let's use the void_app. So before we flash the app, Let's see what's happening on the board.
Now, since flash is totally erased, there's nothing on flash. Of course, we get invalid header. So that's normal. Now, we are going to flash the signed bootloader image from the Docker container. OK, the bootloader image is flashed onto the device. We're going to flash the partition table and the app image.
Okay, now the flash program is done. So let's see what's happening on the device. Okay, good sign. We see this message, "I'm the void app, I do nothing." So the image that the app has booted.
If you look more carefully, like you can see that the Secure Boot is also enabled from the console output. And this message is about the app image verification by the firmware bootloader. So we have Secure Boot enabled and the signed images are up and running. That's a demo for doing the backend signing for ESP32. Now let's talk about the firmware signing in production.
There are many reasons the file-based signing keys are not very suitable for production signing case. So we can't easily use the ESP32 build system to load a private key from file system to sign a production image if you want to protect the key. The reason why file based signing keys are not suitable is because they can be exposed by human beings or by malware. So some of you may still remember the recent firmware signing key leak from MSI. So we don't want to do that.
We don't want to use file-based keys. Then what do we do? So we have a few requirements for signing firmware in production. On the security side, we want the production signing keys not to be exposed to any human operator or software. On the usability side, we still want the day-to-day signing operations to be smooth. So if every signing operation requires a ceremony to go to a safe, to fetch a key, that's very inconvenient.
So, Thistle has a solution based on a KMS-backed production signing keys. So essentially, we are providing a firmware signing and patching service to use KMS protected keys. So the idea is that we manage the firmware signing key in the proper key management system. In this case, the key is generated in the system and also managed in the system. No one, including Thistle, has access to the keys. But the key management system can be used in the Oracle to do the signing operations without leaking the private key.
So conceptually, our idea is this. The input will be ESP32 firmware image, signed using the development signing method you saw earlier. Then this image will be uploaded to the service, will be sent to the service.
The service will parse the image, then extract the signature block. It will use the production signing key in KMS in the key management system to sign the payload, the content of the image, and replace the development signed signature block with the production signed signature block. So the image, so these two images will be almost identical and the only difference is in the signature block. So that idea, that's how we proposed to solve the problem.
And in brief, we implemented a system, shown in this slide. So it's a firmware signing system, you know, consisting a few micro-services. So we have an IAM service to do user management, to do user management and access control, and also the signing configuration. The firmware patching service, now deals with parsing and patching firmware images.
It has no access to cryptography related operations. For cryptography related operations, we have a code signing service, which will use the KMS. So our implementation is based on GCP KMS.
So the code signing service will use the KMS, the managed production signing key to sign a digest and produce the signature block. It will give the signature block to the firmware patching service to patch and get the production signed image in the output bucket. So that's how it works.
We don't get the signing key exposed to any entity, outside of the KMS. So our ESP32 Secure Boot V2 production signing system supports both the ESP-IDF Boot Flow and the MCUboot ZephyrOS-based Secure Boot flow. Now I'm going to show you a demo of doing production signing using the Thistle system.
So in order for you to use the system, you just need to go to the Thistle website, sign up to get an account, Then log in, once we are logged in, you can create a project, in which the firmware bundle will be uploaded. So we just call the project "DevCon23 Demo" and it's for production signing of ESP32 firmware. Once you click the button of create Secure Boot signing key, the key pair will be created in the KMS. Now you can enter the project. What happens here is that in the project, you can upload your boot image and app image. There is the Secure Boot feature.
In settings, you can also get the public key generated for the key generated in KMS. Only the public key is available here, so you can use the key to do the verification independently, if you wish. The private key is never exposed. Now let's go to the Secure Boot field and upload a firmware bundle. We call it a "dummy signed IDF firmware". So these firmware images are signed using the previously mentioned method in web development.
So let's upload the bootloader image and the app image. Once we click on the upload button, the backend will start processing these images during the signing and the patching and put them to a place for you to download. Now the signing is done. These two production signed and patched images are available for you to download and they are readily flashable to your unfused or Secure Boot-enabled devices to enable Secure Boot.
Okay, that's the end of my presentation. I'd like to thank Thistle Technologies for supporting this work, and also my colleagues at Thistle for their feedback. So in particular, I'd like to thank Pierre for telling me about Espressif DevCon, and I'd like to thank Toru and Russell for helping build the production firmware signing service. This presentation can also be viewed from this link at the bottom of this slide. Thank you very much. Now I'm ready to take questions.
2023-09-24 12:47
