cm4: hardware-validate eMMC A/B disk boot and FIPS on CM4 (arm64 header, 0x200000 load, mini-UART console)

Author: dgarske

config/examples/cm4_emmc.config

@@ -0,0 +1,54 @@
+# Raspberry Pi CM4 (BCM2711) - onboard eMMC A/B disk boot.
+#
+# wolfBoot drives the BCM2711 EMMC2 controller (Arasan SDHCI v3.0 @ 0xFE340000)
+# to read GPT A/B image partitions from the onboard eMMC via update_disk.c, and
+# boots the highest valid version with rollback.
+#
+# Transfer mode: SDHCI_SDMA_DISABLED forces PIO. The BCM2711 EMMC2 SDMA
+# boundary-restart and cache coherency are not yet validated (same Arasan
+# family as the Versal quirk); DISK_EMMC compiles in the PIO BRR-race
+# workaround in src/sdhci.c. SDHCI_FORCE_CARD_DETECT: the onboard eMMC has no
+# routed card-detect line.
+#
+# Bring-up: uncomment DEBUG_SDHCI (+ DEBUG_DISK / DEBUG_GPT) for verbose
+# controller/partition tracing over the mini-UART console.
+#
+# For wolfCrypt FIPS 140-3, build with FIPS=1 pointing at a FIPS / FIPS-ready
+# wolfSSL tree (NO_ARM_ASM required; entropy from the RNG200 TRNG). See
+# docs/FIPS.md. e.g.:
+#   make FIPS=1 WOLFBOOT_LIB_WOLFSSL=../wolfssl-5.9.2-gplv3-fips-ready NO_ARM_ASM=1
+#FIPS?=1
+#WOLFBOOT_LIB_WOLFSSL?=../wolfssl-5.9.2-gplv3-fips-ready
+#NO_ARM_ASM?=1
+ARCH?=AARCH64
+TARGET?=cm4
+SIGN?=ECC384
+HASH?=SHA384
+IMAGE_HEADER_SIZE?=1024
+DEBUG?=0
+DEBUG_UART?=1
+DISK_SDCARD?=0
+DISK_EMMC?=1
+CFLAGS_EXTRA+=-DSDHCI_FORCE_CARD_DETECT
+CFLAGS_EXTRA+=-DSDHCI_SDMA_DISABLED
+#CFLAGS_EXTRA+=-DDEBUG_SDHCI
+#CFLAGS_EXTRA+=-DDEBUG_DISK
+#CFLAGS_EXTRA+=-DDEBUG_GPT
+EXT_FLASH?=0
+NO_XIP=1
+NO_QNX?=1
+ELF?=1
+VTOR?=1
+SPMATH?=1
+PKA?=0
+WOLFTPM?=0
+WOLFBOOT_NO_PARTITIONS=1
+CFLAGS_EXTRA+=-DBOOT_PART_A=1
+CFLAGS_EXTRA+=-DBOOT_PART_B=2
+CFLAGS_EXTRA+=-DDISK_BLOCK_SIZE=0x80000
+WOLFBOOT_LOAD_ADDRESS?=0x10000000
+WOLFBOOT_RAMBOOT_MAX_SIZE=0x2BC00000
+WOLFBOOT_LOAD_DTS_ADDRESS?=0x1000
+WOLFBOOT_PARTITION_BOOT_ADDRESS=0x80200000
+WOLFBOOT_PARTITION_SIZE=0x4000000
+WOLFBOOT_SECTOR_SIZE=0x1000

config/examples/cm4_sdcard.config

@@ -5,10 +5,10 @@
 # reads GPT A/B image partitions via update_disk.c and boots the highest valid
 # version with rollback.
 #
-# NOTE: the EMMC2 platform glue in hal/cm4.c is bring-up scaffolding pending
-# on-hardware validation (controller clock/pinmux state left by the GPU
-# firmware, card-detect wiring on the carrier). Use SDHCI_FORCE_CARD_DETECT
-# for the embedded eMMC path.
+# The EMMC2 glue in hal/cm4.c is hardware-validated on the onboard eMMC
+# (see cm4_emmc.config). The microSD path uses the same driver but needs a CM4
+# Lite - an eMMC module disables the carrier's microSD slot. Use
+# SDHCI_FORCE_CARD_DETECT for media with no routed card-detect line.
 #
 # For wolfCrypt FIPS 140-3, build with FIPS=1 pointing at a FIPS / FIPS-ready
 # wolfSSL tree. The FIPS module uses portable-C crypto, so NO_ARM_ASM is

docs/FIPS.md

@@ -55,12 +55,17 @@ The FIPS DRBG needs a seed. wolfBoot's lean configuration compiles out the OS se
 
 The module verifies an in-core integrity hash (HMAC-SHA-256 over the module's code and read-only data) at startup. A fresh build ships with a placeholder, so the first run reports a mismatch; capture the runtime hash and seal it:
 
-1. Build and run with a FIPS callback registered (wolfBoot does this in `src/loader.c`). On a mismatch the module reports the runtime hash; wolfBoot prints it (`FIPS in-core hash = ...`, from `wolfCrypt_GetCoreHash_fips()`), and on the CM4 the test app (`test-app/app_cm4.c`) prints it over UART.
+1. Build and run with a FIPS callback registered (wolfBoot does this in `src/loader.c`). On a mismatch the module reports the runtime hash; wolfBoot prints it (`FIPS in-core hash = ...`, from `wolfCrypt_GetCoreHash_fips()`) before halting.
 2. Copy the reported 64-hex-character hash into `verifyCore[]` in `wolfcrypt/src/fips_test.c`.
 3. Rebuild and re-run. `wolfCrypt_GetStatus_fips()` now returns 0 (operational).
 
 The seal is **specific to the exact binary layout**: any code change that shifts the FIPS module's link addresses changes the in-core hash and requires a re-seal. Re-sealing `verifyCore[]` itself does not shift addresses (same-size rewrite), so once the rest of the build is fixed the seal converges in one pass.
 
+Two practical traps when re-sealing (both cost time on the CM4 bring-up):
+
+- `verifyCore[]` can be sealed via a build define instead of editing the FIPS tree: `CFLAGS_EXTRA="-DWOLFCRYPT_FIPS_CORE_HASH_VALUE=<hash>"` (unquoted; `fips_test.c` stringifies it). But apply it by recompiling **only** `fips_test.o` - `rm "$WOLFBOOT_LIB_WOLFSSL/wolfcrypt/src/fips_test.o"` then rebuild. `verifyCore[]` lives *after* `wolfCrypt_FIPS_last`, so this leaves the hashed region byte-identical and converges in one pass. Passing the define through a **full** rebuild (`make clean` + build) recompiles the whole module and shifts its link addresses, so the hash never stabilizes.
+- `make clean` removes `$(WOLFBOOT_LIB_WOLFSSL)/wolfcrypt/src/*.o`. If you build with `WOLFBOOT_LIB_WOLFSSL=<fips-tree>` but run `make clean` **without** that variable, it cleans the default `lib/wolfssl` instead, leaving the stale FIPS `fips_test.o` in place - the new seal silently never lands. Pass `WOLFBOOT_LIB_WOLFSSL` to `clean` too, or `rm` the object directly.
+
 ## Bare-metal targets
 
 The FIPS module targets a hosted environment; a few things must be provided on bare-metal (the CM4 does all of these):

docs/Targets.md

@@ -3714,11 +3714,11 @@ cp config/examples/cm4.config .config
 make CROSS_COMPILE=aarch64-none-elf-
 ```
 
-The example uses `SIGN=ECC384 HASH=SHA384` (both FIPS-approved). wolfBoot is entered by the firmware at `0x80000` at EL2, matching `hal/cm4.ld`. The image is loaded from RAM: wolfBoot reads the signed application at `kernel_addr` (`0x140000`), verifies it, copies it to `WOLFBOOT_LOAD_ADDRESS`, and boots.
+The example uses `SIGN=ECC384 HASH=SHA384` (both FIPS-approved). wolfBoot is built as an AArch64 Linux kernel image (`kernel8.img`): `src/boot_aarch64_start.S` prepends the 64-byte ARM64 image header (`"ARM\x64"` magic), and `hal/cm4.ld` links at `0x200000`. The VideoCore firmware only transfers control to a 64-bit kernel that carries this header, and it runs the image in place at the 2 MB-aligned load address `0x200000` (it does not relocate a header image down to `0x80000`) at EL2. The image is loaded from RAM: wolfBoot reads the signed application at `kernel_addr` (`0x2C0000`), verifies it, copies it to `WOLFBOOT_LOAD_ADDRESS`, and boots.
 
 ### Signing and assembling the boot image
 
-Sign the application, then concatenate wolfBoot and the signed image so the signed image lands at `kernel_addr` (`0x140000` = `0x80000` load + `0xC0000`):
+Sign the application, then concatenate wolfBoot and the signed image so the signed image lands at `kernel_addr` (`0x2C0000` = `0x200000` load + `0xC0000`):
 
 ```
 make keytools tools/bin-assemble/bin-assemble
@@ -3731,7 +3731,7 @@ tools/bin-assemble/bin-assemble kernel8.img \
 
 ### config.txt
 
-The CM4 UART on GPIO14/15 defaults to the mini-UART because the PL011 is used by Bluetooth. wolfBoot drives the PL011, so route it to the header and fix a known UART clock:
+The debug console on GPIO14/15 is the BCM2711 mini-UART (AUX, Linux `ttyS0`); the PL011 is used by Bluetooth. wolfBoot drives the **mini-UART** by default (it inherits the firmware's stable baud, which `enable_uart=1` fixes by pinning `core_freq`), so no baud reprogramming is needed. Boards where `dtoverlay=disable-bt` actually routes the PL011 onto GPIO14/15 can build with `CFLAGS_EXTRA=-DCM4_UART_PL011` to use the PL011 instead.
 
 ```
 arm_64bit=1
@@ -3761,13 +3761,15 @@ Firmware Valid
 Booting at 0x3080000
 ```
 
-### Optional: eMMC/SD A/B updates
+### Optional: eMMC/SD A/B disk boot
 
-`config/examples/cm4_sdcard.config` enables the disk updater (`DISK_SDCARD`/`DISK_EMMC`), driving the BCM2711 EMMC2 controller through the generic SDHCI driver (`src/sdhci.c`) for A/B image partitions on the boot medium. This path is provided as bring-up scaffolding and is not yet hardware-validated.
+`config/examples/cm4_emmc.config` (onboard eMMC) and `config/examples/cm4_sdcard.config` (microSD) enable the disk updater (`DISK_EMMC`/`DISK_SDCARD`), driving the BCM2711 EMMC2 controller through the generic SDHCI driver (`src/sdhci.c` + the `hal/cm4.c` register glue) to read A/B signed images from GPT partitions. wolfBoot reads the GPT, selects the higher-version image, verifies it, ELF-loads it (`ELF=1`) to `WOLFBOOT_LOAD_ADDRESS`, and boots.
+
+The **eMMC** path (`cm4_emmc.config`) has been validated end to end on CM4 hardware: SDHCI/eMMC card init -> GPT parse -> A/B version select -> SHA-384 integrity -> ECDSA-P384 signature verify -> ELF64 load -> boot of a signed payload. `tools/scripts/cm4/prepare_emmc.sh` builds the GPT layout (FAT boot partition with `kernel8.img` + firmware, plus raw A/B image partitions), signs a minimal test payload (`tools/scripts/cm4/disk_app.S`), and writes it to the eMMC over `rpiboot`. Uncomment `DEBUG_SDHCI` / `DEBUG_DISK` / `DEBUG_GPT` in the config for verbose bring-up tracing. The **microSD** path shares the same driver but is validated only on modules whose SD lines reach the microSD slot (a CM4 with onboard eMMC disables that slot).
 
 ### FIPS 140-3
 
-The CM4 target uses `SIGN=ECC384 HASH=SHA384` (FIPS-approved) and can perform its signature verification with the wolfCrypt FIPS 140-3 module (build `config/examples/cm4.config` with `FIPS=1`, pointing `WOLFBOOT_LIB_WOLFSSL` at a FIPS wolfSSL tree). At boot the module runs its power-on self-test and in-core integrity check, and wolfBoot refuses to boot unless the module is operational. Entropy for the FIPS DRBG comes from the BCM2711 RNG200 hardware TRNG. The full flow has been brought up and exercised end to end on CM4 hardware using the FIPS-ready bundle: FIPS module operational -> SHA-384 integrity -> ECDSA-P384 signature verification -> boot handoff. A production, CMVP-validated deployment additionally requires the licensed validated wolfCrypt FIPS bundle at the validated revision (see [FIPS.md](FIPS.md)). The on-target harness in `test-app/app_cm4.c` prints the runtime in-core hash over the UART for the `verifyCore[]` seal. See [FIPS.md](FIPS.md) for the full build, entropy, and hash-sealing procedure.
+The CM4 target uses `SIGN=ECC384 HASH=SHA384` (FIPS-approved) and can perform its signature verification with the wolfCrypt FIPS 140-3 module (build `config/examples/cm4.config` with `FIPS=1`, pointing `WOLFBOOT_LIB_WOLFSSL` at a FIPS wolfSSL tree). At boot the module runs its power-on self-test and in-core integrity check, and wolfBoot refuses to boot unless the module is operational. Entropy for the FIPS DRBG comes from the BCM2711 RNG200 hardware TRNG. The FIPS configuration builds with the CM4 hardware-boot support (ARM64 image header, `0x200000` load address, mini-UART console) and has been validated end to end on CM4 hardware with the FIPS-ready bundle: after sealing the in-core integrity hash, wolfBoot reports `FIPS 140-3 module operational` and the module gates the boot with SHA-384 integrity and ECDSA-P384 signature verification of the eMMC A/B image (`cm4_emmc.config` with `FIPS=1`; wolfBoot's `src/loader.c` runs the power-on self-test and in-core check before booting). A production, CMVP-validated deployment additionally requires the licensed validated wolfCrypt FIPS bundle at the validated revision (see [FIPS.md](FIPS.md)). On an in-core hash mismatch, wolfBoot prints the runtime hash (`FIPS in-core hash = ...`, from `src/loader.c`) to seal into `verifyCore[]`. Re-seal by recompiling only `fips_test.o` with `-DWOLFCRYPT_FIPS_CORE_HASH_VALUE=<hash>` (a full rebuild shifts the module boundary and the hash); see [FIPS.md](FIPS.md) for the full build, entropy, and hash-sealing procedure.
 
 ## Xilinx Zynq UltraScale
 

hal/cm4.c

@@ -3,10 +3,11 @@
  * HAL for the Raspberry Pi Compute Module 4 (CM4): Broadcom BCM2711,
  * quad-core Cortex-A72 (ARMv8-A).
  *
- * The VideoCore GPU firmware loads wolfBoot (as kernel8.img) to 0x80000 and
- * releases the A72 cores; wolfBoot verifies the appended signed payload and
- * boots it from RAM. hal_flash_* are no-ops (no in-place flash in this mode).
- * Optional eMMC/SD A/B via the generic SDHCI driver is at the end of the file.
+ * The VideoCore GPU firmware loads wolfBoot (an ARM64 kernel8.img carrying the
+ * Linux image header) to 0x200000 and enters it at EL2; wolfBoot verifies the
+ * signed payload and boots it from RAM, or from eMMC/SD A/B via the generic
+ * SDHCI driver (at the end of this file). hal_flash_* are no-ops (no in-place
+ * flash in this mode).
  *
  * Copyright (C) 2026 wolfSSL Inc.
  *
@@ -42,39 +43,67 @@
 /* Fixed addresses (provided by the linker script) */
 extern void *kernel_addr, *update_addr, *dts_addr;
 
-#if defined(HAVE_FIPS)
+/* Enable the identity MMU + caches when the build does more than the trivial
+ * RAM-boot: FIPS (unaligned/SIMD in the module), or the disk path (optimized
+ * code + SDHCI block-buffer memcpy fault on MMU-off Device memory). Normal
+ * cacheable memory permits those accesses and speeds up crypto/disk reads. */
+#if (defined(HAVE_FIPS) || defined(DISK_SDCARD) || defined(DISK_EMMC)) \
+    && defined(__aarch64__)
+#define CM4_USE_MMU
+#endif
+
+#if defined(CM4_USE_MMU)
 void cm4_mmu_enable(void);  /* defined below; called from hal_init */
 void cm4_mmu_disable(void); /* defined below; called from hal_prepare_boot */
 #endif
 
 #if defined(DEBUG_UART)
+/* Console UART select. On this bench CM4 the debug cable on GPIO14/15 is the
+ * BCM2711 mini-UART (AUX, Linux ttyS0), so that is the default. Boards where
+ * dtoverlay=disable-bt actually routes the PL011 onto GPIO14/15 can build with
+ * CM4_UART_PL011 to use the PL011 (0xFE201000) instead. */
+#if defined(CM4_UART_PL011)
 static void uart_tx(char c)
 {
     while (*UART0_FR & 0x20) /* TXFF: wait while FIFO full */
         ;
-    *UART0_DR = c;
-}
-
-void uart_write(const char* buf, uint32_t sz)
-{
-    while (sz-- > 0 && *buf)
-        uart_tx(*buf++);
+    *UART0_DR = (unsigned int)(unsigned char)c;
 }
 
 void uart_init(void)
 {
-    /* The VideoCore firmware has already routed the PL011 to GPIO14/15
-     * (dtoverlay=disable-bt) and set init_uart_clock=48MHz. Program the PL011
-     * for 115200 8N1 directly, without the VideoCore mailbox (a mailbox poll
-     * that never returns post-handoff would hang before any output).
-     * 48MHz UARTCLK: BAUDDIV = 48e6/(16*115200) = 26.04 -> IBRD 26, FBRD 3. */
+    /* Program the PL011 for 115200 8N1 assuming a 48MHz UARTCLK
+     * (init_uart_clock=48000000): BAUDDIV = 48e6/(16*115200) -> IBRD 26 FBRD 3. */
     *UART0_CR = 0;
     *UART0_ICR = 0x7FF;
     *UART0_IBRD = 26;
     *UART0_FBRD = 3;
     *UART0_LCRH = (1 << 4) | (1 << 5) | (1 << 6); /* FIFO, 8-bit */
     *UART0_CR = (1 << 0) | (1 << 8) | (1 << 9);   /* enable UART, TX, RX */
 }
+#else /* mini-UART (default) */
+static void uart_tx(char c)
+{
+    while ((*MU_LSR & MU_LSR_TXFF_EMPTY) == 0) /* wait until TX can accept */
+        ;
+    *MU_IO = (unsigned int)(unsigned char)c;
+}
+
+void uart_init(void)
+{
+    /* The firmware has already enabled the mini-UART at a stable baud
+     * (enable_uart=1 fixes core_freq), so - like the Linux 8250 console with
+     * "skip-init" - wolfBoot inherits that setup and just writes AUX_MU_IO.
+     * Reprogramming the baud here is unnecessary (and error-prone: the mini-UART
+     * clock is core_freq-derived, not a fixed rate). */
+}
+#endif /* CM4_UART_PL011 */
+
+void uart_write(const char* buf, uint32_t sz)
+{
+    while (sz-- > 0 && *buf)
+        uart_tx(*buf++);
+}
 #endif /* DEBUG_UART */
 
 void* hal_get_primary_address(void)
@@ -134,18 +163,19 @@ void hal_init(void)
     wolfBoot_printf("wolfBoot CM4 (BCM2711 Cortex-A72) hal_init, EL%d\n",
         (int)((el >> 2) & 0x3));
 #endif
-#if defined(HAVE_FIPS)
-    /* Bring up Normal cacheable memory before the FIPS POST, which uses
-     * unaligned / SIMD accesses that the MMU-off Device memory rejects. */
+#if defined(CM4_USE_MMU)
+    /* Bring up Normal cacheable memory before any code that uses unaligned /
+     * SIMD accesses (FIPS module, optimized disk path) which the MMU-off
+     * Device memory rejects. */
     cm4_mmu_enable();
 #endif
 }
 
 void hal_prepare_boot(void)
 {
-#if defined(HAVE_FIPS)
-    /* Undo cm4_mmu_enable() before handoff: flush the app out of the D-cache
-     * and return to the MMU-off state the application expects. */
+#if defined(CM4_USE_MMU)
+    /* Undo cm4_mmu_enable() before handoff: flush the loaded image out of the
+     * D-cache and return to the MMU-off state the application expects. */
     cm4_mmu_disable();
 #endif
 }
@@ -173,13 +203,16 @@ void* _sbrk(int incr)
     brk += incr;
     return (void*)prev;
 }
+#endif /* HAVE_FIPS */
 
+#if defined(CM4_USE_MMU)
 /* Minimal identity-mapped MMU + caches for the CM4. wolfBoot's simple startup
  * runs with the MMU off, so all memory is Device-nGnRnE, which faults on the
- * unaligned / 128-bit SIMD accesses the FIPS module and newlib printf perform.
- * Mapping DDR as Normal (cacheable) permits those accesses and speeds up the
- * crypto; the peripheral region (incl. 0xFE000000) stays Device.
- * Four 1GB block descriptors cover the 32-bit VA space at translation level 1. */
+ * unaligned / 128-bit SIMD accesses that the FIPS module, newlib printf, and
+ * the optimized disk/SDHCI code paths perform. Mapping DDR as Normal
+ * (cacheable) permits those accesses and speeds up crypto/disk reads; the
+ * peripheral region (incl. 0xFE000000) stays Device. Four 1GB block
+ * descriptors cover the 32-bit VA space at translation level 1. */
 #define MMU_BLOCK_NORMAL  0x0000000000000701ULL /* block, AttrIdx0, AF, SH inner */
 #define MMU_BLOCK_DEVICE  0x0000000000000405ULL /* block, AttrIdx1, AF, SH none  */
 
@@ -275,7 +308,7 @@ void cm4_mmu_disable(void)
     __asm__ volatile("dsb sy");
     __asm__ volatile("isb");
 }
-#endif /* HAVE_FIPS */
+#endif /* CM4_USE_MMU */
 
 #if defined(DEBUG) && defined(DEBUG_UART)
 /* CM4 bring-up diagnostic: exception handler invoked from cm4_vectors in
@@ -371,9 +404,9 @@ int RAMFUNCTION hal_flash_erase(uintptr_t address, int len)
 /* BCM2711 EMMC2 platform glue for the generic SDHCI driver (src/sdhci.c).
  * EMMC2 is a standard SDHCI v3.0 Arasan block at 0xFE340000. The driver uses
  * Cadence-style SRS offsets (0x200 + std); translate them to the standard
- * Arasan layout, mirroring the ZynqMP path in hal/zynq.c. NOTE: not yet
- * hardware-validated; clock/caps/card-detect quirks may be required once
- * validated on hardware. */
+ * Arasan layout, mirroring the ZynqMP path in hal/zynq.c. The GPU firmware has
+ * already configured the EMMC2 clock/pinmux, so only a controller soft reset is
+ * needed here (hardware-validated on CM4 eMMC). */
 #include "sdhci.h"
 
 uint32_t sdhci_reg_read(uint32_t offset)