feat: aarch64 port - Raspberry Pi 4/400 SD boot (SDHCI, ext2 root + first-boot grow), USB hubs, PCI INTx routing, SMP, DMA fences
This commit is contained in:
daniel committed
1 parent
8c09ccef66
commit
808e1d66fd
66 files changed
+3690
-196
No files matched your search
+41
-9
@@ -104,10 +104,11 @@ dispatch table (`ArchIrqDispatch`), panic-frame printing, and:
|
||||
HHDM maps only the memory map's RAM and firmware regions, never the holes
|
||||
between them (device space on ARM). MAIR: 0 Normal WB, 1 Normal NC
|
||||
(`MapWC`), 2 Device-nGnRE (`MapMMIO`).
|
||||
|
||||
- GICv2 (`Interrupts.cpp`, from the MADT): IRQ slots map onto INTIDs --
|
||||
`IRQ_TIMER` onto the generic timer's PPI, the MSI slots (24-45) onto the
|
||||
SPIs of a GICv2m MSI frame (`ComposeMsi`), the IPI slots onto SGIs 0/1.
|
||||
SPIs of a GICv2m MSI frame (`ComposeMsi`), the IPI slots onto SGIs 0/1,
|
||||
and the platform lines (1-23) onto the SPIs `AssignPlatformLine` gives
|
||||
them (see "Interrupt lines" below).
|
||||
Interrupts are completed (EOI) before the handler runs, as on x86; a
|
||||
level-triggered source drops its line first in an ack hook.
|
||||
- Generic timer tick (`Timer.cpp`): the EL1 virtual timer at
|
||||
@@ -119,13 +120,11 @@ dispatch table (`ArchIrqDispatch`), panic-frame printing, and:
|
||||
instruction fetch does not snoop data writes (QEMU hides this; real
|
||||
cores do not). EL0 may maintain caches itself (`SCTLR_EL1.UCI`).
|
||||
|
||||
Not yet written (each is marked `TODO(aarch64)`):
|
||||
|
||||
- Secondary CPU start-up (`Smp::BootAPs`).
|
||||
- PCI INTx: the PC interrupt lines (slots 1-23) have no ARM meaning, so
|
||||
`RouteLegacyIrq` refuses them. Routing a PCI pin needs its `_PRT`
|
||||
entry and a GSI-based routing call; until then PCI drivers use MSI
|
||||
(GICv2m) or polling.
|
||||
- SMP (`Smp.cpp`): Limine wakes each secondary CPU (PSCI `CPU_ON`
|
||||
underneath) on the bootloader's tables; it programs its own system
|
||||
registers and the kernel translation regime (`Bringup.hpp`), its banked
|
||||
GIC CPU interface and timer PPI, then idles until a reschedule SGI.
|
||||
TLB maintenance is broadcast (`tlbi ...is`).
|
||||
|
||||
The generic kernel is compiled `-fsigned-char` for AArch64: it was written
|
||||
for x86's signed plain `char`.
|
||||
@@ -145,6 +144,39 @@ The kernel console is mirrored to the serial port the ACPI SPCR table names
|
||||
including the log written before the port was found. On QEMU that is
|
||||
`-serial stdio`.
|
||||
|
||||
## Interrupt lines
|
||||
|
||||
Drivers for PCI functions without MSI read the Interrupt Line register and
|
||||
pass it to `Arch::EnableLegacyIrq`. On x86 the firmware fills that register
|
||||
with the function's I/O APIC input (`Arch::PlatformLinesAreFixed()`). On
|
||||
other platforms `Pci::AssignInterruptLines` (`Pci/PciAcpi.cpp`) fills it
|
||||
instead: it finds each pin's global system interrupt in the root bridge's
|
||||
`_PRT` (swizzling through PCI-to-PCI bridges, resolving PNP0C0F link
|
||||
devices through `_CRS`), binds the GSI to a platform line with
|
||||
`Hal::GsiRouting::LineForGsi` and writes the line number into the register.
|
||||
Devices that ACPI describes directly (platform devices) get their line from
|
||||
`LineForGsi` too.
|
||||
|
||||
## DMA
|
||||
|
||||
`Memory/Dma.hpp` serves devices whose DMA does not snoop the CPU caches
|
||||
(ACPI `_CCA = 0`): pages for rings and descriptors come uncached, and
|
||||
`SyncForDevice` / `SyncForCpu` clean and invalidate streaming buffers
|
||||
around a transfer (`Arch::CleanDataCache` / `InvalidateDataCache`). For
|
||||
coherent devices every call is the plain allocation or a no-op. The xHCI
|
||||
driver uses it; it also orders its TRB writes with `Arch::DmaWriteFence`
|
||||
(the cycle bit last, a control TD's Setup TRB after its Data and Status
|
||||
TRBs) and its event reads with `Arch::DmaReadFence`. Those fences cost
|
||||
nothing on x86 and are required on ARM, where stores to normal memory and
|
||||
device registers may otherwise be observed out of order. The other PCI
|
||||
drivers (AHCI, NVMe, e1000, HDA) do not have them yet: they work on QEMU,
|
||||
which does not reorder, but need them before they meet real ARM hardware.
|
||||
|
||||
## Raspberry Pi 4 / 400
|
||||
|
||||
See `docs/raspberry-pi.md`: `make ARCH=aarch64 rpi4-sd`, the firmware it
|
||||
expects and how the Pi's hardware is reached.
|
||||
|
||||
## Userspace
|
||||
|
||||
The programs are architecture-neutral C/C++ built by `<arch>-montauk-gcc`
|
||||
|
||||
@@ -0,0 +1,133 @@
|
||||
# Raspberry Pi 4 / 400
|
||||
|
||||
MontaukOS runs on the Raspberry Pi 4 and 400 through the Pi Firmware Task
|
||||
Force's UEFI firmware ([pftf/RPi4](https://github.com/pftf/RPi4), an EDK2
|
||||
build) in ACPI mode, booting Limine like any other UEFI machine.
|
||||
|
||||
## Building and writing the SD card
|
||||
|
||||
```bash
|
||||
./toolchain/build-montauk-toolchain.sh aarch64 # once
|
||||
make ARCH=aarch64 rpi4-sd # -> montauk-rpi4.img
|
||||
sudo dd if=montauk-rpi4.img of=/dev/sdX bs=4M conv=fsync
|
||||
```
|
||||
|
||||
`scripts/mk-rpi4-sd.sh` downloads the firmware release once into
|
||||
`rpi4-firmware/` (`PFTF_VERSION=v1.53` by default) and builds an MBR image:
|
||||
|
||||
| Partition | Type | Contents |
|
||||
|---|---|---|
|
||||
| 1 | 0xEF, FAT32 | the firmware files, Limine (`EFI/BOOT/BOOTAA64.EFI`), `boot/kernel` |
|
||||
| 2 | 0x83, ext2, label `MONTAUKROOT` | the system tree (the contents of `ramdisk.tar`) |
|
||||
|
||||
Limine loads only the kernel; the `limine.conf` on the card has no ramdisk
|
||||
module. Finding no ramdisk, the kernel mounts the partition labelled
|
||||
`MONTAUKROOT` as `0:/` (`Fs/Boot.cpp`), read through the SD driver
|
||||
(`Drivers/Storage/Sdhci`), and the ESP after it. Files load as they are
|
||||
used instead of the firmware reading the whole 80 MB tree before boot.
|
||||
Unlike the ramdisk, the root is persistent: changes to `0:/` survive a
|
||||
reboot, so shut down from the desktop rather than pulling power.
|
||||
The image is sized to its contents (`ROOT_FREE_MIB`, default 64, sets the
|
||||
free space on the root) and grows to fill the card on first boot.
|
||||
|
||||
### First boot: growing the root
|
||||
|
||||
Only the SD image ships `0:/os/firstboot.elf` (`programs/src/firstboot`,
|
||||
built into `programs/bin-aarch64-rpi/`, which the script lays over the
|
||||
system tree), and the script adds its init service, ahead of all others,
|
||||
and its capability grant (`storage_admin`, `system_image`) to the image's
|
||||
`init.toml` and `capabilities.toml`. On first boot it shows a status card
|
||||
in the login screen's style while it calls `SYS_FS_GROW` on drive 0 in a
|
||||
loop:
|
||||
|
||||
1. The kernel grows the MBR entry of the root partition as far as the
|
||||
filesystem can use (to the end of the card, or 1 TiB), on a 1 MiB
|
||||
boundary (`Gpt::ResizeMbrPartition`).
|
||||
2. The ext2 driver grows the mounted volume (`Fs/Ext2.cpp`, "Online
|
||||
grow"): each call writes the bitmaps and zeroed inode tables of a few
|
||||
new block groups past the old end, then the last call commits the
|
||||
bitmaps, descriptor table and superblock. The descriptor table grows
|
||||
into the blocks `mke2fs -O resize_inode -E resize=2^28` reserved; the
|
||||
leftover reservation is freed and `resize_inode` cleared, so the volume
|
||||
grows once.
|
||||
|
||||
The new groups' inode tables are the bulk of the writes (0.1% of the card,
|
||||
one inode per 256 KiB), written as long multi-block runs. Once the volume has grown, `firstboot.elf` deletes
|
||||
itself and init skips its (optional) service from then on. If it fails it
|
||||
stays and the next boot tries again; an interrupted grow is safe, since the
|
||||
new groups are only referenced once the superblock is written, and a
|
||||
partition larger than its filesystem is valid.
|
||||
|
||||
`make ARCH=aarch64 rpi4-sd ROOTFS=ramdisk` builds the old layout instead:
|
||||
one FAT32 partition with `boot/ramdisk.tar`, which Limine loads through the
|
||||
firmware. It boots slowly but does not depend on the kernel's SD driver.
|
||||
|
||||
`make ARCH=aarch64 run-rpi4-sd` boots the ext2 image on QEMU with the card
|
||||
on `sdhci-pci`, so the root comes through the same driver. QEMU's UEFI has
|
||||
no SD driver, so the card's ESP is also attached on its own over virtio for
|
||||
the firmware to load Limine from. The card is padded to
|
||||
`RPI4_QEMU_CARD_MIB` (default 4096), and every run starts from a fresh
|
||||
copy, so every run is a first boot.
|
||||
|
||||
The firmware defaults are what MontaukOS needs; leave them:
|
||||
|
||||
- **System Table Mode: ACPI.** The kernel reads ACPI only.
|
||||
- **Limit RAM to 3 GB: enabled.** The BCM2711's PCIe can only reach the
|
||||
first 3 GB (its tables say so in `_DMA`). The kernel warns at boot when
|
||||
RAM extends past a device's `_DMA` range.
|
||||
|
||||
## Serial console
|
||||
|
||||
The kernel console (the whole boot log, including what was printed before
|
||||
the port was found) is mirrored to the UART the firmware's SPCR table names.
|
||||
The firmware's `config.txt` applies the `miniuart-bt` overlay, which puts the
|
||||
PL011 on GPIO 14/15: header pins 8 (TX) and 10 (RX), ground on pin 6, 115200
|
||||
8N1, 3.3 V levels. On the Pi 400 the header is on the back.
|
||||
|
||||
## How the hardware is reached
|
||||
|
||||
| Part | Source | Driver |
|
||||
|---|---|---|
|
||||
| 4x Cortex-A72 | MADT, PSCI (TF-A, SMC) | `Arch/aarch64` SMP via Limine MP |
|
||||
| GIC-400 (GICv2) | MADT | `Arch/aarch64/Interrupts.cpp` |
|
||||
| Generic timer | GTDT | `Arch/aarch64/Timer.cpp` |
|
||||
| Display | UEFI GOP (VideoCore) | Limine framebuffer |
|
||||
| Keyboard (built in), USB ports | VL805 xHCI as ACPI `\_SB.SCB0.XHC0` (`PNP0D10`) | `Drivers/USB/Xhci.cpp` platform probe |
|
||||
| USB 2 devices (built-in keyboard too) | VIA hub on the VL805's USB 2 root port | `Drivers/USB/Hub.cpp` |
|
||||
| Reset, power-off | PSCI | `Arch::Reboot`, `Arch::PowerOff` |
|
||||
|
||||
The VL805 sits behind the BCM2711's PCIe, whose config space is not ECAM
|
||||
(the root bridge's `_DSD` says `linux-ecam-quirk-id = "bcm2711"`), so the
|
||||
kernel does not enumerate PCI there. The firmware has already brought up the
|
||||
link and the VL805 and describes the controller as a platform device: a
|
||||
register block at `0x600000000`, GSI 175, and `_CCA = 0`, i.e. its DMA does
|
||||
not snoop the CPU caches. `XHC0._INI` (run by the AML interpreter) enables
|
||||
the controller's PCI memory decoding and bus mastering. The xHCI driver then
|
||||
keeps its rings and contexts in uncached memory, bounces control-transfer
|
||||
data through an uncached page, and cleans/invalidates the caller buffers of
|
||||
bulk transfers (`Memory/Dma.hpp`). It also watches the event ring from the
|
||||
timer tick, so input keeps working if the INTx line never fires.
|
||||
|
||||
The VL805 has one USB 2 root port (port 1) and four USB 3 ones. An on-board
|
||||
VIA hub (`2109:3431`) sits on the USB 2 port, and the USB 2 lines of every
|
||||
socket, and the Pi 400's internal keyboard (`04d9:0007`), hang off it: a
|
||||
keyboard or mouse is always a low/full-speed device behind a high-speed hub,
|
||||
reached through that hub's transaction translator.
|
||||
|
||||
## Not supported yet
|
||||
|
||||
- Ethernet (BCM54213 via GENET v5, `BCM6E4E`).
|
||||
- SD high-speed mode: the driver transfers by polled PIO at 25 MHz on a
|
||||
4-bit bus (multi-block CMD18/CMD25 with Auto CMD12), at most 12.5 MB/s;
|
||||
switching the card to 50 MHz (CMD6) would double that.
|
||||
- Wi-Fi and Bluetooth (CYW43455 over SDIO/UART).
|
||||
- Audio (HDMI via VideoCore, the 3.5 mm jack via PWM).
|
||||
|
||||
## Status
|
||||
|
||||
Everything above is verified on QEMU only, including the Pi-specific path:
|
||||
the firmware's own XHC0 table, retargeted at QEMU's xHCI and loaded with the
|
||||
`bcm2711` quirk on QEMU's root bridge, boots to the desktop with non-coherent
|
||||
DMA. Real hardware has not been tried yet. The ext2 root on the SD card boots on QEMU (`run-rpi4-sd`); the
|
||||
first-boot grow has been tested only against the kernel's ext2 code in a
|
||||
host harness (images grown, then checked with `e2fsck`).
|
||||
Reference in new issue
Block a user