As far as I have tracked it down (quite literally up to the point where it switches into the EFI context to run the respective service handler), the UEFI denies a call to `GetNextVariableName` with `EFI_INVALID_PARAMETER` (that part is actually indicated in `dmesg`) even though the request appears to be specification-compliant (and the existing implementation evidently hasn't been an issue on any other notable hardware).
The main issue with fixing it properly is that I'd most likely have to reverse engineer the Windows kernel or the UEFI firmware itself (note to self: I haven't yet checked whether any of the *BSDs can read EFI variables in general and on this hardware in particular) to figure out where the request is going wrong/what Windows is doing different.
It's not impossible, given that one can unpack the UEFI PI firmware image into all the separate modules, but going through them to figure out where variable management is implemented will still take me a few weeks at least (not due to any particular challenge, it's just consuming a lot of time that I don't have right now).
One thing I've gotten used to over the years is Linux complaining that some part of the machine's EFI is buggy no matter what machine it is installed on. Apparently it's just too complicated for hardware manufacturers to get right. I also feel like there is far too much duplicated effort in the industry with everybody making their own version that is slightly buggy in a unique way.
In the old BIOS + ACPI days, the OS carries hardware specific hacks. These hack were buggy and hard to keep to day.
We (the community as a whole) decided it is better leave the hardware specific hacks to the hardware, UEFI was supposed to provide enough abstraction for all we need.
The result is, of course, the hacks with all its bugs are moved to the firmware.
Uncompress it, and dd it to your USB drive. (dd if=FreeBSD-14.0-RELEASE-amd64-memstick.img of=/dev/sdb bs=1m conv=sync, assuming sdb is your usb stick..)
Makes sense! I wonder if there is a way to dynamically watch the Windows call, to compare it with the Linux one, to avoid the tedious reverse engineering. Or if the syntax of Windows GetNextVariableName() use is generally understood/documented?
This could happen either through somehow getting logging from the Windows end, or somehow changing the UEFI to be one you control and logging there, or finding a different BIOS/OS that can read the vars and getting it to log its work.
Makes sense! I wonder if there is a way to dynamically watch the Windows call, to compare it with the Linux one, to avoid the tedious reverse engineering. Or if the syntax of Windows GetNextVariableName() use is generally understood/documented?
The userspace interface is somewhat documented by third-parties (because it is technically internal). However, the important parts happen kernel side, and I'd rather avoid diving too deep into Windows because some very interesting job postings (understandably) have "No exposure to Microsoft code or reverse-engineering of Microsoft software" in them.
I already tried getting to the service handler implementation via Linux, but memory protections made it weird enough that I was even questioning whether it was returning correct raw data when trying to read it from memory (or I have been looking at the wrong set of headers).
Last I checked (which was about a decade ago) Windows doesn't call GetNextVariableName() - it just accesses variables on demand. We should probably handle that in a cleaner way.
It seems like that is no longer the case. I was able to successfully retrieve a non-standard variable using the `UEFIv2` PowerShell module [1] (which is just a thin wrapper around the undocumented `NtEnumerateSystemEnvironmentValuesEx` function) without actually naming the variable in question.
To the untrained layman like me, this sounds like Windows actually is querying via `GetNextVariableName`, because UEFI doesn't seem to offer any other interfaces that aren't "get/set variable by name".
Ok, yes, sounds like it is in that case. Which means figuring out how Linux is doing this differently to Windows, sigh. The easiest validation is to boot Windows under qemu with a debug-enabled EDK2 build to trace the calls.
Comments
As far as I have tracked it down (quite literally up to the point where it switches into the EFI context to run the respective service handler), the UEFI denies a call to `GetNextVariableName` with `EFI_INVALID_PARAMETER` (that part is actually indicated in `dmesg`) even though the request appears to be specification-compliant (and the existing implementation evidently hasn't been an issue on any other notable hardware).
The main issue with fixing it properly is that I'd most likely have to reverse engineer the Windows kernel or the UEFI firmware itself (note to self: I haven't yet checked whether any of the *BSDs can read EFI variables in general and on this hardware in particular) to figure out where the request is going wrong/what Windows is doing different.
It's not impossible, given that one can unpack the UEFI PI firmware image into all the separate modules, but going through them to figure out where variable management is implemented will still take me a few weeks at least (not due to any particular challenge, it's just consuming a lot of time that I don't have right now).
Good news: FreeBSD is able to read the whole variable list.
Bad news: I just wiped my test drive with all the in-progress kernel patches by accident.
Glad to hear FreeBSD worked. Sorry you wiped your drive though!!
One thing I've gotten used to over the years is Linux complaining that some part of the machine's EFI is buggy no matter what machine it is installed on. Apparently it's just too complicated for hardware manufacturers to get right. I also feel like there is far too much duplicated effort in the industry with everybody making their own version that is slightly buggy in a unique way.
It is just moving the complexity around.
In the old BIOS + ACPI days, the OS carries hardware specific hacks. These hack were buggy and hard to keep to day.
We (the community as a whole) decided it is better leave the hardware specific hacks to the hardware, UEFI was supposed to provide enough abstraction for all we need.
The result is, of course, the hacks with all its bugs are moved to the firmware.
More realistically, we now have them in all the places because vendors. :)
FreeBSD can read and manipulate EFI variables in general. It would be interesting to see what happens when you try efivar or efibootmgr in FreeBSD.
Grab the memstick image from here: https://download.freebsd.org/releases/ISO-IMAGES/14.0/
Uncompress it, and dd it to your USB drive. (dd if=FreeBSD-14.0-RELEASE-amd64-memstick.img of=/dev/sdb bs=1m conv=sync, assuming sdb is your usb stick..)
Makes sense! I wonder if there is a way to dynamically watch the Windows call, to compare it with the Linux one, to avoid the tedious reverse engineering. Or if the syntax of Windows GetNextVariableName() use is generally understood/documented?
This could happen either through somehow getting logging from the Windows end, or somehow changing the UEFI to be one you control and logging there, or finding a different BIOS/OS that can read the vars and getting it to log its work.
The userspace interface is somewhat documented by third-parties (because it is technically internal). However, the important parts happen kernel side, and I'd rather avoid diving too deep into Windows because some very interesting job postings (understandably) have "No exposure to Microsoft code or reverse-engineering of Microsoft software" in them.
I already tried getting to the service handler implementation via Linux, but memory protections made it weird enough that I was even questioning whether it was returning correct raw data when trying to read it from memory (or I have been looking at the wrong set of headers).
Last I checked (which was about a decade ago) Windows doesn't call GetNextVariableName() - it just accesses variables on demand. We should probably handle that in a cleaner way.
It seems like that is no longer the case. I was able to successfully retrieve a non-standard variable using the `UEFIv2` PowerShell module [1] (which is just a thin wrapper around the undocumented `NtEnumerateSystemEnvironmentValuesEx` function) without actually naming the variable in question.
To the untrained layman like me, this sounds like Windows actually is querying via `GetNextVariableName`, because UEFI doesn't seem to offer any other interfaces that aren't "get/set variable by name".
[1] https://www.powershellgallery.com/packages/UEFIv2
Ok, yes, sounds like it is in that case. Which means figuring out how Linux is doing this differently to Windows, sigh. The easiest validation is to boot Windows under qemu with a debug-enabled EDK2 build to trace the calls.
wouldn't that be trivial simply using a VM?
I don't know a thing about BIOS internals, so this might be completely irrelevant
Before doing such complicated things, have you tried with RU.EFI?
I haven't, and (as I already said) I also haven't tried any of the BSDs yet. I put both on the list of things to try.