Wednesday, November 23, 2016

Windows 10 KASLR Recovery with TSX

It is possible to break Kernel Address Space Layout Randomization (KASLR) on modern operating systems running on modern x86 CPU's.

One possible way of doing this is to time certain operations when using the Transactional Synchronization Extensions (TSX) instruction set. TSX makes it possible for unprivileged user mode programs to detect whether certain virtual memory pages are mapped or unmapped in kernel mode. It is also possible to detect whether a kernel page is executable or not.

It has been known since at least 2014 that timing attacks against KASLR, using TSX, is possible. This was discussed by Rafal Wojtczuk from Bromium Labs in the blog post TSX improves timing attacks against KASLR. The technique was popularized and presented at Black Hat US-16 by Yeongjin, Sangho, and Taesoo from Georgia Institute of Technology. Their presentation and white paper is found on the Black Hat site. Example code for Linux was published on Github after the talk.

Since no example code was published for Windows I decided to look into this to see if I could make the technique work reliably on Windows as well. The result is presented in this blog post and the resulting code is found my my Github as kaslrfinder.

Test System
I used my NUC to develop and test kaslrfinder. It is the only system capable of TSX I have access to. It also have plenty of memory and an USB-C/Thunderbolt3 port. This made development easy since I could use PCILeech to query the kernel for addresses over Direct Memory Access (DMA).

  • Intel NUC Skull Canyon with a Skylake i7 CPU. 32GB RAM. M.2 SSD.
  • Windows 10 Enterprise version 1607.
  • No Virtualization Based Security (VBS).
The NUC Skull Canyon test system. PCILeech connected via Apple USB-C to Thunderbolt2 adapter.

The Kernel
The Windows 10 kernel is loaded into consecutive large (2MB) pages in the range 0xFFFFF80000000000-0xFFFFF803FFFFFFFF. The location is randomized. In total there are 13 bits of entropy, or 8192 possible locations. The Windows 10 kernel is further randomized within the large (2MB) pages.

The table below contains ten (10) randomly sampled ntoskrnl.exe base addresses. The addresses are shown both in hexadecimal and binary. The columns in the binary denotes changes in paging levels.

0xFFFFF800B2877000   1111111111111111 111110000 000000010 110010100 001110111 000000000000
0xFFFFF801B2A19000   1111111111111111 111110000 000000110 110010101 000011001 000000000000
0xFFFFF8013DC05000   1111111111111111 111110000 000000100 111101110 000000101 000000000000
0xFFFFF8010A81B000   1111111111111111 111110000 000000100 001010100 000011011 000000000000
0xFFFFF8003FE12000   1111111111111111 111110000 000000000 111111111 000010010 000000000000
0xFFFFF8014B21F000   1111111111111111 111110000 000000101 001011001 000011111 000000000000
0xFFFFF80065891000   1111111111111111 111110000 000000001 100101100 010010001 000000000000
0xFFFFF8020D47F000   1111111111111111 111110000 000001000 001101010 001111111 000000000000
0xFFFFF803F0486000   1111111111111111 111110000 000001111 110000010 010000110 000000000000
0xFFFFF80066203000   1111111111111111 111110000 000000001 100110001 000000011 000000000000

The table makes it clear that while the kernel is mapped into large pages its base address is randomized down to 4kB. Windows randomizes the kernel base address within the large pages. The table indicates, in the 5th binary column, that the kernel is randomized between 0-1MB within the large pages. Since the TSX attack technique is dependent on detecting whether a page is executable or not it cannot be used to further detect randomization within the large pages. KASLR entropy is however effectively reduced from 21 bits (13+8) to 8 bits/256 possibilities.

The Modules
The kernel modules and drivers in Windows 10 version 1607 have been observed to load into standard (4kB) pages in the range 0xFFFFF80000000000-0xFFFFF80FFFFFFFFF. In previous Windows 10 versions the modules have been observed to load into the same more limited range as the kernel.

The modules are randomized to an even 64kB (0x10000) boundary. This holds true for most, but not all, modules. The notable exceptions are win32k.sys and related modules which may be randomized outside this area. Special modules like hal.dll and ci.dll have been observed to load at 4kB, 0x1000, boundaries.

How is it possible?
At the very core of the issue is the XBEGIN and XEND TSX instructions first introduced in some Haswell CPU's. Everything between XBEGIN and XEND is guaranteed to execute as an atomic operation. If some other thread interferes with memory while executing in an XBEGIN-XEND TSX block the TSX operation is guranteed to fail and any changes are rolled back and will become void. In fact if anything happens inside the XBEGIN-XEND block the operation will fail and execution will continue at the address of xabort specified in the XBEGIN op. This includes page faults and various types of access violations.

Normally the control is handed over to the kernel when a page fault or an access violation occurs. This is not the case when executing inside a TSX block. TSX is an unprivileged op that can be run in user mode. There are distinct timing differences between:
  • Reading memory from a "forbidden" mapped kernel compared to an unmapped page.
  • Writing memory to a "forbidden" writable kernel page compared to an unmapped or read-only page.
  • Executing in a "forbidden" mapped executable kernel page compared a non executable or unmapped kernel page.
The addresses we wish to test must also be mapped into the page table of the current process. Windows, Linux and macOS all do this for performance reasons. It shall however be noted that Windows is not mapping hypervisor memory into the process page table. As a result of this it is not possible to disclose hypervisor memory addresses with this technique.

In the assembly code listing below only XBEGIN is shown. XEND is not required since the JMP rsi is always guaranteed to fail.

Assembly listing the XBEGIN op, the JMP to the kernel address and the timing before and after measurements.
How do I try it out?
Download kaslrfinder from Github.A CPU with the TSX instruction set is required. Kaslrfinder have only been tested on Skylake CPU's, but nothing indicates it shouldn't work on Kaby Lake CPU's and even on some Haswells if they have TSX enabled.

Please note that kaslrfinder isn't 100% stable. It may at times fail to uncover the address of the kernel or the driver that is searched for. Timing measurement algorithms may be tweaked some more to improve accuracy in the future.

kaslrfinder locates the address of tcpip.sys in Windows 10 1607/14393 patched with the November 2016 patches.

What can be done to mitigate?
First please note that Kernel Address Space Randomization (KASLR) is an additional protection mechanism designed to make some kernel security issues harder to exploit. Finding out about kernel addresses won't do anything bad unless another security issue exists as well. kaslrfinder is not a malicious program. Please also note that it is not possible to use this technique to find out the address of the actual kernel itself for reasons discussed above.

What can users do to mitigate? Answer: nothing.

What can operating system vendors do? Probably several things, but finding a solution without any downsides will be hard. The best thing for now might actually be to do nothing. I plan to follow up on this in the RS2 Windows insider releases.

It will be interesting to see how this will develop in the future. Will TSX be kept as-is?, will constant timing be guaranteed? or will the instruction behavior change in future CPU's?

This should no longer work if post-meltdown attack patches have been applied to the operating system.
Windows 10 have since 1903 altered the kernel alignment to be mapped into the base of a 2MB large page.

Monday, November 14, 2016

Disable Virtualization Based Security (VBS) on auto-booting systems

I this post I will show how it's possible to disable Windows 10 Virtualization Based Security (VBS), Credential and Device Guard, by corrupting in-memory structures prior to operating system boot.

For this attack to succeed the target computer must not be protected by a pre-boot authentication password. Auto-booting Bitlocker with TPM and/or Network unlock will work. The target computer is also required to have Direct Memory Access (DMA) capable ports and a BIOS which will allow DMA before operating system boot.

The Target
  • Intel NUC Skull Canyon with a Skylake i7 CPU. 32GB RAM. M.2 SSD.
  • Windows 10 Enterprise version 1607.

The NUC have two options for DMA. The obvious choice is the USB-C port at the back which is capable of Thunderbolt 3. Thunderbolt is however secure by default on the NUC - which is unfortunate for us. The Thunderbolt to PCI-Express adapters I use also doesn't seem to be working prior to OS boot even in the less secure Thunderbolt Legacy Mode.

The second option is to use one of the M.2 slots inside the NUC. M.2 is pretty much just another from factor for PCI-Express. The M.2 slots are easily accessible; just unscrew the bottom cover and open up the NUC. In this example I will connect PCILeech to the available M.2 slot.

The M.2 to PCIe adapter and PCILeech mounted in one of the NUC M.2 slots.

Windows 10 Enterprise version 1607 was installed and the computer joined to an AD. Group Policies enabling virtualization based security with UEFI lock and Bitlocker with with secure boot validation was deployed to the system.

The relevant GPO VBS and Bitlocker settings deployed to Windows on the NUC.

The result is that Windows 10 enabled the virtualization based security features. It's not possible to disable them by altering the GPO since the UEFI lock was configured. The test system have a BIOS password configured so it shouldn't be possible to get into the BIOS to disable it. The system should be secure even though auto-booting Bitlocker is deployed.

If we check the task manager we see that the secure system is up and running as well as other secure processes such as LsaIso.

The Secure System including LsaIso is running on the NUC together with Bitlocker.

Overwriting the DMAR ACPI table
The DMAR ACPI table is an in-memory reporting structure used to report the memory mapped location of the IOMMU to the operating system. DMAR is short for DMA Remapping. If the OS cannot read the configuration data from a valid DMAR table it cannot locate the IOMMU and cannot enable the virtualization features.

The DMAR table is not protected or signed. It loads at a predictable memory address on the NUC (also on other tested hardware). It is possible to overwrite the DMAR table on the NUC before the OS boots.

This attack was discussed on a theoretical level already back in 2009 in the paper Another Way to Circumvent Intel® Trusted Execution Technology by Rutkowska et al.

It's possible to search for the DMAR table in memory by using PCILeech since it's starting with the signature DMAR. It turns out that while entirely possible to do so it was a bit problematic on the tested hardware. When PCILeech encounters unreadable memory it needs to be power-cycled before able to read memory again. It was easier to boot into Ubuntu on an USB stick and check the location of the DMAR table with the dmesg command.

The DMAR ACPI table is located at address: 0x3A529CB0.
Once knowledge of the memory address of the DMAR table has been gained it's possible to overwrite it. The address is usually completely static between reboots. It might change if extensive changes are made to the BIOS configuration - such as enabling/disabling secure boot though.

We'll overwrite the beginning of the DMAR table before Windows starts to boot. Just issue the command: 
pcileech.exe write -min 0x3a529cb0 -in 000000000000000000000000000000000000000000000000 
which will effectively overwrite the start of the DMAR table with 24 bytes of null data.

The obligatory before/after image. The original DMAR table to the left. The overwritten DMAR table to the right.

The Result
Even though the beginning of the DMAR table is now overwritten Bitlocker will unlock the OS disk allowing Windows to boot. Windows don't find the DMAR table so it can't locate the IOMMU. Even though UEFI lock has been configured Windows will just disable virtualization based security features and continue to boot normally.

Windows with Bitlocker. Credential and Device Guard not started despite configuration. PCILeech kernel module loaded.
  • It's possible to nuke the DMAR table on other hardware as well.
  • Virtualization Based Security isn't designed to fully protect against physical and firmware based attacks.
  • Issue mentioned to MSRC in July as part of pre-existing case. Case closed after DEF CON talk Direct Memory Attack the Kernel.
  • BIOS version: KYSKLi70.86A.0037.2016.0603.1032.
  • Did I miss something? Please let me know.

  • Computers with active DMA ports and auto-booting full disk encryption might be at risk if physical access could be gained.
  • Windows 10 Virtualization Based Security might not fully protect against physical DMA attacks with PCILeech on autobooting systems.
  • It's possible to disable unused M.2 slots and set a BIOS password on the NUC if one wish to be extra secure.