Tuesday, March 27, 2018

Total Meltdown?

Did you think Meltdown was bad? Unprivileged applications being able to read kernel memory at speeds possibly as high as megabytes per second was not a good thing.

Meet the Windows 7 Meltdown patch from January. It stopped Meltdown but opened up a vulnerability way worse ... It allowed any process to read the complete memory contents at gigabytes per second, oh - it was possible to write to arbitrary memory as well.

No fancy exploits were needed. Windows 7 already did the hard work of mapping in the required memory into every running process. Exploitation was just a matter of read and write to already mapped in-process virtual memory. No fancy APIs or syscalls required - just standard read and write!

Accessing memory at over 4GB/s, dumping to disk is slower due to disk transfer speeds.

How is this possible?
In short - the User/Supervisor permission bit was set to User in the PML4 self-referencing entry. This made the page tables available to user mode code in every process. The page tables should normally only be accessible by the kernel itself.

The PML4 is the base of the 4-level in-memory page table hierarchy that the CPU Memory Management Unit (MMU) uses to translate the virtual addresses of a process into physical memory addresses in RAM. For more in-depth information about paging please have a look at Getting Physical: Extreme abuse of Intel based Paging Systems - Part 1 and Part 2.

PML4 self-referencing entry at offset 0xF68 with value 0x0000000062100867.

Windows have a special entry in this topmost PML4 page table that references itself, a self-referencing entry. In Windows 7 the PML4 self-referencing is fixed at the position 0x1ED, offset 0xF68 (it is randomized in Windows 10). This means that the PML4 will always be mapped at the address: 0xFFFFF6FB7DBED000 in virtual memory. This is normally a memory address only made available to the kernel (Supervisor). Since the permission bit was erroneously set to User this meant the PML4 was mapped into every process and made available to code executing in user-mode.

"kernel address" memory addresses mapped in every process as user-mode read/write pages.

Once read/write access has been gained to the page tables it will be trivially easy to gain access to the complete physical memory, unless it is additionally protected by Extended Page Tables (EPTs) used for Virtualization. All one has to do is to write their own Page Table Entries (PTEs) into the page tables to access arbitrary physical memory.

The last '7' in the PML4e 0x0000000062100867 (from above example) indicates that bits 0, 1, 2 are set, which means it's Present, Writable and User-mode accessible as per the description in the Intel Manual.

Excerpt from the Intel Manual, if bit 2 is set to '1' user-mode access are permitted.

Can I try this out myself?
Yes absolutely. The technique has been added as a memory acquisition device to the PCILeech direct memory access attack toolkit. Just download PCILeech and execute it with device type: -device totalmeltdown on a vulnerable Windows 7 system.

Dump memory to file with the command: pcileech.exe dump -out memorydump.raw -device totalmeltdown -v -force .

If you have the Dokany file system driver installed you should be able to mount the running processes as files and folders in the Memory Process File System - with the virtual memory of the kernel and the processes as read/write.

To mount the processes issue the command: pcileech.exe mount -device totalmeltdown .

Please remember to re-install your security updates if you temporarily uninstall the latest one in order to test this vulnerability.

A vulnerable system is "exploited" and the running processes are mounted with PCILeech.
Process memory maps and PML4 are accessed.

Is my system vulnerable?
Only Windows 7 x64 systems patched with the 2018-01 or 2018-02 patches are vulnerable. If your system isn't patched since December 2017 or if it's patched with the 2018-03 2018-03-29 patches or later it will be secure.

Other Windows versions - such as Windows 10 or 8.1 are completely secure with regards to this issue and have never been affected by it.

Other
I discovered this vulnerability just after it had been patched in the 2018-03 Patch Tuesday. I have not been able to correlate the vulnerability to known CVEs or other known issues.

Updates
Windows 2008R2 was vulnerable as well.
OOB security update released to fully resolve the vulnerability on 2018-03-29. CVE-2018-1038. Apply immediately if affected!

Timeline
2018-03-xx--25: Issue identified in Windows 7 x64. Issue seemed to be patched already. PoC coded. Contacted MSRC with technical description asking if OK to publish a blog entry or if I should hold off publication.
2018-03-26: Green light given by MSRC for me to publish blog entry.
2018-03-27: Published blog entry and PoC.
2018-03-28: Found out that the March patches only partially resolved the vulnerability. Contacted MSRC again.
2018-03-29: OOB security update released by Microsoft. CVE-2018-1038. Apply immediately if affected!

Huge Thank You to everyone at Microsoft that worked hard to resolve this issue. It is super impressive to be able to be able to roll out a complex kernel update in little over a day. It was never my intention to release a fairly potent kernel 0-day publicly. I hope the above timeline explains how this could happen.

Monday, March 12, 2018

Introducing the Memory Process File System for PCILeech


The Memory Process File System for PCILeech is an easy and convenient way to quickly look into memory dumps. The processes in a memory dump and their virtual memory should be mapped as files and folders in less than a second.

Click around the processes, look at their memory maps and corresponding virtual memory! Oh - if you run it in live mode with a supported PCILeech FPGA device you'll be able to write to memory as well! Super convenient if your target system employs software based anti-forensic or anti-cheating functionality since this is all handled in hardware on the target!

Using the Memory Process File System for PCILeech to explore Windows processes in a memory dump file.


Cool! So how do I try it out?
Check out and download PCILeech on Github. PCILeech is open source and totally free of charge! You'll find both source code and pre-compiled binaries on Github.

How does it work?
It works by parsing in-memory page tables which are used by the CPU to translate the virtual memory of a process into physical memory. Since page tables are hardware dependent this means that the Memory Process File System is able to map processes from all 64-bit operating systems running on x64 hardware, if the base of the process page table called PageDirectoryBase or PML4 or CR3 is known.

If parts of the page tables, or other process memory, doesn't exist in the memory, it may have been swapped out to the swap on disk, analysis of that part of memory will fail.

If analyzing Windows memory dumps additional analysis will be performed to identify running processes, modules and which memory regions they belong to.

What about Functionality and Limitations?
- The Memory Process File System is currently only supported when running PCILeech on Windows.
- x64 64-bit target operating systems only, no 32-bit, no ARM.
- Read-only mode on memory dump files, read-write mode if PCILeech FPGA is used on a live system.
- Automatic process identification only in Windows memory dumps.
- Automatic identification of EPROCESS, PEB and DLL addresses in Windows memory dumps.
- May fail on memory dumps taken from Virtual Machines, such as VirtualBox.
- May fail for various other reasons as well.

Looking at the Linux kernel by manually specifying a valid CR3 value (PageDirectoryBase/PML4).


How Forensically Secure is it?
Memory swapped out to disk will also be invisible to the Memory Process File System - there is no guarantee that everything will be mapped.

There is no way for user-mode malware to hide from the Memory Process File System though. Since the Memory Process File System analyzes page tables, which are used by the CPU and thus must remain in memory somewhat unobfuscated.

The Memory Process File System however employs some tricks to filter out what looks like invalid page tables. This means that malware running with kernel privileges may hide from the Memory Process File System at the moment

Acquiring Memory Dumps
If you have access to a PCILeech supported FPGA device this is the preferred acquisition method. Insert the PCILeech FPGA device in your target computer. Then identify the size of the physical memory address space. This is likely to be slightly larger than the amount of RAM in the computer. Do this by running the command "pcileech.exe probe". This will give an output similar to the one below. Here the maximum address 0x21E5FFFFF is important.
Enumerating target system memory with a PCILeech FPGA device (PCIeScreamer in image).
Now we are able to dump the memory. Dump using "pcileech.exe dump -v -force -max 0x21E5FFFFF -out my_dump.raw". The -force option tells PCILeech not to abort when it encounters unreadable memory, such as memory holes between 3-4 GB or memory protected with Vt-d.

MoonSols DumpIt is a software one-click dumper that is easy to run. It may however create corrupt memory dumps on more recent Windows 10 computers, hence it's recommended to use WinPMEM on Windows 10.

WinPMEM is a software memory dumping utility which is part of the Google Rekall memory forensics project on Github. You'll find a signed WinPMEM to download here. Execute, from an administrator elevated command prompt, "winpmem-2.1.post4.exe --format raw -o dump-pmem.raw.aff4". WinPMEM will dump to an .aff4 volume. Open this volume in 7-Zip and unpack the memory dump named PhysicalMemory.

Reading and Writing to Memory
Reading and writing to virtual memory can be done in two ways. Either open the vmem file in the process directory, which contains the whole process virtual memory, or open a file in the vmemd directory which contains individual virtual memory mappings. The memory map is found in the map file.
The memory map, vmem memory file and vmemd directory for the Notepad process.
If using PCILeech FPGA with a hardware based target the virtual memory files will be writable. Please note that writes go the the corresponding physical memory which may be shared between multiple processes! As an example, alterations written to ntdll.dll in one process is likely to be written into the ntdll.dll of all processes.
Looking at the notepad.exe virtual memory in hexedit in Ubuntu WSL.
When reading to, or writing to a kernel address, for example 0xFFFFF801'10000000 please open the vmem file in your favorite hex editor and navigate to the address as seen when cutting away the first four F's. That is 0xF801'1000000 in the example from above. The topmost 16-bits are what is called sign-extended so this won't matter. It's still the same address. This is done since Windows don't support file sizes above 63-bits (file size becomes negative).
Looking at notepads EPROCESS struct in kernel memory. Note that we look at it in the System process (PID 4) and that the leading four (4) FFFF are skipped from the address.

Special Files
The virt2phys file is a special always writable file. The first 64-bit value contains the virtual address and the second 64-bit value is the corresponding physical address. Whenever a new virtual address gets written the physical address will update.
notepads 0x7FF660160000 virtual address maps to the physical address 0x14C40D000.

The win-process file contains the address of the EPROCESS kernel struct that belongs to the process. Since the EPROCESS struct resides in kernel memory one must look for this one in the System (PID 4) process virtual memory.

The win-peb file contains the address of the PEB (Process Environment Block), which resides in process memory. The win-entry contains the process entry point, win-modules the modules (.dlls) in the PEB Ldr list.

The pml4 file contains the physical address of the PML4 a.k.a. the CR3 register a.k.a. the Page Directory Base.

The Future
The future of the Memory Process File System will depend on how useful it is. If useful I see a lot of possibilities, somewhat limited by time available to implement them. The Memory Process File System is not meant to become a serious forensic file system, that space will continue to be reserved by industry standard tools, such as the excellent Volatility. The Memory Process File System should nevertheless provide convenient easy-to use access if performing a quick analysis, or be helpful when patching live processes via DMA.

Comments, suggestions and ideas are very welcome. PCILeech in both source and binary distributions is found on Github.  Also check out Twitter for updated information and announcements.

Wednesday, August 30, 2017

Attacking UEFI

Unlike macs many PCs are likely to be vulnerable to pre-boot Direct Memory Access (DMA) attacks against UEFI. If an attack is successful on a system configured with secure boot - then the chain of trust is broken and secure boot becomes insecure boot.

If code execution is gained before the operating system is started further compromise of the not yet loaded operating system may be possible. As an example it may be possible to compromise a Windows 10 system running Virtualization Based Security (VBS) with Device Guard. This have already been researched by Dmytro Oleksiuk.

This post will focus on attacking UEFI over DMA and not potential further compromises of the system.

What is UEFI?
UEFI is short for Unified Extensible Firmware Interface. It is the firmware that is running on the computer before the operating system is booted. UEFI is responsible for detecting memory, disks and other hardware required to boot the operating system. UEFI is a small operating system in itself. It's also sometimes a bit sloppily called the BIOS.

The Targets
A brand new Intel NUC i3 "Kaby Lake" purchased in June. 8GB RAM, Win10 1703 with Secure Boot, Bitlocker+TPM, Virtualization Based Security (VBS) Device Guard is enabled. BIOS revision: BNKBL357.86A.0036.2017.0105.1112. DMA access via internal M.2 slot.

An older Lenovo T430, 8GB RAM, Win10 1703 with Secure Boot, Bitlocker+TPM, Virtualization Based Security (VBS) Device Guard is enabled. DMA access via ExpressCard slot.

T430 to the left, NUC to the right.
The Problem
The root problem is that many UEFIs still do not protect themselves against DMA attacks, despite the hardware (VT-d/IOMMU) to do so being included in all CPUs for many years. The screenshot below shows PCILeech first searching the memory of a target computer over DMA trying to find where to hook into UEFI. Once inside it's easy to dump memory (also shown) and do other evilness - such as executing arbitrary code despite secure boot being enabled.

Loading a PCILeech module into UEFI, dumping the memory and unloading the module.
The Attack
Taking control is a simple matter of finding the correct memory structures and overwriting them if DMA access is allowed. This process is automated with PCILeech. It's possible to automatically search for the memory address of the EFI system table "IBI SYST" - or even better specify it directly to PCILeech. The EFI System Table contains the location of the EFI boot services table "BOOTSERV" which contains many useful function pointers. The boot services functions are useful for both hooking and also calling into from our implanted module.

In the example below the boot services function SignalEvent() is hooked. Once the PCILeech "kernel" module for UEFI is inserted it's possible to use it to dump memory and execute code - just as any normal PCILeech kernel module. In the example below the PCILeech UEFI implant uefi_textout is called multiple times. The output is printed on the screen of the victim computer.

The text HELLO FROM EVIL PCILEECH IMPLANT !!! is printed multiple times after the PCILeech module for UEFI have been inserted.
Once the attack was completed the kmdexit command was issued to PCILeech and the UEFI implant was unloaded. In this case Windows will start booting as shown below. If targeting the operating system loaded it's better to hook ExitBootServices() - which is called by the EFI based operating system loader when the operating system is taking over control of the computer from UEFI. At this point in time it will be possible for malicious code to modify the operating system loader.

Windows is booting normally once the PCILeech UEFI module is unloaded.
Can I try it myself?
Absolutely! The code is available as a part of the open source PCILeech Direct Memory Access Attack Toolkit on Github.

Conclusions
UEFI DMA attacking with PCILeech is now public, inexpensive and easy to perform. DMA attacks agaunst UEFI are no longer theoretical.

Vendors should enable VT-d to protect against DMA attacks.

Further compromise of the operating system may be possible. It may not be possible to rely on Virtualization Based Security if having a vulnerable UEFI.

Wednesday, January 11, 2017

Attacking UEFI Runtime Services and Linux

Attackers with physical access are able to attack the firmware on many fully patched computers with DMA - Direct Memory Access. Once code execution is gained in UEFI/EFI Runtime Services it is possible to use this foothold to take control of a running Linux system.

The Linux 4.8 kernel fully randomizes the physical memory location of the kernel. There is a high likelyhood that the kernel will be randomized above 4GB on computers with sufficient memory. This means that DMA attack hardware only capable of 32-bit addressing (4GB), such as PCILeech, cannot reach the Linux kernel directly.

Since the EFI Runtime Services are usually located below 4GB they offer a way into Linux on high memory EFI booting systems.

Please see the video below for an example of how an attack may look like.



What are the EFI Runtime Services?
UEFI on PCs, EFI on macs, is the modern day BIOS. UEFI is short for Unified Extensible Firmware Interface. UEFI is responsible for detecting hardware and configuring devices so that the control can be handed over to the operating system that should be loaded.

Two main components of UEFI are the Boot Services and the Runtime Services. Very early on the operating system calls ExitBootServices. After this Boot Services are no longer in use.

The EFI Runtime Services lives on even after the operating system is loaded and running though. They provide various functionality that the operating system can call into. The UEFI specification specifies a fixed set of functions that the Runtime Services should provide to the operating system, as seen below.
The UEFI Runtime Services according to the UEFI specification.
Initially the location of the Runtime Services functionality is communicated to the operating system via the Runtime Services table. The physical address for each function is 64-bit/8-byte long and is stored in memory as little endian. The memory addresses are however in the 32-bit range on all systems looked at so far. The physical memory addresses also seems to be completely static, i.e. no randomization occurs between reboots.

Linux later on maps these addresses into virtual address space and overwrites the initial addresses in the table with the corresponding virtual addresses. An example of the initial table and the table altered by Linux is shown side-by-side below.

The EFI Runtime services table, original to the left, modified by Linux to the right.
The Attack
What if we overwrite the EFI Runtime Services with our own code by using DMA? Or even better what if we overwrite function pointers in the EFI Runtime Services table to point to previously inserted attack code?

Code execution is gained on the target system when the operating system calls into EFI Runtime Services - for example whenever it's reading an EFI variable. Code execution is gained in a special context, in which pages in lower memory are mapped 1:1 between virtual and physical addresses. The Linux kernel is reachable at its normal virtual addresses. Execution is gained in ring0 / supervisor mode.

It is however not possible to call into all functions in the Linux kernel. Some functions will fail due to the special EFI context Linux have set up for Runtime Services to execute in. The way around this is to patch a "random" hook function in the Linux kernel. When a "normal" kernel thread hits that hook "normal" kernel code execution is gained...

The target system running Ubuntu 16.10 with the PCILeech attack hardware connected (to the left). Detailed view of PCILeech (to the right).

The Targets
This have been tested on a Lenovo T430 with 8GB memory and on an Intel NUC Skull Canyon with 32GB memory. The ExpressCard slot was used to gain DMA access on the T430 On the NUC the Thunderbolt mode was set to "Legacy" in the "BIOS" settings - enabling DMA access via Thunderbolt3/USB-C.

The T430 is very straight forward. Just insert the PCILeech device and issue the pcileech.exe kmdload -kmd linux_x64_efi command. PCILeech will search for the EFI Runtime Services table and hook it. The user will be asked to do something resulting in a call to the Runtime Services - triggering the hook.

The NUC is different. PCILeech will encounter unreadable memory before the target is found and will fail. Luckily the location of the Runtime Services table is static and won't change between reboots. This goes for all tested systems. The easiest way to find out the location is by USB-booting a live Linux in EFI mode on the same system, or on a similar system. Once booted issue the command cat /sys/firmware/efi/runtime to see the physical address of the Runtime Services table. Once the address is known as 0x3b294e18 we may issue the command pcileech.exe kmdload -kmd linux_x64_efi -min 0x3b294000 -max 0x3b295000 .
The NUC setup. Displaying the NUC EFI Runtime Services table on the Surface attack computer.

Notes
To try this out yourself please have a look at PCILeech at Github.

The attack is not 100% stable in the sense that the exploit always works. Sometimes the search for the Runtime Services table will fail and sometimes it's hard to trigger a call into the Runtime Services. Target system crashes are rare though.


Conclusions
Linux 4.8 based operating systems, such as Ubuntu 16.10, are no longer completely secure from PCILeech.

Even though your laptop may not have an ExpressCard slot chances are that it will have a mini-PCIe or M.2 slot for a WiFi card if the back cover is unscrewed ...

Operating systems cannot protect themselves from DMA attacks by putting everything of interest in memory above the 32-bit / 4GB address limit. 32-bit hardware such as PCILeech may attack firmware code and data residing below 4GB. Other malicious hardware may also reach into 64-bit address space.

Operating systems should protect both themselves and firmware, such as the Runtime Services, from malicious devices by enabling VT-d.

Thursday, December 15, 2016

macOS FileVault2 Password Retrieval

macOS FileVault2 let attackers with physical access retrieve the password in clear text by plugging in a $300 Thunderbolt device into a locked or sleeping mac. The password may be used to unlock the mac to access everything on it. To secure your mac just update it with the December 2016 patches.

Anyone including, but not limited to, your colleagues, the police, the evil maid and the thief will have full access to your data as long as they can gain physical access - unless the mac is completely shut down. If the mac is sleeping it is still vulnerable.

Just stroll up to a locked mac, plug in the Thunderbolt device, force a reboot (ctrl+cmd+power) and wait for the password to be displayed in less than 30 seconds! Check out the demo video below:


How is this possible?
At the very core of this issue there are two separate issues.

The first issue is that the mac does not protect itself against Direct Memory Access (DMA) attacks before macOS is started. EFI which is running at this early stage enables Thunderbolt allowing malicious devices to read and write memory. At this stage macOS is not yet started. macOS resides on the encrypted disk - which must be unlocked before it can be started. Once macOS is started it will enable DMA protections by default.

The second issue is that the the FileVault password is stored in clear text in memory and that it's not automatically scrubbed from memory once the disk is unlocked. The password is put in multiple memory locations - which all seems to move around between reboots, but within a fixed memory range.

This makes it easy to just plug in the DMA attack hardware and reboot the mac. Once the mac is rebooted the DMA protections that macOS previously enabled are dropped. The memory contents, including the password, is still there though. There is a time window of a few seconds before the memory containing the password is overwritten with new content.

The PCILeech DMA attack hardware attached to a mac victim.
After the PCILeech hardware is connected just run the mac_fvrecover command on the attacker computer.
Retrieving the FileVault password with PCILeech. The correct password is DonaldDuck.

In Details
The password, when entered, is stored in memory as unicode. Every 2nd byte will be zero if a password consisting only of ascii characters is used. Enter a "random" phrase, not naturally occurring in memory, at the password prompt. In this example the phrase eerrbbnn is used. In memory this is stored as 6500650072007200620062006e006e. Search for this using PCILeech and you'll notice that the phrase is stored in multiple memory locations as per the example below.
Searching the mac memory for the test phrase eerrbbnn.

After finding the memory locations it's possible to have a look at it. One might have to re-attach the attack device to the mac if the first read fails.

If the locations found are checked out the password will be clearly visible. In addition to this other signatures that may be scanned for are also found, such as phd0 at the beginning of the memory page read.
The phrase is clearly visible at the memory location.

Can I try it out myself?
Yes - Absolutely! Download PCILeech from Github and purchase the hardware. Password recovery have been tested and found to work on multiple macbooks and macbook airs (all with Thunderbolt 2). The attack is not tested on more recent macs with USB-C.

Password recovery may fail if the user uses special (non ascii) characters in the password. In those cases a memory dump will be saved so that it can be manually searched for the password.

Please note that trying this on other macs than your own might not be legal.

Other Notes
Recovering the password is just one of the things that are possible unless the security update is applied. Since EFI memory can be overwritten it is possible to do more evil ...

The disclosure timeline is as follows:
  • End of July: Issue found.
  • August 5th: PCILeech presented and released at DEF CON 24. (FileVault issue not mentioned).
  • August 15th: Apple notified.
  • August 16th: Apple confirmed issue and asked to hold off disclosure.
  • December 13th: Apple released macOS 10.12.2 which contains the security update. At least for some hardware - like my MacBook Air.

Conclusion
The solution Apple decided upon and rolled out is a complete one. At least to the extent that I have been able to confirm. It is no longer possible to access memory prior to macOS boot. The mac is now one of the most secure platforms with regards to this specific attack vector.

Update:
This issue is classified as: CVE-2016-7585

Wednesday, November 23, 2016

Windows 10 KASLR Recovery with TSX

It is possible to break Kernel Address Space Layout Randomization (KASLR) on modern operating systems running on modern x86 CPU's.

One possible way of doing this is to time certain operations when using the Transactional Synchronization Extensions (TSX) instruction set. TSX makes it possible for unprivileged user mode programs to detect whether certain virtual memory pages are mapped or unmapped in kernel mode. It is also possible to detect whether a kernel page is executable or not.

It has been known since at least 2014 that timing attacks against KASLR, using TSX, is possible. This was discussed by Rafal Wojtczuk from Bromium Labs in the blog post TSX improves timing attacks against KASLR. The technique was popularized and presented at Black Hat US-16 by Yeongjin, Sangho, and Taesoo from Georgia Institute of Technology. Their presentation and white paper is found on the Black Hat site. Example code for Linux was published on Github after the talk.

Since no example code was published for Windows I decided to look into this to see if I could make the technique work reliably on Windows as well. The result is presented in this blog post and the resulting code is found my my Github as kaslrfinder.

Test System
I used my NUC to develop and test kaslrfinder. It is the only system capable of TSX I have access to. It also have plenty of memory and an USB-C/Thunderbolt3 port. This made development easy since I could use PCILeech to query the kernel for addresses over Direct Memory Access (DMA).

  • Intel NUC Skull Canyon with a Skylake i7 CPU. 32GB RAM. M.2 SSD.
  • Windows 10 Enterprise version 1607.
  • No Virtualization Based Security (VBS).
The NUC Skull Canyon test system. PCILeech connected via Apple USB-C to Thunderbolt2 adapter.

The Kernel
The Windows 10 kernel is loaded into consecutive large (2MB) pages in the range 0xFFFFF80000000000-0xFFFFF803FFFFFFFF. The location is randomized. In total there are 13 bits of entropy, or 8192 possible locations. The Windows 10 kernel is further randomized within the large (2MB) pages.

The table below contains ten (10) randomly sampled ntoskrnl.exe base addresses. The addresses are shown both in hexadecimal and binary. The columns in the binary denotes changes in paging levels.

0xFFFFF800B2877000   1111111111111111 111110000 000000010 110010100 001110111 000000000000
0xFFFFF801B2A19000   1111111111111111 111110000 000000110 110010101 000011001 000000000000
0xFFFFF8013DC05000   1111111111111111 111110000 000000100 111101110 000000101 000000000000
0xFFFFF8010A81B000   1111111111111111 111110000 000000100 001010100 000011011 000000000000
0xFFFFF8003FE12000   1111111111111111 111110000 000000000 111111111 000010010 000000000000
0xFFFFF8014B21F000   1111111111111111 111110000 000000101 001011001 000011111 000000000000
0xFFFFF80065891000   1111111111111111 111110000 000000001 100101100 010010001 000000000000
0xFFFFF8020D47F000   1111111111111111 111110000 000001000 001101010 001111111 000000000000
0xFFFFF803F0486000   1111111111111111 111110000 000001111 110000010 010000110 000000000000
0xFFFFF80066203000   1111111111111111 111110000 000000001 100110001 000000011 000000000000

The table makes it clear that while the kernel is mapped into large pages its base address is randomized down to 4kB. Windows randomizes the kernel base address within the large pages. The table indicates, in the 5th binary column, that the kernel is randomized between 0-1MB within the large pages. Since the TSX attack technique is dependent on detecting whether a page is executable or not it cannot be used to further detect randomization within the large pages. KASLR entropy is however effectively reduced from 21 bits (13+8) to 8 bits/256 possibilities.

The Modules
The kernel modules and drivers in Windows 10 version 1607 have been observed to load into standard (4kB) pages in the range 0xFFFFF80000000000-0xFFFFF80FFFFFFFFF. In previous Windows 10 versions the modules have been observed to load into the same more limited range as the kernel.

The modules are randomized to an even 16kB (0x10000) boundary. This holds true for most, but not all, modules. The notable exceptions are win32k.sys and related modules which may be randomized outside this area. Special modules like hal.dll and ci.dll have been observed to load at 4kB, 0x1000, boundaries.

How is it possible?
At the very core of the issue is the XBEGIN and XEND TSX instructions first introduced in some Haswell CPU's. Everything between XBEGIN and XEND is guaranteed to execute as an atomic operation. If some other thread interferes with memory while executing in an XBEGIN-XEND TSX block the TSX operation is guranteed to fail and any changes are rolled back and will become void. In fact if anything happens inside the XBEGIN-XEND block the operation will fail and execution will continue at the address of xabort specified in the XBEGIN op. This includes page faults and various types of access violations.

Normally the control is handed over to the kernel when a page fault or an access violation occurs. This is not the case when executing inside a TSX block. TSX is an unprivileged op that can be run in user mode. There are distinct timing differences between:
  • Reading memory from a "forbidden" mapped kernel compared to an unmapped page.
  • Writing memory to a "forbidden" writable kernel page compared to an unmapped or read-only page.
  • Executing in a "forbidden" mapped executable kernel page compared a non executable or unmapped kernel page.
The addresses we wish to test must also be mapped into the page table of the current process. Windows, Linux and macOS all do this for performance reasons. It shall however be noted that Windows is not mapping hypervisor memory into the process page table. As a result of this it is not possible to disclose hypervisor memory addresses with this technique.


In the assembly code listing below only XBEGIN is shown. XEND is not required since the JMP rsi is always guaranteed to fail.

Assembly listing the XBEGIN op, the JMP to the kernel address and the timing before and after measurements.
How do I try it out?
Download kaslrfinder from Github.A CPU with the TSX instruction set is required. Kaslrfinder have only been tested on Skylake CPU's, but nothing indicates it shouldn't work on Kaby Lake CPU's and even on some Haswells if they have TSX enabled.

Please note that kaslrfinder isn't 100% stable. It may at times fail to uncover the address of the kernel or the driver that is searched for. Timing measurement algorithms may be tweaked some more to improve accuracy in the future.

kaslrfinder locates the address of tcpip.sys in Windows 10 1607/14393 patched with the November 2016 patches.

What can be done to mitigate?
First please note that Kernel Address Space Randomization (KASLR) is an additional protection mechanism designed to make some kernel security issues harder to exploit. Finding out about kernel addresses won't do anything bad unless another security issue exists as well. kaslrfinder is not a malicious program. Please also note that it is not possible to use this technique to find out the address of the actual kernel itself for reasons discussed above.

What can users do to mitigate? Answer: nothing.

What can operating system vendors do? Probably several things, but finding a solution without any downsides will be hard. The best thing for now might actually be to do nothing. I plan to follow up on this in the RS2 Windows insider releases.

It will be interesting to see how this will develop in the future. Will TSX be kept as-is?, will constant timing be guaranteed? or will the instruction behavior change in future CPU's?

Update:
This should no longer work if post-meltdown attack patches have been applied to the operating system.

Monday, November 14, 2016

Disable Virtualization Based Security (VBS) on auto-booting systems

I this post I will show how it's possible to disable Windows 10 Virtualization Based Security (VBS), Credential and Device Guard, by corrupting in-memory structures prior to operating system boot.

For this attack to succeed the target computer must not be protected by a pre-boot authentication password. Auto-booting Bitlocker with TPM and/or Network unlock will work. The target computer is also required to have Direct Memory Access (DMA) capable ports and a BIOS which will allow DMA before operating system boot.

The Target
  • Intel NUC Skull Canyon with a Skylake i7 CPU. 32GB RAM. M.2 SSD.
  • Windows 10 Enterprise version 1607.

The NUC have two options for DMA. The obvious choice is the USB-C port at the back which is capable of Thunderbolt 3. Thunderbolt is however secure by default on the NUC - which is unfortunate for us. The Thunderbolt to PCI-Express adapters I use also doesn't seem to be working prior to OS boot even in the less secure Thunderbolt Legacy Mode.

The second option is to use one of the M.2 slots inside the NUC. M.2 is pretty much just another from factor for PCI-Express. The M.2 slots are easily accessible; just unscrew the bottom cover and open up the NUC. In this example I will connect PCILeech to the available M.2 slot.

The M.2 to PCIe adapter and PCILeech mounted in one of the NUC M.2 slots.

Windows 10 Enterprise version 1607 was installed and the computer joined to an AD. Group Policies enabling virtualization based security with UEFI lock and Bitlocker with with secure boot validation was deployed to the system.

The relevant GPO VBS and Bitlocker settings deployed to Windows on the NUC.

The result is that Windows 10 enabled the virtualization based security features. It's not possible to disable them by altering the GPO since the UEFI lock was configured. The test system have a BIOS password configured so it shouldn't be possible to get into the BIOS to disable it. The system should be secure even though auto-booting Bitlocker is deployed.

If we check the task manager we see that the secure system is up and running as well as other secure processes such as LsaIso.

The Secure System including LsaIso is running on the NUC together with Bitlocker.

Overwriting the DMAR ACPI table
The DMAR ACPI table is an in-memory reporting structure used to report the memory mapped location of the IOMMU to the operating system. DMAR is short for DMA Remapping. If the OS cannot read the configuration data from a valid DMAR table it cannot locate the IOMMU and cannot enable the virtualization features.

The DMAR table is not protected or signed. It loads at a predictable memory address on the NUC (also on other tested hardware). It is possible to overwrite the DMAR table on the NUC before the OS boots.

This attack was discussed on a theoretical level already back in 2009 in the paper Another Way to Circumvent Intel® Trusted Execution Technology by Rutkowska et al.

It's possible to search for the DMAR table in memory by using PCILeech since it's starting with the signature DMAR. It turns out that while entirely possible to do so it was a bit problematic on the tested hardware. When PCILeech encounters unreadable memory it needs to be power-cycled before able to read memory again. It was easier to boot into Ubuntu on an USB stick and check the location of the DMAR table with the dmesg command.

The DMAR ACPI table is located at address: 0x3A529CB0.
Once knowledge of the memory address of the DMAR table has been gained it's possible to overwrite it. The address is usually completely static between reboots. It might change if extensive changes are made to the BIOS configuration - such as enabling/disabling secure boot though.

We'll overwrite the beginning of the DMAR table before Windows starts to boot. Just issue the command: 
pcileech.exe write -min 0x3a529cb0 -in 000000000000000000000000000000000000000000000000 
which will effectively overwrite the start of the DMAR table with 24 bytes of null data.

The obligatory before/after image. The original DMAR table to the left. The overwritten DMAR table to the right.

The Result
Even though the beginning of the DMAR table is now overwritten Bitlocker will unlock the OS disk allowing Windows to boot. Windows don't find the DMAR table so it can't locate the IOMMU. Even though UEFI lock has been configured Windows will just disable virtualization based security features and continue to boot normally.

Windows with Bitlocker. Credential and Device Guard not started despite configuration. PCILeech kernel module loaded.
Other
  • It's possible to nuke the DMAR table on other hardware as well.
  • Virtualization Based Security isn't designed to fully protect against physical and firmware based attacks.
  • Issue mentioned to MSRC in July as part of pre-existing case. Case closed after DEF CON talk Direct Memory Attack the Kernel.
  • BIOS version: KYSKLi70.86A.0037.2016.0603.1032.
  • Did I miss something? Please let me know.

Conclusions
  • Computers with active DMA ports and auto-booting full disk encryption might be at risk if physical access could be gained.
  • Windows 10 Virtualization Based Security might not fully protect against physical DMA attacks with PCILeech on autobooting systems.
  • It's possible to disable unused M.2 slots and set a BIOS password on the NUC if one wish to be extra secure.