Wednesday, October 20, 2021

Modifying the Acorn CLE-215+ FPGA into a PCILeech DMA attack device

PCILeech and MemProcFS allows for easy-to-use user-friendly DMA attacks and hardware assisted memory analysis. This is possible since PCI Express supports DMA. Unfortunately production of compatible hardware, such as the Screamer series has been hit hard by the global silicon shortage.

The goal with this project is to modify the Acorn CLE-215+ / Nitefury / Litefury FPGA boards, in a short time frame and on a relatively tight budget, to support PCILeech and MemProcFS at around 20-25MB/s.

PCILeech DMA with a modified Acorn CLE-215+ and FT2232H.

The Hardware

The Acorn CLE-215+ is a powerful FPGA board with PCI Express M.2 connector which is used for DMA hardware memory acquisition. It also have 12 additional GPIOs in a 20-pin DF52 header. It uses the most powerful Xilinx Artix7 FPGA chip - the 200T. The Acorn CLE-215+ was used for crypto mining but has been discontinued for some time. They can sometimes be found at a nice price at eBay. The Nitefury FPGA board is basically the same board as the Acorn CLE-215+. The Litefury has a smaller, still very capable, Artix 7 100T FPGA.

Acorn CLE-215+ and LiteFury.

PCILeech traditionally connects to the FPGA device over USB3 resulting in DMA around 150MB/s. The now sold-out FTDI FT601 chip is used in synchronous FT245 mode to achieve this. The FT601 uses 40 signals between itself and the FPGA.

The goal with this project is to modify the Acorn CLE-215+ / Nitefury / Litefury to support the FTDI FT2232H USB2 chip. In FT245 mode this should allow for DMA transfer speeds around 20-25MB/s. This should allow PCILeech to work with the CLE-215+ with relatively minor software modifications. 

Alternative ways that does not include hardware modifications would be UART (slow) or a carrier board with additional FPGA/chips (expensive and complex to design). The goal of this project was to create something low-cost in a limited time frame.

The FT2232H is readily available and an inexpensive mini module exists for labs. In this project the FT2232H-56Q mini module which has a micro-USB2 connector will be used.

The FT245 mode we require 15 signals: 8 data, 1 clock and 6 additional control signals. To make things even worse the FT2232H runs at 3.3V while the Acorn only have 4 GPIOs at 3.3V and 8 GPIOs at 2.5V. Also none of the GPIOs on the Acorn are clock capable - i.e. possible to use as a clock input pin.

Two hardware modifications are required:

1) Make the 2.5V GPIOs 3.3V. This is done by removing a 3.3V to 2.5V voltage regulator and connecting the 2.5V power rail to 3.3V. This has been previously discussed. The Schematics also support this change and it should have little or no side effects.

2) Desolder LED2, LED3 and LED4 and use the FPGA connections as three additional GPIOs. The FPGA PIN driving LED4 is also clock capable which is great!

In addition to the above a custom JTAG connector cable may have to be created.


Required Tools:

This is a hardware modification project. In addition to the hardware for the project itself it's assumed that one has access to:
  • Molex crimp tool.
  • Hot-air rework station.
  • JTAG programmer cable.
  • Soldering paste.
Also nice to have, but not required, is access to a logic analyzer or oscilloscope in case something isn't working and some signal debugging is required. A multimeter to check voltage levels and cable connections would also be nice to have at hand.


JTAG connector cable

JTAG is required to program the FPGA with PCILeech a compatible gateware.

The Acorn has a Pico EzMate JTAG header on the bottom side. The LiteFury comes with a nice connector cable which allows for easy connection to your JTAG programmer cable. I use the Digilent HS2 JTAG cable since it's directly compatible with the Xilinx Vivado development environment. The LiteFury cable, shown below in (5), is compatible with the Nitefury and Acorn as well.

If a LiteFury purchase is not desirable it's relatively easy to roll your own cable using the parts list below and molex crimp tool.

Custom JTAG connector cable (1-3). JTAG pinout (4). LiteFury JTAG connector cable (5).

  1. Cut the purchased Pico-EzMate cable in half.
  2. Then crimp standard molex connectors on the cable endings.
  3. Then insert the crimped connectors into the housing resulting to get a working JTAG cable adapter. Make sure the Pico-EzMate JTAG PINs are mapped to the correct PINs on your JTAG programming cable according to the pinout in (4).

Parts list custom JTAG connector cable:

6x Molex crimp connectors (get some extra just in case).
Total: €6 + VAT + shipping.


Flashing the LiteFury with the custom made JTAG cable.



Modification #1 - 2.5V to 3.3V

It's possible to modify the acorn 2.5V GPIOs to 3.3V which would support the FT2232H voltage levels. This has been previously discussed. This is possible by removing the voltage regulator U11 and shorting some of the underlying pads.

When looking at the schematics for the NiteFury/LiteFury this becomes clear. The Voltage regulator U11 converts 3.3V into 2.5V. The 2.5V is only used to drive FPGA Bank34 - which drives the 2.5V LVDS/GPIOs and also the 200MHz DDR clock oscillator - which seems to be supporting 3.3V as well.

2.5V voltage regulator U11 and schematics extracts.

Now let's remove U11 using a hot-air rework station and short the 2.5V power rail to the 3.3 power rail. The removal of U11 is likely to be destructive - the part is likely to fall apart. Please take great care not damaging the Acorn / LiteFury. If you are not familiar with work like this please practice on scrap components before attempting this!

Hot-air rework station setup.

The topmost image below shows the Acorn when the U11 power regulator has been removed. In order to connect the 2.5V power rail to 3.3V a tiny amount of ChipQuik soldering paste as well as a tiny cable fragment is applied on the three "topmost" power regulator pads as shown in the bottom image below. Hot air is applied to make the modification permanent. Now the modification is complete and the Acorn 2.5V GPIOs has become 3.3V.

Removal of power regulator U11 (top). Connecting 2.5V pad to 3.3V (bottom).



Modification #2 - LEDs to GPIOs

The 2nd modification is to desolder three LEDs and replace and solder on three cables on the innermost pad (which goes to the FPGA). The pad closes to the edge goes to ground - care should be taken not to connect the two by mistake.

For cabling I used three short DF52 cables since I had them at hand. Other cables may be used as well, but please try to keep them fairly short. On the other end Molex connectors was crimped on (after the soldering to the acorn was completed). Housing from three standard lab cables was used to shield the crimped Molex connector.

Image (1) shows the acorn LEDs before the hardware modification.

Image (2) shows the LEDs removed using hot air and a cable to the pad of LED3 is being attached with a rather messy application of soldering paste. The wire were individually soldered on using soldering paste and hot air. The wires were then routed in the space behind the DF52 header for some additional stability.

Image (3) shows the end result. The three additional GPIO wires are routed behind the DF51 had Molex connectors crimped on the other end after the soldering was complete. The worst ChipQuik mess was also cleaned up and the end result is decent enough and most importantly - fully functional :)


LED to GPIO hardware modification. (1) before, (2) during, (3) after.


FPGA/FT2232H custom signal cable

Creating the custom signal cable is one of the more tedious parts of the project. Care should be taken to create a good quality relatively short cable to allow for good enough signal quality.

The pinout of the DF52 connector is shown in the table below. PIN1 is at lower part (close to the SQRL logo) whilst PIN20 is on the upper part closest to LED1.

DF52FPGANAMEFT2232H
1. 3.3V (NC)
2. 3.3V (NC)
3. AB8DATA[0]CN2-7 (AD0)
4. AA8DATA[1]CN2-10 (AD1)
5. GND CN2-6*
6. Y9 DATA[2]CN2-9 (AD2)
7. W9 DATA[3]CN2-12 (AD3)
8. GND CN2-6*
9. Y8 DATA[4]CN2-14 (AD4)
10. Y7 DATA[5]CN2-13 (AD5)
11. GND CN2-4*
12. V9 DATA[6]CN2-16 (AD6)
13. V8 DATA[7]CN2-15 (AD7)
14. GND CN2-4*
15. K2 RXF_N CN2-18 (AC0)
16. J2 TXE_N CN2-17 (AC1)
17. GND CN2-2*
18. J5 RD_N CN2-20 (AC2)
19. H5 WR_N CN2-19 (AC3)
20. GND CN2-2*
----
LED2H3 OE_N CN2-23 (AC6)
LED3G4 SIWU_N CN2-22 (AC4)
LED4H4 CLK CN2-24 (AC5)

Start by cutting the DF52 cables in half. Populate the DF52 header. Skip the 3.3V at PIN 1 and 2 or cut  the cables after the cable is completed.

I opted for a ~10cm cable length. I first populated the DF52. Then I put a 2x10 PIN molex connector on the FT2232H CN2 PINs 5-24 - skipping PIN 1,2,3,4,25,26. Ideal would be to have a 2x13 connector but I only had some 2x10 connectors at hand. The parts list should contain a 2x13 pin connector housing though.

Making of a short custom signal cable.

I first crimped one connector then populated the housing on the proper pin with it. Then I cut the next cable to the correct length; crimped it and populated it. Iterating over all signals. For the ground I crimped two cables into the same connector (since there were 6 ground cables and only 3 slots on FT2232H). This iterative process is a good way making sure the cables are of an optimal length.

Upon completion I also glued the cable and taped it making sure every 2nd cable was ground like on the DF52 header. I also made some shorter (around 5cm) less impressive cables.

For the OE_N, SIWU_N and CLK cables a standard lab cable was used to connect.

The custom made signal cable.

Parts list custom signal cable:

12x DF52 pre-crimped cables (get some extra just in case).
20x Molex crimp connectors (get some extra just in case).
Total: €41 + VAT + shipping.


Flashing FT2232H and FPGA

The FT2232H must be flashed in FT245 mode prior to being usable by PCILeech. For the clock signal to enable it's required to both flash the FT2232H and start PCILeech/MemProcFS with the driver.

Download FT_Prog from ftdichip. Connect to the device. On Port A change Hardware to 245 FIFO and Driver to D2XX Direct. Do the same on Port B. IO Pins are set to 4mA and Schmitt Input on both Port A and Port B.

Remember to program the device after the changes have been made.

FT245 FIFO.


D2XX Direct.


To flash the FPGA first install Xilinx Vivado. It's possible to download a free WebPack edition. But beware - it's huge! After Vivado has been installed connect the JTAG cable and programmer. Open up Vivado and then Hardware Manager. Connect to the FPGA. Once connected right click the FPGA chip and select Spansion SPI memory according to the images below. Then program the bitstream / gateware. Please download the latest version from the PCILeech-FPGA project on Github.

Flashing the FPGA.


PCILeech DMA attacks and MemProcFS memory analysis


The Acorn CLE-215+ and LiteFury runs stable with PCILeech and MemProcFS with these modifications. For more information about how to use PCIleech and MemProcFS please check out my Github repos!

When using PCILeech / MemProcFS with the FT2232H make sure to install the FT2232H drivers from ftdichip and put the FTD2XX.DLL alongside the PCILeech / MemProcFS binaries.

Also make sure to specify -device fpga://ft2232h in the connection string.

Examples:

pcileech.exe dump -out memdump.raw -device fpga://ft2232h=1
memprocfs.exe -device fpga://ft2232h=1 -v -vv

Dumping memory using the acorn with PCILeech PCIe DMA attacks.

MemProcFS live memory analysis with the Acorn CLE-215+ works nicely.



Final Notes

This was an interesting project creating a fully usable low-cost (€50 + FPGA board) DMA attack device with decent performance.

Having FT2232H support in the code base - both in LeechCore and in the form of an FPGA IP core may be useful for other projects that doesn't require the higher performance of the FT601.

The device as such is due to the relative lack of available I/O however inferior to the USB3 variants of PCILeech compatible hardware such as the Screamer series. Latency is much higher and transfer speeds are significantly lower on the modified Acorn. But more high-speed alternatives are currently sold out due to the global chip shortage.

The LiteFury with a short custom signal cable.

More performant alternatives would be possible if designing a carrier board - utilizing some of the FPGA GTX lanes for communication with a 2nd FPGA then handling additional GPIOs. Such an approach would however be a larger undertaking and also be more costly. The Acorn Artix7 200T FPGA is however very powerful and is highly suitable for such an approach.

If you're interesting in doing these hardware modifications yourself please do so at your own risk.

PCILeech and MemProcFS is free open source and will continue to stay so. Many people and large corporations are using my tools for free - but few show any appreciation. If you enjoy using PCILeech and MemProcFS please consider becoming a Github Sponsor. Thank You 💖