SoftMC v1.0

SoftMC is an experimental FPGA-based memory controller design that could be used to develop tests for DDR3 SODIMMs. SoftMC currently supports only the Xilinx ML605 board. Soon, we will port SoftMC on other popularly used boards (e.g., Xilinx VC709).

A paper describing SoftMC in detail is published at HPCA 2017 and is available here:

Hasan Hassan, Nandita Vijaykumar, Samira Khan, Saugata Ghose, Kevin Chang, Gennady Pekhimenko, Donghyuk Lee, Oguz Ergin, and Onur Mutlu, "SoftMC: A Flexible and Practical Open-Source Infrastructure for Enabling Experimental DRAM Studies" Proceedings of the 23rd International Symposium on High-Performance Computer Architecture (HPCA), Austin, TX, USA, February 2017.

If you use or build on SoftMC, please cite that paper.

We provide a) prebuilt binaries for quick installation and b) the source code (both in Verilog and C++) that you could modify as you wish.

Prerequisites:

• A Xilinx ML605 FPGA or other compatible board (We only tested ML605)
• A Linux Host Machine, (We have tested on Ubuntu 12.04/14.04)
• SoftMC uses an 8-lane PCIe interface to communicate with the Host Machine. So, you would also need to attach the board to the 8x/16x PCIe slot on the motherboard of the Host Machine
• Single Rank DDR3 SODIMM attached to your FPGA board
• Xilinx ISE 14.6 (in case you want to build your own bitfile from the source)

Installation Guide:

1) Installing the RIFFA driver:

You need to have RIFFA driver installed on your system to enable the communication with the FPGA via the PCIe bus. Do not forget to attach your FPGA board to a PCIe slot.

First switch to the directory of the source files of the RIFFA driver:

$ cd sw/riffa_2.1/driver/linux

Then run make to build the driver

$ make

Then install the driver to your system

$ sudo make install

2) Creating and Downloading the SoftMC bitfile to the FPGA:

a) Quick Installation using Prebuilt Binaries

You will find all that you need inside "prebuilt" folder (including an executable for the Retention Time test).

• You need to download the bitfile into your FPGA using an appropriate tool (Xilinx Impact does the job for ML605)
• After restarting the machine that the FPGA is connected to via PCIe, you should be able to run the Retention Time test application by typing to command below:

$ ./SoftMC_RetentionTime [Target Retention Time in milliseconds]

b) Installation from the Source Code

Generating the SoftMC bitfile is straightforward:

• Use Xilinx ISE 14.6 to open the project, which is located located at hw/boards/ML605/SoftMC.xise. Ignore missing files pop-up windows by clicking the "Cancel" button. The missing files will be generated once the steps below are followed.
• Double-click on xilinx_mig IPCore file to open Memory Interface Generator's (MIG) configuration window. Keep clicking the "Next" button until the windows closes. Doing this will generate the necessary files required by SoftMC in hw/boards/ML605/ipcore_dir/xilinx_mig.
• Open a terminal and go to hw/boards/ML605. Run:

$ sh apply_patches.sh

This will apply modifications to the files generated by MIG.

• Go back to Xilinx ISE and click "Generate Programming File" and then click "Yes" on the dialogs that ask for IPCore to be compiled

  • If you get the following error during synthesis: hw/boards/ML605/ipcore_dir/pcie_endpoint/source/gtx_wrapper_v6.v" Line 277: Instantiating <GTX_DRP_CHANALIGN_FIX_3752> from unknown module <GTX_DRP_CHANALIGN_FIX_3752_V6>
    1. Add `include "gtx_drp_chanalign_fix_3752_v6.v" before the module declaration in "gtx_wrapper_v6.v". Save the file.
    2. Comment-out the line that you have just added. Save the file again.
    3. Click Synthesize again

  The error occurs due to a bug in Xilinx ISE software. A workaround for it is doing the steps that we listed above.

• After the operation completes successfully, you will find the generated bitfile in the project folder

• Then follow the steps in a)

(Where necessary to simulate the SoftMC hardware, we provide a sample testbench module ("tb_softMC_top.v") that you can start with. To enable simulation, you will need to uncomment the SIM definition in "softMC.inc". This will change the I/O interface of the top module to exclude the PCIe signals and let you easily issue instructions to SoftMC)

To compile the sample application (retention time test) that we provide:

$ cd sw/RetentionTest
$ make
$ ./SoftMC_RetentionTest [Target Retention Time in milliseconds]

Known Issues:

• Multi Rank SODIMMs are currently not supported.
• An instruction sequence could consist maximum of 8192 instructions (see our HPCA 2017 paper for details).
• Motherboards with B75 Chipset seems to be incompatible with ML605's PCIe endpoint.

You are welcome to contribute to the project. If you find/solve any issues or port SoftMC to a new FPGA board, please contact the people below.

Contacts:

Hasan Hassan (hhasan [at] inf [dot] ethz [dot] ch)