Ibex

Description

In Ibex we work on an intelligent storage engine for commodity relational databases. The Ibex framework is open source and can be downloaded here. Our prototype works with MySQL, and performs various database tasks on data which is in-flight between an SSD and the CPU. The tasks that can be off-loaded to the FPGA in
clude Projection, Selection and Group By Aggregation. As a result, the amount of records hitting the CPU can be dramatically reduced for most queries, resulting in faster execution times, but also lower energy consumption overall. In addition to this, we are working on providing statistics in the datapath for relational databases, as an exploration of additional operations that could be performed on smart storage elements. Most recently, we evaluated the benefits of offering histograms on data flowing from storage to CPU.
 
The source code of Ibex and the tutorial to compile it can be found here.
Groundhog, the SATA controller running on the FPGA is also open source, available at https://groundhog.codeplex.com/
 

People

  • Zsolt Istvan
  • Louis Woods (PhD Graduate, now at Apcera)
  • Jens Teubner (Postdoctoral Researcher, now Professor at TU Dortmund)
  • Gustavo Alonso

 

Publications 

Louis Woods, Zsolt Istvan, Gustavo Alonso.
40th Int'l Conference on Very Large Data Bases (VLDB). Hangzhou, China, 1-5 September 2014.
 
Zsolt Istvan, Louis Woods, Gustavo Alonso.
2014 ACM SIGMOD/PODS Conference (SIGMOD'14), Snowbird, Utah, US, 22-27 June 2014.
 

Demos

Hybrid FPGA-accelerated SQL Query Processing.
Demo for 23rd International Conference on Field Programmable Logic and Applications (FPL'13), Porto, Portugal, 2-4 September 2013. 
 
Less Watts, More Performance: An Intelligent Storage Engine for Data Appliances.
Demo for ACM SIGMOD Conference on Management of Data (SIGMOD 2013), New York, NY, USA, June 2013
 
 
 

IBEX Source Code and Tutorial

Setting up the Hardware

Ibex runs on a XUPV5 delopment board from Xilinx. An SSD (e.g. OCZ Vertex 4) needs to be connected to the SATA Host 1 port as depicted in the picture below. When the FPGA is configured using the precompiled bitstream (system.bit) provided below or a version that you compiled yourself, the ERROR led ERR1 will light up if the SSD could not be initialized properly. This might happen when the SSD is powered off. When the power supply for the SSD is turned on, the ERROR led will turn off if everything is fine.

 

Ibex Hardware Setup

 

Compiling MySQL

These instructions discuss how to compile MySQL 5.5.25 on Windows using Visual Studio 2010 Professional. Before you start you need to install CMake and Bison next to Visual Studio 2010. Download the MySQL source code and unpack, e.g., in C:\mysql-5.5.25.

Go to the root directory of the source code (e.g., C:\mysql-5.5.25) and run:

C:\mysql-5.5.25> cmake .

This will generate the Visual Studio project file: MySQL.sln in the root directory. You can now open the project in Visual Studio and compile it. Next we will configure the project for debugging.

  • Right click Solution 'MySQL' on the top of the Solution Explorer and select mysqld under 'Single startup project'
  • In the Solution Explorer rightclick the subproject mysqld and select properties
  • Go to Configuration Parameters > Debugging
  • In the 'Command Arguments' field type '--defaults-group-suffix=.1 --defaults-file=C:/mysql-5.5.25/mysql-test/var/my.cnf --log-output=file --gdb --default-storage-engine=MyISAM --loose-skip-log-bin --core-file --loose-debug-sync-timeout=300 --console'
  • In the 'Working Directory' field type '..\mysql-test\var'

Running MySQL

Now you should be able to run MySQL (press CTRL+F5). To connect to the server open the Command Prompt or Windows PowerShell and type:

> C:\mysql-5.5.25\client\Release\mysql.exe -u root --port=13000 --local-infile=1

This assumes that you unpacked the source code C:\mysql-5.5.25 and compiled a release version of the entire project.

Adding the Ibex Storage Engine

To add the ibex storage enngine to MySQL first download the engine here. Unpack the file and copy the directory called 'ibex' into 'C:\mysql-5.5.25\storage\ibex' and select the file ibex.vcxproj. This should add the subproject IBEX and should be visible in the Solution Explorer. You can compile the code by right clicking on IBEX and selecting Project Only > Build Only IBEX. This will generate the library ha_ibex.dll. 

The next step is to register the storage eninge with MySQL. To this end, start mysqld and connect  to the database as explained previously. Install ibex using the following comand:

mysql> INSTALL PLUGIN ibex SONAME 'ha_ibex.dll';

If this command completes successfully, you should see IBEX when you run the following command:

mysql> SHOW ENGINES\G

 

Running Ibex

 Before you attempt to create an Ibex table you should format the SSD/Harddisk. This can be achieved by setting the FORMAT flag in ha_ibex.cc to true. Warning: this will overwirte the first few sectors of the storage device connected to the FPGA when the storage enngine is loaded.  After mysqld starts with the output on the Command Prompt should look as follows:

Now you can start creating tables using SQL. Remember to set FORMAT back to false so that the tables you create will not be erased when you start mysqld the next time. The below an example is given of how to create an IBEX table and load data into it:

When you start mysqld the next time, table statistics will be output on the Command Prompt for debugging purposes:

Generating the FPGA Bitstream

To compile the hardware part of Ibex you should first create an (Xilinx) ISE project following these instructions. Download the Verilog code and add all the source files provided. Also add the UCF files (HBA.ucf and XUPV5system.ucf) Finally, generated the three IP-cores, using the core generator (Block Memory Generator 6.2) as described here. Done. You can now generate the bitstream.

 

Source Files with Operators

The tutorial above covers how to get the core of IBEX up and running and how to plug it into MySQL. Due to the complexity of setup, we chose to provide these explanations on a version of IBEX without operators, to simplify dependencies and also to allow cleaner additions of you rown operators. If you are interested in the version with operators, you can find it below. Feel free to contact us for questions.