Operation Set Architectures for organic FPGA ecosystems

How can we bring different optimized-down-to-the-gate FPGA libraries and latest state-of-the-art ML and AI together?

In my research of 2022 and 2023, I answered this by inventing the Operation Set Architectures. The basic principle is to define operation blocks at an intermediate representation that is 1) low enough to enable compiler optimizations but 2) high enough to still allow easy architecture-specific optimizations.

My research was published in the IEEE Compture Architecture Letters with Advancing Compilation of DNNs for FPGAs using Operation Set Architectures and the top-tier IEEE EDGE conference with DOSA: Organic Compilation for Neural Network Inference on Distributed FPGAs. The first paper evaluates the potential of Operation Set Architectures, while the second paper explores the potential of Distributed Operation Set Architectures.

You can find the PDF of the CAL paper here and the EDGE paper here.

Extended Abstract

The computational requirements of artificial intelligence workloads are growing exponentially. In addition, more and more compute is moved towards the edge due to latency or localization constraints. At the same time, Dennard scaling has ended and Moore’s law is winding down. These trends created an opportunity for specialized accelerators including field-programmable gate arrays (FPGAs), but the poor support and usability of today’s tools prevents FPGAs from being deployed at scale for deep neural network (DNN) inference applications. In this work, we propose an organic compiler — DOSA — that drastically lowers the barrier for deploying FPGAs. DOSA builds on the operation set architecture concept and integrates the DNN accelerator components generated by existing DNN-to-FPGA frameworks to produce an overall efficient solution. DOSA starts from DNNs represented in the community standard ONNX and automatically implements model- and data-parallelism, based on the performance targets and resource footprints provided by the user. Deploying a DNN using DOSA on 9 FPGAs exhibits a speedup of up to 52 times compared to a CPU and 18 times compared to a GPU.

Lightning Talk

For the IEEE Computer Architecture Letters, we recorded a 2 minutes lightning talk.

Open source code

I’m deeply committed to OpenSource.Science and the code of the above research can be found here: github.com/cloudFPGA