A high-throughput readout architecture based on PCI-Express Gen3 and DirectGMA technology

Modern physics experiments produce multi-GB/s data rates. Fast data links and high performance computing stages are required for continuous data acquisition and processing. Because of their intrinsic parallelism and computational power, GPUs emerged as an ideal solution to process this data in high...

Full description

Saved in:
Bibliographic Details
Published in:Journal of instrumentation 2016-02, Vol.11 (2), p.P02007-P02007
Main Authors: Rota, L., Vogelgesang, M., Perez, L.E. Ardila, Caselle, M., Chilingaryan, S., Dritschler, T., Zilio, N., Kopmann, A., Balzer, M., Weber, M.
Format: Article
Language:English
Subjects:
Architecture
Architecture (computers)
Computation
Computer programs
Instrumentation
Memory (computers)
Photons
Citations: Items that this one cites
Items that cite this one
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:Modern physics experiments produce multi-GB/s data rates. Fast data links and high performance computing stages are required for continuous data acquisition and processing. Because of their intrinsic parallelism and computational power, GPUs emerged as an ideal solution to process this data in high performance computing applications. In this paper we present a high-throughput platform based on direct FPGA-GPU communication. The architecture consists of a Direct Memory Access (DMA) engine compatible with the Xilinx PCI-Express core, a Linux driver for register access, and high- level software to manage direct memory transfers using AMD's DirectGMA technology. Measurements with a Gen3 x8 link show a throughput of 6.4 GB/s for transfers to GPU memory and 6.6 GB/s to system memory. We also assess the possibility of using the architecture in low latency systems: preliminary measurements show a round-trip latency as low as 1 [mu]s for data transfers to system memory, while the additional latency introduced by OpenCL scheduling is the current limitation for GPU based systems. Our implementation is suitable for real-time DAQ system applications ranging from photon science and medical imaging to High Energy Physics (HEP) systems.
ISSN:1748-0221
1748-0221
DOI:10.1088/1748-0221/11/02/P02007