#18: Donard: NVM Express for Peer-2-Peer between SSDs and other PCIe Devices

In this presentation we extend previous work to include p2p transfers between NVMe devices and RDMA capable NICs running protocols like Infiniband, RoCE and iWARP. We present experimental results using both 10Gbe iWARP and 56G Infiniband NICs that show how the latency associated with remote transfer of data can be reduced whilst also offloading the CPU allowing it to focus on other tasks. We show how this work can act as a precursor for the NVMe over Fabrics work currently being standardized. We also show how the Controller Memory Buffer (CMB) feature introduced in NVMe 1.2 be utilized in a novel fashion to aid this work. Learning Objectives: 1) What are the benefits of NVM Express 2) How can NVM Express and RDMA be utilized prior to NVMe over Fabrics 3) How Donard code builds on open-source code 4) How latency, bandwidth and CPU offload can all be improved using peer-2-peer