#134: Best Practices for OpenZFS L2ARC in the Era of NVMe

The ZFS L2ARC is now more than 10 years old. Over that time, a lot of secret incantations and tribal knowledge have been created by users, testers, developers, and the odd sales or marketing person. That collection of community wisdom informs the use and/or tuning of ZFS L2ARC for certain IO profiles, dataset sizes, server class, share protocols, and device types. In this talk, we will review a case study in which we tested a few of these L2ARC myths on an NVMe-capable OpenZFS storage appliance. Can high-speed NVMe flash devices keep L2ARC relevant in the face of ever-increasing memory capacity for ARC (primary cache) and all-flash storage pools? Learning Objectives: 1) Overview of ZFS L2ARC design goals and high level implementation details that pertain to our findings; 2) Performance characteristics of L2ARC during warming and when warmed, plus any tradeoffs or pitfalls with L2ARC in these states; 3) How to leverage NVMe as L2ARC devices to improve performance in a few storage use cases.