NVMe 1.3 adds framework for metadata exchange

Article By : Gary Hilson

The update, which represents one of three core specifications for NVMe, encompasses 24 technical proposals that can be spread across client, enterprise and cloud features.

After nearly three years, the NVM Express (NVMe) specification has gotten its first major update.

NVM Express Marketing Committee co-chair Jonmichael Hands told EE Times that the version 1.3 of the NVMe specification for SSDs on a PCI Express (PCIe) bus adds a significant number of new features, something that hasn't been done since November 2014.

This update represents one of three core specifications for NVMe, according to Hands. The others are the NVM Express Management Interface and the NVMe-over-Fabrics specifications, although the latter is not due for an update until late 2018. Micron recently announced it was working ahead of the standard.

It takes time for vendors to take advantage of the new specifications and incorporate them into their products, said Hands. Devices with the NVMe 1.2 specification only began launching last fall. This two-year ramp up is typical, he said, although there's nothing holding device makers back except the time it takes them to update products with the new features.

NVMe 1.3 encompasses 24 technical proposals, said Hands, which can be spread across three major buckets that address client, enterprise and cloud features. The most significant step forward is improved support for virtualisation so developers can more flexibly assign SSD resources to specific virtual machines, he said.

“Right now, if you want to use an NVM Express device in a virtualised environment, the hypervisor's NVMe driver has to emulate an NVMe SSD to the guest OS," Hands said. "They do this pretty well, but there's a latency hit."

And when it comes to very fast storage class memory devices, Hands said it starts to add up since putting a raw device behind a hypervisor can significantly reduce the number of IOPs. The trick to getting the most performance from each SSD in a virtualised environment, he said, is to make it seem like the SSD is attached natively to each virtual machine. NVMe 1.3 takes advantage of the Single Root I/O Virtualisation (SR-IOV) feature in PCIe to support shared storage with direct assignment. “Now you can partition and intelligently allocate resources," he said.

Hands said this offers a lot of value for companies supporting cloud environments and multi-tenancy, but to get the most value from it, developers should be writing this resource allocation into the software-defined storage stack. Some of the large hyperscale customers that are on the NVM Express board of directors are pushing for this feature, he said. The current approach is to use more, smaller SSDs for each workload so they are not impacting the quality of service of other workloads.

Figure 1: The Streams feature in NVMe 1.3 can reduce write amplification for host managed workloads.

One of the most exciting features in 1.3 is Directives, said Hands, which is a new framework for the host and device to exchange metadata. It is particularly well-suited for an all-flash array to support better workload optimisation on each drive. SSDs are getting much larger, he said, with the average size hitting 4TB today and quickly rising. In multi-tenant environment, this means mixing different customer workloads on a single SSD. “Inevitably it's going to hurt your endurance because you're going to have different workloads on the same drive," Hands said.

An early example of the Directives feature is Streams, which enables the host to indicate to the controller that the specified logical blocks in a write command are part of one group of associated data. This information may be used by the controller to store related data in associated locations or for other performance enhancements. Essentially, said Hands, Streams optimises performance and improves endurance for NAND-based SSDs using simple tagging of associated data from different tenants in cloud hosting applications.

Among the other new features in NVMe 1.3 are enhanced debugging tools for SSDs, which until now have been the domain of the SSD vendors, said Hands, as well as more granular control for thermal throttling based not only on the temperature of the system but also the workload. “The host can now tell SSDs where throttle," Hands said.

The latest NVMe specification also supports bootstrapping in low-resource environments, including mobile, said Hands, which will allow for lower cost NVMe devices in smaller spaces. NVMe 1.3 also offers broader operations for SSD erasure that are compliant with government standards.

Not unlike 3D NAND, which looks to be hitting its tipping point in 2018 with wide adoption, NVMe seems to be poised become the dominant interface for SSDs by the end of the year, said Hands. And while there remain markets for SATA and SAS, there are few features being added. “This is where NVMe pulls ahead in terms of innovation," Hands said.

First published by EE Times U.S.