Asia's Source for Enterprise Network Knowledge

Sunday, May 28th, 2017

Storage

FAQ: 3D XPoint memory – NAND flash killer or DRAM replacement?

As the first new non-volatile, mass-marketed storage technology since NAND flash, 3D XPoint made a huge splash when it was first announced in 2015 by development partners Intel and Micron. It was touted as being 1,000 times faster than NAND flash with up to 1,000 times the endurance.

In reality, the performance claims were only true on paper; 3D XPoint turned out to be about 10 times faster than NAND, which requires existing data to be erased before new data is written.

The new solid-state memory, however, is likely to find a place in the data center since it is about half the price of DRAM (though still costlier than NAND). That's because it works with conventional memory technologies to boost performance.

With the growth of transactional data, cloud computing, data analytics and  next-generation workloads will require higher performance storage.

Enter, 3D XPoint.

"This is an important technology that is going to have big implications for data center usage and to a lesser degree on the PC side," said Joseph Unsworth, Gartner's research vice president for semiconductors and NAND flash. "Whether it's your hyperscale data center, cloud service provider or traditional enterprise storage customers, they're all very interested in the technology."

While 3D XPoint won't convince companies to rip and replace all of their server DRAM, it will allow IT managers to cut costs by replacing some of it -- while also augmenting the performance of their NAND flash-based SSDs.

What is 3D XPoint? 

Simply put, it's a new form of non-volatile, solid-state storage with vastly greater performance and endurance than NAND flash. Price-wise, it lies between DRAM and NAND.

DRAM currently costs a little north of $5 per gigabyte; NAND comes in around 25 cents per gig. 3D XPoint is expected to land at around $2.40 per gig for large volume purchases, according to Gartner. And it's expected to be much more costly than NAND through at least 2021.

While neither Intel nor Micron have detailed what 3D XPoint is, they have said it's not based on the storage of electrons, as is the case for flash memory and DRAM, and it doesn't use transistors. They've also said it's not resistive RAM (ReRAM) or memristor -- two emerging non-volatile memory technologies considered possible future rivals to NAND.

The process of elimination (backed by storage experts) leaves 3D XPoint as a type of phase-change memory, as Micron previously developed the technology and its properties closely resemble it.

PCM is a form of nonvolatile memory based on using electrical charges to change areas on a glassy material -- called chalcogenide -- back and forth from a crystalline to a random state. That description matches up with what Russ Meyer, Micron's director of process integration, has said publicly: "The memory element itself is simply moving between two differentresistence states."

In PCM, the amorphous state's high resistance is read as a binary 0; the lower-resistance crystalline state is a 1.

3D XPoint's architecture is akin to a stack of submicroscopic window screens, and where wires cross there are pillars of chalcogenide material that includes a switch allowing access to stored bits of data.

"Unlike traditional DRAM that stores its information in electrons on a capacitor or NAND memory that stores electrons trapped on a floating gate, this uses a bulk material property change of the material itself to store whether [a bit] is a zero or a one," said Rob Crook, GM of Intel's non-volatile memory solutions group. "That enables us to scale to small dimensions and that enables a new class of memory."

Why is 3D XPoint getting so much attention? 

Because 3D XPoint technology delivers up to 10x more performance of NAND flash across a PCIe/NVMe interface, and has up to 1,000 times the endurance. One thousand times the endurance of NAND flash would be more than a million write cycles, meaning the new memory would last, well, pretty much forever.

By comparison, today's NAND flash lasts for between 3,000 and 10,000 erase-write cycles. With wear-leveling and error correction software, those cycles can be improved, but they still don't get anywhere near one million write cycles.

It's 3D XPoint's low latency -- 1,000th that of NAND flash and ten times the latency of DRAM -- that makes it shine, particularly for its ability to deliver on high input/output operations, such as those required by transactional data.

The combo allows 3D XPoint to fill a gap in the data center storage hierarchy that includes SRAM on the processor, DRAM, NAND flash (SSDs), hard disk drives and magnetic tape or optical discs. It would fit between volatile DRAM and non-volatile NAND flash solid state storage.

So why is it good for some data centers? 

James Myers, director of NVM Solutions Architecture for the Non-volatile Memory Solutions Group at Intel, said 3D XPoint is aimed at servicing random, transactional data sets not optimized for in-memory processing. (Intel calls its version of the technology Optane memory.)

"Optane is going to be servicing the highest end of warm and part of the hot tier in terms of storage for architectures that aren't optimized [for in-memory processing]...or even to extend the memory size or space within that hottest tier," Myers said. "Those are very much random transactions."

For example, it could be used to perform limited real-time analytics on current data sets or store and update records in real time.

Conversely, NAND flash will grow in its use for storing near-line data for batch-based, overnight processing -- performing analytics with column-oriented database management systems. That will require queue depths of 32 outstanding read/write operations or greater.

"Not a lot of people are willing to pay a lot of extra money for higher sequential throughput. A lot of those analytics ... can get done between 2 a.m. and 5 a.m. when no one is transacting much business," Myers said.

Intel's first 3D XPoint SSD - the P4800X -- can perform up to 550,000 read input/output operations per second (IOPS) and 500,000 write IOPS at queue depths of 16 or fewer. While Intel's top-tier NAND-flash based SSDs can achieve 400,000 IOPS or better, they only do so with deeper queue depths.

Like DRAM, 3D XPoint can be byte addressable, meaning each memory cell has a unique location. Unlike block-level NAND, there's no overhead when an application goes searching for data.

"This isn't flash and it's not DRAM, it's something in between, and that's where the ecosystem support's going to be important to be able to exploit the technology," Unsworth said. "We've not seen any [non-volatile] DIMM deployed yet. So it's still an area being worked on."

The introduction of 3D XPoint as a new storage tier, according to IDC, is also one of the first major technology transitions to occur since the emergence of large cloud and hyperscale data centers as dominating forces in technology.

When will 3D XPoint be available? 

Intel has carved out its own path separate from that of Micron for 3D XPoint technology. Intel describes its Optane brand as suited for both data centers and desktops, saying it strikes the perfect balance of accelerating access to data while affordably maintaining mega storage capacities.

Micron sees its QuantX SSDs as best suited for data centers. But at least one executive alluded to the possibility of a consumer-class SSD down the road.

In 2015, limited production of 3D XPoint wafers began at IM Flash Technologies, Intel and Micron's joint fabrication venture based in Lehi, Utah. Mass production began last year.

Last month, Intel started shipping its first products with the new technology: the Intel Optane memory PC accelerator module for PCs (16GB/MSRP $44) and (32GB/$77); and the data center-class 375GB Intel Optane SSD DC P4800X, ($1,520) expansion card. The DC P4800X uses a PCIe NVMe 3.0 x4 (four-lane) interface.

The Optane memory PC accelerator module can be used to accelerate any SATA-attached storage device installed in a 7th-generation (Kaby Lake) Intel Core processor-based platform designated as "Intel Optane memory ready." The Optane add-in memory module acts as a type of cache to increase performance in laptops and desktops.

While the DC P4800 is the first 3D XPoint-based data center SSD to be made available, Intel has said more will be coming soon, including an enterprise Optane SSD with 750GB in the second quarter of this year, as well as a 1.5TB SSD that's expected to ship in the second half of this year.

Those SSDs will also be modules usable in PCI-Express/NVMe and U.2 slots, which means they could be used in some workstations and servers based on AMD's 32-core Naples processors.

Intel is also planning to ship Optane in the form of DRAM-style DIMM modules next year.

Currently, Micron expects its first sales of a QuantX product in the second half of 2017, with 2018 being a "bigger year," and 2019 being the "break-out" revenue year.

How will 3D XPoint impact computer performance? 

Intel claims its Optane add-in module cuts PC boot-up time in half, boosts overall system performance by 28% and loads games 65% faster.

The DC P4800 performs best in random read/write environments where it can augment server DRAM. Optane lights up when running random reads and writes, which are common in servers and high-end PCs. Optane's random writes are up to 10 times the speed of conventional SSDs, with reads around three times faster. (For sequential operations, Intel still recommends NAND flash-based SSDs.)

For example, the 375GB DC P4800 SSD retails for about $4.05/GB of capacity, with a random read rate of up to 550,000 IOPS using 4K blocks at a queue depth of 16. It has a sequential read/write rate of up to 2.4GB/s and 2GB/s, respectively.

By comparison, an Intel NAND flash-based data center SSD such as the 400GB DC P3700 retails for $645 or about $1.61/GB. From a performance perspective, the  P3700 SSD delivers 4K random read rate of up to 450,000 IOPS at higher queue depth -- up to 128 -- with sequential reads/writes topping out at up to 2.8GB/s and 1.9GB/s, respectively.

In addition, the new DC P4800 SSD is specified with read/write latency of under 10 microseconds, which is a lot lower than many NAND flash-based SSDs that sport read/write latency in the 30 to 100 microsecond range, according to IDC. The DC 3700, for example, has an average latency of 20 microseconds, twice that of the DC P4800.

"The read and write latency of the P4800X is approximately the same, unlike flash memory-based SSDs, which feature faster writes versus reads," IDC stated in a research paper.

Will 3D XPoint eventually kill NAND flash? 

Probably not. Both Intel and Micron have said that 3D XPoint-based SSDs are complimentary to NAND, filling the gap between it and DRAM. However, as sales of new 3D XPoint SSDs pick up and economies of scale grow, analysts believe it could eventually challenge existing memory technology -- not NAND, but DRAM.

Gartner predicts that 3D XPoint technology will start seeing significant uptake in data centers in the late 2018.

"It's gotten a lot of attention from a lot of key customers -- and not just servers, storage, hyperscale data centers or cloud customers, but also software customers," Unsworth said. "Because if you're able to cost effectively analyze databases, data warehouses, data lakes far faster and cost effectively, that becomes very appealing for the end user to be able analyze more data and do that in real time.

"So we do believe this is a transformational technology," he added.

That transformation, however, will take time. The data center ecosystem will have to adjust to adopt the new memory, including the new processor chipsets and third-party applications that support it.

Additionally, there are currently only two providers: Intel and Micron. Longer term, the technology may be produced by others, Unsworth said.

But there are other kinds of memory coming? 

There are --  namely, competing technologies such as Resistive RAM (ReRAM) and memrisor. But neither one has been produced in high capacities or shipped in large volume.

Last fall, Samsung debuted its new Z-NAND memory, an obvious competitor to 3D XPoint. The yet-to-be released Z-NAND SSDs were purported to sport four times faster latency and 1.6 times better sequential reading than 3D NAND flash. Samsung expects its Z-NAND to be released this year.

OK, so does this mean NAND is dead? 

Not by a long shot. While other non-volatile technologies may eventually challenge 3D XPoint, conventional NAND flash still has a long development road map ahead of it. It's likely to see at least another three rev cycles that will take it through at least 2025, according to Gartner.

While the latest versions of 3D or vertical NAND stacks up to 64 layers of flash cells atop one another for more dense memory than traditional planar NAND, makers already see stacks exceeding 96 layers beginning next year and more than 128 layers in years to come.

Additionally, current 3-bit per cell triple-level cell (TLC) NAND is expected to move to 4-bit per cell quadruple level cell (QLC) technology, further increasing density and driving down manufacturing costs.

"This is a very resilient industry in which we have some of the biggest semiconductor vendors in the world...and China. China wouldn't be getting into the NAND flash industry with billions of dollars if they thought it wouldn't last more than three or four or five years," Unsworth said. "I see 3D NAND slowing down, but I don't see it hitting a wall."