By Brian Beeler - October 1, 2021
"Absolutely phenomenal, we were blown away by the efficacy of this simple to use card and accompanying software. Compared to traditional hardware or software RAID options, GRAID SupremeRAID™ offers amazing ROI for demanding workloads."
GRAID Technology launched its first product over the summer, designed to handle modern throughput challenges that have been introduced by NVMe SSDs. GRAID SupremeRAID, so the claim goes, is “the world’s first NVMe RAID card to deliver 100% available SSD performance.” The little bit of marketing aside, GRAID does hit at a very real issue, traditional RAID cards can’t deliver all of the performance modern SSDs have to offer. GRAID is significantly different, thanks to an entirely novel architecture. You see, GRAID SupremeRAID doesn’t use a legacy RAID card; it uses a GPU to get the job done.
What’s Wrong with Traditional NVMe RAID Cards?
Traditional hardware RAID cards have worked well, up to a point. Now that NVMe SSDs are so fast, hardware RAID is a limiting factor. It’s not that the RAID cards themselves don’t work, it’s simply an issue of math. A server with a Gen4 PCIe slot can only hope to attain 16GB/s out of a x8 slot or 32GB/s out of a x16 slot.
But RAID cards are generally of the x8 variety, which means to leverage even moderate pools of NVMe flash in a single server, you’re looking at multiple RAID cards. A mainstream enterprise SSD like the Intel P5510 for instance can crank out 7000MB/s sequential read bandwidth. Even accounting for spec sheet vs reality difference, in this workload, that means two SSDs will cap a traditional RAID card in this particular workload.
Not every workload will be easy sequential reads of course, but the point remains. Getting 24 NVMe SSDs in a standard 2U server onto standard hardware RAID cards is complicated, expensive, and can compromise system performance. That said, the alternatives like software RAID haven’t been great either. Software RAID is easy to set up with no expense for a RAID card, but it chews through a lot of CPU resources which creates a performance penalty.
Enter GRAID SupremeRAID
As noted, the big difference from the jump is that GRAID SupremeRAID is based on an NVIDIA T1000 GPU. That little bit of detail isn’t terribly interesting to most GRAID customers who only care about results, but for our audience, this is an extremely interesting fact. The GPU has many advantages when it comes to addressing bottlenecks in NVMe RAID.
First off, the SupremeRAID card directly processes the I/O, relieving the CPU of this duty. And, because the card is a GPU, there’s tremendous computational power on the card, which simply doesn’t exist on standard RAID cards. The RAID protection itself is out-of-path, meaning data can flow from CPU to storage without having through the SupremeRAID card. GRAID also has an AI engine, which makes up its secret sauce, that runs on the card and makes the entire ecosystem operate efficiently.
Even though the card itself is fundamental to the GRAID solution, they still categorize themselves as a software RAID solution. Currently, GRAID supports RAID0, 1, 5, 6, and 10. In terms of performance, GRAID has driven around 6 million IOPS and 100GB/s out of a 24-drive system. Incidentally, 24 drives isn’t a limitation for GRAID, that’s just what’s commonly found in a 2U server. SupremeRAID doesn’t have a hard limit when it comes to the number of drives supported. GRAID is also ready for NVMe-oF and a variety of interconnects so that they’re ready for emerging architectures.
One last architecture note. This is a standard off-the-shelf card that’s plug-and-play with no cabling required. The card also does not require a battery backup module.
At the time we started this review, GRAID SupremeRAID only supported Linux, we’ve been recently told Windows is making its way out. We leveraged our Intel Scalable Gen3 server supporting NVMe Gen4 for this review, populated with eight Intel P5510 3.84TB SSDs. We went with RAID5 for both a hardware RAID group with GRAID as well as software RAID using mdadm in Linux.
The software RAID configuration leveraged a stripe size of 16K. To shake out each configuration, we used FIO with a range of synthetic benchmarks covering large-block sequential to small-block random.
FIO Workload Parameters
1M Sequential Read and Write – 16 Thread / 32 Queue
4K Random Read and Write – 32 Thread / 64 Queue
8K Random 70/30 Mixed – 32 Thread / 64 Queue
Going into this review we were somewhat optimistic, given the impressive claims. Not that it’s impossible to perform well in the storage market, but there are a lot of new items being thrown around with this solution. As you can quickly see, GRAID had no problem demolishing traditional software RAID.
We measured 8.88M IOPS 4K random read at 0.23ms, compared to traditional software RAID which measured 2.9M IOPS at 0.89ms latency. Moving to the write performance measurement was frankly nuts. The GRAID HW RAID solution measured 863K IOPS at 2.4ms, where the SW RAID5 group came in at the rather lackluster 52.6K IOPS at 38.9ms.
Switching the focus to a large block sequential bandwidth test, we remained very impressed. From the GRAID RAID5 group, we saw a peak read speed of 30.7GB/s at 16.3ms latency and 11.3GB/s peak write at 44.4ms. This completely smoked our software RAID5 group that came in with a not too shabby 26.2GB/s read at 19.1ms latency, but a sad 1.42GB/s write at 360ms. Software RAID historically has very poor RAID5/6 performance, with a massive hit on write performance.
Lastly, we moved back to a random workload, but this time a mixed read/write spread with an 8K 70/30 profile. Here the GRAID setup was able to continue to perform at a fantastic speed, measuring 1.59M IOPS at 1.3ms of latency, versus software RAID5 that came in at 101.3K IOPS at 20.2ms latency.
The GRAID SupremeRAID SR-1000 solution offers a lot to like if you need to leverage RAID across a group of NVMe SSDs. Traditional hardware RAID solutions have a limit of how much data can flow through them, which ultimately caps the bandwidth of the solution to whatever the PCIe interface card can support. GRAID mitigates that issue by sidestepping the data path. While software RAID solutions have been able to bypass the need for a physical card to route NVMe traffic, performance has historically been abysmal.
In our testing of the GRAID SupremeRAID solution, using eight Intel P5510 Gen4 SSDs in RAID5, the solution offered incredible performance in aggregate. This was in contrast to software RAID5 that came in at but a fraction of the performance, with a massive hit on anything involving write activity.
Highlights really speak for themselves. 4K random read performance through GRAID topped out at 8.88 million IOPS compared to just 2.9 million IOPS through software RAID. 4K random write performance the gap spreads even further, with GRAID measuring 863K IOPS compared to just 52.6K IOPS with SW RAID5.
Moving to a 1MB sequential transfer workload we saw 30.7GB/s read and 11.3GB/s write from GRAID, while SW RAID came in with 26.2GB/s read and 1.42GB/s write. Again if you only look for read performance, software RAID doesn’t appear that bad but write activity just crushes it.
If you’re going to make the investment in Gen4 (and soon Gen5) SSDs, you don’t want to be leaving performance on the table. The GRAID SupremeRAID solution is absolutely phenomenal, we were blown away by the efficacy of this simple to use card and accompanying software. Compared to traditional hardware RAID, or popular software RAID options, the SupremeRAID 1000 offers amazing ROI for demanding workloads.