How PCWorld tests and evaluates external storage

Modern external computer data storage technologies (DAS, or Direct Attached Storage) run the gamut from USB hard drives, to optical discs (yes, still), to USB thumb drives, to the latest, greatest NVMe SSDs in high-speed Thunderbolt or USB 4 enclosures.

PCWorld has been reviewing external storage since the 1980s and the era of floppy drives, though these days we stick with SSDs and the occasional hard drive. Everything else seems, well… quaint. Not to mention, drearily slow.

What does PCWorld look for in an external storage device?

Capacity

Generally speaking, more capacity is better, but determining capacity takes no effort on our part — it’s generally written plainly on the box. Capacity by itself doesn’t score a lot of points unless it’s a jump from the norm, however…

Price

…the cost per gigabyte counts heavily in our evaluations. That said, the star rating is weighted more toward the quality of the product, not how much it costs. But all things being equal, the cost per gigabyte can tip the scales.

Performance

Basic performance obviously depends on the technology involved: HDDs, SSDs, USB, Thunderbolt, etc… We run benchmarks and perform real-world operations to determine its speed relative to others of the same ilk. Performance relative to a device’s peers as well as in the grand scheme counts greatly in the final rating. There’s far more granular info on our testing methodology and consideration below.

Durability

Of course, speed is nothing if a device gives up the ghost in six months. HDD failure has become progressively rarer since the bad old days when up to 15 percent of hard drives failed per year. Early SSDs also had their issues, but the only problem we’ve heard of recently was with the SanDisk Extreme Pro line. Every SSD we’ve tested is still going strong, as are all the hard drives from the last five years or so.

That said, SSDs are still a relatively young technology, and NAND blocks/cells can only be written to so many times. We make special note of the TBW ratings for SSDs. That’s an acronym for terabytes written, or the number of terabytes that may be written before a drive runs out of replacements (overprovisioning) for worn-out cell blocks.

MTBF (mean time before failure) ratings for HDDs are a guess at best, and rarely accurate or provable, so we generally don’t pay attention them.

Seagate Ultra Touch HDD

Seagate Ultra Touch HDD

Seagate Ultra Touch HDD

Given the limited time we have between testing and publication, 100 percent accuracy on long-term reliability is impossible. We keep our test units in service, but it can take years for a problem to crop up.

One thing we can check is how well an external device sheds heat — the general enemy of electronics. If the temperature of an SSD gets too high, the controller will slow down/throttle operations to reduce said temperature to avoid damage to the components.

We also check the quality of construction and the materials used. With SSDs, shock mounting and shock resistant silicone covers are plusses, but with HDDs they’re an absolute must.

Many devices are IP rated — i.e., designed and tested to ward off dust and liquids. We’ll always discuss this with any ruggedized device, and it does score brownie points.

Portability

When it comes to portability, smaller is obviously better. Thumb drives are generally the smallest and most portable. Their integrated Type-A or occasionally Type-C connector eliminates the need for an additional USB/Thunderbolt cable or power adapter, which takes up space and which you must remember to bring with you. If a device is designed to be portable, the overall ease of transport is obviously considered.

Jon L. Jacobi

Jon L. Jacobi

Jon L. Jacobi

Devices designed for your desktop might be portable to a degree, but a bit of size and heft is in order. Anyone who’s ever inadvertently sent a small, light drive flying across the table (or room) understands. It’s especially true for 2.5-inch external hard drives, which can’t absorb nearly as many G’s as an SSD. With desktop drives, we also rate such stability — i.e., we check for non-skid feet, etc.

Power requirements

AC adapters are required with 5.25-inch hard drives (still the only way to get more than 8TB in a single unit!), and with very large-capacity USB 3.x SSDs. Single, external 2.5-inch hard drives rarely require an AC adapter; however, some RAID units with more than one drive will. At least non-Thunderbolt types.

Jon L. Jacobi

Jon L. Jacobi

Jon L. Jacobi

Speaking of which, both USB 4 and Thunderbolt deliver more power over their connections than older USB 3.x, which is why you generally won’t need external power unless, as mentioned, there’s more than one drive in the enclosure.

One exception to this is when using Thunderbolt generational adapters (Thunderbolt 2 to Thunderbolt 3/4), which may not pass through power. In that case, external power, or a dock with power, may be required.

How does PCWorld test external storage?

Connectivity/bus speed considerations

Connectivity, i.e. the transport protocol or bus type, is the essential indicator of expected performance and compatibility and obviously the first thing we consider. Highly compatible 5Gbps USB SSDs will max out somewhere south of 500MBps, while also highly compatible 10Gbps USB SSDs top out at around 1GBps.

20Gbps USB 3.2×2 drives can reach 2GBps, but only on the still somewhat-rare 3.2×2 connector or USB 4. But even that’s not a given. While Macs (and many PC Thunderbolt ports) support USB 4 20Gbps, they only support 10Gbps when using USB 3.2×2 SSDs. Why? Don’t know. USB 4 and Thunderbolt 3/4 SSDs can operate at 40Gbps, or around 4GBps.

In USB 4 and Thunderbolt 3’s case, it’s not actually specified that they support anything greater than 20Gbps. Yeah, it’s a mess of a “standard,” but we always test drives on their native bus at their highest possible speed. Note that the Thunderbolt 4 ports on our test bed also support all flavors of USB, including USB 4.

All USB drives are backwards compatible to at least USB 2.0, but they’re always limited to the speed of the bus they’re on.

Generally speaking, though faster is always better, we compare external storage devices with their technology peers: 10Gbps USB to 10Gbps USB, Thunderbolt 3 to Thunderbolt 3, hard drive to hard drive, etc.

Internal technology considerations

The technology inside a device (SATA, NVMe) also has a lot to do with its expected performance. For instance, a SATA SSD will never go much faster than 500MBps even across USB 4 or Thunderbolt 3/4. SATA HDDs rarely top 250MBps (3.5-inch) or 120MBps (2.5-inch).

NVMe on the other hand is fast enough that the bus is a major limiting factor. For example, the fastest internal PCIe 5.0 NVMe SSD we’ve tested clocked almost 15GBps, while the fastest Thunderbolt SSD could muster only 4GBps.

All HDDs to a small degree, and SSDs to a large degree write slower as they fill up so the benchmark numbers we publish are best case. Drive slowdown is why we recommend that you always buy at least twice the SSD capacity that you might think you need. This is not generally necessary with HDDs, but a good idea.

Again we generally compare similar internal technologies in our reviews: SATA vs. SATA, 20Gbps USB vs. 20Gbps USB, etc.

NAND and SSD design considerations

When it comes to testing, SSDs have “special needs.” Mostly because of the way they cache data to achieve and maintain best performance. Primary caching is either done with onboard DRAM, which offers the fastest random performance, or in the case of cheaper Host Memory Buffer (HMB), your device’s system memory. Recent implementations of the latter have turned in startlingly good numbers with sequential transfers.

But the primary cache is generally a relatively small amount. A couple of gigabytes with DRAM and 58/64MB with HMB. Hence SSDs use another method as secondary caching.

A controller can write 3-bit (Triple Level Cell/TCL) or 4-bit NAND at either their native bit depth or, far faster, as on/off 1-bit SLC. This single-bit writing is used for secondary caching. Data is later rewritten at the greater bit depth to reclaim the lost space.

The reason writing at greater bit depths is slower is due to the error-checking required to determine if the correct voltage (one of 8/16 levels with TLC/QLC) was written to a cell, rather than the simple charged/not charged binary of 1-bit SLC.

How vendors handle this secondary caching technique mostly affects performance during very long writes. Some fix the amount of NAND that can be used for secondary caching, while some allocate it dynamically. We often have to rewrite the 450GB file several times, depending on the capacity of the SSD, before the drive will drop to its native 3-bit/4-bit write speed, which can vary anywhere from 150MBps (slow QLC) to nearly 1GBps (faster TLC).

Note that as a drive fills up, performance may suffer as there is less NAND available to allot as secondary cache. We test with empty drives but warn users that performance may degrade over time. It’s also why we recommend that you overbuy in terms of capacity.

Many NMVe SSDs use DRAM for primary caching and some use your computer’s own memory as cache (Host Memory Buffer/HMB). The former greatly improves random performance, while the latter can actually improve sequential performance.

The number of chips on an SSD can also affect performance, as the more chips there are, the more data paths there are to shotgun data over. This is, again, why we warn you in each review that our results apply only to the capacity SSD we tested.

All these factors must be accounted for and inform our modus operandi when it comes to testing SSDs.

Further reading: Best external drives

What tests does PCWorld run to determine how fast an external storage device is?

Benchmarks

NVMe SSDs, as they have matured, have broken the nearly 1:1 ratio of benchmark to real-world transfer performance that existed right up through SATA SSDs — drastically, when it comes to internal SSDs, and to a lesser, but still noticeable extent with bus-constrained external SSDs.

The benchmarks aren’t errant, they simply use their own I/O, which has matured to take advantage of NVMe’s unique capabilities such as multiple threads and queues.

Operating systems, and the vast majority of software have not. For example, Windows Explorer never uses more than a single queue or thread. Hence our test transfers proceed at a far slower pace than what’s actually possible, and spoken of by the benchmarks. A drive capable of reading at 15GBps in benchmarks, will barely manage 4GBps under Windows.

We benchmark using CrystalDiskMark 8 and AS SSD 2.0. There are other perfectly valid benchmarks, but these are two whose results have consistently matched the level of the technology involved over the years.

Also, we have a huge set of results from years past to compare. On occasion, at vendor request we will run other benchmarks such as ATTO or IOmeter to simulate a particular workflow that a drive has been optimized for.

Real-world tests

To augment the benchmarks and give users an idea of what they’ll actually see under Windows, we perform a series of real-world transfers. Firstly, a 48GB folder full of smaller files and folders, and a single 48GB virtual hard drive (VHD) written to and read from the device. The second drive is actually a 58GB RAM disk that is faster than the fastest NVMe SSD we’ve tested to date. But it’s getting to be a close contest, so we may have to come up with other means in the near future.

Our final test is writing a single 450GB VHD file to the drive. This is not to measure performance as such, but to test the secondary caching. See above.

Note that both our 48GB and 450GB VHD files are filled with data, to minimize the impact of any compression techniques. Some controllers can compress all zeros with lighting speed, which throws a monkey wrench into the results.

What hardware does PCWorld use to test external storage devices?

We test all storage products on a dedicated X790 (PCIe 5.0) motherboard/i5-12400 CPU combo with two Kingston Fury 32GB DDR5 modules (64GB of memory total). The Intel integrated graphics are used as well as Windows 11 (22H2) 64-bit. The 48GB transfer tests utilize an ImDisk RAM disk taking up 58GB of the 64GB total memory. The 450GB file is transferred from a Samsung 990 Pro 2TB, which also contains the operating system.

Jon L. Jacobi

Jon L. Jacobi

Jon L. Jacobi

On occasion, should any aberrations arise on the main test bed, we also test on our older AMD test bed. This consists of an MSI MEG X570 motherboard socketing an AMD Ryzen 7 3700X 8-core CPU and 64GB of Kingston DDR4. The same software is used as on the Intel platform. This machine is also the software test platform.

For Mac-oriented products or those tested for Macworld, a first-gen Mac Studio M1 Max with 32GB of memory and a 1TB internal SSD is employed. This system has the aforementioned interesting limitation common to all Apple Silicon Macs — it supports USB 4 20Gbps, but does not support USB 3.2 2×2 20Gbps. Sigh.

How does PCWorld determine an external storage device’s rating?

As noted, our overall ratings reflect a lot of factors, but the performance/cost ratio carries by far the most weight. And with only a 10-level system (five full stars, five half stars), the star ratings aren’t particularly nuanced.

Therefore, it’s important to actually read the reviews for the particulars, and at least the pros/cons to see the major strengths and weaknesses when trying to decide which storage device you want to buy.