How Benchmarks Work and When to Look at Them
When you buy a new gadget, you want the fastest and most powerful gadget you can get for your money. Tests can help you determine which devices have the most power, but sometimes they are just distracting.
What does benchmarking actually do?
While we use words like “fast” and “powerful” to describe our gadgets, actually measuring their performance is much more difficult than it sounds. For example, you can measure the clock speed of a CPU to see how fast it is, but if you get a CPU with multiple cores, it can do more even if each core is slower. Graphics cards generate tremendous processing power by combining thousands of relatively slow cores .
Benchmarks allow you to measure actual performance without getting bogged down in specs comparison. You don’t need to know how many cores or RAM a graphics card has if it displays Mass Effect: Andromeda at maximum frame rate and maximum detail. There are many different types of performance tests, but here are some of the most common:
- Computational tests measure the processing power of the CPU. At a basic level, processors are designed to do mathematical calculations very quickly. Programs like Passmark use various math tests to find out how much a processor can handle. For example, they can see how much data the CPU can compress in a given period of time, or how fast it can look for prime numbers. The more it can go, the more powerful the processor.
- Hard drive tests measure how fast you can read and write data. It doesn’t matter how fast your processor is if it takes forever for your hard drive to find the data you need to process. Passmark tests how fast you can read, write, and search hard drive data to rank different hard drive models. The faster the drive can read or write data, the less bottleneck it will create for the rest of your system.
- 2D and 3D benchmarks measure the performance of a video card. Packages such as 3DMark and Heaven Benchmark test video cards and GPUs by testing how fast they can display both 2D and 3D graphics. The more frames they can display per second and the more detail they can display in each frame, the better their score will be.
- Smartphone tests measure every component of your phone. Your phone is a complex network of components that are specifically designed to work together, which means isolating one component is often harder than it sounds. Packages like Antutu measure everything from CPU speed to UI rendering speed and even how fast you can read / write to SD card or internal storage to give phones an overall performance rating.
Some benchmarks use real-world situations to measure performance, such as playing Crysis , while others use more general benchmarks . Basically, they are all designed to show how powerful a device is, but benchmarking also has its limits. For example, Samsung phones can get high scores in performance tests, but if Samsung or Verizon floods your phone with malware , you could still end up with a slower phone than if you bought a Pixel phone without extra points with lower scores. So when do tests really matter?
When to Use Benchmarks to Make a Purchase Decision
Benchmarks are most useful when you are measuring something specific, such as the performance of a video card. They can also be handy if the type of device you are looking at has so many characteristics and variables that direct comparison is impossible. Again, like a video card. Below are the situations in which performance tests can be most useful.
To find the best graphics card for your PC’s gaming needs
If you’re building a gaming PC, you only need a graphics card to do one thing well: to make your games look good. Companies like NVIDIA and AMD have so many different models and features for their graphics cards that finding the “best” one becomes a daunting task. Benchmarks can be of great help here.
Sites like Passmark and Futuremark keep a current list of video cards evaluated by their test suites. They also show the lowest price for each card from various online stores. Futuremark also shows the popularity rating of each card so you can see what other users are buying. If you want to upgrade to a higher tier, find your card in the list, then scroll up to find something with a higher score within your price range. This will be the surest bet that you will get the best experience.
Get a processor capable of handling resource-intensive projects like video editing
For the average user, virtually any CPU available will be sufficient for basic tasks such as browsing the web or checking email. However, if you are doing heavy tasks like video editing, graphic design, or 3D modeling, you may need something more powerful. In this case, tests can be helpful.
Passmark divides its rating into several charts to help you narrow down the type of processor you need. The high-end table ranks processors designed for high-performance gaming and editing devices as well as server-grade processors. For more moderate needs, Passmark’s high mid- range list compares more traditional desktop processors if you just want to improve your regular gaming PC without spending a ton of money.
To find faster SD cards for your camera, phone, or Raspberry Pi
Slow SD cards can slow down the performance of whatever you insert them. Whether they force your camera to save images indefinitely or slow down the apps on your phone, they can be a painful bottleneck. SD cards are assigned class ratings based on their minimum write speed, but even within certain classes, some SD cards are much faster than others.
Tests SD-card to find a little more difficult, but such sites as the Pi Dramble and TechFunology, offer a comparison of some of the most popular cards. For casual users, you can just focus on getting a higher grade SD card ( grade 10 is the fastest , with a minimum write speed of 10MB / s), but if you want the fastest card for your Raspberry Pi project or your professional camera photography, visit the sites above to see which are the fastest.
When it is safe to ignore control tests
For devices like smartphones that perform a wide variety of tasks, generic tests can give you a vague idea of how powerful they are. However, your actual use is likely to be so different from your testing conditions that it makes the benchmark almost meaningless. Here are some situations in which you might be better off ignoring the tests.
You can look at smartphone tests, but don’t buy based on them
Antutu maintains a current list of smartphone tests , ranking them according to their performance. This is good information to have, but it is hardly the only factor that you take into account. A phone that scores well in tests may have poor battery life, have a poor camera, or may not work on your carrier’s network.
More importantly, the test result only matters if you plan on using your phone to the limit. If you use your phone to check Gmail and read the news, the insanely powerful processor won’t do you much good. It might be better for mobile gaming, but it depends a lot on how well the game developers are using this power. If you want to keep your phone up to date with tomorrow, it might be a good idea to pick something from the top 20 or so, but don’t sacrifice other features just to get better benchmarks.
It’s also worth noting that the top-rated Android phone on Antutu’s benchmark list is OnePlus3T, which has been accused of artificially inflating the test results . While the mechanism they used to do this is complex and not entirely cheating, it also means that the score does not necessarily reflect day-to-day use. This is not the first time a manufacturer has been accused of trying to tune their system for favorable results . This is another reason why you should be suspicious of phone tests and not the reasoning behind your buying decision.
Skip the hard drive tests and just buy an SSD
Sites like PassMark and UserBenchmark offer extensive ranking lists of both old school platter hard drives as well as solid state drives. However, if you are not building a server for business, you can ignore them. If you’re using a platter drive, the best speed boost you can get is an SSD. If you already own an SSD, you’re better off buying based on how much data they can store or how reliable the particular model is.
Some older platter drives are faster than others (for example, 10,000 RPM drives will be better than 7,200 RPM drives), but SSDs do not use platters at all. This makes them much faster. In fact, almost twice as fast as conventional hard drives . If you have an older hard drive, don’t worry about benchmarks. Just buy an SSD. If your computer is still slow, you will probably see a lot of benefit by updating other components.
Depending on your needs, you may care more about the tests than the average person, and that’s okay. If you’re building a high-performance NAS or buying a phone to write applications, you might want to pay more attention to benchmarks. Overall, however, tests are a useful tool that should not always be regarded as gospel.