You’ll Never Know How Accurate Your Fitness Tracker Is, and That’s Okay
The appeal of smartwatches for fitness tracking is that they have all the answers. They turn the unfathomable mysteries of our soft bodies into precise numbers that we can easily read and analyze. But we’d be fooling ourselves if we believed that our smartwatches always tell the truth. According to a new scientific analysis , not only are wearable devices often wrong, but it’s also impossible to ever know how accurate they are.
This won’t be shocking news to longtime Lifehacker readers. We discussed the fact that some smartwatch metrics are more reliable than others , and that calorie burn is one of the less accurate ones . On the other hand, heart rate variability across devices shows different raw numbers, but all the major recovery-focused devices manage to capture the same rough trend —if you trust my home study with a sample size of one.
So, what do we know about the accuracy of smartwatches on the market, and why is this question so difficult to answer? That’s the problem that a recent analysis by a team of sports scientists and data scientists in Ireland aims to answer. This is a broad overview—a study of studies of studies—that aimed to collect all relevant published data on consumer wearables. Here’s some of what they learned.
Research becomes outdated immediately after publication.
You would think that someone at Apple, Garmin or Fitbit would do thorough research on their technology before introducing it to the public. And they probably do that internally, but their goal is to launch and sell a product, not to test the accuracy of their product compared to others.
So the research that we have is usually done by scientists and it starts after the wearables hit the market. It usually takes at least two years to research a new smartwatch and publish it. By then, these smartwatches won’t be so new anymore.
This new analysis, published in July 2024, used the most recent meta-analyses available, which in turn used the most recent studies available . And what models of fitness watches did they include? I looked at additional charts of the newest models from each major brand. They included:
-
Charge 4 from Fitbit ( Charge 6 released last year )
-
Apple Watch Series 6 (most recent is Series 9, released again last year along with Ultra 2 )
-
Garmin Fenix 5 (Fenix 8 just released)
-
Garmin Forerunner 245 (still popular, to be fair, but the 255 and 265 have since been released; the 265 is a year and a half old now)
-
Oura Ring 2nd generation (now up to 3rd generation)
-
Whoop 3.0 (current model is 4.0)
So if you want to see how the Apple Watch Ultra 2 compares to the Charge 6, Forerunner 265, or Whoop 4.0, you’ll have to wait a few more years—by which point everything will have moved on. number or two.
Accuracy studies are not conducted consistently.
The studies are so varied that it’s hard to compare them to each other, even if you’re interested in learning about older device models. A scoping review found that most of their studies underestimated heart rate and overestimated sleep time, for example, but the authors concluded that they couldn’t really say that wearables in general overestimate or underestimate these things. The studies were too different from each other, each evaluating multiple devices and rarely using the same gold standard metrics to compare them.
“This broad overview reveals complex variability across devices, outcomes, user context, and reference standards,” the authors write, “making it difficult to definitively assess the accuracy of wearable devices.” In other words: we don’t have the data to answer the questions you ask when you go shopping for a new device.
What were the best and worst performance?
That said, if we can take the results with a grain of salt, I think it’s still worth looking at what the general review found. Here are some common features, although we definitely can’t say they are universal:
-
The heart rate was usually correct within +/- 3% of the true value. That’s not bad, but a 6% window is still a lot when you’re trying to keep your heart rate within the 10-point zone .
-
Heart rate variability was “very good to excellent” when readings were taken at rest, but accuracy dropped when readings were taken while moving.
-
Energy expenditure (calorie burning) was low, which we already knew . Sometimes devices were underestimated by 21%, sometimes overestimated by 14%.
-
The number of steps was also quite variable, ranging from 9% less than the actual number to 12% more.
-
Sleep duration was typically overestimated, and sleep latency (how long it takes you to fall asleep) was typically underestimated.
It is more important to ask whether something is useful than whether it is accurate.
To be honest, I don’t judge wearables by whether they are accurate, only by how useful they are. You may remember from my comparison of Whoop, Garmin and Oura that each of the three devices reported different raw resting heart rate and heart rate variability values, but they were all able to track the same trend, providing me with possibly useful information about when my body was well rested and well recovered compared to when it wasn’t.
It’s because of this view of utility that I try to keep people from focusing on burning calories. If you really want to know how many calories you need to eat to maintain your weight, your best bet is to track how many calories you eat and also monitor your weight . Likewise, rather than blindly following the watch’s estimates of whether you’re in zone 2 during a workout, you can use other cues like breathing and internal monologue (“oh my god, when is this going to end?”) to determine how hard we work for you.
While we can’t test the accuracy of every tracker, I know that accuracy is important to most people who buy smartwatches and fitness trackers, so I will continue to cover this topic where appropriate. A GPS-enabled watch should show you the street you’re actually running on, and the heart rate sensor shouldn’t confuse your running cadence with your heart rate. But the most important questions to ask about a wearable device aren’t whether its metrics are accurate , but whether they ‘re useful , even knowing they might be inaccurate.