Why so Much Psychological Research Fails to Live up to Expectations
One of the key aspects of any scientific research is that, no matter what, its results are challenged and verified again. A number of popular psychological studies, such as the idea that smiling makes you happier or that willpower is a limited resource, have fallen short of this test. They are not entirely contrived, but they are not final either. Here’s what’s really going on.
Researchers can manipulate results to fit different theories
Back in 1998, researchers published an article titled ” Ego Drainage: Is the Active Self a Limited Resource?” “In the journal” Personality and Social Psychology “, which completely changed our understanding of willpower. The idea was simple: put some cookies on a plate next to a bowl of radishes, have the participants hang out in the room, and then test them out with an insoluble puzzle. People who ate radishes gave up the puzzle earlier than people who ate cookies, which showed researchers that using willpower (not eating cookies) employs a kind of reserve of willpower. The more self-control you use at the moment, the less you will have to use in the future. Since this landmark study in 1998, ego depletion, which includes self-control or willpower, has become a huge subject of research.
We’ve mentioned this idea of limited willpower a couple of times because, in fact, it seems to be true. In fact, there have been many similar studies done over the years that seemed to support this idea.
In 2010, a meta-analysis of 83 studies of ego depletion published in the Psychological Bulletin showed overwhelmingly that ego depletion is a reality. Then, oddly enough, another group of researchers analyzed much of the same data in an article in Frontiers in Psychology and found no evidence of ego depletion. The same researchers conducted a subsequent meta-analysis in the Journal of Experimental Psychology, which added unpublished research and again found no evidence for ego depletion. We have already talked about bias associated with data analysis , and here it is quite obvious. Researchers can manipulate data to achieve any goal.
So what does all this mean to you as a normal person? This means that scientists are not entirely sure how willpower or self-control works. Self-control may disappear, but we don’t know exactly why. We can have a reserve of willpower that we use every time we make a choice. Or not. After all, ego depletion is still a theory to be tested.
For most of us, it will probably be better to go back to blaming bad decisions as general fatigue. Instead of worrying about some imaginary reserve of willpower, just try to avoid these tempting situations altogether.
Your experience plays a role that is difficult to verify
Have you heard that the mere thought of money makes you selfish? According to a 2013 article published in the Journal of Experimental Psychology , this is certainly the case. In the study, participants completed two questionnaires, one with a solid background and one with a faint background image of a $ 100 bill. They found that participants who saw the $ 100 bill had less empathy for victims and disadvantaged groups and even slightly changed their political views on the free market. Given that we live in a year of big elections, this idea of why some people are prettier than others is certainly in the hands of many. This and subsequent studies by the same researchers have generated headlines such as “ How Money Makes People Become Less Human ” and “ Just looking at cash makes people selfish and less social. “
However, when 36 laboratories tried to repeat the tests, they did not get the same results. Their research, published in Social Psychology, was part of the Many Laboratories replication project designed to replicate psychological experiments internationally. Of 36 laboratories, only one reported the same significant effects as in the original study.
A 2015 study , also published in the Journal of Experimental Psychology , attempted to replicate the same 2013 test on a larger scale with a larger sample size, but failed to obtain the same results.
In response, one author of the original study pointed to several other studies over the years that have yielded similar results, namely that the mere thought of money makes people selfish and even antisocial. This, again, is all there is to it that we still have evidence from both sides.
So what’s really going on here? Should you avoid looking at $ 100 bills? Do you suddenly become an asshole every time you open your wallet? Does the mere fact of viewing a $ 100 bill turn you into a free market capitalist? Depends.
There is an argument here that the biggest influencer is how important money is to you from the start. If it’s important to you, it might make you more selfish, if not, then there probably won’t be any changes in your answers. Or maybe it’s not a factor at all. We don’t really know, this is just a theory. For now, you’re probably better off not paying attention to how much money you have and just fixing your habits .
Social sciences are inherently difficult to test
You’ve probably heard this before: smile and you feel better. There is even research to support this. A study published in 1988 in the Journal of Personality and Social Psychology found that people thought Far Side comics were funnier when researchers made them smile. This is the kind of nonsense that fills the heads of authors of books on self-development. For similar studies were followed by similar results, including one published in Cognition and Emotion, in which tested the effect of a frown, and the other of the journal Creative Research Journal suggests that one and the same pen between the teeth can make people more creative. Unfortunately, these studies are not as straightforward as you might think.
Earlier this year, 17 different laboratories tried to replicate the results of a 1988 experiment, and an article published last month in the journal Perspectives in Psychological Science found their results to be inconsistent with the original study. In the end, nine of the 17 laboratories received similar results to the original study, but the size of the difference was smaller than in the original study. The other eight laboratories either found no evidence of higher ratings or showed completely opposite results.
But that’s not all. There were some differences in how the study was conducted in duplicate, the largest of which was that the participants were videotaped. It can be argued that the participants changed their behavior because they knew that the camera was looking at them. To complicate things a bit, it’s also worth noting that while using the same Far Side cartoons from the original 1988 study might seem like good practice, participants in 2016 definitely have a different sense of humor than people 28 years ago. This alone could skew the results.
Research is difficult to replicate, especially in the social sciences. Even if you do it from a book and reproduce the study accurately, the results may be different. This does not mean that the original results are incorrect, but it does mean that it is worth looking in more detail.
As a more recent example, we recently pointed out that the results of another primary study had been challenged. In this case, the idea was that imitating a specific body language, “strength postures”, reduces stress and improves performance. In terms of strength poses, one of the co-authors, Dana Carney, spoke up and said the results were statistically altered to get the results the researchers wanted. She even adds that the study was not that big , stating that too many participants understood the hypothesis they were testing, which should automatically negate the results.
Whether it is forceful posing or smiling, the question is what effect these priming techniques might have on behavior, if any. They may fall under the mantra “If it works for you, keep doing it,” but it is just as important as the reader to remain skeptical. None of these studies, however fancy heading you read, apply to 100% of people. There are no magic bullets and universal advice, so always be wary of any research that suggests there is one solution to a common problem.
Recently, a reproducibility project was launched with the support of the Association for Psychological Science . The aim of this project is to reproduce psychological research, analyze the results, and propose more reliable testing methods. Replay failure rates are now quite high: about 64% of studies fail. It sounds grim, but it means more tests are needed and that our long-standing beliefs are worth challenging as often as new research.