PsychNotes Index

More PsychNotes: Policies and Issues in Mental Health

Evaluating Psychological Information in the Information Age
by Monica A. Frank, PhD

A problem frequently discussed today is how much inaccurate information we consume on a daily basis. Certainly, most of us know to question information that comes from the neighborhood gossip. But what about information from what we believe to be reliable news sources? We will never stop the proliferation of false information but we can become better consumers of information.

Waiting in a doctor's office I picked up a popular women's magazine and was confronted with another misrepresentation of psychological research. It said “studies show that having unused vacation days makes you less likely to get a raise” and concluded with “so force yourself (to take a vacation) and you may just come back to a bigger paycheck.” Other than the obvious fluff this article illustrates some of the problems with the reporting of psychological information.

What are the problems with this statement?

1) No identification. I haven't read these “studies” they refer to because they didn't indicate where they obtained the information. I realize that magazines don't provide citations. However, most at least provide a name of an author or the university where the study was completed so that the research can be tracked down. Instead, this magazine just referred to some allusive “studies.”

2) Inferring causality. Knowing how psychological research is conducted, I doubt the study said “makes” which is a statement of causality. In other words, it is the same thing as saying “unused vacation days causes you to not get a raise.” The only way research can make such a statement is if they did a randomized study. In this case it would mean having two groups of people at the same workplace who participated in a study their boss didn't know about and one group didn't take vacation days while the other did. That does not sound like a practical study, does it? So, at the most, this research could only show a relationship between vacation days and pay.

3) Other explanations. Since the research can only show a relationship, it can't tell the why of that relationship. For instance, maybe people who take vacation days are more confident and assertive. This is what is known as a “third variable.” So it's possible that it is not the vacation days that causes people to be paid more but personality factors influence both pay and whether a person takes vacation days.

4) Reverse reasoning. Even if the research does show that not taking vacation days is related to not getting a raise, that doesn't mean the reverse is true: that taking a vacation might lead to a raise as this magazine article states. This is reverse reasoning. A very common example is when the research that indicates most abusers were abused as children is reversed to say most abused children will grow up to be abusers. Reversing the research is a completely misleading statement.

Why is this important?

We are confronted daily with information that influences our lives. We make decisions based upon this information. Maybe a lot of the time it doesn't have a tremendous impact on our lives but sometimes it can such as when you are making health choices or major decisions.

For instance, widely publicized research indicating that C-PAP machines don't reduce heart disease may lead some people to not use their C-PAP as instructed. Yet, a careful reading of the research shows that the possible reason for the conclusion is that people aren't using the C-PAP long enough (study participants averaged 3.3 hours a night on the machine when 6 hours is optimal). The popular media didn't report this aspect of the study but instead had glaring headlines that C-PAP doesn't help heart disease (further reading).

Using inaccurate information can lead to poor decisions. So it is important to question the information you read or view. The above provides a few guidelines for doing so.