Research is a tricky business. Why? Well, because it is about people and studying people is a challenge.
In 1958 a man called Henry Landsberger was investigating a series of research studies that had been conducted between 1924 and 1932 by a company called Hawthorne Work near Chicago. They had commissioned a series of studies to find out if the level of light within the building affected the level of productivity of their factory workers. I mentioned this series of studies also in the 6 don’ts of user research. The researchers found that whenever they manipulated the lighting productivity went up. In fact, it went up even when the changes were imperceptible to the human eye (the equivalent of not making a change at all). Going through the documents, Landsberger identified something we still refer to as the Hawthorne effect today. The rise in productivity can be explained by the simple fact that the workers were aware of being observed. As a consequence, they were more diligent and more productive. This effect is also called observation.
We can roughly distinguish between biases that affect the subject of research and biases that affect the researcher.
1. Observation bias happens as mentioned above when the subject is aware of being observed and modifies her/his behaviour accordingly.
2. Social desirability is the tendency of survey respondents to answer questions in a manner that will be viewed favorably by others. It can take the form of over-reporting “good behaviour” or under-reporting “bad”, or undesirable behaviour.
3. Social conformity was studied in an experiment conducted by Solomon Asch in 1951. The focus was to investigate the extent to which social pressure from a majority group could affect a person to conform.
50 male students were invited in for a ‘vision test’. Only one participant of the group was a real participant. All the others were part of the experiment. On some occasions during the experiment all the other “participants” unanimously choose the wrong line which had an impact on the answer of the real participant. Asch conducted several variations of this test and participants gave different explanations to why they went along with the wrong answer of the group. Without going into detail, it became clear that the group has a significant influence on the individual to conform.
Be aware of this wicked effect when you use research methods that are conducted in group settings. Individuals might conform to the group opinion although they might have other points of view or opinions.
1. Interviewer bias happens when the interviewer may subconsciously give subtle clues with body language, gestures or tone of voice, that subtly influence the subject into giving answers skewed towards the interviewer’s own opinions, prejudices, and values.
2. Wording bias can happen in several ways but usually, it happens when the researcher elaborates on respondents answers and by choosing certain words over others put words in their mouth. Another example of wording bias is leading questions which are questions that prompt or encourage a certain type of answers. Example: Don’t you hate when this happens?
3. The Texas sharpshooter fallacy comes from a joke about a Texan who fires some gunshots at the side of a barn, then paints a bullseye target around the spot where the most bullet holes appear and claims to be a sharpshooter. Known also as confirmation bias, this happens more often than you think. It is the tendency to search for, interpret, favor, and recall information in a way that confirms one’s preexisting beliefs or hypotheses while giving disproportionately less consideration to alternative possibilities. In research, it occurs when a researcher forms a hypothesis or belief and uses respondents’ information to confirm that belief. Designers sometimes fall prey to this when they fall in love with their idea or design. It is a selective collection of evidence and it is very dangerous.
Research is not without flaws. Data is not without flaw. People aren’t without flaws. To make sure our research results are valid and valuable we can verify them through triangulation.
Triangulation means that you study something from more than one standpoint.
Read my next post about how triangulation can help your research.