User Research: Smoke and Mirrors

The article delivers an interesting argument about design-driven qualitative research versus QA- and performance-oriented quantitative research. It questions and re-explores the usefulness as well as the skeptism of quantitaive research; whether the numeral results are a good justification for the basis of certain design decisions.

Fahey argues that to many user experience designers and firms, the wide options of scientific tools available and its (overrated?) value given to those tools is sometimes seen as a facade to doing the real job: 'being expert designers who draw on deep experience and good instincts.' Instead of relying blindly on statistical results, experience designers should pry deep into the minds of users and get to the crux of understanding their experiences, which provides far more accuracy than what mere numbers can tell.

The article also debunks the 'fad' over the eyetracking design tool, saying that it provides only a superficial understanding of what users are looking at. Also it cannot making meaningful interpretations but only generalizations. For instance, if a user is look at a particular spot , does it mean that it is because it is pleasurable to do so, or is it because the user is trying to make sense of what it is? Since each person's evaluation on a product is so subjective, how then is it possible to use objective methods and 'cold hard' numbers to understand a subjective issue?

Fahey however, talks about making use of scientific research when it comes in handy, especially when politics come in play- when pitching your ideas to bosses or stakeholders who probably accept or understand nothing but numbers, to convince them them adopt or fix the problem at it's right spot.

It seems important not to belittle what user experience research can offer. User experience research, conducted through relatively cheaper methods like card sorting, ethnographic observations, personas etc. can help to steer design decisions in the right direction and probably even provide better insights than what fanciful and expensive quatitative tools can do.

It seems easy to speak of it, but in actual fact, user experience research is not easy as it sounds. Because of its subjective nature it is difficult to get a definite solution to a design problem. On top of that, it is not the culture of users to 'think aloud' and vocalize their thoughts. Moreover, interviews conducted during usability testing must be carefully conducted such that it is not steered to a biased answer. Comments from the users should come intuitively and naturally. Hence, careful thought must be in place whenever setting tasks and interview questions. The difficulty to grasp the user's trail of thoughts because it is so random is, in my opinion, the main obstacle that most experience designer must tackle. Beyond that it is the materializing of user's thought into design solutions.