bias in surveys

It is easy to get the answers you want from a survey. All you have to do is write the questions to 'encourage' the answers you want. Of course, the purpose of a survey is NOT to get the answers you want, so this is more of an example of what NOT to do.

Here is a good example. This is a survey to see if the respondent has libertarian views.

Question 1a) Do you believe in free speech? Answer 1a) Of course!!!
Question 1b) Do you believe that terrorists should be able to promote hatred of the United States on college campuses and in town centers? Answer 1b) Well . . . .

Question 2a) Do you believe that the government should end corporate welfare and stop giving handouts to companies? Answer 2a) Strongly!!!
Question 2b) Do you believe the government should provide R&D support for alternative energy and other promising new technologies? Of course they should!!

As you can see, the average person would come out as a libertarian using survey a) but not in survey b).

This also shows the importance of using examples. When you use an example, the question's implication becomes much clearer. But what you have to do to ensure objectivity is to provide an example on both sides so that you don't bias the answer.

negative political ads

This week, the Virginia Senate race really heated up (article). The Republican incumbent George Allen read a passage from a novel written by Democratic challenger James Webb about 30 years ago. The novels are historical and based on experiences that Webb had as a journalist in Bangkok. But the scenes in question are very sexually explicit and I can see how they would offend the conservative voter.

But what I want to blog about today is Webb's response. I think he did it exactly the wrong way. From a Human Factors point of view, what Allen did was create connections in voters' minds from Webb to sexually explicit scenes. Anyone against explicitness will have connections to negative affect from this. And no one will really have positive connections, because while many people are in favor of free speech, no one is really in favor of explicitness in itself.

But Webb's response just magnified this. He talked about how legitimate these scenes are because they come from his real experiences. But all this does is create salient repetition in voters' minds from Webb to the sexual explicitness. For those who are offended, his logic wouldn't sway them at all, and may make it worse. And for those who aren't offended, it strengthens connections that have no effect. Either way, he doesn't accomplish anything positive for his campaign.

What he should have done is create brand new connections that would appeal to different kinds of voters while ignoring the specifics of the explicit scenes. For example he could have said something like:

"This is clearly a desperate attempt by my opponent to bring up 30-year old news that has nothing to do with the real issues of this Senate race. I am sure that the citizens of the great state of Virginia can see through this ploy and will vote based on the issues on November 7th."

By avoiding any mention of the sex, he avoids strengthening these connections. And he creates several beneficial connections of his own: That Allen is desperate, that he has a high opinion of Virginia voters' intelligence, that the novel is from many years ago, and that it is irrelevant (whether it really is or not).

Whoever advised Webb on his response needs to study his/her human factors.

conflicts of interest

Disclosure is clearly not the answer to conflicts of interest. A 2005 study in the Journal of Legal Studies (recently cited at Business Week) found that when someone discloses a conflict of interest, they are more likely to exaggerate their position. And those who receive the disclosure do not add the grain of salt they should. So the practical implication is that you should admit to any conflicts without worrying about consequences and then feel free to make up whatever you want (yes, I am exaggerating here !!!).

So what does human factors say about this? My sense is that when making any point, there are two competing incentives - personal payoff, which has high salience and strong low-brain instinctive connections, and ethics, which is more deep-brain connected. And recent studies have shown that short term decisions are made more by the emotional than the rational brain areas. So when speaking or presenting, the payoff connections will be activated first and any excuse to use them will overcome the rational ethics of being completely honest. Not that experts would lie, but exaggerating is easy to rationalize when it is based on truth.

But perhaps for long term discussion with long term payoff (writing journal papers) we are better at being rational and can be more honest. That is a study that still needs to be done.

human factors of sleep

The following was just reported in the Harvard Business Review. The authors discussed the results in terms of how the workforce is hurting productivity, but it applies pretty generally. All of these findings can be explained easily with basic cognitive science as global fatigue of activation, particularly inhibitory connections. But we often don't account for them in design, either of products that people use when they are sleepy or work systems where employees may be sleepy.

• Stay awake longer than 18 consecutive hours and your reaction speed, memory, attention-span and decision-making all start to suffer.
• Five or six hours of sleep a night for several days in a row has a cumulative effect that magnifies these negatives.
• Throughout the waking day, humans build up a stronger and stronger drive for sleep.
• Most people can't get to sleep without some wind-down time, even if they are very tired.
• There is a transitional phase between when you wake up and the time your brain becomes fully functioning. This is why making key decisions at the crack of dawn is never a good idea.
• A person who is sleep deprived has no idea how functionally impaired he or she actually is.

Source: "Sleep Deficit: The Performance Killer," Harvard Business Review, October, 2006, Vol. 84, No. 10. Pages 53-59.

netflix movie recommendations

Netflix is offerring $1 million to anyone who can improve their system for making recommendations by 10% or more. Why is this human factors? Because Netflix's business model is based on getting people to rent many movies. If the customer already knows what movies they want, this is easy. But what happens when he/she runs out of ideas? If they stop renting, Netflix loses. So they need a way to get people to rent movies they didn't know about. Random recommendations would never work, so they need some way to predict their customers' schemas for "good movie." But everyone has a different set of attributes for this. So they need to be able to model their customers' individual schemas of "good movie" based on the limited information they collect about each one, which includes:
1. some basic demographics they collect during registration
2. a list of the movies they have renting so far and when (more recent is more relevant)
3. a list of movies they have browsed but not rented
4. lists of favorite movies or wish lists
5. ratings of movies they have seen in the past

Then they develop algorithms that can include:

1. Movies that people who liked the same movies also liked
2. Movies that people with similar demographics also liked
3. Movies that share attributes with movies that this customer liked.

If you are interested in trying to win the $1 million, let me know. I would be happy to share with you the research I have done in this area.