There’s a myth about self-selection bias. This post will first explain “self-selection bias” in surveys. I’ll illustrate with an example. Then we’ll explore the myth.

Imagine you want to know what fraction of kids prefer blue vs red marbles.[1]

Imagine you bring two bags of 1000 red and 1000 blue marbles to a shopping mall, and two glass bowls, and as kids leave the mall[2], you say, “Would you like a free marble? Please take either a red one,” and you wiggle the bowl with red marbles, “or a blue one?” And you wiggle the blue one.  Imagine that three quarters of kids you approach take a marble. Imagine that after a very long day, all the blue ones are gone and 500 red ones remain. You conclude 2/3 of kids prefer blue.

Self-selection bias warns us that the kids who took marbles did so by choice. Instead of choosing random kids, we let them choose who took a marble. The kids we sampled selected themselves, hence the term “self-select.”

What if the kids who didn’t take a marble were afraid? And what if half the kids who like red marbles are kids who are afraid? That is, if all the kids who were afraid preferred red marbles? If this was true, by letting the kids determine our sample, we biased the results in favor of blue, and the real answer is that 50% of kids prefer blue! The self-selection biased our results. It slanted them to make blue seem preferred 2-to-1, when it’s really 1-1.

But there’s also a myth about self-selection bias. The myth is that if you let people self-select, you won’t get accurate results. I’ll illustrate how this works in the next article.

###

[1] Why would you want to know the ratio of red to blue? Because you always feel guilty giving kids candy on Halloween. And they like getting little toys even more than they like candy. So this year you plan to buy red and blue marbles and want to know what proportions to buy.
[2] It’s important to give the marbles when kids leave the mall. If you give kids marbles as they enter the mall, some kid is likely to drop one and someone could step on it and take a hard fall and sue you. (It could happen in the parking lot, too…)
[3] What you really like is giving kids stuff, Halloween or not. The pretense of the survey gave you an excuse for giving marbles to kids at the mall, too!