There are, apparently, some good bloggers at Psychology Today. But there’s also a heck of a lot of low-hanging fruit for me to pick on, so I can’t resist picking on the bloggers at A Billion Wicked Thoughts. (Fandom folk will know who I’m talking about if I say “SurveyFail.”) Let’s just say that these guys are making broad claims about human sexuality and desire based on Internet searches and a survey of Internet users.
::waits for anyone with a science or social science background to finish cursing:: Yup, they’re generalizing about human desire based on the terms people googled and a really horribly written survey of women who read fanfic. (Actual questions: “58. Which do you think is more influential on shaping sexual identity? Nature. Nurture.” “64. What is most important in a relationship? Emotional sincerity. Passion. Mutual trust. Shared interests. Commitment.”)
The bloggers/”researchers” in question are Ogi Ogas, Ph.D., and Sai Gaddam, Ph.D., who have degrees in computational neuroscience, not, y’know, anything that taught them how to do ethnography. They have written posts with titles like “Why feminism is the anti-Viagra” and “What do shemale porn and slash fiction have in common?”
Note: From what I hear, numerous people in and out of the transgender community have asked them not to use the term “shemale” but they say they’re continuing to use it because it’s a standard genre term in porn. I mentioned the bit where they don’t know jack about ethnography, right?
Look, I don’t know anything about computational neuroscience, okay? Not a damn thing. Which would be why I’ve never attempted to publish a book about it. Hell, I doubt you’ll ever see me even attempt to blog about it, because I know what I don’t know!
But, as many others before me have noted, Ogas and Gaddam appear to be excellent and annoying examples of the Dunning-Kruger effect. They’re so far out of their depth, they haven’t seen shore in years. And the truly amazing thing is how clueless they are about their incompetence.
I think my absolute favorite part of their adorable cluelessness is their analysis of web searches. When asked how they could know what men were searching for and what women were searching for, they replied that “sometimes the gender of an AOL sex searcher is suggested from their search history” and then they list two sets of searches, containing stereotypically male (trucks and baseball cards) and female things (gossip, menstrual cycles, corsets).
Does anyone else see the problem here?
This is known as “begging the question.” When asked how they know what men and women are searching for, they replied (essentially) “by checking to see if they’re searching for male or female things.” Personally, circular logic makes me dizzy. ::snort::
I will grant that using names on credit cards used at porn sites is going to get you a fairly good idea of whether men or women are using that particular site, but I absolutely don’t think you can extrapolate from consumers at porn sites to all men or all women.
And when it comes to their survey, according to the report of the American Psychological Association’s Advisory Group on Conducting Research on the Internet
There is currently no sampling frame that provides an approximate random sample of Internet users, unlike the case of random digit dialing of telephone numbers, which provides an approximate sample of the U.S. population. The problem of representativeness is compounded because many online surveys and experiments rely on opportunity samples of volunteers. As a result, it is not clear exactly how to go about the task of appropriate generalization. For psychologists, who often value internal validity over generalizability, the large and diverse samples online are preferable to the college sophomores on whom much psychological theory rests. But for sociologists, political scientists, and others who attempt to track the pulse of the nation or to generalize to broader groups beyond the participants, these self-selected samples are problematic. [Read the whole report here, with references.]
The money quote is in the last sentence: For people who “generalize to broader groups beyond the participants, these self-selected samples are problematic.” Yeah, they are.
I’m more than ready to argue with their conclusions even if they were just talking about Internet users who watch porn, or Internet users who read erotic fanfic, but when they try to take their questionable conclusions and expand them to include all men and women…fuggedaboutit.
And the APA report pretty much assumes that if you’re doing this research, you’re working through an Institutional Review Board, which is an organizational component created to protect participants, whether they’re rats or people. Any research that goes through a university or government agency must get the IRB’s approval. (My father-in-law has served on several IRBs at the National Institutes of Health for many years, so I have passing familiarity with how they work.)
Ogas and Saddam weren’t working through an institution (although they made it look like they were), so they bypassed the whole thing. Which is too bad, because any IRB with half a brain would have looked at their online questionnaire and laughed them out of the room. Honest to Darwin, I don’t know what my father-in-law would have said if this came over his desk, but the conversation with him that would have followed would have been amusing, I’m sure.
If you’d like to read some wonderful explanations of precisely what was wrong with their methodology, smarter people than me have written them. You can find them over at Fanlore’s page on SurveyFail. Scroll down to the bottom for links to intelligent and angry people.
And when they were done making assumptions, begging the question, and asking intrusive and upsetting questions of thousands of members of fandom…they published a book. Which I wouldn’t own if you paid me to put it on my shelf.
Sexuality and desire are complicated and intertwined with biology and culture and history. Reducing them to neurons and “granny sex” misses pretty much everything that’s interesting and important when it comes to sexuality and gender. Shame on them.
APA Advisory Group on Conducting Research on the Internet. 2002. Psychological research online: Opportunities and challenges. Washington, DC: APA.
Fanlore. N.D. SurveyFail. Wiki.
Matthew Philips. 2011. The neuroscience behind sexual desire: Authors of A Billion Wicked Thoughts answer your questions. Freakonomics blog.
Ogi Ogas and Sai Gaddam. A billion wicked thoughts. Blog.