Tag Archives: research

Gender and Ohloh.net

While I was at Wikimania last week I was talking to a sociologist who is researching open source contributions. Turned out he’d never heard of Ohloh.net so I was glad to be able to introduce him to it. Ohloh, if you’re not familiar, is a site that reports on contributions to a wide range of open source projects over time, by scraping information from version control repositories. It has over 300k projects listed and almost 400k contributors.

Yesterday, when looking at Ohloh, I wondered whether we could guess anything about the gender of contributors from their user profiles there. So I set up a little experiment. Using the Ohloh API, I extracted a bunch of account data, then grabbed a small sample (100 accounts) to mess around with. (I didn’t worry too much about real randomness at this point, as it was just a proof of concept.)

Next I created a Mechanical Turk job where I asked participants to look at Ohloh profile pages and see if they could figure out the gender of the user based on username, avatar, or any other means. I got three people to look at each profile, paying 5c each, so the cost to me was 100 * 15c = $15, plus Amazon’s fee brought it to $16.50.

The results came back in about an hour. I downloaded them and ran them through a quick little Perl script. In any case where at least two of the Mech Turk workers had agreed on “Male” or “Female”, I counted the user as that gender. If the workers couldn’t agree or couldn’t tell, I counted the user as “unknown”.

My results for the test batch of 100 users:

6 female
23 male
71 unknown

Turns out it’s hard to tell gender from Ohloh profiles! Some of them are truly impossible — usernames that are just initials, for instance, and profile pages not filled out at all. And sometimes my MT workers just seemed to have odd opinions, or didn’t know much about names. For example, they all marked someone named Didier Durand as “?” although that is a common French masculine name. Similarly, someone named Pavel Shiryaev also came through as unknown, with two “?” and one “M” though Pavel is the Russian version of the masculine name “Paul”. dianelamb320 got a vote each way for “M”, “F”, and “?”, also resulting in “unknown”, though I would have guessed female. On the other hand, svpavani came through as female (two “F”, one “?”) and I can’t for the life of me figure out why, as there is nothing on the profile page to indicate it.

So… with 71% unknown, I don’t really feel this was successful enough to extend to a wider sample, given that it costs real money to do so. But I do think it was interesting that in the small and not-particularly-random sample I used, 5% were clearly feminine usernames (that is, 6% minus “svpavani”). This is considerably higher than the 1.5% of female contributors usually cited from the FLOSSPOLS survey.

What do you think? Would it be worth trying again with a larger sample? Do you have any ideas for how to get fewer “unknown” responses without compromising the data? Any other ideas on how we could mine Ohloh’s account information to learn things about gender?

Who wants to play Evolutionary Neuro Cognitive Research FAIL?

Let’s all imagine that we’re cognitive neuroscientists and we want to do some “research” about fanfic (why fanfic? nobody knows!) and see if we can get a bunch of womengurlz to support our pet theories about “the unified fabric of human desire” (whatever that is — ilithiana says plaid). Because you can totally tell stuff about brain function from hacked-together surveys on Appspot.

What will we put on our survey? Here are my questions.

1. What sex are you?
a) Man. 100% manly man. GRRR.  
b) Female. *teehee*  
c) Confused.

2. Which statement do you agree with? Choose one:
a) I love cock!
b) All men are heterosexual.
c) One day my prince will come, and he will be Edward Cullen.

3. Which best represents your fanfic reading habits?
a) I fulfil my personal fantasies by inserting myself into fictional scenarios.
b) Because of my sexual inexperience, I read fanfic as research about boys.
c) I read fanfic because I am into depraved kinks like homosexuality and bandom.

Jonquil (who, incidentally, is kicking bottoms and taking names on this one — check recent entries on her journal) suggested via IM:

4. Will you please tell me about your sexual practices? With pictures?

If you need inspiration, check out this transcription of the 70-question survey. Remember, nothing you suggest will ever be reviewed by an IRB, so you can ask anything.

See also: Ten steps to a perfect fanstorm at Hoyden About Town, unfunnybusiness roundup, linkspam roundup on DW, high-larious Ogi/Sai badfic slash (NSFW).

Photo credit: innocentsmith @ dreamwidth

Credit: innocentsmith @ dreamwidth

In conclusion: fandom, I love you. You are smart and funny and don’t take shit from anyone — especially not cave-dwelling neanderthals posing as scientists.

Epistemology and impostor syndrome

Successful women often suffer from impostor syndrome. As wiredferret succinctly explains,

Imposter Syndrome is the pervasive feeling that whatever success or acclaim you might have, it’s all a cosmic accident, and other people really are much smarter and more successful than you.

Armchair psych follows, in which I massively mix descriptive and prescriptive:

The actor-observer bias and the related self-serving bias often cause a person to attribute her actions and their outcomes to certain kinds of causes, and attribute others’ actions, successes and failures to other kinds of causes. In other words, these bias lead me to believe that I caused my own successes and external events caused my failures, but others’ successes are due to luck and their failures are their own fault. Depressives’ biases run the other way; a clinically depressed person often believes that any good thing that happens to her is luck, she causes all her own failures, and her peers and role models and enemies get their deserved successes through their virtues. (Elliot Aronson’s book The Social Animal, chapter 4, “Social Cognition”)

It strikes me that Impostor Syndrome preys on the same epistemological problem as the biases I listed above. How do you know whether you belong, whether you deserve your success, whether your achievements even count as success?

Surprise!  You deserve to think of yourself as successful!

Surprise! You deserve to think of yourself as successful!

How would I know if another person, female or male, were succeeding or failing in my position? Would I judge them by number of emails sent per day, quality of relationships, apartment cleanliness, salary, credentials, orgasms per week, number of FLOSS commits?

Well, I could try to make a yardstick.  Consensus reality has an array of subjective and objective criteria for “is this person a success?” Proxies include money, influence, fame, respect from one’s community, and pride. I can use that data to try to fight the automatic negative thoughts. My bosses and colleagues praise my work unbidden. I’ve written articles I’m proud of. I can make strangers laugh at my jokes. I know so much about technology that my friends and acquaintances consistently ask me for tech advice. At one job I was earning more yearly than my dad ever did. I aimed to do foo and I did it.

“Does this person belong?” Belonging seems trickier, slippery and social. What’s the baseline? What’s a good metric for “does this group really accept and like me”? I can come up with plenty of falsifiable propositions to check whether they act as though they like me, perhaps even whether they are sending costly and hard-to-fake signals that they like me, but I can’t check their internal states.

And besides, it takes a lot of discipline and consciousness to address Impostor Syndrome with data. As long as I’m concentrating, I can believe I’m competent. But the data can be pretty handwavy.  And unless I use that data to change my permanent beliefs, sooner or later I’m subconsciously moving the goalposts on myself.  So at some point I have to just start acting as if I believe I’m good enough, stop believing — without proof! — that I’m a fraud, and allow my identity and beliefs to be fluid enough to catch up.

And we all have nonfalsifiable beliefs that undergird our behaviors. We all make assumptions to get through the day. Maybe you believe that men and women should have equal opportunities in the workplace, or that sunrises are beautiful, or that all humans should behave compassionately, or that God does, or does not, exist. Too many women refuse to add “I am a success” and “I deserve to be here” to their list of beliefs. If your excuse is that you can’t believe it because it’s not objectively provable, well, neither is “I am a failure” or “I don’t deserve this awesomeness” — let’s do some social construction to fit our blueprints for once.

What would I be like, if I were successful and deserved it? Well, I’ll try to act like that, then.

Other useful resources on Impostor Syndrome include Valerie Young’s Overcoming the Impostor Syndrome blog and Anna Fels’s great book Necessary Dreams: Ambition in Women’s Changing Lives.

Open Source research ideas

If you had a research team at your disposal to study women in open source, what would you set them doing?

My biggest wishlist item would be to look at retention of women in open source. How long do we stay on individual projects and in the community as a whole, and what makes us leave?

Another one would be to study how financial support for open source work is distributed. Who gets paid to work on open source, attend conferences, has hardware donated, etc? Is there a gender gap? (Might want to study within companies that use open source, but are not explicitly open source companies, for this one.)

Related: Do people mostly work on open source during work hours or leisure time? How do disparities in available leisure time affect open source contributions? (Would love to see this visualised as coloured timelines.)

What Women-In-Open-Source studies would your team of researchers do?

Standing out in the crowd… in PHP

Elizabeth Naramore has knocked it out of the park with Gender in IT, OSS, and PHP, and how it affects us *all*. It’s honestly the best article on women in open source that I’ve read in the last five years.

Not only does she lay out the situation incredibly clearly, but she backs it up with references to research that supports her points. For instance, here’s what she says on the “standing out” issue:

I did a bit of research and found that it’s not just women who feel uncomfortable in the gender minority, but men do also. Studies have shown that “being in the numerical minority in a mixed-sex team is not a favorable experience for either women or men (Powell & Graves, 2002; LBS, 2007). Men report feeling uncomfortable, somewhat alienated and highly aware of their minority status when faced with being in a female majority group. (Spelman, et al, 1986). As well, numerous studies have shown that when anyone is in the minority in a group, they are more apt to place more emphasis on and be more self-conscious of the quality or trait that makes them distinctive from the rest of the group (O’Leary, et al, 1985; Kirkham, 1985). Bottom line is, it’s human nature to feel out of place when you’re in the minority regardless of gender, and if that means you’re the only female around, you’re going to be even more acutely aware of it. So ladies, do not adjust your set. There is nothing wrong with you.

She also goes into the research behind retention rates, the effects of the free time gender gap, and how diversity affects group effectiveness, creativity, and innovation.

I wish I’d had this article a month ago. I’ll be leaning on it heavily going forward, and I recommend that everyone who’s interested in this stuff bookmarks for quick reference. It’s going to become a standard.