Gender bias in my hiring history

I have hired a lot of people. I have hired more men than women. I have also hired a greater proportion of female applicants than male. Both of these are critical to understanding the bias in my field (software development) and what to do about it. Also, both are indirect results of something else.

Most of the industry optimizes hiring decisions for individual performance. They assume the best company performance comes from strong individual performance.

They are wrong.

Research on collective intelligence has shown that the best group performance comes from the group with the best emotional intelligence in the most individuals. Company results are composed of individual results, but are determined by emotional intelligence.

Aside on social justice

Before I go any further, I want to state that I am explicitly ignoring whether I am genderist. Instead I am looking at the systemic forces that determine which options I get to select from each time I get to make a choice, and which forces impact my decision among those options. Basically, I am assuming Deming was right when he said

95% of the behavior of any individual is due to the system in which he is found. The reverse is not true.

So if you are coming to this from the perspective of wanting to change things—such as the gender bias in the software industry—then the systems are the things to change. I’ll use my experiences to highlight the systems that I see in play and a simple change which will make your company’s system far more effective at building software…and have the side effect of closing the gender gap.

And yes, I do have personal gender biases. Which may or may not be the ones you or I suspect. But most of my behavior has to do with the systems I choose to allow or construct around me.

What determines performance

The key insight is understanding the impact of emotional intelligence on company performance.

Individual performance may be driven by things like intelligence, skill, experience, work ethic, creativity, and especially ability to learn, but group performance is determined by emotional intelligence, not individual performance. In the above research,

the performance of groups was not primarily due to the individual abilities of the group’s members. For instance, the average and maximum intelligence of individual group members did not significantly predict the performance of their groups overall.

The study explored collective intelligence, not group productivity. However, the researchers concluded that “a group’s collective intelligence accounted for about 40 percent of the variation in performance on this wide range of tasks.”

That collective intelligence stems from how well the group works together. For instance, groups whose members had higher levels of “social sensitivity” were more collectively intelligent. [Social sensitivity has to do with how well group members perceive each other’s emotions, according to one of the authors]

Therefore if you want to have the most effective individual, pick one who is good at learning. But if you want to have the most effective business, instead pick people with good social sensitivity.

I hire for emotional intelligence

In my embedded culture (US middle class and above), we train women to be more socially sensitive than men. Therefore, women tend, on average, to be better qualified for my positions. And so each female applicant has a higher probability of making it through my hiring process than each male applicant.

Another quote from the above article:

Only when analyzing the data did the co-authors suspect that the number of women in a group had significant predictive power. “We didn’t design this study to focus on the gender effect,” Malone says. “That was a surprise to us.” However, further analysis revealed that the effect seemed to be explained by the higher social sensitivity exhibited by females, on average. “So having group members with higher social sensitivity is better regardless of whether they are male or female,” Woolley explains.

I am not biasing my hiring towards women. I am trying to assess, as well as I can, the applicant’s  ability on the single most important trait for my team’s success: social sensitivity (a key part of emotional intelligence). It just turns out that most men are less capable at that trait than most women.

Others hire for individual performance

Most software people hire for:

• claimed accomplishments (stuff which you did with others but claim for yourself),
• years of experience with one specific technology, or
• “team fit”—being like the people we already have.

The first of these is more common in the male population, because it actively selects against social sensitivity. A person with greater social sensitivity is far more likely to see the result as the effort of the whole team (and to have been on teams where that is true), so have more difficulty claiming individual accomplishments.

The second selects for whoever has been around longest with the least curiosity. It selects for those who stay with one technology, even when another may be better for a given task. It selects for those who fear learning new technologies. It focuses on those who get really excited by a technology, not by team effectiveness. Again, this tends to select towards males.

The third selects towards whichever gender is predominant at the company. Since this is recursive, it tends to select towards whichever gender is predominant in the founders of the company. Female-founded businesses who select on team fit tend to hire more women, and male-founded businesses who select on team fit tend to hire more men. As more tech companies have been founded by men than women, this tends to select towards the male population.

Better qualified

I define “better qualified for a position” to mean the person most likely to be hired for that position.

The person most likely to make it through the hiring process is not necessarily the person who would result in optimal company results were they to be the one selected for the position. Job getting skills are not the same as job doing skills.

However, we are talking about hiring processes here. I want to explicitly separate what we choose to hire for and who is most likely to be hired on that basis. Then we can align what we hire for with what will give the best company result.

Given this definition, then:

• Men are, on average, better “qualified” for most positions in the software industry.
• Women are, on average, better “qualified” for most positions at any of my companies.

And that is why there are more men in the industry as a whole, and yet women are more likely to be hired at my company. The people most “qualified” get the jobs and become part of the industry / company / team—regardless of who would actually do a better job were they to get the role.

Embedded culture and options

However, even my teams are often majority male. That’s because of the culture I am in. Women leave the software industry because they don’t get hired for the best jobs at the majority of companies.

Unless I make special efforts, the resumes that I receive for a position are typically around 97-99% male. I get 0-2 female applicants. Even if women are more likely to be socially sensitive, the fact that they are sitting next to 50-100 male applicants means the most socially sensitive person in the pile has a high probability of being male.

Social sensitivity is distributed in both populations, probably in a bell curve (because everything is). Even if the peak of the female curve is at higher sensitivity than the male one, if I am only sampling twice from the female curve and a hundred times from the male one, the highest sample I find is likely to be male.

So I actively seek more options. I specifically seek out female applicants.

• I try to write position descriptions to appeal to the people I want (highlight working with a team to change the lives of our customers, not being an individual rockstar performer).
• I seek applicants outside the software field (it is a lot easier to teach programming than emotional intelligence, although I do train both).
• I don’t often use recruiters (who select by keywords and along industry norms).
• I use networking (which biases towards those with social sensitivity).

But still I get a lot more male options to choose from than female ones, and end up with teams that have more men on them than women. Overall in my career, I have hired a couple hundred people. Those people are about 60% men, 40% women. 90+% of them are high in social sensitivity.

Results

I have created several best-of-breed teams in my career. I consistently assemble and grow high-performance, cross-functional teams, regardless of the company culture around me. My teams advance the state of the art in software process, productivity, and product.

This is not accidental. I have about a 70% success rate at creating gelled, high-performance teams within 6 months. I have a 90+% success rate at creating learning organizations at the single-team level. The industry typical manager has created 1 or maybe 2 such teams in their entire career.

My teams and companies are simply better for business. We are more productive, more flexible, have shorter lead times, learn faster, and have less attachment to any one way of doing things than our competitors. We have a market advantage which we exploit to build more, better products at lower cost.

Conclusions

Core to this result is the philosophical difference on how to create an effective team. I believe software dev is a team sport. The most effective results come from strong teams that learn and change fast. These teams are best created by selecting for high emotional intelligence and wide heterogeneity on all other axes, then building a learning environment.

Social justice isn’t the reason you should hire more women. You should hire whoever will make your business more effective. And that is probably women.

Getting effective teams requires changing the criteria upon which you select candidates. When you do so you will find yourself competing fiercely with managers like me for the new top talent—which will mostly be women.

At least until we start training our men to also have high emotional intelligence.