Culture Amp strategist, Steven Huang talks about the importance of inclusion and the survey we developed around it in partnership with Paradigm.
More about our Inclusion Survey here.
(Edited for grammar and syntax)
Steven Huang, strategist at Culture Amp, talks about how to use surveys to promote inclusiveness and drive growth.
Bronwen Clune: Welcome to the Culture First podcast. I’m Bronwen Clune, director of PR and Comms and I’m here with David Ostberg, who is our director of insights.
Bronwen Clune: With us today, we’ve got Steven Huang who is one of the strategists on the Insights team. Steven is here to talk to us about something that I’m particularly passionate about and have been part of, and that’s our inclusion survey. I’d like to think all of us at Culture Amp are passionate about it. Welcome to the Culture First podcast, Steven.
Steven Huang: Thanks, Bronwen
Why study inclusion?
David Ostberg: Just to set the context, can you give some background about how this inclusion survey came into being, what drove it, just where did it come from and then maybe a bit about what it covers?
Steven Huang: That’s a great question. The Inclusion survey started nearly a year ago. We looked at what are the best cultures out there. What are the most culture first companies? What are some of the things that they do? Those are really the things that we should be targeting. What we notice is that the best culture first companies are ones that are really diverse and also very inclusive. We realized that we needed to make a survey to help measure and manage inclusivity in companies.
Bronwen Clune: I guess also the other thing for me in particular was I didn’t know much about engagement before I started working at Culture Amp. The more I looked at what we’re doing I felt like we had a really powerful tool here to uncover some of the things around diversity and inclusion that were beyond demographics because of the way our survey works. You two can probably talk to this better around things like looking at what drives people, what their motivations are, what their experience was of working at a company.
If we actually broke that down by different demographics, I started to wonder if not only could we start to identify different demographic experiences of the company, but we could also orient people to where they could be looking for solutions. That, for me, was particularly compelling. Other than just going, “Hey, 3% of people at this company are American or 30% are women.” We could actually say, “The women in your company don’t feel that there’s two-way communication at the same level as men.” Then there’s something you can specifically address. That was the early ambition around doing this.
The danger of the single metric
Steven Huang: I think that was a driving ambition. I think when we looked at a lot of the work that’s going on now - not to discredit any work that’s been done before - but it was really focused on representation. When you focus on just one metric - David, you and I both know that that’s really dangerous. If you just focus on time to hire for example, your quality starts to go down as you start to just focus on time. In the same way when you just focus on representation, you don’t actually think about the experiences of these people once they actually do come to your organization.
Bronwen Clune: Steve, you actually had some experience in trying to do something around this at Facebook. That was also an interesting experience because you tried to do something and felt you hadn’t had much success. You want to talk about that a little bit because I think that creates a great context for what landed as our approach, the way we designed the survey.
Steven Huang: I would say that my previous roles at Square and Facebook doing diversity metrics you are focused on this one metric. I’m the analyst. I’m in charge of putting together the numbers and building models to say, “How can we make our numbers look best? If we include designers and tech, does that make our numbers look better? What about engineering support? Does that make our numbers look better and it’s like: Can I optimize this?” I’m optimizing for this one metric and I got to a point where I was also running the engagement survey and we see differences between some groups.
I’m like, “How is that happening?” You think about what is the work that you’ve been doing trying to make the picture look rosy but not actually tackling the root cause of why these people might be having different experiences. I’ve always held that in the back of my head, “How can we actually solve this problem?” It’s not really until I got to Culture Amp and Bronwen came up the idea of the diversity inclusion survey that I thought, “This is something that could actually work.”
Diversity does not guarantee inclusion
Bronwen Clune: I should say in the beginning we actually thought of it as a diversity survey. That’s been our own journey. We’ve learned so much about the space and we dived into it. A lot of these conversations were being had in the space - we’re not talking about diversity. We’re talking about inclusion. It was interesting timing too, because so much of the focus to date around improving this has been on improving the pipeline. The way I came to see that was the companies went, “Oh well it turns out there’s not enough female engineers to hire. That’s why our stats look like they do.” That was actually a way for them to deflect responsibility on improving their cultures now.
David Ostberg: Pipeline argument, yeah.
Steven Huang: That was widely panned, wasn’t it?
David Ostberg: Yeah absolutely.
Steven Huang: It’s not valid.
Bronwen Clune: That’s still a fairly recent thing that came out as recently as a few weeks ago and said, “Oh, this is why our results look like this.” As much as it is panned by people who are working in the space, the kind of ultimate message hasn’t really permeated through everything. I think that’s certainly something that until we start improving the company cultures that exist now we’re not ever going to solve the issue.
The other shift I’ve seen as well is the very big shift away from this being a gender discussion to one that’s much more inclusive. I think that’s exciting. When it came to simple things like naming the survey, we realized if we called it a diversity survey, probably every white male in the company wouldn’t take it because they’d go: “That’s not me,” where inclusion is something everyone can relate to and get on board with. That was some of the thinking about it.
One thing I have been interested in is that inclusion feels like something that’s not concrete. I’ve loved our approach to creating a way that we can measure inclusion. I think that’s a compelling part of the story and part of the journey.
Digging into data to identify issues
Steven Huang: When we approached it, we first put together the diversity symposium and it’s like, put the best minds together, what can we come up with? I think that was helpful to get started, to get the momentum, but I think we made huge gains and we brought in Paradigm as a partner and also the brilliant minds of Project Include. They brought this academic rigor and research that took us to the next level. Yes, inclusion is this broad concept of making sure everyone feels included. Y-Vonne Hutchinson really elevated the component of intersectionality. We’re looking at more than just gender.
We’re looking at people of color and sexual orientation, parental status, family status, caregiver status, native English speaking proficiency rates. All those things in combination broadened the concept of inclusion. Then Paradigm brought so much research behind some of the questions. Some of the questions were brought in because we know that there are differences. We’ve already surveyed over a million people. We know there are differences.
Bronwen Clune: I think it’s close to 2 million.
A growth mindset
Steven Huang: Close to 2 million people and we know there’s differences between men and women in some aspects. We’re like, “We know we can ask these things because if women are having a different experience, it’s likely that others are, too.” Then like concepts of the growth mindset, which I actually wasn’t familiar with until about 6 months ago, and there’s tons of research from Carol Dweck and others about how this growth mindset has a large part to play in feeling inclusive.
Bronwen Clune: Explain what growth mindset is.
Steven Huang: I hope I don’t mess this up. The growth mindset is that your company should believe that you can change how intelligent you are. If your company believes that you can grow, you’re more likely to be engaged and be a part of the learning and feeling included. Conversely, if you feel that your company doesn’t believe in your capability or potential, you tend to withdraw and then they believe in you less and then you withdraw some more and that can create this cycle.
Why bad times may matter more than good
Bronwen Clune: I’m interested to explore how improving inclusion can lead to better engagement, which is the core of what we do. My simple explanation is that simply by improving inclusion, you’re always ultimately going to improve your engagement score. You’re getting the best out of the people. You’re getting bigger buy-in and that sort of thing. Is there any research, or perhaps a more nuanced explanation of how the two go together?
Steven Huang: We’re building it. I would say I’m a very cynical, skeptical person. If I don’t see the data, I’m like, “You’re just some other person telling me that inclusion matters.” One thing that we’re doing in our inclusion survey is we asked the five standard engagement questions about what you say about your company, whether you want to save your company, if you want to strive at your company. Our research so far - I mean we’ve only had I think a little over 10 participants right now because we’ve just launched it - but our preliminary results are showing that some of these things are tied to engagement, productivity and intent to stay. There’s belonging questions especially the question around belonging uncertainty.
Belonging uncertainty is measuring whether, even when something bad happens - you get really, really negative feedback, or one of your products fails or something, you have this really bad interaction. Even when something negative happens, do you still feel that you belong at your organization? If you’re in the majority group, you rarely question whether or not you belong. These things just happen, and it doesn’t make you pull away. If you’re in a minority group, the impact might be outside in its effect on you. You might withdraw: ‘I don’t feel included or this happened, then I tend to withdraw.’ Again it creates a cycle of withdrawing. Not feeling included, withdrawing some more.
Bronwen Clune: The impact is greater, I guess.
Steven Huang: Yeah, absolutely. It can be.
The courage to be vulnerable
David Ostberg: You may be a very resilient person in your normal life, but if you do not feel like you’re part of the team, that you really fit in and you experience some traumatic event or some negative event then, yes you may find that it’s just much easier to say, “This is not where I’m supposed to be,” as opposed to, “This is just an episode and it doesn’t reflect on my association and my belongingness.”
Steven Huang: I’m really learning about this question with Paradigm. I was like, “This question is weird to me. Can you explain this?” Then you read on it. Paradigm did different approaches. It’s like in an academic setting, in an applied setting, going through the data of how this belonging uncertainty can impact your engagement and feeling included.
David Ostberg: I think it’s also interesting to think about how that relates to social support because we know people’s ability to persist through traumatic or negative life events has so much to do with their access to support from others, from friends, from family who believe in them, and it’s really corollary to this that if you don’t feel like you belong, you’re less likely to communicate or to share your concerns or your pain with coworkers. It’s much easier to withdraw.
Bronwen Clune One of our values is the courage to be vulnerable, but I guess for people creating safe spaces to talk about these things or the impact on them, we’ve often talked about this being integral to try to make progress in this area. How would you go about measuring people with that ability?
Belonging drives engagement
Steven Huang: There’s a question about being your authentic self, which we changed to true self. Authentic self maybe doesn’t translate. We’re testing which of these things are most tied to engagement. So far, this is a preliminary result, but a sense of belonging, feeling respected and feeling belonging even in the times of uncertainty, those three aspects right now are showing really high correlation to engagement.
Pride, recommendation, motivation and intent to stay. We’ll see if this stays true as we get more data, but it’s actually not surprising to me because of this conversation we’re having right now. The more that you feel that you are yourself, the more you can communicate and it makes you feel more included. It makes you want to stay and do better.
Protecting employees’ identities
Bronwen Clune: There are also a couple of differences between the way we run this survey to how we run our other surveys. With the engagement survey, it’s an attributed survey which is confidential but not anonymous. Culture Amp knows who you are, but the company doesn’t. We aggregate those results based on location, tenure, geography and other demographics. With this survey we decided to make it an unattributed survey. Do you just want to talk to us or explain what that is and why we landed in that space?
Steven Huang: One of the first big topics we had to tackle is anonymity. For most surveys, we really prefer the attributed, the confidential survey. Culture Amp as a vendor knows who your employees are, so we don’t have to ask you what department you’re in or how old you are, your tenure, those kinds of things.
We preload a lot of that in. It’s seamless. In this survey in order to feel safe, you might be the only gay person of color in your organization as an example. You might not feel comfortable even though there are minimum reporting sizes built in. You want to be truly anonymous. You don’t want to be tied to your employee file.
Bronwen Clune: I also think conversely employers don’t want that information tied to their employees either because that could become an issue.
Steven Huang: I think in larger companies that’s a main concern, too. We took extra precautions. This survey is available in a separate account. If you don’t want it even close to your employee file, you can control the administrators more tightly that way, too. The anonymous survey does have its drawbacks though, and that’s why some companies decide to run it attributed and that is an option if your legal counsel has the risk appetite for that. The unattributed survey, it’s hard to measure participation rates because we actually can’t track who’s taken the survey. It’s completely anonymous. You could theoretically somehow take the survey twice. On an engagement survey if you take the survey once, check. You’re not allowed to take it again. With a truly anonymous survey, your responses go through and we don’t know who you are.
Bronwen Clune: One of the reasons we also had to do that is we had to have a way to collect the demographics. We had to ask very personal questions and some of the questions are pretty confronting. We ask around sexuality.
Steven Huang: It’s not for the faint of heart is what I tell people.
Bronwen Clune: I guess at the same time, the more information we have, the better solutions we can come up with.
Surveys as a benchmark against competitors
Steven Huang: Yeah, all of the questions are voluntary. If somebody doesn’t want to answer any one of the questions or several, they don’t have to. We also had to get people really inclusive response options to choose from to ensure that their answers are really anonymous. One person from our launch event said, “Even if this is truly anonymous I still have trouble selecting some of these things because I feel like it’s going to identify myself.” At that point I said, “It’s truly anonymous, and if you can’t answer an anonymous survey or have the conversation in person, how do you expect your leaders to have the awareness?” This is really about bringing awareness to leadership and people that are reviewing these results and say, “I need to do something about this.”
Bronwen Clune: I suspect a lot of what we find is not going to be completely new to people, but I think what we can do is validate a lot of assumptions. I think once companies have the data in front of them, it’s a bit of a map to try and address the issues at hand. I think very rarely we find something completely new or shocking or that people haven’t thought about.
David Ostberg: Like Steven mentioned this earlier, I think this is a really fantastic opportunity, because there are so many companies participating and so many individuals participating, to really get a lay of the land and create a benchmark and create an understanding of what exactly is the makeup out there and what is going on, to really allow companies to understand where they stand in relation to other organizations.
Learn, Act, Repeat
Steven Huang: One company they have agreed that I can use their name but I won’t in the podcast. There are about 1,500 employees. They didn’t even know that they had 11 transgender employees. Self-identified of course. A lot of the companies that have participated early on - they’re doing a pretty good job. They’re staying ahead of the curve. They’re finding out about the survey. They’re running it to stay ahead. I would say some groups you would expect to be having a difficult time, they’re scoring at or above the majority population because these organizations have already done so much to make them feel included.
Bronwen Clune: I also think one of the other important points to make is it’s not a matter of doing the survey once, coming up with the solution, thinking you’ve ticked the box and moving on. We rarely work on the basis of measure. What is it?
Steven Huang: Learn, Act, Repeat.
Bronwen Clune: Learn, Act, Repeat. Continual learning on this and continually testing around what is working and what isn’t for companies. We’re suggesting companies do this once a year because anything bigger than that, it’s a big ask of people to hand over that personal information. I think the more companies that do this over a longer period of time, we’re going to learn more and more about ways we can make a difference in the space, which really excites me. For me I think it’s been one of the most exciting things I’ve worked on in this area.
Steven Huang: I love working on the survey. Really excites me that we’ve made it free for customers and even for non-customers in the tech space for a limited time as we build this benchmark. It’s a survey that we want everyone to take.
Inclusion is a process
Steven Huang: We want these questions to be used and these demographics to be used because the EEOC guidelines are non-inclusive and the focus on representation is not really holistic. We want people to use these questions and think of these as the new standard for measuring inclusivity.
David Ostberg: Back to your point Bronwen, we’ve been encouraging organizations to look at this not as an inclusion project but as a process because it’s not a one-time thing.
Bronwen Clune: I think it is exciting the more companies we have taking it, the stronger it’s going to be. We want as many tech companies out there who are willing to do this to get on board. We’ll walk you through everything, how we’ve developed everything. Steven, you’re running fortnightly or weekly sessions at the moment for interested companies. What’s the best way for them to get in touch with you to do that? Is it to email firstname.lastname@example.org?
David Ostberg: Yup.
Steven Huang: If you just email email@example.com either me or Bronwen or Julie will reach out and schedule a time with you. We’re doing weekly onboarding and informational sessions. Some people just come to learn more so they have enough ammo to pitch it. Others come and are ready to launch the next day. We can set them up and get them running.
Bronwen Clune: The other point to make is the focus at the moment is on tech because that’s the area we have a lot of clients in, but definitely inclusion is not an issue in tech industries alone. There are a lot of industries with similar problems. Anyone who is our customer outside of those industries can always use it and anyone else interested still come and chat us and we’ll see what we can do. It’s something we’re all pretty passionate about here at Culture Amp.
David Ostberg: Even if you’re not a tech company, but you want to learn more about inclusion, come. Use our questions. They’re developed in an open-source manner.