28.10.2011

Staff Engagement Surveys and Rocket Science

For the last few months we have been developing and using a new application for conducting staff engagement surveys – it’s called Murmur. Murmur allows small numbers of people to be polled or surveyed every week so leaders and employees can be kept in the loop all the time - using a realtime dashboard and driver-analytics to guide where to focus action. We liked the name Murmur because it captured what many CEOs and people leaders are often dealing with in terms of feedback from their employees:

Mur – mur (noun): A soft, indistinct sound made by a person or group of people speaking quietly or at a distance.

Staff engagement surveys are not rocket science - although many consultants would certainly like you to think it is similar and that they should be paid accordingly [1] - all we are really doing is tapping employees on the shoulder and asking them what they’re murmuring about. Once we know what's on their mind it’s much more about the dialogue, leadership and action that follows than designing the world’s most hyper-validated survey – if you’re aiming to improve the workplace that is.

I had a chance to learn the dangers of treating engagement surveys like rocket science when accompanying a senior consultant to my very first staff survey results presentation (I wont say when that was but I had more hair then for sure). As the analyst for the project I was asked to do a comprehensive analysis and uncover the structure of the survey for the organisation. In fact, the organisation was a scientific institution and the survey was comprised of around 160 items, so this provided fertile ground for practising survey analytics [2].

As it turned out, most of the results session was spent on scholarly discussions arising from the segmentation of results from the previous survey – the organisation had also moved to surveying once every two years because it was taking people that long to understand their data properly. And after two years of analysing the data they thought it was unwise to use it to guide any actions because the results were out of date.

I learnt the lesson well. Returning to the office I checked the organisation’s survey results over the last 10 years. The survey items had been well designed and the analyses each year were superb, but looking at the trend in their engagement scores the story was clear - staff engagement was never particularly high and it hadn’t moved at all in a decade.

A well designed, validated and benchmarked engagement survey that takes 30 minutes to complete and two years to analyse may please some academics and organisational consultants, but it is not going to improve your organisation.

A good survey that takes 5-10 minutes, fast results and analysis, a focus on a few actions and ongoing and open results tracking - now that is a different matter. More on that to come.

[1] Rocket scientists are generally qualified engineers who work on rocket systems. The only rocket scientist I have met personally was working as a management consultant; he told me this was because there are more poorly organised organisations than rockets that need building at the present time, and, it pays better (management consulting that is).

[2] For this survey we had so much data to make sense of we were using factor analysis to group the items, followed by regression to identify factors most related to an engagement index and then some more regressions to identify the specific items from the factors most related to the engagement index. Finally, because some factors had so many items in them we were not really sure whether they were the best predictors because they represented the most important issue for engagement or whether the factor was just more reliable due to the large number of items.

About Culture Amp

We've brought together experts in technology, data science and organizational psychology to ensure you can get the clarity you need, when you need it. Measuring employee engagement, experience and 360° feedback is simple using Culture Amp.

Learn more