Beware of Over-Simplification

Adam Pressman

Global Director of Client Partnership

“The ability to simplify means to eliminate the unnecessary so that the necessary may speak.”
Hans Hofmann (artist and teacher 1880 – 1966)

“Truth is something which can’t be told in a few words. Those who simplify the universe only reduce the expansion of its meaning.”
Anais Nin  (author 1903 – 1977)

Which of these above quotes is correct?  Is simplification a good or a bad thing when it comes to making business decisions?

No one can argue that business leaders rely on clarity to determine priorities and make decisions.   However, in this effort to provide clarity, it is easy to fall into the trap of over-simplifying.  Over-simplifying a business problem can lead to the wrong conclusions.  The wrong conclusions lead to ineffective actions.

When it comes to working with employee opinion data, this dilemma clearly applies.  The main objective of an employee survey process is to provide clarity for executives on which issues are most important and where to focus for the most impact.  It is important to provide simplicity where it is needed, but not at the expense of effective follow up.

The Instrument Itself
Every survey practitioner knows that in order to have useful data on the back-end, you need to start with a good, reliable survey instrument.   But how do you keep the survey instrument simple and clear for the respondent while ensuring a robust enough data set to provide clear direction?   There is an emerging trend around short, on-going pulse surveys to be able to see trend data throughout the year versus from a single point in time.  Are these short, pulse questionnaires giving us the right amount or the wrong kind of simplicity?  Here are some basic tips:

  1. Shorter is not always better – while 100 item questionnaires are clearly too cumbersome, fewer items often leads to data that is too general and broad to effectively action.   Remember, the end goal of your survey program is not to save your respondents an extra 5 to 10 minutes.  The goal is to get useful and actionable data.
  2. Group the concepts together –dimension structures are a very effective way of grouping concepts to better make sense of the data on the back end.  Working with data from 8 to 12 dimensions is very manageable.  You typically will need 4 to 6 items per dimension to have a reliable measure of a specific concept.   This gives us a simple way of organizing and presenting the data, but also provides enough information to explain the issue in more detail.
  3. Group the items together for the respondents – the best way to keep things clear from a respondent point of view is to group the items by referent.   A referent refers to the object or the concept you want people to think of when responding to a set of items (e.g., my job, my company, my co-workers, my boss, etc…).
  4. Pulse with a purpose! – While it sounds attractive to get on-going data at all times from a pulse survey effort, the data can often create more confusion.  Be clear about what you are pulsing and ensure it is tied to specific priority areas.  Asking the same broad and general questions on an on-going basis through a pulse survey will annoy your respondents and give little direction to your leaders in terms of how to take action.

The Analysis
The goal of your data analysis should be to provide clarity and direction.  So, simplicity will be important to keep executives on the same page as to the key priorities, but over-simplifying the data can be dangerous and lead to the wrong conclusions and actions as a result.  For example, if the data comes back from one of your “compensation” related items showing 45% favorability, does that mean that people are being paid unfairly or below the market?  To answer that accurately, we would need to understand the results from that item, how they compare with responses to other items related to fairness and equity, how the responses differ from the global norm on that item, and how the results compare within groups or segments of the population.  Here are some basic tips for analyzing survey data:

  1. View the data in the context of your business – zero in on issues that are most relevant to your business priorities and ensure to present the data back to executives in that fashion.
  2. Look for ways to combine the data to make sense of it – examine item relationships to understand how the dimensions and items interact.  Use statistical techniques such as correlation analysis, factor analysis, and reliability analysis.
  3. Explore demographic differences – examine the data to find meaningful differences across organizational groups using techniques such as analysis of variance, Chi-square, and CHAID procedures.
  4. Figure out where to focus for the biggest payoff – use multiple regression, relative weights analysis, and/or path analysis to determine which items have the most influence on an outcome such as engagement.
  5. Boil it down to 3 strengths and 3 areas of improvement – help your leaders see the top line story in terms of what is working well and what needs to be improved.  If the analysis is done correctly, you can tell this story in very simple terms while having nuanced and detailed insights underneath.

Simplicity and clarity are critical as long as they are based upon sound data analysis and accurate conclusions.  Simplicity in the form of broad and general survey items, high level feedback, and insufficient data can be dangerous.  The goal is to accurately describe the problem in simple, business terms.  Not to ‘generalize’ the problem based a few pieces of evidence.  In this age of information and complexity, understanding and interpreting what is happening around us is more critical than ever.  It is a natural human tendency to take a few pieces of information, and use that to reinforce our existing beliefs.  We can fight against this tendency in our organizations by remembering some basic and fundamental survey research practices to ensure we are distilling and simplifying information in the right places and to the right degree.

 

Leave A Comment