Faced with a haystack of data, how do we find the needle of insight?

Once upon a time the market researcher was the one who asked the best questions.  This skill is more in demand than ever.  Paradoxically the task of the insight function has become increasingly more difficult in the information era as the expectations of the decision makers have grown.  The growth in processing power, and the access to vast amounts of data, has provided a playground for the newly emerged data scientists – who have appeared like characters from Lord of the Rings – to magic up wondrous solutions to, well, what questions exactly?  The data scientists can dive into the giant haystack of data but without the right questions they won’t know what they are searching for; it might be a needle, or then again, it might be a camel.  Which is where small data comes to the fore.  Without active, smart use of small data the marvellous potential of big data will not be fully realised.  The right questions will come from the customers that your business is trying to serve.  Digital surveys provide quick, acceptable answers to everyday questions – this keeps the business managers happy –  but it requires the social interaction of face to face qualitative research to tease out the needs of your customers and focus the attention of your data scientists on the right areas of exploration.  They need clues.  There is a natural demand for evidence to reduce the risk involved in the decision making process, but there is also a demand for increased speed of delivery.   We believe you will save time and energy by using good, old-fashioned interrogative skills to get close to your customers, and, the simple use of the mobile phone to communicate the findings as they occur.  Then you can steer your data scientists in the right direction.

We believe in keeping it simple.   Simple doesn’t mean easy.  It takes hard work to provide simple solutions.  At the end of the day it is the desire of us all to make life simpler and better for our colleagues and our customers; though not necessarily in that order.  The traditional craft of the skilled, qualitative researcher is the ability to understand not what the customer says, but what the customer really means.  It is one thing to recognise the difference between system 1 and system 2 thinking (thank you Daniel Khaneman) but it is another thing to put it into practice.  There is no short cut when trying to help customers understand the architecture of choice, particularly when they often don’t understand it themselves.  

It is difficult to know who to trust in this day and age, and yet, trust is at the heart of our decision making.  The world of insight continuously evolves but skill and hard work are still the foundation blocks for success, and experience matters!