Phil Hearn: Blogger, Writer & Founder of MRDC Software Ltd.
Market Research Data Processing: Why You Need Choice
In some ways, market research data processing has not changed in years. As an industry, we collect data, clean the data, analyse the data and deliver it in some form to clients. For many, I suspect, the term ‘data processing’ conjures up an image of a factory taking raw materials and producing something more useful. This image, I will argue, is not helpful to our business in 2021. We need to be more agile.
The argument against my viewpoint
There will be some, I suspect, who will already be thinking “he doesn’t understand how research has changed in the last ten years”. The argument against my opening point is likely to centre on one of two points. Point number 1 is that market researchers use and sometimes collect a much broader range of data than ten years ago, e.g. passive data, customer feedback data, KPIs, other business data. Point number 2 is that the deliverables have widened from a set of crosstabs and a PowerPoint report to a world of online dashboards, infographics, models and automation. But, it’s still data processing!
The process of data processing
My dislike of the term ‘data processing’ is that it gives an image of ‘one size fits all’. Consequently, market research agencies often set up a system to process their data in one particular way. This approach may work well if every project that a research agency handles is similar, and clients want the same deliverables, but, increasingly, that is not the case. It would be far better to have flexibility. The barrier to this flexibility is that it means buying more software.
Is this just another software supplier telling us to buy more software?
You might assume at this point that I am just trying to encourage research agencies to spend more on software. Of course, there is an element of truth to that, but efficiency comes at a cost. However, the goal is to have all the tools you need to run an efficient business that a) is more profitable and b) delivers what clients want more effectively. In my DIY toolbox at home, I don’t just have a hammer, an excellent tool for one type of work, I have a selection of tools even that I am hopeless at DIY!
It’s not just about buying more software
I believe it goes well beyond the need to buy multiple software products. It is best if you have differently skilled staff that can use the software, often the same software product, utilising different skills. My argument is far more about having a flexible way of working for each project. Data processing systems can strangle this flexibility and tend to limit choices.
An example of how flexibility matters
a) The projects
To keep this example reasonably simple, let’s assume a research agency handles 50 quantitative research projects per year. Some of these projects may be very small (just a few questions), others may be bigger but straightforward, a few require complex analysis and some are tracking studies, large and small. The deliverables may include summary data, tabulations, charts, PowerPoint reports, online dashboards, or automated data delivery to a client system.
b) The process
The spread of projects in terms of complexity and deliverables does not lend itself to one system for every project. Yet, that it is exactly what happens in many cases. One strategy is applied to every project and then the differences, such as producing an online dashboard, are bolted on to the main standard practices. It’s no wonder that research businesses are not always flexible enough to meet modern customers’ demands.
c) Different skills, different software, different approaches
Many market research software products have multiple functions, e.g. collecting data and producing tables and charts. What you need in place are the right tools to suit your team. Here are some thoughts for more efficient data processing:
- Use more skilled staff for complex work, less skilled staff for easier work
- Consider researchers doing their own analysis where expertise in data management and analysis specification is not needed
- Use software that is suitable for tracking studies, e.g. handles changing questionnaires, code lists, data layouts easily
- Look at ways to automate tracking studies and use software that keeps minor changes as work for clerical staff rather than data processing experts
- Use software that is efficient for the type of work you are handling
- Have a manager who is responsible for considering the demands of each project rather than just processing each project in the same way
- Have a team with different skills – you do not want a highly paid expert data processing person doing simple work
- Have someone in the team who is an expert in understanding data; data is growing in variety, uses and importance.
- Use software that can link to as many other systems as possible easily.
Thinking about the software….
OK, so back to the software. The above may prove helpful. It is not an exhaustive list, just some ideas to trigger ways that data processing can become more efficient, provide clients with what they want and make you more profitable. One of the problems I find with data processing (and, in some cases, software vendors) is the answer to the question ‘can Product X do something?’ Often, the answer is yes, but it’s the wrong question. The question should be ‘can Product X do something easily?’ I have seen staff in data processing departments spend a whole day, for example, importing data from another system. When an error arises in the imported data or more data is supplied, it takes another day to import. ‘Easily’ is important.
Getting ready for more change
Market research deliverables for the last 20 years have generally focused on delivering PowerPoint reports. In that time, market research software vendors have been slow to respond to that challenge. There have been few products that have attempted to make that process easy. E-Tabs and Rosetta Studio are two products in this market, but there are few competitors and whether these products are suitable for every research project is doubtful. The next move in market research is the move to online dashboards, which is, in my opinion, long overdue. However, there are some significant steps to make online dashboards available to all research agencies. For example:
- Products are too expensive for most research agencies, particularly when the target customers work in the Business Intelligence market
- Some products are priced per project which makes it uneconomic for most projects
- Software is difficult to use
- Software is very limited in functionality
- Handling complex data is impossible (or very difficult)
There is still a long way to go, although, we feel that The CYS Platform that we sell has broken some new ground in some of these factors. Most other products fall on most of the above points.
MRDC Software’s approach
2020 proved to be a year in which we made considerable advances in our software. In addition to the productivity gains we provided in the previous ten years, we advanced automation in our software last year.
More choice in MRDC Software’s products
We increased the choice of how you produce cross-tabulations in 2020. For many years, we have provided MRDCL, a product for data processing professionals. We have also supplied QPSMR, which uses the same engine as MRDCL, but allows you to produce cross-tabulations using a more friendly user interface. In February 2021 we will be releasing Resolve, enabling researchers and clients to dip into data to create the main cross-tabulations or delve deeper into the data. The cost of Resolve? It’s free. There will be an inexpensive (about US$500 per year) Premium version of Resolve later in 2021, which will automate PowerPoint reports and allow you to produce complex tables using your own custom templates if you wish. This development is a big step towards improving choice, flexibility and being efficient.
More choice is the key
Increasing the choices of the way you work need not be expensive. It can take time to develop to an organisation that can be as agile as you need to be, but starting to take those steps now will produce rewards. In this article, I have tried to put forward some real arguments and give some likely scenarios, but every company is different. I am always happy to talk to anyone who wants to reappraise the way they work. Having a choice may not mean more expenditure. Being able to be agile and not having one way of doing things can offer great rewards. Arguably, it will be a necessity as we move through the 2020s.
