How to analyse and report on a tracking study

If you have data that you want to track over time, handling the analysis and reporting efficiently, is going to be important. Tracking studies are used in market research, marketing and other fields measuring performance. A well organised tracking study should deliver results quickly to recipients of the information. It is crucial that an efficient system in is place to handle the analysis and reporting of a tracking study. A poorly organised tracking study may take as much as 20 times as much staff time to carry out as a well organised study.

Have the right tools

To manage a tracking study, you might need a range of different skills and software. These might include powerful tabulation software like MRDCL, a high level of skills in Excel and PowerPoint (including knowledge of VBA), programming skills to produce online dashboards or to automate reports and data management skills with tools that can manipulate data.

What outputs do you need?

There are a number of reporting output forms that might be considered – or, indeed, a combination of these forms. These include:

• Excel reports
• PowerPoint presentations
• Online dashboards – either customised or generated through a product like DataDynamic
• Interactive Excel dashboards
• Cross tabulations generated by a product like MRDCL or QPSMR
• Smart reports generated by a product like Snap
• PDF reports
• Word documents

Running the analysis and reporting yourselves vs. buying services from specialists

MRDC Software offers both the software, which means you can do the whole thing yourselves, as well as a team of analysis and reporting professionals that can do all of the work for you. In some cases, we put together systems so that you can run reports yourselves – this can be a good way to use our expertise yet keep costs down. But, what is the right solution for you? There are 5 key questions that you should guide you in the right direction.

1. How complex are your requirements?

At the simplest level, you might get some key figures each month and want to provide these figures in a short PowerPoint or Excel report. Automating reporting from this type of data input is fairly easy to handle. You can easily find some free templates that will facilitate this for you. At most, you might need a simple distribution system that emails reports to recipients of the information.

At the most difficult end, you might have complex data calculations, algorithms, changing data inputs with different structures which need experts to handle efficiently. If you do not have staff highly competent in these skills, you are likely to save money engaging analysis and reporting professionals like those offered by MRDC Software.

2. How many sources of data are being plugged into your analysis and reporting?

Data comes in all shapes and sizes. Your data may come in as Excel or Access file, from a SQL server database, as ASCII data, in Triple-S format, in a SPSS file or some other proprietary form. If files are unusually large or in a format which you are unfamiliar with or you don’t have the tools to manipulate, you are likely to need expert help.

Market research tends to be different to most other commercial data in that it tends to contain a relatively small number of records (less 1000 respondents is not unusual), but it is likely to contain a large number of variables – 1000s in many cases. Most commercial data tends to have a large number of records with a small number of fields. These types of data need different tools to analyse and manage. Again, familiarity with these different forms should guide your decision.

3. How much is your data likely to change in structure?

Whilst building a system to automate analysis and reporting, whether it is simple or complex, can take some time, the task becomes easier as the tracking study continues if data is always available in the same format. If your data is coming from different suppliers, it’s important to stress the importance of consistency.

Putting in a structure to your data can make reporting much easier if there is pre-planning. A simple of example of this might be structuring an Excel spreadsheet such that that data appears in specific rows and columns and varies month to month to an agreed format if, say, there are additional brands being reported.

The more differences between data from period to period in your tracking study, the more effort that is needed to manage your reporting thus increasing the need for expertise in handling efficiently.

4. How much are your analysis and reporting requirements likely to change?

One of the most important considerations when reporting data from a tracking study is to consider all of the stakeholders. It is not uncommon for some users of the information to want brief summary reports that are easy to absorb whereas other stakeholders may want large amount of repetitive or detailed data.

Before producing the analysis and reports from a tracking study, it is important to ensure that all stakeholders have what they want. Making regular changes to the analysis and reporting can prove expensive. It is far better to consider in advance what might be needed and carry out a risk assessment estimating the cost of change. Again, data analysis experts will be needed if more flexible and changeable reporting is required.

5. What skill sets do you have internally?

Managing tracking studies requires some important skills – some require expertise, others require just good organisation. Good organisation is often underestimated. Documenting lists, how a system works etc. is very important, particularly when key staff leave or change roles.

Good technical skills are also important. Understanding how to manage data and program automated analysis and reports is important. A good breadth of skills and software tools is important, so that the most efficient solution can be put together. As an example of this, I recently worked with a company that had excellent Excel skills using formulas and VBA (Visual Basic for Applications), Excel’s in-built programming language. However, whilst they were experts in Excel, they had little other knowledge, so that when a professional tabulation system like MRDCL was needed to analyse results they persisted in trying to use Excel, which wasted many man days and made processing slow (days rather than minutes) even when they had partially succeeded using Excel.

What you need to make the automation of your tracking study analysis and reporting easy

1. Understand the scope – ensure that you understand how the data will arrive and how reporting is required.
2. Anticipate changes – if data or reports may change or be structured differently, be ready for these changes wherever possible.
3. Don’t postpone problems – if something major changes, don’t ignore the problem and find workarounds. You are likely to pay a premium for this in the long run.
4. Have a budget for changes – make sure there is a budget to streamline systems every 6 to 12 waves. This means annually for a monthly project and about every two years for a quarterly project.

Similar can be a lot more expensive than the same

Clients have sometimes been surprised that there is a such a big difference in cost between repeating a process and reproducing something that is similar. Repeating the analysis and reporting from a tracking study that is identical should be a matter of pressing some buttons and checking the output. Where things change, a lot more work is needed. Seemingly, minor differences can mean a lot of minor programming changes, which add up to a whole new re-test procedure.

Making changes to a computer programs is a difficult task, which is prone to error, even with the best documentation and standard operating procedures. Program code is as hard to read as a fairly unfamiliar foreign language.

Think costs

Having worked in data management, analysis and reporting for many years, I have encountered a number of badly handled tracking studies that MRDC has, in many cases, helped to reorganise to make efficient.

One recent project that I encountered was costing just under US$10000 in staff time to process each month. By reorganising the project at a cost of US$15,000, the project was reduced to a monthly cost of about US$1,500. In other words, savings were made within two months.

Organise the project from the start or, if it’s started, organise it now!

The example above is not uncommon. Time and costs can soon run out of control where a tracking study is not well managed. We are not talking about 10% or 20% improvement in efficiency; we are talking about massive potential savings.

Need help or advice? Just ask.