Everything you need to know about survey data collection
Data Collection Software
If you want to conduct a market research survey, there are several methods you can use to collect the data.
The most common methods are:
Additionally, there are some less common methods for collecting survey responses. These include postal surveys, self-completion kiosk surveys and email surveys. You can also collect data passively or from other sources such as transactional data.
Each of the primary methodologies has its advantages and disadvantages. They are as follows:
|Pen & paper interviews||PAPI||• Interviewer present to encourage engagement|
• Easy to manage sample
|• Cost and timescales |
• Needs interviewers to carry out
• Manual data entry needed
|Computer-assisted personal interviews||CAPI||• Data can be uploaded quickly |
• Interviewer present to encourage engagement
|• More expensive than online surveys |
• A project takes longer to complete
• Needs interviewers to carry out
|Computer-assisted telephone interviews||CATI||• Data ready after interview |
• Better respondent engagement
• Sample and quotas managed
|• More expensive than online surveys |
• Requires dedicated CATI installation
• Needs interviewers to carry out
|Computer-assisted web interviews||CAWI/online surveys||• Less expensive |
• Quick turnaround feasible
• Completed at any time
|• Less control over the interview |
• Questionnaires need to be clearer
• Respondents more likely to quit
There is no easy answer to this question. Most surveys nowadays are conducted online as it is cheaper and more convenient to conduct surveys this way. Also, it is easier to carry out an online survey quickly. In contrast, a pen and paper interview, for example, requires far more organisation as well as the availability and management of interviewers. Finding the right people to answer an online survey can be difficult, particularly with ‘hard to reach’ samples, such as senior businesspersons.
Online surveys can be in place quickly and cheaply as there are several free or low-cost platforms. These low-cost platforms may have limited capabilities or make it challenging to analyse the data you have collected, but they can offer a good way to get started, particularly if you have a small survey.
One of the challenges, however, that should be considered is the importance of well-structured questionnaires with carefully worded questions. It is crucial to ensure that poorly framed questions do not lead to a bias in how respondents answer the questions. Professional market researchers can perform an important role in ensuring surveys produce ‘fair’ results.
Let’s now look at best practice for each of the main survey data collection methodologies
Paper surveys conducted by interviewers, often called face to face interviews, are the original method of collecting survey data. The form is still widely used in some countries where communications are limited or where large parts of the population are in remote or rural areas. Although paper surveys have a place at events or where customers are present in suitable numbers, there has still been a move to online surveys where respondents can use their smartphone to complete the survey.
It’s easy to assume that paper surveys are redundant, but they do have their place in the market research world. Two good examples are:
With the rapid growth of online surveys, few modern software products produce interviewer-ready questionnaires. Many people use Microsoft Word or similar products to create questionnaires, but this means that texts have to be re-entered into analysis software for tabulations, charts and reports. Both of our products, Snap and QPSMR, produce paper questionnaires, but only Snap has all the tools you need for professional-looking documents. QPSMR creates a questionnaire that you are likely to want to edit in Word (or similar products) to enhance its appearance.
To proceed to analysis, you have three choices for processing data from paper questionnaires:
It is usually best to use a software package that lets you prepare the questionnaire and automatically sets up data entry for the questionnaires unless you are planning to analyse data manually or use a scanner. If you need to use a separate software package (or something like Excel) to enter data from paper questionnaires, this is likely to add inefficiency and errors. Both QPSMR and Snap automatically generate screens so that data entry staff can enter data from completed questionnaires. QPSMR also has a data entry only module, called MRInput, that is freely distributable so that as many staff as you wish can enter data from questionnaires.
Whatever software you use, it is important to consider the whole process from data collection through to the deliverables you wish to produce. While transferring data from one system can work well by using compatible data, Triple-S or an API (application program interface), it is essential to check that this is a smooth process before starting data collection.
In some cases, it is practical to scan questionnaires so that the data is available without keyboard entry. Firstly, you will need to use a software package like Snap that produces questionnaires that can be scanned – very few software packages have this facility. Secondly, the questionnaire usually needs to be reasonably short in length and not subject to too many changes throughout its lifetime. Scanning is a more suitable route for large volumes. Snap has an additional module to manage scanned surveys. You will usually need tools in the software that can manage tolerances for reading completed questionnaires as well as tools to reject or highlight questionable data where answers have been altered or crossed out. Before considering scanning questionnaires, it is usually best to talk with someone skilled in this technique.
Paper self-completion questionnaires are also used from time to time for mailed surveys or ‘at event’ surveys. Everything that applies to interviewer-administered paper surveys applies equally to self-completion paper surveys. There is a general trend of replacing self-completion surveys being replaced by online surveys, often through a mobile device such as smartphone.
Interviewers conduct CAPI surveys, but rather than using paper questionnaires, they use an electronic device, usually a tablet. There are several important considerations when choosing the right CAPI software for your business. The key issues range from security through to the interviewer’s internet connectivity. The primary benefit of CAPI surveys over paper surveys is that the data can be uploaded and centralised without the need for data entry staff to enter the data. There are several considerations that you also need to take into account.
Some CAPI software products work by being connected online at all times. Effectively, these are the same as online surveys with an interviewer entering the answers on behalf of respondents. The best software products work by storing data from interviews on the device and uploading if and when an internet connection is available – with, of course, appropriate reminders to the interviewer to upload data when a signal has not been present. CAPI software that relies on an internet connection is generally not practical. Where there are signal fluctuations or if there is a need to carry out interviews in uncovered areas, storage of data on the device is a necessity. In our view, CAPI software should work both offline and online, even in areas where there is usually a good internet connection.
Year on year, data security has become a more serious issue. The loss of any personal data can have a severe, detrimental effect on any company. It is crucial, therefore, that the data collected from respondents is safe, both on the device itself is safe, the transmission is safe and the database in which it is centralised is safe. Our product, Snap, maintains the highest standards adhering to ISO27001. Further, data on the device is encrypted so that if a device is lost or stolen, the data could not be examined or used by anyone.
In addition to the importance of CAPI surveys working offline or online, there is a need to ensure that CAPI software works from the device itself. This becomes particularly important where part of the CAPI interview requires respondents to watch videos or view images. If the media is accessed via the internet, picture or video quality may be unreliable, distorted or slow if it depends on internet connectivity. Snap works by downloading media to the device itself so that there is no such risk.
The CAPI software market can be more complicated as a market than it might appear. While there are several products capable of working as a CAPI product, it is, in almost all cases, essential to ensure that you are using a product that has all the facilities stated above. Indeed, once you have a team of interviewers, there is a need to have the right measures in place. It is easy to give too little consideration to these measures.
The method by which a software platform transmits surveys and interviews to each interviewer needs to be as simple as possible to administer and practical to manage. The level of comprehensiveness of these administrative tools that you will need will be dependent on the complexity and size of your operation. Again, Snap is a top of the range tool managing all these administrative aspects of your project.
One of the problems we hear from users of some CAPI products is the loss of interviews when they are uploaded. While this may only happen occasionally, it can still be a costly problem or cause delays in delivering final results to clients. The protocols for uploading any data can range from low levels to very reliable. It is important to use a product like Snap that uses the most reliable protocols to upload its data.
Although this is the last topic covered under CAPI surveys, it is, of course, important that the software you choose has all the features you need. The features are generally similar to other data collection methods, such as different question types, routing, answer logic etc. What is of less importance than, say, for online surveys is the look and feel of the questionnaire on the interviewer’s screen. As long as it is understandable and practical, it should be fine. The features offered by Snap are summarised, but it is a complete system which will handle more or less all the needs you will have for successful CAPI surveys.
Survey respondents usually complete CAWI surveys without an interviewer present. For this reason, survey managers must present questionnaires in a clear way and with questions that are easy to understand and unambiguous. If a respondent is unable to understand anything, they are likely to quit, leaving you with an incomplete survey. Online surveys are about clarity and avoiding anything is boring to complete or repetitious. Further, online surveys should not be too long – 10 minutes is usually enough. However, this may be dependent on whether your respondents are passionate about the subject involved and the rewards you offer.
The main motives for using online surveys are significant speed improvements and significant cost reduction. Researchers can put a questionnaire into field and have analysed data within 24 hours. The future of market research may depend on these factors, prompting continuing change.
It does not need saying that if your potential survey respondents fail to complete your survey, their data is of limited or no use. People need incentives to complete a survey. They may be financial rewards by offering a fee, by entering those completing the survey to enter a prize draw or they may be for more altruistic reasons such as a charitable donation. But, financial incentives are not necessarily all that is needed to encourage people to complete your survey.
Preparing a survey that is the right length, is easy to understand and as interesting as possible to complete will bring far better success than long, dull questionnaires where respondents become disengaged. It is important to learn how to make surveys engaging. If online survey respondents are confused or get bored, it’s important to remember that they are one click away from leaving your survey. While keeping a survey short is important, it has been shown that longer interesting surveys are more likely to be completed than shorter uninteresting ones. An example of this is a rating scale. By giving respondents the task of dragging statements into piles of very good to very poor, for example, the respondent will be more engaged even though it might take a little more time than checking boxes in a long grid. Further, respondents are more likely to read questions more carefully if they are switched on.
We like to think that we offer good advice and have the right software tools that will engage your potential respondents so that you achieve good response rates. We have provided a checklist of things you should think about before you launch your survey. Don’t be fooled by suppliers of software that will tell you need high-level scripting software for most surveys. It’s simply not true. It will not improve your response rates, data quality, research quality or anything else. It will just reduce your profits and reduce the chances of winning more business.
The software you use as your survey platform must be good enough in many different ways, but there is no real to exceed ‘good enough’. We have documented the main considerations you should make. Having the most expensive, feature-rich software for online surveys does not guarantee high responses, better questionnaires or more successful survey outcomes. Surveys should be engaging and interesting enough to complete, but they do not need to be overly-designed in most cases.
It is easy to put too much emphasis on detailed or ‘nice-to-have’ features within the platform you use. If a product does not have a feature which is fundamental to the questionnaires you want to use, you will need to seek an alternative product. However, it is important not to forget the business-focused issues, which may mean the difference between winning and losing a client or chunk of business. We cover questionnaire features separately as we believe they need separate consideration.
They range from security to being able to output the data you need. Here are some ‘business-related’ features that you should consider.
There are numerous questionnaire features that you might need to consider, but here is a list of the top 20 features (and sub-features) that you are likely to need. If you want additional modules or functionalities such as tables, charts, online dashboards, these are covered separately in the next section.
This is not an exhaustive list, but these are features that you are likely to need for many surveys and probably constitute minimum requirements if you are offering an online survey service to customers.
One of the key considerations when choosing a CAWI platform is whether you need any other functionality. Let’s consider a few types of users that want to use a CAWI platform.
|User profile||What else do you need to consider?|
|Wants to conduct a one-off survey of 25 people asking 5 questions||Probably none. You could hand count the answers!|
|Wants to conduct a one-off survey of 100 people with 20 questions||You could hand count the results, but it would be better if you could drop the data into Excel or something superior for analysis and reporting to produce some counts and percentages|
|Wants to conduct a one-off survey of 1000 respondents with 50 questions||Hand counting is now out of the question. Excel might be good enough, but if it's not you will probably want the platform to carry out analysis and reporting or it needs to be easy to transfer the data to another platform.|
|Wants to conduct several projects with 100+ respondents with questionnaires of varying length||You need to be 100% certain that the CAWI platform can do all the analysis and reporting you need or you can transfer data. Data can often be transferred successfully using Triple-S or SPSS format. Make sure this works.|
|Wants to conduct several projects and expects CAWI platform to produce tables and charts||Snap is the right tool for you. It has all the tools you need for all these tasks. Further, if you want to link the survey data to another platform or your client wants the data in a different form, Snap handles almost all output formats you will need.|
|Wants to conduct several projects and expects CAWI platform to produce multiple automated reports||Snap has a highly effective Smart Reporting tool which can automate reports for you. Very few CAWI platforms have any tools like this.|
|Wants to conduct several projects and expects CAWI platform to produce online dashboards||While Snap can easily link to most dashboard systems, The CYS Platform is the right tool for you as it can collect data, read data from other sources (including Snap) and produce first-class data visualisation facilities within the platform.|
|Wants to conduct some projects where data is connected to an external database to read data or send data back to a customer service centre or a database||Both Snap and The CYS Platform have tools to do this. Few platforms offer this type of functionality.|
|Wants to combine results from different surveys together with CAWI survey data and other business data such as KPIs to show in a dashboard||The CYS Platform handles all these things in one platform. Its data management facilities are second to none, and its data visualisation tools are unmatched.|
Considering how important each of the above features are for your purposes should help to guide you to the right software choice. We have put together a guide which will help you to decide which features are important to your organisation or clients. The reason this matter is so important is that CAWI software comes in all shapes and sizes. There are some free or nearly free products, some low-cost products, some mid-range well-featured products and some top of the range product. Making a mistake can be costly. Buying a top of the range platform when you do not need can be a massive waste of money. At the other extreme, using a free product or buying a product at the low end of the scale can lead to an expensive error too. We have seen buyers make mistakes at both end of the spectrum.
We do not advocate buying top of the range software unless you need it. And, the fact of the matter is, many companies do not need expensive products to carry out professional-looking market research surveys. If you are new to online surveys, we suggest you start with a free or low-cost product to learn about online surveys. The two platforms we offer target different markets and will not be the right platforms for everyone. Our Snap platform is aimed at those looking to do professional surveys of any size using (relatively) easy to use tools (the middle-upper market). Our CYS Platform is more geared to those collecting customer experience (CX) feedback, employee research or smaller surveys, particularly where the data needs to be merged with other business data for highly-customised reporting and online dashboarding. But, let’s look at the range of tools available.
You can conduct small, simple surveys for free using Google Forms. Maybe, this is the place to start if you are new to online surveys. If this platform is good enough for your particular needs, there is no need to look further. Understanding its limitations is important, but we believe that Google Forms has its place for online surveys. At the other end of the scale, you can subscribe to platforms like Confirmit, but to get the real benefit you will highly skilled staff, high volumes of work and a high percentage of clients wanting complex surveys.
The range of software for online surveys is wider than any other aspect of market research software. There are some important guidelines that you should consider when choosing software. The low-cost entry products can be ideal for many people wishing to conduct online surveys. The main drawbacks to consider are:
At the more expensive end of the market, you need to consider:
The decisions are not easy, but buying at the right level can be a significant decision with long term implications to your costs in running online surveys.
Most platforms break down into one of these types with some platforms restricting features.
These are typically the most restrictive systems and often limited output or data analysis options. They may suit your needs for a ‘quick survey’ but are unlikely to be of much use for most commercial needs.
Some licence fees include a maximum number of interviews that you can collect. The further up the price range you go, the more interviews you will able to carry out. Many of these platforms start inexpensively, but become far more expensive than other platforms like Snap once you collect a large number of interviews. As you buy ‘bigger’ licences, you are often entitled to additional logins and the ability to run more projects or surveys.
Some software is sold with two fees – one for the licence (or licences) often reflected by the number of users plus a cost for a bundle of interviews. Our Snap solution is sold with a licence fee which reflects the number of users and a cost for bundles of interviews. The larger the block of interviews you buy, the cheaper the cost per interview. For example, Snap interviews are priced at only about 25% dearer for 10,000 interviews than 5,000 interviews. Snap is an ideal product for companies running commercial surveys with 10000 or more interviews per year.
Some platforms have other additional fees. Our CYS platform is sold as a package which includes a number of users, projects and collaborators. As you go up the price ranges, these packages increase in size. There is a small extra fee based on the number of projects that have active.
Many packages deliberately restrict key features, which can make the lower-cost versions of little use commercially. The CYS Platform is sold as packages with increasing features, but they are in line with the level of use that each type of user demands. Snap has no limitation on any of its features – once you buy a licence, you have access to all the main features excluding one or two specialist needs such as scanning, for example, which needs an additional module.
The total cost, therefore, comes from:
Speak to MRDC Software to find out how you can do CAWI surveys effectively.
Interviewers carry out CATI surveys by telephone. The interviewers are usually in a central location with a supervisor present to manage the team and the projects. In some operations, a separate person will be responsible for the projects. The interviewer to respondent connection means that longer surveys can be more successful using CATI than self-completion online (CAWI) surveys.
Most CATI centres will operate using software that has interviewers networked to a server so that potential respondents can be issued to each interviewer as they become ready for the next interview. Software controls the sample file, which contains the telephone numbers and, possibly, other respondent details, such that callbacks, appointments and unavailable numbers can be recorded centrally and utilised appropriately. Setting up a CATI installation is a fairly straightforward if you follow these steps.
It is possible to handle telephone interviews without professional CATI software. However, there will be a need for a lot of manual recording of the status of potential respondents, which may become impractical to manage if there are more than one or two telephone operators. Online software systems can offer a way to conduct telephone interviews, but sample management soon becomes problematic except for small surveys.
Software for CATI surveys tends to be specialised. Generally, the CATI software market has not developed as fast as the online survey market as more interviews are online using self-completion by respondents to collect data. CATI, however, offers a convenient way to handle complex/longer surveys or where the presence of an interviewer will improve quality or increase response rates. CATI surveys are generally quicker to prepare as the survey appearance has only to be clear enough for interviewers. Some of the key considerations are as follows:
Some CATI systems need scripting to produce questionnaires. As with CAWI surveys, this can be expensive route to follow as you need highly-skilled staff to use such software. Most of the better systems can be used with an easy-to-use interface. Our QPSMR CATI product needs little training and is easy to understand.
Supervisor functionality is a necessity unless you are running one or two small projects. Supervisor functionality means that:
CATI software needs a full range of features to route through a questionnaire appropriately and to control the responses given. These are broadly in line with the questionnaire features needed for CAWI surveys.
Our QPSMR CATI software has a unique design that puts the minimum load on the server that you use. This design means that your expenditure on hardware can probably be lower than any other system on the market – an important consideration. The QPSMR CATI design means that an interviewer’s computer or terminal only connects with the server at the beginning and the end of the interview. This means that the risk of lag during an interview is negated – something that other systems can suffer from where there is inadequate hardware.
Data security has increased in importance in recent years. Data from QPSMR CATI surveys are stored on your own servers or, if you wish, in the Cloud. The telephone numbers that are dialled that are subject to data protection laws in many countries around the world can be encrypted so that interviewers and project managers have no knowledge of the numbers they are dialling. You can even receive encrypted telephone numbers from your clients so that you have no risk of losing any personal data
CATI centres can prove costly to set up, but with the right approaches and tools, you can get good value for money. Staff and staff training costs need to be considered as well as hardware costs
QPSMR has a full set of tools for managing paper surveys. The upgrade path to using the CATI version of QPSMR simply means learning how to use a few more tools. Most clients need some assistance to handle their first project, but after that, you can manage each project in the same way. We provide free training as well as complete set of videos to show you how to upgrade
The tools in CATI module need no scripting to use. They cover system or user data that you may want to record, such as the date/time of interviewer, the interviewer’s ID or name, the length of the interview or the time taken to complete a section of the questionnaire. There are also other controls to manage the number of times that a respondent is called back and the dates/times of any appointments that are made.
From time to time, there is no need to run multimode surveys. This means using more than one data collection method that we have covered already. The need for multimode might be because some respondents prefer to complete their surveys in a different way, e.g. on paper rather than online or, perhaps, because you have a hard-to-reach sample that are best accessed in a different way.
Some software systems allow mixed modes of data collection, but they will store each data file separately. This can lead to spending considerable time merging data files together so that it can all be analysed as one set of data. Multimode software stores data from any source in one database making it more convenient to analyse.
Snap is a full multimode system. Whether your data is from paper questionnaires, CAPI surveys, online interviews or scanned questionnaires, the data will reside in one database which will allow you to analyse data as one, splitting it by data collection methodology if you wish. Snap does not have a full CATI supervisor module, although it is used for some smaller CATI surveys.
If you want to merge Snap data with QPSMR CATI, you would need to merge the data files together if you wanted to analyse the data together. The amount of work is minimised as all our products adhere to the Triple-S standard, which makes transferring data an easy task.