Effective sample size calculator – with free Excel working model

Target weighting is a technique used in market research to scale a sample of respondents to a known or desired target or population. For example, you may have 40% males and 60% females but want 50% of both males and females. MRDC Software can provide you with a free working model in Excel if you email phil.hearn@mrdcsoftware.com.

Scope

The purpose of this article is to explain how target weighting works and to understand the effects of the weighting on your data. After you have applied weighting factors to your data, th

e effective sample size will usually be reduced.

The article only explains how target weighting works and some basic checks that you should apply.

If you want a working model in Excel, please email phil.hearn@mrdcsoftware.com and you can receive our free target weighting calculator which also give you the effective sample size. There is a video that shows you how to use the target weighting and effective sample size calculator.

If you want to know how to carry out target weighting using the free target weighting and effective sample size calculator, please see this video.

What is target weighting?

Target weighting allows you to scale survey respondents to a targeted number of respondents. To do this, you will usually be weighting respondents to one or more questions or variables. For example, you may wish to adjust your sample so that it represents 50% males and 50% females. If you survey sample had 40% males and 60% females, you would apply a weighting factor to the male sample of 0.83333 (50% divided by 60%) and to the female sample of 1.2 (50% divided by 40%). If these weighting factors are applied to the data, you will get a weighted sample which is 50% males and 50% females.

Target weighting is often carried out using a two-dimensional matrix. For example, gender within age group. For example, you may have the following targets:

Males 18-34 – 20%

Males 35-54 – 15%

Males 55+ – 15%

Females 18-34 – 15%

Females 35-54 – 25%

Females 55+ – 10%

You should note that weighting targets can be figures or percentages. The examples here use percentages but the working model allows either to be used.

In this example, if Males 18-34 were 15% of the sample, they would get a weight of 1.33333 (20% divided by 15%). The weights for the other five cells would be calculated in the same way.

The targets do not need to be a matrix – you could have four age groups for females and three for males, for example. However, it is important that the percentages add to 100% and that every respondent falls into one (and only one) of the categories.

Three (or more) dimensional weighting matrices are also possible, but care should be taken not to spread the sample too thinly as your effective sample size may reduce too much (see next section).

You will note that a factor is calculated for each cell of your weighting matrix by dividing target by the actual number of respondents.

What is effective sample size?

The effective sample size is the sample which could have been achieved by an effective unweighted random sample. In other words, if your sample size of 200 has an effective sample size of 100, you could have achieved results that were as statistically accurate with a random sample of 100 that met your target criteria.

The effective sample size is an important figure that should always be checked when weighting survey data. An extreme example shows this. Let’s say your targets are 50% males and 50% females in a sample of 200 people, but you have 99% males and 1% females in your sample. When you apply weighting as described in section 2, your effective sample would be 7.92. In other words, your sample of 200 people has reduced to the validity of a random sample of 7.92 people.

What does this mean? It is easy to think that target weighting can be used to solve sampling defects. If it can, it comes with a penalty. It is good practice to observe the weighting factors being applied to each cell of your weighting target matrix, but, as an absolute minimum, you should always check (and preferably show) the effective sample size in any analysis you do.

Other things to consider

• Each respondent must fall into one (and only one) of the categories in your weighting target matrix. If a respondent has missing data such as no age where you are scaling data to age groups, there is no magic solution; you will need to remove the record or apply a weight to this record manually.
• Target weighting uses interlocking cells for its targets. For example, where you have two genders and three ages, you would need targets for the each of the six cells in the target weighting matrix. If you only have targets for the two genders and the three age groups, you will need use rim weighting – see link.
• Try to avoid using too many cells, particularly when you have a small sample. For example, it is possible to weight to 2 gender categories by 4 age groups by 5 regions by 4 social grades, but this means that each respondent will fall into one of 160 cells (2 * 4 * 5 * 4). If your sample size is only 300, you are almost certain to get some high or low weights.
• Where there is no one in your sample for a particular cell, you cannot weight no one to a target. This may sound obvious, of course, but can be an undiscovered problem, particularly where a software package is making the calculations automatically. It is important to check this. The usual solution is to collapse some cells together – maybe 3 age groups rather than 6 age groups.

If you have any further questions, please do not hesitate to contact me.

Just email phil.hearn@mrdcsoftware.com and I will send you a working example in Excel that you can use to calculate target weights and effective sample size yourself.