Wednesday, November 9, 2011

Computation of Response Rates in RDD Telephone Surveys

Suppose you are interested in examining the fat-free food items in Target. You can quickly screen out the clothing and furniture items by taking a quick look. You may then go to the food department and look at the labels on all the items whether they are fat-free. Some will be, some will not, and a third set you will not be able to establish (maybe they are out of reach...).

You can apply a rate to those unknowns, based on the knowns, multiplying the unknowns by the calculated "fat-free" rate (fat-free/(fat-free + non-fat-free)).

If you want to show a lower fat-free food rate the store, you can compute the fat-free rate as: (fat-free/(fat-free + non-fat-free + clothing & furniture items)). This is bound to be an underestimate since the clothing and food items never went through the same "data collection" process of looking at labels.

Odd example, but not unlike telephone surveys in which we need to calculate an eligibility rate to apply to the numbers that we have not established eligibility. Here, the prescreened out clothing and furniture items are the telephone numbers that prior to data collection have been purged through comparisons against lists of known nonworking and business numbers and possibly by dialing each number and hearing a tri-tone.

AAPOR provides an invaluable standardization of response rates, but unfortunately, does not provide clear guidance on how to calculate this eligibility rate, or "e". In its absence, some telephone surveys use the prescreened numbers in the denominator of "e", only later to apply to cases that have already been prescreened.

Given the very large proportions of telephone numbers that are prescreeened and that end up without established eligibility at the end of data collection, this is a critical aspect of response rates that needs to be addressed and guidelines set forth.