Large ongoing national surveys such as the Consumer Expenditure surveys, the National Crime Victimization Survey, the National Health Interview Survey, the National Survey on Drug Use and Health, and the National Immunization Survey (these are just a few haphazard examples), field major methodological experiments, the data from which are not intended to be used in official estimates but rather, the inform changes in the design of the studies.
Unfortunately, the data from these large scale methodological experiments are usually not publicly released. This is not through inadvertent oversight. At a minimum, there is cost associated with preparing and releasing data files - resources that are needed on the particular survey being redesigned.
The downside, however, is that these carefully designed and often costly experiments are used to inform a set of design decisions on a survey and, at best, result in a limited number of publicly released reports. Yet these data could be used for a wide array of methodological work by students and researchers. Some of these studies include the collection of validation data, experimental manipulation of questions, random administration of questionnaire modules, mode experiments, incentive structure, etc.
Not being able to use some of the largest, carefully designed survey experiments, seems like a missed opportunity for research, informing better and more efficient survey designs, and helping the development of the next generation of researchers.
The agencies and organizations conducting these surveys have little impetus to change this. It takes up scares resources from the survey being redesign, both monetary and personnel time.
It does make me wonder if external research funding could be dedicated to making data from such important and valuable survey experiments public (even if some of these data are released as restricted use files with special applications or to be used in special locations, such as the Census Research Data Centers). There are a number of great methodological experiments to inform survey design conducted over the past five decades that contain invaluable information.
Furthermore, such funding could be seen as cost reduction (albeit unmeasurable) as some future experiments may not be deemed necessary as answers may be found in extant data, and methods may be improved from "free" research conducted by the research community.
