Results of Cognitive Testing

Appendix C

Results of Cognitive Testing of the M-3DOD Form

The M-3DOD form underwent cognitive testing for two months by the Office of Statistical Methods and Research for Economic Programs (OSMREP) staff at the U.S. Census Bureau prior to the OMB submission. It involved two rounds of testing with changes being made after each round. This proved to be very beneficial because many important changes were implemented. A summary of the testing is described below.

Before testing began, the OSMREP staff recommended a few changes to the M-3DOD form based on their expertise. Once the changes were made, one member of the OSMREP staff and one member of the M-3DOD team went on company visits. At the meetings, respondents were given copies of the form with instructions to describe the process by which they would complete it. As respondents went through the form, they were asked non-leading questions about their interpretations of the items and how they would retrieve data. Their responses were probed until a thorough and accurate understanding was obtained of the response process for each item. A total of fourteen cognitive testing visits were conducted by OSMREP and M-3DOD staff. After each round of testing, the OSMREP staff presented their findings and recommendations to the M-3DOD team. Some of the changes that were made include:

• Re-wording of the questions to be clearly understood.

• Moving text to different locations.

• Moving the purpose statement to the back of the form.

• Moving information on reporting period to the front of the form.

• Adding more examples of agencies that should be excluded from Department

of Defense in the instructions.

As a result of cognitive testing, the improved M-3DOD form is presented in Appendix A.