Carol_White_PEP_Supporting_Statement_Part_B_Revised_4-01-2011

American Institutes for Research®

Evaluation of the Carol M. White Physical Education Program -
Supporting Statement for Paperwork Reduction Act - Part B: Description of Statistical Methods

April 1, 2011

Prepared By:

American Institutes for Research^®

1000 Thomas Jefferson St., NW

Washington, DC 20007

Prepared for:

U.S. Department of Education

Policy and Program Studies Service

Office of Planning, Evaluation, and Policy Development

Supporting Statement for Paperwork Reduction Act

B. Description of Statistical Methods

1. Universe and Respondent Selection

The surveys of the Carol M. White Physical Education Program (PEP) project administrators will be given to the entire universe (or population), rather than a sample, of the 77 PEP grantees that are FY 2010 cohort PEP grant recipients. This is because the size of the population is small enough (77 grantees) that sampling would not be appropriate. Furthermore, each of the PEP projects is somewhat unique and the U.S. Department of Education (ED) seeks a comprehensive picture of PEP activities.

2. Procedures for Collecting Information

The AIR study team will collect information about the design and implementation of the PEP through four survey instruments developed by the study team: Year 1 and Year 3 Program Design and Implementation (PDI) surveys for LEAs and Year 1 and Year 3 PDI surveys for CBOs (the survey instruments are contained in separate files).

The surveys will be administered on-line. Paper-and-pencil copies of the surveys will also be available upon request. Administrators of all 2010 PEP grantees will be contacted via email to take the survey. Given that PEP grantees made the commitment to participate in the evaluation in their funding applications, we expect all survey recipients to complete the surveys. In addition, administrators of all PEP grantees will receive a pre-notification email about the study emphasizing its importance and requesting their participation. Three to five emails over a period of two months (depending on response rates monitored on a weekly basis) will follow up with non-respondents to encourage participation (draft contact emails and letters are contained in a separate file).

3. Methods to Maximize Response Rates

As mentioned above, one method to maximize response rates was to require grantees’ commitment to participating in the evaluation in their grant applications. In addition, ED publicized the study at the grantee kick-off meeting and will provide updates to grantees at all grantee meetings. The follow-up procedures discussed in question 2 above will be implemented to follow up with and convince non-respondents of the importance of participation.

It is expected that the response rate for the PEP project administrators will be 100 percent because all PEP grantees have agreed to participate in the study and they are all current grantees of the U.S. Department of Education. Although a response rate of 100 percent is expected, we will test for non-response bias if the actual response rate is less than 100 percent. Specifically, respondents and nonrespondents will be compared on available background information (e.g., types of grantees, involvement of partners, and grade levels served). We will report any significant differences in characteristics between respondents and nonrespondents based on available information and discuss their implications for the interpretation of study findings in the Year 1 brief and the final report.

4. Tests of Procedures

The data collection instruments were developed, in part, based on a review of similar questionnaires used previously by ED and vetted by experts advising the evaluation.

Usability of the website for the surveys will be examined by AIR’s subcontractor Firepig Partners who specializes in on-line survey design and administration. Based on their recommendations, revisions to the design of the online surveys will be made.

5. Contacts for Statistical Aspects and Data Collection

The following individuals were involved in the design and statistical aspects of the study and its data collection:

• Stéphane Baldi, Principal Research Scientist, American Institutes for Research

• Mengli Song, Senior Research Scientist, American Institutes for Research

• Tanya Taylor, Research Analyst, American Institutes for Research

• Yibing Li, Research Analyst, American Institutes for Research

Notice of Trademark: “AMERICAN INSTITUTES FOR RESEARCH” and “AIR” are registered trademarks. All other brand, product, or company names are trademarks or registered trademarks of their respective owners.

1000 Thomas Jefferson Street, NW | Washington, DC 20007‑3835