supp_statement_b - revised

Supporting Statement Outline – Boren Scholarship and Fellowship Survey

NOTE: Complete Part B for Survey ICR Requests

SUPPORTING STATEMENT – PART B

B.  COLLECTIONS OF INFORMATION EMPLOYING STATISTICAL METHODS

If the collection of information employs statistical methods, it should be indicated in Item 17 of OMB Form 83-I, and the following information should be provided in this Supporting Statement:

1.  Description of the Activity

Describe the potential respondent universe and any sampling or other method used to select respondents.  Data on the number of entities covered in the collection should be provided in tabular form for the universe as a whole and for each of the strata in the proposed sample.  Indicate the expected response rates for the collection as a whole, as well as the actual response rates achieved during the last collection, if previously conducted.

The potential respondent universe includes all Boren scholars and fellows who have completed their service requirement which is approximately 2,000 people. DLNSEO maintains a database of all Boren Scholarship and Fellowship recipients and has information on the approximately 2,000 alumni at the time they completed their service requirement. Of those 2,000 alumni, the DLNSEO database has email addresses for approximately 1,800 people. Because of our concern that some of these email addresses may be out of date, we requested email addresses from the Boren Forum, the alumni organization associated with the Boren Program. With the list of email addresses provided to us by the Boren Forum, we were able to update over 700 email addresses in the DLNSEO database with more up-to-date email information.

We are planning a one-time survey, which will collect data from Boren scholars and fellows about their program participation and their subsequent career trajectories. Once the data are collected, we will aggregate and analyze responses based on the time period the award was given, the length of time the student was funded to study abroad, and whether the recipient was a scholar or a fellow. We expect responses to vary within and between these categories.

We are particularly interested in the time period of the award for a variety of reasons. First, the Boren program has evolved over the years including the level of support that NSEP/DLNSEO has offered to students in terms of mentorship and assistance finding jobs upon graduation. Further, the alumni organization known as the Boren Forum which hosts career fairs and provides other networking opportunities didn’t exist in the earlier years of the program. These differences could contribute to different perspectives that may be reflected in respondents’ opinions regarding the program and the impact it has had on their careers.

We are attempting a census. No sample is being selected. We estimate our response rate to be about 40 percent based on our efforts to obtain the most accurate and up-to-date contact information as well as a high level of alumni participation in the Boren Forum. It is possible that more recent graduates may respond to our survey at a higher rate; but we have no previous data collection using only this population to serve as a comparison.

Table 1. Potential subjects: Number of Boren recipients who have completed their service requirement

	Boren scholars and fellows with completed service requirement
Approximate total number	2,000
Approximate number with email addresses	1,800
Estimated number of responses	720

2.  Procedures for the Collection of Information

Describe any of the following if they are used in the collection of information:

1. Statistical methodologies for stratification and sample selection;

Not applicable

2. Estimation procedures

We will conduct chi-square tests of goodness of fit to assess whether the demographic distributions in our sample are representative of the population. A decision to weight the results to population proportions on key demographics will be based on the results of these tests. The key demographics that will be considered are the time period the award was given, the length of time the student was funded to study abroad, and whether the recipient was a scholar or a fellow.

3. Degree of accuracy needed for the Purpose discussed in the justification;

Assuming a 40 percent response rate, the allowable sampling margin of error is 4 percent.

4. Unusual problems requiring specialized sampling procedures; and

Not applicable

5. Use of periodic or cyclical data collections to reduce respondent burden.

Not applicable

3.  Maximization of Response Rates, Non-response, and Reliability

Discuss methods used to maximize response rates and to deal with instances of non-response.  Describe any techniques used to ensure the accuracy and reliability of responses is adequate for intended purposes.  Additionally, if the collection is based on sampling, ensure that the data can be generalized to the universe under study.  If not, provide special justification.

As noted above, we are surveying the entire population of Boren recipients who have completed their service requirement who we have contact information for. To maximize response rates, we first updated the DLNSEO Boren Awardee database with membership contact information from the Boren Forum. The Boren Forum information should provide us with the most up-to-date contact information possible for those Forum members who have completed their service requirement. Next, DLNSEO will be emailing the initial survey email and follow-up emails. We assume that most Boren awardees would view an email from a cna.org address as SPAM and not open it. Finally, DLNSEO will be sending a follow-up email reminding awardees to complete the survey. We have attached copies of the initial survey email and follow-up email.

We are attempting to conduct a census and the collection is not based on a random sample. Because responding to the survey is voluntary, we plan to assess the potential for selection and non-response biases using chi-square tests of goodness of fit to assess whether demographic distributions in our sample are representative of the population. If we learn that those who have stayed in contact with the program through the Boren Forum and more recent graduates are more likely to reply, we will weight the results to population proportions on key demographics based on the results of these tests. The key demographics that will be considered are whether the recipient was a scholar or a fellow, the year of the award, and the length of study time funded as we might expect different responses based on these characteristics. Additionally, because this study is retrospective in nature, and we do not have a comparison group, we do not plan to causally characterize the program’s effects. Rather, our goal is to make descriptive statements about career trajectories of program participants.

4.  Tests of Procedures

Describe any tests of procedures or methods to be undertaken.  Testing of potential respondents (9 or fewer) is encouraged as a means of refining proposed collections to reduce respondent burden, as well as to improve the collection instrument utility.  These tests check for internal consistency and the effectiveness of previous similar collection activities.

We have pre-tested the survey with Boren scholars or fellows who have completed their service requirement (fewer than 9). We received their feedback on how long it took to take the survey as well as their feedback on the survey questions and format.

5.  Statistical Consultation and Information Analysis

a.  Provide names and telephone number of individual(s) consulted on statistical aspects of the design.

Dr. Yevgeniya K. (Jane) Pinelis

Research Scientist

CNA

4825 Mark Center Drive

Alexandria, VA 2231

Email: [email protected]

Phone: 703-824-2052

Dr. Paul Rosenfeld

Survey Design, Analysis, & Operations Branch Chief

Human Resources Strategic Assessment Program (HRSAP)

Defense Manpower Data Center (DMDC)

4800 Mark Center Drive

Suite 04E25

Alexandria, VA 22350-4000

Office: 571.372.0987

[email protected]

b.  Provide name and organization of person(s) who will actually collect and analyze the collected information.

Jessica Wolfanger

Research Scientist

CNA

4825 Mark Center Drive

Alexandria, VA 2231

Email: [email protected]

Phone: 703-824-2842