Supporting Statement

OMB Approval No. 1225-0059

CUSTOMER SATISFACTION SURVEY AND CONFERENCE EVALUATION CLEARANCE FORM

Division of Energy Employees Occupational Illness Compensation (DEEOIC) Customer Satisfaction Survey Supporting Statement

A. Justification

A.1 & A.2 Circumstances Necessitating Data Collection & How, By Whom, And For What Purpose the Information Is To Be Used

The Energy Employees Occupational Illness Compensation Program (EEOICP) began on July 31, 2001 with the Department of Labor’s implementation of Part B; Part E implementation began on October 28, 2004. The mission of the program is to provide lump-sum compensation and health benefits to eligible Department of Energy nuclear weapons workers (including employees, former employees, contractors and subcontractors) and lump-sum compensation to certain survivors if the worker is deceased. Since the program began in 2001, over 175,000 claims have been filed, representing the cases of 71,000 workers. Over $4.7 billion has been paid on 37,000 cases.

On September 11, 1993, President Clinton issued Executive Order 12862, “Setting Customer Service Standards” which clearly defined his vision that the Federal agencies will put the people first. To do that, the President called for a “revolution within the Federal government to change the way it does business.” He expected this process to require continual reform of government practices and operations with continual reform of government practices and operations with the result that, when dealing with the Federal agencies, all people would receive services that match or exceed the best service available in the private sector.

Section 1(b) of that Order required agencies to “survey customers to determine the kind and quality of services they want and their level of satisfaction with existing services” and Section 1 (e) requires agencies to “survey front-line employees on barriers to, and ideas for, matching the best in business.” These Presidential requirements established a need for the Division of Energy Employees Occupational Illness Compensation (DEEOIC) at the Department of Labor (DOL) to be engaged in an interactive process of collecting information and using it to improve program services and processes. Agencies are therefore authorized to measure customer satisfaction and report results. In addition, the March 22, 1995 Presidential Memo “Improving Customer Service,” states that customer views should be obtained to determine whether standards have been met on those matters which most concern the customer.

This evaluation of the Energy Employees Occupational Illness Compensation Program (EEOICP) will assess satisfaction among Department of Energy employees and/or their representatives who have filed a compensation claim and gone through the EEOICP claims process. This evaluation will assess the following:

• Responsiveness to inquiries

• Courtesy of staff

• Explanation of the claims and review processes

• Clarity of correspondence

• Ability to treat personal information privately and confidentially

• Explanation of claim decisions

• Desire for additional services

A.3 Use of Technology to Reduce Burden

Collecting data solely by Internet survey is not feasible, because email address information is not regularly collected from claimants. Supplementing a mail survey with an Internet-based questionnaire to collect data was considered, but DEEOIC decided against it because of concerns about additional costs given the limited benefits it was likely to provide. The target population’s receptiveness to online surveys is anticipated to be low. Almost seven out of ten (69.6%) EEOICPA claimants are age 60 or older. Internet usage among people age 64-72 in the general U.S. population is 56%. Among those older than 72, it is only 31%.¹ In addition to the specific problems posed for conducting an Internet survey with DEEOIC’s customers, it is unclear that adding a web option actually improves a survey’s response rate.² For a study of this size, the cost savings per unit from replacing surveys returned by mail with surveys returned online are likely to be smaller than the fixed costs from setting up and hosting an online survey.

A.4 Efforts to Identify Duplication

The information to be supplied on these surveys will not be duplicated on any other information collection. No other surveys of customer satisfaction at DEEOIC have been conducted.

A.5 Methods to Minimize Burden on Small Businesses

Not applicable. The questions are being asked only of individual customers of claimants.

A.6 Consequences of Less Frequent Data Collection

DEEOIC’s ongoing program improvement plans involve obtaining an outside assessment of the program’s performance. Along with a review by OIG, this study is an essential component of evaluating whether the program is effective and achieving its results. Failure to conduct this survey would deprive DEEOIC of an important tool in making such an evaluation and of identifying areas for improvement.

A.7 Special Circumstances For Data Collection

The collection of information in the proposed study is consistent with all applicable guidelines contained in 5 C.F.R. § 1320.5(d)(2).

A.8 Consultation outside the Agency

The survey was developed within DEEOIC and involved consultation with Synovate, the survey contractor, regarding formatting and the specific wording of questions. In addition, DEEOIC consulted the Bureau of Labor Statistics (BLS) on the statistical methods employed by this survey.

A.9 Payment of Gifts to Respondents

No payment or gift will be provided to respondents in this study.

A.10 Confidentiality Assurances

In addition to identifying on the cover letter that participation is strictly voluntary and that answers will remain anonymous, the following statement is also provided within the cover letter, in accordance with OMB recommended guidelines:

“Privacy: Responses to this data collection will be used only for statistical purposes. The reports prepared for this study will summarize findings across the sample and will not associate responses with and individual. We will not provide information that identifies you to anyone outside the study team, except as required by law.”

A.11 Additional Justification for Sensitive Questions

This data collection will not involve sensitive questions.

A.12 Estimates of the Burden of Data Collection

The respondent burden for this data collection is estimated to be under 10 minutes. During the pre-test, it was assessed that taking the survey to be approximately 5 minutes, and we have added an additional 5 minutes to this estimate in order to accommodate additional written comments that respondents may provide. With an estimated 1000 respondents completing the survey, that suggests 167 hours of respondent time. Finally, the cost per respondent should be negligible. Participation is voluntary and will not require start‑up, capital, or labor expenditures by respondents.

A.13 Estimated Cost to Respondents

The cost to respondents should be negligible. Participation is voluntary and will not require start‑up, capital, or labor expenditures by respondents. Postage for returning the surveys will be pre-paid.

A.14 Estimates of Annualized Costs to Federal Government

The cost to the government includes the firm-fixed price contract awarded to Synovate for $50,070.27.

A.15 Changes in Burden

This is a new data collection.

A.16 Publication of Results

Results will primarily be for internal use and process evaluation. Results will be shared with the office of the secretary of labor and may be published if deemed appropriate.

A.17 Approval Not to Display OMB Expiration Date

Not Applicable.

A.18 Exceptions to OMB Form 83i

Not Applicable.

B. Collection of Information Employing Statistical Methods

B.1. Description of Universe and Selection Methods Used

The Energy Employees Occupational Illness Compensation Program Act (EEOICPA) provides compensation and medical benefits to employees who became ill as a result of working in the atomic weapons industry. The EEOICPA also offers benefits to their survivors. Workers who developed certain illnesses as a result of work performed in the production and testing of nuclear weapons while they were employees of the Department of Energy (DOE), its predecessor agencies, or its contractors and subcontractors, are eligible for benefits. Employees of DOE designated Atomic Weapons Employers (AWE) and beryllium vendors are also eligible for compensation.

There are two different benefit programs— Part B and Part E. In some cases, employees, or their survivors, are eligible for compensation from both programs. Part B covers current or former workers who have been diagnosed with cancers, beryllium diseases, or silicosis, whose illness(es) was caused by exposure to radiation, beryllium or silica while working directly for DOE, DOE contractors or subcontractors, a designated AWE or beryllium vendor. Under Part B, silicosis is only covered for employees who worked during mining of atomic weapon test tunnels in Nevada or Alaska. Part E provides coverage to DOE contractor and subcontractor employees who developed any illness, including cancer, beryllium disease, and silicosis, as a result of occupational exposure to any toxic substances at a covered DOE facility. Section 5 uranium miners, millers, and ore transporters (or their eligible survivors), and certain Section 4 RECA individuals may be eligible for benefits under the EEOICPA under both Part B and Part E.

As of July 22, 2009, there have been over 86,000 claimants for EEOICPA benefits whose cases have been resolved since the program’s inception in July 2001, and it is that group that DEEOIC intends to survey. The population for this examination of customer satisfaction is being limited to claimants whose cases have been completed and for whom a final determination is available. Claimants without a determination may be at a number of different stages in the claims process, so examining their perceptions and attitudes in aggregate may not be optimal, but segmenting them based upon their current stage in the claims process may be imprecise, since further action on their claim may have taken place between sample selection and the arrival of the survey. Claimants who recently completed the claims process are likely to have similar experiences at any individual stage of the process to those who have not yet had a final determination; little is likely to be lost from excluding this latter group.

The DEEOIC claimant universe is divided almost equally between those who have been awarded benefits (50.4%) and those who have been denied benefits (49.6%). Most of these decisions have been made in the last two years, with 22.1% from the past year and 45.7% of decisions having been made between one and two years ago. Approximately one in five decisions (19.3%) was made four or more years ago.

A majority of claimants (62.5%) filed for Survivor benefits, while 37.5% of claimants are former employees of DEO or DEO contractors or subcontractors. Most claimants filed for benefits under both Part B and Part E (72.2%), while 22.2% filed for Part B benefits alone and 6.2% filed for Part E benefits alone.

Four regional offices and one national office adjudicate final decisions. Almost a third (32.5%) of decisions are adjudicated at the national office, while Cleveland (16.4%), Denver (17.5%), Jacksonville (21.3%), and Seattle (12.1%) make determinations in all other cases. As the following section (B.2.1) will explain, the sample will be stratified so that it is divided equally among these Final Adjudication Branches in order to facilitate a comparative evaluation of their services and identify specific areas for improvement at each. Within each branch, we intend to select a random sample (see. B.2.a below).

B.2. Describe the procedures for the collection of information including:

• Statistical methodology for stratification and sample selection,

• Estimation procedure,

• Degree of accuracy needed for the purpose described in the justification,

• Unusual problems requiring specialized sampling procedures, and

• Any use of periodic (less frequently than annual) data collection cycles to reduce burden.

DEEOIC has determined that a mail survey is the optimal means of assessing claimants’ satisfaction and has commissioned Synovate to conduct the survey of 1000 claimants. The survey instrument was developed to assess satisfaction with claimants’ different interactions with the division and their recommendations for improvements. The questionnaire will be mailed to claimants in a survey packet that also includes a cover letter and a return postage-paid envelope. The cover letter will make clear that this survey is from DEEOIC and will be used confidentially to help identify strategies to improve customer service throughout the country. Participants will be asked to return a completed survey within two weeks of receipt, via the pre-paid envelope provided. A reminder post card will be sent to each participant approximately two weeks after sending the surveys. The returned surveys will be processed and scanned, and the data captured through Synovate’s automated Optical Mark Recognition (OMR) software.

Synovate will provide DEEOIC with tabulated data, coded verbatim responses to open-ended questions, and a report containing analysis and graphical representations of the data. The report will identify statistically significant differences between key subgroups in the population.

B.2.a. Statistical Methodology for Stratification and Sample Selection

The goal of the Division of Energy Employees Occupational Illness Compensation’s Customer Service Satisfaction Survey (CSSS) is to provide feedback on the application process for current or former employees who applied for benefits under the Energy Employees Occupational Compensation Program Act. The survey is to cover five separate locations. The survey is intended to yield a sample size of 1,000 applicants.

The preliminary number of applicants across the five FABs is given the following table. Along with providing comparisons among FABs, an additional goal is to maximize statistical power for comparisons between the Awarded and Denied samples. These requirements indicate the need to have an equal number of respondents from applicants that were awarded benefits and from applicants that were denied benefits across locations. The following table summarizes the sample design. The expected yield rates – the number of useable returns divided by the number mailed out -- is assumed to be 32.6%. The expected mail out sizes are given for each stratum. The final population sizes will be determined once the sample frame is processed, and the actual sample sizes will be counted once the survey is completed.

Location	Target Sample Size	Mail Out Size	Expected Yield Rate	Interim Population Size	Sample Size
FAC – Cleveland FAB	200	613-614	32.6%	14255	n1
FAD – Denver FAB	200	613-614	32.6%	15180	n2
FAJ – Jacksonville FAB	200	613-614	32.6%	18476	n3
FAS – Seattle FAB	200	613-614	32.6%	10505	n4
FAB – National FAB	200	613-614	32.6%	28248	n5

An underlying assumption is that claim type, adjudication status, current age of decision, part type and current age of claimant are each completely randomly distributed across the locations. This may not be the case. If it is not the case, then any of these may not have a distribution that properly mirrors the population. Synovate recommends providing a larger sample than necessary and check the outbound samples for claim type, adjudication status, current age of decision, part type and current age of claimant. If the outbound sample is appropriate, then a random subset will be mailed. If the samples for claim type, adjudication status, current age of decision, part type and current age of claimant are irregular, then Synovate will further stratify the sample and apply a second-stage sampling to meet sample targets.

 The steps for selecting the sample are as follows:

• Process the sample frame by assigning each applicant in one and only one of the 18 strata.

• Assign a uniform pseudo-random number to each element of the sample frame.

• Count the number of applicants in the sample frame within each stratum.

• Sort and rank order the applicants on the sample frame from lowest to highest uniform pseudo-random number within each stratum.

• Randomly round the mail out size to an integer for the final mail out size

• Select the number the ranked cases less than or equal to the final mail out size within a strata.

The selected cases are a simple random sample within the stratum.

B.2.b. Estimation Procedure

Estimation without adjustment for non-response will be based on standard estimation procedures for a stratified design. The final population counts and sample sizes may not be exactly equal across all strata, and the estimation procedure accounts for this. Sample design weights will be calculated using the following formula:



for respondent i in the location h and awarded/denied category k. N_h,k and n_h,k are the population size and sample size cited in previous table for location h and awarded/denied category k. These are derived from the standard formula for estimation from a stratified sample where the estimator is

.

is the score for stratum h, k. Its variance primary estimator is

.

The estimator using the weights w_i is of the form

,

where x_i is the response value for respondent i.

This form is more flexible for estimation when there is an analysis of item non-response or of sub-domains.

An analysis of the responding sample versus the sample frame will be conducted to examine differences in response propensities among groups of individuals. There are 6 different population characteristics to examine. They are location, claim type (employee versus survivor), Adjudication status (Accept versus Deny), Age of Decision (less than 1 year, between 1 year and under 2 years, between 2 years and under 3 years, between 3 years and under 4 years, between 4 years and under 5 years, between 5 years and under 6 year, and 6 or more years), Age of claimant (Under 18, 18-29, 30-39, 40-49, 50-59, 60-69, 70-79, 80-89 and over 89 years), and part type (Both B and E case, B only case and C only case). The analytical plan involves three steps:

1. Identify variables with greatest variations in response rate for each of these variables using cross-tabulations and chi-square testing. The crosstabulations will begin with an examination of differences between population subgroups within one variable. The classification factors that will be examined are listed in the paragraph above. If multiple variables seem to affect response rate, or if one variable shows significant differences, we may use crosstabs to explore second order effects on response rate (e.g. examining age subgroups within Adjudication status). The following stage of the analysis, using a Classification and Regression Tree (CART) is more ideal for examining smaller subgroups, we may therefore limit the crosstabs to a broader summary of response rate differences. We will employ the Pearson chi-square test for two way tables (see immediately below) to examine whether differences between subgroups are statistically significant:

Where

.

This test assumes row and column variables are independent and there will be (R-1)*(C-1) degrees of freedom.

2. Use automatic interaction detection and tree analysis to identify groupings with the largest differences in response rates within each stratum. Synovate will use CART to build classification trees for predicting response/ non-response for unit non-response and identified variables for item non-response. The segmentation analysis constructs a tree with split nodes. The algorithm splits on the variables that are used to predict membership in the respondent/ non-respondent categories. The split at each branch or node is that which generates the greatest improvement in predictive accuracy as measured by the Gini index of node impurity. The GINI index measures the relative homogeneity for the cases in the terminal nodes. If all cases in each node show identical values, then node impurity is minimal, homogeneity is maximal, and prediction is perfect. The Gini index of node impurity is the measure most commonly chosen for classification-type problems. It reaches zero when only one class is present at a node. The Gini Impurity measure for node t is calculated by

.

The change in the Impurity Measure for split rule s is captured by

,

where is the Impurity measure for the left branch node, is the fraction of the node going to left branch, and the subscript R denotes the right branch. The split rule selected is the maximum over all possible splits.

The algorithm will continue until an appropriate number of nodes is achieved. The basic rule is the Minimum n Rule. This allows splitting to continue until all terminal nodes are pure or contain no more than a specified minimum number of cases within a node. We typically use a minimum n of 20.

3. Confirm differences and interactions using logistic regression. Synovate will conduct a logistic regression to serve as check on the CART analysis by providing confirmation regarding the key drivers of unit non-response. The regression will use the explanatory sample variables explored in the other tests above and will be in the form:

with response probability

Where

• h = 1, 2, ... , H is the stratum number with a total of H strata

• j = 1, 2, ... , is the unit number of stratum h, with a total of units

• w_hj denotes the sampling weight

• x_hj denotes the k-dimensional row vector of explanatory variables for the jth member in stratum h. If there is an intercept, then .

•   is the total sample size

Using Taylor approximation, the estimated covariance matrix of is

where

And matrices of partial derivatives .

These analyses will determine groupings with relatively uniform response propensities. The goal will be to identify mutually exclusive weighting adjustment cells where the sample units have approximately equal response propensities. The weights are then calculated for the cells within each stratum. This is a post-stratification strategy that keeps the overall response probabilities constant within cell. The use of more cells than less reduces the potential for bias. The analysis and modeling results will identify cells that can be combined to keep response propensities constant within cells.

The estimator under post-stratification is similar to the stratified estimator. In this case the number of strata is expanded to include the weighting cells from the analysis. The estimator is

,

where is based on the responding units. This implicitly accounts for the response propensities across the weighting adjustment cells. Weights accounting for nonresponse and the response propensities captured in the adjustment cells are accounted for in the following weight formula:

where is the population cases within the weighting cell within stratum, is number selected in the design and is the number cases that responded in the cell. This accounts for the design, the post-stratification allocation, and the response propensity with cell.

Using these weights under this estimation scheme, the estimator is now

.

where

Sample Variance

The Sample Variance for the sample mean is as given before

.

This can be applied to the post-stratification variable, but it does not account for the additional variability due to the post-stratification. The recommended approach is to use Taylor-series approximation where the variance uses the formula:

The SAS procedure SURVEYMEANS will be used to calculate variances, sampling errors, statistical tests and confidence intervals. It uses the Taylor expansion method to estimate sampling errors of estimators. For our stratified design, the procedure pools stratum variance estimates to compute the overall variance estimate. For t tests of the estimates, the degrees of freedom equals the number of sampling units minus the number of strata in the sample design.

B.2.c. Degree of Accuracy Needed for Purpose Described in the Justification

The sample design will provide a margin of error of about ± 3.1% at the 95% level of confidence for all applicants. This accounts for the sample design and scores at about 70%. The margin of error for estimates for the applicants with awarded benefits will be about ± 4.0% at the 95% level of confidence and scores around 70%, and the same ± 4.0% for applicants with denied benefits. The design will require a difference of 6.2% for comparisons between those awarded and those denied benefits for scores around 50%, a difference of 5.7% for scores around 70% and 3.7% for scores around 90%. These are for tests with using a 0.05 test criteria, 95% level of confidence.

The degree of accuracy is sufficient to meet the goals of the survey.

B.2.d Unusual Problems Requiring Specialized Sampling Procedures

There are no unusual problems requiring specialized sampling procedures.

B.3. Describe methods to maximize response rates and to deal with issues of non‑response. The accuracy and reliability of information collected must be shown to be adequate for intended uses. For collections based on sampling, a special justification must be provided for any collection that will not yield "reliable" data that can be generalized to the universe studied.

Response Rates

Because no prior survey of the DEEOIC claimant population has been conducted, historical data could not be used to estimate a response rate. DEEOIC asked Synovate for guidance in estimating a response rate for this population, given Synovate’s experience conducting mail surveys of Social Security disability applicants. Like DEEOIC claimants, the Social Security Administration (SSA) disability applicants have gone through an adjudication process to determine whether or not they were eligible to receive support to replace income lost through impairment. The broad similarities of the claimant populations, in terms of age and disability, and the overall resemblance of the claims adjudication processes through which respondents pursue benefits, provides at least a modicum of guidance regarding response rate. Synovate’s assistance was drawn mainly from experience conducting SSA’s Hearings Process Report Card survey. This survey of disability applicants was conducted in English and Spanish using an initial survey packet, a reminder postcard, and a follow-up survey packet. An equal number of surveys were mailed to applicants awarded and denied benefits. This study yielded a raw response rate of 47.8%.³ Synovate’s experience was that 62-65% of returns were returned in the initial mailing. We have made a general assumption that a small percentage of the respondents who returned surveys during the second mailing would have done so from the first mailing and based upon these experiences, Synovate arrived at an overall response rate of 32.6%. DEEOIC has chosen to use a one page questionnaire and to send a follow-up reminder postcard in an effort to increase the response rate over a standard, one contact survey.

Unit Non-response Analysis

Upon completion of the survey, Synovate will conduct a systematic analysis of the demographic traits of the respondent and non-respondent population based upon the sample frame. This analysis will have two goals:

1. Identify groups with a low propensity to respond so that DEEOIC can understand its population better and adjust for low response rates during sampling in any future survey projects.

2. Adjust the survey data to account for response rate differences by applying post-stratification weights that incorporate different response propensities.

The analysis will be done in the three steps: introductory crosstabulations to identify overall response behaviors in the population, a classification and regression tree (CART) to identify particular subgroups with low response propensity, and logistic regression to confirm the observations of the CART analysis. The details of each test, including the sample variables to be examined in crosstabs; the significance tests that will be employed; the procedure, controls, and tests that will be used in CART; and the equation for the logistic regression, are all included in section B.2.b. To meet goal 1, we will provide the tables of response rate for each of the population subgroups examined in the crosstabulations, the graphical display of the CART results (i.e. the tree showing how the population divides into groups based upon response rate, and a summary of logistic regression’s findings. The equations to meet goal 2 by applying the weights to the survey data to adjust for differing response propensities (and the resulting equations for the estimator and sample variance) are provided in section B.2.b.

Item Non-Response Imputation

Question-level missing data limits multivariate analysis. Most of the questions in this survey should have very small amounts of item non-response for qualified respondents. Synovate will conduct an item response analysis identifying completion rates of all questionnaire items, and the item completion rates for the key demographic groups -- location, claim type (employee versus survivor), Adjudication status (Accept/Award versus Deny), Age of Decision (less than 1 year, between 1 year and under 2 years, between 2 years and under 3 years, between 3 years and under 4 years, between 4 years and under 5 years, between 5 years and under 6 year, and 6 or more years), Age of claimant (Under 18, 18-29, 30-39, 40-49, 50-59, 60-69, 70-79, 80-89 and over 89 years), and part type (Both B and E case, B only case and E only case). An in-depth examination will be conducted for unusual completion rates under 70%, and when there are large differences in completion rates by demographic group. At this time, we anticipate only examining first-order effects (differences within one demographic category at a time). If we determine that item non-response appears to correlate with subgroups of multiple demographic categories, we will explore second-order effects (e.g. examining item response rates for Adjudication status within each location, as in the cells below) and possibly third order effects (e.g. examining item response rates within different age groups in each of the adjudication/location cells). We will employ the Pearson chi-square test for two way tables (see immediately below) to examine whether differences between subgroups are statistically significant:

Where

.

This test assumes row and column variables are independent and there will be (R-1)*(C-1) degrees of freedom.

If a low item response rate exists, statistical adjustments to the data will be conducted to impute values where they are missing. We anticipate using a nearest neighbor hot deck approach to statistical imputation. This approach identifies groups of respondents with similar attributes (based upon other responses or demographic characteristics from the sample data) and then chooses the most similar respondent (or a randomly selected respondent from among the most similar respondents), known as a donor, and applies that donor’s response for the missing value.⁴

The sample will be divided into 10 cells (based upon the sampling strata, see table below).

Cleveland – Award	Denver – Award	Jacksonville – Award	Seattle – Award	National – Award
Cleveland – Denial	Denver – Denial	Jacksonville – Denial	Seattle – Denial	National – Denial

We believe that these cells will have some predictive effect on responses, because respondents from similar final adjudication locations are likely to have similar experiences, and respondents with similar outcomes are likely to have similar views of the process (i.e., respondents whose claims are denied are likely to be more dissatisfied than those whose claims were approved). We will verify these assumptions using through the crosstabulations described above, making sure to examine the statistical differences among the 10 cells.

Once respondents have been assigned to a particular cell, we will screen that sample cell for prospective donors. In order to qualify as a prospective donor, records must meet the following criteria:

• All questions for which the respondent was qualified to answer were completed,

• The respondent has not used the same response (a practice known as straightlining) for all questions on Q6-Q8 (the questions employing the 5-point poor-excellent rating), and

• The respondent’s answer about employer or survivor benefits most match the information in the sample.

We will then identify the four records closest to the missing-data record in terms of another key variable. We expect this key variable to be length of time since decision. This information is available in continuous, rather than categorical form, making it easier to identify the closest neighbors and because respondents who filed claims at similar times are likely to have dealt with similar people, their experiences may have been similar. To test our hypothesis, we will conduct a logistic regression for to determine the primary drivers of the variable for which we are imputing a response. The regression will employ the same 6 demographic variables listed above. Categorical data will be included in the regression using a 1 or 0 to represent each value within a demographic variable. The regression will be in the form described for the unit non-response analysis in section B.2.b.

Assuming the regression indicates we should employ the length of time since decision category, we will add a small random number from 0 to .1 to all values in the category (given in number of days between the decision and the mail out date) to break ensure there will be no ties as to nearest neighbor. The four donor records that are nearest in length of time since decision in the appropriate cell will then be selected and numbered 1 through 4. A random number between 0 and 1 will then be selected. If the number is less than or equal to .25, the value from donor 1 will be selected and imputed for the missing data; if the random number is greater than .25 and less than or equal to .5, then the value from donor 2 will be imputed, etc.

Records will be eligible to serve as potential donors multiple times; this strategy could present a problem, because the same record could have an outsized effect, particularly if there are few qualified donors. We do not think this issue presents a major flaw for the plan, because the cell sizes should all be approximately 100 people, and, assuming 60-70 qualified potential donors, records should not be repeated too frequently.

B.4. Describe any tests of procedures or methods to be undertaken.

A pre-test was conducted by nine DEEOIC personnel who were selected due to their intimate familiarity with respondent issues. The subsequent comments provided by these pre-testers enabled several minor changes to the survey, and included realignment of questions for a better thought-flow process, adding choices to question responses, and providing additional space for free-flow comments. We have incorporated these changes into the questionnaire.

B.5. Provide the name, affiliation (company, agency, or organization) and telephone number of individuals consulted on statistical aspects of the design and the name of the agency unit, contractor(s), grantee(s), or other person(s) who will actually collect and/or analyze the information for the agency.

For further information on statistical methods, please contact:

Alan Roshwalb, Senior Vice President at Synovate (PhD in Statistics and Business Administration, University of Michigan, 1987) 703-663-7260

Zachary Lewis, Study Director at Synovate (MA in Social Sciences, The University of Chicago, 2004) 703-663-7235

Julia Batishev, Study Director at Synovate (MA in Statistics, University of Maryland, 2006) 703-663-7241

1 Sydney Jones & Susannah Fox, Pew Internet Project Data Memo: Generations 2009 1/28/2009 http://www.pewinternet.org/~/media//Files/Reports/2009/PIP_Generations_2009.pdf accessed 3/8/09

2 Jan Brøgger, Wenche Nystad, Inger Cappelen, & Per Bakke, “No Increase in Response Rate by Adding a Web Response Option to a Postal Population Survey: A Randomized Trial” Journal of Medical Internet Research 9:5 Oct_Dec 2007 http://www.pubmedcentral.nih.gov/articlerender.fcgi?artid=2270416, accessed 3/8/2009

3 The raw response rate is based upon Synovate’s internal totals for the study. The official response rate reported to SSA (excluding out-of-scope cases and undeliverable surveys) was 51.05%.

4 The nearest neighbor hot deck methodology employed here is similar to that used by the Bureau of Labor Statistics on its Job Openings and Labor Turnover Survey. Cf. “Comparing Impact of Alternative Approaches for Item Non-Response in the Job Openings and Labor Turnover Survey, Proceedings of the American Statistical Association, 2004. (http://www.amstat.org/sections/SRMS/proceedings/y2004/Files/Jsm2004-000936.pdf) Accessed December 17, 2009.