Supporting Statement A

Supporting Statement A

National Survey of Children's Health: Research to Support the Development of Mail and Internet Questionnaires with Measurement Equivalence (NSCH Redesign Project)

OMB Control No. 0915-0379

Terms of Clearance: None

A. Justification

1. Circumstances Making the Collection of Information Necessary

The Maternal and Child Health Bureau (MCHB) of the Health Resources and Services Administration (HRSA) of the United States Department of Health and Human Services (DHHS), is redesigning the National Survey of Children’s Health (NSCH) and National Survey of Children with Special Health Care Needs (NS-CSHCN) into a single combined survey that will utilize an Address- Based Sampling (ABS) frame. Previously, these surveys utilized the State and Local Area Integrated Telephone Survey (SLAITS), a mechanism for conducting surveys that uses the same sampling frame as the National Immunization Survey. This frame consisted primarily of telephone landline numbers, with some cell telephone supplementation, and utilized Random Digit Dialing (RDD) sampling techniques. These surveys were then conducted through telephone interviews using Computer Assisted Telephone Interview (CATI) software to collect data from households.

The telephone interview approach utilized for these surveys allowed for a complex questionnaire as it ensured that skip patterns were properly followed. Furthermore, it protected against data entry error through preprogrammed range and logic checks on responses. Interviewers were able to address respondent questions and concerns as they arose, helping reduce response error. In recent years, declining willingness of the public to participate in surveys and changes in household telephone use has resulted in declining response rates in these and other surveys. Of particular concern is the increasing prevalence of households that have substituted their landline telephone for wireless service only. Efforts to include these non-landline households within the telephone sampling frames for the NSCH and NS-CSHCN have resulted in both an increase in costs and a substantial decline in response rates.

The decline in response rates and the increase in costs are no longer sustainable. Considerable work has already been done to determine how to address these concerns, and a decision has been reached to utilize a two-phase multimode data collection design for a combined survey, henceforth known as the NSCH. The NSCH will consist of two questionnaires: 1) an initial household screener to assess the presence of children in the home and facilitate the selection of a target child within the household (with oversampling of children with special health care needs), and 2) a substantive topical questionnaire that combines selected content from the past NSCH and NS-CSHCN questionnaires along with some newly relevant content. Mail is anticipated to be the primary mode of data collection, but web-based administration will also be used, probably in a “web first” design. This change in the mode of data collection, as well as revisions to the survey’s content, requires a redesign of both the initial household screener and topical questionnaires.

This submission requests approval for the following activities:

1. Cognitive interviews to assess how respondents interpret survey items and response options and what difficulties they may face in completing the questionnaires.

2. Usability testing to assess response times for items across data collection modes, respondent behaviors such as changing their initial answer, and if difficulties exist in the navigation or usage of web and paper-based instruments.

3. Validity testing that utilizes both test-retest and criterion-based methodologies in order to assess the validity of parent-report of key indicators such as medical diagnoses and insurance status.

4. Limited field testing of the data collection instruments to

   1. Evaluate the influence of data collection mode and survey length on response rates.

   2. Assess tendency of estimates from the mail and internet-administered surveys to differ from estimates from telephone survey administration.

2. Purpose and Use of Information Collection

Data from the NSCH are used to measure progress on national performance and outcome measures under Title V MCH Services Block Grant. It is therefore critical that HRSA assess how changes in the survey’s content, sampling frame, and data collection mode will impact the reliability and validity of such estimates.

In the absence of these preliminary research activities, HRSA would pursue research activities without pretesting, which could lead to increased burden time for respondents, decreased data quality, and less efficient data collection procedures.

3. Use of Improved Information Technology and Burden Reduction

This data collection will assess the feasibility of a web-based version of the questionnaire, a data collection mode that allows for features that will reduce respondent burden. In general, respondents find it less taxing to provide sensitive information about their children in self-administered surveys; however, because of the significant number of filter questions, paper and pencil versions of the survey appear quite lengthy. The web-based survey allows for the programming of skip patterns similar to telephone interview version of the survey. Thus, the web-based format allows for the comfort of self-administration with the ease of seeing and subsequently answering only questions relevant to a particular respondent.

4. Efforts to Identify Duplication and Use of Similar Information

Efforts to identify published information on a comparable redesign of a national survey that produces state-level estimates on similar content areas were unsuccessful. It is unlikely that an entity external to HRSA would undergo research on the redesign of a survey overseen by the Maternal and Child Health Bureau.

5. Impact on Small Businesses or Other Small Entities

The data collection forms and procedures of this project will be designed to minimize burden on all respondents. It is anticipated that approximately 30 health care providers, some of whom may work for small businesses or other small entities, will be contacted to provide documentation. To minimize the burden experienced by these entities, the information being requested will be held to the absolute minimum required for the validation of parent-report.

6. Consequences of Collecting the Information Less Frequently

This data collection is a one-time activity. One-time activities cannot be conducted less frequently. The consequences of doing this activity less frequently would be not collecting these data at all.

7. Special Circumstances Relating to the Guidelines of 5 CFR 1320.5

This data collection will be consistent with the general information collection guidelines of 5 CFR 1320.5. No special circumstances apply.

8. Comments in Response to the Federal Register Notice/Outside Consultation

Section 8A:

As required be 5 CFR 1320.8(d), a 60-day Federal Register Notice was published in the Federal Register on December 11, 2013, vol. 78, No. 238; pp. 75353. No substantive comments were received.

Section 8B:

A convenience sample of eight (8) individuals completed the questionnaires to assess questionnaire administration time.  Two of these individuals were parents of children with special health care needs; six of the individuals were parents of non-special health care needs children.  The questionnaires were completed between 7/9/2014 and 7/14/2014.  Name and contact information for these individuals is being omitted to protect respondent confidentiality.

9. Explanation of any Payment/Gift to Respondents

Respondents will receive monetary remuneration for their participation in such a way that is congruent with other governmental surveys that use incentives and that have been approved by the Office of Management and Budget¹. A detailed overview of remuneration amounts by respondent activities and burden estimates is outlined in Table 9A.

Respondents who participate in cognitive interviews and validation tests will each receive compensation up to $50 cash. Compensation will be provided to these respondents to both help encourage their participation and to thank them for their time and effort. Particularly, compensation is motivated by the following reasons:

• This project requires substantial participation by respondents with specific characteristics (e.g. child’s age, special health care need status, diagnosed medical conditions). The more specific the characteristics, the more difficult it is to recruit eligible respondents.

• Cognitive interviews require an unusual level of mental effort, as respondents are asked to explain their mental processes as they hear the question, discuss its meaning and any ambiguities, and describe why they answered the questions the way they did.

• Compensation will help thank participants for their mental effort to answer questions and for any inconvenience related to traveling to the place where interviews will be conducted in the city.

• All participants who start the interview will be able to receive the incentive payment; those who are scheduled but do not show up will not be compensated.

Respondents who participate in the mode effects experiment will each receive monetary incentives up to $10 cash. The amount of the incentive is dependent on the mode condition the respondent is assigned to, and whether the respondent is eligible to complete the topical questionnaire. Those assigned to the mail mode will receive a $1 prepaid cash incentive with the mailed Screener packet; those who return a completed screener and who are eligible will receive a $2 prepaid cash incentive with the mailed Topical packet. For the web mode, potential respondents will receive a $2 prepaid cash incentive with the mailed invitation letter. Those assigned to the telephone mode will not receive any prepaid incentive; however, refusal cases will be offered a $10 promised cash incentive.

Remuneration will be provided to these respondents for the following reason:

• Survey research indicates that incentives are a necessary and cost-effective expense for achieving a response rate that minimizes non-response bias.² Due to a preponderance of such research, incentives were used in all previous administrations of the NSCH and NS-CSHCNs.

• Recent experimentation within a general population mixed mode (web and mail) survey, found that the use of a prepaid incentive nearly doubled the response rate within that population from 25% (no incentive) to 56% (with incentive) (Messer and Dillman 2011).

9A. Remuneration Amounts by Respondent Burden

Remuneration Amount	Number of Respondents	Burden per Respondent	Activities
$30	120	60 minutes	Cognitive Interview/Usability Testing Respondents
$1	2,250	4 minutes	Prepaid incentive for mail mode Screener respondents
$2	842	18-30 minutes*	Prepaid incentive for mail mode Topical respondents. Burden varies depending on questionnaire version received (Long vs. Short; 0-5 vs. 6-11 vs. 12-17)
$2	2,250	4-30 minutes^	Prepaid incentive for web mode respondents. Burden varies depending on whether respondent screens in for Topical and on Topical questionnaire version received (Long vs. Short; 0-5 vs. 6-11 vs. 12-17).
$10	80	4-30 minutes^	Paid incentive to phone refusal cases. Burden varies depending on whether respondent screens in for Topical and on Topical questionnaire version received (Long vs. Short; 0-5 vs. 6-11 vs. 12-17).

^* 0-5 Long = 30 minutes; 6-11 Long = 27 minutes; 12-17 Long = 27 minutes; 0-5 Short = 22 minutes; 6-11 Short = 18 minutes; 12-17 Short = 22 minutes.

^{^} Screener = 4 minutes; 0-5 Long = 30 minutes; 6-11 Long = 27 minutes; 12-17 Long = 27 minutes; 0-5 Short = 22 minutes; 6-11 Short = 18 minutes; 12-17 Short = 22 minutes.

10. Assurance of Confidentiality Provided to Respondents

Data will be kept private to the extent allowed by law. Individuals and organizations will be assured of the confidentiality of their replies under Section 934(c) of the Public Health Service Act, 42 USC 299c-3(c). They will be told the purposes for which the information is collected and that, in accordance with this statute, any identifiable information about them will not be used or disclosed for any other purpose.

11. Justification for Sensitive Questions

Sensitive questions are generally not included on the NSCH. However, it is possible that respondents may find some questions related to their children’s health or disease status to be sensitive in nature. Respondent participation is entirely voluntary and individuals are free to refrain from answering any question that they do not feel comfortable responding to. Items related to race and ethnicity are included; however, the U.S. Department of Health & Human Services (HHS) requires that race and ethnicity be collected on all HHS data collection instruments.

12. Estimates of Annualized Hour and Cost Burden

Estimates of annualized hour burden and annualized cost to respondents are laid out in Tables 12A and 12B, respectively. The total number of estimated respondents is 9,486 annually. The total number of annual burden hours is 1,595. The estimated total annual respondent cost is $16,734.30.

The number of estimated respondents varies by data collection activity. A total of 120 respondents will be recruited to complete the Qualitative Assessment (Cognitive Interview/Usability Testing). Of these respondents, half (n=60) will be assigned to a test-retest validation condition whereby they will be recontacted two weeks subsequent to the Qualitative Assessment interview and re-administered a sub-set of items from the original interview. The other half of respondents will be assigned to a criterion-based validation testing condition, and will be asked to bring in documentation (e.g., prescription bottles; health insurance cards) to allow for validation of responses for a subset of items (conditions questions; health insurance status questions). If respondents are unwilling or unable to provide the requested documentation, the study will request permission to contact the child’s health care provider to validate these items. It is estimated that approximately 30 cases will require follow-up with a health care provider.

For the Mode Effects Experiment, 6,750 respondents are expected to complete the screener; 421 respondents per age group/questionnaire length condition are expected to complete the Topical questionnaire.

The frequency of response is the same across data collection activities – each instrument requires one response per respondent.

The average burden per response was determined by timing instruments administration with 9 or fewer respondents.

Estimates of the total annual respondent cost for the collection of information use the appropriate wage rate categories. For individuals, the wage rate is $10.00 per hour. This is based on the average hourly earnings for employees as reported by the Bureau of Labor Statistics (http://www.bls.gov/news.release/realer.t01.htm).

For health care providers, the wage rate is $88.43 per hour. This was calculated based on the mean hourly wage for Family and General Practitioners as reported by the Bureau of Labor Statistics (http://www.bls.gov/oes/current/oes291062.htm).

12A. Estimated Annualized Burden Hours

Type of Respondent	Form Name	No. of Respondents	No. Responses per Respondent	Average Burden per Response (in hours)	Total Burden Hours
Qualitative Assessment (Cognitive Interviews & Usability Testing)
Adult Parents	Qualitative Assessment Instrument	120	1	60/60	120
Adult Parents	Test/Retest Instrument	60	1	20/60	20
Health Care Providers	Provider Validation Instrument	30	1	20/60	10
Mode Effects Experiment
Adult Parents	Screener	6750	1	4/60	450
Adult Parents	0-5 Long Instrument	421	1	30/60	211
Adult Parents	6-11 Long Instrument	421	1	27/60	189
Adult Parents	12-17 Long Instrument	421	1	27/60	189
Adult Parents	0-5 Short Instrument	421	1	22/60	154
Adult Parents	6-11 Short Instrument	421	1	18/60	126
Adult Parents	12-17 Short Instrument	421	1	18/60	126
Total		9,486			1,595

12B. Estimated Annualized Burden Costs

Type of Respondent	Total Burden Hours	Hourly Wage Rate	Total Respondent Costs
Qualitative Assessment (Cognitive Interviews & Usability Testing)
Adult Parents	120	$10.00	$1,200.00
Adult Parents	20	$10.00	$200.00
Health Care Providers	10	$88.43	$884.30
Mode Effects Experiment
Adult Parents	450	$10.00	$4,500.00
Adult Parents	211	$10.00	$2,110.00
Adult Parents	189	$10.00	$1,890.00
Adult Parents	189	$10.00	$1,890.00
Adult Parents	154	$10.00	$1,540.00
Adult Parents	126	$10.00	$1,260.00
Adult Parents	126	$10.00	$1,260.00
Total	1,595		$16,734.30

13. Estimates of other Total Annual Cost Burden to Respondents

There are no direct costs to respondents other than their time to participate in the study.

14. Annualized Cost to the Federal Government

This data collection will be carried out under a contract awarded to NORC in the amount of $562,444. This contract spans an eighteen month project period, and represents an annual cost of $374,962.

Additionally, the cost to the government consists of the salaries of the HRSA staff who (1) determine the content of the data collection instruments, (2) oversee the scope of work conducted under the aforementioned contract, and (3) assist in the analysis of the results and recommend changes in questionnaire wording:

Managerial (G15-10) 0.25 FTE $32,703¹

Professional (G15-4) 0.75 FTE $83,015¹

¹Based on 2014 OPM Salary Table (http://www.opm.gov/policy-data-oversight/pay-leave/salaries-wages/salary-tables/pdf/2014/saltbl.pdf).

Annual Total (contracts and staff) $490,680

15. Explanation for Program Changes or Adjustments

This is a new information collection.

16. Plans for Tabulation, Publication, and Project Time Schedule

Below is NORC’s schedule for reports, tabulation, and data delivery with descriptions of each process. In addition, in combination with authors from MCHB, NORC will submit a series of scientific papers for presentation at national and regional conferences and ultimately submit these works to peer-reviewed journals.

Reports, Tabulation, and Schedule.

Qualitative Assessment Report	Wed 10/15/14	Report summarizing findings from the Cognitive Interviews, Usability Testing, and Validation Testing.
Quantitative Data Files	Mon 2/16/15	Will include screener and topical interview data files containing a record for each completed screener and main interview along with variables containing the responses to all questionnaire items, a variable identifying the mode of completion for each case, and variables identifying any experimental treatment group to which the case was assigned.
Questionnaire Modifications Report	Fri 2/27/15	Based on results of Phase I and II, and in consultation with MCHB, NORC will modify the questionnaires and produce a report detailing how and why changes were made.
Estimates Report	Fri 2/27/15	NORC will provide a report describing the potential for estimates from the mail and internet-administered survey to differ from prior year estimates from telephone administration.
Questionnaire Length Experiment Report	Fri 2/27/15	Report detailing results from experiment designed to test difference in response rates between long and short versions of the questionnaire.

17. Reason(s) Display of OMB Expiration Date is Inappropriate

Not applicable. Not requesting exemption.

18. Exceptions to Certification for Paperwork Reduction Act Submissions

Not applicable. No exception requested.

1 E.g., OMB No. 0920-0805 (Racial and Ethnic Approaches to Community Health across the U.S. (REACH US) Evaluation); OMB No. OMB No. 0920-0406 (National Survey of Children with Special Health Care Needs).

2 See Brick, J. Michael, Douglas Williams, and Jill M. Montaquila. 2011. Address-based sampling for subpopulation surveys. Public Opinion Quarterly 75 (3): 409-28; Foster, Erin B., Alicia M. Frasier, Heather M. Morrison, Kathleen S. O’Connor, and S.J. Blumberg. 2010. All things incentive: Exploring the best combination of incentive conditions. Paper presented at the American Association for Public Opinion Research annual conference, Chicago, IL.

5