Part B - NSSRN 2018 Production OMB

Department of Commerce

United States Census Bureau

National Sample Survey of Registered Nurses: Supporting Statement B

OMB Control No. 0607-####

B. Description of Statistical Methodology

1. Statistical Design and Estimation

The National Sample Survey of Registered Nurses (NSSRN) is being conducted by the U.S. Census Bureau on behalf of the National Center for Health Workforce Analysis (NCHWA) in the Bureau of Health Workforce at the Health Resources and Services Administration (HRSA), U.S. Department of Health and Human Services (HHS). The objective of the survey is to sample and estimate the characteristics of the registered nurses and nurse practitioners in the nursing workforce. These data will provide the means for the evaluation and assessment of the evolving demographics, educational qualifications, and career employment patterns of RNs.

The NSSRN has gone through a significant redesign for the 2018 cycle. Decreasing response rates and budgets prompted the HRSA NCHWA to initiate a redesign of the NSSRN. This redesign included combining the original NSSRN with the National Sample Survey of Nurse Practitioners’ (NSSNP) into a single survey that is now known solely as the NSSRN. The NSSRN will use a probability based sample design derived from a RN universe of the U.S. resident population holding an active nursing license in one or more states, covering the 50 states and the District of Columbia. The NSSRN will be collected from March 2018 through July 2018. Registered Nurses will be sampled as described in section B.1.1. An invitation to participate in the NSSRN with login information for the online survey instrument will be sent to each sampled RN, and half the sample will also receive a paper instrument. Both modes are accompanied by Telephone Questionnaire Assistance (TQA). Contact methods are discussed further in section B.2.

1. Sampling Registered Nurses

The population of interest for the NSSRN is all registered nurses (RNs) and nurse practitioners (NPs) currently living in the U.S. with a valid nursing license as of January 1, 2018. For the 2018 NSSRN, a sample of 100,000 RNs will be selected. The sample is selected from a compilation of files provided by the State Boards of Nursing and the National Council of the State Boards of Nursing (NCSBN) and supplemented with administrative records-based information. The Census Bureau’s Center for Administrative Records Research and Applications (CARRA) is maintaining the files and compiling them into a RN universe file. As background, CARRA is an interdisciplinary group in the Research and Methodology Directorate and is charged with the strategic re-use of administrative data from federal, state, and commercial providers. Information is combined from multiple sources to create new data products that would be impossible to produce using single data sets. While the majority of the states provide their nursing data to the NCSBN, there are 10 states that do not. These State Boards of Nursing had to be contacted individually for their nursing data: Alabama, Alaska, Delaware, Hawaii, Massachusetts, Michigan, Nevada, New York, Oklahoma, and Wisconsin. Additional data were needed to identify Nurse Practitioners. There are 24 states from the NCSBN file and 3 states from the individual state boards of nursing files where NPs could not be identified. Additional files were provided by the following states containing advanced practice registered nurses (APRN) licensure information:

Alaska	Maryland	Ohio
California	Michigan	Pennsylvania
Colorado	Minnesota	Rhode Island
Connecticut	Mississippi	South Carolina
Washington, DC	Nebraska	Tennessee
Georgia	New Hampshire	Utah
Indiana	New Jersey	Virginia
Kansas	New York	West Virginia
Louisiana	North Carolina	Wyoming

After the files are combined, missing data on race, ethnicity, and gender will be filled with the most recent information (if available) from CARRA’s administrative records. A Protected Identification Key (PIK) is provided for each record that is identified and then assigned a unique number (control number) to every record with the same PIK. The final sampling file will contain the unique control number and the PIK will be removed. The creation of the unique control number will allow for data security of the administrative records CARRA uses to fill in data gaps.

The following RN records will not be eligible to be sampled:

• Nurses who have died

• Addresses that are incomplete, i.e., not mailable

• Addresses that are out of the country

• Nurses whose license expired prior to January 1, 2018

A sampling stratum that indicates the primary licensing state will be created. For most records this will remain unchanged, except for RNs who live in AK or HI. For nurses who reside in AK or HI, their sampling stratum variable will be reassigned to match their state of residence, regardless of where they were licensed.

When duplicate records exist, the record indicating the license with the latest expiration date will remain eligible, while the other record(s) will be made ineligible. If the expiration date is missing for all records in a set of duplicates, the record in the state with the highest sampling rate will remain eligible, while the rest of the records in the set of duplicates will be made ineligible.

There will be two strata per state, one for RNs holding an NP license and another for all other RNs. There will be different sampling rates for RNs and NPs across states, with the goal of having a large enough sample of RNs and NPs in each state to produce reliable state estimates of each population.

The universe counts for each strata within each state will be obtained from record counts in the universe file after duplicate records are removed. The expected response rate for each state in the 2018 NSSRN will be estimated as the lowest response rate observed between the 2008 NSSRN and the 2012 National Sample Survey of Nurse Practitioners (NSSNP). Since the primary estimates of interest are employed nurses, taken into account is the percent employed in the state from the 2008 NSSRN. Lastly, the resulting estimated sample size is taken and the expected variances will be calculated for the state estimates of employed RNs and employed NPs.¹ The state sample sizes are then increased enough to detect a minimum significant difference of 5% at the 90% confidence level for both RN and NP estimates. The variance calculations assume a conservative estimate of 50% from the base of employed RNs and NPs. The state level variances include a finite population correction factor since the NP sample is a large portion of the NP population in many states.

The nursing data will be sorted by age, gender, race, ethnicity, county, and zip code within each of the sampling strata prior to sample selection in order to improve the distribution of these variables in our sample and decrease the variance of survey estimates.

2. Expected Yield

The respondent universe for the NSSRN is all registered nurses (RNs) and nurse practitioners (NPs) currently living in the U.S. with a valid license as of January 1, 2018. Currently, the sampling universe includes about 4.56 million RNs and approximately 200,000 NPs nationwide. The initial sample size for the NSSRN is 100,000 RNs and NPs or about 60,000 RNs and 40,000 NPs. Table 1² shows the sampling rate for each stratum, total sample size, RN license alone (stratum 1) sample size, RNs with a NP license (stratum 2) sample size by each state and Washington, DC.

In estimating the NP universe, it is assumed an 87% match rate between the APRN file and the NCSBN file. (The 87% match rate is an average of the current match rates excluding Utah which has a match rate of 25% due to differential treatment of the NPs.) The 87% of the current NP frame total as our population size for each state. The sampling rate is the sample size divided by population size for each state. As previously mentioned, the sample size for each state is the minimum sample size needed to detect a significant difference of 5% on estimates of 50% of the employed RN and NP population by state at the 90% confidence level.

Table 1. States Sample Sizes

	Table 1. State Sample Sizes3
		Stratum 1	Stratum 2	Total	Stratum 1	Stratum 2
State		Sampling Rate	Sampling Rate	Sample	Sample	Sample
Alabama		0.012	0.245	1,650	850	800
Alaska		0.084	0.516	1,450	1,000	450
Arizona		0.012	0.157	2,000	1,100	900
Arkansas		0.023	0.391	1,800	950	850
California		0.009	0.073	4,911	3,411	1,500
Colorado		0.013	0.179	1,600	900	700
Connecticut		0.014	0.207	1,700	850	850
Delaware		0.056	0.467	1,500	1,000	500
District of Columbia		0.068	0.520	2,350	1,550	800
Florida		0.009	0.068	3,697	2,497	1,200
Georgia		0.010	0.156	2,193	1,193	1,000
Hawaii		0.078	0.775	2,000	1,500	500
Idaho		0.048	0.448	1,500	1,000	500
Illinois		0.010	0.114	2,688	1,788	900
Indiana		0.010	0.180	1,909	1,109	800
Iowa		0.018	0.243	1,550	900	650
Kansas		0.018	0.223	1,700	950	750
Kentucky		0.012	0.144	1,500	800	700
Louisiana		0.015	0.216	1,700	900	800
Maine		0.039	0.431	1,500	900	600
Maryland		0.013	0.175	1,850	1,000	850
Massachusetts		0.010	0.102	2,009	1,209	800
Michigan		0.010	0.144	2,242	1,442	800
Minnesota		0.010	0.146	1,667	1,017	650
Mississippi		0.024	0.271	1,950	1,050	900
Missouri		0.010	0.113	1,739	1,039	700
Montana		0.069	0.583	1,500	1,000	500
Nebraska		0.030	0.396	1,400	800	600
Nevada		0.038	0.452	1,950	1,300	650
New Hampshire		0.041	0.450	1,500	850	650
New Jersey		0.010	0.139	2,183	1,183	1,000
New Mexico		0.049	0.436	2,000	1,250	750
New York		0.009	0.073	3,823	2,323	1,500
North Carolina		0.010	0.143	2,059	1,259	800
North Dakota		0.060	0.592	1,250	800	450
Ohio		0.009	0.088	2,588	1,688	900
Oklahoma		0.023	0.371	1,900	1,150	750
Oregon		0.015	0.176	1,400	800	600
Pennsylvania		0.009	0.091	2,678	1,748	930
Rhode Island		0.052	0.517	1,450	950	500
South Carolina		0.017	0.290	1,850	1,050	800
South Dakota		0.047	0.593	1,250	800	450
Tennessee		0.010	0.068	1,719	859	860
Texas		0.009	0.070	3,559	2,459	1,100
Utah		0.029	0.359	1,550	850	700
Vermont		0.081	0.677	1,350	950	400
Virginia		0.010	0.182	1,920	970	950
Washington		0.010	0.145	1,643	893	750
West Virginia		0.030	0.437	1,650	950	700
Wisconsin		0.010	0.149	1,749	949	800
Wyoming		0.072	0.749	1,350	1,000	350
National				99,624	60,734	38,890
					60.96%	39.04%

3. Estimation Procedures

There will be written specifications for weighting the data that will be collected in the 2018 NSSRN. Instructions for computing adjustments (e.g., coverage and nonresponse) and several final RN and NP weights will be provided.

4. Nonresponse Bias Analysis

Standard 1.3 of the OMB Standards and Guidelines for Statistical Surveys (2006) states that “Agencies must design the survey to achieve the highest practical rates of response, commensurate with the importance of survey uses, respondent burden, and data collection costs, to ensure that survey results are representative of the target population so that they can be used with confidence to inform decisions.” Implicit in this standard is the assumption that the frame variables used at the design stage are sufficiently predictive of the collection variables for this to be feasible. Under this assumption, standard nonresponse bias analyses techniques can and will be applied to frame data variables to study potential areas of nonresponse bias (both item and unit) in the survey estimates.

2. Survey Collection Procedures

This section describes the data collection procedures that will be used in the NSSRN. The Census Bureau will request survey participation from 100,000 RNs via one of two modes: Web survey or paper questionnaire. Half of respondents will receive a letter invitation with the Web URL and login ID included in the letter. The other half of respondents will be mailed a paper questionnaire with the Web URL and login ID included in the questionnaire package. Those RNs that receive a paper questionnaire in the first mailing will be randomized. See attachment B for sample letters.

The 2018 NSSRN will include multiple contact strategy experiments to reduce both follow-up costs and nonresponse bias. There will be a non-monetary incentive experiment in the first mailing. Approximately half of the sample will be mailed an inscribed syringe pen and lanyard for the first contact attempt. Receiving a syringe pen and lanyard were discussed with the members of the nursing workforce and the feedback was that syringe pens are common in the nursing field due to the nature of the occupation. Non-monetary incentives have been used in past research studies to reduce non-response bias⁴. The syringe pen and lanyard will be experimentally tested again in the second follow-up mailing with a segment of the non-responding sampled RNs/NPs from the control group from the initial mailing. Half of those non-respondents will be mailed a syringe pen and lanyard.

Additionally, the 2018 NSSRN will include an experiment to test the efficacy of an infographic in the third contact attempt. Fifty percent of the RN sample will be randomly assigned the treatment group. While the content and design of the infographic is still being developed, it is anticipated that providing respondents with a visually pleasant overview of the survey, including survey design, key estimates from past iterations, and information on how the data can benefit their community, will encourage response. Additionally, there will be logos of the several nursing groups that endorse the NSSRN. See attachment C for a list of the organizations and infograph. Higher response early in data collection can reduce follow-up costs and nonresponse bias.

The NSSRN survey (for both Web and paper) will cover the following content areas: eligibility and education; principle nursing employment; job changes; secondary jobs in nursing; nurse practitioners; nurses not working in nursing; previous nursing employment; national practitioner data bank; general demographic background; and license and certification. See attachment D for a copy of the questionnaire.

3. Methods to Maximize Participation Rates and Deal with Nonresponse

In designing the various modes of the NSSRN questionnaire, attention is placed on the following design elements to help increase cooperation by prospective respondents.

• In developing and refining specific questions, the goal will be to create a logical, clear questionnaire with concrete question wording and simple grammar.

• The Web and paper versions of the questionnaire will be attractive with clear and simple instructions on how to complete specific questions.

• Questions will be grouped according to subject areas.

• Questionnaire formatting will maximize readability, including appropriate question spacing, font type and size and easy to follow skip instructions.

• Questionnaire formatting considerations will also include the use of color and pictures to enhance respondent comprehension.

• Respondent contact strategies and letters have been carefully designed to grab the attention of the respondent and pique interest in the subject matter.

• Some respondents will receive a pen and lanyard as an unconditional non-monetary incentive in the survey. See Section A.9 of the supporting statement for more information on incentives.

• In addition to the methods above, an infographic experiment will evaluate the efficacy of an infographic as a means of increasing respondent cooperation with the NSSRN.

Data collection for the NSSRN will involve a series of mailings and nonresponse follow-up activities, encouraging questionnaire completion (see attachment A). Our proposed approach to data collection and nonresponse follow-up is based on previous project experience and recommendations made by Dillman and colleagues (2009)⁵.

Invitation Letter. An initial invitation letter will be mailed to all potential respondents providing details about the study, a Web URL with the login ID for accessing the Web version of the questionnaire, and a toll-free number for the individual to call if there are questions or comments. Half of the sample will receive a paper questionnaire in the initial mailing (paper and web treatment). In addition to the invitation letter, 50% of the sample will also receive a token of appreciation (a pen and lanyard).

Additional mailings. Subsequent to the first invitation mailing, the Census Bureau will send all sample RNs a reminder pressure-sealed postcard containing the Web URL with the login ID. The NSSRN employs multiple contacts with nurses to maximize response. Non-respondents in the web first groups, contact strategies include up to three web invitation letters, three pressure sealed reminder letters, and two paper questionnaires. For those non-respondents in the paper first groups, contact strategies include up to three web invitation letters with a paper questionnaire, two pressure sealed reminder letters, and one final reminder letter in a manila envelope.

4. Individuals Responsible for Study Design and Performance

The Census Bureau will collect the information on behalf of HRSA NCHWA. Contact information for the Census Bureau’s principal staff on the project is listed below:

Jason Fields

Survey Director

U.S. Census Bureau, ADDP, HQ-7H153

4600 Silver Hill Road

Washington, DC 20233-0001

Phone: (301) 763-2465

Email: [email protected]

Daniel Doyle

Assistant Survey Director

U.S. Census Bureau, ADDP, HQ-7H051

4600 Silver Hill Road

Washington, DC 20233-0001

Phone: (301) 763-5304

Email: [email protected]

Elizabeth Sinclair

Survey Team Lead

U.S. Census Bureau, ADDP, HQ-7H036F

4600 Silver Hill Road

Washington, DC 20233-0001

Phone: (301) 763-3748

Email: [email protected]

Michael Brennan

Survey Team Member

U.S. Census Bureau, ADDP, HQ-7H036B

4600 Silver Hill Road

Washington, DC 20233-0001

Phone: (301) 763-6196

Email: [email protected]

List of Attachments:

Attachment A – 2018 NSSRN Questionnaire Changes

Attachment B – Letters

Attachment C – Infograph listing supporting organizations

Attachment D – 2018 NSSRN Questionnaire

1 The calculation of the variance for the state estimates of RNs is and the calculation of the variance for the state estimates of NPs is .

2 The sample sizes and sampling rates will change when the state universe counts are adjusted for each stratum. This is expected to have little effect on the sample sizes by stratum, generally, the most noticeable effect will likely be the stratum 2 sample sizes for states with a small NP population.

3 To calculate the sample size for RN and NP for Alabama and Hawaii the totals are from NCSBN and American Association of Nurse Practitioners (AANP). Georgia, North Carolina, South Carolina, and Virginia NP totals are from AANP as well.

4 White, E. Carney, P. & Kolar, A. (2005). Increasing Response to Mailed Questionnaires by Including a Pencil/Pen. American Journal of Epidemiology, 162(3):261 -266.

5 Dillman, D.A.; Smyth, J.D.; Christian, L.M. (2009). Internet, mail and mixed-mode surveys: The tailored design method, 3rd edition. Hoboken, NJ: John Wiley & Sons.

25