MOPSHP_SupportingStatementB_022221

Department of Commerce

U.S. Census Bureau

OMB Information Collection Request

Management and Organizational Practices Survey-Hospitals

OMB Control Number 0607-<XXXX>

Part B – Collections of Information Employing Statistical Methods

Question 1. Universe and Respondent Selection

The Management and Organizational Practices Survey-Hospitals (MOPS-HP) is a sample survey of approximately 3,200 establishments classified as a general medical or surgical hospital. The sample represents a universe of approximately 5,000 establishments, based on the Census Bureau’s Business Register and 2017 Economic Census data. Establishments were selected if they were classified as a general medical or surgical hospital and associated with a firm included in the Service Annual Survey’s (SAS) sample.

The SAS unit response rate has been in the 65-70% range in recent years. We estimate the final unit response rate for the MOPS-HP will be approximately 70%.

The MOPS-HP target population and sample is in essence the same target population and sample as the SAS for 6221. For 6221, where revenue is the key item in SAS, the total quantity response rate (TQRR), which is what is used to dictate the need for a nonresponse bias study in that survey's case, is consistently 80% or higher. This is above the threshold of requiring a nonresponse bias study. Although the SAS is primarily measuring total revenue and the MOPS-HP is primarily measuring management practices, which is not a dollar volume measurement, the target population for 6221 is represented in both the SAS and MOPS-HP by the same representative proportion of sample, with weights applied in both instances based on dollar volume measures of size.  For this reason, there is no evidence to believe that the respondents will differ significantly from the non-respondents with any more consistency than they do for the SAS. However, if the MOPS-HP response rate does not meet these standards, a nonresponse bias study will be conducted at that time.

Question 2. Procedures for Collecting Information

1. Description of Reporting Forms

We will mail forms to approximately 3,200 establishments associated with firms included in the SAS sample and classified under NAICS code 6221, General Medical and Surgical Hospitals.

2. Sampling Methodology

The MOPS-HP sampling frame includes establishments classified under NAICS code 6221 (General Medical and Surgical Hospitals) and associated with firms included in the SAS sample. The information used to create these sampling units will be extracted from data collected as part of the SAS, Economic Census, and from establishment records contained on the Census Bureau's Business Register. We are currently developing the sampling methodology so all information in this section is subject to change.

To create the sampling frame, records will be extracted for all employer establishments located in the United States that are classified in NAICS code 6221 as defined by the 2012 NAICS and that are associated with firms included in the SAS sample. The current SAS sample was selected based on data from the 2012 Economic Census and Business Register (BR) data from 2012, 2013, and 2014. For the MOPS-HP, we may extract the universe of establishments in 6221 on a more recent year of BR data to provide a more up-to-date frame with revenue at the establishment level.

The MOPS-HP will use a stratified sample design. This mimics the current sample design for the SAS. Stratification for the MOPS-HP will most likely be done by tax-status and Census region with sub-stratification on revenue.

3. Non-Response

Imputation methodologies are still under development. In general, data will be imputed using survey data as input for unit non-response, item non-response, and for responses that fail computer or analyst edits.

4. Estimation Procedure

Estimation procedures are currently being developed. Conditional on data quality, indices on management practices are planned and will be used in tabulations and empirical analysis. These indices will be similar to those previously developed to measure management practices using data from the WMS¹ and MOPS.² For example, collected data from MOPS have been aggregated via simple averaging into a single metric or management score ranging between 0 and 1. A management score of 0 indicates the least structured management practices – little monitoring of performance indicators, only annual targets, weak incentives such as promotions based solely on tenure, or no effective action taken for underperforming workers. A score of 1 indicates the most structured practices – processes for continuous improvement, a mix of short- and long-term targets, and performance-based promotions. Four indices are proposed using the MOPS-HP’s collected data: (1) an index comparable to the 2015 MOPS’ index for manufacturing (Section C on the survey form in Attachment A), (2) an index for management practices for team interactions and staffing allocation decisions (Sections E and F), (3) an index for the adoption of standardized clinical protocols (Section G), and (4) an index for managing multiple objectives – clinical and financial (Section H).

Variance estimation will be performed using the Random Group methodology, which is the same method used to compute variance estimation for the SAS. The Random Group method of variance estimation was chosen for its ability to handle complex survey designs and for its versatility in dealing with different types of estimates (e.g. totals, ratios, etc.). We will utilize four random groups for MOPS-HP.

Question 3. Methods to Maximize Response

The initial letter (Attachment C) explains the necessity and use of the data, states the respondents’ authentication code, and provides the website where the respondent can report online and access the current year worksheet (Attachment A). In an effort to promote electronic reporting, paper forms have been eliminated from the initial and follow-up mailings, and respondents are instructed to provide data electronically.

A due date reminder will be mailed approximately two weeks before the survey is due and emailed approximately one week before the survey is due. The SAS utilizes two follow-up mailings and three follow-up e-mail reminders for delinquent cases. The schedule for the respondent contacts is as follows:

Initial mail out	April 2021
Due date reminder	April 2021
Due date	May 2021
First mail follow-up	May 2021
First e-mail follow-up	June 2021
Second mail follow-up	June 2021
Second e-mail follow-up	July 2021
Telephone follow-up	July – August 2021
Third e-mail follow-up	August 2021

Due to the nature of the respondents, this schedule may be impacted by the effects of COVID-19. The Census Bureau is monitoring the ongoing situation and will adjust dates as necessary as the collection start date approaches as we do not want to add burden to an overly burdened sector of the economy.

Firms are given at least 30 business days to respond to the initial mailing and are given extension dates upon request. The Census Bureau also provides a telephone number for assistance with any questions or concerns about the survey.

The Census Bureau prioritizes providing quality customer service to respondents to maximize response. With the Respondent Portal, respondents can communicate more easily with Census Bureau staff. From within the portal, they can send secure messages directly to survey representatives. The Census Bureau staff also provides assistance to respondents by walking them through forms if necessary, explaining specific items on forms, granting extensions, and helping with access to forms and any technical issues.

Question 4. Tests of Procedures or Methods

As part of the Census Bureau’s statistical quality standards,³ the content for the MOPS-HP has undergone cognitive testing. The final content can be seen in Attachment A. The Census Bureau interviewed respondents to help ensure that the questionnaires and supplemental materials support a balance between collecting high quality data and minimizing respondent burden. This testing was conducted in 2018 during two separate rounds with respondents who primarily held the title of either Chief Nursing Officer (CNO) or Chief Financial Officer (CFO); testing revealed that CNOs were the more appropriate respondents for the planned content. The MOPS-HP content was tested with thirty hospitals across seven states and the District of Columbia.⁴ The findings helped the Census Bureau incorporate industry-tested terminology, provide examples and instruct respondents as needed, order and word questions, provide hospital-appropriate responses, and add and delete questions.

1. Defining Key Terms

   1. Clinical managers: Initial drafts of the content tested in round 1 referred generally to managers, as defined by those involved in clinical/operational decision making. However, respondents made various suggestions and the questions were subsequently changed to focus more narrowly on clinical managers, defined as “those who are involved in patient care decision-making” (Q4).

   2. Providers: Following feedback obtained during cognitive testing that respondents considered physicians to be providers, and distinct from frontline clinical workers (FCWs), an additional definition was added. Based on respondent feedback, providers are defined as “physicians, physicians’ assistants, advanced practice nurses, and others responsible for evaluating, diagnosing, and treating patients”. (Q5)

   3. Frontline clinical workers: FCWs were initially defined as clinical staff with non-managerial responsibilities, including physicians, staff nurses, and medical assistants. During round 1 of cognitive testing, however, the Census Bureau was advised that physicians are not considered FCW. Respondents suggested adding a listing of which clinical positions should be included or excluded in the definition of FCWs to promote consistency. After testing well during round 2, revisions were adopted for the final definition – “FRONTLINE CLINICAL WORKERS include all clinical staff with direct patient care responsibilities (such as nurses, nurses’ aides, physical/occupational/speech therapists, radiology and laboratory technicians), who do NOT have employees directly reporting to them. Do NOT include non-clinical frontline staff such as food services, housekeeping, or maintenance staff” (Q6).

   4. Key performance indicators (KPIs): Round 1 cognitive testing revealed that a more specific and thorough definition of KPIs was needed in place of listing examples that included metrics on cost, waste, clinical quality, financial performance, absenteeism, and patient safety. A new definition was tested in round 2 which defined KPIs as “quantifiable metrics used to evaluate the success of any clinical or non-clinical activity or function”. This definition was further refined to focus only on clinical activities, since tested respondents asked for clarification on whether to include KPIs that might be monitored for financial, dietary, and/or human resource activities (Q4).

2. Providing Examples

   1. While providing examples can potentially limit the options considered by the respondent, cognitive testing revealed that respondents found the following examples of hospital-wide goals for patient care to be helpful: “infection rates, readmission rates, and wait times” (Q8).

   2. Respondents repeatedly provided the same examples for financial goals, which suggested that the MOPS-HP did not need to provide examples for additional clarification (Q11).

   3. During cognitive testing, respondents were asked about their interpretation of “data” when asked about its use in meetings dedicated to the discussion of clinical outcomes (Q26). Their answers included key performance indicators, readmission rates, infection rates, and scorecards which made including examples unnecessary.

   4. In round 1, examples of standardized clinical protocols were tested and these included checklists or patient bar-coding. Described by some as clinical pathways or maps, most respondents had a clear interpretation of protocols thereby eliminating the need to include examples (Q33).

3. Instructing Respondents

   1. Cognitive testing of the MOPS-HP revealed that adding instructions to include time spent on remediation was important when asking about management practices for underperformance (Q17-19). Respondents explained that remediation, such as training, performance improvement plans, and mentorship were common approaches for addressing underperformance of clinical managers, providers, and frontline clinical workers.

   2. As a result of testing, instructions have been added to “exclude serious reportable events that result in patient harm or death and are due to a lapse or error in the hospital”, when asking how the hospital addresses problems with patient care delivery (Q20-21).

4. Ordering Questions

   1. Feedback during cognitive testing helped to redesign the order of the MOPS-HP questions. In round 1, the first question asked how the hospital addressed problems; however, respondents suggested that the survey start with a less difficult question. In the final question ordering, questions 20 and 21 now ask how the hospitals typically addresses problems with patient care delivered by providers and FCW, respectively. The first two questions on the MOPS-HP now ask for the respondent’s tenure at the hospital and as a manager at the hospital.

   2. Initially Section E asking about the management of team interactions began with a question on how frequently these meetings were held, but cognitive testing results suggested that the first question should ask who participated (Q23).

5. Wording Questions

   1. The MOPS-HP asks for the number of licensed beds. Initially this question asked for the number of staffed beds. However, in cognitive testing, respondents were confused as hospitals may have differing counts for beds that are staffed, budgeted, operated, or licensed. In round 2, respondents were asked for the number of licensed beds and they indicated that they would not have any difficulty providing this information (Q3).

   2. The MOPS-HP asks how often clinical managers review KPIs (Q4), and two separate questions ask how often clinical KPIs are given to providers (Q5) and given to FCWs (Q6). Initially, respondents were asked how often providers and FCWs reviewed the KPIs; however, testing revealed they could only report how frequently these indicators were distributed to providers and FCWs.

   3. Initially questions referred to “targets”, but tested respondents felt this term was less relevant for hospitals and suggested the term “goals”, which was cognitively tested and subsequently adopted (Q8-13).

   4. MOPS-HP questions initially referred to “patient care goals”, however, respondents suggested saying “hospital-wide goals for patient care” to avoid confusion with individualized patient care plans (Q8-10).

   5. When cognitively testing how FCWs and clinical managers are promoted, “relationships” was used as an example of factors other than performance and ability. However, respondents suggested dropping “relationships” and adding “managerial potential” to encapsulate other factors such as experience, tenure, and the ability to get things done (Q14-16).

   6. The MOPS-HP asks about the hospital’s actions when providers incompletely documented patients’ medical records (HP-38). This wording reflects respondents’ emphasis on complete documentation rather than saying “poor” documentation (Q38-39).

6. Listing Responses

   1. In question 1, the MOPS-HP asks what year the respondent started working at the hospital. If respondents indicate they were not at the hospital being surveyed in 2019 then only 2020 information will be requested for subsequent questions to reduce respondent burden and to maximize data quality.

   2. The MOPS-HP asks who was aware of the hospital-wide goals for patient care and financial goals (Q10, Q13). Initially respondents were asked to check all that apply from options for senior non-clinical managers (Chief Financial Officer, Chief Executive Officer), senior clinical managers (Chief Nursing Officer, Chief Medical Officer), department chiefs/nurse managers, and frontline clinical staff. However, we added options for the board of directors and/or hospital president as well as non-clinical staff based on respondents’ suggestions.

   3. Following feedback from testing, one additional response option was added for questions on how problems with patient care delivery are addressed – “We tried to fix it, but did not remediate the problem” (Q20, Q21).

   4. The quality of the MOPS-HP’s management training response options benefited from both rounds of cognitive testing (Q22). Initially the only graduate-level training program listed in the responses was a Master of Business Administration (MBA). After round 1, an option for a Masters’ program in health care administration was added but found to be insufficient for measuring the many advanced degree programs found among CNOs. For example, CNOs may have a Master of Science in Nursing, a Master of Health Administration, a Master of Healthcare Management, or an MBA. Rather than trying to develop an exhaustive list of names for non-MBA programs, the final wording refers to “other graduate-level degree programs lasting at least one year or more full-time that included management coursework”.

   5. When asked about the management of team interactions, respondents suggested adding the board of directors to the possible responses (Q23, Q28). Along with this addition, the responses listing clinical staff were also edited to move away from terms such as “department chiefs/nurse managers, physicians, nurses, and other support staff”. The final response options include: “Board of Directors and/or President,” “Senior clinical managers”, “Clinical Managers”, “Non-clinical managers”, “Providers”, and “Frontline clinical workers”.

   6. When asked how work was allocated to the hospital’s clinical staff, respondents during testing suggested that the list of possible responses be expanded beyond just senior managers (Chief Nursing Officer, Chief Medical Officer) and department chiefs/nurse managers (Q29). The revised responses for round 2 included senior clinical managers, clinical managers, physicians, and FCWs. As noted elsewhere, the term physicians were replaced with providers after round 2, and responses were added for senior non-clinical managers (Chief Executive Officer, Chief Financial Officer, and Chief Operating Officer) and non-clinical managers.

   7. When asking about the hospital’s actions when providers incompletely document in patients’ medical records, new response options were suggested during testing (Q38). These included: “Required provider to meet with compliance office” and “Required provider to undergo peer review” (e.g., by the hospital’s medical staff). A response for “Required provider to meet with other staff not listed above” has been added to include individuals from medical records, health information departments, and/or clinical documentation experts that may carry different titles across hospitals. Cognitive testing also led to a suggested response option for “Provider was penalized financially”, which some respondents thought could be suspensions or removal of admitting privileges. Some of the original responses also required some editing based on respondent feedback. For example, rather than requiring providers to meet with hospital administrators, we were advised to replace this with “Hospital senior managers or supervisors”.

7. Adding Questions

   1. Respondents frequently commented that various management practices can differ for providers and FCWs. For example, since some providers may not be employed by the hospital and instead have other contractual arrangements, respondents stressed the importance of asking the question about underperformance separately for providers and FCWs as responses may vary for employees versus contractors (Q18-19). Similarly, separate questions are asked on management practices for when clinical key performance indicators are provided (Q5-6), providers and FCWs are promoted (Q15-16), and problems with patient care delivery are addressed (Q20-21).

   2. Following communications between the testing staff, survey director, and survey partner, two questions were added related to COVID-19 and hospitals’ ability to respond to shocks to their organization and the health care system (Q30 and Q36). Since this content was a late addition, the questions did not go through cognitive testing due to insufficient time. In accordance with the Census Bureau’s Statistical Quality Standard A2, Developing Statistical Data Collection Instruments and Supporting Materials⁵, the new content was reviewed independently by two cognitive experts at the Census Bureau. Subsequently, the cognitive reviewers, survey managers, and subject matter experts met to discuss the reviewers’ recommendations, seek clarifications, and collaboratively developed acceptable alternatives, upon which mutual agreement was reached. The expert reviews and a summary of the discussion can be seen in Attachment H.

8. Deleting Questions

   1. In round 1 of cognitive testing, respondents were asked how many key performance indicators (KPIs) they monitored. They revealed that the number of KPIs could be in the hundreds or even the thousands thereby reducing the value of collecting these data and this question was dropped after round 1.

   2. Respondents were asked for the hospital’s typical “nurse to patient ratio” and the typical “medical assistant to patient ratio” during round 1 of testing. These two questions were subsequently dropped, since respondents explained these ratios could differ by a number of factors, including budgetary reasons and/or patient acuity which can differ by hospital unit or by day. Respondents also advised that an average ratio would not be representative and these data would be difficult if not impossible to report.

   3. The MOPS-HP asked respondents how many standardized clinical protocols were used at the hospital during round 1 of cognitive testing. However, many respondents explained that these existed anywhere in the hospital that standard definitions for providing patient care existed and could be specific to multiple departments. They advised that the number of protocols could be in the hundreds if not the thousands and even estimates would be difficult to report. Since the interest was in whether a hospital used any protocols, and cognitive testing revealed that for respondent hospitals this was always the case, this question was dropped.

   4. After round 1, the question asking whether FCWs were aware of documenting key words for reimbursement was dropped. This question tested poorly and respondents advised that FCWs in the hospital generally do not document nor code medical records.

   5. After round 2, a question asking who interacted with systems and tools used for documenting patient medical records was dropped. Many tested respondents interpreted systems and tools as being electronic health records, but indicated the collected data would show little variation since most clinical staff interacted with these systems and tools.

   6. With the late addition of two questions in the MOPS-HP content related to COVID-19 and hospitals’ ability to respond to shocks to their organization and the health care system (Q30 and Q36), adjustments were made to keep the total number of questions unchanged. In an effort to limit respondent burden while adding this content, discussion with the survey partner led to removing two existing questions about the documentation of patients’ medical records, specifically regarding training and response time to queries. 

Additionally, procedures in every phase of the MOPS-HP production will be tested – from mailout and data capture to editing and publication.

Question 5. Contacts for Statistical Aspects and Data Collection

Direct questions regarding the planning and implementation of this survey to Edward Watkins, U.S. Census Bureau, (301) 763-4750 or via email at [email protected]. Questions regarding survey methodology should be directed to Katrina Washington, U.S. Census Bureau, (301) 763-7212 or via email at [email protected].

Attachments to the Supporting Statement –

Attachment A: MOPS-HP Questionnaire

Attachment B: MOPS-HP Content Justification

Attachment C: Initial Letter to Respondents

Attachment D: Due Date Reminder Letter

Attachment E: Reminder Letter

Attachment F: Screenshot of Introductory Centurion Screen

Attachment G: BEA Letter of Support for MOPS-HP

Attachment H: Expert Reviews of Additional MOPS-HP Content and Decision Document

Attachment I: Title 13 Cited Authorities

1 Bloom, N. and J. Van Reenen. 2007. “Measuring and Explaining Management Practices Across Firms and Countries.” The Quarterly Journal of Economics 122(4): 1351–1408.

2 Buffington, C., A. Hennessy, and S. Ohlmacher. 2018. “The Management and Organizational Practices Survey (MOPS): Collection and Processing.” U.S. Census Bureau’s Center for Economic Studies Working Paper Series CES 18-51.

3 U.S. Census Bureau. “U.S. Census Bureau Statistical Quality Standards.” https://www.census.gov/about/policies/quality/standards.html. July 2013.

4 The number of tested hospitals and states has been reviewed to ensure no confidential information is disclosed (CBDRB-FY19-EWD-B00002).

5 U.S. Census Bureau. “Statistical Quality Standard A2: Developing Data Collection Instruments and Supporting Materials.” https://www.census.gov/about/policies/quality/standards/standarda2.html. May 2015.