Supporting Statement Part A -- Evaluation of the Guide to PFE 4-18-2011

Supporting Statement Part A -- Evaluation of the Guide to PFE 4-18-2011.doc

Development of the Guide to Patient and Family Engagement in Health Care Quality and Safety in the Hospital Setting

OMB: 0935-0176

Document [doc]
Download: doc | pdf





SUPPORTING STATEMENT


Part A







Evaluation of the Implementation of the Guide to Patient and Family Engagement in Health Care Quality and Safety in the Hospital Setting





Version: April 12, 2011







Agency for Healthcare Research and Quality (AHRQ)



Table of Contents


1. Circumstances that make the collection of information necessary 3

2. Purpose and use of information 8

3. Use of Improved Information Technology 8

4. Efforts to Identify Duplication 8

5. Involvement of Small Entities 8

6. Consequences if Information Collected Less Frequently 8

7. Special Circumstances 8

8. Consultation outside the Agency 9

9. Payments/Gifts to Respondents 9

10. Assurance of Confidentiality 9

11. Questions of a Sensitive Nature 9

12. Estimates of Annualized Burden Hours and Costs 10

13. Estimates of Annualized Respondent Capital and Maintenance Costs 12

14. Estimates of Annualized Cost to the Government 12

15. Changes in Hour Burden 12

16. Time Schedule, Publication and Analysis Plans 12

17. Exemption for Display of Expiration Date 14



A. Justification


1. Circumstances that make the collection of information necessary


The mission of the Agency for Healthcare Research and Quality (AHRQ) set out in its authorizing legislation, The Healthcare Research and Quality Act of 1999 (see Attachment A), is to enhance the quality, appropriateness, and effectiveness of health services, and access to such services, through the establishment of a broad base of scientific research and through the promotion of improvements in clinical and health systems practices, including the prevention of diseases and other health conditions. AHRQ shall promote health care quality improvement by conducting and supporting:


1. research that develops and presents scientific evidence regarding all aspects of health care; and


2. the synthesis and dissemination of available scientific evidence for use by patients, consumers, practitioners, providers, purchasers, policy makers, and educators; and


3. initiatives to advance private and public efforts to improve health care quality.


Also, AHRQ shall conduct and support research and evaluations, and support demonstration projects, with respect to (A) the delivery of health care in inner-city areas, and in rural areas (including frontier areas); and (B) health care for priority populations, which shall include (1) low-income groups, (2) minority groups, (3) women, (4) children, (5) the elderly, and (6) individuals with special health care needs, including individuals with disabilities and individuals who need chronic care or end-of-life health care.


Improving the quality and safety of health care in the United States is one of the most significant challenges facing the American health care system. Too many Americans continue to receive health care that is not grounded in a reliable evidence base of what is proven appropriate, safe, and effective. Extensive studies conducted during recent decades demonstrate that the U.S. health care system provides continuing unwarranted variation and costly, inefficient, and simply unsafe care.1,2,3 Involving patients and families in improving quality and safety in hospitals has the potential to improve health care experiences, delivery, and outcomes. AHRQ has been at the forefront of supporting increased involvement for patients, families, and the public in all aspects of health care.


This project will develop a program to help patients, families, and health professionals in the hospital support one another to improve quality and safety. To accomplish these goals, patients and families must be able to express what they want from their hospital care and how they want to be involved and then effectively communicate this information with health professionals. Conversely, health professionals must be able to understand what patients want to do and what is appropriate for them to do and feel that they have the system supports and tools to facilitate these actions.


To address this issue and help fulfill AHRQ’s mission of health care quality improvement, AHRQ will evaluate the implementation of a set of draft interventions and materials, entitled the Guide to Patient and Family Engagement in Health Care Quality and Safety in the Hospital Setting (“the Guide”), for use by patients, their family members, hospital staff, and hospital leaders to foster patient and family engagement around the issues of hospital safety and quality.


The goals of this project are to:


1) Identify the barriers and facilitators to implementing the Guide, including how barriers were overcome;


2) Assess hospital staff satisfaction with implementation of the Guide and identify changes in hospital staff behavior before and after implementation of the Guide (this could include changes in the hospital’s organizational culture with respect to patient- and family engagement and patient- and family-centered care);


3) Assess patient satisfaction with implementation of the Guide and changes in patient experience of care before and after implementation (this could include level of patient/family involvement in their own health care and patient/family involvement in quality improvement and patient safety activities);


4) Refine and update the Guide as necessary to incorporate lessons learned from the evaluation of the implementation.


We will evaluate the implementation of the final draft version of the Guide in three hospitals. These hospitals have not yet been selected, but will be selected for heterogeneity in terms of size, location, teaching status, and ownership.


The Guide has been developed and is in a final draft version (see Attachments B, C, and D). Hospitals will be asked to implement at least one of the strategies in Component 1 on a medical-surgical hospital unit. They will be asked to implement Component 2 across the hospital, and to read Component 3. The components and how hospitals will be asked to implement them are described in Exhibit 1 on the next page.

Exhibit 1. The three components of the Guide and how they will be implemented

Description of Guide Component

How hospitals will implement

Component 1: Hospital-stay Active Involvement

Materials to facilitate patient and family engagement during the hospital stay. Component 1 includes three strategies:

Each of the 3 hospitals will implement at least one of the Component 1 strategies on a medical-surgical unit. We will ensure that all strategies are implemented at least once across all 3 hospitals/units.


Strategy 1: Communicating to Improve Quality (admission communication packet)

Hospitals will distribute materials to patients and families at admission to help them understand who is who in their care, opportunities for engagement, and how to ask questions. Hospitals will educate staff about effective communication using tools in the Guide.

Strategy 2: Bedside change of shift

Hospitals will move their existing nurse change of shift report from the nurses’ station to the bedside. Hospitals will distribute materials to explain bedside change of shift to patients and families and train nurses in how to conduct shift report at the bedside.

Strategy 3: Discharge plan


Hospitals will implement a process that focuses on including patients and families in discharge planning discussions and addressing patient and family questions / concerns about discharge. Hospitals will distribute a discharge checklist to patients and families and train staff on the discharge process.


Component 2: Organizational Partnership Materials

Materials to help hospitals involve patients and families at the hospital organizational level (e.g., as short-term advisors, members of advisory councils, hospital committee members)

Each of the three hospitals will implement Component 2 across the entire hospital (i.e., this is not a unit-specific implementation).

Hospitals will begin to work with patient and family advisors. This involves identifying, recruiting, and training patient and family advisors, creating opportunities for advisors to provide input, and preparing hospital staff to work with advisors.


Component 3: Organizational Change and Leadership Materials

Information to help hospital leadership understand the importance of patient and family engagement and organizational changes to support Guide implementation.

Hospital leadership will read the materials in Component 3.



To evaluate the implementation of the Guide, the following data collections will be implemented:


1) Semi-structured interviews (see Attachments E, F, G, and H) will be conducted in-person with hospital staff and hospital leaders from each of the three participating hospitals. We will conduct both pre- and post- implementation interviews, using separate interview guides for staff and leaders. Pre-implementation interviews will focus on current knowledge, attitudes, and beliefs around patient and family engagement and on the current organizational culture and climate surrounding patient and family engagement. Post-implementation interviews will be conducted to understand the hospital’s experiences implementing the Guide interventions, including how easy or difficult the Guide was to implement; the perceived effects of the Guide implementation; and the sustainability of the Guide interventions.


2) Collection of documentation from each participating facility (see Attachment I). The purpose of this collection of documentation is to gather documentation of the implementation of the Guide and to document hospital policies and procedures related to patient and family engagement through a review of records and other materials. To the extent that it is available, the following types of documentation will be collected:

  • Background on organizational structure and vision

  • Policies and procedures related to Component 1 and Component 2 strategies

  • Tools used to foster communication between patients, family members and health care team

  • Policies and procedures related to patient and family engagement, patient- and family-centered care, quality and safety


This task will consist of forwarding emails and or photocopying and sending documents to the project team both pre- and post- implementation.


3) Bi-weekly semi-structured interviews (see Attachment J) will be conducted by telephone with the implementation coordinators from each participating hospital. At each hospital site, an implementation coordinator will be responsible for overseeing implementation activities and serving as a primary point-of-contact. Interviews with these individuals will provide a robust understanding of the Guide implementation and the ability to track the implementation in real time. These interviews will occur bi-weekly for 9 months.

4) Observation of Guide implementation (see Attachment K) around different activities targeted in the Guide components. The purpose of these observations is three-fold. First, we want to directly assess how the Guide is being implemented. Second, we want to assess any changes in patient-provider interactions after Guide implementation. Finally, observation will allow us to determine what, if any, follow up questions should be asked of hospital staff during the in-person interviews. As such, observations will occur both pre-and post-implementation. Observations will be conducted by the project staff as time permits during the pre-implementation and post-implementation site visits, with at least two observations per strategy of Component 1 (admission packet, bedside change of shift, discharge planning). Since this data collection does not impose a burden on the participating hospitals OMB clearance is not required; therefore it is not included in Exhibit 2 (estimated burden hours).


5) Focus groups (see Attachments L, M, and N) with patients and family members at each of the participating hospitals. The purpose of these groups is to elicit information about patients’ and families’ experiences of care at the hospital along with their reactions to tools in the Guide and their implementation. Three focus groups of up to 8 individuals will be conducted at each hospital post-implementation. One focus group will be conducted with patients only, one with family members only, and one with patients and family members together.

6) Staff Survey with hospital staff (see Attachment O). A web-based survey will be administered to hospital staff pre- and post-implementation. The purpose of the Staff Survey is to assess changes in organizational culture related to patient safety and engagement, and to assess significant changes in staff knowledge, attitudes, and behaviors. Items from the Medical College of Georgia (MCG) Patient- and Family-Centered Care Culture Survey will be used in this data collection activity. The survey items will be supplemented with questions from AHRQ’s Hospital Survey on Patient Safety Culture (HSOPS) and from the Army Medical Department Climate Survey (for complete sources, see Attachment L). At each of the three hospital sites, it is estimated that survey responses will be collected from at least 50 health professionals. The email invitation to invite staff to participate is included in Attachment P. The same questionnaire will be used at pre- and post-implementation.

7) Patient Survey (see Attachment Q). The patient survey will be administered pre-implementation and again post-implementation. The survey will be built around the CAHPS® Hospital Survey (HCAHPS) domains that assess aspects of patient-physician interaction around the hospital stay, including Communication with Nurses, Communication with Doctors, Communication about Medicines, Responsiveness of Hospital Staff, and Discharge Information. These scales directly assess the aspects of the hospital stay and encounters that we are hoping the Guide will affect. Additional questions to address any aspects of care covered by the Guide that are not adequately addressed by the HCAHPS composites will also be included in this survey. Finally, measures from the Patient Activation Measures (PAM) Survey will also be included (for complete sources, see Attachment N). The invitation to participants which will be distributed with all surveys is included in Attachment R. The same questionnaire will be used pre- and post-implementation.

This study is being conducted by AHRQ through its contractor, the American Institutes for Research (AIR), pursuant to AHRQ’s statutory authority to conduct and support research on healthcare and on systems for the delivery of such care, including activities with respect to the quality, effectiveness, efficiency, appropriateness and value of healthcare services and with respect to quality measurement and improvement. 42 U.S.C. 299a(a)(1) and (2).


2. Purpose and Use of Information


The purpose of the data collection is to inform improvements to the Guide. AHRQ will use this information to understand how hospitals implement the Guide and how patients, families, and hospital staff respond to implementation of the Guide. This data collection will also collect lessons learned that will be incorporated into the final version of the Guide.

3. Use of Improved Information Technology

In order to reduce respondent burden, the Staff Survey data will be administered online using web-based survey technology. Each potential respondent will receive an email containing a hypertext link to the survey’s website (Attachment M). Using this online system rather than a paper-based survey makes completing and submitting the survey less time-consuming for respondents. Any skip patterns included in the survey (that is, questions that are only appropriate for a proportion of respondents) will be automatically programmed into the Web-based form, to insure that inappropriate items are not accidentally answered by the respondent.

4. Efforts to Identify Duplication

The information collected as part of the qualitative data collection is unique to this project as it pertains specifically to the Guide to Patient and Family Engagement. Moreover, a recent literature scan conducted as part of this project revealed that materials designed to improve patient and family engagement have not been evaluated and impacts have not been studied.

Data regarding patient experiences of care (similar to the patient survey) are being collected through the CAHPS Hospital Survey, which is mandated by CMS for reimbursement. However, the existing HCAHPS data collection protocol cannot be used for our evaluation due to the inability to purposely identify and select patients who have had experiences with the Guide materials.

5. Involvement of Small Entities

Collection of information will not impact small businesses or other small entities, as information will be collected for existing hospitals and health systems which are not small entities.

6. Consequences if Information Collected Less Frequently

This is a one-time study. There are no plans to repeat it.

7. Special Circumstances

This request is consistent with the general information collection guidelines of 5 CFR 1320.5(d)(2).

8. Federal Register Notice and Outside Consultations

8.a. Federal Register Notice

As required by 5 CFR 1320.8(d), notice was published in the Federal Register on (date and page number of 60 day notice) for 60 days (see draft 60 Day FRN in Attachment P).

8.b. Outside Consultations

AHRQ has consulted with staff of American Institutes for Research (AIR), Consumers Advancing Patient Safety (CAPS), the Institute for Patient- and Family-Centered Care (IPFCC), the Joint Commission, and the Health Research and Educational Trust (HRET). AIR staff are experts in study design, as well as quantitative and qualitative methodologies. CAPS and IFCC are experts in patient and family engagement and patient-centered care. The Joint Commission and HRET are experts in the applied research, specifically related to healthcare delivery systems.

9. Payments/Gifts to Respondents

Incentives will be provided for the patient survey and for the patient focus groups. The American Association of Public Opinion Research lists monetary incentives as a procedure to consider in ensuring adequate survey response rates. Specifically, a $2 incentive will be offered with each patient survey to encourage completion. Research has shown that an incentive as small as one to two dollars mailed with the first questionnaire can increase response rates 5 to 8 percent.4


For each focus group, patients will be offered an incentive of up to $75 for participating in the group. Incentives are necessary for focus groups to compensate for transportation costs and lost opportunity costs (associated with other uses of time that must be foregone to allow participation).5 In addition, AIR’s Institutional Review Board (IRB) reviews all incentive payments to insure that the amount is not so large as to be considered “coercive”.

10. Assurance of Confidentiality

Individuals and organizations will be assured of the confidentiality of their replies under Section 934(c) of the Public Health Service Act, 42 USC 299c-3(c). They will be told the purposes for which the information is collected and that, in accordance with this statute, any identifiable information about them will not be used or disclosed for any other purpose.

11. Questions of a Sensitive Nature

Questions of a sensitive nature or other matters that are commonly considered private will not be asked.

12. Estimates of Annualized Burden Hours and Costs

Exhibit 2 shows the estimated burden hours for the respondents’ time to participate in this project. The total annualized burden hours are estimated to be 2,085 hours. The burden estimate comprises the following activities:

  • Planning activities: Hospitals will need time to review the Guide materials, participate in discussions about which strategies to implement, and plan for implementation. These activities are included in “Planning for Guide Implementation” in Exhibit 2. The estimate assumes that approximately 6 staff from each hospital will be involved in Guide implementation planning.


  • Implementation activities: Hospitals will be asked to implement at least one of the strategies in Component 1 on a medical-surgical hospital unit. They will be asked to implement Component 2 across the hospital, and to read Component 3. The estimate assumes that, on average, hospitals will implement two strategies from Component 1 as well as Components 2 and 3. We assume that 50 hospital staff will be involved in implementation. Fifty staff represents the maximum number of staff in units that will be implementing the Guide across all three hospital sites.


  • Data collection activities:

    • Semi-structured interviews will be conducted with 4 hospital staff members both pre- and post-implementation, and will take 1 hour to complete.

    • Semi-structured interviews will also be conducted with 2 hospital leaders, pre- and post-implementation, and will take 1 hour to complete.

    • Collection of documentation will occur pre- and post-implementation at each hospital and require 4 hours to complete each time.

    • Bi-weekly semi-structured interviews will be conducted with the implementation coordinator at each hospital (3 total). A total of 18 interviews per hospital over a 9 month period will occur with each interview taking about 30 minutes.

    • Focus groups will be conducted with patients, families, and both patients and their families. Three focus groups will occur at each hospital, post-implementation. Each focus group will last for an hour and a half.

    • The staff survey will be completed by approximately 50 hospital staff members from each hospital, pre- and post-implementation, and require 15 minutes to complete.

    • The patient survey will be conducted twice, pre- and post-implementation, by about 885 patients across all 3 participating hospitals and will take 30 minutes to complete.

Exhibit 2 shows the estimated annualized cost burden associated with the respondents’ time to participate in this project. The total cost burden is estimated to be $58,768.

Exhibit 2.  Estimated annualized burden hours

Activity

Number of respondents

Number of responses per respondent

Hours per response

Total burden hours

Planning and implementation

Planning for Guide implementation

3

6

8

144

Implementing Guide Components

3

50

5

750

Data collection

Semi-structured leader interviews – pre-implementation

3

4

1

12

Semi-structured leader interviews – post-implementation

3

4

1

12

Semi-structured staff interviews – pre-implementation

3

8

1

24

Semi-structured staff interviews – post-implementation

3

8

1

24

Collection of documentation

3

2

4

24

Bi-weekly semi-structured interviews

3

18

30/60

27

Focus group with patients

24

1

90/60

36

Focus group with patients’ family

24

1

90/60

36

Focus group with patients & family

24

1

90/60

36

Staff survey

3

100

15/60

75

Patient survey

885

2

30/60

885

Total

984

na

na

2,085



Exhibit 3. Estimated annualized cost burden

Form Name

Number of respondents

Total burden hours

Average hourly wage rate*

Total cost burden

Planning and Implementation

Planning for Guide implementation

3

144

$43.74

$6,299

Implementation of Guide components

3

750

$33.51

$25,133

Data collection

Semi-structured leader interviews – pre-implementation

3

12

$43.74

$525

Semi-structured leader interviews – post-implementation

3

12

$43.74

$525

Semi-structured staff interviews – pre-implementation

3

24

$33.51

$804

Semi-structured staff interviews – post-implementation

3

24

$33.51

$804

Collection of documentation

3

24

$21.16

$508

Bi-weekly semi-structured interviews

3

27

$33.51

$905

Focus group with patients

24

36

$20.90

$752

Focus group with patients’ family

24

36

$20.90

$752

Focus group with patients & family

24

36

$20.90

$752

Staff survey

3

75

$33.51

$2,513

Patient survey - pre-implementation

885

885

$20.90

$18,497

Total

984

2,085

n/a

$58,768

*Based upon the mean of the wages for 11-9111 Medical & Health Services Manager ($43.74), 29-000 Healthcare Practitioner and Technical Occupations ($33.51), 43-6011 Executive Secretaries and Administrative Assistants ($21.16) and 00-0000 All Occupations ($20.90), May 2009 National Occupational Employment and Wage Estimates. United States , “U.S. Department of Labor, Bureau of Labor Statistics.” http://www.bls.gov/oes/current/oes_nat.htm#b29-0000

13. Estimates of Annualized Respondent Capital and Maintenance Costs

Capital and maintenance costs include the purchase of equipment, computers or computer software or services, or storage facilities for records, as a result of complying with this data collection. There are no direct costs to respondents other than their time to participate in the study.

14. Estimates of Annualized Cost to the Government

Exhibit 3 below breaks down the costs related to this study. Since this study will span two years, the costs have been annualized over a two year period. The total annualized cost is estimated to be $536,396.50.

Exhibit 3. Estimated Total and Annualized Cost

Cost Component

Total Cost

Annualized Cost

Guide Development

$526,214

$263,107

Data Collection Activities

$310,006

$155,003

Data Processing and Analysis

$110,620

$55,310

Project Management

$20,270

$10,135

Overhead

$105,683

$52,842

Total

$1,072,793

$536,396.50

15. Changes in Hour Burden

This is a new collection of information.

16. Time Schedule, Publication and Analysis Plans


AIR’s contract for this project lasts from September 21, 2009 to March 20, 2013. Data collection will begin in June 2011 or when OMB clearance is received (if clearance is received after June 2011). Findings from these data collections will be analyzed using standard qualitative and quantitative methods. An implementation and evaluation report will be submitted in September 2012. The Guide will also be made publicly available at the conclusion of this project.

Any publications and presentations of study findings will explicitly state the limitations of the methodology.


The focus group and interview data will be systematically analyzed to identify patterns and themes related to the goals of the data collection activities (as described above). Any changes in the documentation or any changes from in behaviors noted during observation will also be analyzed and used to provide context to the other findings.


Qualitative Data Analysis Methods

Qualitative data analysis methods will be used to analyze data from:

  • Semi-structured interviews with hospital leaders and hospital staff

  • Collection of documentation

  • Bi-weekly semi-structured telephone interviews with staff coordinators

  • Observation of Guide implementation

  • Focus groups with patients and families



All qualitative data will be managed and analyzed using NVIVO 8.0 or Atlas.ti, qualitative data analysis software programs. Either program can accommodate diverse types of qualitative data, including observation or interview notes “written-up” in Microsoft Word, electronic copies of reports (originals or scanned) or electronic report summaries.

We will define and apply a coding system for the notes from the semi-structured interviews and focus groups. This coding system will be applied using the qualitative software to facilitate the retrieval, summary, and synthesis of data. The data are coded by assigning labels to the units, clustering the codes into categories and hypothesizing about relationships among the categories. This process facilitates the identification of patterns and themes that explain the outcomes in different cases.

For all qualitative data, we will employ a variety of qualitative techniques to draw conclusions from the data (e.g., noting patterns and themes, plausibility, relationships between variables, and finding intervening variables)6.


Quantitative Data Analysis Methods

Quantitative data analysis methods will be used for the staff survey and patient survey. The staff and patient survey items will be used to create continuous scales defining the constructs of interest, which include staff behavior and patient experience of care. Data entry and verification procedures will include double data entry and validity checks, e.g., checks for out of range values. We do not intend to impute missing data.


We will report the results descriptively, and we will compare mean scores on the constructs between the pre-test (baseline) and post-test (follow-up).  Using linear regression techniques, the means will be adjusted to control for personal characteristics as well as unit and hospital characteristics.  The analyses will indicate whether, holding personal and provider characteristics constant, there is a statistically significant difference in staff behavior or patient experience of care after implementation of the Guide.  Finding the beta coefficient for the pre-post indicator not equal to zero is a test of whether the regression-adjusted mean of the post group (X = 1) differs from the regression-adjusted mean of the pre group (X=0), controlling for the other adjusters in the model. 


There are a number of threats to validity (i.e., alternative explanations for the observed results) that we cannot control in this evaluation. We will not know if observed differences would have occurred anyway (maturation effects) or as a result of other interventions that the hospital is engaged in (environmental effects). A comparison group would control for these threats; however the number of hospitals required for a comparative design is beyond the scope of this project. Even if funds were available to recruit a larger sample of hospitals, we would still encounter other threats inherent in quasi-experimental studies, such as selection bias introduced by including volunteer hospitals (as opposed to selecting a representative sample of hospitals). Despite these threats to validity, the results of the evaluation will inform the development of the Guide materials and are sufficient to accomplish the data collection objectives which are to inform improvements to the Guide and to understand how hospitals implement the Guide and how patients, families, and hospital staff respond to implementation of the Guide.

17. Exemption for Display of Expiration Date

AHRQ does not seek this exemption.

List of Attachments:

  • Attachment A: AHRQ's Authorizing Legislation

  • Attachment B: Component 1 Materials

  • Attachment C: Component 2 Materials

  • Attachment D: Component 3 Materials

  • Attachment E: Health Professional Pre-Implementation Protocol

  • Attachment F: Hospital Leaders Pre-Implementation Protocol

  • Attachment G: Health Professional Post-Implementation Protocol

  • Attachment H: Hospital Leaders Post-Implementation Protocol

  • Attachment I: Collection of Documentation Protocol

  • Attachment J: Bi-weekly Telephone Interview Protocol

  • Attachment K: Observation Protocol

  • Attachment L: Focus Group Protocol for Families on Component 1

  • Attachment M: Focus Group Protocol for Patients on Component 1

  • Attachment N: Focus Group Protocol for Families on Component 2

  • Attachment O: Staff Survey

  • Attachment P: Invitation to Participate in Staff Survey

  • Attachment Q: Patient Survey

  • Attachment R: Invitation to Participate in Patient Survey

  • Attachment S: 60 Day Federal Register Notice

1 Asch, D., & Werner, R. (2004). The unintended consequences of publically reporting quality information. JAMA, 293, 1239-44.

2 Institute of Medicine (IOM). Crossing the quality chasm: A new health system for the twenty-first century. Washington, DC: National Academy Press.

3 Wennberg, J. (2004). Practice variations and health care reform: Connecting the dots. Health Affairs, Web Exclusive, VAR140-VAR144.

4 James, J.M. and Bollstein, R. “The effect of Monetary Incentives and Follow-Up Mailings on the Response Rate and Response Quality in Mail Surveys.” Public Opinion Quarterly. 54:346-361 (1990).

5 Focus groups: A practical guide for applied research RA Krueger, MA Casey – 2008, page 77.

6 Miles, M. B., & Huberman, A. M. (1994). Qualitative data analysis

(2nd ed.). Thousand Oaks, CA: SA


15


File Typeapplication/msword
File TitleOMB Clearance Application
Authorhamlin-ben
Last Modified Bywilliam.carroll
File Modified2011-04-18
File Created2011-04-18

© 2024 OMB.report | Privacy Policy