Supporting Statement B
Questionnaire and Data Collection Testing, Evaluation, and Research for the Health Resources and Services Administration (HRSA)
OMB Control No. 0915-0379
Extension
The purpose of collections under this generic clearance is to obtain formative information from respondents to develop new questions, questionnaires and tools and to identify problems in instruments currently in use. This clearance request is limited to formative research activities emphasizing data collection, questionnaire and toolkit development, estimation procedures and reports for internal decision-making and development purposes and does not extend to the collection of data for public release or policy formation.
It is anticipated that these studies will rely heavily on qualitative techniques to meet their objective. In general, these activities are not designed to yield results that meet generally accepted standards of statistical rigor; rather, these activities are designed to obtain valuable formative information to develop more effective and efficient data collection tools that will yield more accurate results and decrease non-response.
These research activities will generally employ non probability samples; however, some research may involve a probability sample drawn from a list-based sampling frame. Most studies would be conducted with convenience sampling. “Quota” sampling may be used to ensure that the convenience sample is broad enough to include enough persons with particular characteristics and ensure representation of the target audience and/or a reasonable degree of diversity in key demographic characteristics.
Depending on the particular project, participants may be recruited through advertising (for instance, when a project necessitates respondents with particular rare characteristics) or through places of employment or medical care when those places serve the target population. Telephone samples may be selected with random digit-dialing (RDD) techniques, or with stratified sampling of telephone area codes. Address Based Samples (ABS) may be selected from a list-based sampling frame. Over-recruiting will be used, as needed, to compensate for non-response and/or the inability to recruit a sufficient number of participants.
The following types of research activities may be employed for these formative evaluation and data collection methods and techniques under this general clearance: Surveys conducted by mail or via email which may include telephone non-response follow-up; telephone surveys; web-based surveys; focus groups; automated data collection (for example, testing of database software, computer-assisted personal interviewing [CAPI], computer assisted telephone interviewing [CATI], audio computer-assisted self-interview [ACASI] or other automated technologies); and cognitive testing.
Professionally recognized procedures will be followed in each information collection activity to ensure high quality data. Examples of these procedures are likely to include
Monitoring by supervisory staff of a certain percent of telephone interviews;
Conducting cognitive interviewing techniques, including think-aloud techniques and debriefings.
Data-entry from mail or paper-and-pencil surveys will be computerized through scannable forms or checked through double-key entry;
Observers will monitor focus groups, and focus group proceedings will be recorded; and
Data submitted through web-based surveys will be subjected to statistical validation techniques to ensure accuracy (such as disallowing out-of-range values).
Each request under this umbrella clearance will specify the specific procedures to be used.
Participation will be fully voluntary, and non-participation will have not affect eligibility for, or receipt of, future HRSA health services research activities or grant awards, recruitment or participation. Specific testing and evaluation procedures will be described when we notify OMB about each new request. Consent procedures will be customized for each information collection activity, but will include assurances of confidentiality and the legislative authority for the activity. If the encounter is to be recorded, the respondent’s permission to record will be obtained before beginning the interview.
Recruitment – Respondents will be recruited by means of advertisements in public venues or through techniques that replicate prospective data collection activities that are the focus of the project. For instance, a survey on physician communication, designed to be administered following an office visit, might be pretested using the same procedure. Each submission to OMB will specify the specific recruitment procedure to be used.
Screening - When screening is required (e.g., quota sampling), the screening will be as brief as possible and the screening questionnaire will be provided as part of the submission to OMB.
Collection methods - The particular information collection methods used will vary, but may include the following
Individual in-depth interviews – In-depth interviews will commonly be used to ensure that the meaning of a questionnaire or strategy is understood by the respondent. When in-depth interviewing is used, the interview guide will be provided to OMB for review.
Focus groups – Focus groups will be used to obtain insights into beliefs and understandings of the target audience early in the development of a questionnaire or tool. When focus groups are used, the focus group discussion guide will be provided to OMB for review.
Expert/Gatekeeper review of tools – In some instances, tools designed for patients may be reviewed in-depth by medical providers or other gatekeepers to provide feedback on the acceptability and usability of a particular tool. This would usually be in addition to pretesting of the tool by the actual patient or other user.
Record abstractions - On occasion, the development of a tool or other information collection requires review and interaction with records rather than individuals.
Multimode pretesting – In some instances, the proposed pretesting will constitute fielding the intended questionnaire in multiple modes (self-administered web-based, self-administered mail based, CATI, etc.) in order to assess measurement error and usability across modes.
“Dress rehearsal” of a specific protocol - In some instances, the proposed pretesting will constitute a walkthrough of the intended data collection procedure. In these instances, the request will mirror what is expected to occur for the larger scale data collection.
The design of testing and evaluation procedures will include approaches to better understanding how to maximize response rates and quality of collected information, while retaining the voluntary nature of the effort.
The “default” for much of this work will be not to offer incentives. However, when deemed necessary, for instance, when individuals are recruited to travel to a cognitive interviewing site, respondents may be eligible for an incentive of $20-40. This amount is designed to remunerate for travel associated costs as well as child-care and related expenses in addition to encouraging travel. If higher remunerations are be requested due to documented difficulties in identifying eligible participants, they will be evaluated on a case-by-case basis for particularly difficult recruitments. For example, focus group remuneration might occasionally rise higher than the $20-$40 range but never above $100. It is sometimes important to offer remuneration sufficient to attract the full range of needed respondent types, and specifically across different modes. Inadequate respondent recruitment limits the effectiveness of the questionnaire evaluation. Requests and justification for remuneration will be included in each individual collection submission.
Before each information collection is implemented, HRSA will pilot test the instrument(s) and method of data collection. Lessons from the pilot test will be identified and changes will be incorporated into the method and instrument, as needed. All pilot tests will involve no more than nine individuals unless OMB clearance is sought for more than nine.
Whenever it is methodologically sound and appropriate to do so, programs will obtain input from statisticians regarding the development, design, conduct, and analysis of the testing and evaluation methods planned for the data collection and estimation procedures. This statistical expertise will be available from HRSA statisticians/contractors. Technical assistance in data collection and estimation procedures may be sought, in some cases, outside of HRSA from experts in other Federal Agencies or outside of the government.
| File Type | application/vnd.openxmlformats-officedocument.wordprocessingml.document |
| File Title | Instructions for writing Supporting Statement B |
| Author | Jodi.Duckhorn |
| File Modified | 0000-00-00 |
| File Created | 2021-01-14 |