Evaluation of the Communities for Immunity (C4I) PROJECT
SUPPORTING STATEMENT A
The Association of Science and Technology Centers (ASTC) has contracted SRI International (SRI) to conduct an independent evaluation of the Communities for Immunity (C4I) Project. The C4I Project, administered by ASTC in partnership with the American Alliance of Museums (AAM) and the Network of the National Library of Medicine (NLM) and supported by the Institute of Museum and Library Services (IMLS) and the Centers for Disease Control and Prevention (CDC), supports libraries, museums, and their partners to engage local communities with the aim of increasing COVID-19 vaccine confidence and, ultimately, of improving community vaccination rates. The C4I Project also aims to increase libraries’ and museums’ organizational capacity to partner in addressing critical national and local issues.
This new emergency clearance request is for the study design and instruments that SRI proposes to use in evaluating C4I Round 2 award activities. Most of the proposed instruments are the same as those used in the Round 1 data collection, which OMB approved on November 15, 2021, under OMB Control Number 3137-0129. In consultation with OMB, IMLS has submitted two separate clearance requests to accommodate two sequential timelines and some variation in approach. Data collection activities for Round 1 activities were scheduled for November through December, 2021, and those for Round 2 activities are scheduled for January through April, 2022. Very brief surveys were appropriate for all Round 1 projects, which had a lower per-project funding ceiling, whereas the higher per-project funding ceiling in Round 2 is likely to result in more in-depth project activities that warrant slightly expanded data collection efforts.
This proposed data collection fills an urgent need to understand promising strategies for mitigating vaccine hesitancy and improving vaccination-seeking behavior and vaccination rates in communities around the country. More than 800,000 Americans have died of COVID-19-related causes since February 2020 and, despite widespread availability of the vaccine in the US, only 61 percent of Americans of all ages nationwide are fully vaccinated.1 In a 2020 survey measuring potential acceptance of a COVID-19 vaccines, just two-thirds (66%) of Americans said that they would definitely or probably get vaccinated when a COVID-19 vaccine became available; subsequent surveys have shown that vaccine hesitancy is a key reason for this choice.2 Sources of vaccine hesitancy included concerns that fast vaccine approval could reflect lowered quality standards, concerns about the newness of mRNA vaccine development, and misinformation circulated on social media—all despite evidence that vaccines are safe to use.3
As COVID-19 continues to cause severe illness and death, increasing vaccination rates in the U.S. remains a top national priority. It is also vital to build confidence among parents and caregivers in the recently approved vaccine for children ages 5-11.
The proposed evaluation will describe engagement strategies that libraries, museums, and cultural institutions and their partners used in funded projects, and participant self-reports of changes in vaccine confidence, including regarding the new vaccine for 5-11 year-olds.
In Round 2, SRI proposes to field slightly longer awardee/partner and participant surveys for people associated with a subset of projects involving more intensive or longer activities, and to field the already approved shorter surveys in other projects. SRI also plans to use the already approved interview protocols in Round 2.
The project’s short timeframe reflects the urgency of mitigating COVID-19 impacts, particularly in disproportionately affected communities where the project especially aims to make a difference. IMLS is seeking approval for this collection by January 25, 2022, in order to accelerate the timeline for sharing findings among IMLS stakeholders and with broader national audiences.
The proposed evaluation is a descriptive study that aims to characterize the funded projects; describe participant self-reported vaccine confidence and plans to seek vaccines; describe participants attitudes and understanding regarding the role of museums and libraries in their communities; characterize awardee satisfaction with C4I; and describe project leader reports of attitudes, capacity, knowledge, and strategies for leading similar efforts.
SRI will aim to minimize respondent burden by maximizing use of available data (i.e., document review, analysis of publicly available data and project administrative data), and by aligning awardee reporting forms with evaluation and performance measurement needs. New data collection activities include: (1) a survey of awardees and partners, (2) a survey of participants in funded activities, and (3) interviews with a subset of awardees, partners, and participants.
This second round of evaluation data collection proposes to use short surveys with Round 2 awardees/partners and project participants in projects with lower funding levels and/or that offer only brief or less intensive participant experiences, and to use slightly longer surveys with those Round 22 awardees/partners and participants in projects with higher funding levels and/or that offer longer or more intensive participant experiences.
Drawing on its database of Round 2 awards, SRI will identify the projects with the highest funding amounts and consider the types of project activities offered to select approximately 20 of the total 52 projects in which to field slightly longer versions of the awardee/partner and participant surveys. Specifically, SRI will consider administering longer surveys for all projects funded at $25,000 or more. In addition to funding levels, researchers will also consider projects with features such as (1) opportunities to ask questions or interact with interpreters or other staff, community leaders, and/or experts; and/or (2) opportunities for multi-faceted participation, such as activities for both children and adults, options to participate on-site and then continue learning at home with an additional take-away activity; or (3) opportunities to attend an event or exhibit and then get a shot at a vaccine clinic. Lastly, SRI will review the projects identified by applying the first two criteria and consider the degree to which the target audiences are representative of the demographic range reflected in the Round 2 awards overall. We estimate that from these three considerations, we will identify about 20 projects in which to administer the longer versions of the surveys.
The shorter awardee/partner surveys and participant surveys are the same as those OMB already approved for the Round 1 data collection (OMB Control Number 3137-0129). The longer versions of the surveys will require approximately five minutes more than the shorter versions to complete. The longer awardee/partner survey will require approximately 15 minutes, as opposed to 10; the longer participant survey will require approximately 10 minutes, as opposed to 5. Survey introductory and consent language is provided in Appendix A. The longer and shorter awardee/partner surveys and longer and shorter participant surveys are provided in Appendices B and C. A matrix illustrating how survey and interview questions map to evaluation objectives is provided in Appendix D. All appendices are presented in the Supplementary Documents file.
Both surveys will address community engagement strategies used, perceived project reach, satisfaction with the C4I Project, respondents’ perceptions of project successes and challenges, and perceptions of library and museum awardees as trusted sources of information. The surveys will also feature questions to elicit respondents’ views about their library’s or museum’s capacity to continue to engage target communities around critical issues. The longer survey additionally addresses perceived utility of resources provided in the C4I online community and whether and how project leads used and/or adapted resources, and views regarding partner relationships. It also features an open-ended item about suggestions for project improvement or for others’ undertakings similar efforts.
These surveys will gather information about the types of activities in which respondents engaged; whether people learned about COVID-19 vaccines for people aged 12 and older and/or for children aged 5-11; perceptions of vaccine confidence and plans to seek vaccines; attitudes towards museums and libraries as trustworthy sources of information about public health and/or as trustworthy community resources; and race and ethnicity. The longer participant survey additionally addresses their attitudes regarding awardee organizations and perceptions of awardee organizations as trusted sources of information. SRI will work with awardees to administer participant surveys locally when their project activities end or as soon afterward as is feasible, pending OMB approval. The longer survey also has an item about willingness to be contacted for a follow-up interview and, for those who indicate yes, space to enter a name, email address, and telephone number.
SRI will conduct interviews with awardees, partners, and participants associated with five Round 2 projects. Interviews will require approximately 30 minutes (participants) or 45 minutes (awardees/partners) and will be semi-structured, using guiding language, questions, and prompts. Interviews will be tailored based on trends in survey responses (as timing allows) and based on interviewees’ statements in the moment to elicit the most useful insights. Interview consent forms are in Appendix E. Interview protocols are in Appendix F. Email communications to invite interview participation and to follow up are in Appendix G.
Library and museum awardee and partner interviews will gather respondents’ perspectives regarding how the project aligns with community needs, project implementation successes and barriers, changes in perceptions of their capacity to build relationships with the community and engage the community as a trusted source of information, and the role of museums and libraries in supporting community well-being. Participant interviews will collect detailed information about engagement with C4I activities and resources; perceptions of the project activities; whether, why, and how participants do or do not feel participation influenced their vaccine confidence and vaccine plans; prior and current sources of vaccine hesitancy; and trustworthiness of the project lead organizations as sources for information on issues important to them.
This evaluation is a descriptive study that aims to provide insights into C4I project management and implementation, and into project leader and participant self-reports of their attitudes and beliefs related to vaccines and the role of library of museums in their communities. This is not an audit of awardees or their individual performances. The evaluation will be guided by these primary objectives:
Characterize C4I funded projects (including in groupings of similar target audiences, organizational partnering strategies, and engagement strategies).
Describe participants’ perceptions of vaccine confidence and plans to seek vaccines.
Describe participants’ perceptions of attitudes towards and understanding of awardee organizations and their partners as trusted sources for timely, relevant information and community resources.
Characterize awardee satisfaction with the C4I program, in terms of its alignment to their organizational and community needs, staff capacity, and their views of project success.
Describe project leaders’ reported attitudes, capacity, knowledge, and strategies for undertaking similar efforts for community improvement.
The evaluation is intended to provide IMLS with descriptive information of C4I project activities and participant and project leader perceptions to inform the cooperative efforts of IMLS and ASTC to strengthen the project and facilitate an online community among awardees; and enable IMLS and ASTC to effectively communicate about project activities with awardees, policymakers, and other audiences.
IMLS anticipates that numerous, diverse stakeholders will be interested in the evaluation findings. Audiences include other agency and federal partners; cooperator ASTC and its partner associations and their respective memberships; and other funders, project managers, and community health advocates at federal, state, and local levels interested in community engagement efforts that aim to increase COVID-19 vaccination rates and/or to augment the role of libraries and museums as trusted resources for improving community well-being. The study will result in a series of reports and briefings primarily developed for grant funders and administrators as well as other products to be shared widely.
IMLS takes very seriously its responsibility to minimize burden on respondents and worked with SRI to design this evaluation with that goal in mind. SRI will use information technologies to aid in characterizing Round 2 projects, and to maximize use of publicly available information, streamline efforts and increase security for administering surveys and conducting interviews, as well as to widely disseminate evaluation results.
All administrative documents are in a secure digital format, accessible to only award administrator ASTC and the SRI evaluation team. SRI is reviewing its database of Round 2 awards to characterize projects in terms of awardee and partner organization types, community engagement strategies and activities, timing of activities, and target populations. This review helped inform instrument development (e.g., by ensuring questions are pertinent to awardee/partner and participant experiences) and avoid collection of redundant information. SRI will continue to use a variety of technologies and methods to maximize data collection efficiency and minimize respondent burden.
First, SRI will conduct Internet searches to maximize use of extant, publicly available data. These data may include: vaccine hesitancy data collected by Carnegie Mellon University in concert with Facebook and provided at the ZIP-Code level by IHME and the COVID Collaborative; county-level CDC vaccinations and vaccine hesitancy estimates based in part on results from the Census Household Pulse survey; and locally generated data on vaccinations by census tract, as available, using descriptions of activity locations from awarded applications to identify relevant census tracts. In addition, SRI conducted a brief document scan to identify previously validated survey items and to gather the state of understanding and evidence related to community efforts to influence vaccine hesitancy and confidence.
SRI also plans to conduct analysis of social media activity (e.g., use of project hashtags) and to use Google Analytics and reporting from Higher Logic (the platform that powers the online community hosted by ASTC) to gain insights into uptake of awardee-provided resources and project resources provided by IMLS and the CDC. SRI can program in R automated searches of social media hashtags to be able to assess the prevalence of participants’ vaccine-related attitudes expressed on these platforms. SRI will work with projects planning to use hashtags to ensure the hashtags are unique to support this analysis. SRI will thus be able to associate hashtag use with specific projects but will not gather information about any individual social media users.
Similarly, SRI will review website data in Google Analytics such as numbers of page views over time and numbers of click-throughs to collections of vaccine resources, and data in captured by Higher Logic including numbers and types of resource downloads, levels of community members and levels of community activity (page views, numbers of discussion posts, engagement with discussions). SRI will review these data only in the aggregate, without regard to individual users. SRI will also review discussion posts to learn pertinent details about changes in project plans and note project successes and challenges shared with the community as part of evaluating project implementation. All these steps will ensure new data collection is focused on aspects of the evaluation for which data are not already available.
For all survey data collection, SRI will use Qualtrics, a FedRAMP secure online survey platform. Qualtrics is a user-friendly, customizable tool that enables detailed analyses of survey responses. Qualtrics also enables the use of skip logic that will save respondents time by presenting them only with the questions relevant to them. Administering surveys online will also minimize data entry error and the need to follow up with respondents.
To administer the awardee/partner surveys, SRI will send unique links to each respondent via email. This strategy will enable automated, targeted follow-up (i.e., sent only to those who have not completed the survey) and to match survey data and interview respondents. Email texts for awardee/partner survey administration and follow-up are in Appendix H.
To help awardees administer participant surveys, SRI will provide awardees with QR codes on flyers to display at their events, along with guidance on displaying the flyers and engaging participants to aim their phone cameras at the code to automatically open the survey. Awardees can also use their own phones or a tablet belonging to their organization to assist participants less comfortable with technology or who did not bring a smartphone to the event in completing the survey to minimize any potential bias introduced by the use of QR codes. In addition, SRI will provide the survey in translation as needed and as the evaluation budget allows. (As examples, SRI provided Round 1 participant surveys in French, Haitian Creole, Russian, and Spanish, as well as in English.) SRI will provide awardees with the QR code linking to the survey, and participants will select the preferred language at the start of the survey. Guidance to help project leads collect participant data is located in Appendix I.
SRI proposes to provide each awardee with an individualized QR code paired with a unique link to enable the research team to link participant responses with C4I projects. SRI does not intend to collect personal information from participants, partly due to the sensitive nature of asking about vaccine status and attitudes, apart from asking for the name and contact information for those who indicate willingness to participate in follow-up interviews in the longer participant survey. SRI will not have opportunities for survey follow-up with most participants after they leave a project-funded event or activity (with the exception of those who indicate willingness to be interviewed) because the surveys will typically be administered as people leave an event or exhibit.
Interviews will be conducted virtually by telephone or using Zoom.gov virtual conferencing software—the most secure version of Zoom available—in English, French, Spanish. Interviews will be recorded with participants’ consent. Interviews will be automatically transcribed using a Zoom feature, and professionally transcribed as needed in cases of inadequate automated capture. Survey and interview data will be stored securely in SRI’s Qualtrics platform and in SharePoint on SRI’s secure servers. Data will be accessible only by the research team. Awardees will be provided with SRI team members’ telephone numbers and a central email address ([email protected]) to use, should they need assistance.
Lastly, SRI will leverage technology to share evaluation results as widely as possible, including via social media, on the project and SRI websites and with the community of practice, and via blog posts and webinars. IMLS will coordinate with the CDC, ASTC, and other association stakeholders to help SRI share these nationally relevant results well beyond the awardee community.
The proposed evaluation is a descriptive study of a unique awardee community that was selected for a one-time, short-term project. SRI will take into consideration that a very small number of Round 1 awardees received Round 2 awards to expand and/or adapt their initial efforts. These project awardees will still be asked to complete surveys specific to their Round 2 work. These projects’ longer involvement in C4I may warrant consideration in sampling for interviews, depending on the nature of project activities.
Some organizations included in the project lead survey data collection are small nonprofit organizations and businesses (e.g., small museums, libraries, and/or community-based clinics or other partners with limited staff capacity). To minimize burden on these entities, the evaluation will provide clear, concise instructions and ensure the data collection process limits requests for personnel time and the need to collect additional, follow-up information. The instruments are designed to minimize respondent burden.
Surveys will be administered only once, are brief (estimated ten minutes to complete the shorter awardee/partner survey, 15 minutes to complete the longer one; five minutes to complete the shorter participant survey, 10 minutes to complete the longer one), are simply written and free of jargon or inappropriate assumptions about respondents’ background knowledge or experience and include primarily closed-response questions that do not present cognitive challenges to complete.
Project leads and participants associated with eight projects will be additionally asked to participate one time in an interview to last no more than 45 minutes (project leads) or 30 minutes (participants). Interviews will be voluntary, conducted virtually, and scheduled at the convenience of the interviewee.
The research team plans to collect survey data specific to Round 2 project activities from respondents only once and interview data from a small sample of respondents only once. The study will provide IMLS with insights into C4I project implementation and project lead and participants’ self-reported perceptions of project strategies and activities and attitudes about COVID-19 vaccines.
Conducting the collection less frequently (essentially foregoing the collection) would impede the evaluation’s ability to provide meaningful insights and to identify and broadly share promising practices for reducing vaccine hesitancy and increasing vaccination rates. Ultimately, not collecting and sharing study results could limit stakeholders’ ability to read descriptive findings that may inform local efforts. Further, without the data collection, IMLS, CDC, ASTC, and other partners would lack information on awardee satisfaction with C4I and on participant and awardee self-reported perceptions of project implementation and activity-related attitudes and understanding.
None of these special circumstances applies.
This is an emergency, one-time, new data collection. The evaluation design and data collection instruments developed by SRI were reviewed by SRI senior methodologists and SRI’s Institutional Review Board. ASTC project administrators also reviewed and provided input on the study design and methods, including survey instruments and interview protocols. The research team has reviewed and considered all input from these groups and revised the design and methods as appropriate.
Input on survey instruments led SRI to reword certain questions and to delete items that gathered information already available in project administrative data. SRI staff also tested the programmed surveys (with skip logic) to ensure they could be completed within stated timeframes. In addition, surveys translated into other languages will reviewed by native or fluent speakers at SRI, when possible, to assure translations are clear and of high quality.
Interview protocols were reviewed by two SRI researchers outside the evaluation team with community engagement and public health expertise. Based on input, SRI simplified and consolidated a few questions and improved alignment of questions to evaluation objectives.
The evaluation team will provide a $50 gift card to project participants for interviews, as these require a greater time commitment (up to 30 minutes each). SRI does not plan to provide incentives for awardee or partner interviews, nor for awardee/partner or for participant survey respondents. This approach follows a practice SRI has used successfully to increase participation in data collections for other federally funded research and evaluation studies, including for the U.S. Department of Education and the U.S. Department of Justice. The use of incentives in this study is also approved by SRI’s Institutional Review Board.
In guidance for awardees on administering surveys locally, SRI notes that all responses to the short participant survey are anonymous, and that people who complete the longer participant survey can also choose to do so anonymously or can choose to provide a name and contact information if they indicate willingness to voluntarily participate in a follow-up interview. The guidance also states that PII will be stored separately from survey data files and destroyed once interviews are complete. It is important to underscore the anonymous, confidential nature of participant survey responses to alleviate concerns about any possible repercussions or consequences for sharing views on a sensitive topic. Respondents to the longer participant survey and partner survey who provide their name and contact information for possible selection for interviews will be assured researchers will keep their information secure and confidential on password-protected servers accessible only by members of the research team. In addition, SRI will use information provided only for the stated purpose and will anonymize and destroy the PII of people who agreed to be interviewed once interviews are complete.
All potential interview respondents will receive written communication describing the purpose of the study and procedures for maintaining participant anonymity, confidentiality, and data security. (Interview consent forms specify that data are kept in secure, password-protected files, participants’ names are not shared outside the research team, and responses will appear only in summaries of all interviews without anyone identifying information.) Potential respondents also receive the plan for data collection prior to participating in the interview. All written communications will explain that the interviews will not commence without the verbal consent of the respondent(s) to participate in the interview and be audio-recorded. Interviewees who wish to participate but not be recorded will be advised they may still do so. Survey and interview respondents will be provided the telephone number and email address of the evaluation principal investigator and SRI’s Institutional Review Board to use, should they have questions or concerns.
Because of the potentially sensitive nature of the subject matter, the study team plans to conduct only individual interviews unless interviewees request to participate in a small-group interview (please see a potential scenario for a small-group interview request in the response to A11 below). The explanation of the consent process will assure interviewees that the study will report on information shared only in the aggregate and will not publicly reveal any information that could lead to an interviewee being personally identified. In addition, all written communication will indicate that researchers will take the steps delineated here to safeguard any confidential information provided. The description of the study will also advise respondents that the study team may include direct quotes in reports but that identifying information will not accompany quotes.
In the event that a respondent withholds consent for audio recording, the study team will make arrangements for a second data collector to be present in the interview to take near-verbatim notes on a laptop computer. So that we will know to include a second data collector in the interview, we will ask interviewees whether they consent to audio-recording the interviews. Should they not consent to audio-recording, we then will request consent for a second data collector to take notes as part of the interview process.
Safeguards to protect the privacy and anonymity of all respondents include the following:
All team members will participate in a training that will cover procedures for assuring participant anonymity.
The study team will provide secure environments for all data collected for the study.
The study team will immediately anonymize all data collected during the study that can potentially be linked to individual respondents.
Only authorized members of the study team will have direct access to the safeguarded study databases.
The team will not share data obtained in this research with any entity or individual other than ASTC, IMLS, and CDC.
Datasets provided by SRI to ASTC and IMLS at the end of the study will not contain any personally identifying information (PII) – such as name or address of respondents or their organizational affiliation – that could permit disclosure or identification of respondents, directly or by inference. SRI will destroy all PII at the end of the study.
The survey instruments and interview protocols have been reviewed and approved by SRI’s Institutional Review Board (IRB) prior to initiating any evaluation activities. SRI’s IRB operates according to the Common Rule on the Protection of Human Subjects found in Title 45 of the Code of Federal Regulations, Part 46 (45 C.F.R. 46). The information requested under this collection is protected and held private in accordance with 42 U.S.C. 13-6, 20 C.F.R. Parts 401 and 402, 5 U.S.C. § 552 (Freedom of Information Act), 5 U.S.C. § 552a (Privacy Act of 1974) and OMB Circular No. A-130. The IRB submission included the following information:
The purpose of the data collection and data collection methods being used
The respondent populations and how they will be identified and accessed
Whether the data will be anonymous, confidential, or neither, and if the data are confidential or neither, explanation of why identifiers are necessary
How data will be stored (e.g., electronic files, hard copies)
Who will have access to data and for how long
Potential risks and burdens of the project to participants.
The C4I Project aims to support libraries, museums, and their community partners in improving vaccine confidence in projects’ target communities. Understanding participant self-reports of vaccine confidence and vaccine plans is fundamental to describing their experiences in the funded activities. We recognize that attitudes and preferences regarding personal health decisions—such as whether or not to get vaccinated or have one’s children vaccinate—are sensitive matters, but survey instruments do contain items about vaccination status and plans in alignment with evaluation and project goals. SRI also plans to ask similar questions in interviews.
To minimize risk associated with responding to sensitive questions, SRI’s guidance to project leads who will administer participant surveys underscores that participation is voluntary. This is echoed in the survey consent language, which also notes participants can stop at any time, and that survey responses are recorded in a manner that establishes anonymity. The participant survey does not collect any personally identifiable information (except the names and contact information of respondents who indicate willingness to participate in interviews) and responses will be aggregated in analysis. SRI will also program the Qualtrics survey so that respondents can skip questions they are not comfortable answering.
The study team will take every step to ensure that respondents feel comfortable responding to interviews and protect respondents from potential threats posed by sensitive research. For example, in some cases, participants in a given community may feel more comfortable participating in a small-group interview rather than meeting one-on-one with a researcher. They also may prefer having a trusted community member involved in the project activities present in the interview. SRI will be flexible to ensure that data collection is conducted in ways that are culturally responsive to local participants.
Exhibit 1 presents the projected burden hour estimates for completion of data collection for the awardee/partner and participant surveys as well as for the interviews with awardees/partners and participants. We assume that the cost per hour for each awardee/partner’s and participant’s time is $38.91, including wages and salaries as well as benefits, based on the Bureau of Labor Statistics estimates from June 2021 for civilian workers. We use this same estimate for both the general public as well as for project leads, because project lead staff may range from individuals with graduate degrees to interns.4
All Round 2 surveys will be single use (i.e., invited respondents will be asked to complete it only once); however, some awardee/partner and participant survey respondents will be invited to voluntarily participate in a semi-structured interview.
Surveys will be brief. The awardee/partner survey will require no more than 15 minutes (longer one) or 10 minutes (shorter one), and the participant survey will require no more than 10 minutes (longer one) or five minutes (shorter one). In total, surveys represent a burden of 167 hours at a cost of $6,497.97.
In total, the team plans to conduct 28 interviews: 15 with awardees/partners and an estimated 13 with participants associated with five sampled projects. Awardee/partner interviews will require no more than 45 minutes; participant interviews will require no more than 30 minutes. In total, interviews represent a burden of 17.75 hours at a cost of $690.65.
Exhibit 1. Estimated number of respondents and labor hours for each information collection
Respondent category |
Number of individual survey participants |
Number of grantee/partner survey participants |
Number of interviews |
Estimated response time per individual (in minutes) |
Total labor (in hours) |
Cost1 |
Surveys |
||||||
Longer awardee/partner survey |
|
54 |
|
15 |
13.50 |
$525.29 |
Shorter awardee/partner survey |
|
81 |
|
10 |
13.50 |
$525.29 |
Longer participant survey |
480 |
|
|
10 |
80.00 |
$3,112.80 |
Shorter participant survey |
720 |
|
|
5 |
60.00 |
$2,334.60 |
Interviews (5 projects) |
||||||
Interviews with awardee/partner leads |
|
|
15 (3 leads per project x 5 projects) |
45 |
11.25 |
$437.74 |
Interviews with participants |
|
|
13 (participants in activities of 5 awards) |
30 |
6.50 |
$252.92 |
Total to complete the data collection (over five months) |
|
184.75 |
$7,188.62 |
|||
1Hourly compensation rate for all civilian workers assumed to be $38.91, including wages and salaries as well as benefits, based on average estimate from Bureau of Labor Statistics (2021). USDL-21-1647. September 16. https://www.bls.gov/news.release/ecec.nr0.htm. |
||||||
A13. Estimates of Cost Burden to Respondents
There are no additional cost burdens to respondents beyond the labor cost of burden-hours described in item A12 above.
The estimated cost to the federal government for Round 2 data collection is $32,000. This total includes 194 hours contributed by 6 members of the consultant staff, ranging from $89.85 to $260.20 per hour, as well as $1,015.17 for transcription of interview audio-recordings, $2,081.64 for translation of participant consent forms and surveys, and $2,494.00 for Qualtrics survey programming and administration.
This is a new collection for emergency use, so no changes apply.
Surveys
SRI will administer two versions (longer and shorter) of two, one-time surveys, beginning for project participants once OMB approval is received and ending on a rolling basis as project activities conclude by the end of March 2022. Prior to the start of data collection, SRI will join ASTC in a virtual project kickoff meeting with awardees to explain its role, the purpose of the evaluation, and the importance of the awardee participation in evaluation data collection.
SRI will contact awardees to initiate survey administration with information reiterating the purpose of the survey and how their responses will be used. This email communication will direct respondents to the survey on Qualtrics, which will include introductory language and consent language. Respondents must indicate their consent to complete the survey. SRI will also send awardees guidance and unique QR codes that link to participant surveys to aid them in collecting survey data locally.
Survey results will be analyzed following completion of the surveys and preliminary analysis is expected to be completed within two weeks after end of the survey data collection.
Interviews
SRI will also conduct one-time virtual interviews with up to 24 awardees/partners associated with eight Round 2 projects, and up to 17 participants associated with those projects. Interviews will begin in January 2022 and end approximately two months later, with flexibility to time interviews closely following participation (participants) or just following the conclusion of project activities (awardees/partners). The start of data collection will be preceded by a kickoff with awardees in which ASTC and SRI will convey the importance of the evaluation, participation in data collection, and SRI’s role. SRI will then schedule introductory kick-off calls with groups of awardees to learn about their projects and describe and answer questions about evaluation activities.
SRI will contact selected respondents to participate in an interview with information reiterating the purpose of the interview, the use of their responses, and the types of questions that will be asked, and will invite them to participate by indicating their availability. Participant interviewees will be selected from among those people who indicate in the longer participant survey willingness to complete a follow-up interview, provide valid contact information, and are associated with the eight projects to be sampled for interview data collection. All interviews will be semi-structured and will be tailored and adapted to each respondent’s unique context and background as well as the natural flow of the interview.
Interview data will be analyzed on a rolling basis and analysis is expected to be completed within one month of the end of interview data collection (estimated April 2022).
Report and Publication
The study will result in briefings and/or reports that inform ASTC, IMLS, and CDC about strategies that awardees are using to combat vaccine hesitancy, participant and awardee self-reported perceptions related to those strategies, and any formative feedback for continuous improvement. Examples of formative feedback include information that may support ASTC with award administration or inform partner agencies of supports they could offer to awardees in the online community or for similar projects in the future. The research team will also work with ASTC to identify and develop other dissemination products (e.g., blog posts) that can be used to effectively share evaluation findings with awardees and the broader public.
All products will present data interpretation and conclusions, as well as a description of study goals, the study methodology used, and will include detailed information about study limitations. Products will make clear the descriptive nature of the study, which does not enable claims about project impacts. Products will also contain information-rich, reader-friendly graphics that communicate important, actionable information to policymakers, program administrators, and the broader research and practitioner communities. A Round 2 project timeline is in Exhibit 2.
Exhibit 2. Round 2 project timeline
Task |
Due |
Final survey and interview protocols due to IMLS |
11/29/2021 |
Round 2 grant award project activities begin |
12/3/2021 – 3/31/2021 |
Round 2 PRA request submitted to OMB for approval |
1/14/2022 |
Launch awardee and participant surveys |
Upon OMB approval |
Complete review of database with Round 2 awarded application data |
12/13/2021 |
Share target interview sample with IMLS |
3/11/2022 |
Begin scheduling interviews |
3/25/2022 |
Surveys close |
4/1/2022 |
Complete analysis of survey data |
4/18/2022 |
Conclude interviews |
4/22/2022 |
Complete analysis of interview data |
5/13/2022 |
Initial briefing with IMLS |
5/20/2022 |
Second briefing with IMLS |
6/17/2022 |
Not applicable. All data collection instruments will include the OMB data control number and data collection expiration date.
Not applicable. There are no exceptions requested.
1 Coronavirus in the U.S.: Latest Map and Case Count, New York Times, https://www.nytimes.com/interactive/2021/us/covid-cases.html (updated December 20, 2021).
2 Challenges in ensuring global access to COVID-19 vaccines: production, affordability, allocation, and deployment. Wouters, O. J., Shadlen, K. C., Salcher-Konrad, M., Pollard, A. J., Larson, H. J., Teerawattananon, Y., & Jit, M. Health Policy 397(10278): 1023-1034. https://doi.org/10.1016/S0140-6736(21)00306-8 (March 13, 2021)
3 “Is the COVID Vaccine Safe?” Lisa Maragakis, MD, MPH and Gabor David Kelen, MD, Johns Hopkins Medicine Health, https://www.hopkinsmedicine.org/health/conditions-and-diseases/coronavirus/is-the-covid19-vaccine-safe (updated November 9, 2021).
4 Employer Costs for Employee Compensation Summary, U.S. Department of Labor, Bureau of Labor Statistics (2021), USDL-21-1647, https://www.bls.gov/news.release/ecec.nr0.htm (September 16, 2021).
| File Type | application/vnd.openxmlformats-officedocument.wordprocessingml.document |
| Author | Daniel Princiotta |
| File Modified | 0000-00-00 |
| File Created | 2022-01-17 |