Awareness and Utility Survey Justification

Request for Approval under the “Generic Clearance for the Collection of Routine Customer Feedback” (OMB Control Number: 3137-0081)

TITLE OF INFORMATION COLLECTION: Awareness and Utility Survey of IMLS Data Products for the Public Libraries Survey (PLS) and State Library Administrative Agency (SLAA) Survey 

PURPOSE:

The Institute of Museum and Library Services (IMLS) is conducting a study to better understand the needs of our audiences and identify opportunities to improve our data products and services. As part of this study, we are conducting a survey to determine which audience groups are aware of the IMLS data products derived from the annual Public Libraries Survey (PLS) and the biennial State Library Administrative Agencies (SLAA) survey.

These IMLS data products include the data sets and associated datafile documentation, an online search and query tool, statistical tables, and reports. The survey will determine participants’ awareness of the products, their interest in using the products, perceived usefulness of the products, and the challenges they may face in using the products. The data from this survey will be used to improve and enhance the data products over the next five years and guide dissemination efforts to increase use of the products. IMLS is working with our contractor RTI International to conduct this survey.

DESCRIPTION OF RESPONDENTS:

Respondents are library practitioners, library administrators, journalists interested in libraries and other public institutions, and social science researchers who may or may not be aware of the IMLS data products but who might benefit from using the products in their work.

TYPE OF COLLECTION: (Check one)

[ ] Customer Comment Card/Complaint Form [X] Customer Satisfaction Survey

[ ] Usability Testing (e.g., Website or Software [ ] Small Discussion Group

[ ] Focus Group [ ] Other: ______________________

CERTIFICATION:

I certify the following to be true:

1. The collection is voluntary.

2. The collection is low-burden for respondents and low-cost for the Federal Government.

3. The collection is non-controversial and does not raise issues of concern to other federal agencies.

4. The results are not intended to be disseminated to the public.

5. Information gathered will not be used for the purpose of substantially informing influential policy decisions.

6. The collection is targeted to the solicitation of opinions from respondents who have experience with the program or may have experience with the program in the future.

Name: Matt Birnbaum

To assist review, please provide answers to the following question:

Personally Identifiable Information:

1. Is personally identifiable information (PII) collected? [ X ] Yes [] No

We will collect email addresses in order to process incentives (Amazon gift cards sent via email). This PII will not be entered into a system of records and will be kept separate from participant responses. It will be stored in a folder with access restricted to the researchers and will be immediately destroyed at the end the project. Emails of participants will be deleted immediately after we send the digital gift card incentive.

2. If Yes, will any information that is collected be included in records that are subject to the Privacy Act of 1974? [ ] Yes [X ] No Not Applicable

3. If Yes, has an up-to-date System of Records Notice (SORN) been published? [ ] Yes [ ] No Not Applicable

Gifts or Payments:

Is an incentive (e.g., money or reimbursement of expenses, token of appreciation) provided to participants? [X] Yes [ ] No

Participants who complete the survey will be offered a $15 Amazon gift card as an incentive and token of appreciation for their time. As participants often have competing demands for their time, incentives are used to encourage participation, show respect and gratitude, and provide compensation for their time and contribution. While research may require little skill by participants, it takes time, effort, and endurance.¹ Numerous empirical studies have shown that incentives can significantly increase response rates. Even small amounts of money have been shown to increase response rates for surveys.^{2 3 4} Low or no incentives can also potentially result in a difficult and lengthy recruitment process. This can cause delays in initiating data collection, which can lead to overall timeline delays and increased costs to the government. In our experience with multiple research studies, in situations where we have not offered incentives, time to recruit increased, as did our no-show rate. In fact, recruitment organizations have a standard practice of offering incentives, as participating in research takes people away from their work; thus, they should be paid, unless the research activity is considered part of their normal job responsibilities.^{5 6}

Additionally, since the participants will include both those who already use IMLS data products as well as those who do not, such as journalists, recruitment of this latter group poses additional challenges, necessitating the use of an incentive. While current users may feel more intrinsically motivated to provide input on the data products – either personally or professionally, which may bias their awareness of the products and their perceptions of their utility; providing incentives – or extrinsic motivators – can help us recruit a more diverse set of participants, including those who are not already interested in the IMLS website and data products.⁷

BURDEN HOURS

Category of Respondent	No. of Respondents	Participation Time	Total Minutes	Total Burden Hours	Value of Time8
Individuals - journalists	30	15 minutes	450	7.50 hours	$166.88
Individuals – library staff	20	15 minutes	300	5.00 hours	$143.05
Individuals - researchers	25	15 minutes	375	6.25 hours	$241.06
State or territory administrative staff (library administrators)	30	15 minutes	450	7.50 hours	$214.58
Federal government staff (researchers)	5	15 minutes	75	1.25 hours	$48.21
Totals	110		1,650	27.50 hours	$813.78

FEDERAL COST: The estimated annual cost to the Federal government is $43,432

Item/Activity	Details	$ Amount
IMLS oversight of contractor and project	Estimated labor hours.	$10,000
Recruitment, data collection including incentives, materials, online platform hosting, analysis, travel, overhead	Labor hours and Other Direct Costs	$33,432
Annual Total		$43,432

If you are conducting a focus group, survey, or plan to employ statistical methods, please provide answers to the following questions:

The selection of your targeted respondents

1. Do you have a customer list or something similar that defines the universe of potential respondents and do you have a sampling plan for selecting from this universe? [X] Yes [ ] No

If the answer is yes, please provide a description of both below (or attach the sampling plan)? If the answer is no, please provide a description of how you plan to identify your potential group of respondents and how you will select them?

IMLS has a list of stakeholder organizations, universities, professional associations, journalists, researchers, and other individuals that are well connected to the key audience segments for this study. RTI will reach out to members of this list to disseminate the survey, as well as reach out directly to researchers and journalists who use the IMLS data products. In addition, RTI will also leverage social media posts and advertisements on Facebook and direct messages on LinkedIn that target the job titles and organizations of the audience segments to reach those who many not already be aware of the IMLS data products.

The survey will be distributed to the IMLS-identified stakeholders and to RTI-identified researchers and journalists with an instruction to share the link with others (snowball sampling). Using a screener, RTI will implement a quota sample to gather at least 100 and no more than 110 total responses distributed across the respondent categories as shown in the “Burden Hours” table, above.

Skip-logic in the online survey is used to limit the questions presented to respondents to those to they are likely to be able to answer based on their current level of experience with IMLS data products. In addition, to minimize burden on respondents and limit the time commitment to 15 minutes, we will only ask follow-up questions about three products. If a respondent responds that they currently use more than 3 products, we will ask them questions about 3 products, based on the order in which they are ranked by priority by IMLS (where priority is places data products hypothesized to be less frequently used at the top of the list). We anticipate that the number of respondents who use more than 3 products will be extremely small (less than 10 people).

Administration of the Instrument

1. How will you collect the information? (Check all that apply)

[X] Web-based or other forms of Social Media

[ ] Telephone

[ ] In-person

[ ] Mail

[ ] Other, Explain

2. Will interviewers or facilitators be used? [ ] Yes [X] No

Please make sure that all instruments, instructions, and scripts are submitted with the request.

Instructions for completing Request for Approval under the “Generic Clearance for the Collection of Routine Customer Feedback”

TITLE OF INFORMATION COLLECTION: Provide the name of the collection that is the subject of the request. (e.g., Comment card for soliciting feedback on xxxx)

PURPOSE: Provide a brief description of the purpose of this collection and how it will be used. If this is part of a larger study or effort, please include this in your explanation.

DESCRIPTION OF RESPONDENTS: Provide a brief description of the targeted group or groups for this collection of information. These groups must have experience with the program.

TYPE OF COLLECTION: Check one box. If you are requesting approval of other instruments under the generic, you must complete a form for each instrument.

CERTIFICATION: Please read the certification carefully. If you incorrectly certify, the collection will be returned as improperly submitted or it will be disapproved.

Personally Identifiable Information: Provide answers to the questions. Note: Agencies should only collect PII to the extent necessary, and they should only retain PII for the period of time that is necessary to achieve a specific objective.

Gifts or Payments: If you answer yes to the question, please describe the incentive and provide a justification for the amount.

BURDEN HOURS:

Category of Respondents: Identify who you expect the respondents to be in terms of the following categories: (1) Individuals or Households; (2) Private Sector; (3) State, local, or tribal governments; or (4) Federal Government. Only one type of respondent can be selected per row.

No. of Respondents: Provide an estimate of the Number of Respondents.

Participation Time: Provide an estimate of the amount of time (in minutes) required for a respondent to participate (e.g., fill out a survey or participate in a focus group)

Burden: Provide the Annual burden hours: Multiply the Number of Respondents and the Participation Time then divide by 60.

FEDERAL COST: Provide an estimate of the annual cost to the Federal government.

If you are conducting a focus group, survey, or plan to employ statistical methods, please provide answers to the following questions:

The selection of your targeted respondents. Please provide a description of how you plan to identify your potential group of respondents and how you will select them. If the answer is yes, to the first question, you may provide the sampling plan in an attachment.

Administration of the Instrument: Identify how the information will be collected. More than one box may be checked. Indicate whether there will be interviewers (e.g., for surveys) or facilitators (e.g., for focus groups) used.

Submit all instruments, instructions, and scripts are submitted with the request.

1Grady, C. NIH, Department of Clinical Bioethics Clinical Center. Ethical and practical considerations of paying research participants. Accessed January 8, 2021 from: https://www.niehs.nih.gov/research/resources/assets/docs/ethical_and_practical_considerations_of_paying_research_participants_508.pdf

2 Singer E, Ye C. The Use and Effects of Incentives in Surveys. The ANNALS of the American Academy of Political and Social Science. 2013;645(1):112-141. doi:10.1177/0002716212458082

3 Göritz, A. (2006). Incentives in Web Studies: Methodological Issues and a Review.

4 National Research Council 2013. Nonresponse in Social Science Surveys: A Research Agenda. Washington, DC: The National Academies Press. https://doi.org/10.17226/18293.

5 Boyd, C. (2020). The Ultimate Guide to User Research Incentives. User Interviews. Accessed on January 8, 2021 from: https://www.userinterviews.com/blog/the-ultimate-guide-to-user-research-incentives

6Sigritz, A. (2016). Paying Incentives for Federal User Research. GSA, DigitalGov. Accessed on January 8, 2021 from: https://digital.gov/2016/02/10/paying-incentives-for-federal-user-research/

7 18F. Fundamentals: Foundational methods for practicing design research. Accessed January 6, 2021 from: https://methods.18f.gov/fundamentals/incentives/

8 Value of time (total burden hours) used wage rate data from the Bureau of Labor Statistics, “May 2019 National Occupational Employment and Wage Estimates, United States” accessed online at: https://www.bls.gov/oes/current/oes_nat.htm#00-0000 (Access date: 6 January 2021). “News analysts, reporters, and journalists” (27-3023) had a median wage of $22.25/hour; “Librarians and media collections specialists” (25-4022) earned a median of $28.61/hour; and researchers, are likely to be “Social scientists and related workers” (19-3000) with a median wage of $38.57/hour. The value of time for state or territory administrative staff (library administrators) was estimated using the median wage for Librarians and media collections specialists, while that for federal researchers was estimated using the median wage for Social scientists and related workers.

7