Users Experience Interviews Fast Track

Request for Approval under the “Generic Clearance for the Collection of Routine Customer Feedback” (OMB Control Number: 3137-0081)

T ITLE OF INFORMATION COLLECTION: Assessment of Audience Demand for Data Products for the Public Libraries Survey (PLS) and State Library Administrative Agency (SLAA) Survey: User Experience Interviews

PURPOSE:

The Institute of Museum and Library Services (IMLS) is conducting a study to better understand the needs of their audiences and identify opportunities to improve their data products and services. This request is the second of two such generic clearance requests associated with this study, the first such request being for a short survey of current and potential users of IMLS data products. This second request is for generic clearance of 24 user experience interviews to assess how various audience segments use the IMLS data products and their perceptions of utility, challenges, and opportunities for improvements.

The IMLS data products are derived from the annual Public Libraries Survey (PLS) and the biennial State Library Administrative Agencies (SLAA) survey and include the public use data sets and datafile documentation, an online search and query tool, statistical tables, and reports. The interactive user experience interviews will determine how users engage with one or more PLS and/or SLAA data products, including learning about their knowledge of the products and underlying data. Findings from these in-depth user experience interviews will complement the survey findings (referenced above as a separate generic clearance request) to enable IMLS to improve and enhance the data products over the next five years and guide dissemination efforts to increase use of the products. IMLS is working with our contractor RTI International to conduct these interviews.

DESCRIPTION OF RESPONDENTS:

Respondents are library practitioners, library administrators, journalists, and researchers that make use of one or more of the IMLS data products. In particular, participants for these interviews will be super users (who use the data products frequently), light users (who occasionally use the products), and non-users (specifically journalists).

TYPE OF COLLECTION: (Check one)

[ ] Customer Comment Card/Complaint Form [ ] Customer Satisfaction Survey

[X] Usability Testing (e.g., Website or Software) [ ] Small Discussion Group

[ ] Focus Group [ ] Other: ______________________

CERTIFICATION:

I certify the following to be true:

1. The collection is voluntary.

2. The collection is low-burden for respondents and low-cost for the Federal Government.

3. The collection is non-controversial and does not raise issues of concern to other federal agencies.

4. The results are not intended to be disseminated to the public.

5. Information gathered will not be used for the purpose of substantially informing influential policy decisions.

6. The collection is targeted to the solicitation of opinions from respondents who have experience with the program or may have experience with the program in the future.

Name: Matt Birnbaum

To assist review, please provide answers to the following question:

Personally Identifiable Information:

1. Is personally identifiable information (PII) collected? [X] Yes [ ] No

We will collect email addresses in order to process incentives (Amazon gift cards sent via email). This PII will not be entered into a system of records and will be kept separate from participant responses. It will be stored in a folder with access restricted to the researchers, which will be deleted immediately after we send the digital gift card incentive.

2. If Yes, will any information that is collected be included in records that are subject to the Privacy Act of 1974? [ ] Yes [X] No

3. If Yes, has an up-to-date System of Records Notice (SORN) been published? [ ] Yes [ ] No Not/Applicable

Gifts or Payments:

Is an incentive (e.g., money or reimbursement of expenses, token of appreciation) provided to participants? [X] Yes [ ] No

Participants who participate in the user interview will be offered a $50 Amazon gift card as a token of appreciation for their time. As participants often have competing demands for their time, incentives are used to encourage participation, show respect and gratitude, and provide compensation for their time and contribution, especially given the lengthy time commitment we are asking of the user experience interview participants (75 minutes).¹ Numerous empirical studies have shown that incentives can significantly increase response rates.^{2 3 4} Low or no incentives can potentially result in a difficult and lengthy recruitment process. This can cause delays in initiating data collection, which can lead to overall timeline delays and increased costs to the government. In our experience with multiple research studies, in situations where we have not offered incentives, time to recruit increased, as did our no-show rate. In fact, recruitment organizations have a standard practice of offering incentives, as participating in research takes people away from their work; thus, they should be paid, unless the research activity is considered part of their normal job responsibilities.^{5 6}

Additionally, since the participants will include both those who already use IMLS data products as well as those who do not, such as journalists, recruitment of this latter group poses additional challenges, necessitating the use of an incentive. Current users may feel more intrinsically motivated to volunteer 75 minutes of their time to provide input on the data products – either personally or professionally, which may bias their awareness of the products and their perceptions of their utility; providing incentives – or extrinsic motivators – can help us recruit a more diverse set of participants, who are less interested in the IMLS website and data products.⁷

BURDEN HOURS

Category of Respondent	No. of Respondents	Participation Time	Total Minutes	Total Burden Hours	Value of Time8
Individuals - journalists	6	75 minutes	450	7.50 hours	$166.88
Individuals – library staff	6	75 minutes	450	7.50 hours	$214.58
Individuals - researchers	6	75 minutes	450	7.50 hours	$289.28
State or territory administrative staff (library administrators)	6	75 minutes	450	7.50 hours	$214.58
Totals	24		1,650	27.50 hours	$885.32

FEDERAL COST: The estimated annual cost to the Federal government is $21,200

IMLS oversight of contractor and project	Estimated labor hours	$3,000
Recruitment, data collection including incentives, materials, online platform hosting, analysis, travel, overhead	Labor hours and ODCs	$18,200

If you are conducting a focus group, survey, or plan to employ statistical methods, please provide answers to the following questions:

The selection of your targeted respondents

1. Do you have a customer list or something similar that defines the universe of potential respondents and do you have a sampling plan for selecting from this universe? [X] Yes [ ] No

If the answer is yes, please provide a description of both below (or attach the sampling plan)? If the answer is no, please provide a description of how you plan to identify your potential group of respondents and how you will select them?

Purposive sampling will be used based on individuals compiled from IMLS lists of stakeholder organizations, universities, professional associations, and individuals that are well connected to the key audience segments for this study. RTI will reach out to members of this list, as well as reach out directly to researchers and journalists who use the IMLS data products. RTI will also leverage a snowball sampling approach, where those who participate in the user interview can also recruit additional participants.

Administration of the Instrument

1. How will you collect the information? (Check all that apply)

[X] Web-based or other forms of Social Media

[ ] Telephone

[ ] In-person

[ ] Mail

[X] Other, Explain Video interview with open-ended questions, followed by screensharing and task-based usability analysis.

2. Will interviewers or facilitators be used? [X] Yes [ ] No

The attached interviewer guide includes many questions and probes. As user experience interviews, the specific questions any given interviewee will be presented will vary, with some questions used to elicit usability information from the interviewee. Other questions will be used to guide interviewees to additional data products or accessibility features to both educate interviewees as well as to elicit information about how IMLS can improve these products and features. The moderator will only guide interviewees through scenarios based on the tools they use. If an interviewee uses more than two products, the moderator will prioritize two based on the list of prioritized data products, provided by IMLS.

Please make sure that all instruments, instructions, and scripts are submitted with the request.

1Grady, C. NIH, Department of Clinical Bioethics Clinical Center. Ethical and practical considerations of paying research participants. Accessed January 8, 2021 from: https://www.niehs.nih.gov/research/resources/assets/docs/ethical_and_practical_considerations_of_paying_research_participants_508.pdf

2 Singer E, Ye C. The Use and Effects of Incentives in Surveys. The ANNALS of the American Academy of Political and Social Science. 2013;645(1):112-141. doi:10.1177/0002716212458082

3 Göritz, A. (2006). Incentives in Web Studies: Methodological Issues and a Review.

4 National Research Council 2013. Nonresponse in Social Science Surveys: A Research Agenda. Washington, DC: The National Academies Press. https://doi.org/10.17226/18293.

5 Boyd, C. (2020). The Ultimate Guide to User Research Incentives. User Interviews. Accessed on January 8, 2021 from: https://www.userinterviews.com/blog/the-ultimate-guide-to-user-research-incentives

6Sigritz, A. (2016). Paying Incentives for Federal User Research. GSA, DigitalGo. Accessed on January 8, 2021 from: https://digital.gov/2016/02/10/paying-incentives-for-federal-user-research/

7 18F. Fundamentals: Foundational methods for practicing design research. Accessed January 6, 2021 from: https://methods.18f.gov/fundamentals/incentives/

8 Value of time (total burden hours) used wage rate data from the Bureau of Labor Statistics, “May 2019 National Occupational Employment and Wage Estimates, United States” accessed online at: https://www.bls.gov/oes/current/oes_nat.htm#00-0000 (Access date: 6 January 2021). “News analysts, reporters, and journalists” (27-3023) had a median wage of $22.25/hour; “Librarians and media collections specialists” (25-4022) earned a median of $28.61/hour; and researchers, are likely to be “Social scientists and related workers” (19-3000) with a median wage of $38.57/hour. The value of time for state or territory administrative staff (library administrators) was estimated using the median wage for Librarians and media collections specialists.

5