FY2022 PLS Cognitive Testing Interview Protocol_Appendix A

FY2022 Public Library Survey (PLS) Cognitive Testing Interview Protocol

Appendix A Interview Protocol

Survey Development for Public Libraries Survey

Elements about Partnerships and Overdue Fines

Cognitive Interviewing Protocol 

 

Interviewer
Date
Time (Start to stop)
Interview ID

Verbal Consent Script

Hello, my name is [first and last name], and I will be conducting your interview today. I work for the American Institutes for Research, or AIR. The Institute of Museum and Library Services, or IMLS, has asked AIR to help develop potential new national PLS data elements related to two topics: (1) library partnerships and (2) overdue fines. The first set of data elements, or questions, ask whether libraries have partnerships with various types of organizations. The second set ask about current overdue fine policies. Early next year, the State Library Administrative Agencies (SLAAs) will vote on whether to add some or all of these elements to the FY 2022 PLS.

Before submitting some or all of these elements for SLAA consideration, we want to get feedback on the data element definitions to ensure they make sense and are relevant. We have asked you to participate in this interview because you are a [title] and we value your expertise. We will use your feedback to improve the data elements. 

The way this will work is I will show you a data element definition on the screen and ask you to read out loud. I would then like for you to “think aloud” or talk me through your thoughts as you think about how you would respond to the element. Then I’ll ask you some follow-up questions. Before we begin, I’ll demonstrate how to think aloud, and we’ll do a practice run.

I completely understand your library may not currently collect some of the information asked about in the draft elements. For the purposes of this interview, there are no right or wrong answers to the elements. I am more interested in how you would arrive at your answers and whether it is feasible for you to answer, rather than in the answers you give me.

The interview will last one hour. Your participation is completely voluntary. You can skip any questions that you do not want to answer, and you may stop the interview at any time for any reason. The feedback you provide will be kept confidential. Any information that could be linked to you personally, like your name or name of your library, will not be included in our report to IMLS, and your responses will be used for research purposes only.

There are no risks or direct benefits to you for participating. However, you will be helping to shape these data element definitions to ensure that they are relevant to and meet the needs of public library systems across the country, including yours.

If you have questions or concerns about this effort, please contact the project director, Evan Nielsen, at [email protected].

Do you have any questions at this time? [Answer any questions] 

Do you consent to participate in the interview?  o YES o NO [Check box; if participant does not consent, thank them and end call.]

Great. To ensure that we capture all the information you provide, we would like to audio record today’s conversion. You may also see me taking notes while we are talking. As a reminder, your information will be kept private, and we will not include your name or your library’s name in any reports.  

Do I have your permission to begin the recording?  o YES o NO [Check box; if participant does not agree to recording, proceed without recording the interview. Otherwise, BEGIN RECORDING.]

Great. Let’s get started. 

Instructions

I will be showing you one data element at a time on the screen. As I mentioned, I would like for you to read the element and its definition and “think aloud” as you respond. More specifically, I’d like you to say out loud what you are thinking. This will help us determine whether the elements are clear and understandable. 

Let’s first start with an example where I’ll show you how to think aloud. [Show example question.] 

QUESTION: How many glasses of water did you have today? 

Model the think aloud process, including reading the question out loud.

• Elaborate on what “glass” means to you, what you include as “water” (e.g., coffee, juice, etc.), what time of day you drank them, e.g., “Glass to me means a cup, and I had 1 cup of coffee this morning, but I would not count coffee as water. Then I drank 2 cups of water before I met up with you and I know it’s 2 cups because I remember going to the kitchen twice to get water today, so in total, I had 3 glasses of water today.” 

Now, I’d like for you to try. Please read the question and talk through how you would answer it. 

• If participant gives you a short answer, ask them to tell you how they reached their answer. What is a “glass”? What do they consider as “water”?

Great. This is the kind of detail we are looking for. Okay, let’s get started. As a reminder, please read each element and its definition out loud as if you’re taking the survey. Tell us what you are thinking as you read and answer each question.

General Probes

The following probes may be used in conjunction with the item-specific probes.

• What comes to mind when you hear the word/term [state term/phrase]?

• How easy or difficult would it be for your library to answer this question?

• In your own words, what is this question asking?

• How did you come up with your answer?

• You said [answer]. Can you tell me more about that?

• Are there any examples you would add or remove from the definition?

• [If participant seems confused] Did these feel like the right [partner organizations/response options/etc.] for your library system? 

PARTNERSHIP DEFINITION

This first page includes a definition of “partnership” that helps explain what kind of collaboration and coordination should be included when answering this set of survey questions.

Partnership Definition

NOTE: A partnership is a formal arrangement between the library and another organization that enables the two entities to support each other and work together towards shared goals.

Partnerships can involve varied levels of coordination, such as communicating information about each other's programs, services, and/or resources (for example, by distributing promotional materials, giving referrals, setting up displays, or making presentations); providing mutual assistance (for example, by sponsoring activities, donating incentives, or recruiting volunteers); or working together to jointly develop and deliver programs or services (for example, by sharing staff, resources, and/or costs).

• In your own words, what is a “partnership”?

• How about a “formal arrangement”? What are the key elements of a formal arrangement between a library and another organization?

• Does your library have arrangements that you would consider “informal”?

  • How would you say that an informal arrangement differs from a formal arrangement? What does an informal arrangement look like?

• In the second paragraph of this definition, how well do these examples align with the kinds of coordination your library has with its partner organizations? Are there examples that should be added or removed?

PARTNERSHIPS WITH VARIOUS TYPES OF ORGANIZATIONS

This next section asks whether your library has partnerships with various types of organizations. Please read the elements aloud and let me know what you are thinking as you imagine how you would answer them.

Partnership with State Government Agency

Did the library have a formal partnership with a state government agency during the reporting period?

Do not report state funding or participation in partnerships or programs run by the state library administrative agency.

• Yes

• No

• In your own words, what is a “state government agency”? Do you mind sharing a few examples?

• Take a moment to imagine that your library received funding from your state library agency for a digital media lab or makerspace.

  • Would you answer this question differently in that situation? Please explain.

  • How would you answer if the state library agency provided materials for summer learning program events?

• Is there any coordination with a state government agency that your library participates in that you are not sure whether it should count as a formal partnership?

• [If yes to survey question]: How many partnerships does your library have with state government agencies?

• Can you tell me more about the state government agency with which you partner? Probe names/descriptions.

• On what types of activities or initiatives do you partner?

Partnership with Local Government Agency

Did the library have a formal partnership with a local government agency, other than a public library or school, during the reporting period?

Do not report consortia, cooperatives, or federations with other public libraries. Do not report partnerships with schools or school districts; report them in Partnership with K-12 School or District. Do not report partnerships with Native American tribal governments; report them in Partnership with Tribal Organization.

• Yes

• No

• In your own words, what is a “local government agency”? Do you mind sharing a few examples?

• Does your library coordinate with any local government agencies that you’re not sure whether to count as a formal partnership? Please explain.

• [If yes to survey question]: How many partnerships does your library have with local government agencies?

  • Could you tell me more about the local government agency with which you partner? Probe for names/descriptions.

  • Can you tell me more about the types of activities or initiatives on which you partner?

Partnership with K-12 School or District

Did the library have a formal partnership with a primary or secondary (K-12) school or district during the reporting period?

Include partnerships with public and private schools. Do not report partnerships with vocational schools; report them in Partnership with Institution of Higher Education.

• Yes

• No

• Take a moment to imagine that your library has a partnership with a charter school.

  • Would you report that here? Why or why not?

  • What about a private school?

• Does your library coordinate with a K-12 school or district that you are not sure whether to count as a formal partnership? Please explain.

• [If yes to survey question]: How many partnerships does your library have with K-12 schools or districts? Probe for names/descriptions.

  • Could you tell me more about the K-12 school or district with which you partner?

  • On what types of activities or initiatives do you partner?

Partnership with Institution of Higher Education

Did the library have a formal partnership with an institution of higher education during the reporting period?

Include partnerships with vocational schools, community colleges, 4-year colleges, and universities.

• Yes

• No

• Take a moment to imagine that your library partners with a career or technical school with courses for high school students and adults. Would you report that here? Why or why not?

• Does your library coordinate with an institution of higher education that you are not sure whether to count as a formal partnership? Please explain.

• [If yes to survey question]: How many partnerships does your library have with institutions of higher education? Probe for names/descriptions.

  • Could you tell me more about the institution of higher education with which you partner?

  • On what types of activities or initiatives do you partner?

Partnership with Museum or Historical Society

Did the library have a formal partnership with a museum or historical society during the reporting period?

Do not report partnerships with museums or historical societies that share a facility or budget with the library. Do not report partnerships with museums operated by a local or state government agency; report them in Partnership with Local Government Agency or Partnership with State Government Agency.

• Yes

• No

• Imagine there is a museum within your library. Would you report coordination with that museum in this element? Why or why not?

  • Would it matter whether the museum shares a budget with your library?

• Take a moment to imagine that your library has a formal partnership with a children’s museum. Would you report coordination with that museum in this element? Why or why not?

• Does your library coordinate with a museum or historical society that you are not sure whether to count as a formal partnership?

• [If yes to survey question]: How many partnerships does your library have with museums or historical societies?

  • Could you tell me more about the museum or historical society with which you partner? Probe for names/descriptions.

  • What types of activities or initiatives do you partner on?

Partnership with Non-Profit Organization

Did the library have a formal partnership with a non-profit organization during the reporting period?

• Yes

• No

• Imagine your library partners with a museum that is run by a non-profit organization. Would you report that here? Why or why not?

• Does your library coordinate with a non-profit organization that you are not sure whether to count as a formal partnership?

• [If yes to survey question]: How many partnerships does your library have with non-profit organizations?

  • Could you tell me more about the non-profit organization with which you partner? Probe for names/descriptions.

  • For what types of activities or initiatives do you partner?

Partnership with For-Profit Organization

Did the library have a formal partnership with a for-profit organization during the reporting period?

Do not report sponsorships or donations to the library from for-profit organizations.

• Yes

• No

• Imagine a business in the area provides prizes or giveaways for the summer learning program. Would you report that coordination in this element? Why or why not?

• Does your library coordinate with a for-profit organization that you are not sure whether to count as a formal partnership?

• [If library reports a partnership]: How many partnerships does your library have with for-profit organizations?

  • Could you tell me more about the for-profit organization with which you partner? Probe for names/descriptions.

  • Do you mind sharing more about the types of activities or initiatives on which you partner?

Partnership with Tribal Organization

Did the library have a formal partnership with a tribal organization during the reporting period?

• Yes

• No

• Does your library coordinate with a tribal organization that you are not sure whether to count as a formal partnership?

• [If library reports a partnership]: How many partnerships does your library have with tribal organizations?

  • Could you tell me more about the tribal organization with which you partner? Probe for names/descriptions.

  • Do you mind sharing more about the types of activities or initiatives on which you partner?

Partnership with Other Organization

Did the library have a formal partnership with any other type of organization during the reporting period?

• Yes

• No

• Does your library coordinate with another organization that you are not sure whether to count as a formal partnership?

• [If library reports a partnership]: How many partnerships does your library have with other organizations?

  • Could you tell me more about the other organization with which you partner? Probe for names/descriptions.

  • Do you mind sharing more about the types of activities or initiatives on which you partner?

PARTNERSHIP OVERALL SECTION PROBES:

• Overall, how easy or difficult would the questions in this section be to answer?

• Are there any partnerships that your library has that you feel unsure in which of the above categories you would put them? If so: What is making you feel unsure? What kind of organizations are these partners?

OVERDUE FINES

This next set of elements asks about overdue fines. Please read the elements aloud and let me know what you are thinking as you imagine how you would answer them.

Current Overdue Fine Policy

As of the beginning of the reporting period, did the library charge overdue fines to users when they fail to return physical materials on or before the date due?

NOTE: Overdue fines are monetary penalties that typically increase according to the number of days the materials are overdue. Overdue fines are not replacement costs for lost or damaged materials.

Select one of the following:

• Yes, overdue fines were charged to all types of users and for all types of physical materials

• Yes, but overdue fines were charged only to some types of users or only for some types of physical materials

• No, overdue fines were not charged to any types of users or for any types of physical materials

• In your own words, what are “overdue fines”?

• Does your library charge overdue fines for anything other than physical materials?

• How do you interpret the difference between the first two response options?

• [If yes, some users/some materials] To what types of users / for what types of physical materials does your library (not) charge overdue fines?

• Does your library charge any other types of fines that you are not sure whether to include here?

  • [If yes] What is making you feel unsure? Can you tell me more about those fines?

[Ask next item only if library charged overdue fines]

Overdue Fine Revenue

If the library charged overdue fines, report the total amount of overdue fines collected during the reporting period. Do not count replacement costs collected for lost or damaged materials. This amount is a subset of Other Operating Revenue (data element #303).

$___________

• Unknown

• How easy or difficult would it be for your library to answer this question?

• Are you able to isolate overdue fines from any other fees (e.g., lost/damaged materials, non-resident users) in your financial records?

• Are your fine revenues passed through to another agency’s budget, and thus not typically included in Operating Revenue?

[Ask next item only if library did not charge overdue fines]

Date of Overdue Fine Policy Change

If the library did not charge overdue fines, when was this policy enacted?

_ _ / _ _ _ _ [MM/YYYY]

• Unknown

• Not applicable, library has never charged overdue fines

• How easy or difficult would it be for your library to answer this question?

• Did this date align with a new fiscal year?

• Did your library change the overdue fine policy for certain users or material types at different times, or all at once? For example, did overdue fines for children’s accounts or materials phase out before the policy was changed for everyone?

• If library recently (last 3 years or so) set this policy, before the policy changed, would you have been able to isolate overdue fines from any other fees (e.g., lost/damaged materials, non-resident users) in your financial records?

OVERDUE FINES OVERALL SECTION PROBES:

• Do you have any other feedback about these overdue fine questions?

B-4