Desk and Screening Protocol - final

ARL/Ithaka S+R Case Studies on Sustainability of Digitized Special Collections

Desk and Phone Screening Protocol

Desk Research Protocol

Selection Criteria: Segmentation

1. Type of Institution

• Is this project part of an academic library or a museum/public library?

  • This may be determined through primary observation.

• Record findings: academic library or museum/public library.

2. Budget size of institution

• Is the host institution of this project large or small?

  • For academic libraries, determine large vs. small by using the statistics provided by the National Center for Education Statistics in Academic Libraries: 2010 First Look.

  • For museums and public libraries, determine large vs. small by using the IMLS report Status of Technology and Digitization in the Nation’s Museums and Libraries.

• Record findings: large or small.

3. Source of initial funding

• Was this project created through internal investment or with an external grant?

  • This may be determined through investigation of the project’s website, which may list site funders and other sponsors, and, if needed, a web search for press releases and articles announcing the project.

• Record findings: internal or external.

Selection Criteria: Sustainability Factors

4. Longevity

• Has the resource been available publically for more than 2 years?

  • Look on the website of the project for evidence of its start date, either on its homepage or in a section on its history. If this is unsuccessful, search the web for press releases or other articles with these details. Try to verify results by looking at grant reports, where available.

• Record findings: yes or no.

  • If no, conclude research. Project does not fit our criteria.

  • If yes, continue.

5. Financial Stability

• Does the resource have a plan in place that permits it to cover its costs and invest in needed upgrades, whether through internal support or external funding?

  • Determine financial stability by looking for evidence of creative and varied attempts to generate revenue (advertising, requests for donations, evidence of sponsors, pay models, etc.) on the projects site or in other related locations (e.g., the host’s site). If available, look in the project’s history for financial history. If the project is still active, check to see if its site is current and has been updated recently and regularly.

• Record findings: Rate on a scale of 1-5. A score of 5 will indicate that we identified a variety of robust revenue streams; 1 will indicate that the project is no longer in operation.

  • If the project rates 3 or higher, continue.

6. Pubic Benefit

• Does the project have significant value for the community it was intended to serve?

  • Determine impact by looking for a visitor counter and user activity in comment areas of project blogs and other user-submitted areas. Look to see if the project has a social media presence (Facebook, Twitter, etc.) and if there is a community response. Via internet search, check to see if the project is cited in academic journals, the sites of professional organizations, or in the news media, and look for awards and accolades.

• Record findings: Rate project on a scale of 1-5, with 5 signaling a strong public benefit, whether it is for a small but active niche group or for a wide-ranging group that is less active. 1 will mean that the project shows no signs of having built an audience, audience contribution, or other forms of impact.

  • If the project rates less than 3, conclude research. Project does not fit our criteria.

  • If the project rates 3 or higher, add the site to the list of projects to be considered for phone screening.

Phone Screening Protocol

As a result of the desk research phase, the research team anticipates being able to narrow the field of potential case studies by excluding those that do not meet selection criteria. In some cases, this will be clear (projects no longer in operation; projects not live for more than 2 years, etc…). For certain criteria, including “public benefit” and “financial stability” where obtaining accurate data from desk research alone may not be possible, we will be careful to not exclude cases simply because we do not have access to information. Here, we will use the ratings to prioritize the strongest cases, and then a phone screen will allow us to more accurately assess the degree to which projects fit our selection criteria. Below are the questions we will raise in the phone screen stage:

I would like to inform you that this information is solicited under the authority of the Museum and Library Services Act of 2010, as amended. Your participation is entirely voluntary and your decision whether or not to participate will in no way adversely affect your institution. Your cooperation is extremely valuable in obtaining much needed information to ensure the completeness and accuracy of the case studies.  Any information you designate as “confidential” during the course of this study will be protected from disclosure to the extent permitted by law.

Selection Criteria: Segmentation

1. Type of Institution (pre-screened through desk research)

2. Budget size of institution (pre-screened through desk research)

3. Source of initial funding

• What was the source of your initial funding to create this resource?

• Record findings: internal or external.

Selection Criteria: Sustainability Factors

4. Longevity

To confirm findings from desk research about whether or not the site has been available publically for more than 2 years.

Screening questions:

• When did this project become available to users?

• When did the site launch?

Assessment:

Yes (available for 2+ years) or no.

• If no, conclude research. Project does not fit our criteria.

• If yes, continue.

5. Financial Stability

To determine if the resource has a plan in place that permits it to cover its costs and invest in needed upgrades, whether through internal support or external funding.

Screening questions:

• How is this resource currently supported?

• Are these forms of support ongoing?

• Do they permit continued maintenance and upgrades as needed?

• Is there a plan in place for the future of the project?

Assessment:

Rate site on a scale of 1-5. 5 will mean that the project has a strong financial outlook, 1 will mean that it is defunct, and 3 will mean that it is covering its costs.

• If the project rates less than 3, conclude research. Project does not fit our criteria.

• If the project rates 3 or higher, continue.

6. Pubic Benefit

To determine if the project has a significant impact on the community it was intended to serve.

Screening questions:

• How many users does the project have (and how is this measured?)

• What kind of feedback has the site received and from whom?

• What kinds of other attention has the site received (citations, news articles, etc.)?

• What other ways do you measure the impact of the resource, and how are these measured?

Assessment:

Rate project on a scale of 1-5, with 5 signaling a project with demonstrated public benefit, whether reaching a high volume of users, or a smaller group of users, but in a way that demonstrates their appreciation of it (awards, active usage, contributions). 1 will mean that the project appears to have had no impact.

• If the project rates less than 3, conclude research. Project does not fit our criteria.

• If the project rates 3 or higher, add the site to the list of projects to be considered to serve as a case study.