ED Response to OMB Passback

MEMORANDUM OMB # 1850-0781

DATE: July 12, 2010

TO: Shelly Martinez

Office of Information and Regulatory Affairs, Office of Management and Budget

FROM: Tai Phan

National Center for Education Statistics

THROUGH: Kashka Kubzdela

National Center for Education Statistics

SUBJECT: Response to 7-7-2010 and 7-9-2010 OMB Passback for ALS

 

7-7-2010 OMB Passback:

1. Burden.

1. Why does the survey take over 8 hours on average per library? 

The larger libraries take longer than 8 hours because they have to gather the data from multiple branch libraries.  The smaller libraries take less because they usually have only one library.

2. What is the source of the burden estimate?

The source of the burden estimate is from the 2008 survey respondents, who were asked to provide an estimate of how long it took to complete the survey.

3. Besides automation 10 years ago, what efforts has NCES made to address this burden?

NCES has worked with the ALS advisory committee to refine the survey questions and instructions and also has updated the edit parameters to lessen the editing burden on the respondent.

2. Confidentiality.

1. In SS A10 and A11, please clarify to whom confidentiality is being pledged, if anyone, and for which data.

Section A10 and A11 have been revised as indicated below, in response to point 2b.

2. If NCES is merely using disclosure avoidance as a “professional practice” as it does in IPEDS to avoid identifying individual staff salaries then it should use the language approved for IPEDS in this regard.

The Supporting Statement Part A.10 has been revised as follows:

“ALS data are not collected under a pledge of confidentiality.  However, NCES and the Census Bureau take steps to protect potentially individually identifiable data (The Census Bureau will collect the ALS data under an interagency agreement with NCES). The data are collected as statistical aggregates without any direct identifiers for individuals beyond the name of the library. On the final edited file, for all institutions where any data reported for staff are based on data from one or two staff members, the data for salaries, wages, and benefits will be suppressed. The remaining data will only be reported in statistical summaries that preclude the identification of any individual library staff participating in the survey.”

The 2^nd paragraph of Supporting Statement Part A.11, and the relevant text in the letters to Library Directors and Chief Academic Officers, have been revised to state:

“The cover letters to Library Directors and Chief Academic Officers will state the following: ‘The collection of salary and benefits data is an important component of this voluntary library survey. NCES and the Census Bureau take steps to protect potentially individually identifiable information such that when there are only one or two FTEs within a staffing category, salary and benefits data for that category will not be shown.’”

3. SS A16 is supposed to cover both schedule and publication/analysis plans.  Please provide a paragraph on the latter.

The Supporting Statement Part A.15 has been updated with the following 3 methods of ALS data dissemination:

• Unimputed suppressed data will be released on the Compare Academic Libraries Tool on July 6, 2011.

• Imputed suppressed data files and file documentation will be released on the NCES website in SAS, Text, Access, and SPSS on December 8, 2011.

• First look publication and supplemental publication will be released on the NCES website as a PDF file on December 8, 2011.

The new data elements collected for FY2010 will be included on all data file and publication releases.

3. SS B1.  Where does the list of contacts come for the advance letters?  How are the specific respondents identified?

As stated in Supporting Statement Part B.1, the ALS uses the IPEDS directory file to select its universe by using the following criteria: at least a 2-year, degree granting institution that is eligible for Title IV funding.

4. SS B5.  What were the results of the 2008 study on nonrespondents?

We don't have any results yet.  We expect to have preliminary results in September 2010.

5. Letters: Please provide the letters referenced in the text as well as any other supporting materials such as FAQs provided or available to respondents.

The following four files have been added to the ALS clearance package under supplemental materials:

• ALS 2010 Survey Changes.docx

• ALS 2010 Library Representatives List.doc (we are currently verifying whether the Library Representatives from 2008 still want to serve for the 2010 survey)

• ALS 2010 Q and A.docx (which is given to the help desk)

• ALS FAQ.docx (the current Q and A in the web collection application)

6. Screener.

1. Was any testing done of the new screen questionnaire?

Only the question pertaining to total library expenditure is new in the screener, so the screener did not require testing because the new question was derived from the reported expenditure data. Only 18 out of a total of 3,827 libraries (0.47%) reported expenditures less than 10,000.

2. Question “b” seems vague – what is the scope of “other materials” and is “a combination thereof” the clearest way to ask the question?

This questions has been used historically in the screener. Libraries are comprised of print, non-print, and electronic (digital) materials. Using the term “other materials” is not vague to the responding libraries.

3. Question “d” seems to be asking about the adequacy of space.  Is the real purpose to ask about have a physical facility regardless of its adequacy?

Questions d (actually “e” – the numbering has been corrected) is not about adequacy of square footage, it is about space dedicated [necessary] to house collections, staff, and library users. The purpose of this question is to eliminate institutions responding when all they have is a shelf of books in someone’s office and they call it a library.

4. What is the purpose of the statement “Data collected by NCES are used for statistical and directory purposes only” on the screener?  This is not a complete PRA or confidentiality statement, so seems inadequate and/or misplaced.

This is not a complete PRA or confidentiality statement, so seems inadequate and/or misplaced. This statement was added because we found that institutions that really did not meet the eligibility standards were reluctant to answer no in fear that it would affect their accreditation.

7. Questionnaire: In questions 200-207, please clarify if the right-hand column is supposed to be average or total salaries and wages.  Are instructions or definitions provided for these data elements?  Can the heading be clarified?

These are total, not averages. This column applies to questions 202-205 only, which specify “total” (e.g. “Total librarians and other professional staff”). This wording has been used for many years and the questions have been always interpreted correctly.  We have added the word “total” to the specific instructions on how to report salaries, which are included in the “Library Staff, Fall 2010 and Salaries/Wages, FY 2010” section of ALS 2010 2012 Instructions Appendix C.doc:

"Column (2), Salaries and wages (items 202-205) - Report total expenditures in FY 2010 for full-time and part-time salaries and wages before deductions. Exclude employee fringe benefits provided by your institution for all regular library staff that may be reported in item 207."

7-9-2010 OMB Passback:

8. Re 1a. We would like additional elaboration on the response to question one. Specifically, this collection’s burden estimates are higher per institution than most of the IPEDS collections, which have come under considerable scrutiny on burden by Congress and the GAO.  Please explain what NCES knows about why the library survey is so burdensome.   Are schools having to search paper records?  Are schools having to do custom programming?  Etc?

Prior to the 2008 collection, the estimate of burden hours was based on discussions with a small sample of institutions. To get a more accurate estimate the Census Bureau added a question to the collection asking how many hours it took to complete the survey.  The current time estimates are based on these responses.  To complete the survey there are many factors that come into play:  (1) the data need to be compiled from different areas within the library (financial, personnel, collections, etc.); (2) some libraries have multiple branches and multiple campuses for which data need to be aggregated; (3) some libraries require the review/approval (lock) from multiple individuals; and (4) the collection instrument has many built-in quality assurance edits, which place additional burden "up-front" but reduce additional edit follow-up burden when the data are being reviewed.

We do not know the degree of efficiencies that individual libraries have to report the data, other than that larger institutions tend to have the data available in electronic format, but may still need to pull the data from various sources and/or may need to develop customized programs to meet the reporting needs, while many small libraries tend to have "paper systems" which require time to extract the required information.

The ALS data are currently being collected every other year. The burden associated with completing this voluntary survey has never been brought up as a problem by the respondents or the Academic Library Advisory committee. On the contrary, NCES is frequently asked by the academic libraries to begin collecting the ALS data on an annual basis, in order to make the financial information available annually.

9. Re 1b. Related, does NCES know which items on the questionnaire are the most burdensome and why?  Has NCES ever conducted debriefings or other more intensive means to understand what items are most time consuming?

We have no metrics on which data elements are most burdensome. We suspect that the financial data are the most burdensome to provide because they need to conform to financial standards and audits before they can be released and the data need to be supplied by another area within the institution, whereas, much of the other information can be reported from the library itself.

Although NCES has not done formal evaluations of the burden of individual data items, we do consult with the Academic Library Advisory committee, representing various library types (small, large, public, private, etc.), on each question as to feasibility and ease of repotting. Again, the time burden is not seen as an obstacle, and the committee continues to recommend and request for the ALS collection to become annual, rather than bi-annual.

10. Re 3. Our question was about the individual names of library administrators or whomever NCES uses as contacts.  It is our understanding that IPEDS does not provide names of library staff.  Please provide a new answer.

We do not maintain or use names in our communications. We only use titles. We address the Chief Academic Officer letter to "Chief Academic Officer" and send it to the address provided by IPEDS, and similarly, we address the Library director letter to “Library Director.” During follow-up calls to non-respondents, if we cannot reach the person who can provide information on the library, we ask for a name and contact information for the persons who can. Otherwise, we use only titles in our correspondence.

11. Re 6c – We are concerned that the library space question could be interpreted as an opportunity to express a concern about inadequate space, thereby screening out respondents inaccurately.   Please reassure us that this has not been a problem by providing evidence that this question is well understood by respondents.  For example, has this question been tested in a cognitive lab or the results benchmarked against an independent estimate from associations etc?

This question has been asked in the Academic Libraries Survey since at least 2000 and no issues have been raised in reference to it.  Of the 3,445 libraries that responded to this eligibility question, only 125 answered no to this question.  Among these 125 libraries, 117 answered no to more than one eligibility question, and only 8 answered no to only this eligibility question.

However, in the next ALS administration, we will follow up with those who answer no only to this questions to find out if they are possibly misinterpreting it. We will also raise the wording of this question with the Academic Library Advisory committee, to determine whether they think it may be misleading and whether they recommend changing the wording.

12. Re 6d – If the statement we asked about is not intended to be the PRA statement, then please show us where the required PRA statement appears on the questionnaire.  Providing a screen shot of the web instrument would be important if the respondent is seeing something different from the paper instruments provided to us.

The PRA is provided on the login page of the survey (see screen shot below) under "Institutional Burden."