PEQIS Memo to OMB distance Ed

U.S. DEPARTMENT OF EDUCATION INSTITUTE OF EDUCATION SCIENCES NATIONAL CENTER FOR EDUCATION STATISTICS

TO:	Brian Harris-Kojetin	April 18, 2007
THROUGH:	Kathy Axt
FROM:	Edith McArthur
SUBJECT:	Request for Clearance for the Proposed Postsecondary Education Quick Information System (PEQIS 16): Online and Distance Education Courses at Postsecondary Institutions.

Justification

The National Center for Education Statistics (NCES), U.S. Department of Education (ED) proposes to use the Postsecondary Education Quick Information System (PEQIS) to conduct a survey of online and distance education courses at postsecondary education institutions. This survey, under OMB clearance #1850-0733, will be the fourth survey conducted by ED to provide nationally representative data about the prevalence and delivery of distance education courses at postsecondary institutions. The first study, conducted in 1995, collected the first nationally representative data about distance education course offerings in higher education institutions. Since then, a second study was conducted in 1998 and a third study in 2002.

The 2002 study is widely cited as the most recent national data on distance education at postsecondary institutions. The study provides a national profile of the prevalence of distance education, including the number of distance education courses, enrollments, and degree and certificate programs designed to be completed entirely through distance education. The 2002 study also describes the types of technologies used as a primary mode of instructional delivery for distance education courses, factors keeping institutions from starting or expanding distance education offerings, the goals of distance education programs, participation in a consortia, and accommodations and accessibility of web sites for students with disabilities.

Current literature and anecdotal information suggest that postsecondary distance education continues to undergo significant changes in its structure and delivery. The growth of the Internet and the World Wide Web and reductions in telecommunications costs have combined with the widespread use of personal computers to facilitate new ways of designing, delivering, and administering distance education courses. As new technologies become available, distance education continues to evolve to create opportunities to better serve various student populations. These factors have spurred significant growth in the number and types of distance education courses and programs schools offer. In particular, an explosion of articles has appeared in recent years to discuss the increasing number and importance of online courses and hybrid or blended online courses. Hybrid or blended online courses are consistently defined in the literature as the combination of in-class and online instruction, resulting in reduced in-class seat-time for students. Anecdotal data suggest that these courses are the fastest growing type of distance education courses, primarily because of the high degree of flexibility in meeting student demand for flexible schedules and the institution’s push to maximize the use of its existing facilities.

A new PEQIS survey on distance education will provide much needed data about the prevalence, structure, and delivery of online and distance education at the nation’s postsecondary institutions.¹ In addition to providing national data about online and hybrid/blended online courses, the study will collect information about all other types of distance education courses. It will collect data on the prevalence and delivery of distance education courses in the 2006-07 12-month academic year, including the number of courses and enrollments for online courses, hybrid/blended online courses, and all other distance education courses; the number of degree or certificate programs designed to be completed totally through distance education; technologies used for the instructional delivery of credit-granting distance education courses; and distance education offerings for elementary or secondary students. The study will also examine the factors that affect the institution’s decisions regarding credit-granting distance education courses and the means by which institutions acquire or develop their distance education courses.

To capture the changing face of postsecondary distance education, the new PEQIS survey is designed to collect new baseline data on the prevalence, structure, and delivery of distance education courses, including online and hybrid or blended online courses. Thus, most of the data collected from the survey will not be comparable to information in the three previous PEQIS surveys. Decisions to collect new baseline data and the types of data to be collected were based on extensive review and discussions with the Office of Educational Technology (OET) and NCES at every stage of the survey development process.

The survey is authorized under Section 153 (a) of the Education Science Reform Act of 2002 (Public Law 107-279), which states that the purpose of NCES is “to collect, report, analyze, and disseminate statistical data related to education in the United States and in other nations.”

Data collection instruments

A questionnaire and cover letter (enclosed) will be mailed to each institution in August of 2007. The cover letter requests the participation of the institution and introduces the purpose and content of the survey. It also notes that the survey should be completed by the person or persons most knowledgeable about online and distance education at the institution. The cover letter also includes instructions on how to complete and return the survey, as well as contact information in case of queries. Included in the mailing will be information about the option to complete a Web version of the survey.

The questionnaire asks whether the institution offered any distance education courses in the 2006-07 12-month academic year. Institutions that do not offer distance education are asked to complete the contact information on the front page and return the questionnaire to Westat. Institutions that offer distance education are asked whether they offered any college-level credit-granting distance education courses, and those that offer such courses will complete the entire questionnaire. The first three sections of the questionnaire focus on three types of college-level credit-granting distance education courses—online courses, hybrid/blended online courses, and all other distance education courses. For each category of distance education courses, respondents who indicate that they offer such courses are asked to provide counts for number of courses and enrollments at the undergraduate and graduate or first professional levels. Respondents are also asked to provide additional information on the criterion used to designate courses as online courses. This information will provide insight on the extent to which institutions vary in their definitions of online courses.

The fourth section of the questionnaire focuses on degree or certificate programs designed to be completed totally through distance education. These are the same questions that were asked in the previous PEQIS surveys and would therefore allow for analysis of data over time. The remainder of the questionnaire focuses on all distance education courses and programs, including online and other modes of instruction. Institutions that offer credit-granting distance education are asked about the factors that affect the institution’s decisions regarding college-level, credit-granting distance education courses. This question is designed to address current factors that affect distance education decision-making at the institutional level, including factors such as seeking to increase student enrollment, making more courses available, and meeting student demand for flexible schedules and reduced seat-time on campus. Respondents are also asked to indicate the extent to which the institution used various types of technologies or media for the instructional delivery of credit-granting distance education courses, including asynchronous Internet-based technologies, synchronous Internet-based technologies, two-way interactive video, one-way prerecorded video, one-way video with two-way audio, and written correspondence.

Insititutions are also asked about whether they offer distance education courses for elementary or secondary students, and if so, what types of courses. In addition, institutions are asked about requests to provide accommodation for students with disabilities in their distance education courses, a question that was also asked in the 2002 PEQIS survey. All institutions that offer any distance education courses are asked about whether their institution offers noncredit distance education and how the institution acquires or develops its distance education courses, both credit and noncredit.

Substantial development work was carried out for this survey. This included a literature review, consultation with experts, four rounds of feasibility calls (where respondents provide comments about survey topics or questions, but do not complete the survey), and two survey pretests. Each round of feasibility calls and each pretest included fewer than 10 respondents. In addition, the survey used for each iteration was very different. These rounds of instrument design and testing were aimed at refining the survey definition, organization, and question wording.

Overview of data collection

The data collection will be accomplished by means of a self-administered survey of approximately 1,600 4-year and 2-year Title IV degree-granting postsecondary institutions in the 50 states and the District of Columbia. Respondents will have the option of completing the survey with a traditional paper and pencil questionnaire or with a Web version of the questionnaire. The paper version of the questionnaire is limited to three pages of questions. The information needed to complete the survey will be readily available to respondents, and the survey can be completed by most institutions in 45 minutes or less. These procedures are typical of PEQIS and FRSS surveys, and result in minimal burden on respondents.

Questionnaires and cover letters will be mailed to the PEQIS coordinator of institutions in the PEQIS panel (see description of the PEQIS panel under respondent universe). The cover letter will include a detailed description of the most appropriate respondent(s).

Telephone followup for nonresponse will begin about 3 weeks after the questionnaires have been mailed to the institutions. Experienced telephone interviewers will be trained to conduct the nonresponse followup and will be monitored by Westat supervisory personnel during all interviewing hours. The response rates for all PEQIS surveys have been greater than 90 percent.

Westat will collect the information for the Early Childhood, International, and Cross-Cutting Studies Division, NCES, U.S. Department of Education, through PEQIS. Westat is responsible for the questionnaire development; sample design; data collection; telephone followup; editing, coding, keying, and verification of the data; and production of tabulations and the report detailing the results of the survey.

Review by persons outside the agency

Various individuals have been consulted during the development of the questionnaire, including recognized experts who focus on educational technology, as well as staff at OET and NCES. Additional input was received from respondents at the postsecondary institutions who participated in the feasibility calls and pretests of drafts of the questionnaire. Respondents included directors of distance education programs and faculty with expertise in distance education programs and technologies.

Survey cost

The survey is estimated to cost the Federal government about $380,000, including about $350,000 for contractual costs and $30,000 for salaries and expenses. Based upon costs of past PEQIS surveys, contractual costs are divided into the subtask costs shown in Exhibit 1.

Exhibit 1. Estimated contractual costs by subtask

Subtask	Cost
Survey preparation	60,000
Data collection	175,000
Data analysis	40,000
Report preparation and dissemination	75,000
Total	350,000

Time schedule

The questionnaires are scheduled to be mailed in August 2007. About 3 weeks after mailout, Westat will begin telephone followup for nonresponse. Data collection is scheduled for completion about 18 weeks after mailing. See Exhibit 2 for the anticipated time schedule.

Exhibit 2. Anticipated data collection schedule

	Cumulative workdays
	From submission to RIMG/OMB	From RIMG/OMB approval
Package to OMB	0	-
Package approved by OMB	30	0
Mail-out of questionnaire	40	10
Follow up started	45	25
Follow up completed	130	100

Plan for tabulation and publication

Most of the analyses of the questionnaire data will be descriptive in nature, providing NCES and other information users with tables, charts, and appropriate explanatory text. Survey responses will be weighted to produce national estimates. Crosstabulations of data items will be made with selected classification variables such as institutional type and size of the institution. Questionnaire data will be analyzed by the following characteristics:

• Type of institution (2-year public, 2-year private, 4-year public, 4-year private)

• Size of institution (less than 3,000; 3,000-9,999; 10,000 or more)

Weighted frequency distributions will be produced for all items. Crosstabulations by the analysis variables listed above will be produced for all the categorical items, and sums by the analysis variables will be produced for the counts of distance education courses, degree programs, and enrollments. Analyses will include bivariate comparisons across subgroups of the classification variables with appropriate statistical testing for significance at the 0.05 level. The final report will follow NCES’s editorial and statistical standards. Tables and figures will also be designed and formatted in accord with current NCES standards. The detailing of findings in the text will be descriptive in nature and limited only to statistically significant results.

The survey has been requested by the Office of Educational Technology, U.S. Department of Education (NCES). It is being conducted by Westat under contract to the Early Childhood International and Crosscutting Studies Division, NCES. Westat will mail the questionnaires; collect data by mail and telephone; edit, code, key, and verify the data; and produce tabulations and the survey report.

Reviewing statisticians

Statistician Adam Chu of Westat (301-251-4326) was consulted about the statistical aspects of the design.

Respondent universe

This survey will be sent to approximately 1,600 postsecondary institutions in the PEQIS panel. The PEQIS panel was originally selected and recruited in 1991–92, and is periodically updated to reflect changes in the postsecondary education universe that have occurred since the original panel was selected. A modified Keyfitz approach is used to maximize overlap between the 1991–92 panel and the periodic updates. The sampling frame for the current PEQIS panel is constructed from the 2005 Integrated Postsecondary Education Data System (IPEDS) “Institutional Characteristics” file. Institutions eligible for the PEQIS frame include 2-year and 4-year (including graduate-level) Title IV-eligible degree-granting institutions located in the 50 states and the District of Columbia. In 2006, the frame included a total of 4,264 institutions.

The PEQIS sampling frame is stratified by instructional level (4-year, 2-year), control (public, private nonprofit, private for-profit), highest level of offering (doctor’s/first-professional, master’s, bachelor’s, less than bachelor’s), and total enrollment. Within each of the strata, institutions are sorted by region (Northeast, Southeast, Central, West), and whether the institution has a relatively high minority enrollment. The sample of institutions is allocated to the strata in proportion to the aggregate square root of total enrollment. Institutions within a stratum are sampled with equal probabilities of selection. In 2006, the sample included a total of 1,627 institutions. The modified Keyfitz approach resulted in 79 percent of the institutions in the 2006 sample overlapping with the 2002 panel.

Each institution in the PEQIS panel was asked to identify a campus representative to serve as survey coordinator. The campus representative facilitates data collection by identifying the appropriate respondent for each survey and forwarding the questionnaire to that person.

Statistical methodology

Nonresponse weight adjustments will be used to correct for unit nonresponse. Variances will be estimated using the jackknife replication method. Estimates produced during the PEQIS panel design stage, based on characteristics of the institutions, yielded coefficients of variation (cv’s) in the range of 2 to 4 percent for most national estimates, with estimates for subgroups somewhat higher. Similar cv’s are expected for this survey.

1 The Sloan Foundation studies provide important insights into the growing importance of online and hybrid or blended online courses at postsecondary institutions. However, the data are based on a relatively low response rate; for example, the most recent study conducted in 2002 was based on a 32 percent response rate.