Tracking #:
Prepared for:
U.S. Department of Education
Contract ED
Submitted to:
Trudy Turner
Office of Vocational and Adult Education
400 Maryland Ave., SW
Prepared by:
MPR Associates
Program for Adult Education
Washington, D.C.
Date Submitted:
Contents
Introduction 3
A. Justification 4
A1. Circumstances Making Collection of Information Necessary 4
A.2. Purposes and Uses of the Data 4
A. 3. Use of Information Technology 7
A.4. Efforts to Identify Duplication 7
A.5. Method Used to Minimize Burden on Small Businesses or 7
Other Small Entities 7
A.6. Frequency of Data Collection 8
A.7. Special Circumstances of Data Collection 8
A.8. Consultants Outside the Agency 8
A.9. Provision of Payments or Gifts to Respondents 8
A.10 Assurance of Confidentiality 9
A.11. Sensitive Questions 9
A.12. Estimates of Response Burden 9
A.13. Estimates of Cost 10
A.14. Costs to Federal Government 11
A.15. Reasons for Changes in Response Burden and Costs 11
A.16. Publication Plans and Time Schedule 11
A.17. Approval to Not Display Expiration Date for OMB Approval 11
A.18. Exceptions to Certification for Paperwork Reduction Act Submissions 12
Appendix A: ANI Content Assessment Form A
Appendix B: ANI Content Assessment Form B
Appendix C: Teacher Background Survey
Appendix D: Program Administrator Background Survey
Appendix E: Teacher Follow-up Survey
Appendix F: Program Administrator Follow-up Survey
Appendix G: Teacher Interview
This initiative involves three Adult Numeracy Instruction (ANI) Professional Development Institutes. Twenty teachers and ten program administrators from ten adult education programs from each of two states will participate in a field test of the professional development Institutes. The goals of the institutes are to:
Increase and deepen mathematics content knowledge among teacher and program administrator participants;
Increase the repertoire of instructional skills among teachers working with adults in ABE, pre-GED, and GED classes;
Increase the ability of program administrators to provide support to teachers to enable them to teach effectively;
Increase state capacity to ensure quality instruction in local programs.
After each of two states is selected to have teachers and program administrators participate in the professional development, ANI staff will work with a representative from the state’s professional development staff to plan how the project will contribute to the state’s capacity to ensure quality instruction, to recruit teachers and administrators from local programs to participate in he professional development, and to plan activities for the state to build on the ANI project in the following year.
The study will involve the administration of the following instruments:
Pre/Post surveys of participating teachers and program administrators;
Pre/Post administration of a cognitive assessment to participating teachers, and
Post-professional development interviews with participating teachers.
This document supports the clearance of selected data elements, materials, and procedures under the Paperwork Reduction Act of 1995 and 5 CFR 1320, as amended, for the Strengthening America’s Competitiveness Through Math Instruction – Task 6: Field Test the Teacher Training Initiative task. MPR Associates Inc. will administer this survey under contract to the Office of Vocational and Adult Education (OVAE) (Contract Number [ED-04-CO-0121/0004]).
OVAE is investigating the effectiveness of a teacher-training initiative to be piloted during 2009-2010. Specifically, OVAE is interested in changes in teacher knowledge and repertoire of instructional skills. OVAE is also interested in concomitant changes in the knowledge of program administrators and their ability to support teachers in improving the quality of instruction. The project is also designed to bring about changes in state capacity to support teachers in the area of mathematical instruction. The latter will be accomplished as the ANI staff work with the state representative responsible for professional development to plan how to follow up on the ANI activities to increase state capacity to ensure quality instruction in local programs. To that end, we will conduct a field test of the initiative, and the instruments included in this submission will be used as part of that field test.
This project is authorized under the Adult Education and Family Literacy Act, Title II of Public Law 105-220, Section 243, National Leadership Activities. Section 243 allows the Secretary of Education to establish and carry out a program of national leadership activities to enhance the quality of adult education and literacy programs nationwide.
This activity shall support Goal Three of the Strategic Plan, “Ensure the accessibility, affordability, and accountability of higher education, and better prepare students and adults for employment and future learning.”
The main purpose of the data collection will be to support and improve the Adult Numeracy Instruction (ANI) professional development services. Data will provide information on the effectiveness of the institute in improving instruction in adult numeracy. The primary focus will be on the impact on teachers and local program administrators.
The data collection will include surveys, interviews, and a cognitive assessment.
Under OMB requirements, we will submit the following documents for review:
Pre/post surveys of teachers who participate in the professional development programs offered under this contract;
Pre/post cognitive assessments, using two different forms, of teachers who participate in the professional development;
Pre/post surveys of program administrators who participate in the professional development offerings; and
Interview protocols for use with a sample of teachers immediately after the professional development concludes and one year later.
The survey of teachers and program administrators will determine respondent background and program information, perspectives on learning and teaching math, and practices prior to and at the completion of the professional development. The surveys were adapted from two main sources: The Teachers Investigating Adult Numeracy (TIAN) questionnaire, from a project funded by the National Science Foundation (Grant No. ESI-0455610); and the 2006 Local Systemic Change Teacher Questionnaire, from Horizon Research, Inc.
For both teachers and program administrators, the background portion of the survey will collect information about:
Credentials and years teaching (or supervising) adults and mathematics;
Current teaching/program situation, including type or level and characteristics of students in math classes;
Institutional sponsorship of adult education program;
Demographic characteristics.
For teachers, the surveys will also collect information on:
Confidence in teaching math, specifically, in teaching the content areas relevant to the professional development (Number and Operation Sense; Patterns, Functions, and Algebra; Geometry and Measurement: Data, Statistics, and Probability);
Current instructional practices and support provided by local administrators for their teaching;
Perspectives on learning and teaching math;
Challenges and barriers to providing strong programs;
Opportunities for professional development; and
Expectations/goals for participating in the professional development.
For program administrators, the surveys will also collect information on:
Awareness of good practices and expectations for instructional practices in their program;
Current support offered to teachers and understanding of good support practices;
Challenges and barriers to providing strong programs;
Opportunities for professional development; and
Expectations/goals for participating in the professional development.
For the cognitive assessment, two separate, parallel forms have been developed. The items from these forms were selected from the Teacher Knowledge Assessment System (TKAS) from the Learning Mathematics for Teaching (LMT) Project at the University of Michigan. Items were selected that cover the topics and constructs that will be covered in the ANI. The forms were created by LMT-trained professionals, and a table describing the summary statistics is presented along with the instruments. Two additional items were added from the TIAN Project Assessment 3 (A Focus on Data and Graphs and Algebraic Thinking). A rubric for scoring these items is included with this package. These forms include teacher learning on topics specifically addressed during the professional development sessions, including:
Number and Operation Sense
Patterns, Functions, and Algebra
Geometry and Measurement
Data, Statistics, and Probability
Finally, the interviews will allow researchers to obtain detailed information about teacher confidence, current instructional practices, and learning from the professional development services. An additional interview, conducted one year after the third Institute, will provide information on lasting changes to instructional practices and understanding of mathematics.
Some of the primary issues to be investigated in this study include:
What do teachers learn from the institutes about concepts related to Algebra, Geometry, Numbers and Operations, and Data, Probability, and Statistics?
What effect does the professional development have on teacher’ confidence in teaching math?
What effect does the professional development have on teachers’ plans for using specific instructional practices and on their practices in the classroom?
What effect does the professional development have on teachers’ perspectives on learning and teaching math?
How does the professional development affect teachers’ need for support, i.e., the kind of support they need?
What are teachers’ expectations for the institutes and how well do they feel the goals were met?
What specific benefits do teachers report for the professional development and what improvements do they suggest?
What effect does the professional development have on administrators’ knowledge of math and comfort in supporting teachers?
What effect does the professional development have on administrators’ awareness of instructional practices and perspectives on teaching math?
What effect does the professional development have on their plans for supporting teachers in their programs?
What are administrators’ expectations for the institutes and how well do they feel the goals were met?
What specific benefits do administrators report for the professional development, and what improvements do they suggest?
Answers to these and other questions will help OVAE understand the impact of the professional development on teachers’ content knowledge, confidence in teaching math, and plans to change instruction and classroom practices.
In order to maximize response rates, the survey will be administered to participants at the professional development sessions. Respondents will complete the survey using pencil and paper. Survey responses will be hand-entered into a database and entries will be double-checked for quality assurance. In the case of this data collection where respondents are together for a professional development session, it seems more feasible to collect data using paper and pencil instruments rather than try to have computers available for electronic collection. Since we don’t currently know the venues where the professional development will take place in each state, we don’t know what computer resources will be available at the facilities. If, however, we find that the facilities do offer such resources, we will endeavor to use them to reduce the paper burden.
Because this project represents the evaluation of a new professional development program, no duplicate data collection efforts exist.
The survey will be administered to teachers and program administrators who participate in the professional development, as part of the professional development institute; therefore, we do not expect data collection to place a large burden on small entities. In addition, the survey is designed to include only a small number of questions critical to addressing the research questions listed above. We estimate that the survey will take only about 20 minutes to complete, on average. Teachers and program staff should be able to answer all of the questions based upon their own attitudes and knowledge and should not require search of records.
Data will be collected at various intervals during this study. The frequency of data collection varies by respondent. Teachers will receive the survey and cognitive assessment at the first and after the third professional development sessions. Program administrators will receive the survey at the first and after the third professional development sessions. Teachers will be interviewed at the end of the third professional development session and at the end of the school year following the first institute. The institutes will take place in October and December of 2010 and March of 2011. The teachers will participate in the last follow-up interview between March-April 2011.
No special circumstances of data collection are anticipated.
There was no publication in the Federal Register related to this project. It was not applicable to this project.
During the initial stages of design and development, the Agency consulted with: Mary Jane Schmitt and other staff members of TERC, subcontractor to this project and a not-for-profit education research and development organization dedicated to improving mathematics, science, and technology teaching and learning; Lynda Ginsburg, an education researcher at the Center for Mathematics, Science and Computer Education at Rutgers University, a subcontractor to this project; and Myrna Manly, project consultant and a mathematics teacher with experience at many academic levels, who was the Mathematics Specialist for the 1988 version of the GED test. These individuals gave guidance and feedback on the instruments and ensured alignment between the data collection and the professional development efforts.
There will be no payments or gifts provided to respondents for completing this survey.
The survey instructions indicate that responses will only be reported in summary form. The data collection contractor, MPR Associates Inc., is a licensed user of individually identifiable data from the National Center for Education Statistics. They adhere strictly to the requirements of the Privacy Act of 1974, 5 U.S.C. § 552a, Public Law No. 93-579, which imposes strict constraints on the use of individually identifiable data and serious consequences for the unlawful disclosure of these data. In addition, it is standard practice at MPR to separate survey responses from respondent identifying information, to assign a random respondent ID, and to maintain the cross-walk between the random ID and the identifying information in a secure, locked location. These requirements go above and beyond federal requirements. MPR has always received the highest marks for data confidentiality procedures during site visits for compliance by the federal government. Furthermore, MPR personnel with access to the completed surveys have completed “Contractor Security Screening” in accordance with OM:5-101.
The survey does not include any questions of a sensitive nature.
Projected estimates for response burden for the cognitive assessment are based upon information from the item developers as well as a small-scale pilot test. The summary information is presented in the table below. The time it took to complete the cognitive assessment, including reviewing instructions, and completing the test, was approximately 20 minutes. Similarly, we expect the survey to take approximately 20 minutes to complete. The same group of 20 teachers in each site will take the pre-survey at the beginning of the first institute and the post-survey at the end of the third institute. They will take the first cognitive assessment at the beginning of the first institute and the second cognitive assessment at the end of the third institute. This equals a response burden of 80 minutes for the 20 teachers in each of two sites for a total of 53.33 hours. The ten program administrators in each of two sites will receive the survey two times, for a total of 13.33 hours of administrator time. This equals a grand total of 67 hours of response burden. Again, because these instruments will be completed during the professional development sessions, they do not represent additional burden on the adult education programs.
Field Test |
Response Time |
Teachers
|
Total Response Time |
Cognitive Assessment (Form A) |
20 mins. |
40 |
800 mins. |
Cognitive Assessment (Form B) |
20 mins. |
40 |
800 mins. |
Pre-survey |
20 mins. |
40 |
800 mins. |
Post-survey |
20 mins. |
40 |
800 mins. |
|
80 mins. |
40 |
3200 mins. = 53.33 hrs. |
Instrument |
Response Time |
Administrators
|
Total Response time |
Pre- Survey |
20 mins. |
20 |
400 mins. |
Post-Survey |
20 mins. |
20 |
400 mins. |
|
40 mins. |
40 |
800 mins. = 13.33 hrs |
Total for instruments for teachers = 53 hrs.
Total for instruments for administrators= 13 hrs.
Total for teachers and administrators = 67 hrs.
The first round of interviews will be conducted at the end of the third Institute, and we will be conducting them with a sample of fewer than 10 of the participants (30 minutes each for a total of 4.5 hours). The final data collection effort, the interviews conducted at the end of the school year after participation in the professional development, are also expected to last approximately 30 minutes. Researchers will interview up to 9 teachers, for approximately 4.5 hours of total time.
There are no capital, startup, or operating costs to programs for participation in the survey. No equipment, printing, or postage charges will be incurred by respondents.
The total cost to the federal government of the data collection, analysis, and reporting will be approximately $236,468. The breakdown of these costs include $82,779 for developing the instruments; $153,689 for data collection, data analysis and reporting. Direct costs to the federal government include time for staff to review and comment on the reports.
This is a new program, so changes are due to program change.
The formal contract for this project requires a final technical report on the results of the evaluation of the professional development. The current data collection and reporting schedule is as follows:
Activity Date
Data collection begins 09/1/2010
Data collection complete 06/30/2011
Final report first draft 08/15/2011
Final report final draft 09/10/2011
The expiration date for OMB approval of the information collection will be displayed on the data collection instrument and materials. No special exception to this request is requested.
There are no exceptions to the certification statement identified in the Certification for Paperwork Reduction Act Submissions of OMB Form 83-i.
| File Type | application/vnd.openxmlformats-officedocument.wordprocessingml.document |
| File Title | Measuring Learner Progress in Adult Education Study: 2007 |
| Author | sharona |
| File Modified | 0000-00-00 |
| File Created | 2021-02-03 |