The Department of Labor (DOL) contracted with Abt Associates in partnership with the Urban Institute, NORC at the University of Chicago, George Washington University, and Capital Research Corporation to conduct the National Evaluation of Round 4 of the Trade Adjustment Assistance Community College and Career Training (TAACCCT) grant program. DOL is seeking approval from OMB under the Paperwork Reduction Act (PRA) for data collection instruments associated with the evaluation. The TAACCCT grant program provides community colleges and other eligible institutions of higher education with funds to expand and improve their ability to deliver education and career training programs that can be completed in two years or less and are suited for workers who are eligible for training under the Trade Adjustment Assistance for Workers (TAA) Program. TAACCCT-supported programs prepare participants for employment in high-wage, high-skill occupations. Through these multi-year grants, DOL is helping to ensure that institutions of higher education are helping adults succeed in acquiring the skills, degrees, and credentials needed for high-wage, high-skill employment while also meeting the needs of employers for skilled workers. DOL is implementing the TAACCCT program in partnership with the U.S. Department of Education. A total of 49 grants were awarded in Round 1 in FY 2011, 72 in Round 2 in FY 2012, 57 in Round 3 in FY 2013, and 71 in Round 4 in FY 2014. Some of the grants were awarded to single institutions and others to consortia of institutions.
OMB previously cleared three information collection requests (ICRs) for this and prior rounds of TAACCCT. Under OMB Control No. 1291-0004, OMB approved a baseline information form (BIF) and a self-administered questionnaire (SAQ) for an Impact Study of selected Round 4 grantees and a semi-structured interview guide for an implementation study of Round 4 grantees. Under OMB Control No. 1291-0007, OMB approved a semi-structured interview guide for site visits to grantees in Rounds 1-3, a guide for focus groups involving students served by Rounds 1-3 grantees, and a College Survey for all colleges associated with Rounds 1-3 grantees. A third ICR was for an extension of the collection of performance information for the TAACCCT grants (Control No. 1205-0489). Information is still being collected under all three ICRs.
The Round 4 evaluation, funded by the DOL Chief Evaluation Office (CEO) in partnership with the Employment and Training Administration (ETA), will use a multi-pronged approach including 1) an Outcomes Study1 of 9 selected grantees, 2) an implementation study, 3) a synthesis of grantee third party evaluations, and 4) a study of strong grantee-employer partnerships.
In this document, CEO and ETA request OMB clearance for the following Round 4 data collection activities: 1) a follow-up survey of Outcomes Study participants, 2) a participant tracking form for Outcomes Study participants, 3) a college survey for the implementation study, and 4) a discussion guide for a qualitative study of strong grantee-employer relationships.
The TAACCCT program is authorized by Division B of the American Recovery and Reinvestment Act of 2009 (P.L. 111-152), and the Health Care and Education Reconciliation Act of 2010 appropriated $500,000,000 annually from Fiscal Years 2011-2014 for competitive grants to eligible institutions of higher education. The program aims to improve education and employment outcomes for TAA-eligible workers and other adults attending community college and other higher education institutions by helping these institutions build capacity to provide effective occupationally-focused education and training programs of two years or less in duration in the 50 States, the District of Columbia and Puerto Rico. Funding for evaluation activities under this program have been designated by the DOL Employment and Training Administration (ETA) for third-party evaluations of each grant and by CEO for a national evaluation. These evaluation activities will assist DOL in identifying evidence-based programs that are the most promising for TAA-eligible workers and other adults, examining how capacity building and systems reform in community colleges can be achieved, and developing strategies for research and evaluation of similar interventions.
This evaluation is designed to answer the following research questions:
What service delivery and/or system reform innovations resulted in improved employment outcomes and increased skills for participants?
Under what conditions can these innovations most effectively be replicated?
What outcomes were achieved by the participants of the TAACCCT Round 4 grantees’ training programs?
What are the types of emerging ideas for service delivery change and/or system reform that seem the most promising for further research? Under what conditions are these ideas most effective?
What directions for future research on the country’s public workforce system, and workforce development in general, were learned?
To address these research questions, the evaluation of Round 4 TAACCCT will include the following:
Baseline data collection (at programs selected for the Outcomes Study)
Implementation site visits (at Outcomes Study sites)
A participant tracking form (for students in programs selected for the Outcomes Study)
A 12-month follow-up participant survey (for students in programs selected for the Outcomes Study)
A college survey (of all colleges participating in Round 4 TAACCCT grants)
Employer interviews
A match to the National Directory of New Hires (NDNH) to obtain earnings histories (for students in programs selected for the Outcomes Study)
This submission is to request clearance for the third through sixth data collection components listed above. All four proposed data collection activities – the participant tracking form, the follow-up survey, the college survey, and the employer study – are necessary to ensure that the evaluation can adequately document and assess the overall TAACCCT program. Data collection plans for the first two components have been approved, as discussed above. The seventh component is not subject to PRA review, but it is an important part of the evaluation design that greatly reduces the response burden on study participants by decreasing the length of the 12-month follow-up survey.
The immediate users of data collected under this ICR will be the evaluation team (Abt Associates), who will use the data from the participant survey to measure outcomes achieved in select programs at nine grantees. The evaluation team will use college survey data to understand the ways that the grantees spend their grant dollars on program design and improvement. The team will rely on data from the employer interviews to produce a special study on strong employer relationships.
Indirect users of both types of data will be policy analysts at DOL, Congress, and others who may base future policies, at least in part, on the findings of the evaluation. Program designers will also be indirect users of the information through the findings of the evaluation. Community colleges seeking to serve a range of students and tie their programs to local labor demand, and students looking for information about the outcomes of available programs will also be able to make use of the planned final report. All materials developed from the analyses of these data collection efforts are intended to reach multiple audiences including:
DOL and other federal agency staff
Institutions of higher education and their trade groups
State and local workforce agencies and organizations
Industry groups
Researchers
Policymakers at the local, state and federal levels of government looking to design similar programs
Others interested in understanding the experiences and lessons from the community college training and capacity-building programs
Finally, a Public Use File covering information about study participants will be prepared and made available to future researchers.
Follow-Up Survey
The follow-up survey administration will use a tri-modal approach of web, telephone follow-up, and in-person follow-up. All three modes will use the same Computer-Assisted Interviewing (CAI) technology. CAI technology reduces respondent burden, as respondents and interviewers can proceed more quickly and accurately through the survey instruments, minimizing the interview length. Computerized questionnaires ensure that the skip patterns work properly, minimizing respondent burden by not asking inappropriate or non-applicable questions. Computer-assisted interviewing can build in checkpoints, which allow the interviewer or respondent to confirm responses, thereby minimizing data entry errors. Finally, automated survey administration can incorporate hard edits to check for allowable ranges for quantity and range value questions, minimizing out of range or unallowable values.
Participant Tracking Form
The tracking letter offers both a website and a toll-free number that respondents may use for updating their contact information if they find that less burdensome than the traditional option of a paper form and business return envelope.
College Survey
Respondents will receive a link and password to a web-based version of the college survey. The survey will be created in Qualtrics, a commercial software application for development and administration of online surveys. The automated skip patterns embedded in the online survey place less of a burden on the respondent than the customary “if-then go to” instructions of a paper and pencil questionnaire. The questionnaire will be in modular formats that allow the primary respondent to pass sections or questions on to other staff members who may be better equipped to address particular topics.
This will be the first Outcomes Study of the TAACCCT grant program, and the first to present a national view of the characteristics of TAACCCT participants, their service receipt, and their outcomes related to education, employment, earnings and receipt of public benefits.
Regarding the follow-up survey, there is minimal duplication of data collection. The follow-up survey will ask about employment status despite the availability of some employment information in the NDNH data. This survey question is necessary as a screener to asking about job characteristics, information not available through NDNH. Additionally the follow-up survey will ask about participation in education and training programs and attainment of credentials. Some data on college records and service receipt are collected by grantee colleges; however, they are college specific and likely of varying quality. For the Outcomes Study, it is critical that the evaluation team collect identical information from study participants across all nine grantees. Similarly, there is no other method to track study participants between the time of study intake and the follow-up survey.
Regarding the college survey, grantees currently submit narrative quarterly reports to DOL, but not in a standardized way that allows data analysis and does not present the detail needed for a national evaluation. In contrast to the quarterly reports, the college survey collects data in an objective and quantifiable manner that is necessary for analysis and summation in the evaluation report. In addition, the survey captures data at the college level, not just at the grantee level, allowing us to understand the breadth of activities undertaken with TAACCCT grant funds.Regarding employer interviews, the evaluator will collect limited information during 9 implementation research site visits about the role of employers in the grant programs. The employer interview protocol is designed to collect in-depth information on a subset of employers selected for the strength of their relationship with grantees.
The data collection for the employer study may involve small businesses. The data collected under this request is the minimal necessary for the small businesses involved. None of the other data collection operations will involve small businesses.
The follow-up survey will be administered approximately 12 months after study intake to participants. This is the only follow-up survey planned. It is a one-time data collection activity, so it would not be possible to collect this data less frequently. While the evaluation could still report on the employment outcomes of TAACCCT-funded programs in the absence of a survey using NDNH data, the survey allows for a look at a broader set of education and employment-related outcomes of interest to DOL, including attainment of credentials, enrollment in and completion of other education and training programs, characteristics of jobs, total income, and public benefit receipt.
Achieving a high response rate is dependent on being able to successfully re-contact study participants. Conducting quarterly tracking activities that provide study participants the opportunity to update their contact information increases the likelihood that the evaluator will be able to locate the participant for the follow-up survey.
The college survey, administered once, is critical to the implementation study, which will document how TAACCCT-funded programs are operated across all grantees. Given the significant expenditures involved in the TAACCCT grants, and the role that this and similar grant programs are intended to play in shaping the nation’s workforce system, it is important to document the different models and projects that are operating under the initiative, examine and assess the implementation to date, and identify innovative features and potentially promising strategies. The college survey represents the only opportunity to gather comprehensive and in-depth information on implementation from all grantees in Round 4.
The team will conduct interviews with employers only one time. This data will be the only information available to document the development and nature of strong partnerships, which will inform future DOL grant programs.
There are no special circumstances for the proposed data collection.
In accordance with the Paperwork Reduction Act of 1995 (Pub. L. 104-13 and Office of Management and Budget (OMB) regulations at 5 CFR Part 1320 (60 FR 44978, August 29, 1995)), DOL published a notice in the Federal Register announcing the agency’s intention to request an OMB review of this information collection activity. This notice was published on Wednesday, June 22, 2016, Volume 81, Number 120, pages 40,720 to 40,721, and provided a 60 day period for public comment. A copy of this notice is included as Appendix C. During the notice and comment period, the government did not receive any public comment or request for copies of the instruments.
The individuals listed in Exhibit A8 made a contribution to the project and are external to DOL.
Exhibit A8: Individuals Consulted Outside of the Agency
Name |
Telephone Number |
Role in Study |
Karen Gardiner |
(301) 347-5116 |
Project Director |
David Judkins |
(301) 347-5952 |
Co-Principal Investigator |
Lauren Eyster |
(202) 261-5621 |
Co-Principal Investigator |
Burt Barnow |
(202) 994-6379 |
Project Advisor and Task Lead |
For the follow-up survey to be most successful, the evaluator determined that monetary gifts should be provided to study participants in appreciation of the time they spend on data collection activities. These tokens of appreciation are a powerful tool for maintaining low attrition rates in longitudinal studies. The use of monetary gifts for the follow-up survey can help maximize response rates. Three factors helped to determine the gift amount for the follow-up survey:
Respondent burden;
Costs associated with participating in the interview at that time; and
Other studies of comparable populations and burden.
Previous research has shown that sample members with certain socioeconomic characteristics are significantly more likely to respond to surveys when there is a monetary gift. In particular, sample members with low incomes and/or low educational attainment have proven responsive to incentives (Duffer et al. 1994; Educational Testing Service 1991).
Sample members that complete the 12-month follow-up interview will receive a check for $25 as a token of appreciation for their time. The Round 4 TAACCCT 12-month follow-up survey is comparable to other surveys of similar populations conducted by Abt Associates, such as the Pathways for Advancing Careers and Education (PACE) Evaluation 15-month follow-up survey and the Health Profession Opportunity Grants (HPOG) Evaluation 15-month follow-up survey (both conducted for the Administration of Children and Families at HHS), and the Green Jobs and Health Care Impact Evaluation 18-month follow-up survey (conducted for DOL).
For the study participant tracking letters mailed at 3 months and 9 months post study enrollment (but not following study intake and at 6 months), $2 in cash will be included in the letter as a thank you for updating or confirming contact information. An extensive body of research documents the effectiveness of prepayments (see Church 1993, Yammarino, Skinner, and Childers 1991; Edwards et al. 2002; Edwards et al. 2005).
Respondents of the college survey and employer interviews will not receive payment.
The evaluation team developed strong protocols to help maintain the privacy of respondents to the extent permitted by law. All research staff working with personally identifiable information (PII) will follow strict procedures to protect private information and they will sign the standard DOL contractor nondisclosure agreement, stating that they will keep all information gathered private to the extent permissible by law. All papers that contain respondent names or other identifying information will reside in locked areas and passwords will protect any computer documents containing identifying information.
The evaluation team will take the following actions to protect survey respondents’ privacy:
Use rigorous security measures for follow-up survey data. The contractor has established safeguards that provide for the privacy of data from participants on all of its studies. All data users are aware of and trained on their responsibilities to protect participants’ personal information, including the limitations on uses and disclosures of data. All personal data (identifiable and de-identified data analyses files) will reside on a secure workstation or server that is protected by a firewall and complex passwords, in a directory that can only be accessed by the network administrators and the analysts actively working on the data. Survey data collected are stored in secure CATI servers. Data transfer to and from Abt and Abt SRBI (survey firm) will occur through Abt’s secure online file transfer platform that utilizes FIPS 140-2 validated cryptographic modules. Evaluators assign generic study identifiers – not based on PII – for each study participant to link participant data. PII is removed from all electronic files prior to analysis.
Notification of data security breaches. All study partners, including the grantees that are participating in the outcomes evaluation, are aware that they must notify Abt Associates within one hour of a breach of PII privacy per OMB rules. Study partners are also aware that they must notify Abt within 24 hours from the time any study partner knows of a breach/deviation from the data security plan developed for the study. Evaluators will notify DOL of any data security breaches, including breaches of protocol no later than 24 hours after Abt staff is made aware of the breach.
Restricting access to the study network folder. Secure servers will store all data collected that contains PII for the Round 4 TAACCCT Outcomes Study. Access to the study network will be restricted by assigning a password to each relevant staff member.
In addition to these study-specific procedures, the evaluator has extensive corporate administrative and security systems to prevent the unauthorized release of personal records. These systems include state-of-the-art hardware and software for encryption that meets federal standards and other methods of data protection (e.g., requirements for regular password updating), as well as physical security that includes limited key card access and locked data storage areas.
Respondents to the follow-up survey will be assured that their responses will be kept private to the extent allowed by law and will not be shared with anyone outside of the evaluation team in a manner that would allow respondent identification unless the evaluation team is legally ordered otherwise. All findings from the survey will be presented at the aggregate level and with a minimum cell size of 3 to avoid re-identification. Please see the first pages of each data collection instrument – Appendix B – for the respondent privacy statements imbedded in the informed consent procedures.
The college survey will not collect PII. While the survey is still active, access to any data with identifying information will be limited only to contractor staff directly working on the survey and will require special usernames and passwords. Once the survey is closed to respondents, responses will be downloaded for analysis from the SQL server database and kept on a controlled access, encrypted network drive. Hard copies of the survey will be entered into the electronic format and kept in a locked file cabinet. All survey hard copies will be shredded upon completion of the evaluation.
Similar to the college survey, the employers interviewed of for the employer study will be assured that their responses will be kept private allowed by law and will not be shared with anyone outside of the evaluation team in a manner that would allow respondent identification unless the team is legally ordered otherwise. To protect the privacy of the employer respondents, interview notes (with identifiers) and recordings will be secured and destroyed once findings are published.
None of the questions on the 12-month follow-up survey are sensitive in nature. The only somewhat sensitive questions relate to income and public benefit receipt. Since TAACCCT programs should result in increased income and reduced public benefit receipt, these are important domains to measure. Interviewers will remind study participants during the interviewing process that they may refuse to answer individual items. Interviewers will also provide assurances to participants that their responses will be kept private to encourage candid responses.
None of the questions on the tracking form, college survey and employer interview protocol are sensitive.
Exhibit A12 presents the estimated respondent burden on study participants for the follow-up survey, participant tracking form, college survey, and employer interviews.
The evaluator estimates that the follow-up survey will take respondents approximately 20 minutes (0.33 hours) on average to complete and the participant tracking form will take five minutes (0.083 hours) to complete each of the two times it is administered. The evaluation team used estimates from similar surveys, for the PACE Evaluation 15-month follow-up survey (OMB No. 0970-0397) and the HPOG Impact Evaluation 15-month follow-up survey (OMB No. 0970-0394), both conducted for HHS, and the Green Jobs-Health Care Impact Evaluation 18-month survey (OMB No. 1205-0506), conducted for DOL. The burden estimate is based on a total sample of 4,000 respondents (based on an 80 percent response rate for 5,000 field surveys). The burden is annualized by three years.
The evaluation team estimates the college survey will take an average of 1.5 hours to complete. The evaluation team used the survey response time for Round 2 colleges plus an additional 5 minutes for the new questions on employer partners to estimate this average response time. Respondents will be primary representatives of the participating colleges (including the grant organization) deemed to have sufficient knowledge of the TAACCCT grant activities to complete the survey. Specifically, the respondent will most likely be the program coordinator or an administrator at the college. The estimated response rate is 90 percent. Although participation in evaluation activities is required as a condition of the grant award, it is likely that due to changes in staffing, about 10 percent of colleges will not respond to the survey.
The employer interviews are expected to take approximately one hour. Employers will be selected based on the strength of their relationship with grantees. Respondents are most likely to be the human resources managers at firms. As these are highly engaged employers however, the evaluator expects a high response rate of 80 percent based on previous experience with recruiting for interviews from similar DOL grant initiatives.
Exhibit A12: Annualized Estimated Respondent Hour and Cost Burden for the Follow-Up Survey, Participant Tracking Form, College Survey, and Employer Interviews
Instrument |
Total Number of Respondents |
Annual Number of Respondents |
Number of Responses Per Respondent |
Average Burden Hours Per Response (in Hours) |
Total Burden Hours |
Average Hourly Wage |
Total Burden Cost |
Follow-Up Survey |
4,000a |
1,333 |
1 |
0.33 |
440 |
7.25b |
$3,190 |
Participant Tracking Form |
5,000 |
1,667 |
2 |
0.083 |
277 |
$7.25 |
$2,008 |
College Survey |
245c |
82 |
1 |
1.5 |
123 |
43.79d |
$5,386 |
Employer Interviews |
40e |
13 |
1 |
1 |
13 |
56.29f |
$732 |
Total |
|
|
|
|
853 |
|
$11,316 |
a This assumes a sample of 5,000 with an 80 percent response rate.
b The hourly wage of $7.25 is the federal minimum wage (effective July 24, 2009) http://www.dol.gov/dol/topic/wages/minimumwage.htm.
c This assumes a sample of 272 with a 90 percent response rate.
d The median hourly wage for Education Administrators was calculated based on information from the Bureau of Labor Statistics; May 2015 National Industry-Specific Occupational Employment and Wage Estimates found at http://www.bls.gov/oes/current/naics4_611300.htm: Education administrators, postsecondary (NAICS 611300, SOC code 11-9033) wage rate of $43.78.
e This assumes a sample of 50 with an 80 percent response rate.
f The median hourly wage for Human Resource Managers was calculated based on information from the Bureau of Labor Statistics; National Occupational Employment and Wages, May 2015, 11-3121 Human Resources Managers, found at http://www.bls.gov/oes/current/oes113121.htm.
There are no direct costs to respondents, and they will incur no start-up or ongoing financial costs. The cost to respondents involves solely the time involved in completing the follow-up survey, participant tracking form, college survey, or employer interviews. These costs are captured in the burden estimates in Exhibit A12.
The information collection activity and associated instruments have been developed by the evaluation contractor, Abt Associates, in performance of Contract Number: GSA MOBIS 874-01, Order Number: DOL-ETA-14-F-00013. CEO and ETA are funding the costs of the study. The proposed follow-up survey, participant tracking form, college survey, and employers interviews will be fielded starting within one month of receiving OMB approval (participant tracking form), September 2017 (participant survey and college survey), January 2018 for the employer interviews and will end no later than September 29, 2019.
The total annualized cost to the federal government is $1,056,042. Costs result from the following two categories:
1. The estimated cost to the federal government for the contractor to carry out this study is $2,640,309 for follow-up survey data collection (including instrument development), $192,530 for fielding the participant tracking forms, $254,184 for the employer study, and $26,009 for the college survey. Annualized over three years, this comes to (($2,640,309 + $192,530 + $254,184 + $26,009)/3 = $1,037,677.
2. DOL expects the annual level of effort for Federal government technical staff to oversee the contract will require 200 hours for one Washington D.C.-based l GS-14, Step 4 employee earning $57.39 per hour (See Office of Personnel Management 2016 Hourly Salary Table: https://www.opm.gov/policy-data-oversight/pay-leave/salaries-wages/salary-tables/pdf/2016/DCB_h.pdf ). To account for fringe benefits and other overhead costs the agency applies a multiplication factor of 1.6. The data collection period covered by this justification is three years, so the estimated total cost for performance of these duties is $55,095. The estimated annual cost borne by DOL for these duties is $18,365.
This is a new data collection.
The evaluation will cover implementation, outcomes and employer relationship analyses.
Implementation Analysis. The evaluation team will analyze 1) data collected as part of the site visits to the nine Outcomes Study sites (approved under OMB Control No. 1291-004), 2) data collected from the college survey, and 3) data collected from the employer interviews.3 For the nine Outcomes Study sites, following the site visits the team will organize data collected into short summaries that align with the TAACCCT conceptual framework (see Appendix A). This will provide a portrait of each grantee in terms of identified workforce needs, grant activities, capacity building operations, expected outcomes, and program context. These summaries will suggest exploratory questions for the Outcomes Study and will help the team interpret Outcomes Study findings. The evaluation team will also synthesize the findings to describe the similarities, differences, implementation successes and challenges across the nine grantees. Special topics analyses will follow a similar model; the evaluators will create summary reports that highlight key findings. For the college survey, the in Qualtrics software enables the automatic tabulation of responses that reduces both the hours of staff time needed for survey processing and the possibility of introducing human error into the data.
Outcomes Analysis. The research questions for the outcomes analysis will be mostly answered with tabulations of means and percentages along with corresponding standard errors. Comparisons will be made between groups defined both by program characteristics and by participant characteristics. The follow-up survey instrument will support analyses of four major topic areas:
Training receipt, quality, and educational progress. Questions focus on occupational training, such as how long students persisted in the program, how many credits they earned (including credit for prior learning), how many hours of instruction they engaged in, what credentials they earned, and their impressions of the training. The survey includes questions related to hands-on training opportunities (e.g., in labs or simulations) and the specific occupational skills participants acquire.
Training-related supports. These include tutoring, career advising, and job search services, as well as who provided the assistance and whether this assistance yielded useful job leads. Additional questions focus on how participants finance their training.
Employment characteristics. These focus on employment characteristics that are not available through administrative (NDNH) records, such as post-training hourly wages, whether the job is in the area in which the participant trained and whether the job has benefits provided by the employer.
Income and receipt of public benefits. These include total household income, household composition (i.e., the number of adults and children, which will be used to calculate income relative to poverty), and public assistance benefits received.
For administration of the follow-up survey, participant tracking form, college survey, and employer interviews, CEO and ETA are seeking OMB approval beginning within one month of approval and ending September 29, 2019.
Exhibit 16.2 presents an overview of the project schedule for information collection. It also identifies deliverables associated with each major data collection activity.
Exhibit A16.2 Overview of Project Data Collection Schedule
Data Collection Activity |
Timing |
Associated Publications |
|
From September 2017 – November 2018 |
Final report |
|
Within one month of OMB approval |
N/A |
|
September 2017 – December 2017 |
Final report |
|
January 2018 – April 2018 |
Report on Employer Relationships |
All instruments created for the Round 4 TAACCCT national evaluation will display the OMB approval numbers and the expiration dates for OMB approvals.
No exceptions are necessary for this information collection.
Church, Allan H. 1993. “The Effective of Incentives on Mail Survey Response Rates: A Meta-Analysis.” Public Opinion Quarterly 57:62-79.
Duffer, A., J. Lessler, M. Weeks, & M. Mosher 1994. Effects of Incentive Payments on Response Rates and Field Costs in a Pretest of a National CAPI Survey. Proceedings of the Survey Research Methods Section of the American Statistical Association, Vol. 2, pp. 1386-1391.
Educational Testing Service. 1991. National Adult Literacy Survey Addendum to Clearance Package, Volume II: Analyses of the NALS Field Test (pp. 2-3).
Edwards, P., Cooper, R., Roberts, I., and Frost, C. 2005. Meta-Analysis of Randomised Trials of Monetary Incentives and Response to Mailed Questionnaires. Journal of Epidemiology and Community Health 59:987-99.
Edwards, P., Roberts, I., Clarke, M., DiGuiseppi, C., Pratap, S., Wentz, R., and Kwan, I. 2002. Increasing Response Rates to Postal Questionnaires: Systematic Review. British Medical Journal 324:1883-85.
Yammarino, F., Skinner, S. and Childers, T. ( 1991). Understanding Mail Survey Response Behavior: A Meta-Analysis. Public Opinion Quarterly 55:613-39.
1 Since the approval in October of 2015 by OMB of baseline data collection for the planned Impact Study under Control No. 1291-0004, the evaluation plan has changed. DOL and the evaluation team concluded that the 71 Round 4 grants were not appropriate for an impact study. The programs that were of sufficient size for an impact study either had very dispersed activities or there was no natural point where random assignment could occur. After careful consideration DOL opted to replace the planned Impact Study with an Outcomes Study. The BIF approved for the Impact Study will be used for the Outcomes Study. The SAQ will not be used.
2 The Round 4 SGA (as well as SGAs for prior rounds) can be found at http://www.doleta.gov/taaccct/applicantinfo.cfm.
3 The employer interviews will be analyzed in the same manner as the implementation study, but will be included in a separate report.
Abt
Associates Inc. DRAFT-Supporting
Statement for OMB Clearance Request ▌pg.
| File Type | application/vnd.openxmlformats-officedocument.wordprocessingml.document |
| File Title | Abt Single-Sided Body Template |
| Author | Katheleen Linton |
| File Modified | 0000-00-00 |
| File Created | 2021-01-23 |