This appendix provides detailed justifications for items in the Baseline Information Form (BIF) and Self-Administered Questionnaire (SAQ).
As noted in Part A of the package, because some items in the SAQ ask about personal strengths and weaknesses related to readiness for training and employment, individuals may feel more comfortable responding to questions if they know that grant/program staff will not see their answers. This also helps to address potential concerns from participant that their responses would influence their research group assignment. The SAQ will be completed after the BIF but still prior to random assignment. Study participants would fill out the paper form that will include a unique study ID, but no other identifiers. Participants would seal it in an envelope so that grant/program staff would not see the responses. The form would be shipped to the evaluators for data entry.
A developing literature has demonstrated the importance of a variety of psycho-social skills in predicting perseverance in and completion of post-secondary training and in favorable employment outcomes. Major constructs in this category include: (1) the ability to self-motivate, commit to goals, and sustain discipline in pursuing them; (2) the ability to develop and maintain positive beliefs and feelings about one’s self and others; (3) skills needed to engage socially and develop social supports (actual and perceived) for one’s school and work pursuits; and (4) the ability to open oneself to new ideas and experiences, to be creative. A growing body of research is establishing important, if complex and variable, evidence on these factors’ malleability and their connections to education and employment outcomes. Accordingly, the evaluators recommend measuring four specific psycho-social skills as part of the baseline data collection for the TAACCCT evaluation: grit, core self-evaluation, perceived stress, and math anxiety. The evaluation team also proposes measuring one generalized occupational skill that they theorize will be related to post-secondary education success: computer skills. Detailed notes follow on each.
Grit. (Question 1)
The 8-item version of this scale (Duckworth, et al., 2007) has very strong psychometrics (excellent reliability and demonstrated validity). In addition, early analyses of PACE data (Pathways for Advancing Careers and Education, formerly known as ISIS, OMB No. 0970-0397) confirmed that Grit has a strong correlation with post-secondary persistence at 15-month follow-up.
Core Self-Evaluation. (Question 2)
The 12-item version of this scale (Judge, 2009) has very strong psychometrics. In addition, early analyses of PACE data confirmed that Core Self-Evaluation has a strong correlation with post-secondary persistence at 15-month follow-up.
Perceived Stress. (Question 3)
The 4-item version of this scale (Cohen, Kamarck, and Mermelstein, 1983) has very strong psychometrics. In addition, early analyses of PACE data confirmed that Perceived Stress has a strong correlation with post-secondary persistence at 15-month follow-up.
Computer Skills. (Question 4)
This question measures computer skills, as a generalized occupational skill that may be an issue for older workers who are retraining for new professions. A recent DOL report (Heidkamp, Mabe and DeGraaf, 2012) stated that, “The lack of technology skills came up repeatedly in the literature and in interviews as a particular hindrance for many older job seekers. Levine (2010) cited research that found that the high demand for computer skills is one of the factors that is positively correlated with older job seekers’ limited access to employment.” To measure this trait, the evaluation team proposes to use the seven items in the background survey used for the Program for the International Assessment of Adult Competencies (PIAAC) (NCES 2012).
Math anxiety: (Question 5)
Woodward (2004) reviews the importance of math anxiety in the success of post-secondary developmental students. To the extent that programs funded by TAACCCT grants train older and other nontraditional students, we think that a good measurement of this trait could also be a powerful addition to regression models for treatment effects.
The original scale for measuring this trait has 98 items (Richardson and Suinn, 1972), which is long for a baseline data collection tool. Alternatives include the 9-item Abbreviated Math Anxiety Scale (AMAS) (Hopko, et al., 2003). Although the AMAS was developed on a younger sample (undergraduates with a mean age of 19.6 years) than may be enrolling in TAACCCT services, Jameson and Fusco (2014) report using the AMAS on adult learners without problem. In fact, they found no difference in math anxiety between adult learners and traditional undergraduates. Accordingly, the evaluation team added this scale to the SAQ.
Alexander, L. and Martray, C. (1989). The development of an abbreviated version of the mathematics anxiety rating scale. Measurement and Evaluation in Counseling and Development, 22, 143-150.
Cohen, S., Kamarck, T., and Mermelstein, R. (1983). A global measure of perceived stress. Journal of Health and Social Behavior, 24(4), 385-396.
Duckworth, A.L., Peterson, C., Matthews, M.D., and Kelly, D.R. (2007). Grit: Perseverance and passion for long-term goals. Journal of Personality and Social Psychology, 92(6), 1087-1101.
Heidkamp, M., Mabe, W., and DeGraaf, B. (2012). The Public Workforce System: Serving Older Job Seekers and the Disability Implications of an Aging Workforce. The NTAR Leadership Center: Rutgers, NJ. http://www.dol.gov/odep/pdf/NTAR_Public_Workforce_System_Report_Final.pdf
Hopko, D.R., Mahadevan, R., Bare, R.L., and Hunt, M.K. (2003). The Abbreviated Math Anxiety Scale (AMAS): Construction, validity, and reliability. Assessment, 10(2), 178-182.
Jameson, M.M. and Fusco, B.R. (2014). Math anxiety, math self-concept, and math self-efficacy in adult learners compared to traditional undergraduate students. Adult Education Quarterly, 64(4), 306-322.
Judge, T.A. (2009). Core self-evaluations and work success. Current Directions in Psychological Science, 18(1), 58-62.
Levine, L. (2010). Older displaced workers in the context of an aging and slowly growing population. Washington, D.C.: Congressional Research Service. Retrieved from: http://aging.senate.gov/crs/aging22.pdf.
Plake, B.S. and Parker, C.S. (1982). The development and validation of a revised version of the Mathematics Anxiety Rating Scale. Educational and Psychological Measurement, 42, 551-557.
Richardson, F.C. and Suinn, R.M. (1972). The Mathematics Anxiety Rating Scale. Journal of Counseling Psychology, 19, 551-554.
Woodward, T. (2004). The effects of math anxiety on post-secondary developmental students as related to achievement, gender, and age. Inquiry, 9(1). http://www.vccaedu.org/inquiry/inquiry-spring2004/i-91-woodard.html
Abt
Associates Inc. DRAFT-Supporting
Statement for OMB Clearance Request ▌pg.
| File Type | application/vnd.openxmlformats-officedocument.wordprocessingml.document |
| File Title | Abt Single-Sided Body Template |
| Author | Katheleen Linton |
| File Modified | 0000-00-00 |
| File Created | 2021-01-25 |