Cognitive Testing Report

Green Jobs and Healthcare Implementation Study Questionnaire: Survey Pre-Test Results and Changes

Overall, participants did not have trouble taking the survey and felt the questions are clear and response options are adequate. Some participants pointed out suggestions and other recommendations that would make the survey easier to understand. Some overall comments that we heard regarding both grantee and partner surveys are:

• Clarify survey introductions and Frequently Asked Question. Several respondents recommended that “What is the purpose of the study?” and “What information do you intend to collect?” be moved to the beginning of the Frequently Ask Questions section.

• Add navigation instructions to the introduction so that if a respondent does not want to be directed to the Frequently Asked Questions, it is clear they can skip to the first question of the survey.

• Add a feature that allows respondents to save their responses and return to the survey at a later time to complete.

• Provide clarification and reword questions that are unclear. For example, several respondents were unclear about the meaning of the phrase “leveraged any local, state or Federal policies and programs”. The question will be rephrased.

• Increase the allowed text limit for all text boxes. Some participants ran out of space when typing in the text boxes.

• Add examples for response options. Several participants asked for examples of life and soft skills because they were unsure of their meaning.

• Add “I Don’t Know” response option to a number of questions, especially questions that only have a yes or no option. We found in some cases participants did not know the answer and are forced to choose an option that is inaccurate.

• Adjust skip patterns. Particularly, with the question, “Were existing curricula used in developing the training program?” If the respondent answers yes, they are still directed to answer question “How long did it take to develop a new curriculum?” This skip is incorrect and will be adjusted.

• Standardize partner type response options. Some questions only mentioned 3 partner types and other questions mention 4 partner types.

• Correct grammar errors

Respondents made the following recommendations specific to the Grantee Survey:

• Adjust questions/response options to accommodate national grantees that have sub-grantees that operate with individual sets of partners. Several respondents did not know how to answer certain questions because they have multiple sub-grantees and did not know to which sub-grantee the response should apply.

• Adjust questions/response options to accommodate grantees with multiple training programs. We found that respondents had several training programs and could not answer the questions because they were unsure to which training program it should apply.

• Clarify questions regarding timeline for implementation. Some questions are written in past tense, so participants were unsure if they should answer the question about current or past situations.

• Clarify questions that ask about the organization vs. grant program. Some questions are phrased to ask about services that the organization offers and we found that in some cases organizations offer services that are not offered in the Green Jobs / Healthcare grante program. This was changed to reflect the grant program being studied, and not the organization in general.

• Add response options for questions regarding types of individuals that were targeted by the grant program such as incumbent worker and the unemployed.

• Add response option to include sustainability plans that are currently being developed. We found that many participants do not currently have a formal sustainability plan already fully developed, but they are in the process of developing one.

Respondents made the following recommendations specific to the Partner Survey:

• Adjust the Frequently Asked Questions to accommodate partner respondents.

• Add questions pertaining to curriculum development