Additional Context for ICR Changes

OMB CONTROL NO. 1205-0040

SCSEP Customer Satisfaction Surveys

July 22, 2021

Background

Title V of the Older Americans Act as amended in 2016 requires that Senior Community Service Employment Program (SCSEP) to include in its core measures “Indicators of effectiveness in serving employers, host agencies, and project participants.’’ In its Interim Final Rule published on December 1, 2017, the Department defined this new core measure as “the combined results of customer assessments of the services received by each of these three customer groups.” See section 641.710. In the preamble to this section of the regulations, the Department stated that although WIOA was piloting a new measure of effectiveness in serving employers, that new measure would be insufficient to address SCSEP’s statutory requirement to address program effectiveness for all three of its customer groups. As a result, the Department announced that it would continue to use the existing customer satisfaction surveys for each of its three customer groups “as an interim measure at least until the WIOA pilot is complete and a WIOA measure is defined in final form.” SCSEP has been administering these three customer satisfaction surveys nationwide since 2004.

The interim indicator of effectiveness adopted by SCSEP to satisfy the statutory requirement is the American Customer Satisfaction Index (ACSI). The survey approach allows the program flexibility and, at the same time, captures common customer satisfaction information that can be aggregated and compared at the nationwide, national grantee, and state grantee levels. The indicator is created by combining the responses to the three questions that constitute the ACSI. The index is designed to address different dimensions of customers' experience. In addition to the index questions, other questions in the survey provide grantees with information that guides them in effective management of the program.

The ACSI is a widely respected indicator of the degree to which customers value a particular product, program or service. It is used extensively in business communities in Europe and the United States, including more than 200 companies in 44 industries. In addition, over 100 Federal government agencies have used the ACSI to measure citizen satisfaction with more than 200 services and programs.

As stated in the preamble to section 641.710, during this interim period, the Department has explored with grantees, and with its customer groups, options for best measuring the effectiveness of SCSEP’s services. The Department is also exploring ways to improve the efficiency of the current employer surveys, including changes to the administration of that survey.

SCSEP currently funds 77 grantees, (21 national grantees (two of which have two separate grants) operating in 47states, DC, and Puerto Rico; and 52 state grantees, including DC and Puerto Rico; the three overseas territories and USVI do not participate in the surveys).

Proposed Changes

1. Participant Survey. Based on input from grantees and customers, we are seeking to make minor changes to the wording of Questions 6, 21, and 22 and of the instructions at the beginning of the survey, as set forth in the revised survey dated 12/13/19, for sharper focus and greater clarity. There is no change to the substance of the questions or the instructions, and there is no change to the length of the survey or to the burden hours.

Question 6 changes “…understood my employment interests and needs” to “…gave me a host agency assignment that matched my employment interests and needs.” The change in language focuses the question on the benefit to the customer rather than just the actions and intent of the grantee.

Questions 21 and 22, the two open-ended questions, replace what is most valuable about the program and what changes the customer would recommend to what aspects of the program the customer thinks are most effective and what changes would make the program more effective. The change to “effectiveness” in these two questions was recommended SCSEP grantees in order to reflect the new statutory emphasis on effectiveness for this measure.

The wording change to one sentence in the instructions (”frank” instead of “honest” was suggested by customers. In a second sentence, “confidential” is changed to “private,” and an explanation is added that survey results will only be used for program evaluation and improvement.

2. Host Agency Survey. Based on input from grantees and customers, we are seeking to change the wording of Questions 7, 9, 11, 15, and 16 and of the instructions at the beginning of the survey, as set forth in the revised survey dated 12/13/19, for sharper focus and greater clarity. There is no change to the substance of the questions or the instructions. These proposed changes are not anticipated to alter the estimated response burden in any meaningful way.

The instructions in Question 7 were reworded to ensure that all responses are based on the last participant presented for assignment and not on the respondent’s recollection of all assignments that may have been presented to the respondent’s organization. The answers are meant to be mutually exclusive. Both grantees and customers found the original wording confusing.

In Question 9(4), the word “behave” was changed to “interact” at the suggestion of customers.

The original Question 10 is being eliminated at the suggestion of both grantees and customers because it overlaps substantially with the original Question 11 (now renumbered as Question 10) and provides little additional information.

In the renumbered Question 11, “required” is changed to “need” and “health services” is changed to “medical care” at the suggestion of grantees and customers.

The renumbered Questions 15 and 16, the two open-ended questions, have the same changes as the two open-ended questions in the participant survey.

Two sentences in the introduction have the same changes as the corresponding sentences in the participant survey.

1