0056 Supporting Statement Part B

SHIP–SMP SURVEY OF GROUP OUTREACH AND EDUCATION EVENTS 0985-0056 Supporting Statement Part B

Supporting Statement Part B for Collections of Information Employing Statistical Methods

SHIP–SMP Survey

of Group Outreach and Education

Events

1. Describe (including a numerical estimate) the potential respondent universe and any sampling or other respondent selection methods to be used. Data on the number of entities (e.g., establishments, State and local government units, households, or persons) in the universe covered by the collection and in the corresponding sample are to be provided in tabular form for the universe as a whole and for each of the strata in the proposed sample. Indicate expected response rates for the collection as a whole. If the collection had been conducted previously, include the actual response rate achieved during the last collection.

The respondent universe for this survey project will be the number of individuals who attend group outreach and education sessions each year. Based on the most recent data available (2018), there were over 80,000 outreach and education events, with a total audience of over 900,000 individuals. As ACL will sample nine months of presentations each year, the annual sample will be three-quarters of the total population, or roughly 700,000 individuals each year. To generate a sample with a 95% confidence level each year 384 responses will be required (see Table B-1-1 below).

ACL will exceed the response requirement for two reasons:

1. Each year ACL will survey both SHIP and SMP events. Over a nine-month period in 2018, there were 222,000 attendees at SMP outreach events, leading to a requirement of 383 responses at the 95% confidence level. Over the same period in 2018, there were 477,000 attendees at SMP outreach events, leading to a requirement of 384 responses at the 95% confidence level. As a result, the minimum requirement almost doubles, from 384 to 767.

2. Each state will be required to survey at least one event conducted by each program. With 11 individuals completing a survey at an average event in Fiscal Years 2018, 2019, and 2020, the expected total will be closer to 594 responses per program (54 states x 1 event per state x 11 responses per event = 594) than 383 or 384. This brings the total collection to 1188, as opposed to 767.

A traditional sampling methodology cannot be applied to the attendee population because events are only scheduled 2-3 weeks before they occur, and therefore the entire universe of attendees is only known at the end of a given year.

Furthermore, no personal or demographic information is collected from event attendees, further limiting the impact of sampling methodologies and potential stratification. In response to these limitations, ACL will apply the following criteria to each state/territory’s administration of the survey:

1. Each state/territory must survey no less than one presentation from a SHIP outreach event and no less than one presentation from a SMP outreach event (no fewer than two total events).

2. Each state/territory must collect no less than 10 completed surveys from a SHIP outreach event and no less than 10 completed surveys from a SMP outreach event.

3. No location can be surveyed more than once.

4. No presenter can be surveyed more than once.

In addition, ACL will be responsible for selecting events for surveying. At the beginning of each quarter, ACL will randomly select 18 states/territories who will be required to survey one event in the following three months. ACL will ask each of those states/territories to send them a list of events with date, location, presenter, and expected audience included. Based on this information, ACL will then randomly select the event that each state/territory should survey and communicate directly with the presenter to ensure they are prepared for the event. This process will be applied to both the SHIP and SMP programs.

Once an event has been chosen, the presenter will give event attendees an opportunity to complete a comment card survey at the conclusion of the presentation. Table B-1-1 indicates how many completed surveys ACL expects each year.

Table B-1-1: Expected Number of Completed SMP Surveys by Year

Period	Responses Needed for a Representative Survey*	Contacts Needed to Achieve an 80% Response Rate**	Expected Responses
Year 1	767	959	1,200
Year 2	767	959	1,200
Year 3	767	959	1,200
GRAND TOTAL	2,301	2,877	3,600

*These response totals are based on 2018 education session attendees for both the SHIP and SMP programs.

**Because of the nature of this survey, which provides a voluntary survey to individuals immediately after they have attended a presentation but does not attempt to contact them at their home or place of business afterwards, a traditional response rate statistic is difficult to estimate. ACL’s response rate statistic applies more to the event than the attendee, where response rate would be defined as:

Scheduled events that produce one or more completed surveys

/

Total Number of Scheduled Events.

By this measure ACL’s response rate on its existing survey is close to 100% (currently 99% with additional events scheduled but not yet surveyed). ACL conservatively estimates an 80% response rate moving forward.

2. Describe the procedures for the collection of information including:

   1. * Statistical methodology for stratification and sample selection,

The group outreach and education survey will be stratified at the state/territory level to determine which 18 states and territories are required to participate in a given quarter in each survey year. The states/territories will be selected at random. ACL will select individual events to survey for each state/territory by reviewing the state/territory’s schedule of events and considering the relevant survey criteria, including:

1. No less than one presentation from a SHIP outreach event and no less than one presentation from a SMP outreach event.

2. No less than 10 completed surveys from a SHIP outreach event and no less than 10 completed surveys from a SMP outreach event.

3. No location can be surveyed more than once.

4. No presenter can be surveyed more than once.

Once an event has been selected for surveying, the state/territory presenter will bring comment card surveys to his/her presentation. Each comment card will display the OMB Control Number and Expiration Date, the name and location of the event, and the presenter’s first and last name. Prior to the presentation, the presenter will notify attendees of the opportunity to complete a comment card survey. Once the presentation is over, the presenter will instruct the attendees on where to obtain a survey, how to complete the survey, how long it should take, and how to ask questions about the survey. Once the surveys are completed, the presenter will collect the surveys and then scan them to a third-party contractor for data collection, storage, and analysis.

2. * Estimation procedure,

ACL will derive its total sample based on three factors:

1. The number of group outreach and education session attendees in a given year;

2. The number of respondents required to obtain a 95% confidence level, and,

3. A margin of error of +/- 5%.
   
   
   

3. * Degree of accuracy needed for the purpose described in the justification,

In order to generate a national sample of the 54 states/territories, only 384 responses are required. As such, ACL is oversampling, as 600 responses are expected for both the SHIP and SMP programs (1200 responses in total).

4. * Unusual problems requiring specialized sampling procedures, and

There are no unusual problems requiring specialized sampling in this collection.

5. * Any use of periodic (less frequent than annual) data collection cycles to reduce burden.

This is an annual survey.

2. Describe methods to maximize response rates and to deal with issues of non-response. The accuracy and reliability of information collected must be shown to be adequate for intended uses. For collections based on sampling, a special justification must be provided for any collection that will not yield "reliable" data that can be generalized to the universe studied.

ACL is confident in its ability to obtain reliable data through its use of sampling criteria to select education sessions available for information collection. The processes outlined above ensure that the survey responses will meet the requirement for a statistically significant collection (600 total responses per program), will represent all states/territories that work in the SHIP and SMP programs (no fewer than one event and 10 responses per state/territory), and will not be biased by state selection effects (ACL will select the states and events to survey in a particular quarter).

In addition, ACL will take the following steps to ensure adequate responses:

1. Use a notable agency for data collection:

All surveys will contain ACL seals and logos, as well as the OMB Control Number, thereby improving the confidence respondents have with the survey.

2. Describe the survey request, both before and after presentations:

Attendees will be made aware of the option to complete comment cards both before and after the presentation at the outreach event, with specific instructions on where to find the surveys, how to complete them, the number of questions (8), and how long it should take an individual attendee to complete the survey (5 minutes).

2. Describe any tests of procedures or methods to be undertaken. Testing is encouraged as an effective means of refining collections of information to minimize burden and improve utility. Tests must be approved if they call for answers to identical questions from 10 or more respondents. A proposed test or set of tests may be submitted for approval separately or in combination with the main collection of information.

As this is a renewal of an existing collection, ACL has performed extensive testing of the survey instrument and its administration. To date, over 5000 completed surveys have been collected, with an expectation of 7500 completed surveys by May 2020. Based on this data collection, ACL has validated the expected time to complete the survey of five minutes. Informal discussions with State Directors about this survey have been universally positive, indicating that there have been no complaints about undue burden or a lack of utility in the data collection.

2. Provide the name and telephone number of individuals consulted on statistical aspects of the design and the name of the agency unit, contractor(s), grantee(s), or other person(s) who will actually collect and/or analyze the information for the agency.

Two members of the ACL staff will play a primary role in this project:

• Sara Vogler – Program Analyst, 202-795-7461, [email protected]

• Rebecca Kinney – Director, Office of Healthcare Information and Counseling, 202-795-7397, [email protected]

This submission was prepared in consultation with contractors from CG Strategy, specifically David Spak, M.P.P., P.M.P., ([email protected]). Mr. Spak can be reached at 703-527-7001.

With respect to specific tasks, the following individuals were responsible for:

• Designing the data collection: Ms. Kinney, Ms. Vogler, and Mr. Spak.

• Collecting the data: Ms. Vogler and Mr. Spak.

• Analyzing the data: Mr. Spak.

4