Administration on Aging, Administration for Community Living,
U.S. Department of Health and Human Services
April 17, 2015
Supporting Statement for the
Evidence-Based Falls Prevention Program Standardized Data Collection
Part B: Collection of Information Employing Statistical Methods
The Administration on Aging (AoA), Administration for Community Living (ACL), U.S. Department of Health and Human Services (HHS), proposes to use these data collection tools to monitor grantees receiving cooperative agreements in response to the funding opportunity: “PPHF - 2014 - Evidence-Based Falls Prevention Programs Financed Solely by 2014 Prevention and Public Health Funds (PPHF-2014).” ACL/AoA awarded ten “state” and four “tribal” cooperative agreements for a two-year project period beginning in September, 2014.
This data collection is necessary for uniform monitoring of the Falls Prevention grantees and to provide information for reporting about new PPHF awards authorized under Section 411 of the Older Americans Act of 1965, as amended, and the Patient Protection and Affordable Care Act (ACA), Section 4002, 42 U.S.C. § 300u-11 (Prevention and Public Health Fund).
1. Respondent Universe and Sampling Methods
Participants
It is anticipated that the ACL/AoA grantees will reach an estimated cumulative total of 10,000 program participants annually. Each participant will be asked to voluntarily complete paper tools, the Participant Information Form (before or at the beginning of the first program session) and the Post Program Survey Form (at the end of the last session). Collecting data from the universe of participants is preferable because it reduces the burden on sites of selecting and managing a sampling plan.
While the use of the Participant Information Form has been used previously with a response rate over 90%, the Post Program Form has not been previously used. Therefore, there is no expected response rate for the collection as a whole.
2. Procedures for the Collection of Information - Statistical Methodology
The purpose of this data collection (via the Participant Information Form and Post Program Survey Form) is to obtain performance information for this particular grant program by assessment of identified demographic and key outcome variables.
ACL estimates that there will be approximately 1,150 classes held under this grant program. As the purpose of any sampling approach is to use a small number of objects, classes in this case, to represent the larger group from which they are drawn, ACL will construct a stratified random sample. ACL will stratify the sample by grantee type because previous ACL research shows that Aging grantees (State Units on Aging and Area Agencies on Aging) had higher completion rates than did Public Health grantees and classes specifically targeted to a particular racial/ethnic group had higher completion rates than did other classes. Completion rates are of particular importance to ACL because previous evaluations of these programs show that people that complete the programs have better outcomes than do non-completers. There are currently four grantee types:
Tribes (4 grantees)
State Units on Aging (4 grantees) / Area Agency on Aging (1 grantee)
State Health Departments (3 grantees)
Foundations (2 grantees)
ACL estimates that we would need a sample of 725 classes (63%) for a confidence level of 95% and confidence interval of 2.21 based on the following assumptions:
There is no systematic difference across classes in terms of gender or age.
Approximately, 10% of proposed classes will be cancelled requiring a 10% oversampling.
Classes will include an average of 12 participants with little variation between classes
ACL proposes a two-step process for implementing a sampling plan:
ACL will implement a limited convenience sampling procedure immediately. Specifically, ACL will:
work with the grantees to confirm which sites are able to submit class lists prior to holding classes;
select 50% of the classes to participate in information collection using the ‘Post Survey.’ Because of the small number of participants served through the classes held by Tribal grantees and the significance of this population to the “Empowering Older Adults and Adults with Disabilities through Chronic Disease Self-Management Education Programs”, the Title VI Tribal Grants Program and ACL as a whole, classes offered through the Tribal grantees will be selected with certainty.
review the process and make revisions to ensure that this is a sound approach before it is rolled out with all sites.
ACL will incorporate language in the Year 2 continuation Notice of Awards and future Falls grantee award notices requiring sites to submit class lists to ACL prior to holding classes so that ACL can select a random sample of 63% of the classes that will be asked to collect participant data using the ‘Post Survey.’
Information will be collected by paper form by trained leaders and coaches. As noted previously in Part A, Justification, Leaders and Coaches (as grantees) will disseminate and collect paper Program Data Collection Tools at each workshop:
A Program Information Cover Sheet and an Attendance Log will be completed by the leaders/coaches. This information documents the location of the program, type of program, and the number of participants who completed the program.
A Participant Information Form and a Post Program Surveywill be completed by each participant on a voluntary basis. The Participant Information Form documents demographic and health characteristics, including age, gender, race/ ethnicity, types of chronic condition(s), disability status, and education level. It also assesses some key outcome variables, which will be re-assessed in the Post Program Survey, including falls self-efficacy, falls and injury rates, fear of falling, and interference with social activities.
Completed forms will be sent to a central location, entered into a secure database, and the paper forms will be properly destroyed. Personally identifiable information (PII) will not be collected. There is no sampling plan.
Leaders and coaches will be trained in quality control best practices and an approved script will be made available for their use. (See script in Attachment A.)
ACL/AoA Project Officers will review the semi-annual reports and national compiled data.
3. Methods to Maximize Response Rates and Deal with Nonresponse
To maximize response rates ACL will use in-person administration of the information collection because research has shown that in-person survey distribution has the highest average response rates when compared to mail, e-mail, telephone, or web-based data collection. In-person data collection has also been shown to result in more complete data.
In addition, research shows that high response rates are strongly influenced by the following factors that ACL has incorporated into our approach:
The salience of the topic- Respondents are being asked about a program that they are participating in voluntarily to address an issue, falls, that is personally relevant to them.
Personalized request and communication- Respondents are being given the information collection tools from someone that they know. They are being presented with a specific request to respond based on their program participation.
Information collection tool is concise and easy to complete- Based on pretesting the pre and post-program survey form, which are two pages and consist of 8-14 questions, are easy to complete requiring only 6 minutes to complete.
Information collection tool is easy to return-By administering the information collection in-person, respondents are able to return their forms immediately. They will not have to keep track of the form or remember to send it in at a later time.
Showing positive regard-Group leaders who collect the information will thank respondents for their efforts. The group leaders’ script also talks explicitly about the value of the data to ACL for making future program improvements.
Reducing non-receipt of the information collection: ACL’s in-person approach ensures that respondents receive the information collection form and, thus, reduces non-response due to non-receipt.
As a result of this approach, which has resulted in response rates of over 90% with previous programs, ACL expects a similar response rate for this data collection.
Given ACL’s approach to data collection, the primary reason for non-response will be participant refusal (rather than non-contact or in ability to complete the survey). Overall non-response will be measured by dividing the total number of people who respond to the survey divided by the number of people present in the selected sites on the last session. In addition to the overall non-response rate, non-response rates will also be calculated by site, age, gender, and racial/ethnic group. Specifically, ACL will conduct univariate and bivariate cross-tabulations, and multivariate analysis to detect patterns that interactive effects may mask. For example, a natural cross-tabulation would be by age and gender and may show no distinct pattern. However, a multivariate analysis using age, gender, race or ethnicity and site may show substantial variation. If ACL finds that the non-response rates are significantly higher for any subgroups ACL will work with sites that serve the groups with the highest rates of non-response to improve their response rates in the future and to improve instructions that will be used with futures sites that also serve those groups. As the data collection will be conducted on a rolling basis ACL expects that corrections to improve response rates will, over the course of the data collection, result in non-biases data. If the final data set still shows biases in response rates ACL will:
Compare pre-session data for all members of groups with high non-response rates. For example, if Site A had a disproportionately high non-response rate, the data pre-session data from Site A participants will be compared to pre-session data from participants at other sites to determine if the participants from Site A are significantly different in terms of their demographics and falls related characteristics. In addition, site level characteristics from Site A will be compared to site level characteristics of other sites to determine if Site A if significantly different from those sites. If the participants and site are not significantly different from other sites, then the data will be incorporated into the planned analyses. If significant differences are found with the participants or the sites the data will be used with caution and final decisions about program implementation will not be made based on these data.
Compare the results by group to determine whether the data for respondents in population groups that had high nonresponse rates vary significantly on any of the measured variables. For example, if Asian men tend to have disproportionate rates of non-response, their responses will be compared to 1) All respondents, 2) Male respondents, and 3) Asian respondents to determine if there are significant differences between the groups. If there are no significant differences the data will be incorporated into the planned analyses. If there are significant differences, the data will be used with caution and final decisions about program implementation will not be made based on these data.
4. Tests of Procedures or Methods to be Undertaken
Because the Post Program Survey Form is very similar to the Participant Information Form, which previously received OMB clearance and is being used with a similar population, no pilot of this form is planned. The first step of the proposed two-step sampling roll out will constitute a pilot of the sampling methods.
In 2013, ACL/AoA received OMB approval for a CDSME Information Collection set of tools (OMB Approval Number: 0985-0036; expiration date July 31, 2016). The proposed Falls Prevention information collection request is an adapted version of the CDSME set of tools. The approved CDSME participant data collection tool includes name, birth date and zip code. The proposed Falls Prevention participant form does not request these specific items, only the age of the participant.
5. Individuals Consulted on Statistical Aspects and Individuals Collecting and/or
Analyzing Data
Individuals consulted on statistical aspects for collecting and/or analyzing data include Susan Jenkins, PhD and Alice Lynn Ryssman both from ACL..
Data collection will be managed by the Fall Prevention grantees (see complete list below). The data will be submitted via an online database managed by the National Council on Aging (NCoA), a subgrantee of ACL Grantee Lewin. NCOA will review the data and conduct basic descriptive analyses.
Persons involved in designing the data collection tools included the following individuals:
From ACL: Michele Boutaugh, Sam Gabuzzi, Susan Jenkins, Jennifer Klocinski, Laura Lawrence, Shannon Skowronski
From CDC: Margaret Kaniewski, Judy Stevens
Outside program developers: Jane Mahoney, Patricia League
Outside evaluation experts: Matthew Smith, Marcia Ory
The person responsible for receiving and approving contract deliverables is Laura Lawrence.
Michele Boutaugh and Shannon Skowronski are responsible for reviewing reports generated through the database.
Persons who collect the data are the 14 grantees’ local community partner staff and volunteers. The grantees are:
Colorado Department of Public Health and Environment
Elder Services of the Merrimack Valley
Foundation for Healthy Communities
Georgia Department of Human Services
Hardrock Council on Substance Abuse
Health Foundation of South Florida
Iowa Department on Aging
Little Traverse Bay Bands of Odawa Indians
Match-E-Be-Nash-She-Wish Band of Pottawatomi Indians
Minnesota Board on Aging
North Carolina Department of Health and Human Services
Sokaogon Chippewa Community
State of Vermont
Utah Department of Health
ATtachment A
Falls Prevention Program Group Leader/ Coach Script
Read/ paraphrase the following points to participants prior to their completion of the Participant Information Form.
This workshop is made possible by [a grant from the U.S. Administration on Community Living (ACL) and/or support from X funding agencies/ sponsors].
We would like to give you an optional two-page form today and then at the last class we will again ask you to complete a brief post- survey.
Before we share your information with ACL [and X funding agencies or sponsors], we want to explain how your information will be used and protected.
Your information is very valuable to us. We use it to learn who is being reached by this program and to improve our services. It also helps our funding agencies show that they are spending their money wisely.
At the top of the form, we ask for the first two letters of your first and last name and the last two years of the year you were born. We will use this to match your information to an Attendance Log to track how many times you attend a class and to the post-survey. We do not share this information with anyone else.
The form also asks you to provide some personal information such as your age and gender. You may skip any questions that you do not want to answer. While completing the form, you may ask us to explain any questions that you find confusing.
We follow very strict rules to protect all of your information and to keep it private. We will maintain these paper forms securely following standard practices for protecting private data. After a trained person enters your information into a secure computer database, we will destroy the paper forms.
Completing the form is entirely voluntary. If you decide not to it you can still participate in this program.
Please take time now to read the form and let us know if you have any questions.
| File Type | application/vnd.openxmlformats-officedocument.wordprocessingml.document |
| Author | Windows User |
| File Modified | 0000-00-00 |
| File Created | 2021-01-25 |