SUPPORTING STATEMENT
Part B
Examining How Local Public Health Departments Can Leverage Age-Friendly Cities Initiatives to Build Resilience in Elderly Populations
Version 3
Table of contents
B. Collections of Information Employing Statistical Methods 3
1. Respondent Universe and Sampling Methods
2. Procedures for the Collection of Information
3. Methods to Maximize Response Rates and Deal with Nonresponse
4. Test of Procedures or Methods to be Undertaken
5. Individuals Consulted on Statistical Aspects and Individuals Collecting and/or Analyzing Data
Attachment A - Authorizing Legislation
Attachment B - Published 60-day FRN
Attachment C - Survey for Older Adults
Attachment D - Interview Guide for Age Friendly Initiative Staff
Attachment E – Interview Guide for Senior Village Directors
Attachment F – Interview Guide for Local Health Department Representatives
Attachment G – Recruitment Letter for Age Friendly Initiative Staff
Attachment H – Recruitment Letter for Senior Village Directors
Attachment I – Recruitment Letter for Local Health Department Representatives
Attachment J – Institutional Review Board (IRB) Approval Exemption Letter
1.A. Interviews with Key Informants
Potential respondent universe. The target universe for the interviews with key informants comprises three types of respondents (a) Senior Village (SV) executive directors; (b) Age Friendly Initiative (AFI) staff; and (c) Local Public Health Department (LHD) officials. AFIs in the U.S. can be either city-based or county-based and are led by city/county-level administration, health departments, academic centers, and/or volunteer organizations. SVs are neighborhood-based and are grassroots organizations usually led by older adult residents.
Sample size. We will recruit no more than 30 SV executive directors, 31 staff from AFIs, and 15 LHD officials.
Sampling and Selection. Recruitment of the convenience sample involves stratification only for the SVs. We expect to ask a screening question (during recruitment stage) to identify whether EP activities have or will be administered. We will aim for the 30 SV executive director interviewees to be split in half: 15 executive directors of SVs that do provide some type of EP activity and 15 executive directors of SVs that do not provide some type of EP activity.
We will work with the Village-to-Village Network to identify and recruit the SV executive directors to participate in the interviews. Each respondent type will be sent a letter of recruitment (Attachment G – Recruitment Letter for Age Friendly Initiative Staff, Attachment H – Recruitment Letter for Senior Village Director & Attachment I – Recruitment Letter for Local Health Department Representatives). With regard to Age Friendly staff, the administration of AFIs differs widely across sites. We will work with the American Association of Retired Persons (AARP) Public Policy Institute to identify lead staff for AFIs that are registered with the AARP as part of the World Health Organization (WHO) Global Network of Age Friendly Cities and Communities. Local health department officials will be identified through the interviews with AFI staff and SV executive directors who are willing to share their contacts at their local health department. For informants that do not have a LHD contact, RAND will identify the local city/county health department official responsible for developing and implementing health and emergency preparedness services for older adults.
Type of Key Informant |
Community |
Maximum Number of Respondents |
Senior Village Executive Directors |
Senior Villages (i.e., neighborhoods part of the Village-to-Village Network) |
30 |
Age Friendly Initiative Staff |
Cities or Counties registered as an AARP/WHO Age Friendly Community |
31 |
Local Health Department Officials |
City, County, or State |
15 |
Total |
|
76 |
1.B. Surveys of Older Adults
Design. This study is designed to compare adults age 65+ living in 8 communities with senior villages (SV) that engage in emergency preparedness (EP) activities (condition 1), with older adults living in 8 communities with SVs that do not engage in EP activities (condition 2), and with 15 control communities without SVs or other age AFIs (control condition 3). The 16 communities with SVs (with and without EP activities) will first be selected, and then we will sample from within the population of adults 65+ living in these communities. We will then match this sample to our sample of adults 65+ in 15 control communities, based on both individual and community characteristics (more detail below). We will be comparing the following resilience-related outcomes between older adults in these three conditions (i.e., living in a community with a SV and EP activities, living in a community with a SV but no EP activities, living in a control community with no SV or AFIs): disaster resilience, social connectedness, self-sufficiency, emotional resilience, and attention to health needs. We are looking to determine whether adults 65+ have better resilience related outcomes in communities with SVs, and whether these outcomes are even further improved if adults 65+ are living in a community with both a SV that engages in EP activities. Our sample size reflects the minimum sample size needed to compare adults 65+ on these key resilience-related outcomes, accounting for clustering at the community level and taking into account our use of propensity weighting. To maximize our power we will be pooling our samples of adults 65+ living in the 16 communities with SVs (condition 1 and condition 2) to compare resilience-related outcomes to adults 65+ in our 15 control communities. We have also designed the study to have enough power to detect differences between resilience-related outcomes of adults 65+ in our two SV conditions (condition 1- communities with SV and EP activities vs. condition 2 – communities with SVs but no EP activities).
Sample size. The sample will comprise of 400 adults 65+ from 8 communities with senior villages (SV) that engage in emergency preparedness (EP) activities (condition 1), 400 adults 65+ living in 8 communities with SVs that do not engage in EP activities (condition 2), and 750 adults 65+ living in 15 control communities without SVs or other age AFIs (control condition 3). Sample sizes in each of these conditions have been selected in order to make comparisons with minimum detectable differences with 80% power (see power calculations in subsequent Sampling Plan section).
Sampling Plan. A two stage sampling plan will be employed. In the first stage we will select communities in each of three conditions: (a) SVs with EP activities in place, (b) SVs without EP activities in place, and (c) control communities. Then we will select adults 65+ years from each of those conditions.
We will select communities from (a – condition 1) and (b – condition 2) based on the knowledge gained in the key informant interviews which will ask about any emergency preparedness activities taking place in the same location as the SVs location.
Condition 1: To maximize our ability to find differences between adults 65+ in the three conditions, we will select the 8 communities with SVs that are doing the most comprehensive set of EP activities (condition 1). We will identify these EP activities through our interviews with SV directors during which we ask them to describe the scope of their EP activities.
Condition 2: We will select the 8 communities with SVs that are not doing any EP activities and have not done any planning of EP activities. Similar to condition 1, we will identify these communities through our interviews with SV directors during which we ask them about the scope of their EP activities and whether they have started planning for any EP activities.
In the event that there are more than 8 communities identified for each condition, we will purposefully select villages that cover geographically the regions of the US (2 from Northeast, 2 from West, 2 from South, 2 from Midwest), are both urban and rural (at least 3-4 urban, 3-4 rural, and 3-4 suburban), and that vary in their disaster risks (e.g., 2 with major risks of flooding, 2 with major risks of earthquakes, etc).
We will then select control communities (condition 3), by matching each SV community to a control community. Control communities will be selected using a nearest neighbor algorithm to choose comparison communities which are close matches to the condition 1 and 2 communities based on several characteristics below. We will use census tract-level Census/American Communities Survey (ACS) and National Disaster Housing Risk data to determine control and condition 1 and 2 community characteristics.
socioeconomic status such as median household income or % living below the federal poverty level (Census/ACS data),
% of older adults 65 and older (Census/ACS data),
% of minority residents (Census/ACS data),
% urban population (Census/ACS data), and
likelihood of experiencing a disaster (National Disaster Housing Risk data).
The nearest neighbor algorithm finds the closest community that mostly closely matches the condition 1 and 2 communities on the selected characteristics above. This helps to address potential differences due to geographic variation (e.g., having two matched communities in different states with different policy contexts). Whereas SVs most often exist at the neighborhood level (census tract), AFIs most often exists at the city or county level. The presence of an AFI may influence residents’ resilience. As such, we will ensure that the selection control communities do not include communities within an AFI by checking the community against AARP’s list of currently active AFIs.
At the second stage of sampling, we will sample individual seniors from within these communities. SV members (i.e., residents living in SVs) will be identified and recruited in two ways: member lists with contact information will be submitted to the research team by SV executive directors or SV executive directors will send a recruitment letter on our behalf. We will sample to target a sample of 50 adults 65+ from each of the 8 communities with SVs with EP activities for a total of 400 older adults in condition 1, 50 older adults from each of the 8 communities with SVs but without EP activities for a total of 400 older adults in condition 2, and 50 adults 65+ from each of the 15 control communities for a total of 750 older adults in condition 3.
Anticipated response rates. For the 16 communities with SVs (condition 1 and 2), we will select members who are nominated or recruited by SV executive directors, and we anticipate a 75% response rate.
For control communities, we will use random digit dialing, and anticipate a 40% response rate based on prior experience surveying by random digit dialing.
Our nonresponse pool (n=1431) is derived based on the estimated survey response rates for the communities with SVs (with and without EP) and the matched control communities.
Type of Community |
Number of Communities |
Maximum Number of Survey Respondents |
Senior Village with EP |
8 |
400 |
Senior Village without EP |
8 |
400 |
Matched Control Community |
15 |
750 |
Total |
31 |
1550 |
Power to detect differences between resilience-related outcomes in the pooled sample of adults 65+ in communities with SVs (condition 1 and condition 2) and adults 65+ in communities without SVs or AFIs (control condition 3). We estimate the power of the analysis to detect differences between the SV members and matched controls, on a continuous outcome scale. Both the clustering of individuals in communities, and the propensity weights introduce a design effect which reduces the effective sample size (and thereby reduces the precision of the study).
For the design effect introduced by clustering, it is necessary to estimate the intra-class correlation (ICC), that is the degree of similarity of individuals within the same community, on the outcome variables (when any covariates have been incorporated). Based on our prior experience, we estimate an ICC of 0.05. The design effect due to clustering is given by:
dEff = 1 + (m - 1) ICC
Where m is the mean cluster size – that is, the mean number of individuals sampled per community. The sampling plan gives this value as 50, and therefore the design effect due to clustering is equal to 3.45. The design effect due to the survey weights depends on the variance of the weights, which is dependent on the degree of similarity of the samples prior to matching. We anticipate that the samples will be similar and therefore that the design effect due to weights will increase the overall design effect by 20%, giving a final design effect of 4.14.
The effective sample size is given by N / dEff, where N is the original sample size. The effective sample size is therefore given by 1550 / 4.14 = 374.
The effective sample size of 374 individuals, divided equally between intervention (being a senior village member) and control, provides greater than 80% power to detect a difference of d=0.29, that is a difference of 0.29 standard deviation units. This is a conservative estimate of power as we have not incorporated the effects of covariates which are likely to increase power.5
Power to detect differences between adults 65+ in communities with SVs and EP activities (condition 1) and adults 65+ in communities with SVs but no EPs (condition 2). For comparison of SVs with and without EP, assuming the same design effect (which is likely to be more conservative as SV residents will be more similar to one another) we estimate that the study will have 80% power to detect a difference of 0.58 standard deviation units.
We anticipate using the twang1 package within the R6 environment to estimate propensity scores. Regression models will be estimated using Stata, version 13 [or the most recent version available at the time of analysis]. Because our survey procedures will screen out non-eligibles, and takes only 20 minutes to administer, we do not anticipate little missing data, however, if missing data appear to be problematic we will use a multiple imputation approach, as implemented using the MI command in Stata version 13 [or most recent version available].
2.A.
Interviews with Key Informants
Procedures for the collection of information. To assess the variability in AFI and SV communities and identify opportunities for integrating community resilience goals and activities into their development and ongoing activities, we will conduct qualitative interviews using interview guides (Attachment D - Interview Guide for Age Friendly Initiative Staff & Attachment E – Interview Guide for Senior Village Directors). These interview guides ask about the AFI or SVs structure, stage of implementation, linkage with public health departments, and whether (and what types) of EP activities are provided to older adults in their community.
Estimation procedure. We will review interview notes and transcripts for examples that suggest processes, behaviors, and assumptions12 and will look for repetition and variation across content that may indicate relevant themes. After separately examining portions of the transcripts, two researchers will reach consensus about which themes to examine in detail and will then develop a codebook. To help identify all instances of a theme, 2 raters will read the material and code independently. We will analyze data using qualitative data management software (e.g, Atlas ti). Results generated from these interviews will be summarized into a set of findings and potential recommendations to aid LHDs in integrating CR goals/activities into AFI and SV communities.
Degree of accuracy needed for the purpose described in the justification. In qualitative research, saturation of information is emphasized over size or representativeness of the sample. We chose the sample sizes to ensure accuracy of information we will receive within any of the three respondent types. There are currently 31 Age Friendly Cities/Counties and 180 active SVs. We will be talking with all Age Friendly Cities/Counties and 15% of SVs. Based on our prior experience, this should be representative enough to achieve saturation of information.
2.B.Surveys
of Older Adults
Procedures for the collection of information. We will conduct a random digit dial (RDD) survey of 1,550 older adults to evaluate the effects of living in a SV (being village member) versus not living in a SV, and the effects of living in a SV with EP activities versus living in a SV without EP activities. The outcomes and control variables we will measure in the survey of older adults are: disaster resilience, social connectedness, self-sufficiency, emotional resilience, attention to health needs, exposure to AFIs, age, gender, race/ethnicity, length of time living in community, current living situation, income, and presence of chronic health conditions. To improve response rates we will offer a $20 incentive to participants.
Estimation
procedure. We anticipate
using survey weighted regression techniques to estimate the
difference in outcomes between older adults that are members of SVs
(with and without EP in place) and older adults in matched control
communities, which will incorporate both the survey weights and
correct standard errors for clustering. We will use a double
robust3,4
approach in which any covariates, which were failed to be fully
balanced by the propensity model, will be incorporated into the
regression model. All estimates will use the propensity weights
derived from the boosted regression procedure (described above). In
addition, standard errors will be corrected for clustering within
communities using a sandwich estimator approach. Estimates will
adjust for respondent demographic, social, clinical, and behavioral
characteristics obtained via survey.
3.A. Interviews with Key Informants
We have already reached out to the Village-to-Village Network and our research team includes AARP staff who are responsible for overseeing the implementation of Age Friendly Cities/Communities. Thus, through these two high-level organizations, we expect no problems with identification and recruitment of key informants. Our qualitative data collection efforts through key informant interviews are not meant to be representative of all AFI and SV staff, rather, we are striving for saturation of themes that arise across all informants. Thus, we do not expect any problems with non-response bias as we are not making inferences from key informants’ responses on any outcomes.
3.B. Surveys of Older Adults
The older adult respondents will be drawn from three types of communities selected in the first sampling stage, based on whether the community is a SV and whether there are EP activities taking place (described above in “Sampling Plan”). The largest source of non-response bias will likely arise from the second stage of sampling in which individuals are selected within each of these communities.
For the two SV communities (condition 1 and 2), respondents will be recruited via SV executive directors. For SVs that have data on their members’ characteristics (e.g., age, sex, race/ethnicity, marital status, presence of a disability), we will compare our respondents’ distribution of characteristics to the data from SVs. For SVs that do not have data on their members’ characteristics, we will use Census/ACS data on the census tract’s characteristics as proxy data with which to compare our respondents’ characteristics. The census tract will be chosen based on geocoding of the SV to its census tract ID(s) (SVs are approximately the size of census tracts, a cluster of census blocks within a census tract, or located within a cluster of census tracts). In both of these cases, we can use inverse probability weighting to weight the respondents on any differences in characteristics to reduce non-response bias in our data analyses.
For the control community (condition 3), respondents will be recruited via RDD. The contractor selected to conduct the RDD will attempt to ask non-respondents for basic demographic data. These data will be used to determine whether respondents are statistically significantly different from non-respondents. In lieu of these data, we will compare respondent characteristics (e.g., age, sex, race/ethnicity, marital status, presence of a disability) to census tract-level data on targeted community to determine whether respondents’ characteristics are statistically significantly different from those of residents of the same census tract. If differences exist, we can report on those differences or apply inverse probability weighting to weight the respondents on any differences in characteristics to reduce non-response bias in our data analyses.
4.A. Interviews with Key Informants
We conducted pilot testing of each of the interview guides with three respondents who are familiar with Age Friendly Cities/Counties and SVs. The average time of administration for the interview guide was approximately 30 minutes, with the local health department interview being the shortest to complete (average of 20 minutes) and the SV representative interview being the longest to complete (max of 45 minutes).
4.B. Surveys of Older Adults
We conducted pilot testing of the survey with four respondents who are and are not familiar with Age Friendly Cities/Counties and SVs. The average time of administration for the survey of SV members and matched controls was 18 minutes.
Jeremy
Miles, Senior Behavioral Scientist and Quantitative Psychologist at
RAND and professor of quantitative methods in social science at RAND
Pardee Graduate School
Gregg P. Stickeler, Sr. Vice President Client Services, ISA, LLC
References
1. twang: Toolkit for Weighting and Analysis of Nonequivalent Groups, 1.2.5 [computer program]2011.
2. Harder VS, Stuart EA, Anthony JC. Propensity Score Techniques and the Assessment of Measured Covariate Balance to Test Causal Associations in Psychological Research. Psychological Methods. 2010;15(3):234-249.
3. Schafer JL, Kang J. Average Causal Effects From Nonrandomized Studies. Psychological Methods. 2008;13(4):279-313.
4. Neugebauer R, van der Laan M. Why prefer double robust estimators in causal inference? Journal of Statistical Planning and Inference. 2005;129(1-2):405-426.
5. Taylor MJ, Innocenti MS. Why covariance: A rationale for using analysis of covariance procedures in randomised studies. Journal of Early Intervention. 1993;17(4):455-467.
6. R: A Language and Environment for Statistical Computing [computer program]. Vienna, Austria: R Foundation for Statistical Computing; 2014
1 Yin RK. Evaluation: A Singular Craft. In: Reichardt CS, Rallis SF, eds. The Qualitative-Quantitative Debate: New Perspectives. San Francisco: Jossey-Bass; 1994:71-84.
2 Lasker R, Weiss E. Broadening participation in community problem solving: a multidisciplinary model to support collaborative practice and research. J Urban Health. 2003;80(1):14-47.
| File Type | application/vnd.openxmlformats-officedocument.wordprocessingml.document |
| File Title | SUPPORTING STATEMENT |
| Author | wcarroll |
| File Modified | 0000-00-00 |
| File Created | 2021-01-24 |