OMB Question: ACL notes a power analysis is not required to determine if sufficient sample size exists to conduct minimum detectable differences if the analysis is intended to be descriptive only, rather than to provide estimates that would be used to inform policy making.
Could ACL provide the basis for determining the sample size, then? We would want a method to be used as a rule to determine if the minimum sample size is achieved and what sample size would exceed the maximum necessary for the purposes of this descriptive analysis.
The method ACL references should be consistent with the intended use of the information (that is, how the resulting statistics would be used). They might check our guidance at >https://www.whitehouse.gov/sites/default/files/omb/assets/omb/inforeg/pmc_survey_guidance_2006.pdf<.
ACL Response: In response to the question, Supporting Statement Section B.1.3 provides the basis for determining the Local Service Provider (LSP) survey sample. This section outlines the sample size calculation formulas and assumptions for selecting the numbers of Area Agencies on Aging (AAAs) and LSPs to generate descriptive statistics with reasonable confidence intervals and acceptable levels of confidence for the LSP survey.
ACL proposes to recruit 1,000 LSPS from 274 AAAs (per the sample size calculation for AAAs in Section B.1.3) plus the 10 states each operating as a “single state Planning and Service Area.” While the expectation has been that an 80% response rate will yield a dataset of 800 LSPs (about 10% of the 8,000 LSPs providing NFCSP caregiver training/respite services), there are compelling reasons to adjust the expected response rate downward:
There is now a shorter timeframe for conducting the LSP survey before the evaluation contract ends, which will limit time for non-respondent follow ups
Assuming data collection may begin soon, the survey will be fielded during summer vacation months when LSP staff coverage may be more limited
A comparable survey of LSPs for the Nutrition Evaluation conducted Spring 2015 achieved an 63% response rate
It is more likely at this point to expect the LSP survey will be completed with 63% response rate. Of the 1,000 LSPs we expect to recruit, this would yield a total of 630 LSP respondents, slightly more than 10% than the minimum sample size of 559 LSPs required to generate adequate confidence intervals and sufficient representation of LSPs across the two strata of size and urbanicity. We strongly support this oversampling approach since, without advance knowledge about the extent to which LSPs vary depending on their urban/rural settings and the large, medium or small size of the AAA holding their service contract, this approach will ensure adequate representation from these different types of service environments.
In terms of a sample size that would exceed the maximum necessary for the purposes of this descriptive analysis, it would be likely to be one assuming a 99% confidence interval rather than the 95% confidence interval assumed for our calculations. Using the same assumptions including a 63% response rate, the maximum number to be recruited would be 1,456 LSPs, which is significantly higher than the proposed sample size.
| File Type | application/vnd.openxmlformats-officedocument.wordprocessingml.document |
| Author | Dooling, Bridget C.E. |
| File Modified | 0000-00-00 |
| File Created | 2021-01-24 |