Application for New Data Collection:
2010 Study of the Older Americans Act Provider Network
Draft
OMB Review Package
July 12, 2010
Chapter Page
A Justification 1-1
A.1 Circumstances Making the Collection of Information Necessary 1-1
A.2 Purpose and Use of the Information Collection 1-2
A.3 Use of Improved Information Technology and Burden Reduction 1-5
A.4 Efforts to Identify Duplication and Use of Similar Information 1-5
A.5 Impact on Small Businesses or Other Small Entities 1-5
A.6 Consequences of Collecting the Information Less Frequently 1-6
A.7 Special Circumstances Relating to the Guidelines of 5 CFR 1320.5 1-6
A.8 Comments in Response to the Federal Register Notice and Efforts to Consult Outside the Agency 1-6
A.9 Explanation of Any Payments or Gift to Respondents 1-6
A.10 Assurances of Confidentiality Provided to Respondents 1-6
A.11 Justification for Sensitive Questions 1-7
A.12 Estimates of Annualized Burden Hours and Costs 1-7
A.13 Estimates of Other Total Annual Cost Burden to Respondents and Record Keepers 1-9
A.14 Annualized Cost to the Federal Government 1-9
A.15 Explanation for Program Changes or Adjustments 1-10
A.16 Plans for Tabulation and Publication and Project Time Schedule 1-10
A.17 Reason(s) Display OMB Expiration Date is Inappropriate 1-12
A.18 Exception to Certification for Paperwork Reduction Act Submissions 1-12
B. Statistical Methods (used for collection of information employing statistical methods) 2-1
B.1 Respondent Universe and Sampling Methods 2-1
B.2 Procedures for the Collection of Information 2-4
B.3 Methods to Maximize Response Rates and Deal with Nonresponse 2-6
B.4 Test of Procedures or Methods to be Undertaken 2-7
B.5 Individuals Consulted on Statistical Aspects and Individuals Collecting and/or Analyzing Data 2-7
Appendix Page
A Confidentiality Statement A-1
B Pre-site Visit Questionnaire B-1
C AAA Director Interview Script C-1
D Provider Focus Group Moderator’s Guide D-1
Table Page
A-1 Data Collection Topics by Purpose and Use 1-4
A-2 Estimate of the Hour Burden 1-8
A-3 Annualized Cost to Respondents 1-9
A-4 Cost to the Government 1-10
A-5 Project Schedule 1-11
B-1 AoA Service Regions 2-1
B-2 States by AoA Service Region and Percentage of 60+ Population 2-2
Justification |
A |
The Administration on Aging (AoA) is requesting Office of Management and Budget approval to collect information from a sample of Area Agencies on Aging (AAA) and Local Service Providers (LSPs) in order to fill a number of “knowledge gaps” at the Federal level. The information to be collected is primarily qualitative data that will be used to support AoA activities as described in sections A.1 and A.2 below. The survey is being funded by AoA and conducted under a contract with Westat.
Legal Authority: The legal authorities to collect this information are found at 1) Section 205.(a)(1)(C) of the Older Americans Act (OAA) which gives the Assistant Secretary authority to conduct research and demonstrations; and Section 206 of the OAA which establishes the authority to measure and evaluate the impact of all programs authorized by the OAA (http://www.aoa.gov/AoARoot/AoA_Programs/OAA/oaa_full.asp#_Toc153957638).
Study Background: Over the last 10 years a variety of information on two components of the Aging Service Network: 1) State Units on Aging; and 2) Area Agencies on Aging, plus the people who receive services has been gathered by AoA and other groups (e.g. stakeholder organizations). The resulting information has a wide range of uses from planning and policy development to performance measurement to technical assistance. One component of the Aging Services Network, the Local Service Providers (LSPs) has not been studied in a systematic fashion. This is not surprising given that AoA’s direct relationship is with SUAs, which are the entities responsible for reporting program data to AoA, and that both SUAs and AAAs have national organizations that conduct research on and for their members. LSPs are diverse, including but not limited to providers of transportation, nutrition, case management and chores services. No one organization represents their interests and AoA’s relationship with them is not direct. AoA recognizes that basic data on their characteristics and understanding of their relationship with the other Aging Network components and in particular AAAs is an important knowledge gap that is in need of filling.
AoA initially considered a representative sample survey as the approach to filling this gap and the Performance Outcomes Measures Project (POMP) grantees had developed a survey tool that AoA considered using as a basis for a study. However, it became apparent through discussions with POMP grantees and others that a survey study was premature. Evidence suggests that Management Information Systems at the AAA level vary significantly and sampling frame construction would benefit from a better understanding of the range of systems used to track providers. In addition, AoA’s current work designing program evaluations of Titles III-C and E (nutrition and family caregiver programs) has indicated that AAAs also vary in how they define their relationship with LSPs. As a result, a primarily qualitative study designed to better understand the variability in the relationship between AAAs and their LSPs; gain information on the various ways in which AAAs maintain information systems on services providers and the range of service provider characteristics is an important step to any further research on Older Americans Act Local Service Providers.
1. The primary purpose of the study is to better understand the complexity of the Local Service Provider network and the interactions with the Area Agencies on Aging to inform planning, policy development and implementation of the OAA reauthorization provisions. Service providers are a vital component of the Aging Services Network as this is the level that has direct and ongoing contact with older adult clients. Policies and programs at the federal level are ultimately implemented here and any legislative changes in the Older Americans Act (OAA) eventually have consequences at this local level. Information on the variability of AAAs’ provider networks and how AAAs track and monitor these relationships are valuable as AoA prepares for any changes that may stem for the reauthorization of the Older Americans Act, including any future development of OAA standards and regulations. Section C of the AAA pre-site visit questionnaire provides an overall picture of the number and type of providers by service category and Section D provides information on the variety of roles and responsibilities that AAA’s expect their service providers to engage in. The range of mechanisms (e.g. contracts, grants, MOUs…) for establishing the formal relationship between LSPs and AAAs along with the factors used to select providers and the general nature of the network (a stable, closed network or open and fluctuating) will be learned from Section E. The site visit and interview with the AAAs will expand upon the pre-site visit questions and gather richer information on the nature of the AAA’s service provider networks.
As the Older Americans Act goes through reauthorization AoA will benefit from information on the monitoring practices and data systems of AAAs and LSPs to form a picture of the ways in which AAAs and LSPs interact. Of particular use will be Section E of the AAA Interview Guide as it relates to tracking and monitoring of service providers, as well as, Sections C and D from the LSP Focus Group Guide which pertains to the linking of information systems between AAAs and LSPs and their working relationship (see Table A-1 for a list of each questionnaire/interview guide topic and its corresponding uses and purposes).
In addition to preparing for any changes that may result from re-authorization of the OAA, basic information on the range of organizational characteristics of LSPs will provide a broader perspective for both AoA program staff as well as policy, planning and evaluation staff members. Results of this study will provide a valuable resource to AoA staff as they design grant programs and conduct research and evaluations.
2. The study will provide information needed for the design of future representative studies that include Local Service Providers. The design of future Local Service Provider studies and program evaluations that include LSP data collection will be informed by this study in terms of both the development of data collection tools and sampling frame construction.
Sampling frames rely on accurate definitions of the population to be studied and a complete population list without duplications. AoA needs knowledge on how best to define service providers that makes sense to AAAs as well as how best to provide guidance to AAAs in constructing sampling frames of LSPs. Interview and focus group guides for this study cover topics such as the range of LSP organizational characteristics, LSP roles and responsibilities, and the mechanisms used by AAAs for establishing relationships with service providers. These topics will provide information on how best to operationalize definitions of service providers that are meaningful to AAAs, LSPs and AoA. In addition to improving the operationalization of LSP definitions, the process of LSP sampling frame construction will benefit. Section E of the AAA Interview Guide includes several open-ended questions with probes that will increase AoA’s knowledge of the range of Management Information Systems used by AAAs to track providers which will enable AoA to develop better methods for working with AAAs when constructing sampling frames of LSPs for probabilistic sampling.
Future research on LSPs will also benefit from improved data collection tools. This study will enhance question construction as well as answer category development. While this information will be most useful for studies that focus exclusively on service providers it will also be valuable for future OAA program evaluations, and in particular evaluations of OAA Title III-B services. AoA’s approach to program evaluation design includes a component that gathers program data from each level of the Aging Network (SUA, AAA and LSP). Supportive Services (Title III-B) include a range of service categories from transportation services to chore services to case management and hence a potentially complex array of service providers. This complexity poses methodological challenges for survey development. Survey tools will need to be meaningful to this range of service providers. This study will provide AoA with a better understanding of the potential contextual factors that influence service provision, the variability in organizational types, the nature of the relationship with AAAs from both AAA and LSP perspectives, and the many ways that AAAs and service providers divide up roles and responsibilities around service provision, thereby, leading to better data collection instruments.
Table A-1. Data Collection Topics by Purpose and Use
Data Collection Tool Section |
Purpose/Use |
||||||
LSP Organizational Characteristics |
Nature of AAA and LSP Relationship |
Roles and Responsibilities |
Contextual Factors |
Standards and Regulations |
Sampling Frame Definitions/ Construction |
Survey Question Development |
|
Pre-Site Visit Questionnaire |
|||||||
Section A. AAA and Service Area Characteristics |
|
X |
|
X |
|
|
|
Section B. Consumer-Directed Care |
|
X |
X |
X |
X |
|
X |
Section C. LSP Organizational Structure |
X |
|
|
|
|
X |
X |
Section D. LSP Contribution to Program Goals |
|
X |
X |
|
X |
X |
X |
Section E. Arrangements for Provision of Services |
X |
X |
|
X |
X |
X |
X |
AAA Interview Guide |
|||||||
Section A. AAA Approach to Service Provision |
|
X |
X |
X |
|
X |
X |
Section B. Contextual Factors Influencing Service Provision |
|
X |
|
X |
|
|
X |
Section C.AAA as Direct Service Provider |
|
X |
X |
X |
X |
X |
X |
Section D. AAA Relationship with LSPs |
X |
X |
X |
|
X |
X |
X |
Section E. Databases, MIS, Billing and Monitoring |
|
X |
|
|
X |
X |
|
Section F. Final Questions |
|
X |
X |
X |
|
|
X |
LSP Focus Group Guide |
|||||||
Section A. Provider Organizations |
X |
|
|
X |
|
X |
X |
Section B. Client Characteristics |
X |
|
|
|
|
X |
X |
Section C. Working with AAAs |
|
X |
X |
X |
X |
X |
X |
Section D. Client Tracking and Invoicing of AAA |
|
X |
|
|
X |
|
X |
Section E. Final Question |
|
X |
|
X |
|
|
X |
Westat, AoA’s contractor will communicate with the respondent AAAs via telephone and e-mail to introduce the study and schedule the data collection activities. In addition, an email pre-site visit questionnaire will be sent to AAA directors who participate in the study. They will respond to the questionnaire on a PDF fillable form and return it to Westat via e-mail.
This research does not duplicate any other data collection efforts on the part of AoA or any other federal agency. To ensure that prior efforts are not duplicated in the present study, we reviewed previous AoA data collection efforts as well as the published literature.
During the case study site visits Westat will interview staff of AAAs and service providers. Many of the AAAs are agencies within local governments or are nonprofit organizations. The providers include AAAs, nonprofit organizations, or businesses. It is anticipated that the interviews will be relatively brief and will have no effect on the operation of small businesses.
This is a one-time data collection and is the first opportunity for AoA to gather information about how AAAs and providers work together to achieve OAA program goals and to help maximize OAA funding.
The project will be conducted according to the guidelines specified in 5 CFR 1320.6. There are no special circumstances that would cause inconsistency with these guidelines.
The sixty-day notice for public commentary was published in the Federal Register on March 4, 2010. AoA did not receive any comments specific to this information request in response to the sixty-day notice.
The data collection instruments for the proposed information collection address the research questions developed by AoA.
Incentives will not be offered to participants in the study.
Anonymity is an important part of the study design. In response to this concern, Westat will ensure the anonymity of all individuals who provide data. A pledge of anonymity is a major positive incentive for potential respondents to participate in the study. Its absence would be a significant deterrent and could create complications in implementing the study. Westat will take the following precautions to ensure the anonymity of all data collected:
All Westat staff, including analysts, coders, editors, and keypunchers, will be instructed in the confidentiality requirements of the study and will sign statements affirming their obligation to maintain confidentiality;
Information will be reviewed and data will be cleaned only by Westat staff;
Data files that are delivered will contain no personal identifiers for program participants; and
Analysis and reports of study findings will be in terms of aggregated data only and sufficient sample sizes will preclude the identification of individual study participants.
Appendix A presents the confidentiality agreement all Westat staff members must sign. This agreement requires the signer to keep confidential any and all information about individual respondents to which they may gain access. Any Westat employee who violates this agreement is subject to dismissal and to possible civil and criminal penalties.
None of the data collection activities require the collection of any personal information about the respondents such as age, income, ethnicity, and educational attainment. The information asked for is about the mechanisms in which services are provided to OAA clients. For instance, the Pre-site Visit Questionnaire focuses on descriptive information about the AAA s (e.g., organizational structure, characteristics of clients served, services provided) as well as information about provider agencies. The focus of AAA Director Interview Script is on how AAAs and providers work together to achieve program goals. The focus group moderator’s guide probes for the providers’ perspectives of how they work with AAAs to provide services to their clients.
Table A-2 shows the estimates of the hour burden on respondents and the information for calculating the hour burden.
Data Collection Activity |
Number of Respondents |
Frequency of Response |
Annual Hour Burden |
Burden Per Instrument |
Interviews with SUA Directors |
10 |
1 |
.5 |
5 hours |
AAA Directors Pre-site Visit Questionnaire |
30
|
1 |
.5 |
15 hours |
On-site interviews during site visits: |
||||
Project Director |
30 |
1 |
1.5 |
45 hours |
Project Staff |
60 (2 per site x 30 sites) |
1 |
.75 |
45 hours |
Focus Groups: |
||||
Service Providers |
240 (1 group x 8 participants x 30 sites |
1 |
1 |
240 hours |
Total |
370 |
5 |
4.25 |
350 hours |
The total number of responses is 370.
These estimates are based on our previous experience collecting data for the Administration on Aging studies and other organizations. Examples of these efforts are below.
E-mail Surveys: Westat conducted an email survey of AAAs for AOA in 2006. The sample included all 655 AAAs. The instrument probed for information about clients served, available services, client tracking systems, and funding streams. Below is a reference to the survey.
Westat. (2007). 2006 National Survey of Area Agencies on Aging: Data and Codebook. Prepared under contact and submitted to the Administration on Aging, U. S. Department of Health and Human Services.
Focus Groups: Westat staff have conducted numerous focus groups with diverse target groups, including providers, program administrators, and seniors. The focus groups have been conducted in Westat’s focus group facility in Rockville, Maryland, as well as in many other locations within the U.S. and abroad.
In-person Interviews: The Westat staff members of the current project have conducted in-person interviews with diverse target groups, including state unit and area agency on aging staff members, service providers, and seniors. For the administration on aging, we conducted an evaluation of the Operation Restore Trust initiative that included site visits to AAAs and to provider agencies. The reference to the report of the study is below:
Rabinovich, B. A., and Ficke, R. C. (1999). Evaluation of operation restore trust initiative, report of best practices: Coalition for the Rights and Interests of the Elderly (Prepared under contract to the Administration on Aging, U.S. Department of Health and Human Services). Rockville, MD: Westat.
Table A-3 shows the annualized cost to respondents. We used the U.S. Department of Labor average hourly wage guidelines for determining hourly wage rates.
Table A-3: Annualized Cost to Respondents
Type of Respondent |
Total Burden Hours |
Hourly Wage Rate |
Total Respondent Costs |
SUA Directors |
5 |
$36.31 |
$181.51 |
AAA Directors – Pre-site visit Questionnaire |
15 |
$36.31 |
$547.65 |
Project Director |
45 |
$36.31 |
$1,633.95 |
Project Staff |
45 |
$20.93 |
$941.85 |
Service Providers |
240 |
$19.88 |
$4,771.20 |
Total annual cost burden of agency time and respondent time is zero.
The total estimated cost to the government for designing and conducting the study is $399,527.
Category |
Amount |
Personnel |
$314,096 |
Photocopying |
$4,140 |
Computing |
$3,687 |
Supplies |
$9,208 |
G&A |
$5,0435 |
Fee |
$25,788 |
Total Cost |
$399,527 |
This is a new data collection.
Plans for Tabulation – Qualitative Data
Site visitors will prepare a site visit report for each AAA in the sample according to a pre-defined report template. We anticipate that the site visitors will collect data at 3 AAAs within each state for a total of 30 AAA reports. A cross-site analysis identifying common themes and patterns within and across AAAs will be produced using NVivo 8 software for analyzing qualitative data. A summary of the process for analyzing the qualitative data for the site visits is below:
Review reports of each AAA for emerging themes by research question;
Create coding structures (containing individual nodes) based on a review of emerging themes;
Train staff to code site visit reports;
Conduct tests of inter-coder reliability;
Code data;
Refine coding structures as needed;
Create tabulations of codes;
Create matrices (cross tabulations) of codes by attributes (e.g., demographic information from pre-site visit interviews).
NVivo 8 has the capability of importing information on attributes by ID, and subsequently the researcher is able to create matrices of specific attributes by selected codes. This capability will also allow us to identify patterns of results across the different AAAs in the sample.
Plans for Tabulation – Quantitative Data
The majority of the data will be qualitative; however, we will compile information about the AAAs and their providers, such as:
Number of providers by AAA;
Client characteristics; and
Type of MIS system used to track client and provider data.
We will tabulate the quantitative data using statistical software for analyzing quantitative data, such as SAS. Descriptive statistics will be produced on the data elements listed above.
Plans for Publication
The final study report will be for internal AoA use only.
Project Schedule
Below is the project schedule.
Table A-5: Project schedule
Task |
Schedule |
Recruit States and AAAs |
1 month following OMB clearance |
Collect data |
2 months following OMB clearance |
Analyze data |
3 months following OMB clearance |
Prepare final report |
6 months following OMB clearance |
We do not request an exception to display of the Expiration Date for OMB Approval.
We do not request an exception to the certification statement, “Certification for Paperwork Reduction Act Submissions” (OMB Form 83).
Collection of InformationEmploying Statistical Methods |
B |
Sampling for case study site visits
States
To select the locations for the site visits, states were purposefully sampled to examine the range of issues addressed by the study design. To ensure representation from all AoA Service Regions, the states were first organized by region. Table B-1 contains a list of states by AoA Service Regions.1
AoA Service Regions |
||||||||
Region I |
Region II & III |
Region IV |
Region V |
Region VI |
Region VII |
Region VIII |
Region IX |
Region X |
CT |
DC |
AL |
IL |
AR |
IA |
CO |
AZ |
AK |
MA |
DE |
FL |
IN |
LA |
KS |
MT |
CA |
ID |
ME |
MD |
GA |
MI |
OK |
MO |
ND |
HI |
OR |
NH |
NJ |
KY |
MN |
NM |
NE |
SD |
NV |
WA |
RI |
NY |
MS |
OH |
TX |
|
UT |
|
|
VT |
PA |
NC |
WI |
|
|
WY |
|
|
|
VA |
SC |
|
|
|
|
|
|
|
WV |
TN |
|
|
|
|
|
|
Using population data from the Census Bureau 2008 Population Estimates, states were ranked by the percentage of 60+ populations. For each AoA Service Region, the state with the highest percentage of 60+ persons was sampled. The exception to this is Regions II and III, where two states were selected. Regions II and III are technically two regions, but work together from an operational standpoint. Table B-2 contains the states by AoA Service Regions by percent of persons age 60 and older.
Region I |
Region II and III |
Region IV |
Region V |
Region VI |
||||||
State |
% of 60+ |
State |
% 60+ |
State |
% 60+ |
State |
% 60+ |
State |
% 60+ |
|
ME |
21.2% |
WV |
21.7% |
FL |
22.9% |
OH |
18.8% |
AR |
19.7% |
|
VT |
20% |
PA |
20.7% |
AL |
19.1% |
WI |
18.3% |
OK |
18.6% |
|
RI |
19.2% |
DE |
19.4% |
SC |
19.0% |
MI |
18.2% |
NM |
18.1% |
|
CT |
19.0% |
NY |
18.5% |
KY |
18.6% |
IN |
17.7% |
LA |
17.2% |
|
MA |
18.6% |
NJ |
18.3% |
TN |
18.6% |
MN |
17.3% |
TX |
14.5% |
|
NH |
18.5% |
VA |
17.2% |
MS |
17.6% |
IL |
16.9% |
|
|
|
|
|
MD |
17.1% |
NC |
17.6% |
|
|
|
|
|
|
|
DC |
16.7% |
GA |
14.8% |
|
|
|
|
|
|
Region VII |
Region VIII |
Region IX |
Region X |
|
|||||
|
State |
% 60+ |
State |
% 60+ |
State |
% 60+ |
State |
% 60+ |
|
|
|
IA |
19.9% |
MT |
19.9% |
HI |
20.1% |
OR |
18.9% |
|
|
|
MO |
18.8% |
ND |
19.6% |
AZ |
18.1% |
WA |
17.2% |
|
|
|
NE |
18.2% |
SD |
19.5% |
NV |
16.6% |
ID |
16.9% |
|
|
|
KS |
17.9% |
WY |
17.6% |
CA |
15.6% |
OR |
18.9% |
|
|
|
|
|
CO |
15.2% |
|
|
AK |
11.7% |
|
|
|
|
|
UT |
12.6% |
|
|
|
|
|
|
*Source: Population data is from Census Bureau 2008 Population Estimates.
Using this approach, we selected the following states for the site visit:
Region I: Maine
Region II & III: West Virginia and Pennsylvania
Region IV: Florida
Region V: Ohio
Region VI: Arkansas
Region VII: Iowa
Region VIII: Montana
Region IX: Arizona2
Region X: Oregon
Approach for selecting AAAs within sampled states
It is anticipated that that the Westat researchers will conduct site visits to three AAAs in each state, and we will employ a maximum variation approach to sampling the AAAs for the site visits. In this approach, diverse characteristics are identified and used to select the sample cases. We will ensure that selected AAAs vary by structure of provider agencies, geography, and client characteristics. This approach will provide the most complete description of the diversity of service delivery systems that exist across the network. It will also facilitate identifying patterns of common characteristics that exist among provider networks across the state, regardless of the service or the structure of the provider agencies. It will also facilitate the understanding of the differences in service delivery systems that exist across the network, and the factors that contribute to these differences.
The researchers will conduct brief informal discussions with each SUA director to obtain information about the AAAs in her state. We will ask the SUA directors to confirm the total number of providers in the state, the number of providers serving minorities, and the number of AAAs located in rural areas. The states report this information in aggregate in the annual State Program Report. The researchers will ask for this information to be broken down by AAA.
Whenever possible, we will collect information from other sources to obtain information for sampling AAAs. The sources may include the U.S. Census, Annual Plans on the AAA websites, and the State Program Report. The information collected will be organized systematically to ensure that a range of factors are considered when selecting the AAAs for the site visits.
State Units on Aging
Letter of introduction to selected SUAs to introduce the Provider Study
AoA will send a letter of introduction to the state unit directors in the selected states. The letter will include the goals of the project and a brief description of the methodology. In addition, AoA will inform the SUA director that a researcher from Westat will contact her to schedule a telephone call to obtain preliminary information for selecting AAAs for the site visits, documents that will facilitate sampling, and information about state-wide provider networks.
Select AAAs for site visits
We will conduct site visits to three AAAs in each of the sampled states. As discussed above, we will use a maximum variation approach to sampling the AAAs.
Area Agencies on Aging
Initial contact with selected area agencies on aging.
AoA will send a letter to the directors of the selected AAAs introducing the study and informing the directors that their AAA has been chosen for a site visit. The letter will include a brief overview of the study and a summary of AAA involvement in the site visits. In addition, the letter will explain that the AAA involvement will include designation of a site visit liaison that will collaborate with Westat staff on the following activities:
Identify AAA staff for interviews;
Identify provider agency staff to participate in focus groups;
Identify other key informants to contact for in-person or telephone interviews;
Select dates for site visit;
Review site visit schedule;
Pre-site visit data collection
The pre-site visit data collection includes document review (e.g., documents pertaining to providers, such as memorandums of understanding, contracts, guidelines, etc.) and a self-administered AAA Directors Pre-Site Visit Questionnaire (see Appendix B). The final version of the questionnaire will be a fillable PDF document, which the directors will receive via email. They will be asked to complete it at least one week prior to the site visit. The pre-site visit questionnaire will capture contextual information about the AAA, information about the number and type of providers with which the AAA collaborates and other general information. The data gleaned from the pre-site visit questionnaires will serve several purposes: (1) reduce the in-person burden on the AAA directors, (2) provide a contextual basis for the discussions during the site visits, and (3) provide consistent information about the number of providers by service.
Site visits to area agencies on aging
One site visitor will travel to each state. The activities during the site visit will include, but will not be limited to, the following:
Meet with AAA Site Liaison
The site visitor will meet with the site visit liaison for an orientation to the AAA. The liaison will introduce the site visitor to the staff and review the list of documents previously obtained by the site visitor to ensure that the list is complete.
Interview AAA Director
The site visitor will conduct an interview with each AAA director that focuses on: the characteristics of the AAA’s providers, the decision-making process for determining the level of services provided, the distribution of roles and responsibilities for achieving program goals among the AAAs and service providers, and how the AAA and its providers partner in serving older clients. (see Appendix C for interview script).
Interviews with AAA staff members
We anticipate that, along with the AAA Director, other AAA staff members may respond to selected portions of the interview, as relevant. This will depend on the specific staffing configuration of the AAA. For instance, to ensure that certain topics are fully addressed, MIS staff, budget staff, and program staff may need to respond to specific items in the interview. The AAA director in consultation with the site visitors will determine how this process will occur, whether through a group interview or brief interviews with individual staff members.
Focus Groups with Providers
The site visitor will conduct in-person focus groups with staff members of provider agencies. The provider focus group moderator guide includes topics, such as the structure of the provider agency, description of clients served and how client characteristics influence the delivery of services, how providers collaborate with the AAA on achieving program goals, their level of identification with the network, and methods for tracking service delivery and clients. (See Appendix D for the focus group moderator’s guide.)
Site Visits. Several methods will be used to maximize response rates during the site visits. AoA staff will inform the directors of the selected AAAs in advance about the study, its importance and purpose, as well as the extent of the AAA’s involvement. AoA staff will also conduct an orientation for the site visitors prior to travel. This will help facilitate cooperation and address any concerns.
The site visits will be scheduled in coordination with the AAA directors in the site visit locations. The visits will be scheduled at a time that the AAA directors deem most convenient for the majority of staff and service providers. Prior to focus groups and interviews, participants will be provided with background information about the study (its goals and purpose), that participation in the study is voluntary, that the information collected will be reported in the aggregate, and statements made will not be attributed to any individual in reports of the study.
Westat tested the proposed methods using several strategies. First, AoA senior staff members reviewed the data collection instruments to ensure that they accurately captured the purpose of the study. AoA provided feedback and Westat incorporated the feedback into the instruments. Second, Westat staff pre-tested the survey instruments with staff members of an area agency not included in the sampled states to ensure that they accurately captured the intended information. An area agency on aging director completed the pre-site visit questionnaire and Westat completed on-site interviews with the director and additional staff using the interview script. The area agency on aging director and additional staff provided feedback on the instruments. Westat improved the pre-site visit questionnaire and interview script by incorporating the feedback into the instruments (e.g., improved clarity of questions in both instruments and converted pre-site visit questionnaire into a fill able PDF document).
AoA Personnel Responsible for the Deliverables
Ms. Jennifer Klocinski
Administration on Aging
HHS/AoA/CPPE/OPE
Aging Services Program Specialist
1 Massachusetts Ave.
Room 5402
Washington, D.C.
202-357-0146
Contractor’s Personnel
Dr. Beth A. Rabinovich -Role: Study design, data collection, data analysis
Senior Evaluator
Westat
1600 Research Blvd.
Rockville, Maryland 20850
301-315-5965
Dr. Susan Berkowitz - Role: Study design, data collection, data analysis
Senior Evaluator
Westat
1600 Research Blvd.
Rockville, Maryland 20850
301-294-3936
Dr. Dwight Brock – Role: Statistical Analysis
Senior Statistician
Westat
1600 Research Blvd.
Rockville, MD 20850
301-251-4398
Ms. Jessica Harrell – Role: Study design, data collection, data analysis
Analyst
Westat
1650 Research Blvd.
Rockville, Maryland 20850
301-610-4973
Appendix A:
Confidentiality Statement
WESTAT, INC.
EMPLOYEE OR CONTRACTOR'S ASSURANCE OF CONFIDENTIALITY OF SURVEY DATA
Statement of Policy
Westat is firmly committed to the principle that the confidentiality of individual data obtained through Westat surveys must be protected. This principle holds whether or not any specific guarantee of confidentiality was given at time of interview (or self-response), or whether or not there are specific contractual obligations to the client. When guarantees have been given or contractual obligations regarding confidentiality have been entered into, they may impose additional requirements which are to be adhered to strictly.
Procedures for Maintaining Confidentiality
1. All Westat employees and field workers shall sign this assurance of confidentiality. This assurance may be superseded by another assurance for a particular project.
2. Field workers shall keep completely confidential the names of respondents, all information or opinions collected in the course of interviews, and any information about respondents learned incidentally during field work. Field workers shall exercise reasonable caution to prevent access by others to survey data in their possession.
3. Unless specifically instructed otherwise for a particular project, an employee or field worker, upon encountering a respondent or information pertaining to a respondent that s/he knows personally, shall immediately terminate the activity and contact her/his supervisor for instructions.
4. Survey data containing personal identifiers in Westat offices shall be kept in a locked container or a locked room when not being used each working day in routine survey activities. Reasonable caution shall be exercised in limiting access to survey data to only those persons who are working on the specific project and who have been instructed in the applicable confidentiality requirements for that project.
Where survey data have been determined to be particularly sensitive by the Corporate Officer in charge of the project or the President of Westat, such survey data shall be kept in locked containers or in a locked room except when actually being used and attended by a staff member who has signed this pledge.
5. Ordinarily, serial numbers shall be assigned to respondents prior to creating a machine-processible record and identifiers such as name, address, and Social Security number shall not, ordinarily, be a part of the machine record. When identifiers are part of the machine data record, Westat's Manager of Data Processing shall be responsible for determining adequate confidentiality measures in consultation with the project director. When a separate file is set up containing identifiers or linkage information which could be used to identify data records, this separate file shall be kept locked up when not actually being used each day in routine survey activities.
6. When records with identifiers are to be transmitted to another party, such as for keypunching or key taping, the other party shall be informed of these procedures and shall sign an Assurance of Confidentiality form.
7. Each project director shall be responsible for ensuring that all personnel and contractors involved in handling survey data on a project are instructed in these procedures throughout the period of survey performance. When there are specific contractual obligations to the client regarding confidentiality, the project director shall develop additional procedures to comply with these obligations and shall instruct field staff, clerical staff, consultants, and any other persons who work on the project in these additional procedures. At the end of the period of survey performance, the project director shall arrange for proper storage or disposition of survey data including any particular contractual requirements for storage or disposition. When required to turn over survey data to our clients, we must provide proper safeguards to ensure confidentiality up to the time of delivery.
8. Project directors shall ensure that survey practices adhere to the provisions of the U.S. Privacy Act of 1974 with regard to surveys of individuals for the Federal Government. Project directors must ensure that procedures are established in each survey to inform each respondent of the authority for the survey, the purpose and use of the survey, the voluntary nature of the survey (where applicable) and the effects on the respondents, if any, of not responding.
PLEDGE
I hereby certify that I have carefully read and will cooperate fully with the above procedures. I will keep completely confidential all information arising from surveys concerning individual respondents to which I gain access. I will not discuss, disclose, disseminate, or provide access to survey data and identifiers except as authorized by Westat. In addition, I will comply with any additional procedures established by Westat for a particular contract. I will devote my best efforts to ensure that there is compliance with the required procedures by personnel whom I supervise. I understand that violation of this pledge is sufficient grounds for disciplinary action, including dismissal. I also understand that violation of the privacy rights of individuals through such unauthorized discussion, disclosure, dissemination, or access may make me subject to criminal or civil penalties. I give my personal pledge that I shall abide by this assurance of confidentiality.
Signature Print Name Date
Appendix B:
Pre-site Visit Questionnaire
As you know, Westat is conducting a study of provider networks in 10 states for the Administration on Aging. [INSERT STATE] is one of the states and we are conducting site visits to your AAA and two others in your state. The selection of AAAs for site visits is based on the objective of obtaining information on a broad spectrum of service delivery models. The visit will include an interview with [INSERT AAA director name] and other AAA staff may contribute to the interview. We will also be talking to selected providers about their roles and responsibilities.
To prepare for the site visit, we would like to learn more about your AAA and the clients you serve. We would appreciate your completing the attached questionnaire by [INSERT DATE] and returning it to us by email to [email protected]. The responses that you provide will supply some useful background information about your AAA and serve as a jumping off point for the on-site interviews. If you should have questions about the study, please don’t hesitate to call Jennifer Klocinski at AoA 202-357-0146. For questions about completing the pre-site visit questionnaire, please contact Jessica Harrell at Westat 800-937-8281 x4973.
Please complete the following information:
Respondent’s Name:
Role in the AAA:
Date:
1. Name of Agency
2. What type of agency is the AAA? Check one response.
Please specify
3. What type of geographic area does your AAA cover? Check “yes” or “no” for each response option.
Yes No
4. What is the ethnic/racial composition of the 60+ population in your Planning and Service Area?
American Indian or Alaskan Native %
Native Hawaiian/Other Pacific Islander %
Please specify
What percentage of the 60+ population is Hispanic or Latino in your Planning and Service Area ?
Latino/Hispanic %
6. What groups does your AAA serve? Check “yes” or “no” for each response option.
Yes No
Persons 60+ only
Persons younger than 60 with physical and/or developmental disabilities
Other
Please specify
7. How many clients (OAA and others) does your AAA serve?
total number of clients
8. How many OAA clients does your AAA serve?
number of OAA clients
9. What is your annual budget (use most recently completed fiscal year)?
Annual budget
10. What is your annual budget for OAA services (use most recently completed fiscal year)?
Annual budget for OAA services
11. What is the total number of full-time equivalent employees?
Total FTEs
This section is on consumer-directed care. Consumer-directed care is predicated on the belief that consumers should have a choice in who provides care for them and what services they need. There are several models of consumer-directed care. In one model, the care recipient and/or her caregiver is able to choose, hire, supervise, and pay for services. The consumer receives a specific amount of money (usually) on a monthly basis to cover the fees for such care. In the other model, the consumer works closely with a case manager in determining the type of care needed and the specific provider that will deliver the service.
1. Does your AAA have a consumer-directed care option for older adult services?
Yes
No [GO TO END]
1a. If yes, what year did the consumer-directed care begin?
Year
The purpose of this section is to obtain some basic information about your providers. During the site visit, we will talk in more detail about the different types of providers that serve your clients.
1. Below, please enter the total number of providers per service by type of providers as of November 1, 2010. If a provider delivers more than one service, count it for each service provided. If the service can be included in a consumer directed option, please note it in the appropriate box. If the AAA provides the service, please note it in the appropriate box.
Service – Outreach: Information & Referral; Case Management
Stand-alone agency
Agency that is part of an umbrella organization
An individual who provides a service (e.g., an attorney)
Other, please specify ( )
Total
AAA Provides the Service Yes No
Service – Nutrition: Congregate meals; Home delivered meals
Stand-alone agency
Agency that is part of an umbrella organization
An individual who provides a service (e.g., an attorney)
Other, please specify ( )
Total
Cash and Counseling/Consumer Directed option offered Yes No
AAA Provides the Service Yes No
Service – Care: Homemaker; Chore; Personal Care; Adult Day Care; Respite care for caregivers
Stand-alone agency
Agency that is part of an umbrella organization
An individual who provides a service (e.g., an attorney)
Other, please specify ( )
Total
Cash and Counseling/Consumer Directed option offered Yes No
AAA Provides the Service Yes No
Service – Transportation: Transportation; Assisted Transportation
Stand-alone agency
Agency that is part of an umbrella organization
An individual who provides a service (e.g., an attorney)
Other, please specify ( )
Total
Cash and Counseling/Consumer Directed option offered Yes No
AAA Provides the Service Yes No
Service – Legal Assistance
Stand-alone agency
Agency that is part of an umbrella organization
An individual who provides a service (e.g., an attorney)
Other, please specify ( )
Total
AAA Provides the Service Yes No
2. How many unduplicated providers of OAA client services does the AAA have a formal relationship regardless of the services offered?
Number of unduplicated OAA providers
3. What is the number of service providers that also act as Medicaid providers?
Number of Medicaid providers
4. What is the number of service providers that also act as Medicare providers?
Number of Medicare providers
1. We are interested in learning about how providers may work alongside of the AAA to achieve program goals through fundraising, outreach, targeting, and program development. In the table below, for each type of service listed, please check the areas or activities in which providers contribute. During the site visit interview, we will talk in greater detail about provider activities in these areas.
Service – Outreach: Information & Referral; Case Management
Fundraising
Outreach
Targeting
Program Development
Match
Other ways in which providers contribute
Please specify
Service – Nutrition: Congregate meals; Home delivered meals
Fundraising
Outreach
Targeting
Program Development
Match
Other ways in which providers contribute
Please specify
Service – Care: Homemaker; Chore; Personal Care; Adult Day Care; Respite care for caregivers
Fundraising
Outreach
Targeting
Program Development
Match
Other ways in which providers contribute
Please specify
Service – Transportation: Transportation; Assisted Transportation
Fundraising
Outreach
Targeting
Program Development
Match
Other ways in which providers contribute
Please specify
Service – Legal Assistance
Fundraising
Outreach
Targeting
Program Development
Match
Other ways in which providers contribute
Please specify
This section is on types of mechanisms you use to work with service providers. We are interested in whether providers compete with one another to provide the same sets of services and the typical length of contracts, cooperative agreements and other mechanisms for working with providers.
1. What mechanisms does the AAA use to work with provider agencies?
Yes No
2. What factors are considered in selecting providers? Please check “yes” or “no” for each response option.
Yes No
Please specify
3. What is the length of the typical arrangement for service provision?
1 year or less
2 years
3 years
More than 3 years
4. Overall, on a scale of one-to-five, how stable has your service provider network been over the past 2 years?
1 – Not stable
2
3
4
5– Extremely stable
THANK YOU VERY MUCH FOR RESPONDING TO THE PRE-SITE VISIT QUESTIONNAIRE. WE LOOK FORWARD TO MEETING YOU DURING THE SITE VISIT TO YOUR AAA.
PLEASE
EMAIL YOUR COMPLETED QUESTIONNAIRE TO JESSICA HARREL
AT
[email protected].
Appendix C:
AAA Director Interview Script
As you know, Westat is conducting a study of the provider network in your state. We would like to interview you about how services are provided in your area, the characteristics of the providers that deliver services to your clients, and the broader community context in which services are delivered.
The interview, which should last 45-60 minutes, will be tape recorded to ensure accuracy of the information. Please be assured that no one will have access to these data except Westat researchers who have taken a pledge of confidentiality. Furthermore, results will be reported in aggregated form and in a way that will not enable statements to be linked to identifiable individuals.
Are there any other questions before we begin? I am now going to turn on the tape recorder, Okay?
Thank you for completing the pre-site visit questionnaire. The first few questions are about your general approach to service delivery and wherever possible, I want to use your responses from the pre-site visit questionnaire as a starting point for our discussion.
Can you describe your AAA’s overall philosophy of providing services? What would you say drives that philosophy? (Probe on key features and drivers as needed, e.g. state and local policies, needs assessments.) Do you have a written mission statement—if so, can you say a little about it and when, how and by whom it was developed?
Now I’d like to ask specifically about how you would characterize your AAA’s approach to working with service providers. (First let respondent articulate, then probe as needed for clarification/elaboration.) In particular, I’d like to know more about how you view the roles and responsibilities of the AAA in relation to service providers as well the ways in which you may work together to achieve common goals.
Next I’d like to turn to your approach to delivery of services. How would you describe your overall approach to service delivery? (Only ask if not covered in Q1.) OR
Based on what you said earlier, this seems to be your approach to service delivery (briefly summarize). Did I get the gist of what you said? (Ask respondent to elaborate, clarify, with respect to intensity and mix of services.)
[If not already addressed] What role, if any, does consumer-directed care play in your overall approach? What factors influence the development of a service package for any given individual and how does case management typically fit into the picture?
[If not already addressed] What are your thoughts about short-term versus long-term provision of services?
Has any part of your overall approach to service provision (as discussed above) changed in an important way in the last 5 years? If so, please say more about the nature of the change and the factors contributing to any such change(s).
In the pre-site visit questionnaire, you noted that the AAA is situated in an (urban, rural, suburban, or a combination of areas). Could you say more about this, especially about recent changes or other geographic or demographic characteristics that in your view are distinctive to your service area?
Overall, how would you say these demographic and geographic characteristics impact the provision of services? How has this changed, if at all, in the past 5 years?
Apart from these geographic and demographic characteristics, are there any other factors that influence how the AAA provides services? (e.g., municipal or county governmental structure, political culture.) If so, please explain.
In the pre-site visit questionnaire, you noted that the AAA provides [NAME OF SERVICES] to your clients. We want to know more about the factors that contributed to the AAA becoming a provider of these services. (If not explicitly noted, probe on political situation, lack of other available services.)
For the services that the AAA directly provides, please briefly describe: a) the service, b) the clientele, c) the location where the service is provided, and d) any in-kind resources that are leveraged (e.g., free rent on space for service, materials, staff time, etc.) to help provide the service.
We are also interested in learning about AAAs that provide services to other organizations. What services, if any, does the AAA provide to other organizations and how did this arrangement evolve?
In this section I’ll pose a series of questions about your AAA’s relationships with the organizations and individuals that provide services to your clients.
Now I’d like to ask about the type(s) of mechanisms – including contracts, cooperative agreements, or other arrangements–you employ in working with your service providers. (Probe on variations; which most used/why; how different mechanisms developed over time.) Can you show me an example of the vendor package you normally use? Overall, how well would you say these mechanisms work?
Which major funding sources are used to pay providers? How do you determine how resources are allocated? (Probe on any inconsistencies/anomalies in billing—e.g., different rates for same service from different providers.)
Federal funds (OAA,USDA, DOT, Medicare, Medicaid, Medicaid waiver)
State funds
Local government funds
Foundations (specify)
United Way
In-kind contributions
Participant contributions
Other (specify)
Your response to the pre-site visit questionnaire indicates that you work with [INSERT NUMBER PRIOR TO THE SITE VISIT] providers of various types. Please choose a few examples of major types of provider organizations and tell me about their organizational structure:
An individual service provider (e.g., 1 person)
A stand-alone provider agency
An umbrella organization that provides a service
Some other type of organization (specify)
Earlier we talked a bit about the AAA’s overall approach to working with service providers. Now, can you elaborate more specifically on how providers contribute to achieving AAA program goals? [USE INFORMATION FROM ANSWERS TO PRIOR QUESTIONS AND PRE SITE-VISIT QUESTIONNAIRE AS APPLICABLE] For providers that you described in the previous question, please give an interesting example of how that provider is contributing/has contributed to achieving AAA program goals in one or more of the following areas. If it makes sense to do so, please build on the examples you just gave or choose additional providers to describe how they work with the AAA (or independently in achieving program goals):
Fundraising
Outreach
Targeting
Program development
(If
not addressed, probe on factors contributing to each provider’s
role, degree of independence/
collaboration, how
responsibilities are shared.)
Your pre-site visit questionnaire indicates that your AAA does offer a consumer-directed care option. [DRAW ON PRIOR ANSWERS AND SITE-VISIT QUESTIONNAIRE AS APPLICABLE] Can you tell me more about how this works and what contributed to the initiation of this option for your clients? What have been the facilitators as well as the barriers to its implementation? How has having this option affected your relationships with service providers?
Earlier you noted that [FILL IN ANSWERS FROM ABOVE] are the most common mechanisms you use for working with service providers. As the final question in this section, I’d like to ask about the continuity and duration of the AAA’s arrangements with your service providers. (First let respondent speak to this subject. Then, if needed, probe on: average length of arrangements, newest and oldest, AAA’s role.)
What are the main reasons why you need to change service providers? How do you go about doing so? If you’ve had to terminate a relationship with a provider in the past 3 years, please say more about the circumstances surrounding this.
I’d like to ask about how you track provider and client service data.
How do you maintain information about your service providers? (Probe on format – electronic, paper records.) Is there a central location where the information is stored? If the information is not in a central location, is it decentralized? (Probe on how it is decentralized.)
What information about providers is collected and maintained? (Probe: contact information, point of contact, type of contract or grant, service provided, staff certifications, assurances, geographic coverage, monitoring data…) Who maintains the information on providers? (Probe: contracts office, program staff, …) How frequently is it updated?
Can you briefly describe how you track information about providers in terms of the delivery of services? (Electronic versus paper records, software packages (COTS or commercial), where data reside.) Which data elements are tracked for each provider, each client? (Probe on cost/billing data) How was it decided which specific items would be tracked? How interlinked are your MIS systems with those of the providers? (Probe on how communicate with providers about service requests, payments, tracking.)
(If not addressed in the response to question 1, ask the following question.)
Briefly describe the payment system that the AAA uses for providers (fee-for-service/rate, capitated, unit-based, etc.). To what extent does the payment system and/or amount vary by funder? What is the frequency for billing? (Weekly, monthly, per occurrence of service.) If consumer direction is offered, how is this tracked and paid?
Briefly describe the system used to monitor service providers and evaluate their performance (Probe: client assessments of service quality, complaint mechanisms, site visits, document reviews, performance targets…). How do you determine that clients receive the appropriate service and unit of service?
How does your agency contact providers and clients during a manmade or natural emergency, such as a snowstorm, hurricane, or earthquake that may make it difficult for providers to deliver services or for clients to access services? (Probe: If not mentioned in the answer, is there a formal plan for emergency communications?) Has your agency had an emergency in which the plan was used? If, yes how did it work? Were any changes made based this experience?
This is the last section of the interview. I have two questions for you.
If there is one thing you could change about your relationship with the service providers in your area, what would you change?
Is there anything else that you would like to tell me about how you work with providers?
Thank you so much for taking your time to speak with us today. Is there anything else you’d like to add to help us better understand your provision of services?
Thanks, again, for all your help.
Appendix D:
Provider Focus Group Moderator’s Guide
My name is [NAME OF MODERATOR] and I work for Westat, a research firm, located in Rockville, Maryland. Westat is conducting a study of the provider network under the auspices of the U.S. Administration on Aging (AoA). AoA is interested in learning about the characteristics of provider agencies and how providers work with AAAs on activities, such as targeting, planning programs, and outreach. For the study, we interviewed AAA staff and now we are interested in finding out about your organizations.
For those of you who have not previously participated in a focus group, here is a brief description of what will happen. For the next hour, I will pose questions to the group and everyone is encouraged to participate in the discussion. I am interested in what everyone has to say and there are no right or wrong answers.
My colleague [NAME OF NOTETAKER] will take notes during the focus group. She will also audiotape the discussion to ensure that we accurately capture the content of the discussion. Everything you say during the focus group will be kept confidential. Reports of the focus groups will not contain any names and the results will be reported in the aggregate.
Do you have any questions before we begin? I am going to turn on the tape recorder now. Please nod to acknowledge that you know the recorder has been turned on.
First, let’s go around the table and introduce ourselves. Please tell us your name, your organization, and the service (s) that you provide to the AAA’s clients.
First, I’d like to know a bit more about each of your organizations, especially its basic mission and when and how you became a provider for the AAA. (Probe on non-profit/profit, government, stand-alone versus umbrella organization if not stated.) Do you provide services to other organizations (besides the AAA)? If so, please tell us about the other organizations/services you provide and their respective clienteles.
Apart from your relationship with the AAA, are you involved with any other organizations in the Aging Network–at the local, state or national levels? (Probe on which, nature of tie, whether formal or informal.) Do you belong to any professional provider organizations? (Probe on which, whether hold offices, how participate.)(If relevant.) How long have you been part of these organizations?
Can you briefly describe the characteristics of the AAA clients that you serve? (First let respondent articulate; then probe on characteristics, including Medicaid, minority/underserved, persons with disabilities.) How has the makeup of your clientele influenced how you provide services? (If not raised, ask about cuisine, staff, and distance.) If this has changed in the last 5 years, please say a bit about how and why.
Now I’d like to ask you to talk about your work with the AAA. To start with, overall, how would you describe your working relationship with the AAA??
Can you tell us about how you work together in the following types of activities:
Targeting clients
Conducting outreach
Planning programs/activities
Carrying out programs/activities
Are there any other ways/areas in which your organizations work with the AAA? (For each type of activity, as relevant, probe for concrete examples and greater depth.) How has the working relationship with the AAA changed/developed over time??
(Consumer
Directed Care – Only for areas that have a consumer directed
care option.)
We
understand that the AAA has a consumer-directed care option. What
does this mean in operational terms? How, if at all, does having
this option affect your working relationship with the AAA?
What types of systems do your organizations use to track information about clients and delivery of services? (paper versus electronic, software, etc.) How does your system link with the AAA’s system? How does this work for billing purposes? What sorts of issues come up in making the two systems work together?
Thank you all for sharing so much information with us. In closing, I’d like to ask each of you a “what if” question. If you could change one thing about your organization’s relationship with the AAA, what would it be—and why?
THAT
IS ALL THE QUESTIONS WE HAVE. THANK YOU VERY MUCH FOR
PARTICIPATING
IN THE FOCUS GROUP.
1 Per AoA direction, U.S. territories are not included in the sample design.
2 Because of cost issues, Arizona replaces Hawaii.
| File Type | application/msword |
| File Title | 2010 Study of the Older Americans Act Provider Network: PRA Request |
| File Modified | 2010-07-14 |
| File Created | 2010-07-12 |