Key Informant Interviews

BOND Implementation and Evaluation

Contract No. SS00-10-60011

Resource Bank: Study Topic Areas and Questions

Abt Associates Inc.

Appendix F. Resource Bank for Developing Key Informant Interview Guides
and Focus Group Moderator’s Interview Guide

F-11

BOND Implementation and Evaluation

I.

Contract No. SS00-10-60011

SITE BACKGROUND 1

Description of geographic region/service area. What is the size of the BOND service area? Does it
include multiple states? Is it mostly urban, suburban, rural, or mixed? Are most beneficiaries concentrated
within one or two communities or are they scattered across the region?
Demographic characteristics. What are the demographic characteristics of those living in the service
area? How diverse is the community? What are the socioeconomic conditions of individuals living the
service area? What is the high school completion rate?
Characteristics of people with disabilities. What are the characteristics of people with disabilities living
in the service area? What types of disabilities are most common? How does the size of the SSDI caseload
compare to the SSI caseload? How do the demographic characteristics of those receiving SSDI compare
to the general population in the area?
Description of local labor market in the designated service area, in general and for those with
disabilities. What are the current labor market conditions in the service area? What affect has the
economic recession had on the availability and types of jobs? What employment opportunities are
available to clients with disabilities? How adequate are these opportunities? Do clients have the skills and
experience to fill available jobs? To what extent are individuals with disabilities working in the service
area?
Barriers to employment for SSDI beneficiaries. What are the most significant barriers to work facing
disabled beneficiaries in this area? How do local transportation systems support or impede employment
for people with disabilities?
Additional Relevant Data:
Proportion of SSDI-only and current beneficiaries who work.
Proportion of beneficiaries who left the SSDI rolls through employment last year.
Employment and earnings patterns for SSDI beneficiaries.
Waiting lists for SVRA participants.

1

Most of this information will be gathered prior to the site visit. Select information that may change over time,
such as unemployment rate, will be updated for each site visit.

Abt Associates Inc.

Appendix F. Resource Bank for Developing Key Informant Interview Guides
and Focus Group Moderator’s Interview Guide

F-12

BOND Implementation and Evaluation

II.

Contract No. SS00-10-60011

CURRENT SERVICES AVAILABLE TO SSDI BENEFICIARIES

Existing services available to SSDI beneficiaries through the Work Incentives Planning and Assistance
(WIPA) providers. What programs and services are currently available to SSDI beneficiaries through the
local WIPA providers? What services are provided beyond the standard WIPA services? What services
are above and beyond the standard WIPA services? Who has access to these services (e.g., beneficiaries
who are working, those in a certain geographic region)? How successful has the WIPA been in
encouraging beneficiaries to work?
Accessibility of WIPA services. I’d like to ask some questions about the accessibility of services. How far
to participants have to travel to access WIPA services? Are these services located near public
transportation such as a bus route or subway? How long do beneficiaries typically have to wait for these
services? How many requests do the Community Work Incentive Coordinators (CWICs) handle in an
average week? Are these requests handled primarily by phone, in-person meetings, or e-mail? How
accessible are the CWICs? Do they carry ongoing caseloads? If so, how many ongoing cases do they
serve? What is the average length of time a participant waits for the CWIC to respond to their request?
Other initiatives and services for SSDI beneficiaries. What work-based initiatives and services are
available to workers with disabilities in general and to SSDI beneficiaries, in particular? Which agencies
administer these programs? What types of other services are available, such as health insurance
programs? How accessible are these services? Is the Medicaid Buy-In program available to beneficiaries
within the service area? What is the general attitude (e.g., “Culture of Disability”) around serving
individuals with disabilities?
Success/Challenges of employment support initiatives. What successes have you had with employment
support initiatives? What challenges do you experience in implementing them? How do you address those
issues? Does the SSDI population have any special characteristics or present any particular problems for
employment support initiatives?

Abt Associates Inc.

Appendix F. Resource Bank for Developing Key Informant Interview Guides
and Focus Group Moderator’s Interview Guide

F-13

BOND Implementation and Evaluation

III.

Contract No. SS00-10-60011

AGENCIES INVOLVED WITH DEMONSTRATION PLANNING AND INITIAL
IMPLEMENTATION

Area Work Incentives Counselors (AWIC). What role has the AWIC played in the planning and initial
implementation of the BOND demonstration? What ongoing role does it play in return to work issues?
How actively involved is it in the local community?
Social Security Administration (SSA) field office functions. How many SSA field offices are in the
service area? Which offices serve the most participants? Which will likely have the most demonstration
activity? What role have they played in the demonstration planning and initial implementation? What is
their ongoing role in beneficiary employment issues? In the field offices, who is assigned to help with the
demonstration? Who is responsible for helping beneficiaries return to work that are not involved with the
demonstration?
Work Incentive Liaisons (WILs). What role have the WILs played in the demonstration planning and
initial implementation? What role do they play with employment services for those not involved with the
demonstration?
Work Incentives Planning and Assistance (WIPA) providers. Briefly describe the WIPA providers.
What role have they played in the demonstration planning and initial implementation? How many people
have they served so far as part of the demonstration (if applicable)? What services were provided? How
many SSDI beneficiaries have they served that are not involved with the demonstration? How successful
have they been in encouraging beneficiaries to work?
State Vocational Rehabilitation Agency (SVRA) staff. Briefly describe the SVRA. What role have they
played in the demonstration planning and initial implementation? How familiar are counselors with the
demonstration and what are their reactions to it? What services were provided? How many SSDI
beneficiaries have they served that are not involved with the demonstration? Is the agency operating
under an Order of Selection? If yes, are they currently able to serve? Are there waiting lists for clients
with the most severe disabilities? If yes, how long is it? What is the relationship between SVRA and other
agencies that serve SSDI beneficiaries?
Centers for Independent Living (CIL) staff. Briefly describe the CILs. What role have they played in the
demonstration planning and initial implementation? What services are provided? How successful have the
CIL providers been in encouraging beneficiaries involved with the demonstration to work? How does
their success compare to those not involved with the demonstration?
One-Stop Career Centers (OSCC) and Disability Program Navigators (DPN). Briefly describe the local
DPNs. How many are available within the service area? How have the DPNs been involved in the
demonstration planning and initial implementation? How successful have they been? What has been their
ongoing involvement in employment issues more generally?
Local disability advocates. Briefly describe the number and types of local disability advocates? How have
the disability advocates been involved in the demonstration planning and initial implementation? Which
Abt Associates Inc.

Appendix F. Resource Bank for Developing Key Informant Interview Guides
and Focus Group Moderator’s Interview Guide

F-14

BOND Implementation and Evaluation

Contract No. SS00-10-60011

are the most active and influential local advocacy groups? Which agencies and organizations are most
involved with providing employment-related services? How effective are these organizations in linking
people with disabilities to employment opportunities? Which organization serves at the Protection and
Advocacy for Beneficiaries of Social Security (PABSS) provider? Has the PABSS been involved in the
demonstration planning and initial implementation?
Other agencies. What other agencies have been involved with demonstration planning and the initial
implementation (e.g., employment networks, specialized service providers)? What has been their role?

IV.

DEMONSTRATION SITE RECRUITMENT, PLANNING, AND STARTUP

A.

OVERVIEW OF PLANNING PROCESS

Agencies/individuals involved with the planning process. What agencies were involved with the initial
planning? What was the nature of their involvement? What new partnership relationships were formed?
Were any beneficiaries involved or consulted in planning for the demonstration?
Pre-implementation activities. What were the major steps in the planning process? What was the major
vehicle for the planning process (e.g., meetings, written communications, other)? What was the timeframe
for the planning process? What inter-agency and other agreements and contracts were developed? What
recruiting and informational materials were developed for prospective participants, the public and other
interested parties by the demonstration site office? What materials worked well? Why? What marketing
materials didn’t work as well? Why? Were any special marketing materials developed that are specific to
the service area or local community? If so, please describe? How effective were these materials in
recruiting volunteers?
Designing operational policies and procedures. How were overall operational policies and procedures
for front-line SSA offices, demonstration site offices, and work incentives counseling agencies
developed? What procedures or resources were brought to bear to serve individuals with a variety of
disabilities, including cognitive impairments? What provisions were in place or developed for providing
services and information in languages other than English? What worker manuals were developed in
preparation for the demonstration?
Establishing channels of communication and coordination across agencies. What type of planning took
place to establish procedures for communications between the SSA front line office, demonstration site
office, and providers of regular and enhanced work incentives counseling? How did this planning differ
across sites? How were overall procedures developed for front-line staff to provide EWIC services? What
procedures or resources were brought to bear to serve individuals with a variety of disabilities, including
cognitive impairments? What provisions were made for providing services and information in languages
other than English? How will employment network (EN) provider payments be handled under BOND?
How will agencies/initiatives that have received waivers under BOND be handled? Which agencies does
this affect?

Abt Associates Inc.

Appendix F. Resource Bank for Developing Key Informant Interview Guides
and Focus Group Moderator’s Interview Guide

F-15

BOND Implementation and Evaluation

Contract No. SS00-10-60011

Technical assistance activities. Were the technical assistance resources available to your organization
effective and helpful? Why or why not? In which areas in particular was technical assistance needed?
What type of assistance did you receive? What additional technical assistance needs were not met?

B.

ENHANCED WORK INCENTIVES COUNSELING (EWIC) PROVIDER PLANNING

Identifying and recruiting EWIC providers. Which organizations were approached to be providers of
EWIC services? What selection process was used to choose EWIC providers? How were providers
selected in sites that include more than one state? What kinds of issues were faced in multi-state sites and
sites with large rural areas regarding management and operations of EWIC services? How did they react
to the opportunity? What types of issues arose in identifying and recruiting these organizations?
Required staffing or organizational changes. What changes, if any, in organization or staffing were
required for the demonstration? Has the structure of the organization changed to accommodate the
demonstration? Did you need to hire additional staff? Did you need to add office space or facilities? If so,
how smoothly did the process go? What challenges arose during the reorganization? What training on
BOND (or other resources) did counselors receive?
Planned roles and responsibilities. How did the selected and recruited providers plan and develop EWIC
services? Were the programs developed from scratch or was it based off of existing program? How was
planning for EWIC services handled?

C.

REGULAR WORK INCENTIVES COUNSELING (WIC) PROVIDER PLANNING

Identifying and recruiting WIC providers. What types of organizations were considered by SSA and Abt
Associates to serve as providers of additional WIC services, beyond the WIPA grantees? What selection
process was used to choose WIC providers? How were providers selected in sites that include more than
one state? What kinds of issues were faced in multi-state sites and large rural areas regarding management
and operations of WIC services? What types of issues arose in identifying and recruiting these
organizations? What training on BOND (or other resources) did counselors receive?
Required organization or staffing changes. What changes, if any, in organization or staffing were
required for the demonstration? Has the structure of the organization changed to accommodate the
demonstration? Did you need to hire additional staff? Did you need to add office space or facilities? If so,
how smoothly did the process go? What challenges arose during the reorganization?
Planned roles and responsibilities. How did the selected and recruited providers plan for WIC services
for the treatment groups? How was planning for the WIC handled?

D.

ASSESSMENT OF PLANNING AND INITIAL IMPLEMENTATION PERIOD

Experience with initial planning. To what extent did planning and startup activities occur as planned and
on schedule? Were there any delays? If so, what were they? Why did they occur? How did delays affect
other aspects of early implementation? In retrospect, was site planning adequate? Was your organization

Abt Associates Inc.

Appendix F. Resource Bank for Developing Key Informant Interview Guides
and Focus Group Moderator’s Interview Guide

F-16

BOND Implementation and Evaluation

Contract No. SS00-10-60011

well-prepared for the startup of the program? If not, how and why? How long did it take for sites to
become fully implemented?
Startup successes/challenges. What worked well during the demonstration startup? What factors
contributed to these outcome? What didn’t work as planned? How were those issues addressed? What
solutions were most effective? What solutions were least effective? How long did it take for sites to
become fully implemented?
Lessons learned. What recommendations and lessons regarding planning can be made for new sites
gearing up to implement a similar program?

Abt Associates Inc.

Appendix F. Resource Bank for Developing Key Informant Interview Guides
and Focus Group Moderator’s Interview Guide

F-17

BOND Implementation and Evaluation

V.

Contract No. SS00-10-60011

SAMPLE SELECTION, RECRUITMENT, AND ENROLLMENT

Demonstration outreach (Stage 1). What type of information did Stage 1 treatment group members
receive about BOND? What efforts, if any, were used to inform SSDI beneficiaries about the
demonstration? What information was available through the SSA field office? What information was
available from the WIPA(s)/CWICs? For those assigned to the demonstration, what factors increased the
likelihood that they would agree to use the offset? What was the process for enrolling these participants?
What types of information do they request about the demonstration? Who do they typically ask? How are
they identified in the SSA administrative data for purposes of administering benefits? What efforts have
been made to respond to the volume of calls from individuals assigned to the treatment group?
Concurrent beneficiaries (Stage 1). How are concurrent beneficiaries (those who receive SSI and SSDI)
handled? What is the process for handling the offset? Who works with the beneficiary? What works well
about this process? What might be improved?
Demonstration outreach and recruitment (Stage 2). How was outreach and recruitment conducted for
the Stage 2 demonstration group? How were potential volunteers for the study contacted and recruited?
How did project staff describe the demonstration to potential volunteers? What incentives, if any, were
used to attract participants? Were they effective? What was the level of effort and time required – on
average and by different recruitment method – to enroll recruits in the demonstration? Who contributed
most to this process?
Intake and baseline survey (Stage 2). What was the process for obtaining informed consent? Were there
any problems associated with informed consent? How were volunteers for the demonstration surveyed at
baseline and randomly assigned? What information about the evaluation is conveyed to volunteers before
and after random assignment? Were the intake and baseline survey implemented in the same way for all
beneficiaries? If not, in what ways did they vary or change?
Recruitment success rate (Stage 2). What proportion of potentially eligible participants volunteered
during Stage 2? What factors contributed to these outcomes? What do you think are some of the factors
related to willingness to participate in the demonstration? (Age, number of months on SSDI, nature of
disability, education, employment status/ experience, other factor.)

Abt Associates Inc.

Appendix F. Resource Bank for Developing Key Informant Interview Guides
and Focus Group Moderator’s Interview Guide

F-18

BOND Implementation and Evaluation

VI.

Contract No. SS00-10-60011

DEVELOPMENT AND STRUCTURE OF DEMONSTRATION

Demonstration lead. Which agency is the lead organization for the demonstration? Why was this agency
selected?
Date of demonstration implementation. When did the beneficiaries first enroll in the demonstration?
Was the date of implementation as planned? If not, was it sooner or later than anticipated? What factors
contributed to the schedule changes?

A.

DESCRIPTION OF EWIC PROVIDER

Description of EWIC provider. What is the organization’s primary mission? Whom do you serve? What
is your service area? How many beneficiaries did you serve during any given month before the
implementation of the demonstration? About how many beneficiaries do you serve now? What is your
overall operating budget? What are your primary funding sources? What other agencies that serve
SSI/SSDI clients do you work with? Do you also serve non-demonstration SSDI beneficiaries? Were
there other agencies or organizations that worked closely with the EWIC provider? What was the nature
of the collaboration? Did it change over the course of the demonstration?
Administrative and staffing structure. What is the basic administrative and staffing structure? (NOTE to
interviewer: Request and organizational chart). How many staff work with treatment group participants?
How are BOND treatment group members identified? Who is responsible for each task or area of program
operations related to the demonstration in this agency?
Types and amount of services provided. What specific counseling services are available to BOND
treatment group members receiving EWIC services? How are services provided? How do these services
compare to the services available prior to the demonstration startup?
EWIC provider service capacity before and after demonstration implementation. What was the capacity
of the organization to deliver EWIC services prior to program implementation? Did its capacity change
over the course of the demonstration? If so, what contributed to these changes?

B.

DESCRIPTION OF WIC PROVIDER

Description of WIC provider. How did WIPA and the Demonstration Contractor plan and develop WIC
services tailored to BOND participants? To what extent were WIPA programs expanded in the
demonstration sites to accommodate additional demand for WIC services resulting from BOND? What
was the nature of any such expansion? (e.g., Was staffing increased at existing organizations? Were new
organizations added?) Were there other agencies or organizations that worked closely with the WIC
provider? What was the nature of the collaboration? Did it change over the course of the demonstration?
Training on BOND?

Abt Associates Inc.

Appendix F. Resource Bank for Developing Key Informant Interview Guides
and Focus Group Moderator’s Interview Guide

F-19

BOND Implementation and Evaluation

Contract No. SS00-10-60011

Administrative and staffing structure. What is the basic administrative and staffing structure? (NOTE to
interviewer: Request and organizational chart). How many staff work with treatment group participants
receiving basic WIC services? Within the agency, who is responsible for program operations?
Types and amount of services provided. What specific counseling services are available to treatment
group study participants assigned to basic WIC services? How are services provided? How do these
services compare to the services available prior to the demonstration startup?
WIC provider service capacity before and after demonstration implementation. What was the capacity
of the organization to deliver WIC services prior to program implementation? Did its capacity change
over the course of the demonstration? If so, what contributed to these changes?

VII. DEMONSTRATION SERVICE DELIVERY

A.

DESCRIPTION OF EWIC SERVICE DELIVERY

Outreach and recruitment efforts. What efforts were made by EWICs to engage those assigned to the
demonstration group? How often did they contact the beneficiaries? How did they contact them (e.g., email, telephone, letter, home visit)? What strategies seemed to work best? Which were least helpful?
Initial and ongoing assessments. How do EWICs assess participants’ barriers to employment and service
needs? When were they assessed? Did they use standardized screening tools and/or assessments? What
kind of information is collected? How is this information used?
Process for creating and revising the Work Incentive Plan (WIP).What is the process for creating the
WIP? What information is used to decide which activities to include? Is there a standardized form used by
all EWICs for case planning? How often is the plan reviewed? What is the process for revising the plan?
What activities are typically included in the plan?
Income and work incentives counseling. Roughly what proportion of beneficiaries in the enhanced
counseling treatment group request EWIC services? On average, how often does an EWIC talk with
participants assigned to them? Do they communicate primarily in person or by telephone? How often do
they communicate? For what purposes? What type of information do they request most often?
Linking demonstration participants to outside providers. To what extent did EWICs link participants to
outside employment service providers such as Ticket to Work Employment Networks? To local One-Stop
Career Centers? To the Medicaid Buy-In program? To health care providers? To specialized treatment
providers? To the State Vocational Rehabilitation Agency?
In-house job readiness/job placement services. What types of job readiness and job placement activities
are available to the treatment group? To what extent do treatment group participants use these resources?
How successful have these services been with helping people transition to employment?

Abt Associates Inc.

Appendix F. Resource Bank for Developing Key Informant Interview Guides
and Focus Group Moderator’s Interview Guide

F-20

BOND Implementation and Evaluation

Contract No. SS00-10-60011

Availability and types of work supports. What types of work supports are available to the treatment
group? Which services are provided most often? Who provides these services? How accessible are these
services?
Job retention services. What services are available to participants after they are employed? How long are
these services available? What services are requested most often? Do EWICs continue to follow up with
working participants? If so, for how long? What are their primary service needs after they begin working?
Process for monitoring earnings. Once a participant begins working, what is the process for monitoring
earnings and ensuring that benefits are adjusted appropriately? Is this process uniform across EWICs or
does it vary by worker? Have there been any delays with adjusting benefits? If so, how long are the
delays? How often do EWICs contact working participants? Is the follow up regularly scheduled or on an
as needed basis? For what purposes do they meet? Are meetings mostly by phone or in person? What is
the average caseload for an EWIC? To what extent do they have a handle on their caseload? Does their
workload create any delays with following up with participants?
Other activities. Are there any other activities and/or supports available to those enrolled in the enhanced
services treatment group? If so, please describe.
Use of EWIC services. To what extent have SSDI beneficiaries sought information on BOND from the
counselors? How do participants access counseling services? How many participants has this agency as a
whole served since demonstration start-up? Approximately what percentage of your clients are BOND
beneficiaries? Do services available to the treatment group differ, in any way, from those provided to
other clients? If so, what are the differences?

B.

DESCRIPTION OF REGULAR WIC SERVICE DELIVERY

Process for accessing services. How do participants access WIC services? How are they informed about
the services available to them? What is the average wait time for an information request? What factors
influence the timeliness of responding to these requests?
Income and work incentives counseling. Roughly what proportion of beneficiaries in the regular
counseling treatment group request WIC services? Once a participant requests services, how often does
the CWIC talk with them? Do they communicate primarily in person or by telephone? How often do they
communicate? For what purposes? What type of information do they request most often?
Information and referral to employment services. Were there other agencies or organizations that
worked closely with the EWIC provider? What was the nature of the collaboration? Did it change over
the course of the demonstration?
Process for monitoring earnings. Once a participant begins working, what is the process for monitoring
earnings and ensuring that benefits are adjusted appropriately? Is this process uniform across CWICs or
does it vary by worker? What is the average caseload for a CWIC? To what extent do the CWICs have a
handle on their caseload? Does the CWICs’ workload create any delays with following up with
participants?

Abt Associates Inc.

Appendix F. Resource Bank for Developing Key Informant Interview Guides
and Focus Group Moderator’s Interview Guide

F-21

BOND Implementation and Evaluation

Contract No. SS00-10-60011

Description of other activities. Are there any other activities and/or supports available to participants with
CWIC services? If so, please describe.
Comparison of WIC services pre- and post-demonstration. In what ways has the demonstration
influenced the amount and types of WIC services available to beneficiaries? How did services provided
before the demonstration compare to the services available now? How do the caseload sizes of CWICs
compare?
Use of WIC services. To what extent have SSDI beneficiaries sought information on BOND from the
counselors? How many participants has this agency as a whole served since demonstration start-up?
Approximately what percentage of your clients are BOND beneficiaries? Do services available to the
treatment group differ, in any way, from those provided to other clients? If so, what are the differences?

C.

PROCESS FOR ADJUSTING BENEFITS

Responsibility for adjusting SSDI benefits. Who is primarily responsible for adjusting the benefits of
treatment group participants once they begin working? For Stage 1 treatment group participants, who
makes the adjustments for concurrent beneficiaries (those receiving SSI and SSDI)?
Process for adjusting benefits. What is the process for adjusting the benefits based on earnings? For
Stage 1 treatment group participants, how are adjustments made for concurrent beneficiaries? What is the
timeliness of these adjustments? What is the process for handling overpayments? How often does this
occur? What has been the average amount of the overpayments? Over what period of time?

Successes/Challenges. What has worked well with adjusting the benefits based on earnings? What have
been some of the challenges? How have these challenges been addressed?

VIII.

PARTICIPATION PATTERNS AND EXPERIENCE

Understanding of $1 for $2 benefits structure (treatment group). How well do you understand the $1 for
$2 benefits structure? How well do you think that treatment group members understand it?
Understanding of WIC services (treatment group). How well do you understand the regular work
incentives counseling? How well do treatment group members understand it?
Understanding of EWIC services (treatment group). How well do you understand the enhanced
counseling? How well did treatment group members understand it?
Understanding of other program features (treatment group). How well do you understand other
program features? How well did treatment group members understand other features?

Abt Associates Inc.

Appendix F. Resource Bank for Developing Key Informant Interview Guides
and Focus Group Moderator’s Interview Guide

F-22

BOND Implementation and Evaluation

Contract No. SS00-10-60011

Treatment group members’ opinions and assessment of BOND features. What are your perceptions
about treatment group members’ opinions and assessments of intervention features?
Additional relevant data:
Dynamics of participation in the benefits offset
Number of treatment group individuals who:
Are contacted by work incentives counselors.
Receive regular work incentives counseling.
Receive intensive work incentives counseling.
Are enrolled in other benefit programs (e.g., Medicaid Buy-In, SNAP, housing assistance).

Abt Associates Inc.

Appendix F. Resource Bank for Developing Key Informant Interview Guides
and Focus Group Moderator’s Interview Guide

F-23

BOND Implementation and Evaluation

IX.

Contract No. SS00-10-60011

COMMUNICATION, COORDINATION, AND INTERAGENCY RELATIONSHIPS

Interagency relationships. How would you describe your working relationship with SSA in the field
offices? Has that relationship changed since the demonstration was initiated? If so, how has the
relationship changed? How easy or difficult has it been to engage employment support providers in
serving SSDI beneficiaries? What reservations, if any, have service providers had? To what extent have
employers been involved with hiring demonstration participants? What has been your experience working
with them?
Contracts and formal agreements. What contracts and formal agreements are in place to operate the
demonstration? What is the primary vehicle for specifying demonstration activities (e.g., contract,
performance-based contract, memorandum of agreement, and other vehicle)?
Communication and service coordination between agencies. What is the process for handling payments
with EN providers under BOND? What is the process for handling waivers granted under BOND? With
what other agencies or institutions does your organization interact as part of the demonstration? Who are
your main partners? What is the nature of the interactions? What mechanisms, formal or informal, have
been developed to facilitate communication and coordination among partner organizations?
Perceptions of partner agencies about BOND. In your opinion, is BOND well designed to meet its
goals? Why or why not? What experiences have you had with reimbursement under the BOND? How
does reimbursement under BOND interact with payments under the Ticket to Work demonstration?

X.

MONITORING AND TRACKING

Systems used for tracking treatment groups. What administrative systems are used to track Stage 1
treatment groups? Stage 2 treatment groups? How well have these systems functioned?
Monitoring and tracking process. What is the process for tracking participants’ earnings and benefits
adjustments (Stages 1 and 2)? What is the process for monitoring their progress and outcomes (Stage 2)?
What tasks are involved in this process? How often are participants monitored and for how long? What
information is gathered? What is the level of effort required to monitor and track? How does the process
compare for Stage 1 and Stage 2 participants?
Consistency and quality of monitoring and tracking. How consistent are the monitoring and tracking
earnings and benefit adjustments across the service area? What is the process for tracking activities? How
does the monitoring and tracking in rural areas compare to the more urban communities? How would you
describe the quality of the monitoring and tracking? How accurate are the data? How does this compare
for Stage 1 and Stage 2 participants?

Abt Associates Inc.

Appendix F. Resource Bank for Developing Key Informant Interview Guides
and Focus Group Moderator’s Interview Guide

F-24

BOND Implementation and Evaluation

XI.

Contract No. SS00-10-60011

ASSESSMENT OF THE DEMONSTRATION IMPLEMENTATION

Preventing contamination or crossovers/integrity of random assignment. What efforts were made to
keep the treatment and control groups from socializing and sharing knowledge about the demonstration?
What efforts were made to keep the treatment groups (enhanced versus regular counseling) from sharing
knowledge about work incentives counseling? Was the integrity of the random assignment process or the
maintenance of separate research groups compromised in any site? If so, in what ways and to what
extent?
Differences in the quality, quantity, and mix of services offered to treatment and control groups. What
were the differences in the quantity, quality, and mix of services offered to members of each treatment
group and to control group members?
Comparison of enhanced and regular work incentives counseling. How does the enhanced work
incentives counseling compare to the regular counseling? What are the primary differences (e.g., staffing,
outreach, assessment, accessibility)? In what ways is the counseling similar?
Fidelity of implementation. Was the demonstration implemented as planned, overall and for each
demonstration group? If not, how and why not? How consistent were project activities and services with
project design? How closely did the provision of enhanced work incentive counseling adhere to its
design? In what ways did it differ? How and why did project operations or services change?
Extent to which expectations matched actual outcomes. What was the final participation rate among
eligible beneficiaries? Does this match expectations? Why or why not? In regard to services, have EWIC
services in this site been implemented as planned? If not, in what ways and why?
Experience with BOND implementation. How smoothly has the enhanced counseling been integrated
into the ongoing operations of this agency/organization? What has been the staff response to operating the
demonstration? How easy or difficult has it been for management and staff to understand the key features
of the demonstration? To understand and implement random assignment? To what extent were
instructions and training materials for staff clear and helpful? To what extent were research activities
carried out as planned? Were there any issues that delayed or interfered with the research activities? If so,
please describe.
Implementation strengths/challenges. Do you think that the demonstration is well designed to meet its
goals? Why or why not? Which aspects of the program design and operations are most effective? Do you
have any issues or concerns about the demonstration? If so, what are they? Which components limit the
success of the intervention?
Lessons learned with the implementation of BOND. In what ways has the demonstration grown or
become more effective since it became operational – in terms of its organization, operations, service
provision, achieving its goals, etc.? Why do you feel that way? What evidence is there? How may the
experience and lessons of the demonstration sites be used to recommend changes in program design or to
recommend best practices?
Abt Associates Inc.

Appendix F. Resource Bank for Developing Key Informant Interview Guides
and Focus Group Moderator’s Interview Guide

F-25

BOND Implementation and Evaluation

Contract No. SS00-10-60011

Additional relevant data:
Participation rate among eligible beneficiaries.
Number and proportion of beneficiaries working.
Percentage of Stage 1 participants who accepted the invitation to participate in BOND
Participation rate for Stage 2 BOND participants who received enhanced counseling
Participation rate for Stage 2 BOND participants who received regular counseling

XII.

SUCCESSES, CHALLENGES, AND LESSONS LEARNED

Site strengths/limitations. What are the primary strengths of the site? What have they done well in
implementing the demonstration? What are some of the limitations of the site? How did these limitations
play out in the implementation of the demonstration? What efforts were made to address these
limitations? How successful were these efforts?
Advantages of the benefit offset. What are some of the advantages of the benefit offset for SSDI
beneficiaries? How were these advantages achieved? What factors contributed to the success of the
demonstration?
Strengths/challenges of EWIC services. What are some of the strengths of the EWIC services? What
difference have they made in encouraging beneficiaries to return to work? What were some of the
challenges of EWIC? How were these challenges addressed? How successful were these efforts?
Unexpected consequences of the demonstration. What were some of the expected consequences of the
benefit offset? What factors increased the likelihood of these consequences?
Promising practices. What were some of the promising practices in the implementation of the BOND
demonstration? What can we learn from these practices? To what extent are these practices replicable in
another site?

Abt Associates Inc.

Appendix F. Resource Bank for Developing Key Informant Interview Guides
and Focus Group Moderator’s Interview Guide

F-26