OMB Memo on SPADA

MEMORANDUM
MEMORANDUM TO:

Robert Sivinski
Office of Statistical and Science Policy
Office of Management and Budget

THROUGH:

Jeffrey H. Anderson
Director
Bureau of Justice Statistics

FROM:

Suzanne M. Strong
Statistician, Prosecution and Judicial Statistics Unit
Bureau of Justice Statistics

DATE:

June 28, 2018

SUBJECT:

BJS request to conduct a pilot test for frame development for the
Survey of Publicly Appointed Defense Attorneys (SPADA), under
the OMB generic clearance agreement (OMB Number 1121-0339).

Introduction
The Bureau of Justice Statistics (BJS) is planning a new data collection effort to gain a better
understanding of the experiences, workloads, roles, training needs, and quality of representation
provided by publicly appointed defense attorneys through the Survey of Publicly Appointed
Defense Attorneys (SPADA). The developmental work for SPADA includes developing a survey
instrument, sampling strategy, nonresponse follow-up strategy, and developing the universe. In
later development work, BJS plans to test the sampling and recruitment strategies, and, assuming
successful outcomes, plans to fully implement the survey. The current development work for
SPADA was awarded as a cooperative agreement to the Urban Institute, National Association for
Public Defense (NAPD), and Indigent Defense Research Association (IDRA).
SPADA is a component of the BJS Indigent Defense Program. Within this program, BJS
previously conducted the Census of Public Defender Offices (CPDO) in 2007 and the National
Survey of Indigent Defense Systems (NSIDS) in 1999 and 2013 (all three surveys under OMB
Control Number 1121-0095). Both the CPDO and NSIDS are administrative surveys that collect
information from agencies about their provision of indigent defense services. These agency-level
surveys collect information on the number of attorneys employed, the number and types of cases
closed, the available training areas offered, the characteristics used to determine indigence, and
the fees associated with representation. The SPADA is intended to collect information from the
attorneys actually providing representation to those who qualify for indigent defense.
BJS, the project team, and the expert panel have designed a survey, cognitively tested the survey,
and developed a proposed nonresponse follow-up strategy that will be tested in later
1

development work. The focus of this request is the development of the universe frame in order to
develop a nationally representative sampling strategy.
There is no list of publicly appointed defense attorneys in the United States. Research conducted
by BJS and the project team has determined that 28 states and the District of Columbia are able
to provide statewide rosters of active publicly appointed defense attorneys (i.e., attorneys who
have provided indigent defense services in the past year), this is Frame 1; see Attachment A.
These states reported in the 2013 NSIDS that they have a centralized statewide indigent defense
system and are able to generate lists of publicly appointed defense attorneys. While the
completeness of these lists and the quality of the data that are available is unknown, BJS prefers
to focus the attention on the states unable to generate lists before devoting further effort to Frame
1 states. Therefore, the states in Frame 1 will not be contacted for rosters until later in the
development process.
The remaining 22 states, Frame 2, are organized at the local level, not centralized at the state
level, and are unable to provide statewide lists of active publicly appointed defense attorneys
(see Attachment B). In these states, 1,466 Primary Sampling Units (PSUs) were formed
according to how public defense provision is organized at the local level. In 18 of the 22 states in
Frame 2, the PSUs are defined exclusively as counties (n=1,334 counties), while in the other four
states (Georgia, Florida, Kansas, and Tennessee), the PSUs are defined as clusters of counties
(n=132) representing the organization of the criminal justice system in those states. BJS seeks
OMB approval to contact a sample of these jurisdictions to determine their ability to provide lists
and, for a subsample of those contacted, to collect the lists and assess the utility of those lists for
universe development. After stratifying the PSUs by population size and urbanicity (i.e., whether
the county or county cluster is urban or rural, according to U.S. Census designations for
population density), the project team randomly selected 74 PSUs for the purpose of testing
whether rosters can be compiled and obtained from local leaders, or if other methods of sampling
are necessary (see Attachment C for the PSU leader contact scripts). In 16 of the 74 PSUs, BJS
will request that the PSU leader(s) compile and deliver the list, so that the lists can be reviewed
for completeness (see Attachment D for the data elements requested of the PSU leaders).
Summary of current request. BJS requests clearance under the generic clearance agreement
(OMB Control Number 1121-0339) to determine the ability of PSUs in Frame 2 to compile
complete lists of publicly appointed defense attorneys in order to develop a nationally
representative sampling strategy for the full implementation of the future SPADA. For the 22
states in Frame 2 that are unable to provide statewide lists, BJS proposes to contact a sample of
74 PSUs randomly selected from a list of the 1,466 PSUs stratified by population size and
urbanicity to determine the level of effort necessary for PSUs to compile and deliver a list of
active publicly appointed defense attorneys. For 16 of the 74 PSUs, BJS proposes to request the
list and evaluate the completeness of the rosters provided.
Request for Pilot Testing of Universe Generation
The ability to generate rosters in the 22 Frame 2 states is highly variable. These counties or
county clusters may use multiple indigent defense delivery methods, making the compilation of
the lists potentially challenging. One of the members of the project team is the National
2

Association for Public Defense (NAPD), a membership organization serving primarily public
defenders, but also some assigned and contract counsel. NAPD has knowledge about the
organization of public defense in the 22 decentralized, locally organized states and advises that
there is variation in the ability of counties to provide a single list of all attorneys that provide
indigent defense. Lists will likely have to be compiled by multiple sources, with sources having
varying abilities to generate such lists. For example, a county could have a public defender
office, an assigned counsel program, and a contract with a law firm to handle conflict cases. That
county would likely have at least 2-3 people to contact to obtain three separate lists. However, of
the three, the public defender might be the only office with the ability to generate a single list of
active attorneys who represent indigent clients. If the county does not have a person who
manages contracts or assigned counsel, such as the court administrator, then the assigned counsel
lists are likely kept by each judge in the county. That could include contacts to each judge, who
may or may not keep a computerized list, or who may be unwilling to share such a list. Even the
law firm that handles contract cases may not be able to accurately identify which of their
attorneys are responsible for handling the public defense cases.
BJS and the project team estimate that approximately 6 contacts per PSU will be necessary to
generate the rosters. At one end of the spectrum is the PSU where the public defender manages
the primary providers (defenders) and the conflict providers. At the other end of the spectrum is
a PSU where each judge’s chambers maintains its own list of appointed attorneys. Based on the
knowledge and the experiences of the expert panel, the project team expects that most PSUs will
have a primary contact person and an additional 2-5 contacts. BJS is planning for 6 contacts per
PSU.
The project team and expert panel expect that rural jurisdictions will be the least able to compile
and provide these lists. Rural jurisdictions are less likely to have adequate funding for
computerized case management systems, are more likely to cover a wider jurisdiction (e.g.,
multiple counties may cluster together to provide indigent defense) that requires coordination
across counties, and rural jurisdictions are more likely to be under-staffed. Thus, the project team
used population size and urbanicity to stratify Frame 2 prior to generating PSUs.
A total of 1,466 PSUs made up of 1,334 counties (from 18 states) and 132 county-clusters (from
four states). The population groups are shown below in Table 1. Population group 1 consists of
the top 40 jurisdictional clusters (PSUs) in terms of population size, which collectively account
for 40% of the total population of the states in Frame 2. Population group 2 consists of 60 PSUs,
which collectively account for 20% of the total population of Frame 2. Population group 3
consists of 150 PSUs, which collectively account for 18% of the total U.S. population of Frame
2. Finally, population group 4 consists of 1,216 PSUs that collectively comprise 22% of the total
population of Frame 2. The first three population groups contain PSUs that are all defined as
urban (per U.S. Census designations), while the fourth population group contains both urban and
rural areas.

Table 1. Frame 2 PSU development and selection

3

PSUs
drawn

% of
total
PSUs
Sampled

Margin
Of Errora

Population
Group

Population
Range

Number
of PSUs

% of
Frame 2
PSUs

1

>1,000,000

40

3%

40%

30

75%

9.06%

2

440,000-999,999

60

4%

20%

15

25%

22.10%

3

158,000-439,999

150

10%

18%

13

9%

26.06%

% of Frame
2 Population

4
< 158,000
1,216
83%
22%
16
1%
24.35%
Margin of Error was calculated assuming a sample proportion of 50% (of contacted PSUs are able to generate
rosters) with a 95% confidence interval. As margin of error decreases as the sample proportion approaches 0 or 1,
this is the maximum margin of error that will accompany the sample proportions calculated from this pilot test.

a

Total Population (22 States) = 207,201,242
The objective of SPADA is to generate nationally representative statistics about publicly
appointed defense attorneys. BJS focused on drawing a higher proportion of PSUs from the more
populous counties (drawing larger samples in the more populous groups) because there are more
attorneys located in those counties and BJS is seeking to maximize its understanding of its ability
to generate national estimates given limited resources. Population group 4 represents 22% of the
Frame 2 population. It is also the only population group that contains rural counties. Some of the
smaller counties in population group 4 likely group together to provide indigent defense, so the
number of PSUs in that group may be overstated. The level of effort to determine which of the
1,216 counties are combined would have exhausted much of the project team’s remaining time
and resources for this project, and since the proportion of the population in stratum 4 is low, BJS
and the project team determined that a smaller sample was justified.
Assuming a 95% confidence interval and that half of the PSUs contacted will be able to provide
rosters (a sample proportion of 50%), the margin of error for PSUs representing 40% of the
Frame 2 population is ±9.06% (with marked increases for subsequent strata). While BJS would
certainly prefer more precision in its estimates of who is able to provide a list as a result of this
development work, this margin of error will likely be sufficient to guide future universe
development and inform BJS decisions about its ability to combine Frame 2 PSUs with Frame 1
states to generate national estimates.
Of the 74 PSUs drawn for the sampling test, BJS will inquire about the effort needed to generate
the lists, as well as request full rosters from 16. The 16 PSUs were selected with a focus on the
PSUs least able to provide rosters in population group 4. BJS and the project team, based on
conversations with the expert panel, expect that larger PSUs likely have more resources, more
staff, and are generally more able to generate lists than smaller PSUs. Within population group 4,
BJS and the project team expect that rural jurisdictions would be less able than urban ones to
generate lists of publicly appointed defense attorneys. Of the 16 counties, 7 are from stratum 4 (3
urban and 4 rural by Census definitions). The remaining nine were split evenly between
population groups 1, 2, and 3 (3 PSUs in each) (see Table 2 for the distribution of selected
jurisdictions). The remaining 58 PSUs will be asked to describe the level of effort necessary to
generate the rosters, but will not ask the county to complete that work.
Table 2. Sampling Frame 2 PSU selections where the roster will be requested
4

Population Group
1
2
3
4

Number of PSUs selected
3
3
3
7

Urban or rural
Urban
Urban
Urban
3 Urban/4 Rural

In order to identify the first contact, the team will consult internal sources (NAPD, IDRA) and
external sources (the expert panel, internet searches) to determine if a public defender office
exists in the PSU. The public defender is the most likely source of a list and knowledgeable
about other potential sources of indigent defense providers. After contacting the public defender,
or if there is no public defender, the protocol outlined in Table 3 will be used to identify and
contact other PSU leaders as needed. If the last contact (state bar association) is not successful,
the project team will conduct further research to see if the PSU is part of a circuit or other shared
indigent defense arrangement.
Table 3. Protocol for identifying PSU contacts
Contact
attempt
1

Organization

Action if successful

Action if unsuccessful

Public defender office

Go to contact 2

2

3

Other centralized defender
organization (e.g.,
managed counsel program,
defense coordinator)
Court administrator

Obtain additional
contacts
Obtain additional
contacts

Go to contact 4

4

Local Bar Association

5

State Bar Association

Obtain additional
contacts
Obtain additional
contacts
Obtain additional
contacts

Go to contact 3

Go to contact 5
Conduct further research
on PSU

The initial PSU leaders will be contacted to determine the number of persons needed to generate
a complete list of publicly appointed attorneys in each PSU. A second contact will be sent to all
PSU leaders identified by the initial contact to inquire about the level of effort required to
generate the list by each person. For those 16 PSUs where the roster has been selected to be
provided, the project team will ask the PSU leader and additional contacts they identify to
provide the roster. If the PSU leader or contacts indicate that a list can be provided but it would
take too long, the project team will capture the level of effort required. Refusals will be recorded
and not replaced. It is important for the final sampling effort to know how many potential
refusals could occur in Frame 2.
Some jurisdictions may require agreements in order to provide the lists of attorney names and
contact information. The project team will record the circumstances under which the list can be
5

obtained (e.g., preparation of a Memorandum of Understanding or PSU leaders will draw a
sample of attorneys). If the PSU leaders cannot provide a list, the project team will explore if
there are other methods for obtaining the list, documenting these processes and the levels of
effort required to obtain the lists. For example, Andrew Davies, a member of the project team, is
the Director of Research for the New York State Office of Indigent Legal Services. He expects
that some county indigent defense leaders would be unable to generate lists of attorneys readily,
but in those counties, there is a central budget office responsible for processing reimbursement
for attorneys who represent indigent clients. The County budget offices could be able to provide
the names of the attorneys and contact information, which is necessary for payments to be made,
but Dr. Davies is unsure what other information those budget offices would be willing or able to
provide. Other counties in other states may also have alternate sources to generate these lists of
attorneys. Once the rosters are obtained, the project team will compare the lists to the requested
data elements in Attachment D to assess the completeness of the lists. The remaining 58
jurisdictions will only be asked about the persons necessary to compile a roster, the level of
effort and time needed, and the challenges of compiling the lists.
Even though public defenders and attorneys are likely to be listed in public directories, BJS and
the project team will ensure that the rosters will be delivered securely. The Urban Institute has a
secure FTP file delivery system. If this system is not convenient for PSU leaders, Urban Institute
will instruct the PSU leader to password protect the file, email the file to Urban, and in a separate
email, provide the password to the file.
Burden Hours
The burden hour estimates for the respondents are provided in Table 4. The project team expects
that each PSU will require up to six persons to compile a list; therefore, BJS estimates that the
burden on respondents will be approximately 260 hours. The time required to ask Frame 2
leaders about their ability to generate rosters will be approximately 20 minutes for each of the 58
PSUs, to include time to follow up with multiple contacts in counties with multiple delivery
methods for indigent defense. BJS estimates that the 16 sampled PSUs will be able to generate
lists with an average effort of 240 minutes.

6

Table 4. Burden Hour Estimates for Respondents

Task #
1

2

3
4

Task Description
Email (or call) to primary PSU leaders to
asking them to identify additional contact
persons
Email (or call) scheduling phone call with
all leaders identified in PSU (may require
more than one contact)
Follow-up phone call to the 58 PSUs not
producing lists
Compile and share list of publicly appointed
defense attorneys
Total burden

Number of
respondents

Estimated
burden
(in minutes)

Total burden
(in hours)

74

5

6.2

444 (74x6)

10

74

232 (58x6)

20

116

16

240

64
260.2 hours

Institutional Review Board
The Urban Institute’s Institutional Review Board (IRB) determined the pilot testing protocol to
be compliant with informed consent and data confidentiality standards (Attachment E).
Contact Information
Questions regarding any aspect of this project can be directed to:
Suzanne M. Strong
Statistician
Bureau of Justice Statistics
U.S. Department of Justice
810 7th Street, NW
Washington, DC 20531
Office Phone: (202) 616-3666
E-mail: [email protected]
Attachments
Attachment A:
Attachment B:
Attachment C:
Attachment D:
Attachment E:

States Able to Provide Lists (Frame 1)
States Not Able to Provide Lists (Frame 2)
Frame Email and Phone Scripts
Data elements requested of the 16 selected PSUs
IRB Approval

7