408(b)2 Focus Group SS part B 8-26-2014

B. Collections of Information Employing

Statistical Methods

The agency should be prepared to justify its decision not to use statistical methods in any case where such methods might reduce burden or improve accuracy of results. When Item 17 on the Form OMB 83-I is checked, "Yes," the following documentation should be included in the

Supporting Statement to the extend that it applies to the methods proposed:

1. Describe (including a numerical estimate) the potential respondent universe and any sampling or other respondent selection methods to be used. Data on the number of entities (e.g., establishments, State and local government units, households, or persons) in the universe covered by the collection and in the corresponding sample are to be provided in tabular form for the universe as a whole and for each of the strata in the proposed sample. Indicate expected response rates for the collection as a whole. If the collection had been conducted previously, include the actual response rate achieved during the last collection.

EBSA provided its contractor, ICF International (ICF), three data sets to be used in selecting plan fiduciaries for the focus groups. The data sets are based on the OPR 2011 Form 5500 Private Pension Plan Research File which contains information from the Form 5500, Short Form (SF), and schedules H and I. The full research file is available on the EBSA website.¹ A few additional variables were added to the research file.

The data sets contain 51 variables. The first data set contains plans with less than one hundred participants and has 270,114 plan observations. The second data set contains plans with between 100 and 999 participants and has 51,807 plan observations. The third data set contains plans with between 1,000 or more participants and has 8,217 plan observations.

The dataset contains only plans meeting the following criteria:

• Plans where there are positive administrative provider payments (SF5500 lines 8F+8G>0 or on F5500 Schedule H Part II (i) line 5 >0 ; Schedule I Part I (2) lines h+i>0

• Excludes pension plans with benefit codes “3F²”, “3C³”, “3E⁴”, “2N⁵” and “3B. ⁶”.

• Excludes plans with less than $1,000 in assets per participant.

ICF is to draw from the data sets plans to contact in order to obtain subjects for the Focus Groups. ICF is to use the following criteria when constructing the focus groups:

• Geography- Focus groups will occur in cities where ICF has offices. DOL anticipates using zip code information on the Form 5500 to locate plans in the area.

• Finding the responsible plan fiduciary- While the plans sponsor and the plan administrator could be different people, in all plans in the data set they report the same phone number. ICF should ensure that the individual invited to participate in the focus group is a “responsible plan fiduciary,.” a plan fiduciary with authority to cause the plan to enter into a service arrangement.

• For the small plans EBSA wants each focus group to contain plans with varying sizes based on the number of plan participants. EBSA would like about 1/3 of each focus group to come from each of the following categories: 1-24, 25-74, and 75-100 participants.

• For the small plans, EBSA would like to include in each focus group a mix of plans filing the Form 5500 versus the Short Form. It is believed this could be a proxy for bundled versus unbundled services to the plan. Filing type is also correlated with plan size. In 2011, about 17 percent of plans filed the Form 5500 while about 83 percent of plans filed the Short Form.

TABLE 1.--Small Plans by the Number of Participants and Filing Type
	Form 5500	Short Form	Total
1-25	27,627	139,446	167,073
26-75	13,917	72,769	86,686
76-100	4,029	12,326	16,355
Total	45,573	224,541	270,114

• For plans from the 100-999 participants group, EBSA wants the focus group to contain plans with varying sizes based on the number of plan participants. EBSA would like about 1/3 of each focus group to come from each of the following categories: 100-399, 400-699, and 700-999 participants.

• For plans from the 1,000+ participants group EBSA wants the focus group to contain plans with varying size, but is not providing a quota. EBSA does want some from the smaller end of the size range.

TABLE 2. --Number of Plans by Plan Size (Measured by Number of Participants)
City	Small Plans Filing F5500			Small Plans Filing Short Form			Medium Sized Plans			Large Plans
	1-25	26-75	76-99	1-25	26-75	76-99	100-399	400-699	700-999	>1,000
Atlanta	609	250	34	2,580	1,240	206	608	121	31	18
Chicago	1,517	581	68	5,444	2,965	481	1,358	247	54	42
LA	1,846	673	88	8,859	4,110	697	1,651	272	67	43
DC	1,164	596	49	6,311	3,184	556	1,400	244	42	47

Geographic Location

Eight Focus groups will be conducted in four different geographic locations, with two focus groups per day in each of the four locations.

1.    Atlanta, Georgia. Atlanta will represent the Southeast United States. Atlanta was selected over Columbia, South Carolina and Durham, North Carolina because plans in Atlanta have particularly high average expenses ($73,000 per plan) and a higher proportion of participants file Form 5500 (18 percent) versus the short form than in Columbia (13 percent) and Durham (16 percent).  Two focus groups of small plan fiduciaries will be conducted here.

  

2.    Chicago, Illinois. Chicago is the third most populous city in the United States, after New York City and Los Angeles. With 2.7 million residents, it is the most populous city in the American Midwest. Due to their large population base, either Chicago, Illinois or Los Angeles, California will likely host the focus group session for the fiduciaries of large-size plans. A focus group for fiduciaries of small plans and a focus group for fiduciaries of large plans will be conducted here, with the expectation that fiduciaries of large plans can be recruited from outside the Chicago area and they will travel to Chicago.

3.    Los Angeles, California. Los Angeles was chosen over other California Metropolitan Statistical Areas (MSAs) (Sacramento, San Diego, and San Francisco) because plans in Los Angeles have the highest average annual plan expenses ($90,000) of the MSAs reviewed.  As noted in the previous bullet, Los Angeles may host the focus group session for the fiduciaries of large-size plans. A focus group for fiduciaries of small plans and a focus group for fiduciaries of medium-sized plans will be conducted here.

4.    Washington, DC. Washington, DC will represent the Mid-Atlantic region and was chosen over the three surrounding office locations (Baltimore, MD; Charleston, WV; and Richmond, VA) for its low percentage of profit-sharing plans (91 percent versus the 96 percent national average).  Two focus groups of small plan fiduciaries will be conducted here.

2. Describe the procedures for the collection of information including:

• Statistical methodology for stratification and sample selection,

• Estimation procedure,

• Degree of accuracy needed for the purpose described in the justification,

• Unusual problems requiring specialized sampling procedures, and

• Any use of periodic (less frequent than annual) data collection cycles to reduce burden.

ICF will use purposive sampling, in other words, ICF will select participants based on the knowledge of the population (fiduciaries) and the purpose of the study (to collect feedback about participants’ experience receiving disclosures from their retirement plan’s service providers). In instances in which we receive lists from USDOL, ICF will randomly select participants from those lists. In both instances, ICF will create both a primary and an alternate list. Specifically, in order to address the inevitability of recruits who may not be able to attend at the last minute (generally 5 to 10 percent), we will create an alternate list of participants that ICF can call on to fill those slots. ICF will inform those recruits that they are on the alternate list and that we may call them in the event that we need to fill a slot and that their agreement to be on that list may include asking them to be prepared within a 24-hour span of time.

It is important to state that even though we may randomly select participants for inclusion in the focus group, this method of data collection does not result in generalizable results (findings cannot be extrapolated to populations outside of the test population or to all people similar to the participants). However, other researchers can use the results to judge whether the findings apply in their context. In order to enhance transferability, the Principal Investigator will be thoroughly described from the research context, as well as the assumptions that were central to the study.

3. Describe methods to maximize response rates and to deal with issues of non-response. The accuracy and reliability of information collected must be shown to be adequate for intended uses. For collections based on sampling, a special justification must be provided for any collection that will not yield "reliable" data that can be generalized to the universe studied.

To increase response rates the ICF is mailing each prospective focus group participant a letter explaining the purpose of the focus group and to encourage their participation prior to contacting them by phone to request their participation. Non-response issues are not being addressed.

A generalizable sample is not being sought. The Department has chosen the method of focus groups to solicit information on plan fiduciaries personal experiences and to obtain input on how to better improve the experiences and usability of the information. Funding availability limits the number to participants that would allow of focus group testing to be generalizable.

4. Describe any tests of procedures or methods to be undertaken. Testing is encouraged as an effective means of refining collections of information to minimize burden and improve utility. Tests must be approved if they call for answers to identical questions from 10 or more respondents. A proposed test or set of test may be submitted for approval separately or in combination with the main collection of information.

ICF conducted interviews with two subjects in April 2014. The interview subjects both work as advisors at investment firms. During these interviews, the subjects were asked all of the questions on the focus group script and asked to provide their assessment of whether the questions were appropriate, whether they would be understood by the focus group participants, and whether the questions needed any revisions. These comments were considered when revising the script in response to public comments.

5. Provide the name and telephone number of individuals consulted on statistical aspects of the design and the name of the agency unit, contractor(s), grantee(s), or other person(s) who will actually collect and/or analyze the information for the agency.

Dr. Jon Hecht (415-677-7145) from ICF will be conducting the focus groups.

1 http://www.dol.gov/ebsa/publications/form5500dataresearch.html#planbulletins

2 Plan Sponsor(s) receiving services of leased employees, as defined in Code Section 414(m), during the plan year.

3 Plans not intended to be qualified- A plan not intended to be qualified under Code Sections 401, 403, or 408.

4 Out-dated code for prototype plans

5 Code Section 408 accounts and annuities.

6Plan covering self-employed individuals.