Clearance for Focus groups Template

Volume I:
Request for Clearance for Focus Groups for New
Items to Measure Certificates and IndustryRecognized Certifications among Adults in the
United States
1850-0803

December 1, 2009

Justification
Recent OECD data indicate that the United States has fallen behind other nations in the
educational attainment of the adult population, which has far-reaching consequences for our
future economic prosperity and ability to compete globally. In his February 2009 State of the
Nation address, President Obama stated an ambitious goal: by 2020 America will once again
have the highest proportion of college graduates in the world. He then asked every American
“…to commit to at least one year or more of higher education or career training.
This can be a community college or a four-year school, vocational training or an
apprenticeship. But whatever the training may be, every American will need to
get more than a high school diploma.”
The American Graduation Initiative was introduced by the President in July 2009 to directly
support this goal by helping “an additional 5 million Americans earn degrees and certificates in
the next decade.”
Attaining a postsecondary credential has become increasingly important for securing
opportunities to get high-return jobs in the United States in the 21st century. However, NCES
has traditionally only collected data on postsecondary certificates and degrees awarded through
credit-bearing instruction in traditional institutions of higher education that participate in Title IV
federal student aid programs. These comprise only a portion of subbaccalaureate education and
training American adults seek and complete to learn the skills they need to find and keep goodpaying jobs. In fact, a 2008 study using student unit record data from Florida1 found that, in
many cases, industry-recognized certifications have a greater economic value than associate
degrees.
The importance of measuring educational attainment was underlined in a December 2000 report2
from the Federal Interagency Committee on Measures of Educational Attainment:
Analyses of social and economic issues often use educational attainment as an
explanatory variable. The importance of education in shaping life experiences
and outcomes has been well documented in relation to health status, labor force
experience, earnings, criminal activity, and participation in democratic processes
as well as various income support programs. The importance accorded this
measure is demonstrated by its inclusion in virtually all Federal
social/demographic data collection efforts (including surveys, programmatic, and
administrative data collections). Agencies that collect educational attainment data
include the National Center for Education Statistics, the National Center for
Health Statistics, the Bureau of the Census, the Bureau of Labor Statistics, the
Division of Science Resources Studies of the National Science Foundation, the

1

 Jacobson, L. and Moker, C. “Pathways to Boosting the Earnings of Low‐Income Students by Increasing Their 
Educational Attainment,” The Hudson Institute and CNA, November 2008. 
2
 “Federal Measures of Educational Attainment: Report and Recommendations,” prepared by The Federal 
Interagency Committee on Measures of Educational Attainment, December 2000. 

Department of Veterans Affairs, the Defense Manpower Data Center of the
Department of Defense, and the Bureau of Justice Statistics.
Measuring progress toward the President’s goal and improving how we collect data on
educational attainment in the federal statistical system require a coordinated effort to define and
enumerate certificates and industry-recognized certifications.
This research is intended to aid development of a small set of items that will enumerate the stock
of certificates and certifications among the US adult population. These items would then become
candidates for inclusion in a national household study of US adults. The focus of the current
work is on industry-recognized certifications, but we will include some questioning about
certificates to inform later item development in that area.
The research was recommended in a meeting held on November 3rd and 4th with over 50 experts
from federal statistical and program agencies including Education, Labor, and Census;
representatives from the Council of Economic Advisors; representatives from foundations; and
representatives from nonprofit groups spanning workforce, education, industry, and standardssetting groups
The scope of this effort includes educational certificates awarded outside of credit-bearing
instruction in Title IV institutions and industry-recognized certifications, with the current focus
being on certifications. Based on the discussion at the meeting on November 3rd and 4th and on
standards established by the American National Standards Institute, the National Organization
for Competency Assurance (NOCA), and the Office of Personnel Management, we are defining
“certificate” and “certification” as follows:
Certificate: A credential awarded by a training provider or educational institution based on
completion of coursework. A certificate may be issued based completion of educational
requirements and/or some type of assessment that the content (knowledge or skills) has been
learned. A certificate is often based on a narrow scope of specialized knowledge or skills. It is
awarded for life (like a degree); although some certificates may indicate a term of validity.
Certificates cannot be taken away from the individual for disciplinary reasons. Certificates of
attendance or participation are not in scope for this work.
Certification: A credential awarded by a certification body based on an individual
demonstrating through a standardized examination process that they have acquired the
designated validated knowledge, skills and abilities. Certification is often voluntary but may be
mandatory when tied to state licensure. The identified competencies must be derived from a
formal process often called a job analysis. The examinations can be written, oral, or
performance based, but must meet psychometric rigor to demonstrate the examinations are fair,
valid and reliable. Certification is a time-limited credential that is renewed through a recertification process. A certification can be taken away from the individual for ethical violations
or incompetence.

This request for clearance is for focus groups to solicit clarification from respondents about the
language and terminology they use to talk about certifications and other properties of
certification such as time-limits, assessment, provider, and industry recognition.

Design of Focus Groups
We will conduct 3 focus groups in a professional focus group facility in Bethesda, Maryland,
during evening and lunch hours. The facilities have an observation room. A professional focus
group moderator will conduct the groups using a pre-designated focus group script (attached).
The script has been designed in collaboration with the item development working group.
The objectives of the focus group research are to:
1. identify language and terminology used by certification holders when talking about their
certifications;
2. determine properties of certifications that distinguish them from certificates and other
types of similar qualifications;
3. identify key aspects of certifications that are recognized as indicators of quality; and,
4. find commonalities and differences in language and descriptions used across different
certifications and among people with different qualifications (e.g. certificates, licenses).
Target population. To adequately test the items, it is necessary to recruit participants who can
represent the major variations of experience in the target population. We want to hear from
participants who have certifications, as well as respondents who have other types of
qualifications (e.g. certificates) to identify appropriate language recognizable to people with
similar, but different qualifications. To address this objective, we will recruit adults whose
educational attainment is at least a high school diploma or GED but who do not have a
bachelor’s degree or higher. Further, we will target certification holders in the following
industries – IT, Health, Construction, and Business, which represent four of the four groups of
people with industry-recognized certifications.
Our target age range is adults ages 21-40.
We expect to conduct a total of 30 interviews in 3 groups of 10. Two groups will be certification
holders in the 4 specified industries (IT, Health, Construction, and Business). The 3rd group will
be adults with some other similar qualification or interest in obtaining one (e.g certificate, other
certification, license, certificate from a college or trade school). The purpose of the 3rd group is
to provide a comparison group for examining differences and similarities in language and other
properties of these types of qualifications compared to industry-recognized certifications.
Table 1 shows the key groups. We will also endeavor to recruit respondents from different
racial/ethnic backgrounds, ages, and genders. We will not specify quotas for these groups
because we place greater emphasis on finding people with the specified certifications or other
qualifications.

Per industry standard, we will recruit 12 respondents per group with the expectation that 8-10
respondents will show-up on the day of the focus group. However, we will include in the focus
groups all recruited participants that show-up.
Estimated Response Burden
We expect the focus groups to be approximately 90 minutes in length. This will result in 15
burden hours per group. Thus, the estimated total respondent burden for 3 groups is 45 hours.
Table 1. Focus groups by burden
Expected
Group
number of
participants
Other
10
qualifications

Burden per
participant
(minutes)
90

Total
burden
(hours)
15

Certifications I

10

90

15

Certifications II

10

90

15

Estimated Cost Burden
There is no direct cost to respondents.
Recruitment and payment to respondents
Participants completing the focus-group will receive $75. The recruitment and research firm,
Shugoll Research, working in partnership with the American Institutes for Research, will recruit
participants from prescreened research volunteers. Shugoll Research will recruit participants
using their database of residents in the Washington, DC metro area. Information on file for
database members includes education level, occupation/employment and other demographic
characteristics such as age, gender and area of residence.
A recruitment script is attached.
Assurance of confidentiality
Participation is voluntary and participants will be asked to sign a confidentiality pledge before
interviews are conducted: Shugoll Research and the American Institutes for Research are
conducting this study for the National Center for Education Statistics (NCES) of the U.S.
Department of Education. This study is authorized by law under the Education Sciences Reform
Act (Public Law 107-279). Your participation is voluntary. Your responses are protected from
disclosure by federal statute (P.L. 107-279, Title 1, Part E, Sec. 183). All responses that relate to
or describe identifiable characteristics of individuals may be used only for statistical purposes
and may not be disclosed, or used, in identifiable form for any other purpose, unless otherwise
compelled by law.

No personally identifiable information will be maintained after the focus group analyses are
completed.

Project Schedule
Activity
Recruitment
Focus groups 1 and 2
Focus group 3

Timeline
December 2 (or upon approval by Dec. 3)
to December 16, 2009
December 17 (at 6pm and 8pm)
December 18 (at noon)