Supporting Statement Part B_6-25-10

Supporting Statement Part B_6-25-10.pdf

Child Welfare Information Gateway

OMB: 0970-0303

Document [pdf]

Download: pdf | pdf

COLLECTION OF INFORMATION EMPLOYING STATISTICAL METHODS

B1.

Respondent Universe and Response Rates

This evaluation is designed to reach the various types of customers using Child Welfare
Information Gateway services such as professionals, students, and customers with a personal
situation. Data will be collected by delivering surveys to customers receiving Information
Gateway services via the Web, e-mail, live chat, in print, and telephone. Web surveys are
scheduled to appear during randomly selected one-hour time slots throughout the year.
Telephone, e-mail, and live chat customers are offered the survey on 52 randomly selected
business days throughout the year. Publication surveys will be provided to request input on 10
selected publications annually and an online tool / Web section survey will help us understand
customers’ experiences with specific web services and offerings (e.g., databases, learning
centers, specific websites/sections). Webinar surveys will collect feedback to improve webinars
and understand future webinar needs. Focus groups and comment cards will be used to collect
information on a range of Information Gateway services from a variety of user groups (e.g., on
the Web, at conferences). Customer service representatives will gather information on the
professional background and geographic location of customers that call the toll-free phone line
for assistance. Expected response rates for each user group are provided below:

Table B-1: Expected Response Rates by Strata
Expected
Expected Number of Responses by Strata
(based on FY 2009 data)
Response
Response
Total
Rate
Individuals
State / Local
Private
Offers
Responses
(based on
Federal
/
/ Tribal
Sector
Government
FY 2009
Households
Governments
data)
Customer
Survey
(Website,
Email,
Print, Live
Chat, and
Phone)
Publication
Survey
Comment
Card
(General
Web and
Conference
e versions)
Online
Tool / Web
Section
Survey
Webinar
Feedback
Survey
General
Focus
Group
Guide
User Needs
Assessment
Focus
Group

22,602

5.38%

846

182

187

1,215

N/A

124

N/A

300

431

7,175

229

287

2,657

62%

597

1,049

1,646

100%

4,106

95%

2,730

608

561

3,899

4,191

1,526

885

1,049

7,650

The response rates for the customer survey (delivered via Website, email, print, live chat,
and phone) are based on response rates obtained for data collected during FY 2009. Random
selection of survey delivery time blocks was weighted to increase the representation of business
hours in order to reach more professional survey respondents. No statistical sampling
methodology will be used beyond randomization of survey offers and over-representation of
business hours. Following is a description of how response rates were calculated for each
evaluation methodology.
Customer surveys. Sample size estimates for customer surveys (delivered via Web,
phone, email, live chat, and print) are based on FY 2009 data collection activities. We expect that
that approximately 22,001 invitations to participate in the Web survey will be made annually,
with a 4.2% response rate, resulting in 922 completed Web surveys. For customers that contact
Information Gateway via telephone, we anticipate offering the survey to 220 customers per year,
with a 64.5% response rate, resulting in 142 completed surveys. For customers that contact
Information Gateway via email, we anticipate offering the survey to 64 customers per year, with
a 22% response rate, resulting in 14 completed surveys. We are unable to calculate the response
rate for surveys mailed to customers that order publications online1; however, we expect to
collect 102 print surveys as this was the sample collected during FY 2009 and our methodology
will remain the same. Surveys will also be offered to customers via live chat. During FY 2009,
there was an average of 4.94 live chats per day and the live chat survey will be delivered on 52
days each year. Since this is the first time that live chat surveys will be delivered, we are using
the response rate for the email survey (22%) as a basis for the response rate estimate. We
1

A new tracking system is in place that will allow us to calculate a response rate in the future.

estimate that we will collect 56.5 live chat surveys each year (4.94 live chats / year * 52 survey
days * 22%).
Publication survey. We anticipate collecting 124 publication surveys each year as this
was the total sample during FY 2009 and we are using the same data collection methodology.
We are unable to calculate a response rate for the publication survey since we previously were
unable to calculate the total universe of customers that accessed publications online.
Comment Cards. Child Welfare Information Gateway anticipates that 313 customers
will complete a comment card via Web and 118 customers will complete a comment card at a
conference. These estimates are based on the number of customers that submitted comments via
Web and at conferences during FY 2009.
Online Tool / Web Section Survey. The online tool / Web section survey is a new
addition to Information Gateway’s evaluation package. This survey will help us understand
online users’ experiences with specific web services and offerings (e.g., database, tools, learning
centers, specific websites/sections). Since this is a new data collection activity, we are using the
response rate from the Web survey (4%) for the sample size calculation. We expect to deliver
7,175 online tool / Web section surveys resulting in a total sample size of 287 surveys.
Webinar Feedback Survey. During FY 2009, 2,657 people participated in a webinar
sponsored by Information Gateway. Since this is the first year we are delivering surveys to
webinar participants, we are drawing upon our experiences from another data collection effort
(operating under OMB control number 1121-0277) that also administered surveys to webinar
participants as a basis for our response rate estimate. We expect there to be a 62% response rate,
which will result in a total sample size of 1,647 webinar surveys.
Focus groups. Child Welfare Information Gateway will conduct six focus groups per
year with an average of eight respondents per group for a total annual sample size of 48
respondents. Three focus groups will use the general focus group guide and three will use the
user needs assessment focus group guide.
Customer Services Information Questions. Information Gateway’s customer service
representatives will ask customers contacting us via telephone, email, letter, or fax to provide
information on their professional background and the geographical location of their workplace.
During FY 2009, Information Gateway’s customer service representatives responded to 4,106
requests for information either via telephone, email, letter or fax. We anticipate that the majority
(95%) of customers will agree to provide this information to customer service representatives
4

resulting in a total sample size of 3,900. We estimate that the response rate for this activity will
be high since such a high volume of customers already provide us with this type of information;
however, we are adding this as an additional evaluation activity in order to collect this
information in a more systematic way.
B2.

Procedures for the Collection of Information
The specific procedures utilized for each data collection instrument are described below.

Surveys
Customer Survey. The goal of the customer survey is to assess customer characteristics;
the kind and quality of services customers are looking for; customer’s level of satisfaction with
those services; and customers’ access to, use of, and need for Child Welfare Information
Gateway services. The target population includes all general customers with a specific focus on
professional customers.
Website: During 495 pre-determined randomly selected hours throughout the year, website
visitors will receive an invitation to complete an electronic survey after clicking on the website five times. The visitor will have the option to decline the survey, have the survey offered
at a later time, or answer the survey. The goal of the evaluation is to reach 1,545 web surveys
during each fiscal year. During FY 2009, 90 percent of the pre-selected hour time blocks
were during business hours and 10 percent were during non-business hours. (Appendices B,
C, & D)
E-mail: On 52 randomly selected workdays throughout the year, customers who e-mail Child
Welfare Information Gateway will be offered the opportunity to complete a customer survey.
These customers will receive an invitation to complete the survey using a hyper-link in the
response e-mail to their information request. (Appendices B, C, & D)
Telephone: On 52 randomly selected workdays throughout the year, customers who call
Child Welfare Information Gateway will be offered the opportunity to complete a customer
survey over the telephone. If they agree to complete the survey, they will be transferred to a
separate phone surveyor who will read them the survey and record their answers.
(Appendices B, C, & D)
Requests for printed publications: On 52 randomly selected workdays throughout the year,
customers who call or e-mail Child Welfare Information Gateway to order publications or
who order publications online will be offered the opportunity to complete a customer survey
5

by mail. They will be sent a print version of the customer survey along with their requested
publication and a pre-paid business reply envelope. (Appendices B, C, & D)
Live Chat: On 52 randomly selected workdays throughout the year, customers that solicit
assistance via live chat will be offered the opportunity to complete a customer survey. They
will be sent a hyperlink to complete the survey online at the conclusion of the live chat
conversation. (Appendices B, C, & D)
Publication Survey. The Publication survey is designed to elicit feedback on specific
publications. Customers receiving selected publications throughout the year will be asked about
their satisfaction with the individual aspects of the publication. Each year, Child Welfare
Information Gateway will select up to 10 new publications to be assessed. Approximately 500
hundred customers receiving publications via mail will receive a paper-and-pencil publication
survey, along with a business reply envelope for survey return. (Appendices E, F, & G)
Online Tool / Web Section Survey. The online tool / Web section survey will be used to
better understand online users’ experiences with specific web services and offerings (e.g.,
database, tools, learning centers, specific websites/sections). Each year, Child Welfare
Information Gateway will select up to ten Web services to be assessed. Customers that use one
of ten selected Web services will be offered an opportunity to complete a voluntary survey
online. (Appendices K, L, & M)
Webinar Feedback Survey. Webinar participants will be sent an email asking them to
complete a voluntary survey at the conclusion of the event. We expect to solicit feedback for 25
webinar events each year. (Appendices N & O)
Customer Services Information Questions. Information Gateway’s customer service
representatives will ask all customers contacting us via telephone, email, letter, or fax to provide
information on their professional background and the geographical location of their workplace.
This information will be used by Information Gateway and the Children’s Bureau to better
understand where to target marketing and outreach efforts. (Appendices T, U, & V)
Comment Cards
A standard comment card will provide a formal opportunity for customers to provide
suggestions as well as to comment on current Information Gateway products and services. The
comment card will be placed on Information Gateway’s Website and will also be handed out to
professionals at conferences. (Appendices H, I, & J)
6

Customer Service Improvement Focus Groups
On an annual basis, Child Welfare Information Gateway staff will conduct six focus
groups with customers to collect in-depth information from various user groups (e.g., Web
visitors, various professional groups) regarding the kinds and quality of services that customers
are seeking, customers’ satisfaction with those services, and the evaluation process. These focus
groups will primarily happen during conferences the customer already plans to attend and will be
conducted by trained Information Gateway staff. The user needs assessment focus group guide
will evaluate the usefulness of a specific product or tool offered by Child Welfare Information
Gateway. The general focus group guide will be used to supplement data collected from the
customer survey. (Appendices P, Q, R & S)
B3.

Maximizing Response Rates and the Issue of Non-Response

Because there is no tracking of individuals who choose not to complete the survey, it is
impossible to gain insight into factors that effect response rates. However, Information Gateway
has taken steps to maximize response rates. Specifically, we have made the instruments short
and clear, as we understand that customers are unlikely to be willing to spend more than a few
minutes on the survey. In addition, those customers receiving requests online will be given an
“Ask me later” option that he or she may use in order to complete the survey at a more
convenient time. Finally, the random selection of survey time blocks was weighted to increase
the representation of business hours in order to increase the number of professional survey
respondents.
B4.

Pretesting of Procedures or Methods

All instruments for this data collection effort were tested for usability by Child Welfare
Information Gateway staff.
B5.

Contact Information

Should you have any questions about the contents of this OMB submission package,
please contact one of the following individuals:

La Chundra Lindsey, Children’s Bureau, 1250 Maryland Avenue, SW, Room 8148,
Washington, DC 20024, telephone: 202.205.8252, email:
[email protected]

Helena Wallin-Miller, ICF International, 9300 Lee Highway Fairfax, VA 22031,
telephone: 910.255.6124, email: [email protected]

Christine Leicht, ICF International, 9300 Lee Highway Fairfax, VA 22031, telephone:
703.934.3000, email: [email protected]

Kathleen Wang, ICF International, 9300 Lee Highway Fairfax, VA 22031, telephone:
703.934.3000, email: [email protected]

Appendices
Appendix A: Executive Order
Appendix B: IC1: Customer Survey
Appendix C: IC2: Customer Survey
Appendix D: IC3: Customer Survey
Appendix E: IC4: Publication survey
Appendix F: IC5: Publication survey
Appendix G: IC6: Publication survey
Appendix H: IC7: Comment Card –Website and Conference versions
Appendix I: IC8: Comment card – Website and Conference versions
Appendix J: IC9: Comment card – Website and Conference versions
Appendix K: IC10: Online Tool / Web Section Survey
Appendix L: IC11: Online Tool / Web Section Survey
Appendix M: IC12: Online Tool / Web Section Survey
Appendix N: IC 13: Webinar Feedback Survey
Appendix O: IC 14: Webinar Feedback Survey
Appendix P: IC15: General Focus Group Guide
Appendix Q: IC16: General Focus Group Guide
Appendix R: IC17: User Needs Assessment Focus Group Guide
Appendix S: IC18: User Needs Assessment Focus Group Guide
Appendix T: IC19: Customer Services Information Questions
Appendix U: IC20: Customer Services Information Questions
Appendix V: IC21: Customer Services Information Questions

File Type	application/pdf
File Title	Microsoft Word - Supporting Statement Part B_6-25-10.doc
Author	15058
File Modified	2010-06-29
File Created	2010-06-29