Department of Health and Human Services
Administration for Children and Families,
Children’s Bureau
Data Collection for the Customer Satisfaction Evaluation Plan for
Child Welfare Information Gateway
Office of Management and Budget
Clearance Revision Package Supporting Statement
and Data Collection Instruments
March 2008
TABLE OF CONTENTS
Section Page
A. Justification 1
A1. Circumstances Requiring the Collection of Information 1
A2. Use of the Information 1
A3. Use of Improved Technology 2
A4. Duplication of Effort 2
A5. Burden on Small Businesses 2
A6. Consequences of Less Frequent Data Collection 2
A7. Special Data Collection Circumstances 3
A8. 5 CFR 1320.8: Public Comment and Consultation 3
A9. Respondent Compensation 3
A10. Confidentiality 3
A11. Sensitive Questions 3
A12. Respondent Burden 4
A13. Annualized Cost for Respondents 4
A14. Annualized Cost to the Federal Government 4
A15. Program Changes in Sections 13 and 14 of OMB Form 83-1 5
A16. Schedule and Reporting 6
A17. Waiver of Expiration Date Requirement 6
A18. 5 CFR 1320.9: Exceptions to Certification for Paperwork Reduction Act Submissions 6
B. Collection of Information Employing Statistical Methods 6
B1. Respondent Universe and Response Rates 6
B2. Procedures for the Collection of Information 8
B3. Maximizing Response Rates and the Issue of Non-response 10
B4. Pretesting 10
B5. Contact Information 10
LIST OF TABLES
Number Page
A-1, Burden Estimates 4
A-2, Annualized Cost to the Federal Government……………………………………………….. 5
A-3, Project Schedule 5
B-1, Expected Response Rates by Strata 7
LIST OF APPENDICES
Appendix A: Executive Order
Appendix B: IC1: Customer Satisfaction Survey – Web, Phone, E-mail, and Print versions
Appendix C: IC2: Customer Satisfaction Survey – Web, Phone, E-mail, and Print versions
Appendix D: IC3: Customer Satisfaction Survey – Web, Phone, E-mail, and Print versions
Appendix E: IC4: Publication Survey
Appendix F: IC5: Publication survey
Appendix G: IC6: Publication survey
Appendix H: IC7: Comment Card –Web and Print versions
Appendix I: IC8: Comment card – Web and Print versions
Appendix J: IC9: Focus Group Guide
Appendix K: IC10: Focus Group Guide
Supporting Statement for the
Paperwork Reduction Act of Submission
The National Clearinghouse on Child Abuse and Neglect Information (NCCAN) and the National Adoption Information Clearinghouse (NAIC) received OMB approval to collect data for a customer satisfaction evaluation under OMB control number 0970-0303. On June 20, 2006, NCCAN and NAIC were consolidated into Child Welfare Information Gateway.
In response to this consolidation, the proposed information collection activities include revisions to the Customer Satisfaction Evaluation approved under OMB control number
0970-0303.
Child Welfare Information Gateway is a service of the Children’s Bureau, a component within the Administration for Children and Families, and is dedicated to the mission of connecting professionals and concerned citizens to information on programs, research, legislation, and statistics regarding the safety, permanency, and well-being of children and families. Child Welfare Information Gateway’s main functions are identifying information needs, locating and acquiring information, creating information, organizing and storing information, disseminating information, and facilitating information exchange among professionals and concerned citizens. A number of vehicles are employed to accomplish these activities, including, but not limited to, website hosting, discussions with customers, and dissemination of publications (both print and electronic).
The Customer Satisfaction Evaluation was initiated in response to Executive Order 12862 issued on September 11, 1993 (Appendix A). The Order calls for putting customers first and striving for a customer-driven government that matches or exceeds the best service available in the private sector. To that end, Child Welfare Information Gateway’s evaluation is designed to better understand the kind and quality of services customers want, as well as customers’ level of satisfaction with existing services. The proposed data collection activities for the evaluation include customer satisfaction surveys, customer comment cards, selected publication surveys, and focus groups.
The evaluation plan focuses solely on issues of direct concern to our customers. Specifically, customers are asked their opinions on the efficiency and quality of services provided by Child Welfare Information Gateway. Collectively, evaluation data is currently used to improve and help inform:
General Child Welfare Information Gateway services—to ensure the easy accessibility and usefulness of the information provided;
Product development and acquisition—to acquire and create publications in which customers are most interested;
Outreach plans and strategies—to increase the awareness and use of various services; and
Web site content and functioning—to ensure the usability and usefulness of this primary communication vehicle.
The primary means of gathering information is through electronic format including the surveys and comments through the web-site as well as e-mail linked surveys. The evaluation team also imports online survey data from a password protected server into SPSS, a statistical software package used for storing, managing, and analyzing quantitative data (i.e. survey data). Per guidance outlined in 5 CFR 1320.8, the focused use of electronic data collection methods is intended to reduce the burden on respondents.
The Customer Satisfaction Evaluation Plan is the only evaluation of customer service for Child Welfare Information Gateway; therefore, there is no similar information already available.
We do not plan to survey small businesses.
A6. Consequences of Collecting the Information Less Frequently
The current evaluation plan includes periodic data collection methods and utilizes the minimum amount of data collection necessary to obtain valid, reliable, and useful information that can help inform Child Welfare Information Gateway products and services. Reducing data collection further would jeopardize the quality and integrity of the results including the generalizability to all Information Gateway customers.
Responding to this data collection effort is completely voluntary. Individuals recruited for this effort will be given the opportunity not to respond at all, and to respond whenever it is convenient for them during the data collection period. Additionally, it is possible that the same individual could receive a Web site survey request online and then contact Child Welfare Information Gateway via telephone on a day that surveys are being administered to telephone customers, and be asked to complete it again. All customers have the option of refusing a survey request without penalty or to request the survey be offered at a more convenient time for them.
Child Welfare Information Gateway staff are consulted regarding the frequency and usefulness of the data collection effort on a quarterly basis. Respondents regularly provide comments regarding the data collection effort through their completion of surveys. Comments from both Child Welfare Information Gateway staff and respondents have been incorporated into the revisions of the data collection instruments including the provision of an opportunity to complete the survey at a later, more convenient time.
As required by CFR 1320.8, a Federal Register notice appeared on November 13, 2007 (Volume 72, Number 218) Page 63911-63912. The first notice received no public comment.
Not applicable, the data collection plan does not call for payment to respondents for participation.
We will assure that the anonymity of respondent identity and information are safeguarded and that respondents understand the voluntary nature of their participation. Identifying information is not collected as part of this data collection effort.
Sensitive issues will not be raised at any time during the conduct of the survey or focus groups.
Estimates of response burden are outlined in the following table. These estimates were developed using FY2007 response rates from data collection approved under OMB Control Number 0970-0303.
Table A-1: Annual Burden Estimates
Instrument |
Affected Public |
Number of Respondents |
Number of Responses per Respondent |
Average Burden hours per Response |
Total Burden Hours |
Customer Satisfaction Survey (Website, Email, Print, and Phone) |
Individuals/ Households |
1,208 |
1 |
0.07759 |
93.72872 |
Private Sector |
258 |
1 |
0.07759 |
20.01822 |
|
State, Local, or Tribal Governments |
310 |
1 |
0.07759 |
24.0529 |
|
Publication Survey |
Individuals/ Households |
57 |
1 |
0.052 |
2.964 |
Private Sector |
13 |
1 |
0.052 |
0.676 |
|
State, Local, or Tribal Governments |
15 |
1 |
0.052 |
0.78 |
|
Comment Card (Web and Print versions) |
Private Sector |
132 |
1 |
0.014 |
1.848 |
State, Local, or Tribal Governments |
132 |
1 |
0.014 |
1.848 |
|
Focus Group Guide |
Private Sector |
3 |
1 |
1.0 |
3.00 |
State, Local, or Tribal Governments |
25 |
1 |
1.0 |
25.00 |
|
TOTAL Estimated Total Annual Burden Hours |
173.915 hours |
||||
There will be no cost to survey respondents or focus group participants.
We estimate the annualized costs to the Federal government to be $60,400. This cost estimate is based on the work plan under the cooperative agreement for this effort. As outlined below, the estimated annual Federal costs associated with this effort include costs to manage and implement the data collection activities outlined in this application.
Table A-2: Annualized Cost to the Federal Government
|
Total Project Cost |
# of Years |
Annualized Cost |
Gateway Staff Hours – (design, development, testing, phone survey interviews, focus group moderation, management, data analysis, reporting) |
300,000 |
5 |
$60,000 |
Indirect Costs – (printing, mailing, web costs, maintenance, conference calls) |
2,000 |
5 |
$400 |
Totals |
$302,000 |
5 |
$60,400 |
The current OMB inventory estimated 177 hours as the reporting burden; however, the proposed evaluation estimates 174 hours as the reporting burden. Adjusted calculations were based on response rates obtained during fiscal year 2007 and alterations to survey questions resulting from the consolidation of NCANCH and NAIC into Child Welfare Information Gateway.
Table A-3 presents the data collection and reporting schedule:
Submit OMB revision package: March, 2008
Ongoing data collection : March 2008 - March 2011
Reporting: Quarterly and semi-annual internal reporting with the purpose of impacting products and services; Annual Comprehensive Evaluation Reports to the Federal Government
Analysis of data will primarily include basic descriptive statistics such as frequencies, means, and percentages. Additional analyses may include ANOVAs, t-tests, and chi-squares if appropriate.
This section does not apply. We are not seeking approval to not display the
expiration date for OMB approval.
A18. Exceptions to Certification for Paperwork Reduction Act Submissions
No exceptions are requested.
B. COLLECTION OF INFORMATION EMPLOYING STATISTICAL METHODS
This evaluation is designed to reach the various types of customers using Child Welfare Information Gateway services such as professionals, students, and customers with a personal situation. Data will be collected by delivering satisfaction surveys to customers receiving Information Gateway services via the Web, e-mail, and telephone. Web surveys are scheduled to appear during randomly selected one-hour time slots throughout the year. In order to better reach Information Gateway’s target population, child welfare professionals, the randomly selected time slots over-represent business hours with 80% of timeslots during business hours and 20% of timeslots from 6-7am EST and 8-11pm EST. Telephone and e-mail customers are offered the survey on 52 randomly selected business days throughout the year. Additionally, publication surveys will be provided to request input on 10 selected publications annually. Focus groups and comment cards will be used to collect information from a variety of user groups (e.g., Web, conference). Expected response rates for each user group are provided below:
Table B-1: Expected Response Rates by Strata |
||||||
Response Offers |
Expected Response Rate (based on FY 2007 data) |
Expected Number of Responses by Strata |
Total Responses |
|||
Professionals |
Students |
Personal |
||||
Web-site Survey |
22,071 |
7% |
680 |
262 |
603 |
1,545 |
E-mail Survey |
171 |
17% |
14 |
5 |
10 |
29 |
Telephone Survey |
364 |
47% |
53 |
10 |
108 |
171 |
Print Survey |
194 |
16% |
30 |
1 |
0 |
31 |
Publication Survey |
531 |
16% |
85 |
0 |
0 |
85 |
Customer Focus Groups |
28 |
100% |
28 |
0 |
0 |
28 |
Comment Card |
N/A |
N/A |
114 |
40 |
110 |
264 |
Total |
1,004 |
318 |
831 |
2,153 |
||
The response rates for the website survey, email survey, telephone survey, and print survey listed in the table above are based on response rates obtained for data collected during FY 2007. Random selection of survey delivery time blocks was weighted to increase the representation of business hours in order to reach more professional survey respondents. No statistical sampling methodology will be used beyond randomization of survey offers and over-representation of business hours. Following is a description of how response rates were calculated for each evaluation methodology.
Customer satisfaction surveys. Based on FY2007, Child Welfare Information Gateway anticipates that approximately 22,071 invitations to participate in the Web survey will be made annually, with a 7% response rate, resulting in 1,545 completed Web surveys. For customers that contact Information Gateway via telephone, we anticipate offering the survey to 364 customers per year, with a 47% response rate, resulting in 171 completed surveys. For customers that contact Information Gateway via email, we anticipate offering the survey to171 customers per year, with a 17% response rate, resulting in 29 completed surveys. For customers that request a publication, we anticipate offering the survey to 194 customers per year, with a 16% response rate, resulting in 31 completed surveys.
Publication survey. As the publication survey is a new component of the Information Gateway evaluation plan, response rates are based on results from the FY 2007 print customer satisfaction surveys and average mailing numbers per publication. It is anticipated that 531 selected publication surveys will be mailed with an average response rate of 16%, resulting in 85 completed publication surveys.
Comment Cards. Child Welfare Information Gateway anticipates that 264 customers will complete a comment card via Web or at a conference. This estimate is based on the number of customers that submitted suggestions and questions to the Website during FY 04-05.
Focus groups. Child Welfare Information Gateway anticipates that approximately four focus groups per year will be conducted, with an average of seven respondents per group.
The specific procedures utilized for each data collection instrument are described below.
Customer Satisfaction Survey. The goal of the customer satisfaction survey is to assess customer characteristics; the kind and quality of services customers are looking for; customer’s level of satisfaction with those services; Web site accessibility, usage, and satisfaction; and customers’ access to, knowledge, and use of Child Welfare Information Gateway services. The target population includes all general customers with a specific focus on professional customers.
Web-site: During 200 pre-determined randomly selected hours throughout the year, website visitors will receive an invitation to complete an electronic survey after clicking on the web-site five times. The visitor will have the option to decline the survey, have the survey offered at a later time, or answer the survey. Eighty percent of the pre-selected hour time blocks are during business hours (160) and twenty percent are during non-business hours (40). (Appendices B, C, & D)
E-mail: On 52 randomly selected workdays throughout the year, customers who e-mail Child Welfare Information Gateway will be offered the opportunity to complete a customer satisfaction survey. These customers will receive an invitation to complete the survey through providing a hyper-link in the response e-mail to their information request. (Appendices B, C, & D)
Telephone: On 52 randomly selected workdays throughout the year, customers who call Child Welfare Information Gateway will be offered the opportunity to complete a customer satisfaction survey over the telephone. If they agree to complete the survey, they will be transferred to a separate phone surveyor who will read them the survey and record their answers. (Appendices B, C, & D)
Request for publications: On 52 randomly selected workdays throughout the year, customers who call or e-mail Child Welfare Information Gateway will be offered the opportunity to complete a customer satisfaction survey by mail. They will be sent a print version of the customer satisfaction survey along with their requested publication and a pre-paid business reply envelope. (Appendices B, C, & D)
Publication Survey. The Publication survey is designed to elicit feedback on specific publications. Customers receiving selected publications throughout the year will be asked about their satisfaction with the individual aspects of the publication. Each year, Child Welfare Information Gateway will select up to 10 new publications to be assessed. Approximately 500 hundred customers receiving publications via mail will receive a paper-and-pencil publication survey, along with a business reply envelope for survey return. (Appendices E, F, & G)
Comment Cards
A standard comment card will provide a formal opportunity for customers to provide suggestions as well as to comment on current Information Gateway products and services. The comment card will be placed on Information Gateway’s Website and will also be handed out to professionals at conferences. (Appendices H & I)
Customer Service Improvement Focus Groups
On an annual basis, Child Welfare Information Gateway staff will conduct focus groups with customers to collect in-depth information from various user groups (e.g., Web visitors, various professional groups) regarding the kinds and quality of services that customers are seeking, customers’ satisfaction with those services, and the evaluation process. These focus groups will primarily happen during conferences the customer already plans to attend and be conducted by trained Information Gateway staff. This information will supplement the customer satisfaction survey data and will be used to enhance Information Gateway services and strengthen the customer satisfaction evaluation process. (Appendices J & K)
B3. Maximizing Response Rates and the Issue of Non-Response
Because there is no tracking of individuals who choose not to complete the survey, it is impossible to gain insight into factors that effect response rates. However, Information Gateway has taken steps to maximize response rates. Specifically, we have made the instruments short and clear, as we understand that customers are unlikely to be willing to spend more than a few minutes on the survey. In addition, those customers receiving requests online will be given an “Ask me later” option that he or she may use in order to complete the survey at a more convenient time. Finally, the random selection of survey time blocks was weighted to increase the representation of business hours in order to increase the number of professional survey respondents.
B4. Pretesting of Procedures or Methods
All instruments for this data collection effort were tested for usability by Child Welfare Information Gateway staff.
B5. Contact Information
Should you have any questions about the contents of this OMB submission package, please contact one of the following individuals:
La Chundra Lindsey, Children’s Bureau, 1250 Maryland Avenue, SW, Room 8148, Washington, DC 20024, telephone: 202.205.8252
Helena Wallin-Miller, ICF International, 10530 Rosehaven Street, Suite 400, Fairfax, VA 22030, telephone: 910.255.6124, email: [email protected]
Christine Leicht, ICF International, 10530 Rosehaven Street, Suite 400, Fairfax, VA 22030, telephone: 703.385.3200, email: [email protected]
Kathleen Wang, ICF International, 10530 Rosehaven Street, Suite 400, Fairfax, VA 22030, telephone: 703.385.3200, email: [email protected]
Appendices
Appendix A: Executive Order
Appendix B: IC1: Customer Satisfaction Survey – Web, Phone, E-mail, and Print versions
Appendix C: IC2: Customer Satisfaction Survey – Web, Phone, E-mail, and Print versions
Appendix D: IC3: Customer Satisfaction Survey – Web, Phone, E-mail, and Print versions
Appendix E: IC4: Publication Survey
Appendix F: IC5: Publication survey
Appendix G: IC6: Publication survey
Appendix H: IC7: Comment Card –Web and Print versions
Appendix I: IC8: Comment card – Web and Print versions
Appendix J: IC9: Focus Group Guide
Appendix K: IC10: Focus Group Guide
| File Type | application/msword |
| File Title | Supporting Statement for the |
| Author | Shamegan |
| Last Modified By | Bob |
| File Modified | 2008-03-14 |
| File Created | 2008-03-14 |