Supporting Statement for Paperwork Reduction Act Submissions, PART B.
COLLECTION OF INFORMATION EMPLOYING STATISTICAL METHODS
OMB Control Number 1090-0008
1. Describe (including a numerical estimate) the potential respondent universe and any sampling or other respondent selection method to be used. Data on the number of entities (e.g. establishments, State and local government units, households, or persons) in the universe covered by the collection and in the corresponding sample are to be provided in tabular form for the universe as a whole and for each of the stata in the proposed sample. Indicate expected response rates for the collection as a whole. If the collection had been conducted previously, include the actual response rate achieved during the last collection.
All surveys covered under this clearance will have these specific characteristics:
They will be conducted using the ACSI methodology.
They will measure customer satisfaction with federal government websites and related media.
Only a small percentage of each website’s visitors will generally be qualified to take the survey.
The criteria for qualification will vary by agency and will determine when and where the survey will be presented.
The survey will be served up randomly as visitors experience the website.
Collection of personal or demographic information through the surveys is not required by the ACSI methodology.
Government agencies will be the sole owner of the data results.
The ACSI model measures multi-variable components that are reported as indices. Various models have been developed to meet the objectives of each type of website. The models contain between 25-40 questions depending on the needs of the agency for additional custom questions. We are able to decrease the actual number of questions presented which maximizes response rates and minimizes obtrusiveness to website visitors. To accomplish this, we use an imputation statistical methodology to estimate the missing values in a data set using the information that is available. Because multiple questions are used for each element of the econometric model, stable results are obtained using a sample size of 300 respondents.
While there is a quota of 300 respondents per sample, the selection of each respondent is made as a random probability selection from the universe of visitors on the website at any given time. Thus, the maintenance of random sampling is at the individual level rather than at the total sample level. The historical response rate to these surveys of government websites conducted under this clearance ranges between 1.5% and 35% and has averaged about 8%. These results are comparable to the response rates for surveys of private sector websites. No follow-up is attempted if a citizen selected to take a survey fails to complete a portion of or the entire questionnaire.
2. Describe the procedures for the collection of information including: --Statistical methodology for stratification and sample selection.
Data will be collected through an on-line survey that will be presented to website visitors randomly. Virtually every survey will require a unique set of questions and an individualized sample design.
Survey respondents are identified through a number of conditions that are contingent on the website’s traffic and architecture. The trigger code – which causes the survey to be presented – has a number of options that can be customized to the specific website. There is a variable that causes the survey to be presented randomly to a percentage of website visitors as they experience the website. The trigger can also utilize another variable that is referred to as a loyalty factor. This would prevent a visitor from getting the survey before seeing more than a specified number of pages. The survey will run continuously over the time period specified by the agency. The agency will be able to access and analyze all data that is collected over the time period.
The survey is presented continuously throughout the subscription period, because the Internet is constantly changing and websites are continuously updating. Government agencies must have a steady pulse on what is taking place on their website through the various scores and analysis that they receive. Another reason for the survey to run continuously is that it is not possible to know how quickly the necessary data will be collected or how often agency management must have results.
No personal or demographic information of the respondents is acquired through the ACSI methodology on the survey. The government agency has the option to add custom questions that are deemed necessary or beneficial in understanding citizen concerns and priorities. Often those questions are demographic in nature.
An on-line reporting facility will be available for government personnel to access the results of the data collection. The facility is hosted on a secure remote server, and a username and password are established for clients to retrieve their data. All data will be owned by the government agency involved. The results are used to create indices, which are compiled from aggregated data and measurements.
Projected estimates for 2012 – 2014 are as follows:
The total sample is 225 X 5,000 = 1,125,000. This number is derived from the number of surveys being conducted (225) multiplied by the estimated number of respondents per survey per year (5,000).
The total sample is 275 X 5,000 = 1,375,000. This number is derived from the number of surveys being conducted (275) multiplied by the estimated number of respondents per survey per year (5,000).
The total sample is 325 X 5,000 = 1,625,000. This number is derived from the number of surveys being conducted (325) multiplied by the estimated number of respondents per survey per year (5,000).
3. Describe methods to maximize response rates and to deal with issues of non-response. The accuracy and reliability of information collection must be shown to be adequate for intended uses. For collections based on sampling, a special justification must be provided for any collection that will not yield (reliable) data that can be generalized to the universe studied.
To maximize the response rate, the surveys are kept short and take only 2-3 minutes to complete. Questions are brief and easy to answer. The welcome text will indicate that the data is being collected by an independent, third party; that the purpose of the survey is to improve the citizen satisfaction with the agency website; and that no data will be used for sales calls or other purposes. Also, the survey is presented only to a small sample of the visitor universe yet provides truly actionable information. Citizens offered a survey are not likely to receive more than one invitation to take the survey during the subscription period.
Because the index approach employs multiple questions to create the index
and because the 1 – 10 rating scale used for the majority of the questions
generates a mean (as compared to a proportion) which is then converted to a
0 – 100 scale and because we know empirically that the standard deviation
tends to be approximately 20 for this survey data, a sample size of
approximately 300 yields confidence intervals in the range of +/- 1.5 to +/- 3.5
at the 95% confidence level and confidence intervals of +/- 2.3 to +/- 2.6 at
the 90% confidence level on the 0 – 100 scale.
The intended purpose of these data collections is to guide leaders and
managers in making managerial decisions about ways to improve the quality
of government websites and experience by visitors to the sites. Data
collected in these surveys are not used to make policy decisions.
4. Describe any tests of procedures or methods to be undertaken. Testing is encouraged as an effective means of refining collections of information to minimize burden and improve utility. Tests must be approved if they call for answers to identical questions from 10 or more respondents. A proposed test or set of tests may be submitted for approval separately or in combination with the main collection of information.
Background
Those involved with the ACSI are international leaders in assessing and understanding customer satisfaction. Dr. Claes Fornell, is the Donald C. Cook Professor of Business Administration at the University of Michigan and creator of the ACSI. Along with other economic objectives – such as employment and growth – the quality of output (goods and services) is a part of living standards. Like other objectives, the quality of goods and services should be subjected to systematic and uniform measurement. This is the rationale for the ACSI. In the most general sense, the ultimate purpose of the ACSI is to help improve the quality of goods and services available to American citizens.
The ACSI is the only internationally recognized cross-industry, cross-agency methodology for obtaining comparable measures of customer satisfaction. It is a unique customer satisfaction measurement tool that utilizes the proprietary methodology behind the ACSI model to link the drivers and consequences of satisfaction. This patented econometric model enables agencies to obtain insights for valuable, high-return, customer-focused decisions. An important advantage, in contrast to methods that rely solely on survey questions, is that it produces results with statistical stability and low chance variation. This helps ensure uniform and consistent results that allow cross-agency, cross-company, and cross-industry comparisons.
In 1999, the Office of Management and Budget approved a request from the General Services Administration for an emergency collection clearance for the ACSI. That clearance was extended in 2000, when the Office of Management and Budget approved the General Services Administration’s request for a three-year generic clearance. In 2001, the General Services Administration agreed to transfer its generic clearance to the Federal Consulting Group, a franchise in the US Treasury. The Federal Consulting Group contracted with a vendor to offer the ACSI to Federal agencies for website measures and was granted a clearance in 2002. A request for a three-year extension of the generic clearance was
granted in 2005 and 2008. The Federal Consulting Group transferred to the US Department of the Interior in November 2008 and continues as the Executive Agent for the ACSI in the Federal Government.
Testing
No tests of procedures or methods will be undertaken. The ACSI methodology is patented, and the standard questionnaire has undergone extensive, rigorous testing and study by the University of Michigan and throughout academia. The surveys have already been used by the University of Michigan and the ACSI for fifteen years with great success as a key quality metric for the U.S. economy and in over 20 foreign countries. More specifically, the ACSI website surveys have been used in the Federal Government at numerous agencies over the past decade and also have undergone extensive testing to identify the most reliable set of questions that increase reliability and utility while reducing burden.
5. Provide the name and telephone number of individuals consulted on statistical aspects of the design and the name of the agency unit, contractor(s), grantee(s), or other person(s) who will actually collect and/or analyze the information for the agency.
Questions regarding any statistical aspects employed or data collection procedures used should be directed to:
Russ Merz, PhD, Research Director
ForeSee
2500 Green Road, Suite 400
Ann Arbor, MI 48105
Telephone: (734) 205-2604
Fax: (734) 352-6308
Email: [email protected]
Web: www.foresee.com
American Customer Satisfaction Index (ACSI)
Professor Claes Fornell
Stephen M. Ross School of Business
University of Michigan
Ann Arbor, MI 48109-1234
Telephone: (734) 763-9767
Fax: (734) 763-9768
Email: [email protected]
Web: www.theacsi.org
Administrative questions regarding the use of this generic clearance by
the U.S. Interior’s Federal Consulting Group should be directed to:
Ronald Oberbillig
Chief Operating Officer
Federal Consulting Group
1849 C Street, NW
Washington, DC 20240
Telephone: (202) 513-7677
Fax: (202) 513-7686
Email: [email protected]
Web: http://fcg.nbc.gov
| File Type | application/msword |
| Author | bjinnoha |
| Last Modified By | Department Of The Interior |
| File Modified | 2011-10-13 |
| File Created | 2011-10-12 |