SUPPORTING STATEMENT FOR PAPERWORK REDUCTION ACT SUBMISSION
Qualtrics Platform Survey for the Bureau of Educational and Cultural Affairs (ECA)
OMB Number 1405-XXXX
Why is this collection necessary and what are the legal statutes that allow this?
The Department of State’s Bureau of Educational and Cultural Affairs (ECA) regularly monitors and evaluates its programs through the collection of data of program performance in order to enable ECA staff to assess the effectiveness of its programs and to modify and plan future programs. The Qualtrics survey platform will create a standardized method for survey distribution, data collection, and data visualization.
As a normal course of business and in compliance with OMB Guidelines contained in Circular A-110, recipient organizations are required to provide, and the U.S. State Department required to collect, periodic program and financial performance reports (See A-110: www.whitehouse.gov/omb/circulars/a110/a110.html). The responsibility of the State Department to track and monitor programmatic and financial performance necessitates a database that can facilitate this in a consistent and standardized manner. ECA would like Qualtrics to serve as this database for program performance monitoring data.
Legal authorities and administrative requirements that necessitate the collection of these data can be found below:
OMB Circular A-110 (.51 and .52)
Government Performance and Results Act of 1993 (GPRA)
Government Performance and Results Modernization Act of 2010 (GPRAMA)
Mutual Educational and Cultural Exchange Act of 1961, as amended, 22 U.S.C. 2451 et seq (also known as the Fulbright-Hays Act)
Foreign Aid Transparency and Accountability Act of 2016
Foundations for Evidence-based Policymaking Act of 2018
What business purpose is the information gathered going to be used for?
The primary users of the collected data through the Qualtrics platform will be ECA and its implementing partners. The information collected will be used to monitor performance at the program, office, and bureau levels to inform any beneficial program adjustments to strengthen the utility and cost-effectiveness of the programs. Having access to raw program performance data allows ECA to conduct additional analyses at the bureau level, which we are unable to do with the data currently being reported.
Is this collection able to be completed electronically (e.g. through a website or application)?
The collection of information is completely electronic. Users will obtain a login from the ECA MELI Unit and access Qualtrics by logging in through the internet (here: https://stateecafedramp.gov1.qualtrics.com/); once the login is setup, they will receive a User Guide and link to a training that outlines the steps in the process to create a survey.
Qualtrics provides functionality that will minimize time required both to administer and respond to the survey, including:
Individualized links to the survey for easy tracking of response rate by key categories
Automated reminders only to those who have yet to complete the survey
Strong validation measures to reduce errors and to generate cleaner raw data
Options to configure surveys to be compliant with the National Institute of Standards and Technology (NIST).
Qualtrics provides the ability to preview surveys in final forms, both through web browser and mobile platforms (both Apple and Android) to ensure that all questions display properly. We expect that all submissions will be collected electronically.
Does this collection duplicate any other collection information?
There is no centralized system used to collect, aggregate, and visualize all ECA program survey raw data. Qualtrics will collect raw individual data points that can then be aggregated further. The volume and greater level of detail of the data collected through Qualtrics will allow for more accurate insights and greater reliability of data, leading to more effective program implementation.
Describe any impacts on small business.
We do not envision significant impacts on small businesses. In fact, offering the Qualtrics platform at no cost allows implementing partners, particularly small businesses, to save money in that they would not need to budget for costs for their own survey platform to conduct ECA-required surveys. In terms of a burden of time, the Qualtrics platform will provide implementing partners with a pre-populated survey tool, allowing for efficient survey creation by selecting relevant questions and adding any potential custom questions. Implementing partners will be provided with the information relevant to their program(s) and will be able to select these questions easily through the pre-populated survey tool.
What are consequences if this collection is not done?
The ability to effectively assess the performance of individual ECA programs and a broader portfolio of programs necessitates standardized reporting in an electronic format. The use of Qualtrics as a survey platform will streamline data collection, aggregation, visualization, and reporting for implementing partners and ECA program offices. Without use of the Qualtrics platform, ECA will only have access to aggregated data instead of a full set of raw data – limiting the ability to draw meaning from the data. The access to data provided through the use of the Qualtrics platform will provide ECA with the information necessary to understand the effectiveness of our programs and provide insight on where we can make adjustments (if necessary).
Are there any special collection circumstances?
This data collection involves no special circumstances.
Document publication (or intent to publish) a request for public comments in the Federal Register.
The 60-day Federal Register Notice was published on August 6, 2024 (89 FR 64037). No comments were received in that period. The Department will publish a notice in the Federal Register soliciting public comments to OMB for a period of 30 days.
Are payments or gifts given to the respondents?
No payments or gifts are proposed for respondents.
Describe assurances of privacy/confidentiality
ECA and its external contractors follow all procedures and policies stipulated under the Privacy Act of 1974, as amended. In addition, each survey requires the participant to consent, such as:
CONSENT
TO PARTICIPATE
By
clicking the button to begin the survey below, you are giving your
consent to participate in this survey. If you do not wish to
participate, please click the exit survey link below.
Are there any questions of a sensitive nature asked?
We do not anticipate that ECA and implementing partners will ask questions of a sensitive nature in their surveys.
Describe the hour time burden and the hour cost burden on the respondent needed to complete this collection.
There are no additional hours added to customary and usual business practices as implementing partners already conduct their own surveys as part of their award requirements; the only difference is now they will be using ECA-procured Qualtrics survey platform. It is estimated that setting up and conducting a survey (an activity implementing partners are already required to do) takes each implementer an average of 5 hours for their first survey but substantially less for subsequent rounds, for an average of 3.5 hours per survey. ECA estimates that the Department will, on average, provide awards to approximately 100 grantees per year in the future, in which all will be requested to use Qualtrics for survey creation and distribution. The estimated annual burden for survey creating and distribution via Qualtrics is 875 hours. The Department used the Performance Management Evaluation and Learning Services Indefinite Delivery Indefinite Quantity (PMELS – IDIQ)1 – a contract mechanism designed to provide Department of State bureaus with a streamlined contracting process to acquire monitoring and evaluation services – to estimate the burden hour cost for mid-level performance monitoring specialist, which is $168.29 in 2017 USD. Annually, the estimated cost is $147,254 ($168.29 per hour x 875 hours annually).
Describe the monetary burden to respondents (out of pocket costs) needed to complete this collection.
There is no monetary burden to implementing partners who administer surveys or the respondents who take the surveys. The ECA Bureau will bear all costs associated with Qualtrics.
Describe the cost incurred by the Federal Government to complete this collection.
The estimated cost burden to the Federal government is $663,420 per year. The Federal Government estimate is based on the cost to ECA for the Qualtrics survey platform and responses, as well as the number of hours ECA staff devote to maintaining and updating the Qualtrics platform. The average number of hours the Department estimates being devoted to this purpose is 1,000 per year. The average hourly salary of $56.71 (GS-13 Social Science Analyst, Step 1) was used in making this calculation. The Department estimated the cost of fringe benefits (using a rate of 69 percent2 and an overhead rate of 31 percent3) leading to an hourly rate of 113.42 with benefits and overhead.
Explain any changes/adjustments to this collection since the previous submission.
This is a new collection.
Specify if the data gathered by this collection will be published.
Information collected is not intended for publication, although ECA may share data in part or whole with external stakeholders such as the U.S. Congress or through social media posts to highlight results from specific programs. Information is used to evaluate current programs, assess future programming needs, and maximize the effective use of resources.
If applicable, explain the reason(s) for seeking approval to not display the OMB expiration date. Otherwise, write “The Department will display the OMB expiration date.”
The Department will display the OMB expiration date.
Explain any exceptions to the OMB certification statement below. If there are no exceptions, write “The Department is not seeking exceptions to the certification statement.”
The Department is not seeking exceptions to the certification statement.
[describe and list descriptive statistics; aggregate numbers and disaggregate by program, look at key numbers]
This collection does not employ statistical methods.
2 Source: Congressional Budget Office, “Comparing the Compensation of Federal and Private-Sector Employees, 2011 to 2015” (April 2017), https://www.cbo.gov/publication/52637. The wages of Federal workers averaged $38.30 per hour over the study period, while the benefits averaged $26.50 per hour, which is a benefits rate of 69 percent.
3 Source: U.S. Department of Health and Human Services, “Guidelines for Regulatory Impact Analysis” (2016), https://aspe.hhs.gov/system/files/pdf/242926/HHS_RIAGuidance.pdf. On page 30, HHS states, “As an interim default, while HHS conducts more research, analysts should assume overhead costs (including benefits) are equal to 100 percent of pretax wages….” To isolate the overhead rate, the Department subtracted the benefits rate of 69 percent from the recommended rate of 100 percent.
| File Type | application/vnd.openxmlformats-officedocument.wordprocessingml.document |
| Author | Kearney, Paul M |
| File Modified | 0000-00-00 |
| File Created | 2024-11-01 |