ASFR Section B Statistical Methods 62823

ASFR Section B Statistical Methods 62823.docx

Customer Experience in The Office of the Assistant Secretary for Financial Resources Service Delivery

OMB: 0990-0489

Document [docx]

Download: docx | pdf

B. Collection of Information Employing Statistical Methods

Respondent Universe and Sampling Methods

The respondents will be comprised of future/potential applicants to federal funding mechanisms (grants and/or cooperative agreements). The collections are low-burden for respondents (based on considerations of total burden hours, total number of respondents, or burden-hours per respondent) and are low-cost for both the respondents and the Federal Government.

Procedures for the Collection of Information

Instruments/guides for data collection listed below will be attached with this statement. Additionally, an evaluation plan is also attached for your convenience.

Post-submission survey

Upon submission on Grants.Gov of an application package in response to a Notice of Funding Opportunity (NOFO), potential applicants will encounter a voluntary survey (10-15 mins) that pops up in after they submit their response to an ACL, ACF, CDC, or SAMHSA NOFO. Given the nature of funding mechanisms, it is impossible to estimate how many applications will be received by each awarding agency as many factors including funding impact the number of applicants on an annual basis. However, a sample size of 1000 participants provides enough statistical power to make assumptions based on findings.

Post-submission interview

Survey participants who were interested in sharing additional experiences and who applied for the awards that had NOFO prototypes will be scheduled for a 60 minute interview. The anonymous feedback gathered would be valuable in informing the redesign of the grant announcement and help ensure it is accessible for future applicants.

Moderated usability test

Moderated usability tests will be conducted with potential applicants and are comprised of two parts. Each part has a menu of NOFO components that the facilitator can choose from when hosting a session. The two parts are:

1. Cloze tests + content questions to measure comprehension

2. Comparative usability tests to measure readability/ease of use

Estimated time to complete: 60 minutes

Focus group

Focus groups will be comprised of grantees representing underserved communities. In the focus groups, we will collect feedback about the NOFO prototypes. HHS awarding agencies will provide ASFR with a list of eligible grantees to facilitate contact.

3. Methods to Maximize Response Rates and Deal with Nonresponse

In our efforts to collect sufficient data to ensure we can make logical arguments based on numerical/statistical significance, we never lost sight of participant burden. Thus, our goals are to maximize response rate while minimizing burden. Efforts to deal with this include but are not limited to:

Using an online/virtual platform for data collection
Having clear instructions
Keeping questions short and simple
Considering question type and question flow
Safeguarding privacy information
Increasing visibility of the survey invitation
Approaching participants at the right time
Providing compensation

4. Tests of Procedures or Methods to be Undertaken

Currently, four Notice of Funding Opportunities (NOFOs) have been developed in partnership with HHS Awarding Agencies (ACF, ACL, CDC, and SAMHSA respectively). The goal is to use these 4 as a pilot during the current fiscal year as an effective means of refining questions to minimize burden and improve utility in subsequent years where an N = 100 NOFOs is the goal. At the time of this submission, a Fast Track Clearance request was under OMB review. That request was to allow initial user experience testing (for internal purposes only). At the time of this submission and to the best of our knowledge, none of the instruments hereto outlined have been previously used in data collection. For the moderated usability script (attached), the researchers employed a Cloze test which has been an industry standard that is empirically supported.

5. Individuals Consulted on Statistical Aspects and Individuals Collecting and/or Analyzing Data