SUPPORTING STATEMENT - PART A
Evaluation of the Family Advocacy Program’s Domestic Violence Awareness and Child Abuse Prevention Campaigns – 0704-XXXX
1. Need for the Information Collection
Outreach programs are critical to improving gaps in key areas, including a lack of awareness of available resources. The Defense Department, Service branches, and the Family Advocacy Program (FAP) have historically applied child abuse prevention and domestic violence awareness efforts during national awareness months to leverage increased visibility and media coverage. The 2021 United States Government Accountability Office (GAO) report, “Domestic Abuse: Actions Needed to Enhance DOD’s Prevention, Response, and Oversight” (report can be accessed at https://www.gao.gov/products/gao-21-289), indicated that ongoing efforts are needed to build and maintain top-of-mind awareness of available resources, reporting options for those experiencing domestic abuse, and evaluation of the effectiveness of awareness campaigns reaching targeted audiences. Evaluation of the reach and penetration of these campaigns is imperative to assess the effectiveness of the messaging developed and delivered by the FAP.
The Office of the Secretary of Defense (OSD) Military Community and Family Policy (OSD MC&FP) contracted with Clearinghouse for Military Family Readiness at Penn State (Clearinghouse) for this data collection to evaluate the effectiveness of the Family Advocacy Program’s (FAP) Domestic Violence Awareness (DVA) and Child Abuse Prevention (CAP) campaigns at increasing knowledge and awareness. This project will also ensure FAP’s compliance with the Fiscal Year 2022 National Defense Authorization Act (FY22 NDAA) Section 527, “Activities to Improve Family Violence Prevention”.
The authorization for this evaluation comes from 10 U.S.C 136, Under Secretary of Defense for Personnel & Readiness and DoD Instruction 6400.01.
2. Use of the Information
Information will be collected to gain insight into the (1) effectiveness of the campaigns (respondents include military service members and their partners), and to (2) evaluate campaign implementation (respondents include FAP staff).
(1) To assess campaign effectiveness, respondents will include military service members and their partners who are 18 years of age or older, are associated with a military installation participating in the project and are English literate. Installations from each branch of service will be selected, and we will seek to recruit 150 participants from 16 installations to ensure sufficient power to observe statistical significance. These respondents are being selected to complete the four online data collection instruments since they are the target audience of the DVA and CAP awareness campaigns. Thus, their responses can provide the best insight into the impact of the campaigns. Respondents will be able to earn up to $70 in Amazon gift cards as an incentive for their participation, and they will receive these gift cards in increments after completion of each data collection instrument.
Respondents will be recruited using a variety of methods. Methods may include DMDC administrative data, flyers, email/listserv messages, or social media posts (i.e., Facebook, Twitter, Instagram). Since we are recruiting from 16 specific installations, we will request the email addresses for individuals residing within a 5 to 10-mile radius of the installations from DMDC, based on zip codes. Approval for recruitment using social media will be obtained prior to recruitment.
Respondents will have the opportunity to complete four online data collection instruments at different time points (see “Data Collection Type and Timeline” under item #6). The instruments will be completed before and after the launch of each respective campaign and will assess knowledge and beliefs related to domestic violence awareness and child abuse prevention. All data collection instruments will be accessed via a link or QR code that will take them to each respective survey.
After clicking on the link or scanning the QR code on the recruitment material, respondents will be directed to a REDCap survey (“Gift Card Survey”) to provide their name and email address. Immediately after they click the “next” button, they will be directed to a Qualtrics survey to complete the eligibility and demographics questions (a list of demographic variables are provided under item #11). If eligible, the respondent will receive the link to complete Survey 1 via the email address they provided. For subsequent surveys (Surveys 1 through 3), respondents will receive the survey links through the email address they provided. Respondents can access the collection instrument in their own time and in any location that is most comfortable to them. Online surveys are to be completed outside of normal working hours and will be completed at the convenience of the respondent. After completing the collection instrument, the respondent will submit their responses online.
Respondents will receive up to three email reminders to complete the surveys during the four waves of data collection. To support follow-up and distribution of the electronic gift cards, respondents will provide their name and an email address. Respondents will receive their electronic gift card code via email. The gift card survey data will be separate from the data collection instruments (i.e., stored in REDCap vs. Qualtrics) and will not be stored with respondent data. The respondent will be informed that their PII will not be attached to their responses in any way and will solely be used for tracking and gift card distribution purposes.
After submission of the data collection instruments, data will be stored on Penn State’s secure server and within Qualtrics. Penn State’s Clearinghouse for Military Family Readiness research staff will collect and analyze survey data to determine the impact of the DVA and CAP campaigns. The research staff have been trained and certified by the Penn State Institutional Review Board as required by governmental regulations. The findings will inform DoD about the impact of the campaigns on the participants’ knowledge and awareness.
(2) To assess campaign implementation, FAP staff will be invited to participate in program improvement conversations to provide information about successes and challenges with campaign implementation. These interviews will occur in November/December 2024 after implementation of both campaigns. Penn State Clearinghouse staff will conduct these conversations via Microsoft Teams or Zoom and the conversations will last no more than one hour. FAP staff will be asked to provide a description of campaign activities during the conversations. With the permission of FAP staff, conversations will be recorded to aid in qualitative data analysis. Penn State has a single-use contract with both Zoom and Microsoft Teams. All data is required to be kept confidential and is subject to very specific data security contract requirements, including encryption.
In addition, Penn State Clearinghouse research staff will assess campaign implementation at the selected installations and online. During site visits, implementation checklists will be used to record the presence of campaign materials, their frequency, visibility, etc. For online campaign implementation, PSU research staff will record the frequency of website pages, social media posts, etc. that promote campaign content.
3. Use of Information Technology
All responses (100%) will be collected electronically since survey collection instruments and conversations will be online.
4. Non-duplication
The information obtained through this collection is unique and context-specific and is not already available for use or adaptation from another cleared source.
5. Burden on Small Businesses
This information collection does not impose a significant economic impact on a substantial number of small businesses or entities.
6. Less Frequent Collection
Data will be collected at four time points from each participant over a 1-year period (see “Data Collection Type and Timeline” table below). This evaluation will occur one time in 2024. The evaluation is being structured to assess whether patterns in the target outcomes metrics will exhibit some change in awareness of DVA and CAP campaigns, respectively, from baseline (pre-test) to post-test to pre-post-post-test to post-post-post-test with the baseline starting in August/September for the year-long campaign. The new theme for DVA is launched in October, and outreach efforts using that theme will be year-long. Since CAP campaign efforts may occur prior to the launch in April, the data collection instrument distributed in November will include some CAP-related items. The collection of four data points related to DVA and three data points related to CAP enables us to measure impact of the campaign and outreach efforts over time and whether that change or awareness of resources and/or the information is continuous. Moreover, utilizing four data points will allow for a vigorous set of data analyses by employing a multi-level modeling tool known as growth curve analysis. The consequences of conducting fewer than four waves of data is that there would be no way to accurately assess knowledge gained over time for both campaigns. Moreover, each wave of data collection is timed to be immediately before and after a campaign thereby increasing the linkage of any knowledge change to the campaigns. A table showing the timing of data collection is below:
Data Collection Type and Timeline
March 2024 (Pre-test: CAP items) |
April 2024 (New Theme for CAP) |
May 2024 (Post-test: DVA and CAP items) |
September 2024(Pre-and Post-Posttest: DVA and CAP items) |
October 2024 (New Theme for DVA) |
November 2024 (Post-Post-Posttest: DVA and CAP items) |
|
|
November/December 2024: Conversations with FAP staff |
|||||
March 2024-October 2024: Implementation metrics collected by Clearinghouse research staff |
||||||
For information collected from FAP staff, conversations will occur at the conclusion of the campaigns in November/December 2024. These conversations will occur at one time point to reduce the burden on FAP staff. The feedback session scripts were reviewed by OSD MC&FP to ensure that feedback items were necessary for program monitoring purposes.
7. Paperwork Reduction Act Guidelines
This collection of information does not require collection to be conducted in a manner inconsistent with the guidelines delineated in 5 CFR 1320.5(d)(2).
8. Consultation and Public Comments
Part A: PUBLIC NOTICE
A 60-Day Federal Register Notice (FRN) for the collection published on Monday, April 24, 2023. The 60-Day FRN citation is 88 FR 24770.
No comments were received during the 60-Day Comment Period.
A 30-Day Federal Register Notice for the collection published on Monday, August 21, 2023. The 30-Day FRN citation is 88 FRN 56807.
Part B: CONSULTATION
No additional consultation apart from soliciting public comments through the Federal Register was conducted for this submission.
9. Gifts or Payment
To increase the likelihood of participation and to improve retention, Penn State will provide incentives after completion of each data collection instrument for those participating in the pre/posttest portion of this project. Respondents will receive an electronic Amazon gift card after completion of each instrument. They will receive a $10 gift card after completion of the first instrument, $15 after the second, $20 after the third, and $25 after completion of the fourth instrument, for a possible total of $70. These amounts were chosen to ensure that respondents were appropriately reimbursed for their time and to serve as an incentive for completion of each data collection instrument. Research finding have supported the use of monetary incentives related to participation in evaluation surveys. Incentives increase throughout the evaluation to decrease attrition. Respondents can use the gift cards to purchase basic living necessities or other items of their choosing.
10. Confidentiality
A Privacy Act Statement is not required for this collection because we are not requesting individuals to furnish personal information for a system of records.
A System of Record Notice (SORN) is not required for this collection because records are not retrievable by PII.
A Privacy Impact Assessment (PIA) is not required for this collection because PII is not being collected electronically.
Data will be retained by Penn State for a minimum of 5 years from the end of the calendar year in which the project terminates.
11. Sensitive Questions
No questions are being asked that are sensitive in terms of personal experience with domestic abuse/violence and child abuse. However, given the content of the campaigns (i.e., domestic abuse/violence and child abuse), some questions will address general attitudes toward domestic abuse/violence and child abuse prevention. Since the purpose of this project is to assess knowledge and attitudes related to campaigns focused on domestic abuse/violence and child abuse, these questions are required for the evaluation. To assess the impact of the campaigns on various individuals’ attitudes and knowledge about domestic abuse/violence and child abuse, this data is necessary. Respondents will be able to skip any questions that they may find uncomfortable.
In addition, a demographics survey will be administered as part of the wave 1 survey. The demographic variables include gender, age, race, ethnicity, level of education, relationship status, connection to the military (i.e., active duty, spouse/partner), paygrade, service branch, and tenure in the military. These demographic questions are necessary to better understand the project sample and to assess the impact of the campaigns on various individuals’ attitudes and knowledge about domestic abuse violence and child abuse prevention. To prevent identification of respondents through the aggregation of their demographic data, several strategies will be utilized. First, blurring will be used for several variables. For example, respondents will report a range when answering the items about their age and tenure in the military. In addition, identifiers collected from the gift card survey (name and email or phone number) will not be connected to the demographic data in any way. Finally, the re-identification risk of the data is very low due to the number of respondents participating in this evaluation (n = 2,400) and the number of variables. Race & ethnicity is collected in a manner consistent with OMB Statistical Policy Directive No. 15, “Race and Ethnic Standards for Federal Statistics and Administrative Reporting,” 62 FR 58782, Oct. 30, 1997.
For the conversations with FAP staff, questions will focus on campaign implementation and not on personal experiences; therefore, these conversations will not include sensitive information.
12. Respondent Burden and its Labor Costs
Part A: ESTIMATION OF RESPONDENT BURDEN
Collection Instrument(s)
FAP Outreach Effectiveness Project Survey 1 (before launch of CAP campaign)
Number of Respondents: 2,400 (150/installation)
Number of Responses Per Respondent: 1
Number of Total Annual Responses: 2,400
Response Time: 15 minutes
Respondent Burden Hours: 600 hours
FAP Outreach Effectiveness Project Survey 2 (after launch of CAP campaign)
Number of Respondents: 2,400 (150/installation)
Number of Responses Per Respondent: 1
Number of Total Annual Responses: 2,400
Response Time: 15 minutes
Respondent Burden Hours: 600 hours
FAP Outreach Effectiveness Project Survey 3 (before launch of DVA campaign)
Number of Respondents: 2,400 (150/installation)
Number of Responses Per Respondent: 1
Number of Total Annual Responses: 2,400
Response Time: 15 minutes
Respondent Burden Hours: 600 hours
FAP Outreach Effectiveness Project Survey 4 (after launch of DVA campaign)
Number of Respondents: 2,400 (150/installation)
Number of Responses Per Respondent: 1
Number of Total Annual Responses: 2,400
Response Time: 15 minutes
Respondent Burden Hours: 600 hours
FAP Staff Program Improvement Conversations
Number of Respondents: 32 (1-2/installation)
Number of Responses Per Respondent: 1
Number of Total Annual Responses: 32
Response Time: 1 hour
Respondent Burden Hours: 32 hours
Total Submission Burden
Total Number of Respondents: 9,632
Total Number of Annual Responses: 9,632
Total Respondent Burden Hours: 9,632 hours
Part B: LABOR COST OF RESPONDENT BURDEN
Collection Instrument(s)
FAP Outreach Effectiveness Project Survey 1
Number of Total Annual Responses: 2,400
Response Time: 15 minutes
Respondent Hourly Wage: $22.64
Labor Burden per Response: $5.66
Total Labor Burden: $13,584.00
FAP Outreach Effectiveness Project Survey 2
Number of Total Annual Responses: 2,400
Response Time: 15 minutes
Respondent Hourly Wage: $22.64
Labor Burden per Response: $5.66
Total Labor Burden: $13,584.00
FAP Outreach Effectiveness Project Survey 3
Number of Total Annual Responses: 2,400
Response Time: 15 minutes
Respondent Hourly Wage: $22.64
Labor Burden per Response: $5.66
Total Labor Burden: $13,584.00
FAP Outreach Effectiveness Project Survey 4
Number of Total Annual Responses: 2,400
Response Time: 15 minutes
Respondent Hourly Wage: $22.64
Labor Burden per Response: $5.66
Total Labor Burden: $13,584.00
FAP Staff Program Improvement Conversations
Number of Total Annual Responses: 32
Response Time: 1 hour
Respondent Hourly Wage: $27.40
Labor Burden per Response: $27.40
Total Labor Burden: $876.80
Overall Labor Burden
Total Number of Annual Responses: 9,632
Total Labor Burden: $55,212.80
The respondent hourly wage was determined by dividing the step one base annual salary for GS-9 (https://www.federalpay.org/gs/2022) by 52 and then by 40.
The FAP staff respondent hourly wage was determined by dividing the step one base annual salary for GS-11 by 52 and then by 40.
13. Respondent Costs Other Than Burden Hour Costs
There are no annualized costs to respondents other than the labor burden costs addressed in Section 12 of this document to complete this collection.
14. Cost to the Federal Government
Part A: LABOR COST TO THE FEDERAL GOVERNMENT
Collection Instrument(s):
Surveys 1 through 4
Number of Total Annual Responses: 9,600
Processing Time per Response: 0.15 hours
Hourly Wage of Worker(s) Processing Responses: $40.51
Cost to Process Each Response: $6.08
Total Cost to Process Responses: $58,368
FAP Staff Program Improvement Conversations
Number of Total Annual Responses: 32
Processing Time per Response: 0.15 hours
Hourly Wage of Worker(s) Processing Responses: $40.51
Cost to Process Each Response: $6.08
Total Cost to Process Responses: $194.56
Overall Labor Burden to the Federal Government
Total Number of Annual Responses: 9,632
Total Labor Burden: $58,562.56
Part B: OPERATIONAL AND MAINTENANCE COSTS
Cost Categories
Equipment: $0
Printing: $0
Postage: $0
Software Purchases: $0
Licensing Costs: $0
Other: $315,000 (Grant cost includes operation costs)
Total Operational and Maintenance Cost: $315,000
Part C: TOTAL COST TO THE FEDERAL GOVERNMENT
Total Labor Cost to the Federal Government: $58,562.56
Total Operational and Maintenance Costs: $315,000
Total Cost to the Federal Government: $373,562.56
15. Reasons for Change in Burden
This is a new collection with a new associated burden.
16. Publication of Results
Penn State will complete and submit a report to DoD in June 2025. The report is being completed to provide information to DoD FAP about the effectiveness of awareness campaigns and to provide guidance as recommended refinements to increase campaigns effectiveness.
17. Non-Display of OMB Expiration Date
We are not seeking approval to omit the display of the expiration date of the OMB approval on the collection instrument.
18. Exceptions to “Certification for Paperwork Reduction Submissions”
We are not requesting any exemptions to the provisions stated in 5 CFR 1320.9.
| File Type | application/vnd.openxmlformats-officedocument.wordprocessingml.document |
| Author | Kaitlin Chiarelli |
| File Modified | 0000-00-00 |
| File Created | 2023-09-29 |