Supporting Statement A 1660-0139 OCC edits (003) (002) (a.carter)

January 25, 2018

Supporting Statement for

Paperwork Reduction Act Submissions

OMB Control Number: 1660 – 0139

Title: Ready PSA Campaign Creative Testing Research

Form Number(s): FEMA Form 008-0-21; FEMA Form 008-0-22

A. Justification

1. Explain the circumstances that make the collection of information necessary.

Identify any legal or administrative requirements that necessitate the collection. Attach a copy of the appropriate section of each statute and regulation mandating or authorizing the collection of information. Provide a detailed description of the nature and source of the information to be collected.

This collection is in accordance with Executive Orders 12862 (http://www.archives.gov/federal-register/executive-orders/pdf/12862.pdf) and 13571 (http://www.gpo.gov/fdsys/pkg/FR-2011-05-02/pdf/2011-10732.pdf) requiring all Federal agencies to survey customers to determine the kind and quality of services they want and their level of satisfaction with existing services. The Government Performance and Results Act (GPRA) ( https://obamawhitehouse.archives.gov/omb/mgmt-gpra/gplaw2m )

requires Federal agencies to set missions and goals and to measure agency performance against them. The GPRA Modernization Act of 2010 requires quarterly performance assessments of government programs for the purposes of assessing agency performance and improvement. The Federal Emergency Management Agency is collecting information through focus groups to improve its public service advertising campaign on disaster preparedness.

2. Indicate how, by whom, and for what purpose the information is to be used. Except for a new collection, indicate the actual use the agency has made of the information received from the current collection. Provide a detailed description of: how the information will be shared, if applicable, and for what programmatic purpose.

FEMA’s mission is to support our citizens and first responders to ensure that as a nation we work together to build, sustain, and improve our capability to prepare for, protect against, respond to, recover from, and mitigate all hazards. Launched in February 2003, the Ready campaign is a national public service advertising (PSA) campaign that works in support of this mission and is designed to educate and empower Americans to prepare for and respond to emergencies including natural and man-made disasters. In support of this mission, FEMA is requesting clearance to conduct information collections in the form of Focus Group Studies over the next three years. This information collection is intended to assess the target audience’s reaction to the creative concepts developed for the Ready PSA campaign and to determine the clarity, relevance, and motivating appeal of the concepts prior to final production of the advertising – for a multi-media national advertising campaign that launches new PSAs each September. The Ready PSA campaign targets parents and guardians, age 25-54, who have children age 6-17 years old in the household. The campaign will include materials in English and Spanish; thus, focus groups are planned among both English and Spanish speaking parents and guardians.

All creative concepts will be presented in rough, unfinished formats because the research is conducted prior to producing the PSAs. The agency will recommend the format for each media type, which will be shared with FEMA prior to the research. This information collection will be used to determine whether the advertising will be effective and motivating to elicit the intended change in behavior and will help determine whether revisions should be incorporated into the creative concepts prior to final production. Focus groups are an important information gathering technique because they allow for more in-depth feedback from parents and guardians than other types of studies. Focus groups serve the narrowly defined need for direct and informal opinion on a specific topic.

Specifically, the key objectives of creative message testing are to determine:

• How well the target audience understands the concepts

• The overall appeal of the concepts

• The strengths and weaknesses of the concepts

• The relevancy of the concepts to the target audience

• How motivating the concepts are to the target audience to follow through on the call-to-action, or main “ask” of the advertising

For this specific information collection, focus groups will be used as a qualitative research tool to better understand parents’ and guardians’ attitudes and emotions in response to creative concepts designed to communicate information about preparing for emergencies.

FEMA Form 008-0-21, Recruitment Screener (script) – will be used to survey all potential respondents and ensure the individuals qualify to participate.

FEMA Form 008-0-22, Focus Group Discussion Guide – will be used by the moderator to guide the focus group discussion and ensure the necessary information is collected.

Focus groups do not yield meaningful quantitative findings. They can provide public input, but they do not yield data about public opinion that can be generalized to the population at large. As such, they cannot be used to drive the development of policies, programs, and services.

3. Describe whether, and to what extent, the collection of information involves the use of automated, electronic, mechanical, or other technological collection techniques or other forms of information technology, e.g., permitting electronic submission of responses, and the basis for the decision for adopting this means of collection. Also describe any consideration of using information technology to reduce burden.

Focus group studies are directed group discussions that enable skilled observers to infer the underlying views and assumptions of the group. To facilitate interpretation, the focus group discussions are recorded and videotaped so that both a visual record and written transcript of the discussion are available for review.

4. Describe efforts to identify duplication. Show specifically why any similar information already available cannot be used or modified for use for the purposes described in Item 2 above. 

This information is not collected in any form, and therefore is not duplicated elsewhere.

5. If the collection of information impacts small businesses or other small entities (Item 5 of OMB Form 83-I), describe any methods used to minimize.

This information collection does not have an impact on small businesses or other small entities.

6. Describe the consequence to Federal/FEMA program or policy activities if the collection of information is not conducted, or is conducted less frequently as well as any technical or legal obstacles to reducing burden.

If this information is not collected, a vital link in gathering information by FEMA to develop appropriate creative messages will be missed.

Without examining the reactions of parents and guardians to various creative approaches, FEMA cannot adequately determine which approaches are most likely to resonate with these audiences and to ultimately increase preparedness behavior in the United States. Without the information obtained from this data collection, FEMA may make assumptions about attitudes, knowledge, and understanding of proposed messages that are not accurate or useful. The agency could potentially waste time and resources pursuing ineffective approaches.

7. Explain any special circumstances that would cause an information collection to be conducted in a manner:

1. Requiring respondents to report information to the agency more often than quarterly.

 (b) Requiring respondents to prepare a written response to a collection of information in fewer than 30 days after receipt of it.

3. Requiring respondents to submit more than an original and two

copies of any document.

3. Requiring respondents to retain records, other than health,

medical, government contract, grant-in-aid, or tax records for more than three years.

3. In connection with a statistical survey, that is not designed to

produce valid and reliable results that can be generalized to the universe of study.

 (f) Requiring the use of a statistical data classification that has not

been reviewed and approved by OMB.

 (g) That includes a pledge of confidentiality that is not supported by authority established in statute or regulation, that is not supported by disclosure and data security policies that are consistent with the pledge, or which unnecessarily impedes sharing of data with other agencies for compatible confidential use.

 (h) Requiring respondents to submit proprietary trade secret, or other confidential information unless the agency can demonstrate that it has instituted procedures to protect the information’s confidentiality to the extent permitted by law.

This information collection is conducted in a manner consistent with the guidelines in 5 CFR 1320.5(d)(2).

8. Federal Register Notice: 

 a. Provide a copy and identify the date and page number of publication in the Federal Register of the agency’s notice soliciting comments on the information collection prior to submission to OMB. Summarize public comments received in response to that notice and describe actions taken by the agency in response to these comments. Specifically address comments received on cost and hour burden.

A 60-day Federal Register Notice inviting public comments was published on March 14, 2018, 83 FR 1 comment related to the information collection was received. It states, “Whereas the abstract states, ‘FEMA proposes conducting qualitative research in the form of focus groups in order to test creative concepts developed for FEMA's national Ready public service advertising campaign, which aims to educate and empower Americans to prepare for and respond to emergencies. The research will help determine the clarity, relevance, and motivating appeal of the concepts prior to final production of the advertising.’ It does not state research will include the effectiveness of the PSA. Clearly it is difficult for metrics to be measured because they are not an exact science, and there is no way to measure success. What can be measured are ‘results’. Therefore, PSAs should be written with the level of creativity where they generate results, then results or the effectiveness of the PSA can be measured. With no means of measurement stated in the abstract, it seems like determining if there is actual ‘qualitative testing’ will not be captured.” This is qualitative research. The discussion guide will be used to facilitate discussion around the creative PSA product developed and to identify patterns in response to determine if the concepts are relevant, deliver the message clearly, and have potential to motivate behavior change. The results of the advertising will be measured through different methods and is not included as part of this data collection.

A 30-day Federal Register Notice inviting public comments was published on October 15, 2018, 83 FR 51976 No comments were received.

 b. Describe efforts to consult with persons outside the agency to obtain their views on the availability of data, frequency of collection, the clarity of instructions and recordkeeping, disclosure, or reporting format (if any), and on the data elements to be recorded, disclosed, or reported.

FEMA has reviewed reports on emergency preparedness and other qualitative information collections to identify areas of interest and concern. FEMA used experienced contractors to develop focus group plans and materials.

c. Describe consultations with representatives of those from whom information is to be obtained or those who must compile records. Consultation should occur at least once every three years, even if the collection of information activities is the same as in prior periods. There may be circumstances that may preclude consultation in a specific situation. These circumstances should be explained.

The same type of research was conducted with parents and guardians in 2014; based on the previous project, no major revisions were deemed necessary.

9. Explain any decision to provide any payment or gift to respondents, other than remuneration of contractors or grantees.

It is standard practice to reimburse focus group respondents for their time. Each respondent will be provided with $75 following their participation in a session. This amount is in line with the industry standard and will avoid bias of receiving responses only from individuals generally predisposed to be helpful.

The full justification for seeking $75 incentive per respondent is provided as an attachment.

10. Describe any assurance of confidentiality provided to respondents. Present the basis for the assurance in statute, regulation, or agency policy.

A Privacy Threshold Analysis (PTA) was approved on January 17, 2018. A Privacy Impact Analysis for this collection was not determined to be needed. 

11. Provide additional justification for any question of a sensitive nature (such as sexual behavior and attitudes, religious beliefs and other matters that are commonly considered private). This justification should include the reasons why the agency considers the questions necessary, the specific uses to be made of the information, the explanation to be given to persons from whom the information is requested, and any steps to be taken to obtain their consent.

There are no questions of a sensitive nature. 

 12. Provide estimates of the hour burden of the collection of information. The statement should:

FEMA plans to conduct a total of ten focus groups, each lasting 75 minutes. Each group will consist of 4 participants. For recruiting these participants, a total of 50 respondents (5 per group) will be recruited via dialed telephone screening calls, which are estimated to take no more than 10 minutes (.1667 hour) per response. Based on experience, it is prudent to recruit up to 5 individuals per group in order to help ensure at least 4 will actually appear at the research facility at the appointed time. All participants will be either a parent or guardian of a child age 6-17. Thus, the total annual estimated burden imposed by this collection of information is approximately 58 hours. This accounts for 8 hours of screening calls (50 participants x 10 minutes (.1667)) and 50 hours of focus group discussions (40 participants x 75 minutes (1.25)). This information collection will only comprise 50 respondents in total since the 40 focus group participants will also be included in the screening calls.

Estimated Annualized Burden Hours and Costs
Type of Respondent	Form Name / Form Number	No. of Respon-dents	No. of Respon-ses per Respon-dent	Total No. of Responses	Avg. Burden per Response (in hours)	Total Annual Burden (in hours)	Avg. Hourly Wage Rate	Total Annual Respondent Cost
Individuals or Households	Recruitment Screener (survey script) / FEMA Form 008-0-21	50	1	50	0.1667 (10 minutes)	8	$35.52	$284.16
Individuals or Households	Focus Group Discussion Guide / FEMA Form 008-0-22	40	1	40	1.25 hours (75 minutes)	50	$35.52	$1,776
Total		90		90		58		$2,060.16

• Note: The “Avg. Hourly Wage Rate” for each respondent includes a 1.46 multiplier to reflect a fully-loaded wage rate.

Therefore, the total annual estimated burden imposed by this collection of information is approximately 58 hours annually. According to the U.S. Department of Labor, Bureau of Labor Statistics website (www.bls.gov) the wage rate category for all occupations is estimated to be $35.52 per hour including the wage rate multiplier, therefore, the estimated burden hour cost to respondents all occupations is estimated to be $2,060.16 annually. This is calculated in the following equation:

$24.33per hour¹ x 1.46² multiplier = $35.52 x 58 interviewing hours = $2,060.16

13. Provide an estimate of the total annual cost burden to respondents or recordkeepers resulting from the collection of information. The cost of purchasing or contracting out information collection services should be a part of this cost burden estimate. (Do not include the cost of any hour burden shown in Items 12 and 14.)

There are no record keeping, capital, start-up or maintenance costs associated with this information collection.

14. Provide estimates of annualized cost to the federal government. Also, provide a description of the method used to estimate cost, which should include quantification of hours, operational expenses (such as equipment, overhead, printing and support staff), and any other expense that would have been incurred without this collection of information. You may also aggregate cost estimates for Items 12, 13, and 14 in a single table.

Annual Cost to the Federal Government

Item	Cost ($)
Contract Costs [Describe]
Staff Salaries* [salary for attending the focus groups and reviewing output] Total: 45 hours 1 GS 13, Step 13 -- 23 hours x $52.91 ($36.24 hourly rate x 1.46 = 52.91) =   $ 1,216.93 1 GS 15 Step 1 -- 22 hours x $73.54 ($50.37 hourly rate x 1.46 = $73.54 hourly rate) =   $ 1,617.88 $1,216.93 + $1,617.88 = $2,834.81	$2,834.81
Facilities [cost for renting, overhead, etc. for data collection activity]	45,000
Computer Hardware and Software [cost of equipment annual lifecycle]
Equipment Maintenance [cost of annual maintenance/service agreements for equipment]
Travel	5,000
Printing [number of data collection instruments annually]
Postage [annual number of data collection instruments x postage]
Other
Total	$52,834.81

* Note: The “Salary Rate” includes a 1.46 multiplier to reflect a fully-loaded wage rate.

The Agency incurs costs to set up the focus groups including hiring the contractor (facilitator or moderator), renting meeting space, recruitment of respondents, data collection, reporting, travel and subsistence and the payment of a de minimis cost in the form of a token stipend. For these expenses, FEMA estimates the costs to be approximately $52,834.81 annually.

15. Explain the reasons for any program changes or adjustments reported in Items 13 or 14 of the OMB Form 83-I in a narrative form. Present the itemized changes in hour burden and cost burden according to program changes or adjustments in Table 5. Denote a program increase as a positive number, and a program decrease as a negative number.

A "Program increase" is an additional burden resulting from a federal government regulatory action or directive (e.g., an increase in sample size or coverage, amount of information, reporting frequency, or expanded use of an existing form). This also includes previously in-use and unapproved information collections discovered during the ICB process, or during the fiscal year, which will be in use during the next fiscal year.

A "Program decrease" is a reduction in burden because of: (1) the discontinuation of an information collection; or (2) a change in an existing information collection by a Federal agency (e.g., the use of sampling (or smaller samples), a decrease in the amount of information requested (fewer questions), or a decrease in reporting frequency).

"Adjustment" denotes a change in burden hours due to factors over which the government has no control, such as population growth, or in factors which do not affect what information the government collects or changes in the methods used to estimate burden or correction of errors in burden estimates.

Itemized Changes in Annual Burden Hours
Data collection Activity/Instrument	Program Change (hours currently on OMB Inventory)	Program Change (New)	Difference	Adjustment (hours currently on OMB Inventory)	Adjustment (New)	Difference
Recruitment Screener (survey script) / FEMA Form 008-0-21	0
Focus Group Discussion Guide / FEMA Form 008-0-22	0
Total(s)	0

Explain:

There is no change in the burden hours, and a decrease in cost to the Federal Government.

16. For collections of information whose results will be published, outline plans for tabulation and publication. Address any complex analytical techniques that will be used. Provide the time schedule for the entire project, including beginning and ending dates of the collection of information, completion of report, publication dates, and other actions.

FEMA does not intend to employ the use of statistics or the publication thereof for this information collection.

17. If seeking approval not to display the expiration date for OMB approval of the information collection, explain reasons that display would be inappropriate.

FEMA will display the expiration date for OMB approval of this information collection.

18. Explain each exception to the certification statement identified in Item 19 “Certification for Paperwork Reduction Act Submissions,” of OMB Form 83-I.

FEMA does not request an exception to the certification of this information collection.

B. Collections of Information Employing Statistical Methods.



There is no statistical methodology involved in this collection.

1 See U.S. Department of Labor, Bureau of Labor Statistics, “Employer Cost for Employee Compensation —September 2017,” located at: https://www.bls.gov/news.release/pdf/ecec.pdf.

2 Calculated by dividing total compensation for all workers of $35.52 by wages and salaries for all workers of $24.33 per hour (yields a benefits multiplier of approximately 1.46 x wages).

3 U.S. Office of Personnel Management, “Salary Table 2018—GS,” January 2018, located at: https://www.opm.gov/policy-data-oversight/pay-leave/salaries-wages/salary-tables/pdf/2018/GS_h.pdf.

7