Focus Group Supporting Statement B

Focus Group Supporting Statement B.docx

Focus Groups and Usability Study

OMB: 0430-0005

Document [docx]
Download: docx | pdf

Supporting Statement for Paperwork Reduction Act Submission


Certification Statement for Recovery Accountability and Transparency Board


B. Collections of Information Employing Statistical Methods


The agency should be prepared to justify its decision not to use statistical methods in any case where such methods might reduce burden or improve accuracy of results. When the question “Does this ICR contain surveys, censuses or employ statistical methods” is checked, "Yes," the following documentation should be included in the Supporting Statement to the extent that it applies to the methods proposed:


1. Describe (including a numerical estimate) the potential respondent universe and any sampling or other respondent selection methods to be used. Data on the number of entities (e.g., establishments, State and local government units, households, or persons) in the universe covered by the collection and in the corresponding sample are to be provided in tabular form for the universe as a whole and for each of the strata in the proposed sample. Indicate expected response rates for the collection as a whole. If the collection had been conducted previously, include the actual response rate achieved during the last collection.


Total focus group participants: 72 (18 people in each of 4 markets)



Participants will be average American citizens that provide a diverse community based on the following criteria:

  • Registered voter

  • Government-related activities

  • Awareness of ARRA

  • Internet usage

  • Current work status/occupation

  • Level of education

  • Ethnicity

  • Household income


Reference Attachment A: Participant Screener


2. Describe the procedures for the collection of information including:

* Statistical methodology for stratification and sample selection,

* Estimation procedure,

* Degree of accuracy needed for the purpose described in the justification,

* Unusual problems requiring specialized sampling procedures, and

* Any use of periodic (less frequent than annual) data collection cycles to reduce burden.


Discovery: Focus Groups


To “field” test Recovery.gov enhancements, contractor will conduct qualitative research, specifically focus groups, representative of key user populations. Sufficient focus groups will help us engage audiences in a safe (non-threatening) environment, where thorough dialogue, group dynamics and observation of tasks, we can understand how these individuals, from specific audience profiles, might use the site to meet their needs.


The facilitated focus groups will provide probing opportunities so we can truly understand the “why” behind participant behaviors (actions) in the course of seeking information on the site as well as perceptions of the site. We believe that this approach can validate the design concepts and content while gathering any constructive feedback to complete site implementation in the most timely and efficient manner.


We recommend conducting 8 in-person focus groups in 4 geographically representative regions: Northeast, Southeast, Midwest, and West.


Reference Attachment B: Usability Testing Task List


3. Describe methods to maximize response rates and to deal with issues of non-response. The accuracy and reliability of information collected must be shown to be adequate for intended uses. For collections based on sampling, a special justification must be provided for any collection that will not yield "reliable" data that can be generalized to the universe studied.


A participant screener (Attachment A) will be developed to ensure focus group participants are within the specified audiences of concern for site success. Participants, except where not allowed for legal or ethical reasons, will receive an industry standard stipend of $100 each, delivered by check upon completion of the focus group or usability testing session.


In addition to general diversity in the markets, locations for the research were selected based on:

  • Metropolitan Statistical Area: mix of metro, suburban, rural

  • Range of unemployment rates


Market

Unemployment Rate

MSA Rank

BALTMORE

7.2

20

MEMPHIS

10.40

50

DETROIT

14.8

41

PHOENIX

8.4

12



4. Describe any tests of procedures or methods to be undertaken. Testing is encouraged as an effective means of refining collections of information to minimize burden and improve utility. Tests must be approved if they call for answers to identical questions from 10 or more respondents. A proposed test or set of test may be submitted for approval separately or in combination with the main collection of information.


Focus groups will be a two hour guided session that follows a pre-determined discussion guide. The recommended discussion guide is Attachment B.



5. Provide the name and telephone number of individuals consulted on statistical aspects of the design and the name of the agency unit, contractor(s), grantee(s), or other person(s) who will actually collect and/or analyze the information for the agency.


Rob Groat

Chief Technology Officer

Office of the CTO

Smartronix, Inc

301.535.7703


Lindsay Wozniak

Senior Vice President

TMP Government

703.269.0095













Attachment A: Participant Screener


Record Gender: recruit a mix

Male

Female


NAME
Shape1

ADDRESS

Shape2

CITY/STATE/ZIP
Shape3

HOME PHONE WORK PHONE
Shape4

FAX E-MAIL
Shape5

RECRUITED BY DATE
Shape6

CONFIRMED BY DATE
Shape7


PROJECT OVERVIEW

  • Recruit 13 for at least 9-10 to participate per group

  • Represent a mix of ethnicity, employment status, gender, age, education level, U.S. born vs. foreign born, and English as primary/secondary languages.

  • Phase 1 Focus Groups will focus specifically on Average Citizens and Interested Citizens

  • Ensure 508 compliance representation




Proposed Research Locations/Schedule

Locations recommended based on the following:

  • Media market population: mix of metro, suburban and rural areas (based on Metropolitan Statistical Area rankings)

  • Diversity in markets: gender, ethnicity, disabled residents

  • Range of unemployment rates: high, low (based on BLS June 2009 data)


Day

Date

Region

Times

Market

Unemployment Rate

MSA Rank

Thursday

July 8

Northeast

5:30 and 7:30 p.m.

BALTMORE

7.2

20

Monday

July 12

Southeast

5:30 and 7:30 p.m.

MEMPHIS

10.40

50

Tuesday

July 13

Midwest

5:30 and 7:30 p.m.

DETROIT

14.8

41

Wednesday

July 14

West

5:30 and 7:30 p.m.

PHOENIX

8.4

12


Target Audience

  • Group 1: Average citizens/Interested Citizens

  • Group 2: Potential funding recipients/Recipients/Citizens that use Recovery.gov in a professional capacity.


And represent a mix of ethnicity, employment status, gender, age, education level, U.S. born vs. foreign born and English as primary/secondary languages.


Recruiting notes:


Since we are testing creative, we will not be able to include any no/low vision 508 compliance respondents


Shape8



Hello, my name is _________________from C2 Consumer Research. I am calling to invite you to join a paid market research study to discuss government websites. We will pay you $100 to simply share your thoughts and opinions for a little less than 2 hours of your time. This is not a sales presentation or recruiting event. It is merely for market research purposes. We have a number of evening sessions available based on your interests.


VM: If interested or would like to get more information please call us at (877) 222-2909 Ext. xx again that’s (877) 222-2909 Ext. xx.


Live: Does this sound like something you would be interested in participating in? (if yes) Is this a convenient time to ask you a few questions?


a. Yes 1 CONTINUE

b. No 2 ESTABLISH CONVENIENT TIME

TO CALL BACK


1. When was the last time, if ever, you participated in a market research study either at our facility or for another research company? (DO NOT READ)


a. Within the past 6 months 1 TERMINATE

b. Longer than 6 months ago 2 ASK Q2a/2b

c. Never 3 SKIP TO Q3


If within the past 6 months, terminate


2a. How many market research group discussions have you ever participated in?


2b. How many market research one-on-one, in-depth interviews have you ever participated in?


No more than 3 focus groups or one-on-one interviews ever


3. Which of the following categories includes your age? (READ) Recruit a mix.


a. Under 18 1 TERMINATE

b. 18-24 2 CONTINUE

c. 25-34 3 CONTINUE

d. 35-45 4 CONTINUE

e. 45-54 5 CONTINUE

f. 55-64 6 CONTINUE

g. 65+ 7 CONTINUE



4. Which of the following best describes your current residency status?

a. I am a U.S. Citizen 1 CONTINUE

b. I am a Naturalized U.S. Citizen 2 CONTINUE

c. I am not a U.S. Citizen 3 TERMINATE


5. What languages are spoken in your home? ___________________ Recruit a mix of English only and bilingual households


6. On a scale from 1 to 10, where 1= not very interested and 10= very interested, how interested are you in the following subjects?

a. Professional Sports (such as football, baseball, basketball, racing)

b. Fashion & Beauty

c. Internet and/or social networking websites (such as Facebook, Twitter, YouTube)

d. Travel

e. Gardening/Home Decoration/Architecture

f. Politics and government affairs


7. Are you a registered voter?


a. Yes 1 CONTINUE

b. No 2 CONTINUE





8. Which of the following activities have you ever participated in? Again, I want to stress that we are looking for a wide array of Americans so please be as honest as possible.


(Note to recruiter: If A-E—may qualify for Potential funding recipients, Recipients, Interested Citizens quotas. If F—Continue for Average Citizen Quota



a. I voted in the last General Election (November 2008)

b. I follow local, state and national politics closely

c. I have contacted an elected official regarding my thoughts on current events and government policies

d. I have searched the internet/watched TV/listened to the radio/read the newspaper/journals for more information on current events and government policies

e. I have campaigned/volunteered/actively supported initiatives of personal interest

f. None of the above—categorize as Average Citizen


9. Please share an example of your level of inquiry/interests_________________________



10. Are you aware of the American Recovery and Reinvestment Act of 2009 often referred to as the economic stimulus package or Recovery Act?


a. Yes 1 CONTINUE

b. No 2 CONTINUE


10b. What is it? ___________________________________________________________



11A. Do you use the internet?


a. Yes 1 CONTINUE

b. No 2 CONTINUE, SKIP TO 12


11B. If Yes, where do you access the internet?


a. Work 1 CONTINUE

b. Home 2 CONTINUE

c. Library 3 CONTINUE

d. Pay per use (e.g. internet café) 4 CONTINUE



12. On a scale from 1 to 10, where 1=not very comfortable and 10=very comfortable, how comfortable are you with using the Internet? _____________________ (Recruit a mix.)



12B. Which of the following government websites have you ever visited?


a. IRS.gov 1 CONTINUE

b. WhiteHouse.gov 2 CONTINUE

c. Recovery.gov 3 CONTINUE

d. Weather.gov 4 CONTINUE

e. Census2010.gov 5 CONTINUE

f. Other (specify) 6 CONTINUE




The following questions are intended to ensure we get a wide variety of respondents represented in our research study


Note to recruiters: Targets to include in the focus groups

  • Group 1: Average citizens and Interested Citizens

  • Group 2: Recipients and Potential funding recipients


13. Which of the following best describes your current work status or occupation? (Read. Recruit a mix.)


  1. Professional/Executive/Owner

  2. White Collar/ Administrative/ Clerical

  3. Blue Collar/ Service/ Manufacturing 

  4. Part Time Work Outside of the Home

  5. Stay at Home Full Time/Do Not Work

  6. Unemployed/Looking for Work

  7. Student

  8. Retired

  9. Other_____ (Record verbatim, do not read.)

  10. Do Not Know (Do not read.)

  11. Refused (Do not read.)


If A-D ask:


13a. What is your current job title? _________________________________


13b. What is the name of your current employer? _________________________________


14. Have you ever researched or applied for government funding (e.g. grants, contracts or loans) on behalf of your organization?

  1. Yes

  2. No, I do not foresee the need to apply

  3. No but we have considered it—continue for potential funding recipient


15. Did your organization receive government funding/grants/loans/contracts?

  1. Yes—Classify as Funding Recipient

  2. No—Classify as Potential Funding Recipient


The following questions are merely for classification purposes only.


16. Which of the following best describes your last level of education completed? (READ. Data collection purposes only.)


a. No high school diploma 1 CONTINUE

b. High School 2 CONTINUE

c. Associate’s Degree 3 CONTINUE

d. Bachelor’s Degree 4 CONTINUE

e. Masters Degree 5 CONTINUE

f. JD/PhD/MD 6 CONTINUE

g. Post Doctorate 7 CONTINUE


17. Which of the following best describes your ethnicity? (READ. Recruit a mix.)


a. Black/African American 1 CONTINUE

b. Asian 2 CONTINUE

c. White 3 CONTINUE

d. Native Hawaiian or other Pacific Islander 4 CONTINUE

e. Hispanic/Latino 5 CONTINUE

f. American Indian or Alaska Native 6 CONTINUE

g. Other 7 CONTINUE

h. Decline to state 8 CONTINUE


18. For classification purposes only, please tell me into which of the following categories your TOTAL HOUSEHOLD income falls before taxes? (READ LIST.)


a. Under $35,000 1 CONTINUE

b. Between $35,000 and $45,000 2 CONTINUE

c. Between $45,000 and $60,000 3 CONTINUE

d. Between $60,000 and $80,000 4 CONTINUE

e. Between $80,000 and $100,000 5 CONTINUE

f. Over $100,000 6 CONTINUE

g. Refused 7 CONTINUE


19. To ensure our research program is compliant with Section 508 of the Rehabilitation Act and accessible to all segments of the US population, please indicate which of the following applies to you:


(If possible—low/no vision respondents may not be possible for in-person creative concept testing)


a. No vision 1 CONTINUE

b. Low vision 2 CONTINUE

c. Hearing impaired 3 CONTINUE

d. Motor impairments 4 CONTINUE

e. Speech impairment 5 CONTINUE

f. Spinal impairment 6 CONTINUE

g. None of the above 7 CONTINUE



INVITATION

We would like to invite you to participate in a paid market research study to discuss government websites. The focus group is scheduled to last less than 2 hours and will take place on [insert date based on market].


Are you interested in participating in the research discussion?


a. Yes CONTINUE

b. No TERMINATE


That is great, the session will last approximately 2 hours. For your participation we will pay you $xxx. Does this sound like something you would be interested in participating in?


We are investing a great deal of time and effort to coordinate this research study. Your participation and attendance is vital to the success of our study. If for any reason, you are unable to attend the focus group, please notify us in advance so we may find a replacement.


Get mailing address, fax number or email address to send confirmation information.

We are counting on your attendance.  Please do not schedule yourself for this research if you feel that you are not going to be able to attend the focus group discussion. If an emergency does arise, it is imperative that you call us immediately at (877) 222-2909 Ext. xx as we will need to find your replacement as soon as possible. Since our clients expect our respondents to all show up for their research, and since it is necessary that we provide them with the most accurate information, you will be receiving 2 calls confirming your attendance. The first call will be 3-4 days after you are recruited and we will need to re-ask you some of the screening questions, you will also receive a final call to confirm the day before your scheduled appointment. We do need to speak with you on each of these occasions, and we appreciate your prompt return calls.  

Do you have any objection to us placing the 2 calls to confirm your information and attendance?

a. Yes TERMINATE

b. No CONTINUE

You will also be videotaped for market research purposes only during this discussion. Do you have any objection to the use of video?

a. Yes TERMINATE

b. No CONTINUE

If you wear glasses please be certain to bring with you to the session.

Thank you for your understanding and patience during these confirmation calls, and we look forward to having you participate in this study!




Attachment B: Focus Group Discussion Guide


Note to moderator: GROUP MAY NOT EXCEED 2 HOURS IN LENGTH. PLEASE MANAGE TIME ACCORDINGLY.


Introduction (10 minutes)


Hello my name is ____. I want to start off by letting you know that I am an independent moderator. What that means is that I do not work for [insert name of focus group facility]. I am not looking for you to say one thing or feel another. In fact, I am just here to guide the conversation along. There are no right or wrong answers.


I would also like to point out a few things before we get started. I am video taping this group so I could focus on our conversation and I do have some of my colleagues behind the glass taking notes for me. So please feel free to be candid as your individual information will be kept confidential as it relates to this project.


With regard to cell phones - If you have a cell phone with you tonight, please turn it to vibrate so it does not disrupt the flow of our conversation today.

Let’s start off by going around the room. Please introduce yourself, where you are originally from and how long you have lived in [insert city].



Section I: Awareness/Opinions of American Recovery and Reinvestment Act of 2009


But before we get to that point I would like to learn more about your thoughts on the current state of the economy.


Discuss what they know about the economy stimulus and road to recovery.


What do you know about the $787 billion stimulus package? What types of programs or projects is it funding?


What resources do you currently use/or would you use to learn more about the recovery process and allocation of the $787 billion stimulus package? (Create list on flip chart. See if Recovery.gov is mentioned on an unaided basis.)


If not mentioned, how many of you are aware of a website called Recovery.gov?


Section II. Design the Recovery.gov website



Transition: We are conducting this research to better assess what folks just like yourself might like to find on a website called Recovery.gov.


Read description about Recovery.gov: Recovery.gov is the U.S. government’s official Web site providing easy access to data related to Recovery Act spending and is a forum for reporting fraud, waste, and abuse.




Section III: Reveal Enhancements to Recovery.gov

Bring up Recovery.gov/ on a large television monitor.


Conduct an ad-hoc usability study critique of the existing site

Ask respondents to write down their initial response to the homepage. Where would they click first?

Probe all elements of a traditional usability test including homepage, content, navigation, nomenclature.

  • Basic user information location and retrieving

  • Ask respondents to look at each content category/label and supporting topics

  • Discuss if headers/labels and supporting content is clear/intuitive

  • Top-line navigation

  • Aesthetics

  • Economy, i.e. activity was worth the effort for reward achieved

  • Total site architecture

  • Data visualization components


Assess ways to improve the site.



Section V. Conclusion

In closing, please offer one piece of advice to improve Recovery.gov.


Thank respondents again for their time and participation. Please ask them to leave their paperwork behind.

23


File Typeapplication/vnd.openxmlformats-officedocument.wordprocessingml.document
Authorivan.flores
File Modified0000-00-00
File Created2021-02-02

© 2024 OMB.report | Privacy Policy