2017-06-06 SS 1545-1349GENr

SUPPORTING STATEMENT

OMB Control No. 1545-1349

INTERNAL REVENUE SERVICE

STATISTICS OF INCOME

Generic Clearance Approval Request for Cognitive and Psychological Research

(July 2017 through June 2020)

Abstract

This is a request for generic clearance by the Internal Revenue Service (IRS), Statistics of Income (SOI) Division, coordinating with Wage and Investment (W&I), Small Business/Self Employed (SB/SE), Large Business and International (LB&I) and Tax-Exempt and Government Entities (TE/GE) Divisions to conduct research. The research will result in making recommendations for positive changes to current tax forms and customer service procedures. These changes will represent improvements aimed at easing the taxpayer burden in understanding and working with the forms, contributing to a simplification of the processing of the forms within the IRS processing environment, and improving and modernizing customer service avenues. The research and development activities in the forms improvement process will be conducted by external contractors employing state-of-the-art cognitive psychological testing methods. We have demonstrated, by extensive applications over the past decade, the feasibility and value of this approach to questionnaire construction, survey technology, and interview processes. Previous experience in applying these techniques has resulted in recommended improvements to IRS forms and publications, filing requirements, and electronic filing. The planned research and development activities will be conducted during FY 2017 through FY 2020 with the goal of making continued improvements to tax forms and new technologies.

A. Justification

1. Circumstances Necessitating Collection of Information

Currently SOI in collaboration with other IRS organizations along with external contractors conduct cognitive psychological research in design and execution of the data collection process in order to improve the quality of tax forms and taxpayer products and services.

SOI works closely with Wage and Investment (W&I), Small Business/Self Employed (SB/SE), Large Business and International (LB&I), and Tax-Exempt and Government Entities (TE/GE). Wage and Investment is responsible for monitoring the accuracy of the responses given to the general public who call IRS for answers to tax questions or visit IRS assistance centers. Tax Forms and Publications division creates the forms used by taxpayers for tax purposes. Information Technology (IT) is responsible for design and implementation of many new interactive systems and technologies that are being developed as part of the IRS modernization effort. W&I, LB&I, SB/SE, and TEGE conduct a broad range of studies and tests that focus on taxpayer needs.

The purpose of this request for clearance for cognitive psychological research and development activities by IRS working with external contractors is to continue to enhance the quality of the information collected using tax forms and to provide design and implementation testing support to new IRS service delivery methods and technologies. Over the past three years, studies such as the Power of Attorney Notices Focus Groups, Filing and Payment Status Focus Groups, Forms 8863 and 1098T Interviews, Consumer Tipping Usability Study, and Reporting Compliance have proven successful in determining barriers to use, product opinion, and associated burden.

The research techniques and methods to be used in these studies will include analysis of form design as well as survey technology. Analyses will be conducted within the following domains:

a. Question Analysis -- Evaluation of individual forms, survey questions, and scripts appraising line item intention, assessment of semantic clarity, and an examination of relations between line items.

b. Term Analysis -- Evaluation of specific wording and phrases in terms of their psycholinguistic properties and an assessment of respondent interpretation of the meaning of these terms, at both the conscious and unconscious levels.

c. Instruction Analysis -- Inspection of instructions for their semantic clarity, the degree to which they reflect the stated intention, ease of interpretation, and other considerations which may elicit unambiguous and appropriate answers from taxpayers.

d. Format Analysis -- Review of tax forms for perceptual characteristics in order to facilitate better respondent comprehension and promote more active attention to the focus of the form.

e. Task Analysis – Evaluation of users’ ability to complete tasks using online tools and systems.

2. Use of Data

The purpose of IRS data collection is to improve both the theoretical understanding of the data collection process and the practical applications of such knowledge especially as it pertains to emerging technologies. Theories and methods of cognitive science provide useful tools for the development of more effective forms and surveys by giving us a better understanding of how respondents comprehend tax form line items, recall relevant facts from memory, make judgments if they are called for, and select their final responses. The result of this research will then be used, in conjunction with other known questionnaire revision requirements, to modify data collection procedures undertaken by IRS in pursuit of its stated mission.

With this purpose in mind, staff within and outside of IRS will design, conduct and interpret field and laboratory research that contributes new knowledge of the cognitive aspects of human behavior in relationship to forms, survey, and script design. Cognitive psychological research laboratory methods in use include such techniques such as probe questioning, memory cueing, group discussion, and intensive interviewing. Depending on research goals, these methods may be used separately or in combination with one another.

The use of the laboratory approach for the examination of the process noted above has a number of advantages associated with it. These advantages include: rapid and in-depth testing of line items on forms, a more detailed understanding of the respondents comprehension of concepts, and special populations which can be quickly recruited and tested. Different laboratory methods of the questionnaire development process will, therefore, be used in various studies, depending on the type of data collection instruments being studied.

The results of these research efforts may be shared with others by way of seminars, training sessions, reports and/or publications and presentations to professional societies. Concurrent documentation of all research efforts, both on-going and projected long-term projects, is kept in summary form within the Statistics of Income Division.

3. Use of Improved Information Technology to Reduce Burden

Respondent burden in this collection will be held to a minimum. The proposed approach of research data collection methods is designed to obtain the maximum amount of information for the minimum respondent burden. This approach includes such methods as:

a. interview pacing and latency classification,

b. identification and clarification of interview characteristics such as: degree of structure within the interview format, group dynamics, observation and recording of decision behaviors and/or the negotiation processes,

c. vignette classifications, which encompass scenario sort tasks reflecting actual comprehension and/or perception,

d. hypothetical sort tasks reflecting imagined or constructed perceptions,

e. expert analyses (both formal and informal), and

f. "classroom" experiments involving the administration of

experimental forms to groups of respondents or tasks to be completed

in an experimental online environment.

Surveys conducted in conjunction with this research will be narrowly focused on the users or potential users of a particular system and the number of respondents will be small in comparison to most surveys.

4. Efforts to Identify Duplication

The focus of this research will provide critical, ground-breaking, and important supplemental information specific to IRS and beyond that currently available in the field of survey methodology.

This research does not duplicate any outside-of-government research effort, as its purpose is not to replicate survey research studies. External contractors as well as internal customers are cognizant of current research being done in the field of cognitive psychology through attendance at conferences, research reported in professional journals, and through in-house staff meetings and peer review processes. Wage and Investment and Small Business/Self Employed Division’s monitor the quality of service given by the IRS to taxpayers who have tax questions. Tax Forms and Publications Division develops the major tax forms in IRS. IT develops the new methodologies associated with the modernization. SB/SE, LB&I, and TE/GE conduct studies that focus on IRS products, services, and customer needs. These specific tasks are not performed anywhere else, inside or outside of government.

5. Methods to Minimize Burden on Small Businesses or Other Small Entities

All efforts will be made to minimize the burden on small businesses or other small entities in the event that data will be needed pertaining to these entities on emerging IRS technologies. Extensive screening of potential entities will be conducted prior to any contact, and pre-testing of any survey instrument will be performed to ensure proper procedures for contact and re-contacts in an effort to minimize burden. The number of respondents will be held to a minimum. Current public information concerning businesses will be reviewed for any available information pertaining to the study prior to the implementation of the study. All efforts will be made to avoid duplication of any other government or outside-of-government agency's research projects, as the purpose is not to replicate survey research studies.

6. 6. Consequences of Less Frequent Collection on Federal Programs or Policy Activities

Because the planned collection of data is expected to be an on-going effort, it has the potential to have immediate impact on all form design changes and emerging interactive systems within IRS. Its delay would sacrifice potential gain in forms design and improved systems within IRS.

Each research project will be a one-time study. Therefore, although each study will occur only once, there will be a series of one-time projects that will, over time, allow for suggestions, modifications and alterations in an on-going manner.

7. Special Circumstances Requiring Data Collection to be Inconsistent with Guidelines in 5 CFR 1320.5(d)(2)

These laboratory methodological projects are not related to producing statistical estimates. In certain circumstances, a project’s objectives can be accomplished at lower costs and with greatly reduced respondent burden by the use of small purposive samples. For some research projects lab staff will travel to, and test in, the vicinity of respondents residences or they may conduct interviews via telephone. Because some respondents are asked to leave their homes and travel to the laboratory field testing site, they will be reimbursed a designated amount of money for their time in the Lab (approximately 1/2 hour to 2 hours) and for transportation. With the exception of remuneration, there are no circumstances that require data collection to be inconsistent with 5 CFR 1320.5(d)(2).

If circumstances change calling for additional remuneration, then it is our intention to collect data from the contractor measuring the extent of time and effort in recruiting an adequate number of participants for research studies. This information could be used as a basis for changing the reimbursement policy that OMB and IRS agree to as necessary.

8. Consultation with Individuals Outside of the Agency on Availability of Data, Frequency of Collection, Clarity of Instructions and Forms, and Data Elements

a. Other agencies - not applicable.

2. b. Within the IRS, coordination occurs on each SOI project moving through other areas of the IRS. Care is taken to ensure that SOI efforts do not duplicate those of opinion research or surveys conducted by contractors at the request of the IRS subject matter specialists. The individuals responsible for this coordination are:

Laura R. Rasmussen

Internal Revenue Service RAAS:S:SS

1111 Constitution Ave., K-4144

Washington, DC 20224-0002

(202) 803-9748

Michelle Sloan

Internal Revenue Service RAAS:S:SS

1111 Constitution Ave., K-4100-098

Washington, DC 20224-0002

(202) 803-9311

c. There are no unresolved problems.

d. No other comments have been received; however, continual feedback from the scientific community is an important aspect of the proposed research. In both laboratory and field studies, consultation with representatives of various fields of psychology, anthropology, sociology and the like, will have direct impact on the future course of research projects, and promises to determine the feasibility of data collection and the validity of research studies.

We received no comments during the comment period in response to the Federal Register notice dated March 29, 2017 (82 FR 15558).

9. Explanation of Decision To Provide Any Payment or Gift To Respondents

Currently, for the cost of transportation, refreshments, and other incidental expenses associated with their participation, the general rule is that for Focus Groups there is a $75 maximum incentive for up to 2 hours. This is the case unless it can be shown that in previous efforts, this amount has been insufficient to recruit the appropriate number of participants from the targeted population.  For Cognitive Interviews, the maximum allowed for incentives is $40 unless it can be demonstrated through previous efforts, that this amount was insufficient to recruit an appropriate number of respondents.  It is requested that exemptions and exceptions to this payment amount be considered on a case-by-case basis as the IRS is considered by many to be an enforcement agency and depending upon the topic or the target population, recruiting willing respondents can be especially difficult.

10. Assurance of Anonymity of Responses

a. The data collected from respondents will be tabulated and analyzed only for the purpose of evaluating the research in question.

b. Anonymity Policy

All personnel are responsible for maintaining the Anonymity of all data collected. Security requirements are based on The Computer Security Act of 1987 and Office of Management and Budget Circular A-130, Appendices A7B. The security of the data utilized for research projects contained within this package and the privacy of taxpayers to the extent allowed by law will be carefully safeguarded at all times. Physical security measures include a locked, secure office. Audiotapes of focus group interviews will be stored in locked cabinets. Any transcriptions of audiotapes will also be stored in locked cabinets or shredded once the report is written. Data security at the C-2 level will be accomplished via the Windows Operating System. Systems will be password-protected, users will be profiled for authorized use, and individual audit trails will be generated and reviewed.

c. IRS Disclosure Regulations

All data collection will conform to IRS disclosure regulations.

11. Justification of Sensitive Questions

There are no questions of a sensitive nature.

12. Estimated Burden of Information Collection

It is anticipated that the research described will require approximately 90 minutes per respondent for cognitive think-alouds, usability studies, and focus groups in order to achieve its objective. These estimates cover the time respondents (from the public sector) will spend participating in research requirements and being debriefed about the answers they provide.

The total estimated collection burden for completing the work is 6,000 hours annually (18,000 hours for the period of this approval), and is described as follows:

	Estimated Annual Reporting Burden
Type of Collection		No. of Respondents	Annual Frequency per Response	Total responses	Hours per Response	Total Hours
Focus Group Respondents		2,000	1	2,000	1.5	3,000
Cognitive Think Aloud Respondents		1,000	1	1,000	1.5	1,500
Usability Study Respondents		1,000	1	1,000	1.5	1,500
Annual Total				4,000		6,000
Requested 3 year approval				x    3		x 3
2017-2020 Total Burden Hours				12,000		18,000

Each study will, of course, substantively differ from one another. The projects are expected to be complex, involving at times, several cognitive testing procedures/methods which will be used to test the hypotheses of the given research question.

Coverage of the Estimates -- The estimates covers the time that each respondent from the public sector will spend answering questions, including the debriefing procedure concerning the cognitive testing procedure used. The time required to travel to the laboratory is not covered, since distances and modes of transportation are unknown.

Basis for the Estimates -- These estimates are based on the average length of time it takes to complete this type of research over the past three years.

Estimates of the annualized cost to respondents for the hour burdens shown are not available at this time.

13. Estimated Total Annual Cost Burden To Respondents

As suggested by OMB, our Federal Register notice dated March 16, 2017, requested public comments on estimates of cost burden that are not captured in the estimates of burden hours, i.e., estimates of capital or start-up costs and costs of operation, maintenance, and purchase of services to provide information. However, we did not receive any response from taxpayers on this subject. As a result, estimates of the cost burdens are not available at this time.

14. Estimated Annualized Cost to the Federal Government

The maximum estimated cost to the Federal Government is $741,299 per year. The cost includes incentives as well as contractual costs that may be incurred.

15. Reasons for Change in Burden

A change to reduce the current burden of 90,000 hours is being requested. A number of Information Collection Requests (ICR) normally submitted under this Generic Clearance and that were associated with substantial burden are now being submitted and approved under their own standard 3-year clearance or Generic Clearance request. We are making this submission to renew the OMB approval with a burden of 6,000 hours annually for a total of 18,000 hours over the course of this approval.

	Requested	Change Due to New Statute	Change Due to Agency Discretion	Change Due to Adjustment in Estimate	Change Due to Potential Violation of the PRA	Previously Approved
Annual Number of Responses	12,000		(8,000)			20,000
Annual Time Burden (Hr)	18,000		(12,000)			30,000

16. Plans For Tabulation, Statistical Analysis and Publication

This clearance request is for the design of forms and quality assurance, which include the exploratory activities leading to the evaluation of data quality. Both quantitative and qualitative analyses are planned for the evaluation of these activities depending on the circumstances.

The results of these investigations will be used primarily to improve the quality of data collection and assure the total collection quality as it relates to tax administration and statistical use of tax data. Since external contractors may be used, relying on the latest methodology in cognitive psychological testing and techniques, papers may be written which may include some tallies of response problems, recall strategies, or results from other testing procedures used, etc.

Project Schedule -- The project schedule calls for laboratory interviews to commence once approval is received from OMB.

A time schedule is proposed that is continuously ongoing in nature, with research publication dates dependent on data collection limited to the researcher's proposal and subsequent results.

17. Reasons Why Displaying the OMB Expiration Date Is Inappropriate

Displaying the expiration date may cause problems with respondents for data collection programs that overlap the three-year approval periods. Would-be respondents might be inclined to refuse to participate if the form carries a date that is expired or is soon to expire.

18. Exceptions To the Certification for Paperwork Reduction Act Submissions

This collection contains no exceptions to the certification for the Paperwork Reduction Act.

Note: The following paragraph applies to all of the collections of information in this submission:

An agency may not conduct or sponsor and a person is not required to respond to a collection of information unless the collection of information displays a valid OMB control number. Books or records relating to a collection of information must be retained as long as its contents may become material in the administration of any internal revenue law. Generally, tax returns and tax return information are confidential, as required by 26 U.S.C. 6103.

B. Collection of Information Employing Statistical Methods

1. Sample Design

The data collected will be used primarily for research activities that improve data collection processes, including tax forms and interactive technologies, rather than to produce estimates about the population. The objective is to interview a variety of people, rather than a probability sample of the population, or to conduct limited surveys of users and potential users of pilot technology. For most of the research design activities concerning items applicable to the general population, respondents will be recruited by telephone or mail contact. In some cases of surveys of potential users of pilot technologies, probability samples may be used. Detailed sample designs will be provided separately for each of these cases.

For testing some hypotheses, however, some initial screening of volunteers will be done to identify eligible respondents. Eligible respondents are defined as those individuals who have not participated in more than three survey research projects and meet other necessary requirements. Special attempts may be made to recruit from special groups if there are no volunteers from these groups as a result of the general recruiting effort.

2. Data Collection Procedures

Small Group Research

Recruitment: The external contractor will generally recruit respondents through telephone calls. In most cases, the IRS will provide criteria to select appropriate respondents. The public will have the choice of participating in the research. Their participation is voluntary. If the respondent possesses the desired characteristics, an interview is scheduled.

In-person Interview: Once an interview is scheduled, it is the responsibility of the respondent (from the public sector) to travel to the interview site. The rooms are private and soundproof to insure privacy of the interview. To reduce the number of no shows, scheduled volunteers will be sent a reminder letter giving the time of the interview and directions to the laboratory. They should also receive a reminder telephone call before the interview.

When the respondent arrives, he/she will receive an explanation of the purpose of the interview and the procedures to be used. The respondent will then be asked to read a brief description of the study or a description of the study will be presented orally. The respondent is then given an assurance of privacy to read. This assurance is then reread orally by the interviewer to the respondent and the need for audio or video taping of the interview is explained.

Interviewing can last from 1/2 hour to 2 hours depending on the specific laboratory techniques being applied.

The selection of the laboratory technique, in turn, is determined by the hypotheses to be tested. The most commonly used methods include concurrent and retrospective think-aloud interviews and focus group interviews. In one-on-one think-aloud interviews, respondents are asked questions (pertinent to the tax form or procedure in question) and are asked to think-aloud about how and why they answered as they did. The interviewer usually probes extensively to ascertain the degree of comprehension and the recall processes involved.

Focus group interviews usually consist of 6-10 people per group. In these small group sessions, respondents are again asked questions (pertinent to the tax form or procedure in question) and are asked to respond openly to each based on their individual experience. The moderator tries to involve everyone in each question to better understand attitudes, opinions, and concerns.

Debriefing: All respondents should be debriefed. This procedure reiterates the purpose of the project and answers respondents' questions regarding the study.

As a cost-saving method, sometimes focus groups are held via telephone. The interview techniques remain the same, adjusted slightly to accommodate participation via telephone as opposed to having all participants together in one room.

All respondents will be provided with an address to offer comments concerning the data collection process and the Office of Management and Budget (OMB) Control Number.

3. Methods to Maximize Response Rates

As noted, to reduce the number of no-shows, scheduled volunteers will be sent a reminder letter giving the time of the interview and directions to the laboratory. They will also receive a reminder telephone call before the interview.

4. Tests of Procedures and Methods

The tests proposed for research fall into a number of categories which cognitive psychologists utilize to confirm or reject research hypotheses. Some of these tests include those tasks outlined by Michael W. Eysenck (1984) in A Handbook of Cognitive Psychology. Some of the possible tests outlined are:

• developing protocols, scenarios, and question probes--follow-up questions used to gain more information about respondents' strategies for answering questions,

• concurrent think-aloud interview--respondents think aloud while answering questions and responses are probed extensively,

• focus groups and individual interviews--structured and unstructured discussion with groups or individuals,

• retrospective think-aloud interview--respondents answer all questions first, then are asked how they arrived at their answer,

• sorting ranking tasks--respondents sort lists or similar items into groups that go together and rank the items according to specified scale,

• confidence ratings--respondents relate the degree of confidence they have in the accuracy of their answers,

• memory cues--interviewer reads terms which are intended as aid to recall, and

• response latency--measurement of the elapsed time between the presentation of the question and the respondents answer.