Supporting Statement 2010-2013

SUPPORTING STATEMENT

INTERNAL REVENUE SERVICE

STATISTICS OF INCOME

Cognitive and Psychological Research

Generic Clearance

(March 2011 through March 2013)

Abstract

This is a request for generic clearance by the Internal Revenue Service (IRS), Statistics of Income (SOI), coordinating with Wage and Investment (W&I), Small Business/Self Employed (SB/SE), Large and Mid-Size Business (LMSB) and Tax Exempt Government Entities (TE/GE) to conduct research. The research will result in making recommendations for positive changes to current tax forms and customer service procedures. These changes will represent improvements aimed at both easing the taxpayer burden in understanding and working with the forms, as well as contributing to a simplification of the processing of the forms within the IRS processing environment. In addition, new methodologies and technology associated with the IRS modernization will provide design and implementation support. The research and development activities in the forms improvement process will be conducted by external contractors employing state-of-the-art cognitive psychological testing methods. We have demonstrated, by extensive applications over the past decade, the feasibility and value of this approach to questionnaire construction, survey technology, and interview processes. Previous experience in applying these techniques has resulted in recommended improvements to IRS forms and publications, filing requirements, and electronic filing. The planned research and development activities will be conducted during FY 2010 through FY 2013 with the goal of making continued improvements to tax forms and new technologies.

A. Justification

1. Circumstances Necessitating Collection of Information

Currently SOI in collaboration with other IRS organizations along with external contractors conduct cognitive psychological research in design and execution of the data collection process in order to improve the quality of tax forms and taxpayer products and services.

SOI works closely with Wage and Investment (W&I), Small Business/Self Employed (SB/SE), Large and Mid-Size Business (LMSB), and TEGE. Wage and Investment is responsible for monitoring the accuracy of the responses given to the general public who call IRS for answers to tax questions. Tax Forms and Publications division creates the forms used by taxpayers for tax purposes. Modernization and Information Technology Services (MITS) is responsible for design and implementation of many new interactive systems and technologies that are being developed as part of the IRS modernization effort. W&I, LMSB, SB/SE, and TEGE conduct a broad range of studies and tests that focus on taxpayer needs.

The purpose of this request for clearance for cognitive psychological research and development activities by SOI working with external contractors is to continue to enhance the quality of the information collected using tax forms and to provide design and implementation testing support to new IRS methods and technologies. Over the past three years, studies such as the Death and Hard of Hearing Focus Groups,

Health Care HIRE Act Focus Groups, IRS.gov Usability Study, Testing of Form 8822 (Change of Address), and Language Services Interactive DVD (educational tool) have proven successful in determining barriers to use, product opinion, and associated burden.

The research techniques and methods to be used in these studies will include analysis of form design as well as survey technology. Analyses will be conducted within the following domains:

a. Question Analysis -- Evaluation of individual forms, survey questions, and scripts appraising line item intention, assessment of semantic clarity, and an examination of relations between line items.

b. Term Analysis -- Evaluation of specific wording and phrases in terms of their psycho linguistic properties and an assessment of respondent interpretation of the meaning of these terms, at both the conscious and unconscious levels.

c. Instruction Analysis -- Inspection of instructions for their semantic clarity, the degree to which they reflect the stated intention, ease of interpretation, and other considerations which may elicit unambiguous and appropriate answers from taxpayers.

d. Format Analysis -- Review of tax forms for perceptual characteristics in order to facilitate better respondent comprehension and promote more active attention to the focus of the form.

2. Use of Data

The purpose of IRS data collection is to improve both the theoretical understanding of the data collection process and the practical applications of such knowledge especially as it pertains to emerging technologies. Theories and methods of cognitive science provide useful tools for the development of more effective forms and surveys by giving us a better understanding of how respondents comprehend tax form line items, recall relevant facts from memory, make judgements if they are called for, and select their final responses. The result of this research will then be used, in conjunction with other known questionnaire revision requirements, to modify data collection procedures undertaken by IRS in pursuit of its stated mission.

With this purpose in mind, staff within and outside of IRS will design, conduct and interpret field and laboratory research that contributes new knowledge of the cognitive aspects of human behavior in relationship to forms, survey, and script design. Cognitive psychological research laboratory methods in use include such techniques such as probe questioning, memory cueing, group discussion, and intensive interviewing. Depending on research goals, these methods may be used separately or in combination with one another.

The results of this research effort will be shared with others by way of seminars, training sessions, reports and/or publications and presentations to professional societies. Concurrent documentation of all research efforts, both on-going and projected long-term projects, is kept in summary form within the Statistics of Income Division. Thus, research is expected to result in:

• an increase in skills which will result in knowledge of new methodologies,

• a deeper understanding of the survey literature, and

• improvement of data collection instruments.

The use of the laboratory approach for the examination of the process noted above has a number of advantages associated with it. These advantages include: rapid and in-depth testing of line items on forms, a more detailed understanding of the respondents comprehension of concepts, and special populations which can be quickly recruited and tested. Different laboratory methods of the questionnaire development process will, therefore, be used in various studies, depending on the type of data collection instruments being studied.

3. Use of Improved Information Technology to Reduce Burden

Respondent burden in this collection will be held to a minimum. The proposed approach of research data collection methods is designed to obtain the maximum amount of information for the minimum respondent burden. This approach includes such methods as:

a. interview pacing and latency classification,

b. identification and clarification of interview characteristics such as: degree of structure within the interview format, group dynamics, observation and recording of decision behaviors and/or the negotiation processes,

c. vignette classifications, which encompass scenario sort tasks reflecting actual comprehension and/or perception,

d. hypothetical sort tasks reflecting imagined or constructed perceptions,

e. expert analyses (both formal and informal), and

f. "classroom" experiments involving the administration of experimental forms to groups of respondents.

Surveys conducted in conjunction with this research will be narrowly focused on the users or potential users of a particular system and the number of respondents will be small in comparison to most surveys.

4. Efforts to Identify Duplication

The focus of this research will provide critical, ground breaking, and important supplemental information beyond that currently available in the field of survey methodology.

This research does not duplicate any outside-of-government research effort, as its purpose is not to replicate survey research studies. External contractors as well as internal customers are cognizant of current research being done in the field of cognitive psychology through attendance at conferences, research reported in professional journals, and through in-house staff meetings and peer review processes. Wage and Investment and Small Business/Self Employed Division’s monitor the quality of service given by the IRS to taxpayers who have tax questions. Tax Forms and Publications Division develops the major tax forms in IRS. MITS develops the new methodologies associated with the modernization. SB/SE, LMSB, and TE/GE conduct studies that focus on IRS products, services, and customer needs. All divisions will be kept informed of SOI forms research projects.

5. Methods to Minimize Burden on Small Businesses or Other Small Entities

All efforts will be made to minimize the burden on small businesses or other small entities in the event that data will be needed pertaining to these entities on emerging IRS technologies. Extensive screening of potential entities will be conducted prior to any contact, and pre-testing of any survey instrument will be performed to ensure proper procedures for contact and re-contacts in an effort to minimize burden. The number of respondents will be held to a minimum. Current public information concerning businesses will be reviewed for any available information pertaining to the study prior to the implementation of the study. All efforts will be made to avoid duplication of any other government or outside-of-government agency's research projects, as the purpose is not to replicate survey research studies.

6. 6. Consequences of Less Frequent Collection on Federal Programs or Policy

Activities

Because the planned collection of data is expected to be an on-going effort, it has the potential to have immediate impact on all form design changes and emerging interactive systems within IRS. Its delay would sacrifice potential gain in forms design and improved systems within IRS.

Each research project will be a one-time study. Therefore, although each study will occur only once, there will be a series of one-time projects that will, over time, allow for suggestions, modifications and alterations in an on-going manner.

7. Special Circumstances Requiring Data Collection to be Inconsistent with Guidelines in 5 CFR 1320.5(d)(2)

These laboratory methodological projects are not related to producing statistical estimates. In certain circumstances, a project’s objectives can be accomplished at lower costs and with greatly reduced respondent burden by the use of small purposive samples. For some research projects lab staff will travel to, and test in, the vicinity of respondents residences. Because respondents are asked to leave their homes and travel to the laboratory field testing site, they will be reimbursed a designated amount of money for their time in the Lab (approximately 1/2 hour to 2 hours) and for transportation. With the exception of remuneration, there are no circumstances that require data collection to be inconsistent with 5 CFR 1320.5(d)(2).

If circumstances change calling for additional remuneration, then it is our intention to collect data from the contractor measuring the extent of time and effort in recruiting an adequate number of participants for research studies. This information could be used as a basis for changing the reimbursement policy that OMB and IRS agree to as necessary.

8. Consultation with Individuals Outside of the Agency on Availability of Data, Frequency of Collection, Clarity of Instructions and Forms, and Data Elements

a. Other agencies - not applicable.

2. b. Within the IRS, coordination occurs on each SOI project moving through other areas of the IRS. Care is taken to ensure that SOI efforts do not duplicate those of opinion research or surveys conducted by contractors at the request of the IRS subject matter specialists. The individuals responsible for this coordination are:

Laura R Rasmussen

Internal Revenue Service

Statistics of Income Division

500 N. Capitol Street

Washington, DC 20001

(202) 874-1269

Michelle Sloan

Internal Revenue Service

Statistics of Income Division

500 N. Capitol Street

Washington, DC 20001

(202) 874-0683

c. There are no unresolved problems.

d. No other comments have been received; however, continual feedback from the scientific community is an important aspect of the proposed research. In both laboratory and field studies, consultation with representatives of various fields of psychology, anthropology, sociology and the like, will have direct impact on the future course of research projects, and promises to determine the feasibility of data collection and the validity of research studies.

We received no comments during the comment period in response to the Federal Register notice (75 FR 56657), dated September 16, 2010.

9. Explanation of Decision To Provide Any Payment or Gift To Respondents

The only payment authorized to respondents is for those who participate in the lab research. Currently, they are authorized $25 for the cost of transportation, meals, and other incidental expenses associated with their participation in the lab research. However it is requested that exemptions and exceptions to this payment amount be considered on a case-by-case basis.

10. Assurance of Anonymity of Responses

a. The data collected from respondents will be tabulated and analyzed only for the purpose of evaluating the research in question.

b. Anonymity Policy

All personnel are responsible for maintaining the Anonymity of all data collected. Security requirements are based on The Computer Security Act of 1987 and Office of Management and Budget Circular A-130, Appendices A7B. The security of the data utilized for research projects contained within this package and the privacy of taxpayers will be carefully safeguarded at all times. Physical security measures include a locked, secure office. Audiotapes of focus group interviews will be stored in locked cabinets. Any transcriptions of audiotapes will also be stored in locked cabinets or shredded once the report is written. Data security at the C-2 level will be accomplished via the Windows NT operating system. Systems will be password-protected, users will be profiled for authorized use, and individual audit trails will be generated and reviewed.

c. IRS Disclosure Regulations

All data collection will conform to IRS disclosure regulations.

11. Justification of Sensitive Questions

There are no questions of a sensitive nature.

12. Estimated Burden of Information Collection

It is anticipated that the research described will require approximately one and one half hour per respondent for cognitive think-alouds and usability studies, ninety minutes for focus groups and 15 minutes per respondent for surveys in order to achieve its objective. These estimates cover the time respondents (from the public sector) will spend participating in research requirements and being debriefed about the answers they provide.

The total estimated collection burden for completing the work is 37,500 hours (annually), and is described as follows:

FY 2010-2013	Number of Responses	Time per Response	Total Burden Hours
Survey Respondents	60,000	15 minutes	15,000 hours
Focus Group Respondents	10,000	1 ½ hour	15,000
Cognitive Think-Aloud Respondents	2,500	1 ½ hour	3,750
Usability Study Respondents	2,500	1 ½ hour	3,750
Total	75,000		37,500

Each study will, of course, substantively differ from one another. The projects are expected to be complex, involving at times, several cognitive testing procedures/methods which will be used to test the hypotheses of the given research question.

Coverage of the Estimates -- The estimates cover the time that each respondent from the public sector will spend answering questions, including the debriefing procedure concerning the cognitive testing procedure used. The time required to travel to the laboratory is not covered, since distances and modes of transportation are unknown.

Basis for the Estimates -- These estimates are based on the average length of time it takes to complete this type of research over the past three years.

Estimates of the annualized cost to respondents for the hour burdens shown are not available at this time.

13. Estimated Total Annual Cost Burden To Respondents

As suggested by OMB, our Federal Register notice (75 FR 56657), dated September 16, 2010, requested public comments on estimates of cost burden that are not captured in the estimates of burden hours, i.e., estimates of capital or start-up costs and costs of operation, maintenance, and purchase of services to provide information. However, we did not receive any response from taxpayers on this subject. As a result, estimates of the cost burdens are not available at this time.

14. Estimated Annualized Cost to the Federal Government

The maximum cost to the Federal Government is $250,000 per year, and the cost is entirely comprised of the $25 reimbursement per respondent. Costs, such as operational expenses (e.g., equipment, overhead, printing, and support staff), are in place as a function of the laboratory facility and are not contingent or necessary to perform this research.

15. Reasons for Change in Burden

No change in burden is being requested. We are making this submission to renew the OMB approval.

16. Plans For Tabulation, Statistical Analysis and Publication

This clearance request is for the design of forms and quality assurance, which include the exploratory activities leading to the evaluation of data quality. Both quantitative and qualitative analyses are planned for the evaluation of these activities depending on the circumstances.

The results of these investigations will be used primarily to improve the quality of data collection and assure the total collection quality as it relates to tax administration and statistical use of tax data. Since external contractors may be used, relying on the latest methodology in cognitive psychological testing and techniques, papers may be written which may include some tallies of response problems, recall strategies, or results from other testing procedures used, etc.

Project Schedule -- The project schedule calls for laboratory interviews to commence once approval is received from OMB.

A time schedule is proposed that is continuously ongoing in nature, with research publication dates dependent on data collection limited to the researcher's proposal and subsequent results.

17. Reasons Why Displaying the OMB Expiration Date Is Inappropriate

Displaying the expiration date may cause problems with respondents for data collection programs that overlap the three-year approval periods. Would-be respondents might be inclined to refuse to participate if the form carries a date that is expired or is soon to expire.

18. Exceptions To the Certification Statement on OMB Form 83-I

Not applicable.

Note: The following paragraph applies to all of the collections of information in this submission:

An agency may not conduct or sponsor and a person is not required to respond to a collection of information unless the collection of information displays a valid OMB control number. Books or records relating to a collection of information must be retained as long as its contents may become material in the administration of any internal revenue law. Generally, tax returns and tax return information are confidential, as required by 26 U.S.C. 6103.

B. Collection of Information Employing Statistical Methods

1. Sample Design

The data collected will be used primarily for research activities that improve data collection processes, including tax forms and interactive technologies, rather than to produce estimates about the population. The objective is to interview a variety of people, rather than a probability sample of the population or to conduct limited surveys of users and potential users of pilot technology. For most of the research design activities concerning items applicable to the general population, respondents will be recruited by telephone or mail contact. In some cases of surveys of potential users of pilot technologies, probability samples may be used. Detailed sample designs will be provided separately for each of these cases.

For testing some hypotheses, however, some initial screening of volunteers will be done to identify eligible respondents. Eligible respondents are defined as those individuals who have not participated in more than three survey research projects and meet other necessary requirements. Special attempts may be made to recruit from special groups if there are no volunteers from these groups as a result of the general recruiting effort.

2. Data Collection Procedures

Small Group Research

Recruitment: The external contractor will generally recruit respondents through telephone calls. In most cases, the IRS will provide criteria to select appropriate respondents. The public will have the choice of participating in the research. Their participation is voluntary. If the respondent possesses the desired characteristics, an interview is scheduled.

Interview: Once an interview is scheduled, it is the responsibility of the respondent (from the public sector) to travel to the interview site. The rooms are private and soundproof to insure confidentiality of the interview. To reduce the number of no shows, scheduled volunteers will be sent a reminder letter giving the time of the interview and directions to the laboratory. They should also receive a reminder telephone call before the interview.

When the respondent arrives, he/she will receive an explanation of the purpose of the interview and the procedures to be used. The respondent will then be asked to read a brief description of the study or a description of the study will be presented orally. The respondent is then given an assurance of confidentiality to read. This assurance is then reread orally by the interviewer to the respondent and the need for audio or video taping of the interview is explained.

Interviewing can last from 1/2 hour to 2 hours depending on the specific laboratory techniques being applied.

The selection of the laboratory technique, in turn, is determined by the hypotheses to be tested. The most commonly used methods include concurrent and retrospective think-aloud interviews and focus group interviews. In one-on-one think-aloud interviews, respondents are asked questions (pertinent to the tax form or procedure in question) and are asked to think-aloud about how and why they answered as they did. The interviewer usually probes extensively to ascertain the degree of comprehension and the recall processes involved.

Focus group interviews usually consist of 6-10 people per group. In these small group sessions, respondents are again asked questions (pertinent to the tax form or procedure in question) and are asked to respond openly to each based on their individual experience. The moderator tries to involve everyone in each question to better understand attitudes, opinions, and concerns.

Debriefing: All respondents should be debriefed. This procedure reiterates the purpose of the project and answers respondents' questions regarding the study.

Surveys

Limited information from users and potential users of new IRS technology will be collected using self-administered mail surveys, telephone surveys, or electronic surveys of users of interactive applications. Respondents will be advised of the purpose of the survey and its voluntary nature via a cover letter or advance letter in the case of a telephone survey or as part of the survey introduction in the case of electronic surveys.

All respondents will be provided with an address to offer comments concerning the data collection process and the Office of Management and Budget (OMB) Control Number.

3. Methods to Maximize Response Rates

As noted, to reduce the number of no-shows, scheduled volunteers will be sent a reminder letter giving the time of the interview and directions to the laboratory. They will also receive a reminder telephone call before the interview.

Mail surveys will be designed to include a reminder postcard and a second survey mailout as needed to increase response rates. Previous IRS experience with point of service delivery telephone or electronic surveys indicates a high response rate to the initial request to participate. It is further felt that conversion attempts on point of service delivery refusals will antagonize the respondents.

4. Tests of Procedures and Methods

The tests proposed for research fall into a number of categories which cognitive psychologists utilize to confirm or reject research hypotheses. Some of these tests include those tasks outlined by Michael W. Eysenck (1984) in A Handbook of Cognitive Psychology. Some of the possible tests outlined are:

• developing protocols, scenarios, and question probes--follow-up questions used to gain more information about respondents' strategies for answering questions,

• concurrent think-aloud interview--respondents think aloud while answering questions and responses are probed extensively,

• focus groups and individual interviews--Structured and unstructured discussion with groups or individuals,

• retrospective think-aloud interview--respondents answer all questions first, then are asked how they arrived at their answer,

• sorting ranking tasks--respondents sort lists or similar items into groups that go together and rank the items according to specified scale,

• confidence ratings--respondents relate the degree of confidence they have in the accuracy of their answers,

• memory cues--interviewer reads terms which are intended as aid to recall, and

• response latency--measurement of the elapsed time between the presentation of the question and the respondents answer.