TAA Handbook v2.0 TAA Handbook Data Validation Software Users Guide

Employment and Training Data Validation Requirement

TAA-Handbook-v2.0

NFJP, Indian and Native American, and SCSEP Grantees

OMB: 1205-0448

Document [pdf]
Download: pdf | pdf
TAA
Data Validation
Handbook
for

TAADV Software
Release 2.0
December 2006

(REV--6/7/02)

Table of Contents

Chapter

Page

I.

OVERVIEW ................................................................................................... 2

II.

VALIDATION PROCEDURES................................................................... 11

III.

RECORD LAYOUT..................................................................................... 45

APPENDIX A: PERFORMANCE MEASURE SPECIFICATIONS.................... A.1
APPENDIX B: DEV SAMPLING AND ERROR RATE ESTIMATION............ B.1
APPENDIX C: DEV INSTRUCTIONS................................................................ C.1
APPENDIX D: SOFTWARE INSTALLATION .................................................. D.1
APPENDIX E: SOFTWARE REFERENCE GUIDE ............................................E.1

1

I. OVERVIEW
Trade Adjustment Assistance (TAA) programs assist individuals whose
employment has been impacted by increased imports from or shifts in production
to foreign countries. The goal of the Trade Act programs is to help trade-affected
workers obtain the skills needed in today’s economic environment. States submit
Trade Act Participant Report (TAPR) records to ETA on a quarterly basis, and
ETA uses these records to measure the extent to which the program is meeting its
performance goals. Participants who exit the TAA program after receiving
employment and training services are included in three performance measures:
1. The entered employment measure is based on whether the exiter entered
employment.
2. The employment retention measure is based on employment status during
the second and third quarters after exit for exiters who entered
employment.
3. The average earnings measure is based on the earnings of exiters who
entered and were retained in employment after exit.

In order to ensure that the data used to calculate the TAA performance measures
are accurate, states are required to validate the records of participants included in
these measures. According to TEGL 3-03, TEN 9-06, and recent ETA guidance,
states are required to validate their Fiscal Year (FY) 2006 exiter records by April
15, 2007. States conduct data validation after final TAPR records for the fiscal
year have been submitted to ETA. The data to conduct the FY 2006 validation
should be available as of November 15, 2006, when the final TAPR for FY 2006
was due to ETA.
This TAA data validation handbook describes the procedures for states to follow to
review the accuracy of their TAPR records.
The handbook also provides
instructions on using the TAA data validation (TAADV) software that ETA
provides to states.

2

I. OVERVIEW
A. TAPR REPORTING GUIDELINES
The TAA fiscal year runs from October 1st through September 30th. For example,
FY 2006 runs from October 1, 2005 to September 30, 2006. Each state is required
to make four quarterly submissions of TAPR records to ETA according to the
guidelines shown in Tables I.1 and I.2. These tables specify the report quarter, exit
quarter, and TAPR due date for each quarterly submission for FY 2006 and 2007.
The Report Quarter row lists the four reporting quarters in the fiscal year, which
are the time frames during which records of TAA program exiters should be
compiled and the TAPR assembled. The Exit Quarter row lists the corresponding
participant exit dates. For example, to assemble the FY 2006 Q1 TAPR, the state
should assemble records for all participants who exited the TAA program between
7/1/2004 and 9/30/2004. The TAPR Due Date row specifies when the quarterly
TAPR submission is due to ETA. Each submission is due five and a half quarters
after the quarter of exit, which provides sufficient time to obtain the wage data
needed for calculating the retention rate.

Table I.1
Fiscal Year 2006 – Report Periods
Q1

Q2

Q3

Q4

Report Quarter

10/1/2005 –
12/31/2005

1/1/2006 –
3/31/2006

4/1/2006 –
6/30/2006

7/1/2006 –
9/30/2006

Exit Quarter

7/1/2004 –
9/30/2004

10/1/2004 –
12/31/2004

1/1/2005 –
3/31/2005

4/1/2005 –
6/30/2005

TAPR Due Date

2/15/2006

5/15/2006

8/15/2006

11/15/2006

3

I. OVERVIEW
Table I.2
Fiscal Year 2007 – Report Periods
Q1

Q2

Q3

Q4

Report
Quarter

10/1/2006 –
12/31/2006

1/1/2007 –
3/31/2007

4/1/2007 –
6/30/2007

7/1/2007 – 9/30/2007

Exit
Quarter

7/1/2005 –
9/30/2005

10/1/2005 –
12/31/2005

1/1/2006 –
3/31/2006

4/1/2006 – 6/30/2006

TAPR
Due Date

2/15/2007

5/15/2007

8/15/2007

11/15/2007

To conduct data validation and calculate the annual performance measures, the
validation extract file must contain records for exiters for all four quarters in the
fiscal year. For example, the FY 2006 validation extract file must contain records
for all participants who exited between July 1, 2004 and June 30, 2005. The FY
2007 validation extract file must contain records for all participants who exited
between July 1, 2005 and June 30, 2006.

4

I. OVERVIEW
B. OVERVIEW OF DATA VALIDATION
Data validation is designed to accomplish the following goals:
1. Detect and identify data problems with a state’s TAPR to enable the state
to correct the problems.
2. Ensure that critical performance data used to direct incentives and
sanctions and to meet ETA’s Government Performance and Results Act
(GPRA) responsibilities are reasonably accurate by calculating an error
rate for each data element validated.
3. Provide tools that help states analyze the causes of their performance
successes and failures by displaying participant data organized by
performance outcomes.
4. Minimize the burden on states in conducting the validation by providing
standardized software that displays the individual participant records by
outcome and provides built in random samples, validation worksheets,
and automated validation reports.
5. Further minimize the burden on the states by selecting the smallest
possible validation samples necessary to compute valid error rates.

Data validation typically involves two distinct processes—report validation and
data element validation. Report validation assesses the accuracy of state reports
by comparing the values calculated by states to values calculated by the TAADV
software. Because ETA itself calculates performance measures for all TAA
participants, report validation is unnecessary. However, state staff can use the
TAADV software to view and analyze the performance of individual exiters. The
TAADV software also calculates the state’s overall performance and generates a
performance measure report.

5

I. OVERVIEW
Data element validation confirms the accuracy of key data elements in the TAPR
by examining a sample of participant records to assess whether the data in the
sampled records are correct. When a record is selected for validation, state staff
compare specified data elements in the TAPR record to source documentation for
that participant.
Data element validation results in an estimate of the error rate for each data
element selected for validation. Data elements are selected for validation based on
three factors:
• Feasibility—ETA can validate data elements only where it is practical and
efficient to locate and examine supporting evidence within the state case
files. Therefore, such items as race, ethnicity, and gender will not be
validated because these data elements are self-reported by participants and it
is not practical to locate the participant to verify these characteristics.
• Risk—The process for data validation is based partly on the risk that the data
element can be inaccurate. Data elements involving human judgment are
more prone to error than data elements that do not involve human judgment.
For example, determination of employment based on supplemental sources
is higher risk than determination of employment from wage records.
• Importance—Data elements are selected for validation based primarily on
their importance to the integrity of the individual participant records and
their significance for generating performance outcomes.
Error rates are computed by determining the degree to which the accuracy of
the selected data elements is supported by evidence in the case files or by other
sources (such as wage records).

6

I. OVERVIEW
The validation process is designed to compute a reliable error rate using the
smallest possible sample to minimize the state’s burden in performing the
validation. To accomplish this objective, three sampling techniques are used:

1) To lessen the relative burden on smaller states as much as possible, the
precision standards are reduced yielding a smaller sample than for larger
states.
2) Sampling of local offices is performed prior to the selection of participant
records to reduce the number of locations that state staff must visit to
access supporting documentation.
3) Participant records are over-sampled if they are determined to be
important for calculating performance outcomes and if they have a higher
risk of error. The software weights the results of the validation to correct
for over- and under-sampling so that the final results represent the overall
error rate of each data element in the state’s TAPR.

7

I. OVERVIEW
C. HANDBOOK OVERVIEW
The following chapters and appendices guide states through the process of
validating data from their TAPR submission using the TAADV Software.
• Chapter II—Validation Procedures
Chapter II provides a step-by-step description of how to conduct data
validation using the TAADV software. This chapter describes how to build
and import a validation extract file, validate the sample that is selected by
the software, and generate performance measure and summary and
analytical reports.
• Chapter III—Record Layout
Chapter III provides the record layout specification for creating the TAA
validation extract file.
• Appendix A—Performance Measure Specifications
Appendix A provides detailed performance measure specifications used by
ETA and by the TAADV software to generate the TAA performance
measures.
• Appendix B—DEV Sampling and Error Rate Estimation
Appendix B provides the specifications for sampling and error rate
estimation that are used by the TAADV software to select the data element
validation sample, and to calculate error rates on the summary and analytical
report.
• Appendix C—DEV Instructions
Appendix C provides the instructions used by the validator to validate each
data element on the data element validation worksheet.

8

I. OVERVIEW
• Appendix D—TAADV Software Installation
Appendix D provides a step-by-step description of how to install the
TAADV software on a PC.
• Appendix E—TAADV Software Reference Guide
Appendix E provides a condensed list of all the menus and functions of the
TAADV software for quick reference.

D. PREPARING FOR VALIDATION
As indicated, Chapter II of this handbook describes each task that states should
follow to complete validation. Responsibility for completing these tasks will be
divided among various staff.
Managers are responsible for assuring that programmers and validators have the
resources needed to complete the validation as required by ETA. They are also
responsible for keeping the data validation effort on schedule.
Programmers have the primary responsibility for building the validation extract
file in the format specified in the record layout. Programming staff may also be
responsible for loading the file into the TAADV software.
Validators conduct the validation once the extract file has been imported into the
software. Validators should also work closely with programming staff to
determine which state data elements best meet the requirements specified in the
record layout.

9

I. OVERVIEW
Table I.2 summarizes the preparation tasks that states should complete before
embarking on the validation effort, and the staff who will likely be responsible for
completing each preparation task. All of these tasks can begin immediately, so that
the state can maximize the time available to validate its sample of TAPR records.
Table I.2
Preparing for Data Validation
Preparation Activity

Responsible Staff

Handbook Reference

Assemble validation team.

Managers

Not applicable

Review handbook.

Validators, programmers

All chapters and appendices

Install TAADV software.

Programmers

Appendix D

Develop a data validation
schedule and make staff
assignments.

Managers, validators,
programmers

Chapter II

Build and test validation
extract file of all FY2006
TAA exiters. Final file should
be generated as soon as
possible after the final FY
submission in November,
2006, to maximize the time
available to conduct data
validation.

Programmers, validators

Chapter II—Sections A and B,
Chapter III

Review data validation
instructions and update state
version of sources column.

Validators, programmers

Appendix C

After reviewing this handbook, states should contact Mathematica Policy
Research, Inc. by emailing [email protected] with any questions on
validating their TAPR submission.

10

II. VALIDATION PROCEDURES
This chapter provides a step-by-step description of how to conduct data validation.
Following are brief summaries of each section of the chapter.

A. Accessing the TAADV Software
Section A describes how to access the TAADV software and how to identify the
reporting period to be validated. This section also describes how the user can
change the reporting options after the initial sign in.

B. Creating and Importing a Validation Extract File
Section B describes how to build and import a fiscal year file of TAA exiters. This
section also describes how to access and review duplicate detection and error
reports of records rejected by the validation software during the import process.

C. Report Validation
Section C describes how to review the performance measure report generated by
the software, and how to access and analyze records by performance outcome
groups.

D. Data Element Validation
Section D describes how to conduct data element validation for a sample of exiter
records, using the worksheets generated by the software. This section also
describes how to access the summary and analytical report and submit results to
ETA. Section D concludes with a summary table and flowchart of the DEV steps.

11

II. VALIDATION PROCEDURES
A. Accessing the Validation Software
Task 1—Install the Application
Users should refer to Appendix D for detailed instructions on installing the
TAADV software.

Task 2—Open the Validation Software
Go to the Start menu, click on Programs, and then scroll to find TAA. Click on
TAA to open the software.

Task 3—Create a New Database or Open an Existing Database
To create a new database, go to File and click on New. Then use the Create
Database window to find the appropriate location to save the database. Type a
name for the database in the file name cell.
To open an existing database, go to File and click on Open. Use the Create
Database window to find the appropriate database. Click on the database name
and then click on Open.

12

II. VALIDATION PROCEDURES
Task 4—Sign In Screen
On the Sign In screen, the user can either choose to validate records for an entire
fiscal year or for one of the four quarters within that fiscal year.
Enter the fiscal year to be validated in the cell labeled Fiscal Year. Move the
cursor to any other cell and the validation period start and end date range will
automatically fill based on the fiscal year entered. The start and end date range is
the period during which participants must have exited the TAA program to be
included in the TAPR for the fiscal year. As specified in Tables I.1 and I.2, the FY
2006 validation extract file must contain records for all participants who exited
between July 1, 2004 and June 30, 2005. The FY 2007 file must contain records
for all participants who exited between July 1, 2005 and June 30, 2006.
Alternatively, users can leave the Fiscal Year blank and manually enter a date
range, as long as it is within the range of exit dates in the validation file.

Click on the “Sign In” button after specifying the FY or date range to be validated.

13

II. VALIDATION PROCEDURES
Task 5—Change Reporting Options
To change the reporting parameters after signing in to the software, open the
Change Reporting Options menu and select Change Reporting Options.
In the Change Reporting Options screen, the user can select and change several
options: the fiscal year, period start and end date, office name, WIB name, and/or
case manager’s name. This function enables the state to produce substate level
performance measure reports based on the options selected.
While date ranges are auto filled based on the fiscal year selected, users may
overwrite these date ranges to analyze performance for specific time periods. If the
user decides to change the fiscal year after importing the data, the performance
measure report and the performance outcome groups will not accurately reflect the
new selection. For the software to calculate an accurate report, the user must reimport the data after changing the fiscal year.
The WIB, office, and case manager drop down menus include the unduplicated
values in each of these fields on the import file. After the user selects filters from
the drop down menus and clicks on Save, the source table, performance outcome
group table, and performance measure report will all be reconfigured to reflect the
data and results for the selected subgroup. The selected subgroup appears in the
footer of the performance measure report to distinguish it from the state level
report. Users should note that changing these filters does not impact the sample of
exiters selected for validation.

14

II. VALIDATION PROCEDURES
After selecting filters to create the desired report, click on Save to save the options.

15

II. VALIDATION PROCEDURES
B. Create and Import a Validation Extract File
States should consolidate the four TAPR submissions for the Fiscal Year being
validated, and import the file into the TAADV software.
Task 1—Create a File Based on the Record Layout
To view the record layout see Chapter III of this handbook, or open the Import
Data menu and select Source Table Record Layout. The record layout is in the
TAPR format with the addition of six fields. The additional fields are observation
number (field #1), sampling unit (field #60), WIB name (field #61), office name
(field #62), case manager (field #63), and user field (field #64).
The user field and the sampling unit are user-defined fields. The user field
appears on the validation worksheet and can be filled with any additional data
element that the state wishes to have available on the DEV worksheet. For
example, states can include the participant name or other identifying information
that should appear on the validation worksheet to help locate case folders for
sampled participants. It is not mandatory that you use this field.
The sampling unit field is a numeric designator for the physical location of the case
file, and is generally an office or WIB. This field is used by the TAADV software
to create a clustered sample. If the state does not currently have a numeric case file
location designator, it should create one for each case file location. This is not a
mandatory field, so the software will not reject a record if it is blank. However, the
software will treat all records with blanks in the sampling unit field as coming
from the same location and will not be able to create a clustered sample. No two
locations should have the same sampling unit number.
The exit date field (field #43) can represent either the date the participant exited
the TAA program, or the date the participant exited the WIA program if the
participant was coenrolled in WIA. However, for coenrolled participants states
should be sure to use the exit date that was used to obtain post-exit wages.

16

II. VALIDATION PROCEDURES
The extract file must be in ASCII comma-delimited format. The software will also
accept comma separated (CSV) files. Fields must be in the order and format listed
on the record layout. Mandatory fields are specified in the record layout. Blanks
are acceptable in optional fields. However, blank or null values are not valid for
mandatory fields and will result in the record being rejected. The software will
accept files with quotes around all elements or no elements, but not a mix of the
two.
Chapter I provides information about the date ranges for the exiters to be included
in the extract file. The extract file can be generated after the final quarterly file for
the fiscal year being validated has been submitted to DOL.
The validation extract file is equivalent to the four TAPR submissions for the fiscal
year, unduplicated using the most recent record for each participant. One way that
states can create the validation extract file is to aggregate all of their TAPR
submissions for the program year being validated. Before aggregating the files
states should mark each record to indicate which TAPR submission it came from
(e.g. 1st through 4th). States can then unduplicate the aggregated file by retaining
only the last record submitted for exiters with multiple records. The extract file
can be completed by adding observation numbers and sampling units to the
unduplicated records, and then extracting from the state data base the sampling
unit, WIB name, office name, case manager, and any user field applicable to each
record. The original marker for submission quarter may be retained in the user
field if states choose to do so.
Alternatively, states may develop an extract routine to pull the most recent data for
each fiscal year exiter directly from the state’s data base, in the format specified in
the record layout. States using this approach should ensure that they run the
extract routine as close as possible to their fourth quarterly submission, to
minimize any differences between the data in the TAPR submission and in the
validation extract file.

17

II. VALIDATION PROCEDURES
Task 2—Import Extract Data
Once the data are formatted according to the record layout, open the Import Data
menu and select Import from Extract File. Select the file to be imported using
the Select File box. Click Open to import the extract file.
Once this is done, the Import File box will display counters for both the numbers
of records imported successfully and the number of records with errors. The
Import File box also includes a cancel button that can be used to stop the loading
process. It may take several minutes to import the data, depending on the size of
the file. Click Exit when a message appears that the import is complete.

18

II. VALIDATION PROCEDURES
To view the imported file open the Import Data menu and select View Source
Table. This is a read only screen. However, the columns in the source table are
sortable to facilitate state review of the imported records. Users can sort on any
column in the source table.
The total number of records in the source table is provided on the lower left corner
of the screen. If the cursor is placed on a particular row of a column, the software
will show that row number in red at the bottom of the screen. This feature enables
the user to easily count the number of records with a particular characteristic in the
source table. For example, on the following source table the cursor is on row 7 as
shown on the lower right.

19

II. VALIDATION PROCEDURES

20

II. VALIDATION PROCEDURES
Task 3 – Review Error Report
During file import, the TAADV software reads each record to ensure that all fields
are valid based on the record layout and the TAPR edit check logic specified in
Chapter III. For example, any records with missing data in mandatory fields—
such as Observation Number or Date of Birth—are rejected. Duplicate records or
records with invalid values are also rejected. The TAADV will also generate an
error if the number of columns in the extract file does not match the number of
columns specified in the record layout, if text values are placed in a number field,
or if dates are not in the proper format.
The TAADV produces an error report listing the rejected records and records with
warning errors. To view the latest error report, click on the Import Data menu
and then select View Latest Error Report. If no errors are found, no error report
is produced. The error report is also automatically saved to the TAA folder
(C:\Program Files\TAA) and is titled importerror.txt. To save each error report,
the importerror.txt file should be renamed so that it is not overwritten by
subsequent error reports.
The error report indicates which observation has an error and the corresponding
error message from Chapter III. States should use the error report in conjunction
with the record layout and edit check document in Chapter III to determine why
records have been rejected or identified as having warning errors.
After reviewing any error reports generated by the software, staff should determine
if the validation file must be regenerated or reformatted and reloaded. If a very
small number of records are rejected, it may not be necessary to re-import the file
before beginning validation.

21

II. VALIDATION PROCEDURES
Task 4—View Duplicates
This task allows the user to view the duplicate records identified by the software.
No data entry is required, this option is for analytical purposes only.
Click on the Import Data menu and select View Duplicates.
This opens a report that displays the duplicate records, including their observation
number, SSN, and Exit Date. All records that have the same SSN and Exit Date
are considered to be duplicates and are rejected by the software. The user must
determine if it is necessary to fix the extract. If so, the user must determine which
observations to return to the extract file and then reimport the data before
beginning validation.

22

II. VALIDATION PROCEDURES
C. Report Validation
Because ETA calculates performance measures for all TAA exiters, report
validation is unnecessary. However, state staff can use the TAADV software to
view and analyze the performance of individual exiters. The TAADV also
calculates the state’s performance and generates a performance measure report.

Task 1—View Performance Outcome Groups
The TAADV software assigns each participant record to a performance outcome
group following the specifications in Appendix A of this handbook. Organizing
the records into outcome groups serves two purposes. First, it provides the
numerical counts used to calculate the performance measures. Second, it is a useful
tool for analyzing performance outcomes and factors that may influence
performance. Appendix A describes in detail how the software assigns exiters who
received employment and training services to 25 performance outcome groups for
the entered employment, retention, and average earnings measures.
Click on the Report Validation menu. Then click on View Performance
Outcome Groups. This opens a window that displays the 25 performance
outcome groups and the total number of records that have been assigned to each
group. A section of the performance outcome group summary screen is shown
below. Exhibit A.1 in Appendix A displays the full performance outcome group
table. No data entry is required; this function is for analytical purposes only.

23

II. VALIDATION PROCEDURES

Double click on the arrow in the far left column of each performance outcome
group to view the detailed records in that group.

24

II. VALIDATION PROCEDURES
Task 2—View Detailed Records
After double clicking the arrow in the far left corner of a performance outcome
group, the user can view the detailed records in that group.

Sort Detailed Records
To sort records in ascending or descending order, double-click the heading of the
column you wish to sort.

Widen Columns
To increase the width of the columns, go to the intersection of the relevant column
heading, click the mouse button and drag to the desired width.

25

II. VALIDATION PROCEDURES
Task 3—View Performance Measure Report
This function displays a report that calculates the values for the three TAA
performance measures, based on the specifications in Appendix A of this
handbook.
To view the performance measure report, click on the Report Validation menu
and select View Performance Report. No data entry is required; this is a readonly function.
Click on the printer icon to print the report using a live printer. To produce the
report in Adobe Acrobat (PDF) format, click on the export report button to the
right of the printer icon and follow the prompts.

If users have selected a WIB, office and/or case manager filter on the Change
Reporting Options screen, the performance measure report will be generated for
that subgroup. The footer of the report will display the selected subgroup.

26

II. VALIDATION PROCEDURES
D. Data Element Validation
Task 1—Access Sampled Records
The first tasks in DEV are to identify the records that must be validated and to
assemble the validation worksheets and source documentation corresponding to
those records.
The software selects a sample of the TAA exiter records that have been imported
into the software, and presents the data for each sampled record on a worksheet.
Appendix B of this handbook provides the validation sampling specifications.
Click on the Data Validation menu and select Edit Worksheets.
This will open the data validation worksheet summary of sampled records. This
summary screen is read only. Identifying information on this summary list (and on
the individual worksheet for each sampled record) helps state staff locate the
source documents for the selected records. The summary rows for sampled records
that have been completely validated will be shaded in purple.

27

II. VALIDATION PROCEDURES

Summary Worksheet

To access, update, and print individual sampled records, double click on the row of
the applicable record on the worksheet summary. This will open a validation
worksheet for the selected record.

28

II. VALIDATION PROCEDURES
Users can also search for worksheets that contain specific data by clicking the right
mouse button in the field in which they wish to search. A Find window will open.
Enter in the value that the TAADV should search for in the sampled records. If the
value needs to match exactly, check the Exact Match box. Then click the “Find
Next” button. The location of the record that contains the value in the “Find
What” box will be listed in blue.

Find Window

29

II. VALIDATION PROCEDURES
The online worksheet is on two tabs, one for validation fields 1 through 17 and one
for comments. Identifying information at the top of the worksheet helps validation
staff locate the case file for the selected record.
The validation worksheet lists all of the data elements to be validated. Validation
is required for each data element where check boxes are present and unshaded on
the worksheet. States do not need to validate data elements when check boxes are
shaded.

Sample Online Data Validation Worksheet

30

II. VALIDATION PROCEDURES
The validation worksheets can be completed online on a desktop or laptop
computer, or they can be printed and completed by hand. The TAADV may also
be loaded onto the state’s central server, allowing multiple validators to complete
the worksheets on workstations or computers connected to the server. If validators
use paper worksheets, the findings must later be data entered in the online
worksheet.
Task 4 of this section provides detailed instructions for completing the online
worksheet. “Not Completed” is displayed in red in the upper left corner of the
online worksheet until the validator completes every data element on the worksheet
that requires validation. At that point the message changes to “Completed”.

31

II. VALIDATION PROCEDURES
Task 2—Print Worksheets
The validator must assemble the printed worksheets for the sampled records. The
software sorts the worksheets by WIB and then by office within each WIB to
simplify the task of identifying which offices must be visited, and which cases
must be validated at each location.
To print all of the worksheets in batch mode, click on the Data Validation menu
and then click on Print Worksheets. All worksheets for the sampled records are
formatted for printing. Click the printer icon at the top left of the screen to begin
printing. The sampled records will print by WIB.

Sample Printed Worksheet

32

II. VALIDATION PROCEDURES
Task 3—Print Sampled Counts Report
To simplify the task of identifying which offices must be visited and which case
files must be reviewed at each location, the TAADV software sorts the worksheets
by WIB and then by office within each WIB. The TAADV produces a Sampled
Counts Report that lists the number of records in the sample by WIB name, office
name, and sampling unit.
States are also required to validate records on-site for the Workforce Investment
Act (WIA) program. Like the TAADV, the WIA Data Reporting and Validation
System (DRVS) software organizes worksheets by WIB and by office within WIB.
These identical approaches may reduce the burden on the state validator if the
TAADV and the WIA DRVS both select records from the same office. This
reduction in burden, however, may be limited. For the samples to be statistically
valid, each program must sample offices and records independently of each other.
Thus, it is likely that some TAA records will be selected from offices, and possibly
WIBs, that have no sampled records for WIA.
To obtain the Sampled Counts Report, click on the Data Validation menu and
then click on Print Sampled Counts Report. This report shows the distribution
of the sample by location. No data entry is required, this report is for analytical
purposes only.

33

II. VALIDATION PROCEDURES

34

II. VALIDATION PROCEDURES
Task 4—Assemble Supporting Documentation and Complete Worksheets
State staff must obtain the source documentation for each sampled record in order
to complete the validation worksheets. If some of the information is not available
at the state level, the validator must travel to the physical location where the case
files are stored to access the relevant source documentation for each sampled
record. (If the state wishes to follow a different procedure, it must obtain approval
from its regional officer.) The validator reviews the local office’s sampled case
files for each data element present on the worksheet, using the validation
instructions and criteria in Appendix C. Some elements (such as wage records)
may be validated against central computer files. Instructions for validating
elements against central computer files are also included in Appendix C.
Many of the data elements will be blank on the worksheet due to two factors. First,
not every data element applies to each participant. For example, if the participant
has not entered training, the data element for date entered training will be blank on
the worksheet for that participant. Similarly, if employment was established
through supplemental sources, the data element for wages will be blank on the
worksheet.
Second, except for elements that denote employment during the first and third
quarter after exit, negative values are generally not validated and therefore not
included on the worksheets. For example, if the participant did not receive a
relocation allowance (i.e. has a value of “2”), the data element will be blank on the
worksheet.
Blank data elements do not need to be validated unless the instructions specifically
direct the validator to do so. The validator should carefully read the instructions
for each data element because they indicate when blank data elements must be
validated. For example, all earnings fields are validated whether or not they are
blank.
Appendix C provides separate instructions for each data element; the instructions
are presented in the same order in which the data elements appear on the
worksheet. The data element reference numbers on the worksheet correspond to

35

II. VALIDATION PROCEDURES
the data element reference numbers in the validation instructions. Each instruction
specifies the recommended sources for validating the data element. If the validator
locates equivalent source documentation that is not specified in the instructions,
the validator should record the new source in the state version of sources column in
the validation instructions.
After reviewing the source documentation and following the validation
instructions, the validator records the result in the appropriate checkbox for each
element. The two possible validation outcomes for each data element are:
1. Check Pass if the element was supported by/matched the source
documentation
2. Check Fail if the source documentation showed that the data element was
incorrect or that no source documentation was available
The validation worksheets can be completed online by one or more validators
accessing the same database (see Task 8 of Appendix D for more information on
setting this up). However, if validators complete paper worksheets, the results must
later be data entered into the corresponding online worksheet.
The worksheet also includes a comments section. Any comments related to the
validation of a particular record can be entered in the comments text box on the
second tab of the online worksheet. These comments will then appear on the
printed version of the worksheet. Comments can include the reason a data element
failed validation, definitional problems encountered by the validator, the
validator’s name if the state chooses to record it, or other relevant information.
After completing the Pass or Fail checkbox for each data element to be validated in
a sampled record, and entering any relevant comments, the validator should click
on the “Save and Exit” button on the online worksheet (as shown in Task 1 of this
section).
Follow this same series of tasks for each record in the sample.

36

II. VALIDATION PROCEDURES
Task 5—Data Validation Summary/Analytical Report
The software generates a summary and analytical report after all the data validation
sample worksheets have been completed. The report shows the number of errors
and the error rates for each data element validated.
Two types of error rates—Overall Error Rate and Reported Data Error Rate—are
calculated using the information entered on the worksheets by the state validators.
For each data element, the Overall Error Rate is the number of records in which
that data element is in error divided by the total number of records sampled,
weighted to account for the over- and under-sampling of particular types of
records. As particular data elements may not be present in every record sampled,
the Reported Data Error Rate includes in the denominator only those records for
which the particular data element was validated. Please see Appendix B of the
validation handbook for more information about the two types of error rates.
To access the report, click on the Data Validation menu and select Print
Summary/Analytical Report. No data entry is required; this screen is for
analytical purposes only. To print the summary and analytical report, click on the
printer icon on the tool bar.

37

II. VALIDATION PROCEDURES

38

II. VALIDATION PROCEDURES
States may include comments on the summary and analytical report to describe
issues encountered during the validation. To do this, click on the Data Validation
menu and select Summary/Analytical Comments. In the text box, enter any
comments applicable to the data validation effort, up to 250 words. Click the
Update button for the comments to be transferred to the comments section on the
summary and analytical report. Click Cancel to return to previously entered
comments.

States should determine whether their error rates are acceptable or unacceptable for
each data element. Federal quality standards for determining unacceptable error
rates will be developed after sufficient data have been collected on the validation
results.

39

II. VALIDATION PROCEDURES
Task 6 – Submit Validation Results to ETA
After all validation worksheets have been completed, states must submit their
Summary and Analytical Report to ETA via the e-submit functionality of the
validation software. Data validation files, including the completed worksheets,
summary and analytical reports, and copies of supporting documentation, should
be retained on-site for three years for monitoring purposes.
Click on the Data Validation menu and select Export Summary/Analytical.
After a minute or two, the software will open a web browser. Enter the validation
password provided by Traci DiMartini at ETA ([email protected]) and
click on the Login button. The ETA-provided password must be the TAA
validation password, not the TAPR submission password.
Enter your name, e-mail address, and telephone number in the appropriate cells.
Then, click on Browse. Use the Choose File window to find the XML file created
by the software. The location and name of the file are listed in the white cell on
the top of the web browser. Click on the Open button. Then click the Submit
button in the web browser.

40

II. VALIDATION PROCEDURES
Path and file name of XML file

41

II. VALIDATION PROCEDURES
It may take up to a minute for the XML to upload. Once the file has been
successfully transmitted, the DRVS Confirmation of Submission screen will
appear. Users can choose to print a copy of the confirmation, have a copy e-mailed
to them, or logout.

42

II. VALIDATION PROCEDURES
Table II.1 and Figure II.1 summarize the TAA data element validation process.
Table II.1: Overview of Data Validation

Task

Task Description

1

The validator assembles validation worksheets for the sampled TAPR records and
retrieves the case files corresponding to those records.

2

After obtaining the case file for a sampled case, the validator uses the validation
worksheet to locate the first relevant data element that must be validated. Fields for
data that are not relevant to a particular record may be blank. For elements where
data are present or where the absence of data requires validation, the validator uses
the reference number to locate the validation instruction for that data element in
Appendix C. Each data element listed in Appendix A gives instructions and
acceptable source documentation for validating that element only. An edited
definition of the data element is also included in the instructions.

3

The validator obtains one or more of the source documents listed as an acceptable
source to validate the element.

4

Following the instructions in Appendix C, the validator determines whether the
data element meets the validation criteria based on the information in the source
document(s).

5

If the data element is supported by/matches the source document(s), the validator
enters a checkmark in the Pass box on the worksheet. If the source document(s)
does not support/match the data element or no source document(s) for the data
element is in the case file, the validator enters a checkmark in the Fail box on the
worksheet, indicating an error.

6

The validator proceeds through each data element for each sampled record in the
same manner. The “Comments” section of the worksheet can be used to record
notes or to document issues that may be helpful in future validations.

7

After each data element has been validated for every sampled record, the validator
reviews the Summary and Analytical Report generated by the software. The
validator then e-submits the report to ETA.

43

10
11

Date of exit

Employed in first full quarter
after exit

DV W orksheet is
printed or displayed
on a laptop

08

Ref. #

Trade Readjustment Allowance

Data Element

1

20031206

3

Reported
Value

Pass

1

Appendix C
List of Source
Docum ents

Fail

2

44

SUPPORTED

4

3

Enter a checkm ark
in the Pass box

E n t e r a c h e c k m a rk in t h e F a il b o x o n t h e
worksheet if the data elem ent in the validation
file is not supported by/does not m atch the
inform ation in the source docum ent(s), or if
there is no source docum entation in the case
file.

E n te r a c h e c k m a rk in th e P a s s b o x o n th e
worksheet if the data elem ent in the validation
file is supported by/m atches
the inform ation in
3
the source docum ent(s).

Instructions:

APPENDIX C
EXAMPLE

FIGURE II.I Flowchart of Data Validation Tasks

Subm it Sum m ary
and Analytical Report
to ETA

After com pleting all
validation worksheets,
review Sum m ary
and Analytical Report

6

W as data
elem ent
supported?

Determine if the data
element passes all
Determine
characteristic
validationif rules
using:
passes all validation rules
using:
1) W orksheet
2) Data Validation
1) W Instructions
orksheet
2)3)Data
ListElement
of Source
Validation
Instructions
Documents
3)4)Documents
Source Documents

7

Enter a checkm ark
in the Fail box

NOT
SUPPORTED

5

II. VALIDATION PROCEDURES

III. RECORD LAYOUT
Attached is the record layout that should be used to create the extract file of TAA
exiters as specified in Section B of Chapter II of this handbook. The file must be
in ASCII comma-delimited format or comma separated (CSV) format. Fields must
be in the specified order and in the format listed on the record layout. The
software will accept files with quotes around all elements or no elements, but not a
mix of the two.
The record layout provides the following information:
•
•
•
•
•
•
•
•
•
•

Field Number in TAADV validation file
TAPR field number
New TAPR data element name
Old TAPR field number
Old TAPR data element name
Valid values – the record will be rejected if the data element does not meet the
valid value criteria
Changes from old TAPR record layout to this new layout
Edit check logic – the record will be rejected or will receive a warning error if
the data element does not meet the edit check logic criteria
Error type – whether failure to meet the edit check logic results in a warning or
reject error
Error message – the text displayed on the import error report when the data
element fails to meet the edit check logic

45

46

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

New
TAPR
No.

2

1

Field
No.

Pell Grant Recipient

UC Eligible Status

Limited English
Language Proficiency

Employment Status

Eligible Veteran Status

White

Hawaiian Native or other
Pacific Islander

Black or African
American

Asian

American Indian or
Alaska Native

Ethnicity Hispanic /
Latino

Individual with a
Disability

Gender

Date of Birth

Individual Identifier

State FIPS Code

OBS

NEW DATA ELEMENT
NAME

1 = Male
2 = Female
Blank or 0 = did not self-identify

YYYYMMDD

XXXXXXXXX

Sequential number
XX

1 = Yes
2 = No
Blank or 0 = did not self-identify

15

14

13

12

11

10

1 = Employed
2 = Employed, but Received Notice of
Termination of Employment or Military
Separation
3 = Not Employed

1 = Yes, <= 180 days
2 = Yes, Eligible Veteran
3 = Yes, Other Eligible Person
4 = No

1 = Yes
Blank or 0 = did not self-identify

1 = Yes
Blank or 0 = did not self-identify

Pell Grant Recipient

Unemployment
compensation status

1 = Yes
2 = No

1 = Claimant Referred by WPRS
2 = Claimant Not Referred by WPRS
3 = Exhaustee
4 = Neither Claimant nor Exhaustee

Limited English Language 1 = Yes
Proficiency
2 = No

Veteran status

White

Hawaiian Native or other
Pacific Islander

Black or African American 1 = Yes
Blank or 0 = did not self-identify

9

1 = Yes
Blank or 0 = did not self-identify

Asian

American Indian or Alaska 1 = Yes
Native
Blank or 0 = did not self-identify

Ethnicity Hispanic or
Latino

CHANGES

EDIT CHECK LOGIC

No additional edit checks

No additional edit checks

No additional edit checks

A. Mandatory Field

A. Mandatory Field

A. Mandatory Field

A. Mandatory Field

Four (4) values in new TAPR replaces
three (3) values in old TAPR.

A. If TAPR Element #20 (Date of
Application) is less than 09/30/2006, then
accept "blank" on import.

New data element.

A. None. Accept
record as long as
condition is true.

A. Reject

A. Reject

A. Reject

A. Reject

A. Reject

ERROR TYPE

A. Mandatory Field

A. Mandatory Field

A. Mandatory Field

A. Reject

A. Reject

A. Reject

B. If TAPR Element #20 is greater than or B. Reject
equal to 10/01/2006, then reject records that
contain a "blank" on import. TAPR Element
#13 will essentially become a mandatory
field at that point.

A. Mandatory Field

Four (4) values in new TAPR replaces
three (3) values in old TAPR.

One (1) value in new TAPR replaces two No additional edit checks
(2) values in old TAPR.

One (1) value in new TAPR replaces two No additional edit checks
(2) values in old TAPR.

One (1) value in new TAPR replaces two No additional edit checks
(2) values in old TAPR.

One (1) value in new TAPR replaces two No additional edit checks
(2) values in old TAPR.

One (1) value in new TAPR replaces two No additional edit checks
(2) values in old TAPR.

Two (2) values in new TAPR replaces
three (3) values in old TAPR.

Two digit state code in new TAPR
replaces full state name in old TAPR.

Section I: Identification and Participant Characteristics

VALID VALUES

Individual with a Disability 1 = Yes
2 = No
Blank or 0 = did not self-identify

Gender

Date of Birth

Individual Identifier

OBS
State Name

OLD DATA ELEMENT
NAME

8

7

6

5

4

3

2

1

Old
TAPR
No.

TAA Edit Checks and Crosswalk

A. Pell Grant Recipient must be Yes (1) or No (2).

A. UC Eligible Status is missing or invalid.

A. Limited English Language Proficiency must be
Yes (1) or No (2).

B. Employment Status cannot be blank if Date of
Application (TAPR #20) is on or after 10/1/2006.

A. Eligible Veteran Status is missing or invalid.

A. Date of Birth is missing or invalid.

A. Individual Identifier is missing or invalid.

A. State code is missing or invalid.

A. OBS number is missing.

ERROR MESSAGE

47

New
TAPR
No.
17

18

19

20

21

22

23

Field
No.
18

19

20

21

22

23

24

Date of Program
Participation

Program Type

Petition Number

Date of Application

Tenure with Employer at
Most Recent Qualifying
Separation

Most Recent Date of
Qualifying Separation

NEW DATA ELEMENT
NAME
Highest School Grade
Completed

22

21

20

19

18

17

Date of Participation

Program of participation

Petition Number

Date of Application

Tenure with Employer at
Qualifying Separation

Most Recent Qualifying
Separation

Old
TAPR
OLD DATA ELEMENT
No.
NAME
16
Highest School Grade
Completed

YYYYMMDD

1 = TAA
2 = NAFTA-TAA
3 = Both TAA and NAFTA-TAA

00000000

YYYYMMDD

000

YYYYMMDD

VALID VALUES
00 = No school grades completed
01 - 12 = Number of
elementary/secondary school grades
completed
13 - 15 = Number of college, or full-time
technical or vocational school years
completed
16 = Bachelor's degree or equivalent
17 = Education beyond the Bachelor's
degree
87 = Attained High School Diploma
88 = Attained GED or Equivalent
89 = Attained Certificate of
Attendance/Completion
90 = Attained Other Post-Secondary
Degree or Certification
91 = Attained AS/AA Degree or Diploma

CHANGES
87 in new TAPR replaces 12 in old
TAPR.
89, 90, 91 are added in new TAPR.

A. Mandatory Field

A. Mandatory Field

Note: The TAPR on-line submission module
will accept values with and without the field
being padded with "0s".

A. Mandatory Field

A. Mandatory Field

Note: The TAPR on-line submission module
will accept values with and without the field
being padded with "0s". For example, the
system will accept values of "060" or "60"
for the number of months employed with the
employer.

A. Reject

A. Reject

A. Reject

A. Reject

A. Reject

B. Reject

B. Must be less than or equal to TAPR
Element #42 (Date of Exit)

A. Mandatory Field

A. Reject

ERROR TYPE
A. Reject

A. Mandatory Field

Note: The TAPR on-line submission module
will accept values with and without the field
being padded with "0s". For example, the
system will accept values of "01" or "1" for
the number of elementary/secondary school
grades completed.

EDIT CHECK LOGIC
A. Mandatory Field

TAA Edit Checks and Crosswalk

A. Date of Program Participation is missing or
invalid.

A. Program Type is missing or invalid.

A. Petition Number is missing or invalid.

A. Date of Application is missing or invalid.

A. Tenure with Employer at Most Recent
Qualifying Separation is missing or invalid.

B. Most Recent Date of Qualifying Separation is
after the Date of Exit (TAPR #42).

A. Most Recent Date of Qualifying Separation is
missing or invalid.

ERROR MESSAGE
A. Highest School Grade Completed is missing or
invalid.

48

New
TAPR
No.

24

25

26

27

28

29

30

31

32

33

34

35

36

Field
No.

25

26

27

28

29

30

31

32

33

34

35

36

37

Remedial Trade
Readjustment
Allowances (TRA)

Additional Trade
Readjustment
Allowances (TRA)

Basic Trade
Readjustment
Allowances (TRA)

Received Customized
Training

Received Remedial
Training

Received On-the-Job
Training

Received Occupational
Skills (classroom)
Training

Occupational Skill
Training Code

Subsistence While in
Training

Travel While in Training

Training Completed

Date Completed, or
Withdrew from, Training

Date Entered Training

NEW DATA ELEMENT
NAME

33

32

31

Trade Readjustment
Allowances (TRA)

Received Remedial
Training

Received On-the-Job
Training

Received Occupational
Skills (classroom)
Training

Occupational code system

29
30

Occupational Skill
Training Code

Subsistence While in
Training

Travel While in Training

Training Completed

Date Completed, or
Withdrew from, Training

Date Entered Training

OLD DATA ELEMENT
NAME

28

27

26

25

24

23

Old
TAPR
No.

YYYYMMDD
Blank = did not receive training services

1 = Yes
2 = No

1 = Yes
2 = No

1 = Yes
2 = No

1 = Yes
2 = No
Blank or 0 = did not receive training
services

1 = Yes
2 = No
Blank or 0 = did not receive training
services

1 = Yes
2 = No
Blank or 0 = did not receive training
services

1 = Yes
2 = No
Blank or 0 = did not receive training
services

00000000
Blank = code is not available or did not
receive training service

1 = Yes
2 = No
Blank or 0 = did not receive training
services

1 = Yes
2 = No
Blank or 0 = did not receive training
services

1 = Yes
2 = No
Blank or 0 = did not receive training
services

CHANGES

A. Must be 1 or 2 if TAPR Element #24
(Date Entered Training) has a valid date.

A. Must be 1 or 2 if TAPR Element #24
(Date Entered Training) has a valid date.

A. Must be 1 or 2 if TAPR Element #24
(Date Entered Training) has a valid date.

A. Must be 1 or 2 if TAPR Element #24
(Date Entered Training) has a valid date.

Note: The TAPR on-line submission module
will accept values with and without the field
being padded with "0s".

A. Mandatory Field

New data element.
A. Mandatory Field
(Replaces old TRA field values 3 and 4.)
New data element.

A. Reject

A. Reject

A. Reject

C. Reject

B. Reject

A. Reject

B. Reject

A. Reject

ERROR TYPE

A. Reject

A. Reject

A. Reject

A. Reject

A. Reject

A. Reject

A. Reject

A. Must be blank or 00000000 if TAPR #24 A. Reject
(Date Entered Training) is blank

A. Must be 1 or 2 if TAPR Element #24
(Date Entered Training) has a valid date.

A. Must be 1 or 2 if TAPR Element #24
(Date Entered Training) has a valid date.

A. Must be 1 or 2 if TAPR Element #24
(Date Entered Training) has a valid date.

C. Must be less than or equal to TAPR
Element #42 (Date of Exit) if a valid date
exists.

B. Must be greater than or equal to TAPR
Element #24 if TAPR Element #24 has a
valid date.

A. Must be blank if TAPR Element #24
(Date Entered Training) is blank.

B. Must have a valid date if either TAPR
Element #30 or #31 or #32 or #33 is value
1.

Old Trade readjustment allowances
A. Mandatory Field
(TRA) data element was split into two
data elements in the new TAPR.
(Replaces old TRA field values 1 and 2.)

New data element.

Deleted in new TAPR

Numeric code in new TAPR replaces
alphanumeric code in old TAPR.

EDIT CHECK LOGIC
A. Must be blank or less than or equal to
TAPR Element #42 (Date of Exit).

Section II: Activity and Service Record
YYYYMMDD
Blank = did not receive training services

VALID VALUES

TAA Edit Checks and Crosswalk

A. Remedial Trade Readjustment Allowances
must be Yes (1) or No (2).

A. Additional Trade Readjustment Allowances
must be Yes (1) or No (2).

A. Basic Trade Readjustment Allowances must be
Yes (1) or No (2).

A. Received Customized Training must be Yes (1)
or No (2) if a Date Entered Training (TAPR #24) is
specified.

A. Received Remedial Training must be Yes (1) or
No (2) if a Date Entered Training (TAPR #24) is
specified.

A. Received On-the-JobTraining must be Yes (1)
or No (2) if a Date Entered Training (TAPR #24) is
specified.

A. Received Occupational Skills Training must be
Yes (1) or No (2) if a Date Entered Training (TAPR
#24) is specified.

A. Occupational Skill Training Code is specified,
but no Date Entered Training (TAPR #24) is given.

A. Subsistence While inTraining must be Yes (1)
or No (2) if a Date Entered Training (TAPR #24) is
specified.

A. Travel While in Training must be Yes (1) or No
(2) if a Date Entered Training (TAPR #24) is
specified.

A. Training Completed must be Yes (1) or No (2) if
a Date Entered Training (TAPR #24) is specified.

C. Date Completed or Withdrew From Training is
after the Date of Exit (TAPR #42).

B. Date Completed or Withdrew From Training is
before the Date Entered Training (TAPR #24).

A. Date Completed or Withdrew From Training is
specified, but no Date Entered Training (TAPR
#24) is given.

B. On-the-Job (TAPR #31), Remedial (TAPR #32),
or Customized (TAPR #33) Training is indicated,
but no Date Entered Training is given.

A. Date Entered Training is after the Date of Exit
(TAPR #42).

ERROR MESSAGE

49

New
TAPR
No.
37

38

39

40

41

Field
No.
38

39

40

41

42

Other Federal CoEnrollment

Date of WIA Title IB
Participation

Relocation Allowance

Job Search Allowance

NEW DATA ELEMENT
NAME
Waiver from Training
Requirement

38

37

36

35

Other Federal CoEnrollment

Date of registration

Relocation Allowance

Job Search Allowance

Old
TAPR
OLD DATA ELEMENT
No.
NAME
34
Waiver from Training
Requirement

1 = WIA Dislocated Worker
2 = Nat'l Emergency Grant
3 = Both 1 and 2 above
4 = Other Federal Employment and Job
Training Program (in addition to either 1 or
2 or both)
5 = Other Federal Employment and Job
Training Program (not including either 1 or
2)
0 = not enrolled in any other Federal
program

YYYYMMDD
Blank = did not receive services funded by
a WIA Title 1B or National Emergency
Grant program

1 = Yes
2 = No

1 = Yes
2 = No

B. Must be 1, 2, 3, 4, or 5 if TAPR Element B. Reject
#40 (Date of WIA Title IB Participation) has
a valid date.

A. Reject

A. Reject

A. Must be less than or equal to TAPR
Element #42 (Date of Exit) if a valid date
exists
A. Mandatory Field

A. Reject

A. Reject

ERROR TYPE
A. Reject

A. Mandatory Field

A. Mandatory Field

VALID VALUES
CHANGES
EDIT CHECK LOGIC
1 = Recall
Seven (7) values in new TAPR replaces A. Mandatory Field
2 = Marketable Skills
two (2) values in old TAPR.
3 = Retirement
4 = Health
5 = Enrollment Unavailable
6 = Training Not Available
7 = Reason Unknown/Served Prior to 2002
Amendments
0 = did not receive training waiver

TAA Edit Checks and Crosswalk

B. Field value for Other Federal Co-Enrollment
must be 1-5 if a Date of WIA Title IB Participation
(TAPR #40) is specified.

A. Other Federal Co-Enrollment is missing or
invalid.

A. Date of WIA Title IB Participation is after the
Date of Exit (TAPR #42).

A. Relocation Allowance must be Yes (1) or No
(2).

A. Job Search Allowance must be Yes (1) or No
(2).

ERROR MESSAGE
A. Waiver from Training Requirement is missing
or invalid.

50

New
TAPR
No.

42

43

44

45

46

47

Field
No.

43

44

45

46

47

48

Employed in 2nd Quarter
After Exit Quarter

Occupational Code of
Employment (if
available)

Type of Employment
Match 1st Quarter After
Exit Quarter

Employed in 1st Quarter
After Exit Quarter

Other Reasons for Exit
(at time of exit or during
3-quarter measurement
period following the
quarter of exit)

Date of Exit

NEW DATA ELEMENT
NAME

Occupational Code (if
available)

Occupational code system

42

Employed in 1st full
quarter after exit

Date of Exit

OLD DATA ELEMENT
NAME

41

40

39

Old
TAPR
No.

1 = Yes
2 = No

00000000

1 = UI Wage Records (In-State & WRIS)
2 = Federal Employment Records (OPM,
USPS)
3 = Military Employment Records (DOD)
4 = Other Administrative Wage Records
5 = Supplemental through case
management, participant survey, and/or
verification with the employer
Blank or 0 = was not employed in the first
quarter after the exit quarter

1 = Yes
2 = No

Blank or 00 = exited for a reason other
than one of the conditions listed above

98 Retirement
99 Not a Valid SSN

01 Institutionalized
02 Health/Medical
03 Deceased
04 Family Care
05 Reserve Forces Called to Active Duty

YYYYMMDD

VALID VALUES

New data element.

Deleted in new TAPR.

Numeric code in new TAPR replaces
alphanumeric code in old TAPR.

New data element.

New data element.

Section III: Outcomes

CHANGES

C. Reject
C. Mandatory Field

B. Reject
B. Must be 2 if TAPR 48 is Blank or 0.

A. Reject

B. Reject

A. Must be 1 if TAPR 48 (Type of
Employment Match 2nd Quarter After Exit
Quarter) is 1-5.

Note: The TAPR on-line submission module
will accept values with and without the field
being padded with "0s".

No additional edits

B. Must be 0, 5, or blank if TAPR 54 is 0 or
blank.

A. Must be 1-4 if TAPR 54 (Wages 1st
Quarter After Exit Quarter) is >0 and
<999999.99.

A. Reject

C. Reject

B. Must be 2 if TAPR 45 is Blank or 0.
C. Mandatory Field

B. Reject

A. Must be 1 if TAPR 45 (Type of
Employment Match 1st Quarter After Exit
Quarter) is 1-5.
A. Reject

B. Reject

No additional edit checks

A. Reject

ERROR TYPE

B. Must be greater than or equal to TAPR
Element #23 (Date of Program
Participation)

EDIT CHECK LOGIC
A. Mandatory Field

TAA Edit Checks and Crosswalk

C. Employed in 2nd Quarter After Exit must be
Yes (1) or No (2).

B. Employed in 2nd Quarter After Exit must be No
(2) if Type of Employment Match is Not Employed
(blank or 0).

A. Employed in 2nd Quarter After Exit must be
Yes (1) if Type of Employment Match is Wage or
Employment Records (1-4) or Supplemental (5).

B. Type of Employment Match 1st Quarter After
Exit Quarter must be Supplemental (5) or Not
Employed (blank or 0) if Wages is blank or 0.

A. Type of Employment Match 1st Quarter After
Exit Quarter must be Wage or Employment
Records (1-4) if Wages > 0.

C. Employed in 1st Quarter After Exit must be Yes
(1) or No (2).

B. Employed in 1st Quarter After Exit must be No
(2) if Type of Employment Match is Not Employed
(blank or 0).

A. Employed in 1st Quarter After Exit must be Yes
(1) if Type of Employment Match is Wage or
Employment Records (1-4) or Supplemental (5).

B. Date of Exit is on or before the Date of
Program Participation (TAPR #23).

A. Date of Exit is missing or invalid.

ERROR MESSAGE

51

New
TAPR
No.
48

49

50

51

52

53

54

55

56

Field
No.
49

50

51

52

53

54

55

56

57

Wages 3rd Quarter After
Exit Quarter

Wages 2nd Quarter After
Exit Quarter

Wages 1st Quarter After
Exit Quarter

Wages 1st Quarter Prior
to Participation Quarter

Wages 2nd Quarter Prior
to Participation Quarter

Wages 3rd Quarter Prior
to Participation Quarter

Type of Employment
Match 3rd Quarter After
Exit Quarter

Employed in 3rd Quarter
After Exit Quarter

NEW DATA ELEMENT
NAME
Type of Employment
Match 2nd Quarter After
Exit Quarter

48

47

46

45

44

43

Old
TAPR
No.

000000.00
Blank = element does not apply

000000.00
Blank = element does not apply

Third quarter following
exit

000000.00
Blank = element does not apply

000000.00
Blank = element does not apply

First quarter following exit 000000.00
Blank = element does not apply

Second quarter following
exit

Two (2) values in new TAPR replaces
three (3) values in old TAPR.

CHANGES
New data element.

Eight (8) digit figure in new TAPR
replaces seven (7) digit figure in old
TAPR.

Eight (8) digit figure in new TAPR
replaces seven (7) digit figure in old
TAPR.

Eight (8) digit figure in new TAPR
replaces seven (7) digit figure in old
TAPR.

New data element.

Eight (8) digit figure in new TAPR
replaces seven (7) digit figure in old
TAPR.

Eight (8) digit figure in new TAPR
replaces seven (7) digit figure in old
TAPR.

1 = UI Wage Records (In-State & WRIS) New data element.
2 = Federal Employment Records (OPM,
USPS)
3 = Military Employment Records (DOD)
4 = Other Administrative Wage Records
5 = Supplemental through case
management, participant survey, and/or
verification with the employer
Blank or 0 = was not employed in the third
quarter after the exit quarter

1 = Yes
2 = No

VALID VALUES
1 = UI Wage Records (In-State & WRIS)
2 = Federal Employment Records (OPM,
USPS)
3 = Military Employment Records (DOD)
4 = Other Administrative Wage Records
5 = Supplemental through case
management, participant survey, and/or
verification with the employer
Blank or 0 = was not employed in the
second quarter after the exit quarter

Two quarters prior to most 000000.00
recent qualifying
Blank = element does not apply
separation

Three quarters prior to
most recent qualifying
separation

Employed in 3rd full
quarter after exit

OLD DATA ELEMENT
NAME

B. Reject

B. Warning

B. Warning

B. Warning

B. Must not be >50000.00 and <999999.99. B. Warning

A. Must be >=0 and <=999999.99 or blank. A. Reject

B. Must not be >50000.00 and <999999.99. B. Warning

A. Must be >=0 and <=999999.99 or blank. A. Reject

B. Must not be >50000.00 and <999999.99. B. Warning

A. Must be >=0 and <=999999.99 or blank. A. Reject

B. Must not be >100000.00 and
<999999.99.

A. Must be >=0 and <=999999.99 or blank. A. Reject

B. Must not be >100000.00 and
<999999.99.

A. Must be >=0 and <=999999.99 or blank. A. Reject

B. Must not be >100000.00 and
<999999.99.

A. Must be >=0 and <=999999.99 or blank. A. Reject

B. Must be 0, 5, or blank if TAPR 56 is 0 or
blank.

A. Must be 1-4 if TAPR 56 (Wages 3rd
Quarter After Exit Quarter) is >0 and
<999999.99.

A. Reject

C. Reject

B. Must be 2 if TAPR 50 is Blank or 0.
C. Mandatory Field

B. Reject

A. Reject

B. Reject

ERROR TYPE
A. Reject

A. Must be 1 if TAPR 50 (Type of
Employment Match 3rd Quarter After Exit
Quarter) is 1-5.

B. Must be 0, 5, or blank if TAPR 55 is 0 or
blank.

EDIT CHECK LOGIC
A. Must be 1-4 if TAPR 55 (Wages 2nd
Quarter After Exit Quarter) is >0 and
<999999.99.

TAA Edit Checks and Crosswalk

A. Wages 3rd Quarter After Exit Quarter are
invalid.
B. Wages 3rd Quarter After Exit Quarter cannot
be > 50000.

A. Wages 2nd Quarter After Exit Quarter are
invalid.
B. Wages 2nd Quarter After Exit Quarter cannot
be > 50000.

A. Wages 1st Quarter After Exit Quarter are
invalid.
B. Wages 1st Quarter After Exit Quarter cannot be
> 50000.

A. Wages 1st Quarter Prior to Participation are
invalid.
B. Wages 1st Quarter Prior to Participation cannot
be > 100000.

A. Wages 2nd Quarter Prior to Participation are
invalid.
B. Wages 2nd Quarter Prior to Participation
cannot be > 100000.

A. Wages 3rd Quarter Prior to Participation are
invalid.
B. Wages 3rd Quarter Prior to Participation cannot
be > 100000.

B. Type of Employment Match 3rd Quarter After
Exit Quarter must be Supplemental (5) or Not
Employed (blank or 0) if Wages is blank or 0.

A. Type of Employment Match 3rd Quarter After
Exit Quarter must be Wage or Employment
Records (1-4) if Wages > 0.

C. Employed in 3rd Quarter After Exit must be Yes
(1) or No (2).

B. Employed in 3rd Quarter After Exit must be No
(2) if Type of Employment Match is Not Employed
(blank or 0).

A. Employed in 3rd Quarter After Exit must be Yes
(1) if Type of Employment Match is Wage or
Employment Records (1-4) or Supplemental (5).

B. Type of Employment Match 2nd Quarter After
Exit Quarter must be Supplemental (5) or Not
Employed (blank or 0) if Wages is blank or 0.

ERROR MESSAGE
A. Type of Employment Match 2nd Quarter After
Exit Quarter must be Wage or Employment
Records (1-4) if Wages > 0.

52

WIB Name

Office Name

Case Manager

User Field

61

62

63

64

Participated in ATAA

Sampling Unit

58

59

NEW DATA ELEMENT
NAME
Recalled by Layoff
Employer

60

New
TAPR
No.
57

Field
No.
58

Old
TAPR
OLD DATA ELEMENT
No.
NAME
49
Recalled by Layoff
Employer

Text

Text

Text

Text

Number

1 = Yes
2 = No

1 = Yes
2 = No

VALID VALUES

New data element.

CHANGES

No additional edit checks

No additional edit checks

No additional edit checks

No additional edit checks

No additional edit checks

A. Mandatory Field

EDIT CHECK LOGIC
A. Mandatory Field

TAA Edit Checks and Crosswalk

A. Reject

ERROR TYPE
A. Reject

A. Participated in ATAA must be Yes (1) or No (2).

ERROR MESSAGE
A. Recalled by Layoff Employer must be Yes (1)
or No (2).

APPENDIX A

PERFORMANCE MEASURE SPECIFICATIONS

This appendix provides the specifications used by the software to assign the
TAA exiters to performance outcome groups and to calculate the entered
employment, retention, and average earnings measures. Performance is calculated
using the common performance measure calculation specifications used by the
WIA and Employment Service programs.
I. PERFORMANCE OUTCOME GROUP SPECIFICATIONS
As described in Chapter I, participants who exit the TAA program after
receiving employment and training services are included in three performance
measures:

• The Entered Employment measure calculates the percentage of exiters in the
fiscal year TAPR file who were employed in the first full quarter after the
exit quarter. This measure is calculated as follows:
Of those who are not employed at the date of participation:
# of participants who are employed in the first quarter (qtr) after the exit qtr
# of participants who exit during the qtr

• The Employment Retention measure calculates the percentage of those
exiters in the entered employment rate numerator who were still employed in
the second and third full quarters after exit. This measure is calculated as
follows:
# of participants who are employed in both the second and third qtrs after the exit qtr
# of participants who are employed in the first quarter after the exit qtr

A.2

• The Average Earnings measure is the total post-program earnings from
wage records in the second plus third quarters after exit divided by the
number of adults in the employment retention numerator for whom wage
records are available. This measure is calculated as follows:

[Total earnings in the second quarter plus total earnings in the third quarter after the
exit quarter]
# of participants who are employed in the first, second, and third quarters after the exit
qtr

The Performance Measure report generated by the TAADV software lists the
three TAA performance measures as well as the numerators and denominators used
to calculate each measure. The numerators and denominators are derived from the
number of participants assigned to each of the corresponding performance outcome
groups by the TAADV software.
Figure A.1 shows how the individual records in the TAPR file are assigned by
the software to one of 25 outcome groups based on the participants’ pre- and postprogram employment status, employment retention status, and the source used for
recording participant earnings. The TAADV determines the number of participants
who fit each outcome group and computes the three performance measures based
on the number of participants in each group.
Organizing the records into outcome groups serves two purposes. First, it
provides the numerical counts used in calculating the performance outcome
measures (see Appendix B for the detailed specifications). Second, it serves as a
useful tool for analyzing performance outcomes and factors that may influence
performance.
Columns 6 through 10 of Figure A.1 show how the count of participants in
each group, or the sum of their earnings for the average earnings measure, is used
in the calculation of each performance measure. For example, participants in group
1 are included in the numerators and denominators of the entered employment and
retention rates, and their earnings are included in numerator of the average
earnings measure.

A.3

A.4

Emp
Qtr+1

Emp
Status

UnEmp
UnEmp
UnEmp
UnEmp
UnEmp
UnEmp
UnEmp
UnEmp
UnEmp
UnEmp
UnEmp
UnEmp

Emp
Emp
Emp
Emp
Emp
Emp
Emp
Emp
Emp
Emp
Emp
Emp

NA

Group

1
2
3
4
5
6
7
8
9
10
11
12

13
14
15
16
17
18
19
20
21
22
23
24

25

NA

Emp
Emp
Emp
Emp
Emp
Emp
Emp
Emp
Emp
Emp
Emp
UnEmp

Emp
Emp
Emp
Emp
Emp
Emp
Emp
Emp
Emp
Emp
Emp
UnEmp

3

2

1

NA

Wage
Wage
Wage
Wage
Supp
Supp
Supp
Supp
NA
NA
NA
NA
NA

A.4

Emp
Emp
Emp
Emp
Emp
Emp
Emp
Emp
Emp
UnEmp
UnEmp
NA

Emp
Emp
Emp
Emp
Emp
Emp
Emp
Emp
Emp
UnEmp
UnEmp
NA

Emp
Qtr+2

Type
Match
Qtr+1
Wage
Wage
Wage
Wage
Supp
Supp
Supp
Supp
NA
NA
NA
NA

5

4

NA

Wage
Wage
Supp
Supp
Wage
Wage
Supp
Supp
NA
NA
NA
NA

Wage
Wage
Supp
Supp
Wage
Wage
Supp
Supp
NA
NA
NA
NA

Type
Match
Qtr+2

6

NA

Emp
Emp
Emp
Emp
Emp
Emp
Emp
Emp
UnEmp
Emp
UnEmp
NA

Emp
Emp
Emp
Emp
Emp
Emp
Emp
Emp
UnEmp
Emp
UnEmp
NA

Emp
Qtr+3

7

NA

Wage
Supp
Wage
Supp
Wage
Supp
Wage
Supp
NA
NA
NA
NA

Wage
Supp
Wage
Supp
Wage
Supp
Wage
Supp
NA
NA
NA
NA

Type
Match
Qtr+3

8

10

11

12

Yes

No
No
No
No
No
No
No
No
No
No
No
No

No
No
No
No
No
No
No
No
No
No
No
No

No

No
No
No
No
No
No
No
No
No
No
No
No

Yes
Yes
Yes
Yes
Yes
Yes
Yes
Yes
Yes
Yes
Yes
No

No

No
No
No
No
No
No
No
No
No
No
No
No

Yes
Yes
Yes
Yes
Yes
Yes
Yes
Yes
Yes
Yes
Yes
Yes

No

Yes
Yes
Yes
Yes
Yes
Yes
Yes
Yes
No
No
No
No

Yes
Yes
Yes
Yes
Yes
Yes
Yes
Yes
No
No
No
No

Other
Reasons
for Exit EER Num EER Den Ret Num

9

25 GROUPS OF TAA EXITERS USED TO CALCULATE
ENTERED EMPLOYMENT, RETENTION AND AVERAGE EARNINGS

EXHIBIT A.1

No

Yes
Yes
Yes
Yes
Yes
Yes
Yes
Yes
Yes
Yes
Yes
No

Yes
Yes
Yes
Yes
Yes
Yes
Yes
Yes
Yes
Yes
Yes
No

Ret Den

13

No

Yes
No
No
No
No
No
No
No
No
No
No
No

Yes
No
No
No
No
No
No
No
No
No
No
No

Earn

14

II. PERFORMANCE MEASURE CALCULATION SPECIFICATIONS
The following specifications detail how ETA calculates the three TAA
performance measures using the TAPR submission. These specifications are also
used by the TAADV software to calculate the three TAA measures.

Entered
Employment Rate

Numerator:
Count of unique RECORDS where (EMPLOYMENT STATUS AT
PARTICIPATION = 2 or EMPLOYMENT STATUS AT
PARTICIPATION = 3) and ((EMPLOYED 1ST QUARTER AFTER
EXIT QUARTER = 1 and TYPE OF EMPLOYMENT MATCH 1ST
QUARTER AFTER EXIT QUARTER = 5) or (WAGES 1ST
QUARTER AFTER THE EXIT QUARTER > 0 and WAGES 1ST
QUARTER AFTER THE EXIT QUARTER < 999999)) and DATE OF
EXIT is within the report period and (OTHER REASONS FOR EXIT =
00 or OTHER REASONS FOR EXIT = 98 or OTHER REASONS FOR
EXIT is null).
divided by
Denominator:
Count of unique RECORDS where EMPLOYMENT STATUS AT
PARTICIPATION is 2 or 3 and DATE OF EXIT is within the report
period and (OTHER REASONS FOR EXIT = 00 or OTHER REASONS
FOR EXIT = 98 or OTHER REASONS FOR EXIT is null).

A.5

Employment
Retention Rate

Numerator:
Count of unique RECORDS where ((EMPLOYED 1ST QUARTER
AFTER EXIT QUARTER = 1 and TYPE OF EMPLOYMENT MATCH
1ST QUARTER AFTER EXIT QUARTER = 5) or (WAGES 1ST
QUARTER AFTER THE EXIT QUARTER > 0 and WAGES 1ST
QUARTER AFTER THE EXIT QUARTER < 999999)) and
((EMPLOYED 2ND QUARTER AFTER EXIT QUARTER = 1 and
TYPE OF EMPLOYMENT MATCH 2ND QUARTER AFTER EXIT
QUARTER = 5) or (WAGES 2ND QUARTER AFTER THE EXIT
QUARTER > 0 and WAGES 2ND QUARTER AFTER THE EXIT
QUARTER < 999999)) and ((EMPLOYED 3RD QUARTER AFTER
EXIT QUARTER = 1 and TYPE OF EMPLOYMENT MATCH 3RD
QUARTER AFTER EXIT QUARTER = 5) or (WAGES 3RD
QUARTER AFTER EXIT QUARTER > 0 and WAGES 3RD
QUARTER AFTER EXIT QUARTER < 999999)) and DATE OF EXIT
is within the report period and (OTHER REASONS FOR EXIT = 00 or
OTHER REASONS FOR EXIT = 98 or OTHER REASONS FOR EXIT
is null).
divided by
Denominator:
Count of unique RECORDS where ((EMPLOYED 1ST QUARTER
AFTER EXIT QUARTER = 1 and TYPE OF EMPLOYMENT MATCH
1ST QUARTER AFTER EXIT QUARTER = 5) or (WAGES 1ST
QUARTER AFTER THE EXIT QUARTER > 0 and WAGES 1ST
QUARTER AFTER THE EXIT QUARTER < 999999)) and DATE OF
EXIT is within the report period and (OTHER REASONS FOR EXIT =
00 or OTHER REASONS FOR EXIT = 98 or OTHER REASONS FOR
EXIT is null).

A.6

Average Earnings

Numerator:
Sum of (WAGES 2nd QUARTER AFTER THE EXIT QUARTER and
WAGES 3rd QUARTER AFTER THE EXIT QUARTER) where
(WAGES 1ST QUARTER AFTER EXIT QUARTER > 0 and WAGES
1ST QUARTER AFTER EXIT QUARTER < 999999.99) and (WAGES
2nd QUARTER AFTER EXIT QUARTER > 0 and WAGES 2nd
QUARTER AFTER EXIT QUARTER < 999999.99) and (WAGES 3rd
QUARTER AFTER EXIT QUARTER > 0 and WAGES 3rd QUARTER
AFTER EXIT QUARTER < 999999.99) and DATE OF EXIT is within
the reporting period and (OTHER REASONS FOR EXIT is null or
OTHER REASONS FOR EXIT = 00 or OTHER REASONS FOR EXIT
= 98)

divided by
Denominator:
Count of unique RECORDS where (WAGES 1ST QUARTER AFTER
EXIT QUARTER > 0 and WAGES 1ST QUARTER AFTER EXIT
QUARTER < 999999.99) and (WAGES 2nd QUARTER AFTER EXIT
QUARTER > 0 and WAGES 2nd QUARTER AFTER EXIT
QUARTER < 999999.99) and (WAGES 3rd QUARTER AFTER EXIT
QUARTER > 0 and WAGES 3rd QUARTER AFTER EXIT QUARTER
< 999999.99) and DATE OF EXIT is within the reporting period and
(OTHER REASONS FOR EXIT is null or OTHER REASONS FOR
EXIT = 00 or OTHER REASONS FOR EXIT = 98).

A.7

APPENDIX B

DEV SAMPLING AND ERROR RATE ESTIMATION

The primary data validation objective is to identify the sources of error (that is,
which data elements or which kinds of cases are more prone to error) and to
provide error information for selected data elements irrespective of whether or not
they are used to measure performance. This appendix discusses the general
approach to error rate estimation (Section A) and detailed sampling specifications
for TAA (Section B).

A. GENERAL APPROACH
The sample design for TAA is a stratified sample. The software randomly
selects records with differential sampling rates by the type of record. Participant
subgroups are over-sampled if they are determined to be very important for
calculating performance outcomes, and if they have a higher risk of error. Records
are randomly sampled within the subgroups. Sample sizes vary by state, with
smaller states having smaller samples than larger states for two reasons. First, to
achieve a given level of precision, smaller states require smaller sample sizes than
larger states. Second, precision requirements are relaxed to reduce the relative
burden on smaller states of validating data. The sample design also clusters the
sample in a small number of offices.
This design yields estimates that have a variance that exceeds the variance of a
simple random sample. Currently, we assume the variance to be twice that of a
simple random sample (a “design effect” of 2). After data validation has been
implemented for a sufficient period of time, an analysis will be conducted to
estimate the true design effect and to determine how much it varies from current
estimates. The results of this analysis will be used to refine the sample design.
After the records are validated, the software uses the validation results to
generate error rate estimates using two methods. With the first method (Overall
Error Rate), the error rates equal the total number of records in error for a
particular data element divided by the number of records sampled, weighted to
account for the over- and under-sampling of particular types of records.
As particular data elements may not be present in every record, the second
method (Reported Data Error Rate) includes in the denominator only those records
for which that particular data element was validated. With this method of
calculation, the error rates equal the total number of records in error for a particular
data element divided by the number of records for which that data element was

B.2

validated, weighted to account for the over- and under-sampling of certain types of
records.
For example, if 100 records are sampled, only 50 have a particular data
element, and there are 5 errors in that data element, the first method results in an
error rate of 5 divided by 100, or 5 percent, while the second method results in an
error rate of 5 divided by 50, or 10 percent.1 The Data Validation Summary and
Analytical Report uses both methods to list the percent error for each data element
validated.

B.

SAMPLING SPECIFICATIONS

As described in Section A, the software selects a sample of exiters by
creating a clustered, stratified sample in order to minimize the burden that data
validation imposes on states and local offices and still provide sufficiently precise
results. The software first selects a sample of offices.2 From the sampled offices,
the software selects a sample of records of exiters. The size of the samples varies
by state with precision estimates for small states reduced to minimize the burden
validation imposes on them.
To decide which offices to sample, the software weights the records for each
office. The weights are based on the risk that the data may be incorrect and the
importance of the record for measuring performance. Data elements related to
employment and source of employment information are subject to the greatest risk
of error and are the most important for calculating performance. Supplemental
sources for wages are the riskiest data, followed by wage records. Data that
indicate an individual is unemployed present the least risk. Based upon these risk
assessments, records that use supplemental sources to demonstrate an individual’s
employment in the first and/or third quarter after exit receive a weight of 3, records
that use wage records to demonstrate an individual’s employment receive a weight
of 2, and records for individuals who are unemployed receive a weight of 1.
The software creates a weight for each office by summing the weights of each
of the office’s records. The software then selects specific offices based on the
1

Weighting for over- or under-sampling was not considered in this example.

2

Some states may receive data by Workforce Investment Boards (WIBs), not by offices. In this case, the
software selects WIBs as the primary sampling unit instead of offices.

B.3

offices’ weighted values. The number of offices (n) selected is a function of the
number of offices in the entire state (N). The more offices sampled, the greater the
accuracy of the estimates. As more offices are sampled, the burden that validation
imposes on the state increases.
Table B.1 below shows the sampling rule the software uses to determine the
number of offices to sample. Column A provides ranges for the number of offices
in a state. Column B provides the number of offices selected. Thus, for a large state
with 250 or more offices, the software selects 30 offices, as shown in Row 1 of the
table. For a medium size state with 80 offices, the software selects 15 offices, as
shown in Row 4 of the table.

TABLE B.1 Office Sampling

1
2
3
4
5
6
7

A
B
Number of Offices Number of Offices
in State (N)
Sampled (n)
250 or more
30
200-249
25
100-199
20
75-99
15
30-74
10
7-29
7
Fewer than 7
All

The software automatically selects any office(s) that accounts for 0.8 × 1/n or
more of the state’s total weight.3 The remaining offices are randomly selected
based on their weighted value.
Next, the software selects the individual records to validate, selecting a sample
of records from the sampled offices. The probabilities of selection are proportional
to the weights assigned to each record. Consequently, a greater proportion of
3

The initial weights and number of offices are estimates. The pilot validation and early implementation will
provide data to determine if the estimates are correct. If the information gained from these studies suggests that the
estimates are incorrect, they will be adjusted accordingly.

B.4

records with supplemental wage data will be selected than would be in a simple
random sample.
Table B.2 below illustrates how the software determines the sample size.
Column A gives ranges for the number of exiters. Depending upon the number of
exiters, the software selects a level of precision, shown in Column B.4 Column C
provides a range for the number of records to be sampled for validation. For
example, if a state has 480 exiters, the software selects a sample of between 100
and 150 records to validate, providing a precision level of 3.5 percent.

TABLE B.2
EXITER RECORD SAMPLING
A

B

C

# of Exiters

Confidence
Interval5

Range of Sample

1

300 or greater

3.5%

100-1506

2

0-299

4%

0-83

After the sampled records are validated, the software determines error rates for
each of the data elements specified. Because the software clusters and stratifies the
samples, it must weight the results yielding final weighted error rates for each data
element.
4

Precision is determined by the confidence interval. Confidence intervals measure the accuracy of the estimate.
For example, a data element with a 7 percent error rate might have a +/− 2 percent degree of accuracy. The +/− 2
percent is the confidence interval. In this example, the confidence interval means that while the error rate is given as
7 percent, the actual error rate is between 5 and 9 percent.
5

Several factors must be incorporated into the calculation of the confidence interval. Because of the small
sample size, a finite population correction (fpc) needs to be incorporated into the calculations. Assumptions about
the accuracy of the error estimate are also incorporated; the calculations assume a confidence interval at a 0.05 level
for a two-tail test.
6

The sample size will not exceed 150 participants per group.

B.5

APPENDIX C

DEV INSTRUCTIONS

Appendix C provides instructions for validating each data element listed on the
data validation worksheets. The instructions have the following components:

Data Element: The participant characteristic to be validated, as listed on the
record layout and validation worksheet.
Reference #: The number corresponding to each data element on the
validation worksheet.
Format: The format in which each data element value should be recorded in
the imported TAPR file.
Definition: Definition of the data element being validated.
Federal Sources: The definitive federal sources of information for validating
each data element.
State Version of Sources: The state version of the sources of information for
validating each data element. Some state sources may be directly available to
state staff, while other documentation may be stored at the local level. States
have flexibility to determine the most efficient source to use for validation
when more than one source is available.
Instructions: Rules that instruct the validator on how to validate each data
element listed on the worksheet.

The validator identifies each data element on the worksheet and then locates
the corresponding instructions for that data element using the reference number
listed on the validation worksheet. The validator then follows the instructions for
validating that particular data element.
Although the instructions for different data elements may be similar, the
validator should read each instruction carefully because in some cases a blank data
element is not validated, while in others (such as wage records) a blank data
element must be validated.
Please refer to Chapter II, Section D—Data Element Validation—for detailed
information on using these instructions to conduct data element validation.
C.2

Friday, December 01, 2006

Trade Adjustment Assistance Data Element Validation Instructions

Page 1 of 5

1. If a record is missing from the office in which it is supposed to be, the validator should check the box at the top of the worksheet that says "Missing
Record." No other elements should be validated.
2. If the validator cannot determine where a record is supposed to be located, the validator should check the box at the top of the worksheet that says
"Unable to Locate." No other elements should be validated.
3. If a record for a person not registered in TAA has wrongly been included in the extract file, the validator should check the box at the top of the
worksheet that says "Invalid Record." No other elements should be validated.
4. If a record has a wrong Social Security Number (SSN), the validator should check the "Wrong SSN" checkbox AND validate all data elements listed
on the worksheet.

There are 4 conditions used to describe higher level data problems:

MISSING AND INVALID RECORDS

2. If the validation instruction says SUPPORT: Enter a checkmark in the box in the pass column if the data on the worksheet are supported by the data
in the source documentation. Enter a checkmark in the box in the fail column if the data on the worksheet are not supported by the data in the source
documentation or if no source documentation is found.
To support, the data on the source documentation must provide evidence that the data on the worksheet are correct. This instruction is used
when information must be interpreted or processed before it can be used to assess the accuracy of the data on the participant records. For example, source
documentation can support waiver from training requirement.

The validator must validate all data elements that have checkboxes next to them.

1. If the validation instruction says MATCH: Enter a checkmark in the box in the pass column if the data on the worksheet match the data in the source
documentation. Enter a checkmark in the box in the fail column if the data on the worksheet do not match the data in the source documentation or if no
source documentation is found.
To match, the data on the worksheet must be the same as the data in the source documentation. For example, if the worksheet says a participant's date
of exit is July 1, 2002, then the source documentation must also have July 1, 2002 as the exit date.

Two types of validation rules exist:

VALIDATION RULES

Validation Instruction: These instructions present the data elements, reference numbers, formats, element definitions, federal sources, state/grantee
sources, and validation rules needed to perform data element validation. The federal sources are the generic, federally recommended source
documentation. The "State/Grantee Sources" column can be used to enter state-specific versions of the federally approved documentation.

Trade Adjustment Assistance Data Element Validation Instructions

18

Most recent qualifying
separation

Friday, December 01, 2006

12

2

1

Date:
YYYYMMDD

Numeric:
1 = Yes, <= 180
days
2 = Yes, eligible
veteran
3 = Yes, other
eligible person
4 = No

Number Ref # Format

Veteran status

Data Element

Layoff lists or rapid response list
Determination of eligibility form
Notice of termination
Letter from employer

DD-214
Cross-match with Veterans Data
Cross-match with Wagner-Peyser
State MIS

Federal Sources

Trade Adjustment Assistance Data Element Validation Instructions

Record the most recent date of
separation from trade-impacted
employment that qualifies the
individual to receive benefits
and/or services under the Trade
Act. Use the YYYYMMDD format.

Record the code that indicates
whether the individual served in
the active U.S. military, naval, or
air service for a period less than or
equal to 180 days, and who was
discharged or released from such
service under conditions other than
dishonorable, or whether the
individual met the conditions
above for more than 180 days.

Definition

State/Grantee

Page 2 of 5

Match

Support

Instructions

3. Case Notes: Case notes refer to either paper or online statements by the case manager that identify a participant's status for a specific data element, the date
on which the information was obtained, and the case manager who obtained the information.

2. State MIS: Unless otherwise noted, state MIS refers to specific, detailed information that is stored in the state information system that supports an
element. An indicator, such as a checkmark on a computer screen, alone is not acceptable source documentation. For example, state MIS is
acceptable source documentation for date entered training. To be an acceptable source to validate this date, in addition to the date of first training, the
state MIS should have detailed information about the type of training and the organization that provided the training. This detailed information makes
valid source documentation and makes it unnecessary for such states to validate this data element in local offices.

1. Cross-Match: A cross-match requires validators to find detailed supporting evidence for the data element. An indicator or presence of an SSN in a
non-TAA database is not sufficient evidence. For example, veteran status can be determined by a cross-match with the veteran's database. It is not
sufficient to find that the sampled SSN is present in this database; validators must also find dates of service, separation, and discharge information.

For the most part, definition of a particular source is clear. States have, however, had questions about three sources -- Cross-Match, State Management
Information System (MIS), and Case Notes. Definitions for these elements are:

For most data elements, the validation instructions provide multiple forms of acceptable source documentation. If the state collects multiple sources for
the same data element and the sources conflict, the most reliable source should be used to determine if the element passes or fails. For example, for
Employed in first full quarter after exit (Ref# 11) UI wage records are more reliable than supplemental data sources. Validators should use their best
judgment when deciding the most reliable source to validate an element.

TYPES OF SOURCE DOCUMENTATION

8

25

26

Date completed, or
withdrew from, training

Training completed

Basic Trade Readjustment 34
Allowance (TRA)

Friday, December 01, 2006

7

24

Date entered training

6

5

4

21

Petition number

3

20

Numeric:
1 = Yes
2 = No

Numeric:
1= Yes,
completed
approved training
course
2= No, did not
complete
(withdrew)

Date:
YYYYMMDD

Date:
YYYYMMDD

Numeric:

Date:
YYYYMMDD

Number Ref # Format

Date of application

Data Element

State UI records of TRA checks
issued
Request for allowance
State MIS

State MIS
Case Notes about
communications with program
Applicant Statement
UI Records

Vendor training documentation in
case file
WIA Status Form

Vendor training documentation in
case file
WIA Status Form
Training plan

Vendor training documentation in
case file
case notes
WIA status forms
State MIS
attendance lists
UI records

Worker group certification in case
file
State MIS
UI Records
DOLETA Website
Determination of Eligibility form

Application in case file
State MIS
WIA registration form
Designation of Eligibility form

Federal Sources

Trade Adjustment Assistance Data Element Validation Instructions

Record whether the participant
received Basic Trade Readjustment
Allowances (TRA).

Record the appropriate code:

Record the date (using
YYYYMMDD format) when the
participant completed training or
withdrew permanently from training.

Record the date (using
YYYYMMDD format) when the
participant's approved training
began. NOTE: Trade Act-approved
training which begins under the
WIA dislocated worker program
may have a date entered training
that is earlier than the participant's
application date.

Record the petition number of the
certification which applies to the
individual's worker group. If there
is more than one petition number
(for example, certifications under
both the TAA and NAFTA-TAA
programs), record the petition
number of the program from which
the training is paid, unless a
waiver was issued. Do NOT
include any alphabetic suffix;
record the petition number only.

Record the date, using
YYYYMMDD format, on which the
individual first applied for Trade
Act services/benefits under the
applicable certification.

Definition

State/Grantee

Support

Support

Match

Match

Match

Match

Instructions

Page 3 of 5

42

44

49

52

53

Date of exit

Employed in first full
quarter after exit

Employed in third full
quarter after exit

Three quarters prior to
participation quarter

Two quarters prior to
participation quarter

Friday, December 01, 2006

37

14

13

12

11

10

9

Definition

Numeric:
00000.00

Numeric:
00000.00

Numeric:
1 = Yes
2 = No

Numeric:
1=Yes
2= No

Date:
YYYYMMDD

Case file documentation that
includes initial approval and
renewals at 30 day intervals
State UI records of TRA checks
Form from employment counselor

Federal Sources

UI Wage Records
WRIS
Federal Wages Database
Other state wage records

UI Wage Records
WRIS
Federal Wages Database
Other state wage records

Supplemental Data Sources as
defined in TEGL 7-99
State MIS

UI Wage Records
WRIS

UI Wage Records
WRIS
State MIS
Supplemental Data Sources as
defined in TEGL 7-99

Trade Adjustment Assistance Data Element Validation Instructions

Earnings in the second full quarter
prior to the quarter of
participation.

Earnings in the third full quarter
prior to the quarter of
participation.

Record whether the participant was
employed in the third full quarter
after exit.

Record whether the participant was
employed in the first full quarter
after exit.

Record the last date on which
Documentation in case file
the participant received TAA or
State MIS
partner funded services after 90 days WIA exit form
without a service. See TEGL 17-05
for more information

Numeric:
Record the code that indicates
1= Recall
the reason the participant
2 = Marketable
received a waiver from the training
3 = Retirement
requirement.
4 = Health
issued
5 = Enrollment
unavailable
6 = Training Not
Available
7 = Reason Unknown/
Served Prior to 2002

Number Ref # Format

Waiver from training
requirement

Data Element

State/Grantee

Page 4 of 5

Match; Validate blank fields

Match; Validate blank fields

Support; Validate blank

Support; Validate blank

Match

Support

Instructions

55

56

Wages second quarter
after exit quarter

Wages third quarter
after exit quarter

Friday, December 01, 2006

52

17

16

15

Numeric:
00000.00

Numeric:
00000.00

Numeric:
00000.00

Number Ref # Format

Wages first quarter
after exit quarter

Data Element

UI Wage Records
WRIS
Other state wage records
Federal Wage Databases

UI Wage Records
WRIS
Other state wage records
Federal Wage Databases

UI Wage Records
WRIS
Other state wage records
Federal Wage Databases

Federal Sources

Trade Adjustment Assistance Data Element Validation Instructions

Earnings in the third full quarter
after the participant has exited.

Earnings in the second full quarter
after the participant has exited.

Earnings in the first full quarter
after the participant has exited.

Definition

State/Grantee

Page 5 of 5

Match; Validate blank fields

Match; Validate blank fields

Match; Validate blank fields

Instructions

APPENDIX D

TAADV SOFTWARE INSTALLATION

This appendix provides instructions on how to install the TAADV software.
Task 1—Uninstall the Prior Version of TAADV
Before you install a new version of the TAA software, it is recommended that you
uninstall the existing version of the software. Please follow the normal process for
uninstalling software on your computer.
To uninstall the TAA software using Windows 2000 or Windows XP, select Start,
then Settings, then Control Panel. In the Control Panel, double click on
Add/Remove Programs. Click on TAA. Then, click on Change/Remove. Follow
the InstallShield instructions to remove the software.
Users may need to contact their system administrator to uninstall the application.
Task 2—Download Software
Before starting installation of the software, quit any other programs that may be
running.
Download the software to your PC from the Internet at
http://www.doleta.gov/performance/reporting/tools_datavalidation.cfm. Do not
download the software to your C:\Program Files folder as this is the default
installation location.
Click on “setup.exe”. The InstallShield Wizard window will pop up. Click on the
“Next” button. Complete the Customer Information window as necessary, and
click on “Next.” Users may need to contact their system administrator in order to
install the application.
Minimum system requirements:
Operating System - Microsoft Windows 2000 or better
RAM - 256 megabytes
Disk Space - 500 megabytes
Processor - 100 mhz Pentium III (or equivalent processor)
Suggested screen area - 800x600

D.2

Task 3—Restart Computer
After completing installation of the TAADV, reboot your computer before
continuing any further.

Task 4—Fix Database/Data Source Errors During Installation
For some users of Windows 2000, XP and NT, the data source may not be linked
to the correct database or to any databases, which prevents the application from
functioning.
To determine if the correct database is selected, select the Data Sources (ODBC) in
the Control Panel—Administrative Tools, User DSN tab. Check that the TAADV
is linked to the right database. To do this, double click on the TAADV Data
Source (ODBC) and check the path under the “database” section. If the
application was saved to the C:\ drive then the database should be in the
C:\Program Files\TAA folder. If the software was saved to another location, check
to make sure that the data source points to this other folder and database.
Some users of Windows 2000, XP, and NT may also need to set up the ODBC in
the System DSN tab under Administrative Tools. Users should try this solution if
the application freezes on the initial splash screen. This setup procedure is identical
to the procedure used for the User DSN tab.
If you are still experiencing difficulties accessing the software after checking the
data source, you should consult your system administrator.

D.3

Task 5—Manually Register Files
While installing the TAA data validation software, users may get an error message
stating that a file is not registered or cannot be found. In certain cases, users may
get this error message after the software has been installed instead of during the
installation process.
These error messages can be resolved by manually registering the specific files that
failed to register. First, users should check for the unregistered file on their hard
drive. The file should be located in C:\program files\TAA, in C:\winnt\system32 or
in a different directory if the software was not installed in the default directory. If
the file is on the hard drive, users should follow the steps outlined below to
manually register the file. If the file cannot be found, users should contact
[email protected].
To manually register the files, go to the Start menu and select Run. In the Run
box, in the Open field, type in the regsvr32 command in the following format:

regsvr32 "PathName"

where “Pathname” is the full location of the file including the file name given in
the error message. Type in the entire pathname and filename with quotes around it.
For example, users with Windows 2000 or XP would type the following in
response to an error message noting that the file crviewer.dll is not registered:
regsvr32 "c:\program files\TAA\crviewer.dll"

Windows NT users would type in:
regsvr32 "C:\winnt\system32\crviewer.dll"

D.4

After typing in the regsvr32 command click “OK”. This manually registers the file
that did not register during the batch installation process.
Users should see a regsvr32 message that the manual registration succeeded.
Repeat this process for each unregistered file.

D.5

Task 6—Database Setup for Multi-User Access
The following instructions apply to states that want to provide multi-user access to
a pre-loaded database.
Definitions:
In the instructions, the term local is used to describe when a change is made on the
user’s machine, and
the term central is used to describe when a change is made to a database not
located on the user’s machine.
Instructions:
1. One user loads the extract file to be validated to user’s local machine.
2. Compact the database (under the Utilities Menu) and close out the software.
3. Find the local TAA software directory (note the default directory is C:\Program
Files\TAA).
4. Find the TAA database called “TAA.mdb” (or the database into which the
extract was loaded); note that this database contains the extract file that was
imported in addition to the DV worksheets that will be completed for
validation.
5. Save the local TAA database to a central location where other users can access
it in order for them to individually complete the validation worksheets.
Typically, this location will be a drive on a network server.
Users who are completing the validation worksheets using the database on the
server must know where the database is stored and the name of the database. They
must also have the TAA software loaded locally on their machines (note that all of
the users will need to have the same version of the software).

D.6

APPENDIX E

TAADV SOFTWARE REFERENCE GUIDE

This appendix provides a condensed list of all of the software menus and functions
for quick reference.

A. FILE MENU
The first menu on the toolbar is called “File.” Options in this menu include:
1. New
Users can create a new database in which to load their extract file.
2. Open
Users can open an existing database.
3. Save As
Users can save the database that they are currently using under a new name, and
continue working with this new database.
4. Show Tips at Start-Up
Users can choose whether or not the tips appear at start-up.
5. Exit
Select this to exit the program.

E.2

B. IMPORT DATA MENU
The second menu on the toolbar is called “Import Data.” Options in this menu
include:

1.

Import From Extract File

Select this tab to import data into the application. See the source table record
layout for the appropriate data record format.

2.

View Latest Error Report

This function displays a report that lists records that were rejected after importing
a file.

3.

View Duplicates

This function displays a report that lists duplicates that the software identifies and
rejects when importing the extract file.

4.

Source Table Record Layout

This function displays the record layout used to develop the validation file. The
record layout is in the TAPR format with the addition of six fields. A copy of the
record layout can be found in Chapter III of this handbook.

5.

View Source Table

This function displays all of the records that were imported into the software.

E.3

C. CHANGE REPORTING OPTIONS MENU
The third menu on the toolbar is called “Change Reporting Options.” The only
option in this menu is:

1.

Change Reporting Options

This function opens an expanded version of the Sign In pop-up window, where the
user can change the fiscal year and the period start and end dates. The user can also
select a WIB Name, Office Name, and/or Case Manager filter to view subsets of
records and to generate substate reports for management purposes.

E.4

D. REPORT VALIDATION MENU
The fourth menu on the toolbar is called “Report Validation.” Options in this
menu include:

1. View Performance Outcome Groups
This function provides a window where you can see a summary of each
performance outcome group and totals of records and dollar amounts, where
applicable, for each group. By clicking on the arrow in the far left column next to
a performance outcome group, the user can view the detailed records in that group.

2. View Performance Report
This function displays a report that calculates the values for the three performance
measures for the TAA program. The report may be printed by clicking the printer
icon on the tool bar.

3. Export Performance Outcome Groups
This function allows users to export the performance outcome groups into
individual text files. These files are saved are saved in the c:\Program
Files\TAA\Participant Extract directory.

E.5

E. DATA ELEMENT VALIDATION MENU
The fifth menu on the toolbar is called “Data Validation.” Options in this menu
include:

1.

Edit Worksheets

a.

Detailed Summary of TAA Validation Worksheets

This window displays a list of the records sampled. Double click on the relevant
record to view the online version of the worksheet. Any worksheets shaded in
purple have been completely validated. To search for a record with specific data in
a field, right click on the field name that you wish to search, and fill out the Find
box.
b.

Online Worksheet

The online validation worksheet allows users to enter pass or fail for the data
elements for the sampled record. The worksheet is tabbed to allow the user to
easily navigate though all of the data elements.

2.

Print Worksheets

This function enables the user to print the worksheets in batch mode.

3.

Print Summary/Analytical Report

This function displays a report that is generated after the worksheets are
completed. The report calculates two types of error rates for each applicable data
element.

E.6

4.

Print Sampled Counts Report

This function displays a report showing the distribution of sampled cases by WIB
Name, Office Name, and Sampling Unit.

5.

Summary/Analytical Comments

This opens a data entry screen where users can enter in comments related to their
data validation results. The comments then appear on the Summary and Analytical
Report.

6.

Export Summary/Analytical Report

This function exports the Summary/Analytical Report into an XML file and opens
a web browser so that users can e-submit their validation results to ETA.

E.7

F. UTILITIES MENU
The sixth menu on the toolbar is called “Utilities.” This menu includes:

1. Compact Database
Eliminates the temporary memory storage by compacting the database. This
feature removes unnecessary files remaining in temporary memory storage after an
import has been completed, without deleting files required for analysis and
validation. In order for the software and database to function properly and
efficiently during the import of large files, the user should compact the database
prior to every import. Otherwise, errors may result even if the underlying source
file is properly formatted. These overload errors may show up in both the error
counter on the import screen and as database “overload” errors in the import.txt
file. This function is only applicable to Access users. Users should exit the
application after compacting and then reopen the application for further use.

E.8

G. WINDOW MENU
The seventh menu on the toolbar is called “Window.” Options in this menu
include:

1. Cascade
Realigns open windows to appear in a cascade from the top left corner of the
screen.

2. Tile Horizontal
Realigns open windows horizontally.

3. Tile Vertical
Realigns open windows vertically.

This menu also indicates the names of the open windows, with a check next to the
dominant window.

E.9

H. HELP MENU
The eighth menu on the toolbar is called “Help.” Options in this menu include:

1.

Help Contents

This feature is under development.

2.

Contacting Mathematica Policy Research, Inc.

This feature provides contact information for users who would like additional
assistance with installing or using the software.
Users should e-mail
[email protected], and specify the software version being used, the
specific question, and the user’s contact information.

3.

About

This feature provides the software version number and product development
information for the application.

E.10


File Typeapplication/pdf
File TitleMicrosoft Word - 1.TAAhandbook120406.doc
AuthorGRoemer
File Modified2006-12-05
File Created2006-12-05

© 2024 OMB.report | Privacy Policy