Report of Responses to the UI Benefits Operations Self-Assessment Tool

Unemployment Insurance (UI) Benefit Operations Self-Assessment Tool: Data Validation (DV)

UI
Unemployment Insurance (UI) Benefit Operations
Self-Assessment Tool:

Data Validation (DV)

REVIEW PERIOD: Begins

Ends

Unless otherwise noted, all questions are applicable to the review period.

SECTION 1:

Procedures, Policies, and Confidentiality

The reviewer will assess the policies and procedures supporting the state’s DV operations. Resources may
include manuals, handbooks, desk aids, computer help screens, training guides, organized collections
of procedures or policies, or other readily accessible instructions that can help staff do their work
correctly, including ETA Handbook No. 361 and ETA Handbook No. 412 for guidance on Data Validation.
Instructions will normally include general information such as compilations of relevant laws and
regulations, as well as detailed instructions for carrying out individual jobs in the agency.

1.

Does

the state have written policies and procedures for conducting its DV operations apart from
Federal guidelines?
Select Answer
No
Yes

1a. If yes, were the policies and procedures updated or modified during the last validation year?
1b.

Select Answer
N/A
No
Yes
U
 nder what circumstances will the state update its DV policies and procedures?
(check all that apply)

Federal or state law changes
Organizational changes
Technology changes
U.S. Department of Labor guidance
Question 1b check boxes continue on next page

Section 1: Procedures, Policies, and Confidentiality — June 2016, Version 2.0 | 1

Unemployment Insurance (UI) Benefit Operations Self-Assessment Tool: Data Validation (DV)

Other (explain)

2.

How are policies and procedures for DV operations made available to staff? (check all that apply)
Online
Hard-copy handbook
Training materials
Other (explain)

2a. Are all formats consistent and up-to-date?
Select Answer
N/A
No
Yes

3.

Were

there any law or policy changes during the review period that affected the state’s DV policies
and procedures?
Select Answer
No
Yes
3a. If yes, what law or policy changes did the state implement, when were they implemented, and
what was the effect on the state’s DV policies and procedures?

Section 1: Procedures, Policies, and Confidentiality — June 2016, Version 2.0 | 2

Unemployment Insurance (UI) Benefit Operations Self-Assessment Tool: Data Validation (DV)

3b. If yes, have the state’s policies, procedures and training materials been updated to reflect

3c.

4.

these changes?
Select Answer
N/A
No
Yes
If no (to question 3b), explain

Are

policies or procedures in place to ensure all staff use correct definitions and procedures during
the validation process so that systematic and random errors are eliminated?
Select Answer
No
Yes
4a. If no, explain.

4b. As a part of report validation (RV) operations, describe how systematic and random errors
are resolved.

Section 1: Procedures, Policies, and Confidentiality — June 2016, Version 2.0 | 3

Unemployment Insurance (UI) Benefit Operations Self-Assessment Tool: Data Validation (DV)

5.

How does the state produce data used for Federal report and validation?
Directly from the state database
Flat file as transactions are processed
Combination of both methods
Other (explain)

5a. Does the validator identify the state’s reporting and validation sources for each population before

5b.

5c.

6.

determining the specific validation methodology to be implemented?
Select Answer
No
Yes
Are DV extract files generated at the same time as the Federal reports to eliminate differences in
data caused by changes in the database over time?
Select Answer
No
Yes
If no (to question 5b), what is the process for creating DV extract files?

What

policies and procedures are in place to conduct the quality sample validation (Module 4) for
nonmonetary determinations and lower authority appeals samples?

Section 1: Procedures, Policies, and Confidentiality — June 2016, Version 2.0 | 4

Unemployment Insurance (UI) Benefit Operations Self-Assessment Tool: Data Validation (DV)

7. What policies and procedures are in place to conduct the Wage Item Validation (WIV) regarding wage
records submitted on employers’ wage and contribution reports (Module 5)?

Section 1: Procedures, Policies, and Confidentiality — June 2016, Version 2.0 | 5

Unemployment Insurance (UI) Benefit Operations Self-Assessment Tool: Data Validation (DV)

SECTION 1:

Comments

Document any issues that were identified when completing this section. This comment section may also
be used to provide additional information relating to any specific question(s) in this section.

Section 1: Procedures, Policies, and Confidentiality — June 2016, Version 2.0 | 6

Unemployment Insurance (UI) Benefit Operations Self-Assessment Tool: Data Validation (DV)

SECTION 2:

Training

Managers/employees should possess and maintain a level of expertise that enables them to accomplish
their assigned duties. Training systems should be sufficient to ensure that personnel understand and
perform their duties properly. When reviewing training systems, the reviewer must look for formal
training procedures (e.g., the training is conducted using an established schedule and using set guidelines
to make judgments about the quality of work being produced). There should be procedures for identifying
general and specific training needs and for delivering training as needed.

1. Does the state agency have a formal training plan for new DV staff?

Select Answer
No
Yes

1a. Is there a dedicated trainer?
1b.
1c.

Select Answer
N/A
No
Yes
Has all staff conducting DV operations completed a defined training program?
Select Answer
N/A
No
Yes
H
 ow much training has been provided to DV personnel during the last year?
(Estimate number of hours and type of training)

2.

Does the state have a “refresher” training plan to provide continuing training to DV staff?
Select Answer
No
Yes

2a. If yes, how often is the continuing training conducted?
Monthly
Quarterly
Other (explain)

Annually

On an as-needed basis

N/A

Section 2: Training — June 2016, Version 2.0 | 7

Unemployment Insurance (UI) Benefit Operations Self-Assessment Tool: Data Validation (DV)

3. Describe any technology or operational changes that were made during the review period that affected
DV operations.

3a. W
 ere all affected staff trained on these technology or operational changes prior to implementation
of the changes?
Select Answer
N/A
No
Yes

4.

Is there a process to obtain feedback from DV staff regarding the effectiveness of the training given?
Select Answer
N/A
No
Yes

4a. If yes, how is feedback obtained from participants?

4b. Is there a means to obtain feedback from DV management staff on the effectiveness of the training,

4c.

based upon staff performance following training?
Select Answer
N/A
No
Yes
If yes, how is feedback obtained from management staff?

Section 2: Training — June 2016, Version 2.0 | 8

Unemployment Insurance (UI) Benefit Operations Self-Assessment Tool: Data Validation (DV)

5.

What means are used by the state to determine training topics and content? (check all that apply)
Error review monitoring
U.S. Department of Labor guidance
State law/policy changes
Supervisor feedback/input
Other (explain)

Section 2: Training — June 2016, Version 2.0 | 9

Unemployment Insurance (UI) Benefit Operations Self-Assessment Tool: Data Validation (DV)

SECTION 2:

Comments

Document any issues that were identified when completing this section. This comment section may also
be used to provide additional information relating to any specific question(s) in this section.

Section 2: Training — June 2016, Version 2.0 | 10

Unemployment Insurance (UI) Benefit Operations Self-Assessment Tool: Data Validation (DV)

SECTION 3:

Workload Analysis / Management Controls

The state’s ability to manage DV operations, particularly through periods of unforeseen changes
in resources, is analyzed. The reviewer will also address process-improvement initiatives aimed at
improving DV operations.

1.

If the state tracks DV activity backlogs, how often are these reports generated?
Real-time

Daily

Weekly

Monthly

Quarterly

N/A

2. Is there an evaluation tool or checklist used to review the quality and timeliness of completed
DV operations?
Select Answer
No
Yes

2a. If yes, explain.

3. How does the state handle the DV workload if there is a shortage of staff due to retirement, temporary
or permanent staff reductions, etc.?

4.

Did the state experience delays in DV operations during the reporting period?
Select Answer
No
Yes

Section 3: Workload Analysis / Management Controls — June 2016, Version 2.0 | 11

Unemployment Insurance (UI) Benefit Operations Self-Assessment Tool: Data Validation (DV)

4a. If yes, what was the cause of the DV operations delays? (check all that apply)
Operational issues related to facilities, budget or staffing (explain)

Information Technology hardware or software issues (explain)

Law or policy changes (explain)

Other (explain)

Section 3: Workload Analysis / Management Controls — June 2016, Version 2.0 | 12

Unemployment Insurance (UI) Benefit Operations Self-Assessment Tool: Data Validation (DV)

5. During the review period, did the state conduct any business process analysis efforts to identify issues
and recommend improvements regarding DV operations to increase efficiency?
Select Answer
No
Yes
5a. If yes, what changes have been made and what was the result of those changes?

5b. I f yes, what (if any) changes were recommended but not made? (explain why)
(The response to this question may be addressed in the Comments at the end of the section,
if more space is needed.)

Section 3: Workload Analysis / Management Controls — June 2016, Version 2.0 | 13

Unemployment Insurance (UI) Benefit Operations Self-Assessment Tool: Data Validation (DV)

SECTION 3:

Comments

Document any issues that were identified when completing this section. This comment section may also
be used to provide additional information relating to any specific question(s) in this section.

Section 3: Workload Analysis / Management Controls — June 2016, Version 2.0 | 14

Unemployment Insurance (UI) Benefit Operations Self-Assessment Tool: Data Validation (DV)

SECTION 4:

Performance Management

The reviewer will examine the state’s practices for monitoring DV operations and compliance with
Federal and state law and regulations. Preparation and monitoring of corrective action plans are
necessary functions to further program performance improvements.

1.

Was the state required to complete a Corrective Action Plan(s) during the review period concerning DV?
Select Answer
No
Yes

1a. If yes, indicate the performance areas that required a CAP.
Secretary’s Standards:
Data Validation – results not submitted
Data Validation – failing/incomplete submission

2.

What unit or office is responsible for monitoring the performance of the state’s DV operations?

2a. W
 hat methods are used to monitor the DV operations to ensure the program is operated in
accordance with Federal and state law and regulations?

Section 4: Performance Management — June 2016, Version 2.0 | 15

Unemployment Insurance (UI) Benefit Operations Self-Assessment Tool: Data Validation (DV)

2b. H
 ow frequently is the DV program review conducted to ensure compliance and operational
integrity?
Quarterly
Other (explain)

Semi-annually

Annually

3. Does the state monitor UI program activity and performance in addition to monitoring set out in
Federal requirements?
Select Answer
No
Yes
3a. If yes, explain what areas are monitored and the methods used.

Section 4: Performance Management — June 2016, Version 2.0 | 16

Unemployment Insurance (UI) Benefit Operations Self-Assessment Tool: Data Validation (DV)

SECTION 4:

Comments

Document any issues that were identified when completing this section. This comment section may also
be used to provide additional information relating to any specific question(s) in this section.

Section 4: Performance Management — June 2016, Version 2.0 | 17

Unemployment Insurance (UI) Benefit Operations Self-Assessment Tool: Data Validation (DV)

SECTION 5:

Information Technology (IT)

The reviewer will address the state’s IT projects relating to DV operations completed during the review
period and any identified unmet IT needs. The reviewer will assess the IT department’s delivery of
programming and technical support to DV operations.

1.

During the review period, were there any IT projects that impacted the state’s DV operations?
Select Answer
No
Yes

1a. I f yes, what were the projects, are they completed or on-going? If on-going, what is the expected
completion date for each?

1b. W
 hat is the effect of the changes? (The response to this question may be addressed in the Comments
at the end of this section, if more space is needed.)

2. Were any automation or technology upgrades or releases implemented during the review period that
affected the state’s DV operations?
Select Answer
No
Yes
2a. I f yes, have the state’s policies, procedures and training materials been updated to reflect
these changes?
Select Answer
N/A
No
Yes
2b. Is the DV staff aware of the technology changes that were implemented?
Select Answer
N/A
No
Yes
Section 5: Information Technology (IT) — June 2016, Version 2.0 | 18

Unemployment Insurance (UI) Benefit Operations Self-Assessment Tool: Data Validation (DV)

3.

Are the DV computer system and the UI computer system operated on a single computer platform?
Select Answer
No
Yes
3a. I f multiple platforms are in use (e.g. separate databases for Appeals, Tax), describe the actions
taken by the state to minimize the impact of multiple platforms on data integrity?

4.

Does the state have adequate technical staff to support all DV operations and DV computer systems?
Select Answer
No
Yes

5. Were there any IT issues with the software retrieving reported numbers from the state UI database?
Select Answer
No
Yes
5a. If yes, what issues were encountered and have they been resolved?

6.

Were there any IT issues when transmitting the DV results to USDOL?
Select Answer
No
Yes

6a. If yes, what issues were encountered and have they been resolved?

Section 5: Information Technology (IT) — June 2016, Version 2.0 | 19

Unemployment Insurance (UI) Benefit Operations Self-Assessment Tool: Data Validation (DV)

7. During the review period, were there IT needs for the DV operations that were not met that affected
program performance?
Select Answer
No
Yes
7a. I f yes, describe what IT needs were not met, the reason they were not met, what effect it had on
the program, and the state’s plans to meet these needs, if any? (The response to this question may
be addressed in the Comments at the end of this section, if more space is needed.)

Section 5: Information Technology (IT) — June 2016, Version 2.0 | 20

Unemployment Insurance (UI) Benefit Operations Self-Assessment Tool: Data Validation (DV)

SECTION 5:

Comments

Document any issues that were identified when completing this section. This comment section may also
be used to provide additional information relating to any specific question(s) in this section.

Section 5: Information Technology (IT) — June 2016, Version 2.0 | 21

Unemployment Insurance (UI) Benefit Operations Self-Assessment Tool: Data Validation (DV)

SECTION 6:

Staffing

The reviewer will examine staffing levels, staff responsibilities, and organizational changes affecting the
state’s DV operations.

1.

Does the DV staff have any other responsibilities in addition to DV operations?
Select Answer
No
Yes

1a. If yes, explain the other responsibilities.

2.

Does the same staff handle both the Benefits and the Tax DV operations?
Select Answer
No
Yes
2a. W
 hat unit or office is responsible for validating BTQ/TPS populations in Model 4 and Wage Item
Validation (WIV) in Module 5?

3.

How many FTEs were budgeted for DV during the review period?

3a. How many FTEs were dedicated to DV during the review period?

4. Does the state have a contingency plan to handle workload fluctuations—for example, identify staff
with DV experience, maintain list of computer hardware for deployment to a disaster center, etc.?
Select Answer
No
Yes

Section 6: Staffing — June 2016, Version 2.0 | 22

Unemployment Insurance (UI) Benefit Operations Self-Assessment Tool: Data Validation (DV)

5. Were there any organizational changes during the review period that affected the state’s DV unit
staffing or operations?
Select Answer
No
Yes
5a. I f yes, what organizational changes did the state implement, when were these changes
implemented and what was the effect on the state’s DV staffing or operations?

Section 6: Staffing — June 2016, Version 2.0 | 23

Unemployment Insurance (UI) Benefit Operations Self-Assessment Tool: Data Validation (DV)

SECTION 6:

Comments

Document any issues that were identified when completing this section. This comment section may also
be used to provide additional information relating to any specific question(s) in this section.

Section 6: Staffing — June 2016, Version 2.0 | 24

Unemployment Insurance (UI) Benefit Operations Self-Assessment Tool: Data Validation (DV)

SECTION 7:

Concluding Summary Comments for Data Validation

For the following sets of questions, consider the overall operations related to Data Validation. This is an
opportunity to identify successful practices and/or any needed corrective action measures, along with any
other general comments or observations concerning this functional area of UI Benefits. Additional space
for comments and reviewer notes is available on pages 28 and 29.

1. Provide any observations of good and/or exemplary performance in the state’s Data Validation policies,
procedure, or operations that would constitute Successful Practices to share with other states.

Section 7: Concluding Summary Comments for Data Validation (DV) — June 2016, Version 2.0 | 25

Unemployment Insurance (UI) Benefit Operations Self-Assessment Tool: Data Validation (DV)

2. Document any issues detected in Data Validation that adversely affects the state’s performance or its
ability to meet performance standards/measures. Identify any corrective action measures that should
be taken to improve the state’s performance in regards to any weaknesses identified.

Section 7: Concluding Summary Comments for Data Validation (DV) — June 2016, Version 2.0 | 26

Unemployment Insurance (UI) Benefit Operations Self-Assessment Tool: Data Validation (DV)

3. Add any additional comments, concerns, or observations regarding the state’s performance or
operations in this area that have not been addressed elsewhere and should be noted.

Section 7: Concluding Summary Comments for Data Validation (DV) — June 2016, Version 2.0 | 27

Unemployment Insurance (UI) Benefit Operations Self-Assessment Tool: Data Validation (DV)

Additional Comments and Reviewer Notes:

Additional Comments and Reviewer Notes — June 2016, Version 2.0 | 28

Unemployment Insurance (UI) Benefit Operations Self-Assessment Tool: Data Validation (DV)

Additional Comments and Reviewer Notes — June 2016, Version 2.0 | 29

Unemployment Insurance (UI) Benefit Operations Self-Assessment Tool: Data Validation (DV)

Reviewer Information:
REVIEWER
Name:
Title:
Email:
Phone No.:

ADDITIONAL REVIEW TEAM MEMBER
Name:
Title:
Email:
Phone No.:

Reviewer Information — June 2016, Version 2.0 | 30