Reporting

U .S. Department of Labor OMB Control No. 1205-0527

Employment and Training Administration Expiration Date: XX/XX/XXXX

Form ETA 9177

ATTACHMENT I

Unemployment Insurance (UI) Information Technology (IT)

Modernization Pre-Implementation Planning Checklist

Report

Version 1

U.S. Department of Labor

Employment and Training Administration

Form ETA 9177

UI IT Modernization Pre-Implementation Planning Checklist Report
State Name:	Date of Submission:
Project Name:
Scheduled Go Live Date:
Contact Information: (Name, Title, Address, Phone Number. Email Address)

Summary of Project:

Unemployment Insurance (UI) Information Technology (IT) Modernization Pre-Implementation Planning Checklist – Version 1

1. Functionality and Workarounds

Ensure all functionalities are fully available and operational (preferably with no high or critical defects). If a functionality is not fully available, an interim workaround should be in place, with a plan to quickly fix the defects, especially ones that are deemed high or critical. Workarounds should be minimized, fully tested, and exercised with the staff for any system functionality that is deferred or necessary to address known system issues.

1. All Benefit functions have been fully tested, including all service channels such as web, mobile, telephone, etc., and are fully operational (unless work around(s) have been planned for and tested), including, but not limited to:

1. Claims Filing:

For all supported programs: Regular UI, Unemployment Compensation for Federal Employees, Unemployment Compensation for Ex-Servicemembers, Combined Wage Claim, Disaster Unemployment Assistance, Trade Readjustment Allowances, any extensions, etc.;

2. Monetary determinations and monetary re-determinations;

3. Non-monetary determinations:

Separation and non-separation issues, including automated holds, are flagged for adjudication. The effects across Benefits Year Beginnings, especially involving converted claims (claims migrated from the legacy to the new system) and Benefit Charging; non-monetary scheduling of issues, as applicable, have been thoroughly tested;

4. Continued Claims processing;

5. Benefit payment mechanisms; including (as applicable): direct deposit, debit card, and checks;

6. Benefit Payment Control functions and other processes associated with prevention, detection, and recovery of improper and fraudulent payments; and

7. Claimant account portals and/or maintenance (including password reset functionality), as applicable;

2. All Tax functions have been fully tested (unless work around(s) have been planned for and tested), including, but not limited to:

1. Employer Registration:

• Initial registration; and

• Updates to Employer Registration;

2. Employer liability determinations, including reimbursable employers and employer status change;

3. Employer audit;

4. Tax rate computation;

5. Employer delinquency enforcement;

6. Employer tax and wage report and input processing;

7. Tax collections;

8. Tax refunds/overpayments;

9. New business registration; and

10. Quarterly tax filing/data entry;

3. All Appeals functions, such as automated filing of appeals, hearing notices/hearing decision correspondence, etc., have been fully tested;

4. All Reporting functions have been fully tested (unless work around(s) planned for and tested) including, but not limited to:

1. Federal reporting functions, including UI Required Reports, Benefits Accuracy Measurement, and UI Data Validation; and

2. UI program management reporting functions, such as tracking of initial and continued claims by age, management dashboard, and ad hoc and system logging reports;

5. All Interfaces, external and internal, have been fully tested (unless work around(s) have been planned for and tested) including, but not limited to:

1. Interfaces with call center operations (e.g., screen pop, Interactive Voice Response (IVR)). Recovery procedures are established and proven in the event that an interface is down/fails;

2. UI Interstate Connection (ICON) network interfaces, including Social Security Administration cross-match;

3. Treasury Offset Program interface with the Internal Revenue Service;

4. Connectivity with the UI Integrity Center’s Integrity Data Hub;

5. National Directory of New Hires interface with Department of Health and Human Services;

6. State Information Data Exchange System interface with Employers; and

7. Other external interfaces and extracts, such as those required to retrieve wage record data and quarterly employment and wages statistics (Quarterly Census of Employment and Wages (QCEW), Local Area Unemployment Statistics (LAUS), and Longitudinal Employer-Household Dynamics) microdata, and automatic cross matches for identity or other integrity purposes;

6. All Testing has been completed including, but not limited to:

1. System generated forms and correspondence have been fully tested, including any configurability mechanisms and multiple language implementations;

2. All user roles have been tested, and users with appropriate permissions can easily change staff role assignments directly in the application;

3. Printing processes have been fully tested including:

• Verification of printer tap test requirements have been met, including cartridge nozzle performance and printer internal firmware functionality;

• Verification of a process in place to do manual intervention for letters and payments; and

• Verification of mailing addresses and proper display of United States Postal Service barcodes in window envelopes, multiple pages per envelope are possible, and that printers can handle duplex printing (conservation of resources);

4. Imaging and scanning, as needed, have been fully tested.

• Sufficient storage capacity has been provisioned;

• All legacy system image files have been successfully migrated, as applicable;

• Indexing has been tested and validated;

• Optical Character Recognition has been tested and validated as applicable; and

• Imaging system recovery from failure has been tested to ensure critical customer services are not impacted;

5. Batch Processes, including execution timelines, have been fully tested as outlined below:

• Batch processes have been executed for at least a week to ensure that all links are pointing to the correct places, all jobs run as expected and produce expected results;

• Batch processing should be completed within a designated amount of time;

• Batch processing remediation and recovery of failures have been tested and validated;

• Weekly, monthly, quarterly, and annual batch process have been tested and validated;

• If possible, a mock execution has been conducted by states for at least a week with manufactured data to simulate three times the expected load in order to ensure system readiness;

• Time travel/aging test data has been incorporated as part of system testing and User Acceptance Testing (UAT);

• If the testing can be done in an enterprise test environment, production like interaction between systems via interfaces and web services has been simulated to provide insight into potential bottlenecks or issues before launch; and

• After testing, ensure that any environment configuration file(s) have been updated for production;

6. Pilot the process during final testing for reporting system issues;

7. Workflows have been verified and adjusted by conducting thorough review of system-generated issue flags/triggers to ensure they are necessary, and are not creating false issues. This effort should include:

1. Monitoring workflow/work items in new system and ensure proper staff coverage on all queues;

2. Ensuring the capability to either migrate work items or have a process to identify unprocessed/semi processed work on the new system; and

3. Addressing and closing as many work items as possible, across all functions, before “go-live”;

8. System issue handling processes have been established, practiced before “go-live”, and are in operation in order to:

1. Ensure processes are in place to identify, track, and address system errors;

2. Develop procedures to document and communicate any system work-around or resolution; and

3. Ensure system error messages are understandable (and have been fully tested before “go-live”).

2. External Alternate Access Options and Usability Issues Addressed

1. Alternative Access Options are made available in compliance with ETA guidance, including alternative access options for individuals with barriers to filing by phone or on-line, such as those with Limited English Proficiency, disabilities, literacy issues (including computer literacy), and computer access issues;

2. Websites and other forms of communications, like brochures and posters, are in place which clearly identify alternative access points that effectively communicate to the populations with barriers described in section 2.1;

3. All staff have been fully trained on how to assist individuals with access barriers on using alternative filing options;

4. Staff and customer system access to new system has been defined, with outreach to all customers regarding access changes prior to going live, including processes to migrate existing credentials and/or establish new credentials;

5. New credentials are clearly described to the customer when attempting to first access the new system. If security questions or knowledge base authentications are used for self-services, industry best practices should be followed. If email accounts are used as the UserID for self-services login, the state has provided mechanisms for customers to easily create an email account if they do not have one; and

6. Ensure usability and access including:

1. Compliance with the Americans with Disabilities Act including meeting 508 compliance standards for hardware, software, websites, and documentation;

2. Website content has been translated for identified significant population language groups within the state, including correspondence as applicable;

3. Relevant state standards and other standards such as Web Content Accessibility Guidelines have been incorporated;

4. On-line content is clear, understandable, and at appropriate reading level (preferably tested by appropriate customer population);

5. Websites provide information on alternative access for individuals with barriers; and

6. IVR phraseology and messages are clear and understandable.

3. Policies and Procedures

Ensure policies and procedures have been developed, completed, and disseminated, including:

1. Policy and procedural changes coinciding with UI system modernization;

2. Policies for data security, including those for handling privacy and confidential data;

3. Policies for data retention and data disposal;

4. Staff has been fully trained on new policies and procedures;

5. Any updates to organizational structure have been coordinated with human resources department and disseminated agency-wide;

6. Any new or modified roles have been clearly defined, and staff have been appropriately trained; and

7. Disaster Recovery (DR) plan is in place and the Recovery Time Objective and Recovery Point Objective have been established and disseminated.

4. Technical Preparation for System Implementation

Ensure the technical preparation for system implementation is completed prior to “go-live”, including

1. Data architectures/data structures have been addressed and documented. Internal and external data users have been consulted in the development of new data structures;

2. Data conversion is complete and successful, including:

1. State policies and procedures have been followed to inform migration strategy of data to the new system and backlogs are addressed prior to initiating data migration process;

2. Data model diagram (entity relationship model) of the legacy system and the new system have been completed;

3. System design document(s), technical design document(s) and related artifacts have been completed and reviewed for completeness;

4. New system results checked and reconciled against legacy system results;

5. Spot checks have been completed for particular areas of interest (such as: overpayments, certifications, employer charging, ICON Billing (IB6), and accounting balances);

6. System blackout period has been benchmarked, if appropriate, with notification and messaging to customers in advance;

7. Multiple successful data conversions, including record count verifications, and Extract, Transform, Load final load benchmarking have been completed and, process lockdown is in place before going live; and

8. Financial reporting has been thoroughly tested and validated, as Trust Fund/Financial staff interactions are critical;

3. Independent Verification and Validation has been conducted, as needed;

4. Bridging processes/protocols between applications/systems are in place as needed (for example, new benefits system integrated with legacy tax system or the inverse);

5. User roles have been well-defined, configured, and set-up in the system. The use of excessive amount of roles in the system should be avoided in order to ensure a coordinated workflow process;

6. Comprehensive UAT has been performed including:

1. UAT has been conducted with positive results and conducted again following any additional system changes;

2. System defects were systematically tracked and remedied;

3. Defect tracking and remediation processes are in place post “go-live” (in conjunction with development vendor), along with applicable service level agreements (SLAs) for response times and fix turnaround times;

4. System help has been verified (e.g., page level, field level); and

5. Logging and exception handling have been verified, including any tool used for mining application logs;

7. Capacity has been developed to rapidly identify system flaws and make immediate fixes. Hot fix procedures are established, exercised, and validated;

8. Contingency/Back-up plan is in place if new system fails – particularly if no overlap of legacy and new system at go live point;

9. If a DR site is in place and available, testing and validation of the DR site has been conducted;

10. Incoming phone capacity (lines and staff) have been adjusted for anticipated increases in call volume, including:

1. Call Centers, as applicable; and

2. Any other Agency unit staff that will conduct triage of system issues with customers;

11. “Go-live” decision points (show stoppers) have been outlined and continue to be followed;

12. System performance has been checked under peak user loads, including complex transactions, prior to “go-live”;

13. System availability testing has been conducted, including:

1. Demonstrated system availability under heavy loads for sustained period of times; and

2. Monitoring of computing resource consumption (processors, memory, input/output, etc.);

14. System operations reporting is available, including management dashboard, ad hoc reports, and system logging and audit trails, as needed;

15. Production configuration is defined and in place;

16. Automated production build /testing/production environment promotion process is in place for post-deploy defect fixes and enhancements;

17. System security has been tested and validated, including:

1. Verification and validation of Data and System security;

2. Verification of other internal system controls in order to ensure security and confidentiality of data; and

3. Security Penetration and/or Performance Testing has been conducted;

18. Business knowledge has been successfully transferred to appropriate agency/staff to operate the system, including role management, configuration settings and management, dashboards, and reporting;

19. End user support, problem reporting, and resolution protocols are in place;

20. IT knowledge transfer has been completed in order for agency to maintain and support the system. System performance benchmarks documented during build (and start early) have been used by IT staff to complete configuration/programming changes to support performance and system expectations (not just “help” vendor); and

21. Transition planning and execution concerning post-warranty or maintenance phase has been validated with vendor.

5. Call Center /Customer Service Operations

Ensure Call Center /Customer Service Operations strategies are in place including:

1. Call center standard operational procedures adapted, as appropriate;

2. Call center staff and/or customer service representatives (CSRs) have been fully trained, including:

1. CSRs/agents have participated in system UAT;

2. CSRs/agents, supervisors, and managers have been trained in the use of the new system;

3. Staff have been cross trained for use/augmentation, particularly to manage inquiries;

4. Project staff have been assigned/embedded into the customer service operations for at least the first few weeks in order to help the team transition to the new system support procedure; and

5. Call center staff have been included in the triage calls for the first few weeks after going live;

3. Modernization system and operations help desk information have been provided to all front-line CSR staff;

4. Front-line and management staff know triage process when system issues arise, and issue escalation processes have been clearly defined and understood by staff; and

5. Team meetings are scheduled daily and, as needed, to address:

1. New system issues;

2. Customer complaints; and

3. New operational or procedural issues.

6. Business Process

Ensure new business processes have been established, staffed, and that staff have been fully trained on any/all new system operations

1. Business processes and standard operating procedures have been developed and made available to all staff to enable them to understand how the new system has changed their work processes;

2. Roles/responsibilities have been fully defined prior to “go-live”;

3. Staff have been trained on new business processes and roles and responsibilities;

4. Training materials and tools to support new business processes (desk guides, handbooks, etc.) have been fully developed and provided to management and staff. A final training session, where “practice on production” work happens to uncover potential skill gaps (and also possible conversion issues) has been conducted;

5. Service delivery strategies have been clearly identified, made known to staff, and fully communicated to customers;

6. Management and staff have been fully trained on new system features and operations, broadly as well as in relation to their specific job responsibilities including:

1. Training that covers refresher information on program knowledge;

2. Training or refresher training has been scheduled as close to “go-live” as possible; and

3. Recognition that people learn at a different pace — if the training material is online or interactive, ensure the resources are made available to staff for self-learning after formal classes are complete;

7. Additional staffing needs have been identified/secured to address new system implementation including:

1. Call center(s) (calls can dramatically increase during a conversion);

2. Adjudication staff (modernized systems tend to contain more automated flags that raise issues requiring adjudication);

3. Appeals staff (the number of appeals may increase following a conversion as a result of newly automated processes which may impact claimants’ benefits or employers’ liability);

4. Other support functions such as integrity-related activities; and

5. Advance identification and clarification of roles for the members of the triage team (e.g., Frontline and Management staff, Help Desk staff, Call Center staff, Vendor, Labor Market Information (LMI) staff, etc.).

7. Help Desk

Ensure Staff and Help Desk Support is available, including:

1. Key staff have been put on alert and made available during critical times of conversion and deployment;

2. Procedures have been established for escalation processes, triage processes for addressing system technical/functional issues, communication channels, and coordination with Call Centers, as applicable;

3. Dry-runs of procedures with Help Desk staff have been conducted, including:

1. Project staff have been embedded into the help desk team to help the team transition to support the new application and answer questions, etc.; and

2. Help desk staff is included in the triage calls;

4. Help Desk is adequately staffed with modernization testers, including:

1. Subject Matter Experts, Business Analysts, and Project Team members;

2. Subject Matter Experts have been assigned for every 10-15 people in the State Agency in order to assist with questions and training; and

3. Laptops have been distributed to modernization testers (Subject Matter Experts) to report any findings;

5. Contact information has been published and disseminated to appropriate personnel, including LMI staff;

6. Procedures to categorize and track system issues based on help requests are in place

(e.g., credential/access issue, performance, questions needing clarification, etc.); and

7. Ensure Help Desk is available to support staff.

8. Management Oversight

Ensure management oversight is delivered, including:

1. Managers have been fully trained on management features and reporting processes in the system, including those for LMI and Bureau of Labor Statistics (BLS) to ensure that state unemployment and employment statistics can continue to be provided to the state;

2. Managers have been prepared to support staff if system issues arise;

3. Additional management meetings are scheduled during transition to identify and resolve issues; and

4. A support group for managers has been established to share information and relearn how to do their job functions using the new system.

9. Vendor Support/Communications

Ensure vendor support and communications has been established, including:

1. Process for addressing post implementation system fixes with vendor is in place prior to implementation;

2. Process for tracking system and post-implementation system issues is in place;

3. Routine, daily meetings with the vendor during transition has been scheduled to identify and prioritize system issues for as long as needed;

4. Process is in place to track the implementation of deferred functionalities;

5. The “hand-off” process for identified incidents/issues/changes/bugs to the vendor is clear and is understood by all parties;

6. Systems are in place to track the post-implementation support criteria and SLAs defined in the contract; and

7. The triage process has active vendor engagement.

10. Communication Processes and Procedures

Ensure communications processes and procedures have been established, including:

1. Staff Communications including:

1. Both UI and non-UI staff (e.g. LMI staff) have been included in the communications throughout the project;

2. Staff have been fully briefed on implementation plan (in addition to being fully trained); and

3. Staff have been provided with clear instructions on how to handle system issues if they arise;

2. Claimant and Employer Communications including:

1. Employer and Claimant outreach regarding roll-out of new system;

2. System “go-live” date disseminated;

3. System black-out period, if needed, and methods of conducting business during this phase has been clearly communicated to staff and customers;

4. Agency contact information has been widely disseminated;

5. System “how-to information” has been provided to staff and customers; and

6. Processes have been made available to support communication with individuals with limited English proficiency and staff are trained on these processes;

3. Public Communications have been issued prior to “go-live” point, as well as periodically during startup including:

1. State stakeholders have been notified in advance (i.e. Governor’s office, Legislative Services, and partner agencies);

2. System “go-live” date has been disseminated;

3. System black-out period, if needed, has been communicated;

4. A plan has been established to address potential issues/ad-hoc inquiries at the time of launch; and

5. A plan has been put in place to handle the press/media for a successful launch as well as for any unforeseen issues causing the launch to not go as planned.

11. Labor Market Information (LMI) Federal Reporting

Ensure LMI federal reporting functions have followed nationally standardized specifications (available at https://www.bls.gov/cew/unemployment-insurance-modernization/home.htm), tested, validated within project timelines, and are fully operational, including:

1. LMI Notification:

1. LMI has received advance notification of planning timetable for benefits and tax portions of the system implementation milestones; and

2. LMI has been notified of “go-live” dates for benefits and tax systems;

2. LMI Participation:

1. Discussions have occurred with LMI regarding any planned changes to UI numbering schemes, data sources or computations;

2. Collaboration has occurred with LMI on data uses and definitions to help with the development specifications:

• Data dictionary mapping of the fields in the legacy UI system to the fields in the new system has been shared and discussed with LMI; and

• Consultation with LMI on the definitions of employment and wages used in the new system;

3. Any changes to the reporting responsibilities of employers’ have been developed in consultation with LMI and the outreach plan to employers has been shared with LMI;

4. Periodic demonstrations of the parts of the system relevant to LMI have occurred, as the new UI system progresses toward deployment with an opportunity for LMI staff to provide feedback on screens as they are developed and prior to “go-live”;

5. Testing schedule has been discussed and agreed upon with LMI staff;

• LMI staff has been allowed to parallel test the new UI systems as it progresses toward deployment and an opportunity to provide feedback on any identified issue; and

• Issues identified by LMI during testing have been resolved prior to “go-live”;

6. LMI has been granted the same level of access to the screens, forms etc. in the new system as they had in the legacy system;

3. Data Exchange:

1. Collaboration has occurred with LMI concerning the rewrite of UI Tax and Wage record extracts; and

2. Extracts have been tested by LMI staff in order to ensure that it meets LMI requirements;

4. Data Validation:

1. Ensure LMI has completed data validation to produce LAUS and QCEW; and

2. Ensure completion of LMI testing and validation of benefit claims data.

Certification
Certifying Official: (Name, Title)
Signature:
Telephone Number:		Date:
Email Address:

Attachment I-3