On Oct 7, I had the opportunity to testify in front of the Appropriations Committee regarding Legislative Resolution 406: Interim study to examine the implementation of the Office of Chief Information Officer's information technology consolidation initiative to ensure efficient use of Nebraska taxpayer resources.
I was eager to testify on this topic as I wanted to enter into the record the work done by everyone in the OCIO and the impact we have made on the State of Nebraska.
LR406 set out seven items to be addressed as part of this interim study::
- Items (1) through (4) were a review of various budget programs, services, and staffing levels for the Office of the CIO.
- Item (5) related to “evidence of improvements resulting from the consolidation initiative.”; and
- Finally, items (6) and (7) related to potential improvements to the consolidation initiative.
The National Conference of State Legislatures published an article titled “The Case for IT Consolidation” in April of 2018. The article contained the following statement, “Consolidating IT resources across a state, though not an easy process, is clearly the favored path.” Nebraska is one of very few states that has achieved that favored path. The Office of the CIO earned the 2018 Enterprise Technology Management Initiative Award for State Technology Consolidation from the National Association of State Chief Information Officers (NASCIO). This award celebrates the country’s leading government IT services.
In 2015, recognizing that the state’s siloed and disparate legacy agency approach could no longer address the growing and changing needs of the citizens and taxpayers, the Office of the CIO looked for a better solution to provide enhanced services through a consolidated IT environment. What did we mean by “consolidated?” In short, for code agencies the OCIO would be responsible for infrastructure and enterprise applications and agencies would be responsible for agency-specific applications.
Nebraska’s consolidation is:
- One network – one domain;
- Centralized management of active/hot-standby, geographically separated data centers;
- Hardware managed by the OCIO;
- Servers located in only the state’s two data centers managed by the OCIO;
- Enterprise applications managed by the OCIO; and
- Applications that are specific to an agency are managed by the agency, but the technical infrastructure is managed by the OCIO.
Information technology was moved out of its traditional order-taker role to that of a strategic policy, business and service partner. The OCIO’s initial focus included centralizing information technology policy, strategic planning, project management, infrastructure, and addressing enterprise security needs.
Enterprise applications are managed by the OCIO. Applications specific to an agency are managed by the agency business teams which allows them full control of their line-of-business. Line-of-business services consist of those differentiated services core to the strategic mission of the agency.
Consolidation efforts took place in three phases over 18 months:
- Phase 1: Networks,
- Phase 2: Server Administration, and
- Phase 3: Desktop Support.
A number of new functions, processes and organizational units were also created: a centralized Incident Management team to enhance technology support; the Risk Mitigation and Compliance team which addressed concerns about security; a centralized Geographic Information Systems (GIS) team to address cross-boundary data mapping issues; a consolidated infrastructure organization to remediate reliability issues; and the creation of strong centralized project and change management programs.
During this consolidation, no additional funds were requested to upgrade aging hardware. No outside consulting firm was hired to manage or provide guidance for the effort. It was entirely funded from the existing OCIO revolving fund budget.
Consolidation efforts took place in three phases over 18 months:
- CONSOLIDATION BENEFITS
- TACTICAL BENEFITS
- OPERATIONAL BENEFITS
- STRATEGIC BENEFITS
- RESULTS
- IMPROVED SERVICE DELIVERY
- IMPROVED SERVICE QUALITY
- SAVINGS
- STAFF SAVINGS
- PHYSICAL SERVERS ELIMINATED/VIRTUALIZED SAVINGS
- CONSOLIDATED COUNTY AS400 SERVER SAVINGS
- 93 County (109 State Owned AS400 Servers eliminated)
- 88 County (88 County Owned AS400 Servers eliminated)
- CONSOLIDATED DATA CENTERS SAVINGS
- DUPLICATE SOFTWARE SAVING
- TOTAL SAVINGS
- Staff savings $ 30,083,922
- Server savings $ 2,662,562
- County AS400 $ 1,371,500
- Data centers $ 3,999,705
- Duplicate software $ 239,116
- TOTAL $ 38,356,805
We consolidated technical support by restructuring existing agency support positions located outside of the Lincoln area. We had enough resources to meet staffing benchmarks to support eight separate service centers. This allowed us to locate technical support closer to the customers and gives us the ability to recruit from multiple rural areas of the state. We improved service levels and standardized processes across the state with integrated tools and automation.
The included map shows the eight OCIO support regions.
Operational benefits include the time saved through increased system availability, reduced downtime, and enhanced disaster recovery capabilities. Redundant data centers located in Omaha and Lincoln have the capability of running all critical state applications independently. This allows for failover to the alternate data center when issues occur.
A chart with average system availability uptime over the past 12 months is included in the materials.
With respect to critical issues, which we call P1 and P2 events, you can see in the included charts that the number and duration of these events has steadily decreased since consolidation.
In 2020, the OCIO was presented with the responsibility of transitioning a large number of state teammates to a remote working environment. Because of consolidation, the infrastructure necessary to deliver on this responsibility was already in place and functioning. While many states lacked sufficient capacity to adequately support the demand in usage, Nebraska was not one of them.
Since consolidation completion in 2019 we see a steady declining trend in time to resolve for both incident and service requests with high customer satisfaction ratings.
Customer survey results for the past three years are included in the materials.
2019
2020
2021
Consolidation reduced the need for technology resources by 77 FTEs and 16 contractors with an accumulated cost savings of $30,083,922.
Savings from 362 eliminated and/or virtualized servers Contributed to an additional savings of $2,662,562.
197 county AS400 servers were eliminated and virtualized resulting in an accumulated savings of $1,371,500.
A review of states by the National Governors Association (NGA Center for Best Practices – Issue Brief) found that operating consolidated data centers saves states millions of dollars. The OCIO consolidated into two data centers in Lincoln and Omaha. The agency data centers that were closed because of consolidation provided additional cost savings and eliminated risks due to their lack of threat mediation, adherence to basic best practices, and access control.
Agency specific data centers that were no longer needed amounted to approximately 6,000 square feet of office space (as self-reported by agencies in their agency IT plans). Consolidating these data centers into those managed by the OCIO equates to approximately $4 Million in cost savings to date.
Consolidation allowed us to eliminate some duplicate software. Benefits of eliminating redundant applications include reduced software licensing costs, enhanced availability, and enhanced security. The savings from duplicate software elimination add up to $239,116.
Great teams work towards common goals, accept difficult assignments, and share accountability for results. Those results (good or bad) are determined by objective data.
As always, I appreciate your hard work and respect for the taxpayers of the State of Nebraska, which you show each and every day.
Ed Toner