Challenges: Drowning in data, with backups killing application performance
Mahoning County’s IT Department, led by County Auditor Ralph T. Meacham, and IT Director Jacob Williams, supports 50 county offices and departments operating in 20 locations. The staff of 10 provides and maintains all of the information technology the county’s 1,800 employees need to do their jobs.
This includes applications such as the Common Pleas & Probate Court Case Management System, Sheriff’s Jail Management System, Juvenile Justice Courts Systems, Dog Kennel Management and Licensing, Building Permits, Highway and Sanitary Engineers Systems, and Developmental Disabilities Case Management Systems, geographic information systems (GIS) systems used by a number of offices, the 911 system used by Emergency Management Services, and the county-wide ERP/payroll and tax accounting software used by the County Auditor and Treasurer. This is a very partial list. As Meacham says, “We are the IT arm for the entire county.” In addition to maintaining the applications, the IT team manages the network, servers, backup and DR systems, etc.
IT services were introduced to county in the early 2000s and were originally delivered from servers located at eight different sites across the county to provide the optimal computing environment for the majority of users. As the volume of each office’s data grew, backing it up to a central location became a serious problem.
“Over time we accumulated mass sets of data that became our ball and chain,” says Williams. He explains that the majority of the data that county agencies deal with is a public record or potentially a public record, and as a result, nothing can be deleted. “About a year and a half ago we started to realize that there's not enough time in the day to get all of the backing up done,” he adds.
It had gotten so bad that some backups that started on a weekend, and should have been finished by Monday morning, didn’t complete until Tuesday or Wednesday. When backups spilled over into the workday, application performance was terrible. At times it got so bad that court employees, for example, switched back to paper processes until performance improved. This essentially doubled their workload since they had to enter the information into the application eventually.
Williams tried adding bandwidth. He initially had T1 lines connecting the remote sites to the data center. “So we went to five-meg and 10-meg connectivity and it became a cycle of needing more and more bandwidth,” he recalls. With the data volume growing exponentially, this was an expensive, and ultimately futile, strategy.
Application performance wasn’t the only problem related to backup-induced network saturation. As long as the problem persisted, Williams couldn’t automate some of the offices and departments that really needed automation because he couldn’t risk putting more data onto the already burdened network. A digital case management system for the dog warden’s office is a good example. “They were still paper-based and we really needed to bring them into the twenty-first century,” Williams notes. “We needed to come up with new ideas and new ways of delivering a good computing environment, while at the same time covering the issue of data security.”
VIEW MORE VIEW LESS