Why are we still running software on mainframes?
The state of New Jersey recently put out a call for COBOL programmers to help fix problems with their unemployment system. The NJ system…
The state of New Jersey recently put out a call for COBOL programmers to help fix problems with their unemployment system. The NJ system is straining to handle the considerable amount of additional traffic resulting from the COVID-19 pandemic layoffs. I’m not surprised that New Jersey’s unemployment application system is running on 50-year-old software. I’ll hazard a guess that it roughly conforms to the Lindy Effect as described in Nassim Nicholas Taleb’s “Skin In The Game.” The Lindy Effect takes its name from New York’s famous Lindy Delicatessen, located near the Broadway theater district. Actors that frequented Lindy’s noticed a trend that a show that was running for a hundred days was more than likely to remain open for a hundred more. (Read the book, it’s terrific). If we apply the Lindy Effect to New Jersey’s problem, it starts to make sense. A system that’s been running for twenty years is likely to stick around for twenty more.
Old mainframe systems still abound for several reasons. Let’s start by considering why we built many large-scale systems on mainframes in the first place.
It’s only relatively recently, say the last twenty years or so, that we have computing platforms that could replace mainframes for certain types of applications. Before the mid-2000s, IBM’s mainframes were the only computing architecture that could handle many large-scale business data processing problems. You might be able to assemble a suite of technologies to replace the mainframe, but it would require a complicated mix of software from different vendors to match the good old mainframe. I never wrote any mainframe code; I started my career on DEC equipment. However, I have enough experience working with DEC’s ACMS and Tuxedo on Unix to appreciate the complexities of competing against a mainframe with COBOL/CICS for large-scale business software. If you were starting to build a high-volume, secure, reliable application in the late-2000s, you would not have chosen a mainframe as the backbone. However, pre-2000, mainframes were still the leading candidate for building any large-scale business application.
[Side note here. There were some large-scale applications built on proprietary minicomputer platforms from Digital, Prime, Data General, Sequent, Pyramid, HP, and their brethren during the 70s, 80s, and most of the 90s. However, the era of the proprietary midrange operating system declined as the Internet software era began. You should also remember that IBM was the brand of choice through the late 1980s when we built many of these systems. Nobody ever got fired for choosing IBM].
Client/Server applications built in the 90s often relied on big database servers running on midrange computers or mainframes, with much of the heavy-duty processing happening on the “backend” using server-side code built with other programming languages. — PowerBuilder on the desktop and COBOL and a database on the server. The changeover to a web-based architecture in the mid-2000s replaced the desktop PowerBuilder (or Visual Basic, SQLWindows, etc.) apps with HTML pages that front-ended the older mainframe code via an application server.
It’s often prohibitively expensive to replace mainframe applications, and; IBM continues to invest in mainframe hardware updates, so prices continue to drop on a per-MSU basis. I guarantee you that New Jersey is not running on 50-year-old hardware. They probably leased the equipment, and IBM swaps it out every few years with each lease renewal. The “iron” has gotten cheaper, while the cost of a software re-write for a complex system is massive and growing. Thus, the annual “cost” of running these legacy applications is a fraction of the replacement costs.
Business applications in the public sector don’t “compete” with commercial solutions — making legacy systems more common in the public sector.
I’m betting that if I looked at the changelogs (provided that there are any) for the New Jersey unemployment system, I would hazard a guess that there haven’t been too many extensive business-logic changes on an annual basis. You can access the New Jersey unemployment application via their web site, so the state already offers a web-based interface to applying for benefits and checking your application status. Users get a clean, simple way to interact with the software, but based on the news articles, there is a ton of functionality running in the background on the mainframe — written in COBOL (and likely CICS). The state retrofitted the user interface with HTML, and that’s been good enough up until now.
New Jersey is finally starting to reach the limits for keeping its unemployment application up and running. (Which I’ll talk about in my next piece). Replacing it will be very costly, at a time when state budgets are going to be stretched extra-thin.