Here’s something to think about: Despite the fact that IBM stopped manufacturing personal computers in 2005, deciding instead to focus on higher margin business services, the company continues to produce mainframes and by all estimates will continue manufacturing them well into the future. The question is, why? The answer: because there is still significant demand for this type of hardware.
The fact is that a mainframe is a monster when it comes to computation and processing transactions. One mainframe can process 2.5 billion transactions daily. That’s the equivalent of 100 Cyber Mondays in a single day. To give you a clearer idea of the immensity of scale, keep in mind that Cyber Monday sales in 2018 were $7.9 billion. This translates into a mainframe being able to handle a bit more than three-quarters of a trillion dollars in transactions each day, which is almost twice the yearly GDP of Austria!
As I said, the mainframe is a workhorse. Yet, its power and potential are hidden from most people who work in the modern cloud. This is surprising because there’s a good argument to be made that, given its power and scalability, coupled with its support for a large number of concurrent users, the mainframe was the cloud before there ever was a cloud. However, the technology is practically unknown among younger developers, mostly due to cultural reasons and a lack of historical awareness. That more people are not taking advantage of the tremendous opportunities at hand is a misfortune. When it comes to mainframe computing, there’s a lot of interesting tech to be learned and a good amount of money to be made. It’s just a matter of increasing awareness to do this. A good place to start is to redefine the narrative.
Allow me to elaborate.
A Special Club
When mainframes appeared on the commercial landscape, few people interacted with computers and even fewer worked with them directly as programmers or system administrators. Just as in a modern data center, the machines were isolated in climate-controlled rooms to which few had access. Access to the actual physical machine was restricted to a special few. But unlike today, when restricted access is due more to security concerns because there are a lot more people around who are potential black hats, back then access was also restricted due to the exclusivity of the profession.
In the 1950s, 1960s and well into the 1970s, not just anybody could program or be a systems administrator. You could go to a technical school to learn to become a computer operator, which meant you could work at a terminal as a bank teller or travel agent. Or you could learn to become a data processing clerk and work in a back office at a brokerage firm. But to learn to program, you had to go to college. Then after college, you had to go to work for a company or organization that actually used computers in day-to-day operations. These were banks, insurance companies, major manufacturers such as General Motors and General Electric, airlines, very large retailers and government. Computing was a club and you had to be well-educated to get in and be well-liked by the organization to keep your membership. Yes, there were the outliers—Gates, Allen, Joy, Wozniak and Jobs—but for most people, the career path to being a programmer was college and then into big business or government.
As a result, computing had an air of mystery and magic to it. Remember, until the mid-1980s, most people had never touched a computer. It was the stuff of moon landings and election night predictions on network TV. Then the PC came along and changed everything. Within 10 years computers became commonplace. Even elementary schools had them—at least, those that had the budget and foresight. The power that business applications such as the spreadsheet program Lotus 1-2-3 brought to small business motivated even the dry cleaner on Main Street to plunk down the thousands of dollars it cost to buy a PC. The productivity boost was worth it. Also, since the dry cleaner actually owned the machine, the dry cleaner’s kid could learn to program on the box unencumbered, when it wasn’t being used during business hours. Remember, the Microsoft DOS operating system that shipped on every IBM-compatible PC came bundled with the programming language BASIC. You didn’t need to go to college to learn to program; all you needed was a book.
So, PC programming took off. Instead of having to go through the effort required to get into the special club the mainframe programmers belonged to, PC programmers started their own clubs and anybody could get in. All you had to do was show up. PC programming became a meritocracy. Also, it became extremely lucrative work. There were a lot of PCs being shipped and the appetite for applications to run on them was enormous. The unlikeliest of people, such as Lotus Software founder Mitch Kapor and his sidekick, Grateful Dead lyricist John Perry Barlow, became wealthy, thanks to the PC.
The PC became cool and programming for it became cooler. But there was a fact that got obscured as the popularity of the personal computer grew: Compared to the mainframe, the PC was a toy—a productive toy, a liberating toy, but a toy nonetheless, very much in the same way that a go-kart is a toy compared to a Formula One race car.
Feature for feature, the mainframe ate the personal computer’s lunch. The IBM System/370 Extended Architecture Mainframe released in 1983 was a 31-bit machine that could support up to 8MB of memory with a processor running at 38 MHz. By comparison, the IBM PC-XT, also released in 1983, was an 8-bit machine that shipped with 128KB of RAM and an 8088 processor that ran at a speed of 4.77 MHz.
But, as the saying goes, “You should never underestimate the value of cool,” and the PC was very cool. It got a lot of attention while the mainframe stayed hidden away in climate-controlled rooms in the basements of banks and insurance companies. Yet, the mainframe was more powerful, and whereas the PC was meant to meet the needs of a single user, a mainframe can service thousands.
Built to Scale Built from Inception
Mainframe computers from the start were designed to support hundreds (if not thousands) of users. However, more importantly, mainframes were and are designed to support millions of concurrent transactions. When you think about it, while it’s possible for a high number of user requests to exhaust CPU utilization in a typical commodity machine—which is the reasoning for being able to scale up CPU count by increasing VM allocation in the cloud-based computing cluster—the real burden on the CPU is not so much the request count but the computing burden that each request incurs. For example, one request getting a .JPG of a cat will incur nowhere as much burden as a request that needs to convert an audio file in .WAV format to MP3.
The need to alleviate request burden has been around since bank tellers working in the various branches of a big bank had to make deposits and withdrawals at the same time. Mainframe engineers addressed the problem decades ago. Technologies such as batch processing and message queues, which make computational activity asynchronous and are optimized to fully utilize CPU capacity efficiently, were and are commonplace in the mainframe landscape.
IBM’s 360 mainframes supported batch processing in its OS/360 system starting in 1964. Batch processing allows thousands of instructions to be performed on terabytes of data at pre-scheduled times (usually “off-hours”), all without human intervention.
Message queuing allows processes to take data in the form of a message stored in a message queue and perform computational activity according to the data in the message. Processing is performed independent of a request in an asynchronous manner. When processing is complete, a new message is created and sent on to another queue, where another process will do more processing or create a notification indicating that work is complete. IBM MQSeries was an early message-queuing technology first used in 1993 at State Street Bank well before the internet became commonplace.
The reason that support for user and transactional concurrency was possible in mainframes is due to a technology that is commonplace in today’s cloud environment but was unheard of among PC users back in the days when most data was exchanged between standalone PCs using floppy disks. That technology is virtualization.
Virtualization Takes Mainframe Computing to the Next Level
Today we take virtualization—the capability to have software simulate hardware—for granted. Without it, we couldn’t have native cloud providers such as Amazon Web Services, Google Cloud and Microsoft Azure. These services rely on virtualization to provide the segmentation and computational isolation required to meet the needs of the modern computing. Yet, as essential as virtualization is in the world of commodity cloud computing, putting a layer of virtual machines on a commodity PC is a fairly recent event in the history of computing. VMWare, VirtualBox and KVM became mainstream in the mid-2000s. On the other hand, virtualization has been part of mainframe technologies since the mid-1960s. Virtual memory appeared in the mainframe in 1965. Machine virtualization was introduced on the IBM 370 mainframe in 1972 and is still an important part of the mainframe computing experience today.
Every single person who logs onto the mainframe when working in a native mainframe operating system gets an identical address space. You can think of each address space as a standalone virtual mainframe system. These address spaces are created instantly and are protected from interfering with each other. Also, many companies create logical partitions of their hardware called LPARs (logical partitions). The logical partitions are controlled by a native firmware hypervisor called PR/SM (pronounced Prism).
The value that LPARS provide is a further separation between environments. This means not only can you separate users, but you can also create distinct computing environments such as those created when virtualizing X86 machines. For example, you can have a shared development environment as well as environments for testing, staging and production. You can allocate resources and security access to a particular environment based on need, just like you would in a data center full of commodity machines. This means you can configure production systems to be more secure and have higher execution priority and more resources such as memory. In fact, according to the U.S. government, from a security perspective, virtual machines created on the mainframe are considered distinctly separate machines.
LPARs running on the same hardware or different hardware under the native mainframe z/OS operating system can be linked together as a Sysplex for redundancy and to easily provide additional capacity. Dynamic Virtual IP Addressing (DVIPA) is used to dynamically route IP traffic across multiple LPARs in a Sysplex. This is analogous to being able to tie together a variety of web servers behind a load balancer or organize them within a Kubernetes cluster. Being able to tie computers together is an important feature for making systems scalable. Also, linking computers across geographic regions increases performance. It’s always faster to work with a machine that’s closer to you than one that’s further away. As mentioned earlier, being able to scale systems up and down is a key feature of cloud-based computing. Dynamic connection, which is part of mainframe architecture, is critical for scaling to work.
The important thing to understand is that the stuff we consider to be modern in terms of the machine virtualization that we rely upon to make the cloud work has been in play for years on the mainframe. Also, virtualization on the mainframe makes it possible for programmers and system administration staff to work in a variety of operating systems and programming languages as we’ve come to expect in commodity cloud instances. Machine virtualization allows mainframes to support Linux. Whether the computing environment is virtualized Linux or native mainframe, developers can code in Java, C/C++, Python, Perl and a variety of other languages, including something as new as Swift. But, then again, the mainframe was always intended to support many programming languages.
Even back in the ’60s and ’70s, programmers could write in COBOL, PL/I, job control language (JCL) and Customer Information Control System (CICS). Also, writing low-level code in Assembler has always been possible. Clearly, the versatility that modern developers enjoy programming in a variety of languages in a cloud environment has been part of mainframe computing for a while. Yet, the benefit has never been at the forefront of developer awareness. Maybe it’s time to change the narrative.
Changing the Narrative About Mainframe Computing in the Cloud
As I stated at the beginning of this article, IBM continues to make mainframes because there’s a growing demand for the technology—particularly as smartphones and IoT devices proliferate worldwide. Netflix might be spinning up containers on a commodity machine in a data center to let you view past episodes of “Orange is The New Black.” But when it comes time to pay for your Lyft ride using your iPhone, that credit card transaction will most likely be processed on a mainframe that’s providing service via some sort of API gateway.
There is little benefit to arguing that mainframes should or can be used interchangeably with commodity servers. That’s like saying a fleet of automobiles can be used interchangeably with a city bus. You can do it, but it doesn’t make much sense. Each technology has benefits that when used together, make the overall transportation system more efficient. The trick is to understand where and how the technologies fit together in the big picture. In this case, the big picture is the cloud.
The old narrative around mainframe technology is based on perceptions that are no longer applicable. The modern mainframe is no more centralized than the single building that houses a regional data center. In terms of programming languages, mainframes moved beyond FORTRAN and COBOL decades ago. Current mainframe virtualization technology makes it possible to run programs written in just about any programming language. And, mainframe emulators, such as ZD&T, make it possible to get the hands-on experience necessary to get familiar working in a mainframe environment.
The Perceptions of the Past are No Longer Applicable
The modern mainframe is a powerful resource that provides extraordinary benefits for cloud computing. The new narrative is that the mainframe is modern, adaptable and well-suited to accommodate a very high volume of transactions at lightning-fast speeds. And, it’s a good way to make good money.
As the current workforce (which is admittedly aging) approaches retirement, a new breed of mainframe developer will be needed to continue the work of making the mainframe a mission-critical resource in the modern cloud. In addition to understanding the essentials of mainframe computing, these developers will need to be well-versed with the modern tools and techniques that drive cloud computing. It’s an investment that I’ve urged many up-and-coming developers to make.
Putting It All Together
Let me share a piece of personal history: A while back I needed to do some programming. So, I went to my desk, logged into my system, wrote some code and ran it. It worked like a charm! I saved my work, finishing up just in time for dinner. A while later, it was again time to program. I went back to my desk, logged into the system and coded away. Just another day of slinging the code, right? Well, not really.
My first coding session was in 1975. “A while later” is today. Forty-four years have passed. Back then, I was working on a mainframe that was 50 miles away. Today, I logged into an online programming environment that was also miles away, maybe even halfway around the world. That’s how it is with cloud computing: The resources are any way at any time.
Interestingly, despite the passage of a significant amount of time, my current developer experience is not that different than the one I had decades ago. It’s still just me and a terminal. Back then, the terminal was a screen full of green characters. Today, my terminal is able to accommodate a variety of media and input devices. But, at the other end of the wire, who knows what’s going on? For all I know, I might still be connecting to the same mainframe I used back in 1975. Which is my point.
Much of what we consider new and wonderful about cloud computing today has actually been around for a while in mainframe technology. In many ways, the mainframe was the cloud before the cloud. And, it’s still a powerful resource for what we have come to know as modern cloud computing. Given its power, versatility, scalability and support for modern DevOps tooling, the mainframe is a perfect resource for today’s and tomorrow’s cloud computing infrastructure. It’s a computing resource that we’ll do well to take advantage of as we move forward in our development efforts on an increasingly connected planet.
— Bob Reselman
As part of that effort, Chef Application Operations Dashboards provide visibility into both applications and infrastructure to simplify troubleshooting and compliance, said Goldfarb. For example, IT teams will be able to see how individual security and compliance controls are performing across a heterogeneous computing environment that now includes Microsoft Azure and Google Cloud Platform (GCP) alongside Amazon Web Services (AWS), he said.
