Tag: mainframe

  • Compuware Advances DevOps on Mainframes

    Compuware Advances DevOps on Mainframes

    Compuware has updated its Topaz for Total Test suite to enable organizations applying best DevOps practices to the development of mainframe applications to combine unit, functional, integration and regression testing within a single test scenario.

    In addition, Compuware is making it easier to test multiple applications simultaneously. The latest update to Topaz for Total Test adds the ability to not only orchestrate a sequence of steps, but also design tests to measure how programs interact with each other and various data sets.

    Also added is the ability to automate several testing functions, including the ability to launch functional tests within a browser and execute unit, functional and integration tests without having to manually create their own Job Control Language (JCL). According to Compuware, testers now can design test cases upfront based on program specifications before developing a program. What’s more, Topaz for Total Test has been integrated with the DevOps pipeline management platform developed by XebiaLabs.

    Finally, Compuware announced that the Topaz 3270 Emulator now makes it possible complete one-off “green screen” tasks without leaving Topaz. Initiating a CICS transaction or accessing REXX utilities through the Emulator within Topaz, for example, makes it simple for developers to quickly resume Eclipse-based tasks without having to exit its testing environment or toggle between tools.

    Compuware CEO Chris O’Malley said the pace at which IT teams in mainframe environments have been adopting DevOps practices has accelerated considerably in the past year. Not only are applications being developed and deployed faster, the quality of those applications has increased because DevOps tools provided by Compuware make it easier to both visualize a complex application development process and then test those applications as they are being developed. Given the often mission-critical nature of mainframe applications, it’s crucial that any increase in development velocity doesn’t have a negative impact on application quality, he noted.

    DevOps has become a greater imperative in mainframe environments, O’Malley added, as organizations look to leverage investments in existing mainframe applications rather than try replicating functionality that already exists on a mainframe in a distributed computing environment. That approach helps to reduce sharply the amount of technical debt an organization needs to service when they duplicate functionality across two or more platforms. Mainframe teams that historically have relied on waterfall developments process have been challenged by keeping pace with the rate of change. As applications that are deployed on a distributed system look to invoke a mainframe application function, organizations that have deployed those applications need to keep pace with a much faster rate of development, especially when it comes to application programing interfaces (APIs).

    It may take a while longer for DevOps practices on mainframes to become adopted widely. But as the rate of innovation continues to increase, thanks to the rise of various digital business transformation initiatives, it’s apparent that mainframe application development will need to become more agile. The challenge at this point is not so much the availability of DevOps tools for the mainframe, but rather training the teams that build and deploy those applications on when and how to implement DevOps practices.

    — Mike Vizard

  • ABO and the Mystical Art of Source Code Compilation

    ABO and the Mystical Art of Source Code Compilation

    Compilation can make or break the performance quality of an application. A developer might spend weeks creating the most elegant, efficient algorithm known to man, only to have it run at a snail’s pace in production because the compiler used to create the application’s executable binary is old, and not optimized to take advantage of the enhanced performance features offered by modern hardware.

    When it comes to getting the most out of code, companies in the know figured out a while ago that compilers count. Yet, for the most part, they remain a complex—if not somewhat mystical—set of technologies. I’ll admit it, for the longest time my understanding of compilation and compilers went only so far as running a make or build command. I was woefully ignorant about what was going under the covers.

    But that was then, and this is now. Recently, I’ve been taking an interest in IBM’s Automatic Binary Optimizer (ABO) technology. ABO is a technology that optimizes existing COBOL code to run better in mainframe environments, without having to alter source code or modify existing compilations.

    ABO intrigues meet for two reasons. First, it spurs my curiosity about mainframe compilation in general. Second, it makes me wonder how it’s possible to refine code that is already compiled and operating in a production environment.

    So, I decided to go beyond my comfort zone and find the answers to some of the fundamental questions around this mysterious process. Exactly how does a mainframe compiler work? What’s the difference between a good compiler and a bad one? And finally, how does a low-level optimization technology such as ABO do its work, given that it’s being applied to code already compiled?

    The following is the result of my research.

    Understanding Front-End and Back-End Compilation

    Compilation is the process of taking text-based source code and transforming it into a binary file that is executable against a specific piece of hardware. Mainframe computing compilation is divided into two parts: front-end compilation and back-end compilation. Front-end compilation has absolutely nothing to do with clients or GUIs. Rather, the purpose of front-end compilation is surprisingly similar to that of linting, as applied to interpreted languages. Front-end compilation is the process of making sure that the text-based source code is grammatically correct in terms of syntax, semantics, type usage and function declaration.

    Once source code is subjected to front-end compilation, it’s passed on to the back end compiler for further processing.

    Back-end compilation is the place where the artifact created by the front end, called the intermediate representation, is made into the binary format specific to the particular operating system and CPU of the targeted hardware. This binary file is what gets run on the targeted system. (See Figure 1).

    Figure 1: Compiling COBOL source code transforms it into an opaque binary.

    Back-end compilation is where the rubber meets the road in terms of machine-specific performance. A well-designed back end compiler will create binary code that’s optimized to take full advantage of the CPU architecture of the target system, squeezing out every bit of performance benefit possible. A good compiler will create a binary file that is fast to load and fast to run. However, if the underlying source code is inefficient—for example, having code that makes unnecessary trips to the database, thus creating episodic increases in network latency—no amount of back-end compilation is going to improve such runtime inefficiency. However, given that the source code is well-written, a good back-end compiler will improve overall application performance significantly. A compiler that is just so-so can do more harm than good; thus, the importance of compiler technology in the software development life cycle.

    Yet, for all the benefit that it provides, there is a potential problem that is inherent in any piece of compiled code. Once source code is compiled into an executable binary, it becomes difficult to maintain—particularly when the source code is no longer available.

    Compilation Creates Challenges

    Unlike interpreted languages such as Python or JavaScript, in which source code is stored as plain text on a machine and executed at runtime by an interpreter, compiled code exists in a binary format that is opaque to human inspection. It’s machine readable only. Once the code is compiled, it’s “locked down” and opaque.

    This creates two challenges. First, because the binary is locked down, it uses only the instruction sets that were available on the targeted system at the time of compilation. Thus, even though hardware that supports more robust instruction sets evolves, older code compiled for older systems will not be able to take advantage of the features of the modern hardware. All the compiled code can do is execute the instruction sets that were available on the older systems. The result is new hardware, yet old performance. The compiled code just doesn’t know how to move any faster.

    The second challenge is that while it’s easy to change programming behavior later on by rewriting the source code from which the compiled code is created, there’s no easy way that compiled code can be changed directly. Thus, maintenance developers are left in a lurch should they need to upgrade an application for which the source code is unavailable.

    In the world of mainframe computing, this is a big problem. Don’t forget that 80 percent of corporate data still resides in mainframes. There’s a lot of code out there that needs to be upgraded.

    More than one company has been in the situation of having to update code that is decades old, yet both the source code and the person who created the code is nowhere to be found. The result is that companies are left with an asset that is not realizing its full value. The code remains static while significant technological advances have taken place.

    There are tools that can be used for decompiling the binaries back to the original source code against which changes can be made, but the process is time-consuming and error-prone. Even if the decompiler gets it right and manages to reproduce source code that accurately reflects the original version, you still need to find a developer who can make the required changes. Such developers are in high demand in the mainframe world. Many companies simply prefer to leave well enough alone and allocate their mainframe developers to projects that have the highest return on investment.

    Clearly, having a way to transform old, legacy binaries into ones that are optimized to run on modern mainframes, without having to go through a complicated decompilation and recompilation process, will benefit any company that needs to get better code to market, faster and in a cost-effective manner. Fortunately, IBM has provided a way with its ABO technology.

    ABO: Solving the Compilation Challenges

    ABO is designed to modernize old code without having to go through an arduous compilation/recompilation process, as described above. The way it works is interesting.

    ABO works directly on the compiled binary to re-create its program structure as represented in the intermediate representation. The program structure is then passed onto the optimizer, which recompiles a new binary. This new binary takes advantage of the additional instruction sets offered by new hardware. And, it does so in such a way that the original programming logic is not altered in any way. (See Figure 2).

    Figure 2: ABO performs optimization on program structures extracted from the legacy binary without affecting program logic.

    For example, ABO optimizes standard math operations in COBOL such as SUBTRACT, MULTIPLY, DIVIDE and EXPONENTIATION by creating instructions in machine language that are more elegant and execute faster. No doubt, improving math operations is significant considering that COBOL is primarily used for business-oriented number crunching and correlation (being able to do math faster really counts, no pun intended), yet the important thing to understand is that the essential value of ABO optimization is that it’s taking advantage of the advanced instructions sets offered by the new target system. Remember, running older compiled code on new systems is not going to make that much of a difference in performance. In the PC world, it’s the same as running 32-bit code on 64-bit machines. Sure, in most cases it will work, but the code will never go beyond the efficiency of the low-level instructions supported by 32-bit processors.

    Yes, there is some performance boost that can take place without having to recompile due solely to the improved clock speed of the CPU. However clock speeds on new mainframe models are beginning to plateau. As clock speed levels off, so will the time it takes for code to execute on the processor. This fact has not been lost on mainframe designers. New ways of enhancing overall performance, beyond simply moving the bits through the processor faster, are being devised. Sadly, there is no magic in play; for legacy code to realize better performance that’s not dependent solely on clock speed, the code will need to be compiled.

    Recompiling from scratch using the source code is viable, but doing so is expensive. And, if that recompilation affects any part of the source code, additional testing needs to take place. Thus, the cost of change is increased due to both the expense of hands-on development and the added testing.

    ABO, on the other hand, works with existing binaries. The result is an upgraded binary that’s optimized to the latest mainframe technology. In addition to having state-of-the-art code that’s fast and efficient, a company saves time and money in the refactoring process. It’s pretty amazing when you think about it.

    What does such increased efficiency look like? IBM is reporting 24 times the reductions in CPU consumption time. That’s a substantial number.

    Now, granted we’re talking about optimizing existing functionality. Refactoring an existing application to add new features requires code rewrites, no question about it. Yet, there are a lot of applications in play today that are doing what’s required. The problem is that they’re performing poorly and slowing down workflows overall. Optimizing existing binaries means not only that applications can have better performance, but that entire workflows become more efficient as more room is made for other applications to run.

    Putting It All Together

    Making legacy code better is, and has been, a continuous burden for companies since the first days of computing. Updating code is difficult enough when the source code and developer who wrote it are available to do the work. When both are absent and all that’s left behind is an executable binary, refactoring the application becomes a herculean set of tasks. Decompilers will only get you so far.

    Fortunately, an optimization technology such as ABO makes things easier. ABO modernizes legacy binaries without the need to have the source code on hand. The result is improved system performance, reduced costs and faster release cycles.

    As those of us who have been out on the terrain have learned, the rate of change in computer technology never slows down. It only goes faster. Companies that can adapt to the breakneck speed of technological innovations will prosper. This includes companies that rely upon mainframe technology to get work done. Reducing the time it takes to optimize legacy code is essential to stay competitive in today’s marketplace. For those companies that rely upon mainframe technology to get work done, ABO’s ability to optimize code directly against existing binaries provides a transformational advantage that is difficult to ignore.

    — Bob Reselman

  • DevOps Chat: Mainframe DevOps Update With Compuware’s Chris O’Malley

    DevOps Chat: Mainframe DevOps Update With Compuware’s Chris O’Malley

    Christopher O’Malley is one of the leading voices in the DevOps for mainframe community. He is relentless in advocating how the mainframe platform can not only exist but also thrive in today’s Agile/DevOps environments. His evangelism keeps his company, Compuware, front and center in the mainframe DevOps discussion and it really lifts the entire market.

    In this DevOps Chat, we catch up with Chris to talk about the latest release of Topaz, trends in the mainframe market and what we can expect in the near term.

    Chris is always a great interview with lots to say and this interview is right on point. As usual, the streaming audio is immediately below, followed by the transcript of our conversation.

    Transcript

    Alan Shimel: Hey, everyone, it’s Alan Shimel, devopsy.kinsta.cloud, and you’re listening to another DevOps Chat. This DevOps Chat features a return of our friend, Chris O’Malley, CEO of Compuware. Chris, happy 2019 and welcome back to DevOps Chat.

    Chris O’Malley: Thank you and glad to be here, Alan.

    Shimel: Absolutely. You know, Chris, it is. It’s 2019; DevOps continues to mature. And I think one of the trends that we’ve seen, certainly over the last year and I expect it to even accelerate this year, is changes are coming to the DevOps and mainframe market. Right? Mainframes aren’t going anywhere; in fact, shops that are using mainframes, I think their footprints are increasing, rather than decreasing.

    And, from what we see in our readers and in talking to them, is that they are eagerly anticipating new developments, new functionality, that their mainframe software will offer, that will allow them to leverage DevOps types of foundations, DevOps types of functionality. And, I mean, DevOps and Agile and, really, code word: continuous delivery, all of these new ways that we’re defining the modern software factory, if you will. You’re in the driver’s seat for it, right, Chris? I mean, Compuware is kinda leading this charge. What do you see – you know, put on your analyst glasses here. What do you see in the market?

    O’Malley: Well, first, just confirming what you said about the mainframe, is that the z14 has been awakening, I think, for customers and analysts, in terms of its success, so the z14 is the most successful mainframe model in over a decade. It’s had five consecutive quarters of year-over-year growth, which probably hasn’t happened in maybe forever. And, as you said, the reason that’s happening is the mainframe is still unmatched in terms of its reliability, its performance, its security, its transactional efficiency. And all of those things, all those virtues, are incredibly important in a digital age, where you try to basically create a customer experience through software. So the mainframe, as you say, it’s not just doing well; it’s actually doing better than historically because of the digital age.

    Shimel: Mm-hmm.

    O’Malley: So, just giving my disposition on the mainframe and the mainframe’s role within DevOps, I think, if you kinda zoom out a little bit to make sense of what’s happening to the mainframe, you gotta start first with the fact that all of these companies that use mainframe are of a mindset that “We’ve gotta be customer-centric. We’ve gotta serve our customers, whether they’re individuals, if they’re federal agencies, whether they’re citizens or it’s business-to-business.” But, for 100 years and for 100 years from today, companies that are the most customer-centric, obsessed with customers, do the best.

    And, when you look at kinda this modern age, to do well from a customer-centric perspective, you’ve gotta create awesome digital experiences. And that’s done by writing software. So companies – you know, and Amazon has taken the lead in a lot of these kinda more newer-age companies. They’re incredibly good at taking ideas that matter to customers – and, again, it’s usually through digital means, increasingly – rapidly turning those into deliverables that make a difference. And, whether it’s centered on creating these awesome experiences and when they kinda look at the measures of what they’ve gotta do better to turn those ideas into deliverables, they’ve gotta be really good at velocity, how fast they do those things. They can’t do it in ways that compromise quality, so quality has gotta be something that’s constantly getting better and they’ve gotta become more efficient.

    I mean, in the manufacturing space, they talk a lot about getting rid of non-value-added efforts and thinking about retooling things that are a cause for technical debt, that encumber your ability to increase velocity and improve quality, so you’ve gotta fixate on the efficiency side of it. So these three metrics are emerging as foundational to improving things like customer sat and improving financial performance.

    So, when you start now getting closer to large enterprise IT, you’ve got a mainframe, these ideas that matter to customers don’t neatly fall either on the mainframe, in terms of the coding work that’s gotta be done or the cloud or some other platform, almost always, these ideas span these platforms. So there’s a necessity to improve velocity, quality, and efficiency across the entire enterprise.

    So, you know, the mainframe, historically, has been this kind of silo-ed organization that had its own culture, its own set of processes, its own set of tooling. And, as you said, the mainframe’s not going away for all the right reasons. In terms of creating these awesome experiences, reliability matters. Performance matters. Security matters. How efficiently you do that, from a cost standpoint, matters. But you gotta draw the mainframe to be a like system and make it different only in syntax from these other platforms.

    So there’s a strong move within these large enterprise ITs to break down this silo, get rid of a house divided against itself, bring the mainframe into a culture that’s centered on innovation, not on maintenance, bring it from waterfall to Agile, to bring it to a tooling set that’s familiar to non-mainframe platforms, the products like Jenkins and SonarSource and SonarLint or Paris. All these type tools that are used aggressively to improve velocity, quality, efficiencies on these other platforms, bring these to bear on the mainframe.

    So that’s what’s causing, I think, the mainframe to be in this kind of disrupted state of trying to get the culture to move, getting people to think and behave differently, to embrace this kind of new order of how work is done, Agile techniques specifically, and to get away from – you know, the mainframe has always been selectively automated in incredible ways. I mean, the transaction volumes it can support are unmatched within the marketplace, but it’s selective automation, not intrinsic automation, which is a big part of what DevOps tries to push. And getting rid of non-value-added work on the mainframe and getting to the point that the value-added work can be done, again, faster, greater-quality, greater-efficiency.

    So that dynamic is now gone from kind of innovators and early adopters within the mainframe market, you’re now getting into more of the early majority, so you’re seeing it more prod-based, in terms of people looking at the mainframe as that “We’ve gotta make this like any other platform.” And I think the success of the z14 makes sense, why it’s doing so well. Not just better than the z13, but better than previous releases over in the last 10, 20 years, so it’s exciting times in the mainframe. And I think its success is gonna be something you see emerging over the next ten years.

    You’ll see fits and starts and, obviously, problems, but one of the things that’s making me so excited – you’re seeing successes in the market that are outrageous. We’ve got customers at Compuware that are increasing their development productivity by a factor of 4, I mean, a 400-percent increase in productivity, so they’re getting 4 times the number of story points completed within a release cycle. They’re taking testing time from weeks to minutes and with better quality. And, when that is understood by customers, you know, if you can go from weeks to minutes, that’s a lot of non-value-added work that you’re doing, right? That’s like burning money in the parking lot because a customer’s not gonna give you more money to do the same or less quality in what you can do in minutes, as opposed to somebody that’s a laggard doing it in two weeks.

    Shimel: Absolutely.

    O’Malley: So you’re seeing these amazing success stories. Again, it’s not just our tools; it’s _____ –

    Shimel: No, no, it’s important – Chris, it’s that 10X, right? Everyone’s looking for that 10X. And we’re starting – you know, 4X is great; 5X is better. We’re getting those – we’re hearing success stories, the 10X success stories, with mainframe. And it goes back to something else you said, that, instead of people saying, “Oh, I’m winding down my mainframe or I’m migrating mainframe,” no, I’m looking at a ten-year roadmap on mainframes. It’s not – a couple years ago, there was this – I don’t know – fake media, fake news, about “Yeah, well, it’s here now, but we gotta kinda wind down.” What you’re saying about the z14 is exactly right; it’s bigger than anything that’s come before it, in essence, right?

    Here’s another thing, though, Chris, and I’m interested in your take: kudos to the mainframe software market and vendors, who are recognizing – not all of ’em, not all of ’em, but the ones who are, the enlightened ones, who are recognizing this and are capitalizing on it by bringing the tools the market wants to bear. It’s more than just the hardware, right? The hardware’s phenomenal – I’m not gonna deny that; IBM has done amazing with the z14 – but the software vendors, including Compuware, right – take a bow – have really delivered. You guys have a new release coming out or just came out, is that right?

    O’Malley: Right. It’s the 17th consecutive quarter that we’ve come out with new capabilities, upgrades to our classic offerings and integration with awesome and preferred DevOps tools, so, yeah, we’re very excited about this newest release.

    Shimel: So talk to me. Give me the three biggest things that get you excited on this release, Chris.

    O’Malley: So, each quarter, we’re turning the crank in terms of trying to nail the jobs that are changing, right? As I explained earlier, when you’re going from waterfall to Agile and from selectively automated to intrinsically automated, you’re trying to go from maintenance to innovation, the jobs that people have to do change radically. And they change radically in ways that are difficult to achieve. I mean, when you’re working on the mainframe, you’re working with large code bases, large in terms of the quantity of those code bases and also the size of any one of them.

    So these techniques, at a high level, just seem obvious, but the practical things that need to be done from an innovation standpoint to help people to nail these things, it takes a lot of innovation. I mean, we’ve had to do a lot of work to get to the point where you’re seeing orders-of-magnitude improvement in the way people develop, the way people test, so this latest release is just another chapter in that story.

    So we came out with Topaz team profiles, so that’s a good example of a capability that allows those that are working on systems to basically be able to store their configurations, the inputs and outputs of these applications, the test artifacts that are used, and build those artifacts and configurations in ways that can be shared broadly within teams or across the organization. That’s important for a couple things – one is the work to do that, you wanna do it once and then do it once for every one ’cause, in the DevOps mindset of continuous improvement, you wanna get rid of things that aren’t value-added, things that customers won’t pay for.

    So, in creating this capability, we allow now, as teams get spun up, right, you get a team that’s gonna work on a two-week sprint, you want them, out of the gate, to be productive. So being able to share profiles day one, with people that may have never worked on these systems before, is crucial so that you’re productive in the first minute of the first day.

    You also have situations where you’ve got, instead of technical debt issues, maybe a team, a large team, that you wanna go after certain work. It’s gonna help to improve the future throughput of the incremental efforts you have, from a development perspective, in the future. And you’re gonna bring large groups of people to work on something and that may be an outsourcer that you’re engaging; it may be new college recruits that you’re engaging. But you, again, want those people to be as productive as possible, as quickly, so, if you’ve got an experienced person on your staff that can create all those configurations and then share that broadly, again, you’re gonna lessen the ramp-up time for them to do their effective work.

    So, again this is another example of us thinking about the mainframe going from waterfall to Agile, at scale, and having a scenario where it’s not just people that have been on your staff for 30 years, working on the same subsystems of your applications, every day for their entire career. Now, ’cause ideas are paramount and where people work has more to do not with what they know but what the company wants to do in better servicing their customers, so there’s more of a randomness to the work and a changing nature to the work, and what you work on is fluid. So we’re really thinking deeply about the implications of that and, for profiles within Topaz, that’s a major step in the direction of getting a different way of working within IT.

    The second thing that we’ve released is another Jenkins plug-in. Jenkins is a central part of CI/CD pipelines, in terms of the automation side of it, and we’re constantly thinking about “What are the things or plug-ins that need to be created to make the mainframe equal to any other platform in all ways, other than syntax?” So JCL, Job Control Language, is a very important part of how programs are run within the mainframe. Creating a plug-in to help bring it into the CI/CD pipeline efforts and the automations efforts was a crucial thing that needed to be done, so it’s a contribution, from an artifacts standpoint, that, again, takes another step for our customers.

    And then, lastly, is in the area of testing. Compuware is the leader, if not exclusive, provider of unit and functional testing as a combined offering. It’s a necessary part of the job that people have to do within Agile, with this concept of shifting left. The developers aren’t just coding; they’re coding and testing, both in terms of bugs and performance, and becoming much more extended in terms of their role. And so their accountability and responsibility is bigger.

    So they need tools, like unit and functional testing tools, to participate within a two-week sprint process so that their deliverable, at the end of that cycle, is one that’s close to bug-free as possible and not putting future work into the system where it comes back as some quality issue 6 months later, 12 months later, when it runs into production, or, you know, as usability concern. So these three things are another turn of the crank, if you will, and Compuware’s offered to help customers nail these jobs, these new jobs that are created within kind of Agile work environments and DevOps work environments.

    Shimel: Excellent. Anything else on the new release before I go off on another direction?

    O’Malley: No, it’s just we’re very excited ’cause we released a technology called “zAdviser,” six months or so ago, which is a method of collecting what business KPIs for customers and then looking at, actually, stats relative to our products, so we can see kind of the behavior of developers. And we’re working with companies collaboratively to treat their people as they should, like high-performance athletes rather than slaves in a Roman ship galley. You can say, “Work harder. Work faster.” Well, you’ve gotta be constructive in helping to share best practices and methods that have a positive effect in things like velocity, quality, and efficiency.

    So, with every release, we’re seeing a significant increase in this KPI performance, so, as we’re talking about earlier, the mainframe is certainly on a march to become a first-class citizen within the digital age, within these enterprise customers, but the more _____ –

    [crosstalk]

    Shimel: I’m sorry; go ahead, Chris. Go ahead.

    O’Malley: Go ahead.

    Shimel: Well, no, I was gonna say, you know what I find interesting is, yeah, we’re talking about the mainframe machines, but so much of your focus is on the people, on the developers, on the team, because, at the end of the day, that’s the gold. Right? That’s the almost irreplaceable asset, are the people and the developers and the IT teams that are working on these mainframes. And it’s about empowering them. Right? So I think that it’s the right place to put the focus.

    O’Malley: Yeah, people –

    Shimel: Yeah, I –

    O’Malley: Just to restate the obvious here, but people matter most. When you think about the mainframe in the context of Agile and DevOps, asking people to change their role, though they’ve been around for 30 years, 40 years, dramatically – I mean, the day in the life in Agile and DevOps is vastly different than the day in the life of status quo. And it’s important that, as leaders, you’re promoting the appropriate culture and getting people to understand why they need to change, important that you’re giving ’em the necessary education, teaching them how to be effective in Agile sprints. I mean, that’s a different way of working. I mean, it’s not that you’re working 24 hours a day, but you’re definitely working in a more intense way. And then products matter then. The tools have gotta be enablers, not things that fight against your efforts to adopt this new culture and Agile nature.

    And, you know, I really try to express to IT leaders, “You gotta think about these things ’cause – ” you know, we use this example, the Gallup survey. Gallup survey have been doing this thing for 20, 30 years of employee engagement in large enterprises, and what’s most interesting about it is, over 20, 30 years, the employee engagement statistics haven’t really changed. And it’s disturbing when you actually hear about them. Two-thirds of employees in most large enterprises are not engaged, half of them are kinda, “It’s a job; I don’t care that much,” and 16, 17 percent are actively disengaged – they try to sabotage the efforts of the companies.

    So one of the biggest problems is engagement and employee engagement, and so I try to get people to understand – people matter hugely. And you’ve gotta give ’em the culture necessary to thrive in this more competitive age, you’ve gotta give ’em direction in terms of the process, the way that work is done, but you gotta give ’em the tools. You know, asking ’em to use mainframe tools out of a NASA movie from 1965 and thinking that something’s gonna happen is incredibly naive, so –

    Shimel: Absolutely. You’re not landing people on the dark side of the moon with that. Chris, we’ve got a few minutes left. I wanted to touch on another subject – we didn’t really kind of pre-talk about it, but I’m sure you’ve got some great views on it – I was out in Seattle last month for KubeCon, you know, Kubernetes, cloud-native. It’s a funny question to ask the mainframe guy the cloud-native question, but, certainly, containers are no strangers to mainframes.

    O’Malley: I mean, they’ve been on the mainframe for –

    Shimel: 20 years, maybe more, right? 25. So anything specific to Kubernetes in the Topaz or Compuware kinda roadmap that you can discuss right now? Or what effect do you see on your world?

    O’Malley: Yeah, so, when you think about kind of the flow of work in large-enterprise IT and what developers are fixated on, I mean, part of it is innovation and functionality that they’re trying to build, which is most important. I mean, that’s the value-added efforts of developers that make a difference in the eyes of the customer. And you wanna do that as quickly as you can, with the greatest degree of quality.

    But the other things you gotta think about are things like  quality itself. I mean, if you get anything that you release that’s got bugs in the system, how rapidly you can respond to it. That’s always part of the work that’s being done and I don’t know of any organization that doesn’t have some degree of bugs. And bugs may be software written improperly – and, again, our tools are helping to fix that – or usability issues, integration into other systems.

    There’s also the work of compliance and security that you gotta do, but this last thing is this thing, technical debt, right? That any code base that’s had any life to it, has meandered through the efforts to serve customers, it’s something that grows over time. And it has nothing to do with mainframe; it happens everywhere. You know, the cloud – you know, Amazon’s got debt. And you gotta deal with that technical debt, so, as you’re dealing with technical debt, the smart approach is to start using more of these modern techniques. SOA was kind of a philosophical approach to breaking down these larger systems into smaller subsystems. When I think about API enabling, REST Y enabling  these kind of smaller chunks, but you also wanna think about containerizing, using these as a form of containers, Docker being the method of creating the container, Kubernetes of being the way of managing these.

    So we do see this playing an increasing role _____, as people start really looking at affecting technical debt ’cause that gives you the biggest bang for the buck. I mean, you certainly wanna do it with any new functionality you’re doing, but you’ve got some huge numbers of lines of code that you’ve developed over four years that works awesome. I mean, it’s really good working code, but making it more manageable and getting rid of that technical debt. Part of that equation is RESTful APIs; it’s containerizing it and managing it more in a cloud-like fashion. So, yeah, you’ll see us enabling those tools.

    Our approach has always been to go to the edge of the mainframe and then we stop. We take care of what we know best and getting rid of these esoteric differences, but then, when we go off the edge of the mainframe, we work with all these other vendors. We allow SonarLint and SonarQube to do its work on COBOL, as akin to what it does with Java or C. Parasoft is a testing framework, allowing our products to work within that orchestration.

    So there’s all these tools that work in kind of the DevOps world that are preferred and we’re enabling them, so you should think of us doing things in support of Docker and Kubernetes over the long term. But, yeah, it’s an important part of making mainframe IP one that the throughput can constantly be improved and those techniques obviously help enormously.

    Shimel: Absolutely. Chris, I’m sorry to get you going ’cause it doesn’t take much when we start talking mainframes and DevOps, but we’re way over time and I need to wrap up. Hey, we’ll have you on, certainly, next quarter for the – what will it be? – the 45th time?

    O’Malley: [Laughs] We’ll be doing this and it’ll be our 1,000th. You and I will be in wheelchairs, talking about it.

    Shimel: By then, you should be down in Florida with me, but, anyway, Chris, it’s a pleasure having you on. Continued success with Topaz and Compuware. And we’ll catch up with you soon?

    O’Malley: Always a pleasure and anytime I can help.

    Shimel: All right.

    O’Malley: And thank you for the hard work that you do.

    Shimel: Thank you.

    O’Malley: _____ important _____ –

    Shimel: All righty. Chris O’Malley, CEO, visionary, leader at Compuware, on DevOps Chat with me today. This is Alan Shimel and you’ve just listened to another chat.

    — Alan Shimel

  • Open Mainframe Project: Zowe Ready for Prime Time

    Open Mainframe Project: Zowe Ready for Prime Time

    There is a lot of interest in updating mainframe technology/interfaces across traditional enterprises. As development environments and toolsets have evolved outside the mainframe, there is a struggle to keep up—partially because backward compatibility requirements make wild changes difficult and partly because the very architecture of mainframes is different.

    There have been several fragmented efforts over the last few years—some of them quite good—to update how a mainframe is accessed from external source and the ways that developers can code for the mainframe. After all, the number of developers proficient in mainframe toolsets continues to decline while the number of workloads does not, so something has to change.

    The Open Mainframe Project’s Zowe attempts to address these issues in ways that are both more widely applicable and more technologically advanced. It takes advantage of Open Source REST systems to front-end hardcore mainframe functionality, so developers who are accustomed to application server development can call things such as Spring Boot-based interfaces and use languages including Node.Js to access their mainframe systems.

    This week, OMP announced that Zowe 1.0.0 is production-ready. That means serious experimentation can be started at your organization to see if it addresses some or all of your mainframe modernization needs.

    There will likely always be some (at least operational) differences in mainframe systems usage because the software infrastructure is designed a certain way, but increasing modern tool usage makes learning those different parts easier across the board. By implementing a vendor-independent architecture that has guidelines for users and vendors to add functionality from anywhere, Zowe is a powerful and expandable tool for getting the job done.

    I sound a bit like a fanboy, but for most verticals, this is an issue that will continue to be a pressure point. It’s been more than a decade since companies started bringing back retirees to work on mainframe projects because new developers did not have the skills or interest to complete those projects. As more and more skilled CICS/RPG/PLI/etc. developers and mainframe operations specialists retire, this pressure will increase until it is resolved or unbearable. So I am a fanboy of any company/solution that is addressing this space instead of flocking off to the coolest new DevOps tool du jour. I’ve not yet used Zowe, so can’t directly recommend it, but reading over their documentation, I believe it’s a good idea to use tools we’re familiar with. So I’m rooting for it.

    If you test it out, drop a comment and let the rest of us know what you think.

    And keep rocking it. Whether in VS-2019 or VT-70, you are making the company engines hum.

    — Don Macvittie

  • Why 2019 Will Be the Year for Shift-Left Mainframe Testing

    Why 2019 Will Be the Year for Shift-Left Mainframe Testing

    While 2018 was the year for planning and implementing shift-left methodologies in testing, mainframe and server testers were, for the most part, left behind. These legacy infrastructure experts were tied down to old-school testing tools. Mainframes kept functioning, but tools and testing practices often became bottlenecks, preventing performance testing teams from testing more quickly mid-cycle and pre-release.

    Until now.

    The future of testing is in open source, and finally there are open source solutions being developed and tailored for mainframe testers. Now, mainframe testers can also shift left and become agile.

    Here are four reasons why mainframe developers and testers will benefit from shifting lift and embracing open source testing technologies:

    Frequent Releases

    Apache JMeter, BlazeMeter (open-source based), Taurus and Jenkins are ideal substitutes for legacy tools, for two reasons. These open source tools are very easy to use, so developers and testers can use them themselves. In addition, they can be automated easily—once scripts are set up, they just run, and developers can get the results and monitor the system and their code.

    These two reasons are the building blocks of what shifting left is all about: frequent releases. There is a much quicker option than developing, waiting for QA to run tests, arguing with QA about the results, fixing the code, waiting for test results again, realizing you forgot to commit an important part of the fix, sending the code again, waiting for tests and pulling an all-nighter to fix the code for the last time only to realize that your code clashes with a team member’s code and you can’t release.

    By integrating your code tests into the development pipeline, every piece of code is immediately and automatically tested, on its own and encapsulated with other parts of the code. Once the code is approved, a new version is immediately released, to the customer’s/user’s benefit. This also benefits the developers by giving them a clean slate to continue working on. If the code needs fixing, no problem—an alert is sent to the developer, who can easily identify the problem and fix it. The fixed code is immediately tested and released. Now that’s much easier and much more motivating for everyone involved.

    Fewer Bugs, Higher Code Quality

    Shifting left means more automated tests are run—and more frequently. By constantly checking the system for errors and by running tests every build, you and your team are constantly being alerted of errors, which you can then immediately fix. The result is higher quality code and much better system performance.

    Another positive outcome is less hassle for your team when developing, because you don’t have to keep dwelling on previous version errors. Those have already been fixed or dealt with, so you can focus on future plans and exciting developments, which will materialize in higher code quality.

    Cost Reduction

    Shifting left means faster and automated testing, resulting in a higher-quality product. By running more tests in parallel all the time, testers and developers can focus on developing new code, resulting in savings for the business. If you’re running tests in the cloud, you’re also saving infrastructure costs.

    A higher velocity of releases with quicker fixes and new features results in more satisfied customers, who stay with you instead of moving to your competitors. Customers stay with companies they feel is concentrating on the future, which is why Gartner’s Magic Quadrant “Visionary” recognition is considered to be a big deal for the testing industry.

    Better Team Collaboration

    The idea of automation and continuous testing might come across as being detrimental to teamwork, but the opposite is true. Even though developers become more self-sufficient with their own code, this complete ownership actually encourages teamwork. Team members will collaborate more on joint feature development through shared workspaces and dashboards, with each developer bringing the code they produced to integrate with others. They also will work together to discover the system’s shortcomings and learn how they can fix bugs and bottlenecks.

    Shift-left and automation for mainframe means more time for feature development, getting customer feedback and responding to it; quicker fixes; and faster releases. This is crucial for competing in the market, providing the best services, attracting the highest-quality workforce and maintaining product security.

    Start Your Mainframe Testing with Open Source Tools

    BlazeMeter recently contributed the JMeter RTE plugin to the open source community. This plugin enables load and functional testing on JMeter for IBM 3270- and 5250-based applications.

    Running mainframe tests enables mainframe developers and testers to finally introduce agility to their testing habits, bringing mainframe testers in line with their coworkers who already embraced shift left and open source testing toolkits.

    — Refael Botbol Weiss

  • Best of 2018: The Beauty of the COBOL Programming Language

    Best of 2018: The Beauty of the COBOL Programming Language

    As we close out 2018, we at devopsy.kinsta.cloud wanted to highlight the five most popular articles of the year. Following is the first in our weeklong series of the Best of 2018.

    Well-written code is a work of art. Always has been, always will be. A programmer pulls a thought pretty much out of nowhere and transforms it into a working idea that can be used by others. It’s abstract expression made real. Computer programming requires a depth of creativity and discipline of logic that is hard to find elsewhere. Maybe architecture and the theoretical sciences come close, but computer programming stands apart. Computer programming is special and I love it!

     

    Thus, I am always interested to learn a new programming language. Every programming language offers a new way to expand my thinking. It’s always time well-spent.

    Recently I decided to learn COBOL, for no other reason than the fact that there are a lot of mainframe installations out there and it’s the mainstay language for many of them. Mainframes are critical to the operations of many banks, insurance companies, transportation systems and governmental agencies. Learning COBOL been on my Bucket List for while.  So I took the plunge.

     

    What does the term, COBOL stand for? Common Business-Oriented Language

    Working with COBOL in a Modern IDE

    The first thing I needed in my journey to learn COBOL was an IDE. I am a big supporter of coding in an integrated development environment (IDE). I like being able to write, test and run code all in one place. Also, I find the support features that an IDE provides, such as visual code structure analysis, code completion and inline syntax checking, allow me to program and debug efficiently.

    The IDE I found is an open source product, OpenCobolIDE, as shown below in Figure 1.

    Figure 1: OpenCobolIDE provides many tools to make COBOL programming easier

    OpenCobolIDE allows me to write, compile and run code all in one place, without having to go out to the command line, which is very good because I am the world’s worst typist!

    Programming COBOL in the Real World

    When it comes time to do real world, mainframe based COBOL programming, you will do well to look at the IDEs that IBM provides. The tools are designed to go about against its Z Systems environment seamlessly as a set of Eclipse plugins. These tools allow COBOL developers working in the mainframe environment to code, debug, unit test and do problem determination.

    Having set up my development environment, the next thing to do was design my first project.

    Designing My First COBOL Program

    Typically when learning a new language I like to start small and work my way up into more complex tasks. I did the typical Hello World program. You write the code to display string, Hello World, to standard output. Every developer does one. It’s a trivial program, as shown below in Listing 1

    beauty-of-cobol-listing-01
    Listing 1: A Hello World program written COBOL

     

    While the Hello World required that I learn about the concept of a DIVISION, which I will discuss in detail below, in the scheme of things, I needed more. I wanted to write a program that would force me to learn how the following:

    • How to create and use variables.
    • How to structure data into a hierarchy.
    • How to structure code into encapsulated procedures.
    • How to do some basic arithmetic.
    • How to accept user input and then do some manipulation around that input.

    Thus, I create the program, RESEL-WORLD, as shown in Listing 2, below.

    beauty-of-COBOL-Listing-02

    Listing 2: The COBOL program, RESEL-WORLD accepts and manipulates users’ input

    The user input and program output is shown below in Figure 2.

     

    Figure 2: The RESEL-WORLD COBOL program asks a user for first name, last name and age, then manipulates the input

    The RESEL-WORLD program taught me a lot about COBOL and how to program in the language. In the spirit of giving, I am going to share what I learned from writing the program. However, before I delve into the details, please be advised that the information I’m presenting is but a high-level overview. As with any programming language, it takes about a year of consistent coding to become an entry-level professional. My sincere hope is that the information I provide gives the reader a good sense of the language and enough motivation to want to learn more. As I mentioned early, learning COBOL is worth the effort.

    COBOL is Code in a Structured Document

    The most important thing to understand when learning COBOL is that is very strict in terms of code layout. The layout rules relate to the use of columns and characters. Also, the format uses a hierarchical outline structure. The following sections describe the details of layout specification.

    The COBOL Column Specification

    In COBOL a line of code can be no longer than 80 columns in length. You can also think of a column as a character. Columns are segmented into groups, with each group serving a particular purpose. The columns groups are as follows:

    Columns 1-6 is the group in which a programmer defines a sequence number. A sequence number is similar to a line number.

    Column 7 is reserved for special characters. Asterisk (*) starts a line of comment. Hyphen (-) indicates line continuation and slash (/) is form feed.

    Columns 8-11 is called Area A. Area A is the group of columns in which you start a DIVISION, PARAGRAPH and SECTION. We’ll talk more about these below in the next section, The COBOL Structural Hierarchy.

    Columns 12-72, also known as Area B, is where you write code statements.

    Columns 73-80 is reserved for developer use. You can write poetry in there, if you so desire. Just make sure you don’t go past column 80.

     

    Figure 3, below illustrates the columns grouping described above.

    Figure 3: Each column in a COBOL file is specified to serve a specific purpose

     

    COBOL is a compiled language. When it’s time to run your code, the compiler will check to make sure that the code layout adheres to the column grouping specification. If there is a violation, the compiler will error.

    Understanding Performance in Terms of Compilation and Compilers

    COBOL is a compiled language, as are others such as Java, C# and C++. Compilation is the process of taking textual source code and converting into a binary format that the computer can understand.

    There’s a lot of variety in the COBOL world when it comes to compilers, particularly when it comes to cost and performance efficiency. For example, how a compiler orchestrates the way numbers are loaded and then computed in memory matters a lot! A good compiler will make code run really fast.

    IBM has been in the COBOL business for a long time and has a keen understanding about making compilers that are fast and cost-effective. For example, the z14 System compilers improve efficiency by taking advantage of almost 24 new low-level instructions. As a result, computation speeds improve dramatically.

    As you work more with COBOL, you’ll come to appreciate the power that an industry-leading compiler such the IBM Z Systems brings to the coding experience.

    The COBOL Structural Hierarchy

    The concept behind the file format of a COBOL program is based on the structure of a document outline, with a single top-level heading followed by subordinate levels. The organizational units that make up the hierarchy are PROGRAM, DIVISION, SECTION, PARAGRAPH, SENTENCE, STATEMENT and CHARACTER. Figure 4 illustrates the hierarchy.

     

    Figure 4: The structural hierarchy of COBOL program entities

    PROGRAM is the root level of the COBOL code hierarchy. PROGRAM represents the unit of code that the mainframe job scheduler, the JCL, loads into memory to run. The program is identified by the PROGRAM ID statement in the IDENTIFICATION DIVISION. The IDENTIFICATION DIVISION is part of the next level of hierarchy that descends from PROGRAM. PROGRAM must contain the IDENTIFICATION DIVISION.

    There are other DIVISIONs that can be included. These other DIVISIONs are: ENVIRONMENT DIVISION, DATA DIVISION and PROCEDURE DIVISION,  which occur after IDENTIFICATION. Also, these subsequent DIVISIONs must appear in the order defined: ENVIRONMENT, DATA and then PROCEDURE. You can read a detailed description of each DIVISION here. DIVISION names are terminated with a period, for example,

    DATA DIVISION.

    The next organizational level down from DIVISION is SECTION. (Please see Figure 5.) Each DIVISION will contain SECTIONs that are special to it. For example, the DATA DIVISION can contain FILE SECTION, a, WORKING-STORAGE SECTION and/or the LINKAGE SECTION. You can read the details about DIVISIONs and SECTIONs here.


    Figure 5: A COBOL program is structured in a hierarchical manner

    A SECTION name is terminated with a period, for example:

    WORKING-STORAGE SECTION.

    A SECTION contains zero or many PARAGRAPHs. (Typically a SECTION will have at least one PARAGRAPH.) A PARAGRAPH name is terminated with a period, similar to a DIVISION and a SECTION.

    A PARAGRAPH contains SENTENCEs or STATEMENTs. A SENTENCE is a group of STATEMENTs. Usually a PARAGRAPH contains one or more STATEMENTs. A STATEMENT is a line of execution. A STATEMENT is made up of CHARACTERs. A CHARACTER can be an alphanumeric symbol or a special character. CHARACTER is at the bottom of the COBOL code format hierarchy.

    Variable and Data Types

    COBOL allows you to declare variables in a variety of data types. Special to COBOL is the concept of a variable level as represented by a level number. A level number defines a variable in terms of being or having a parent variable. (You’ll see more about this in the section, COBOL Supports Hierarchical Data, later on.)

    The following statement, which will be declared in the WORKING-STORAGE SECTION of the DATA DIVISION, is the declaration of a variable, WS-QUANTITY. WS-QUANTITY will hold a numeric value.

    01  WS-QUANTITY PIC 9(2) VALUE 12

    The expression above above declares a variable , WS-QUANTITY to be a two-digit value with an initial value of 12. Also, the variable is declared to be at level 01, which is the highest level possible. The variable has no parent. The way we know that WS-QUANTITY is a variable for a two-digit value is due to the PIC (Picture) clause. You can think of a PIC clause as a type declaration. Table 1 provides a high-level description of the various PIC clause symbols.

    Symbol Description Example Declaration Sample Value
    9 A numeric value where each occurance of 9 represents a digit 99 or 9(2) 35
    a Alphabetic aaa or a(3) “Bob”
    x Alphanumeric xxxx or x(4) “R2D2?”
    v Implicit decimal* v(3) .175
    s Sign s9(2) -76
    p Assumed decimal* p9 .6

    Table 1: Symbols for a PIC Clause

    *COBOL has a special way of storing numbers such that the numeric conversion take place when the characters are loaded into memory. For example, a variable with a PIC of p9(2) will be stored as 56 and loaded into memory as .56. For a more detailed discussion of usage and the PIC clause, go here.

    COBOL Supports Hierarchical Data

    Storing data in a named group of variables makes programing easier. For example, in the C programming language, we can use a struct to name a group of variables, as shown below in Listing 3.

    struct User { 
      char  first_name[10]; 
      char  last_name[10] 
      int   user_id; 
    };

    Listing 3: A simple structure in C

     

    One of the real nice things about COBOL is that it allows you to organize data according to a named group of values similar to that of a C struct. What’s interesting in a historical sense is that COBOL has had this capability well before Kernighan and Ritchie created the C programming language.

    The way COBOL accomplishes named data grouping is with construct called a record. Figure 6 below shows a graphical representation of a record named, WS-USER.

    Figure 6: The record, WS-USER contains subordinate variable, WS-FIRST-NAME, WS-LAST-NAME, WS-AGE

    The code snippet in Listing 4, which is from the RESEL-WORLD program at the start of this article, shows the COBOL code that declares the record, WS-USER.

     

    beauty-of-COBOL-Listing-04

    Listing 4: The record, WS-USER contains 3 subordinate elements, WS-FIRST-NAME, WS-LAST-NAME, WS-AGE

    Notice in Listing 4 above the there are two levels of data variables in play: 01 WS-USER and 05  WS-FIRST-NAME, 05 WS-LAST-NAME, 05 WS-AGE. The variable, WS-USER is a parent to the subordinate, level 05 variables, WS-FIRST-NAME, WS-LAST-NAME, and WS-AGE.

    Assigning variable to a numeric level is special to COBOL. The declaration logic is that a record variable (the root of the record) has a level number of 01.  Levels 02 to 49 are used to declare subordinates elements. (You can think of a element as a member of the record.)

    You use the reserved word OF to access an element of a record. Listing 5 below shows two statements. The first asks a user to input their first name. The second statement takes the value entered and assigns it to the element, WS-FIRST-NAME of the record, WS-USER.

    DISPLAY "What is your first name?".
    ACCEPT WS-FIRST-NAME OF WS-USER.

    Listing 5: Accessing an element of a record in COBOL

     

    It make sense that records were part of COBOL from the start. COBOL was intended to be a language used for business application. Businesses have been organizing values according to named groups since before the customer form was created. COBOL was designed to reflect business needs. It’s interesting that the fundamental needs of businesses in terms of logical data structures have been surprisingly consistent over time.

    COBOL Has a Natural Language Syntax

    Expressing a statement in COBOL is very similar to the way a person speaks naturally. Listing 6 below shows the statement that adds together the values of the two variables, WS-AGE-DELTA and WS-AGE (from the record WS-USER), and stores the result in the variable, WS-NEW-AGE.

    
    ADD  WS-AGE-DELTA WS-AGE OF WS-USER TO WS-NEW-AGE.

    Listing 6: Adding two integers, WS-AGE-and DELTA WS-AGE, then storing the sum in the variable, WS-NEW-AGE

    Notice that the statement in Listing 6 is very close to saying, “Add WS-AGE-DELTA and WS-AGE from WS-USER to WS-NEW-AGE.”

     

    What I find pretty astounding is that many modern programming paradigms try to capture the ease of natural language expression that COBOL has had for years. What comes to mind immediately is the use of natural language expressions in the NodeJS Chai package. Chai is used in NodeJS unit testing to express BDD assertions. Here’s a Chai expression that checks a variable, myVar to assert that it’s a string.

    expect(myVar).to.be.a('string');
    
    FAST FACT: Is COBOL a case-sensitive language?

    No. COBOL will consider a variable named WS-FIRST-NAME to be the same as one named ws-first-name.

    COBOL supports DRY

    For programming language to be really useful, it needs to support DRY. DRY is an acronym for Don’t Repeat Yourself. Most programmers don’t want to write the same code over and over again, nor should they. What you want to do is to encapsulate code in to a single area of execution that can be called repeatedly. Being able to program by DRY is common for most languages—BASIC, Java, C#, Javascript, Python, to name a few and … you guessed it: COBOL.

    COBOL supports code segmentation and reuse in a variety of ways. One way is through linkage, calling routines between programs. Another way is to segment code inline under a PARAGRAPH in the PROCEDURE DIVISION and then call that PARAGRAPH. (See Figure 7.)

    Figure 7: You can make code reusable in COBOL PARAGRAPHs

    The little RESEL-WORLD program I wrote uses the inline PARAGRAPH technique. Before I go in to the details of calling code encapsulated into a PARAGRAPH, it’s important that you know that in COBOL the entry point of execution into a program is the first line of code after the declaration of the PROCEDURE DIVISION, as shown below in Listing 7.

    beauty-of-COBOL-Listing-07
    Listing 7: COBOL supports reusable code by using PARAGRAPHS to segment execution

    Notice the PERFORM statements right after PROCEDURE DIVISION. These four statements are calling PARAGRAPHs. (For those of you familiar with JavaScript, you can think of each PARAGRAPH as a function definition.) Essentially what the code is saying is, “Execute the code at the paragraph named, GET-DATA, then perform the code at CALC-DATA, SHOW-DATA and then FINISH-UP.”

    Encapsulating code into STATEMENTs within a PARAGRAPH prevents code from become spaghetti that is hard to maintain. Also, the encapsulation enforces the sensibility of DRY. Any language worth its salt needs to support some sort of encapsulation. COBOL meets the need with room to spare.

    An Expressive Language for Now and the Future

    The more I learn about COBOL, the more I like it. The language continues to evolve to meet the needs of our fast-changing times, with revisions as recent as 2014. Since its inception there have been a dozen enhancements to COBOL including a continuing stream of formal standards.

    Today’s COBOL supports modern programming paradigms such as object orientation. The IDEs have grown to keep pace with the demands of modern users. And, given the immense installation base out there, there is a lot of money to be made doing COBOL programming. To quote Wikipedia:

    In 2006 and 2012, Computerworld surveys found that over 60% of organizations used COBOL (more than C++ and Visual Basic .NET) and that for half of those, COBOL was used for the majority of their internal software.

     

    What’s even more amazing to me is the cleverness of engineering the language has promoted. COBOL developers addressed problems in the past that still vex many today. We modern developers can fall prey to thinking that before we came along, there was no cool stuff. It’s like the young, aspiring guitarist who thinks thinks he has his chops down and can shred to the top of the heap. Then one day his music teacher gives him a 1936 recording of Django Reinhardt playing guitar, and playing with only two functional fingers on his left hand! At that point the young artist realizes that virtuosity transcends era and that creativity has no bounds. This is what it’s like for me as I learn COBOL. It’s a beautiful, expressive language that was cool then and is very cool now. Learning it is making me appreciate how much amazing thinking went on back then and continues to emerge. There’s a lot of rich opportunity at hand to make great code for mainframes using COBOL. It’s only a matter of mastery, creativity and discovery.

    Special Thanks

    If I have seen further it is by standing on the shoulders of giants.

    —Isaac Newton, Letter to Robert Hooke, February 5, 1675

    Learning any programming language is hard work. Years ago, when I wanted to learn a new language, I would get a few books on the topic and then hunker down to spend the hours necessary to absorb the required knowledge. If I had the time, I might take a course. Maybe I had an expert friend I could call up when I hit a wall or needed real-world guidance.

    We’ve come a long way since that time. The internet is a game-changer. Online tutorials, videos and interactive, digital books make things a whole lot easier than in earlier times. My experience learning COBOL testifies to the value of these modern benefits. But still, as I’ve been learning, I’ve hit more than one wall that required professional guidance and review. Allan Kielstra and Roland Koo at IBM (who also wrote this great article) provided the expertise that made my learning COBOL not only less difficult, but also fun. I am in their debt. Their enthusiasm for COBOL was infectious. Their commitment to technical excellence is inspiring.

    — Bob Reselman

  • Let’s Bury Bimodal Thinking in Enterprise IT

    Let’s Bury Bimodal Thinking in Enterprise IT

    We still hear a lot about mode-1 and mode-2 in enterprise IT. Like many other large enterprises, you might have adopted the terminology introduced by Gartner and use it every day to shape the way you deliver and operate technology. However, I have always been uncomfortable with the idea of bimodal, as it ignores architectural reality of tightly coupled systems that we have today and the cultural factors. There is also the cultural dimension that is emerging a lot in the transformations that I help clients with.

    The cultural problem with Gartner’s bimodal IT view is that it relegates legacy systems such as mainframe (Z series) or I-series to mode 1 and discourages the new generation of professionals from working on these platforms. If you are in mode 2, you get access to the latest tools and training opportunities, and if you are not, then you are relegated to the end of the queue!

    What do you think? Is bimodal creating two classes of people? Or am I being too harsh in my judgment?

    In a recent article by BMW CIO Klaus Straub, he went on record and said bimodal IT doesn’t work due to cultural issues. As he said in the article, “Until September of last year, I also had a bimodal world in my head—but then the concept was incomprehensible to me. However, it is not suitable for permanently structuring an IT department.”

    BMW is not alone. Other enterprises are joining the voices, such as Peter Jacobs, the CIO of ING Bank Netherlands, who also has expressed similar views on bi-modal. To quote Jacobs: “I would rather work agile at my core bank system than at the channels.”

    The full interview of Straub is below. Unfortunately, the article is in Dutch, so you will need to translate the article to read it: https://www.cio.de/a/amp/bmw-cio-haelt-bimodal-it-fuer-einen-irrweg,3562374

    A few other relevant and interesting articles are below:

    1. https://www.cio.com/article/3193793/cio-role/in-the-digital-era-cios-not-buying-this-bimodal-crap.html
    2. https://www.cio.com/article/3191810/it-industry/why-bi-modal-it-doesnt-really-work.html

     

    — Aditya Vadaganadam

  • Where is the Next Generation of Mainframe Talent?

    Where is the Next Generation of Mainframe Talent?

    Are we potentially facing a shortage of mainframe talent?

    “Three of us set up this tool in 1989. One is retired, and the other person is somewhere in heaven!”

    That was the remark of Alf (name changed), the senior mainframe architect at a large European enterprise, as he pointed toward the ceiling of the meeting room. Everyone else in the room let out a half-nervous smile, unsure how to react to the statement and hoping for some visual cues from Alf. We were all talking about a specific version control and packaging tool on the mainframes, which was heavily customized to suit this particular enterprise. The attendees were a mix of DevOps enthusiasts and mainframe architects and engineers with the purpose of re-evaluating the existing tooling stack on mainframes with a view to modernize them as part of a wider agile/DevOps initiative. Alf’s remark in the meeting has stayed with me ever since.

    Later that week, I was having lunch with a group of new joiners in the technology department, who were all part of a greenhouse initiative to in-source digital skills. During the conversation, I asked which areas where they focused on and the career direction they had in mind. The answers were predictable: data science, python, Nodejs, UX, blockchain, JavaScript etc. If all the new kids on the block are learning the newer technologies, then who will take over from mainframe people over the next few years when they retire?

    The reality is that 96 of the world’s top 100 banks, 23 of the 25 top U.S. retailers and 9 out of 10 of the world’s largest insurance companies run IBM’s System z mainframes. Most digital transformations have barely scratched the surface of the legacy stack.

    Like many other enterprises, this enterprise had outsourced most of its application estate to IT service providers. In few years’ time, Alf will move to a well-deserved, retirement phase of his life and take his mammoth knowledge base that the enterprise  had always taken for granted. There are many more Alfs across various enterprises today with significant mainframe know-how who are set to disappear from workplace over the next few years, making these enterprises entirely dependent on external service providers—so much for those bold visionary statements for the CXOs such as, “Technology is business and business is technology.”

    Most enterprises today have ambitious digital transformation agendas, investing millions on various initiatives ranging from digital, DevOps, agile, UX and blockchain. This will manage to “DevOpsify” the enterprise skin (aka front end, customer experience layer, or mode 2 apps) and, in many cases, possibly, their non-legacy middle-end system as well. However, the majority of the back-end stack, especially in financial services, remains in the legacy world (zSeries  or iSeries).

    One might argue with a seemingly coherent logic that, “Well, we won’t have mainframes in a few years as we will move to newer platforms soon.” But it helps to remember Mark Twain’s quip, “The rumors of my demise are greatly exaggerated,” applies so well to the mainframe world!

    In simple terms, your enterprise most likely will adopt a combination of the following strategies to deal with the legacy challenge:

    • Retire mainframe app, in some cases.
    • Extract business logic and move it to distributed platforms.
    • Componentize the monolithic mainframe code and hook them to APIs for external consumption.
    • Build new platforms to replace legacy apps.

    There is one least common denominator for any of the above strategic direction that your enterprises take: ever-diminishing mainframe knowledge! So, I ask again: Where is the next generation of mainframe talent?

    — Aditya Vadaganadam

  • BMC Modernizes the Mainframe with Automated Mainframe Intelligence

    BMC AMI places machine learning, predictive analytics, domain expertise, and intelligent automation at the core to help enterprises achieve self-managing mainframes

    HOUSTON – October 18, 2018 – BMC, a global leader in IT solutions for the digital enterprise, today announced BMC AMI, automated mainframe intelligence solutions that will deliver higher performing, self-managing mainframe environments to meet the growing demands created by digital business growth. Extending BMC’s leadership in mainframe innovation, BMC AMI solutions combine built-in domain expertise, machine learning, intelligent automation, and predictive analytics to help enterprises automatically manage, diagnose, heal, secure, and optimize mainframe processes.

    “Enterprise IT teams are experiencing complexity at an accelerating rate, and data and transaction volumes are exploding – all while trying to manage a changing workforce, close the skills gap, and optimize costs,” said Bill Miller, President of ZSolutions at BMC. “With AMI, we are freeing up IT staff to work on high-value initiatives by removing manual processes through intelligent automation. BMC AMI will automatically analyze dozens of KPIs and millions of metrics a day, and proactively identify, predict, and fix problems before they become an issue.”

    According to Gartner, by 2020, thirty percent of data centers that fail to apply artificial intelligence and machine learning effectively in support of enterprise business will cease to be operationally and economically viable.1 The BMC AMI solutions enhance mainframe systems through intelligent automation by predicting outages before they occur, solving performance degradations, and mitigating cost increases.

    With BMC AMI, customers benefit from BMC’s expertise in collecting deep and broad z/OS® operational metrics from a variety of industry data sources, built-in world-class domain expertise, and multivariate analysis. The new offerings include:

    • BMC AMI Autonomous Solutions
    • enable IT operations to automatically anticipate and repair performance degradations and disruptive outages before they occur, without manual intervention. This
    • set of intelligent, integrated solutions extends to BMC AMI for Security Management, BMC AMI for DevOps, BMC AMI for Performance and Availability Management, and BMC AMI Cost and Capacity Management.
    • BMC AMI Enterprise Connectors
    • connect business-critical data from the mainframe to the entire enterprise and simplify the enterprise-wide management of business applications. These offer a truly
    • holistic view of enterprise data by streaming
    • mainframe metrics and related information in real-time to a variety of data receivers, including leading Security Information and Event Management (SIEM) solutions
    • such as Splunk, IBM® QRadar®, ArcSight, LogRhythm, McAfee Enterprise Security Manager, and others. The BMC AMI Data Extractor
    • for IMS solution is available now, and additional solutions will
    • be available early in 2019.

    “BMC has been instrumental in helping us grow our business, deliver innovative solutions, and increase customer satisfaction, all while ensuring costs are kept at optimal levels,” said Gregg Kimbrough, Senior Manager Hosting Services, Thrivent Financial. “We look forward to seeing how AMI will help us move closer to achieving a self-managing mainframe so that we can continue to keep up with the demands of today’s digital business.”

    Earlier this month, BMC announced the signing of a definitive agreement to acquire assets of CorreLog, Inc., which provides real-time security management to mainframe customers.  The announcement furthers BMC’s commitment to the security space, expanding the data connectors that are available to clients and accelerating BMC AMI for security offerings.

    To learn more about BMC AMI solutions, visit www.bmc.com/ami

    To register for a webinar on the self-managing mainframe using BMC AMI on October 24, visit https://event.webcasts.com/starthere.jsp?ei=1209255&tp_key=ec7b8500cb

    To learn more about BMC’s mainframe solutions, visit http://www.bmc.com/it-solutions/mainframe.html

    1 Gartner Report: The IT Implications of the 2018 CIO Survey for I&O Leaders by Dave Russell, Hank Marquis Published : 8 March 2018

  • Dependency Based Build Promotes the Freedom to Program

    Dependency Based Build Promotes the Freedom to Program

    Whenever I want to be amazed, all I need to do is reflect on the fact that the continuing development of the Linux operating system is done essentially by a community of volunteers. That’s right! One of the most popular and important operating systems is updated and maintained through donated labor. Any programmer anywhere in the world can make a contribution to improve the Linux kernel.

    That such a herculean undertaking is done though volunteer labor is mind-boggling. What’s more amazing is that the software created has an extraordinarily high level of quality. Let’s face it, working on the Linux kernel is not something a kid in high school with a year’s worth of experience programming in C can do. But, any kid in high school can make a contribution to the Linux code base, provided the code passes muster. That it happens is almost magical, in a way. But it does, every day.

    What enables such freedom? To my thinking, it’s the concept of the the pull request.

    The Powerful Elegance of the Pull Request

    The pull request is a simple yet important part of the open source development process. A developer creates a feature branch off the Master branch of the Linux repository, then adds to or modifies the code in that feature branch. Of course, the developer writes tests that verify the new code works. Once the code is complete, the developer sends a pull request to the project, which is typically hosted on GitHub or another popular Git repository platform. The project maintainers, a group of people who have expert knowledge of the code base, review the pull request using tools that are baked into the Git repository platform. Each maintainer inspects the code and, if the code passes inspection, the reviewer votes to approve. If the code has shortcomings, the reviewer makes suggestions for improvement directly in the code using the Pull Request Commenting features, which are available in practically all the Git Repository Platforms including GitHub, GitLab and Bitbucket.

    It’s a simple yet powerful process and one that has changed the way enterprise-worthy software gets created. Any developer can implement an idea any way they want, as long as the implementation is compatible with the code base. It’s up to the developer to figure out how to do what they desire; nobody is dictating anything. The process is straightforward: Figure out how to make your contribution to the code. Make a feature branch. Do your work. Do a pull request. If your contribution is useful and the coded well, you’re in. If not, we’ll try to help you make it better.

    Bringing Git to Mainframe Development Using DBB

    The pull request is a compelling approach to making software—so much so that IBM has made it a central part of software development on the mainframe. And, its product, Dependency Based Build (DBB), streamlines the mainframe pull request process by reducing the work reviewers need to do in the code review process.  [su_pullquote align=”right”]A Dependency Based Build trial is available in the IBM Z Trial Program at zero cost, and with no installation required. The trial environment includes hands-on tutorials.[/su_pullquote]

    Git version control is supported by practically all the modern IDEs. For example, developers can configure Visual Studio Code, JetBrains products and Eclipse to work with remote Git repositories automatically. Once integrated, Git operates behind the scenes, allowing developers to work under version control without having to issue Git commands directly. IBM’s mainframe integrated development environment, IDz EE, provides Git integration as well. This is a big deal, as Git is now part of the mainframe ecosystem. The developer accustomed to using Git version control when working on a standalone x86 Java app now can move over to developing for a mainframe without having to learn a new source control management (SCM) system. But what’s even more impressive that Git is used as the SCM for more traditional mainframe artifacts; for example, COBOL source code.

    Also, Git integration goes beyond developers working on source code; Git is used throughout the mainframe build process—which, by the way, can be controlled by Jenkins. And, Git is a foundational technology used by IBM DBB, a dependency control agent that works with Git to make sure that all files—source and dependencies—in the mainframe build process are current. Again, this is a big deal: Having all the moving parts in the mainframe deployment process, from programming to build to release, use Git as the common code repository allows z/OS mainframe components to be part of a platform-agnostic deployment process. Use of DBB overcomes a significant hurdle between the mainframe and distributed world.

    So, what does this have to do with the power of pull requests?

    Adding Pull Requests and Code Review to the Mainframe Experience

    The pull request review process is more than just “looking at code.” When you have a project that might have millions of lines of code, such as the Linux kernel, no developer is going to pore over each line of code. They’ll read the documentation, run the unit tests, look at the lines of code that have been added or removed and search for blatant problems and inefficiencies. Now, on a PC, compiling something such as the Linux kernel is straightforward—all the code is in the single repository. On mainframes, however, it’s a bit different: The code is more widely dispersed and dependency management can get tricky. DBB takes care of tracking dependency changes in various Git repositories, so when it comes time to do the pull request, the reviewer doesn’t have to waste time resolving dependency issues to get mainframe code to compile. DBB will take care of that. With regard to the pull request, reviewers can do what they are supposed to do: make sure the code works and the quality is such that it can all be merged back into a master branch.

    This might not seem like much, but anybody who has stayed up until 3 a.m. trying to resolve dependency issues understands the importance of having code that builds all the time, any time. This is the power that DBB brings to the table.

    The Freedom to Program

    Programming is a special vocation. It requires an unusual combination of art, science, creativity and engineering discipline. Also, it’s not a vocation that lends itself to well to prescription. Each programming project is special, requiring new ideas. And new ideas come about when developers are free to use their imagination to think differently. The pull request is one of the best ways to ensure developers have the freedom to program in way that produces useful, high-quality code. DBB extends the freedom to program to the mainframe community.

    History has shown that when a developer is given the freedom to code and a community in which to code, the result is great software. Just ask Linus Torvalds.

    — Bob Reselman