devopsy.kinsta.cloud

Tag: legacy apps

Moving Beyond Legacy API Management Tools and Strategies

What is legacy API management? That’s a question that many organizations haven’t even thought to ask because they don’t realize that the way they manage their APIs is outdated. On the contrary, they assume that if they have API gateways in place to provide security and load balancing for their APIs, they’re already ahead of the curve when it comes to managing and protecting their APIs.

But the fact is that traditional approaches to API management are fast becoming outdated. In a world where the ability to communicate in real-time is more critical than ever, legacy API management solutions can’t keep up, because they’re not designed for APIs that need to share large volumes of data using asynchronous paradigms.

That’s why forward-thinking organizations should be looking for ways to overhaul their API management tools and strategies. They need to embrace next-generation API management solutions, which can do much more than manage traditional, synchronous APIs.

Allow me to explain by walking through the shortcomings of traditional API management tools and explaining what next-generation API management entails.

Legacy API Management

Legacy API management solutions are tools designed to manage and secure synchronous APIs. Typically, these tools come in the form of gateways that sit between backend REST APIs and applications that connect to those APIs using the HTTP protocol. The gateways manage traffic flows and secure communications between the applications and the API backend.

That approach works well if two conditions are true:

1. All of your APIs are synchronous, which doesn’t always work well with continuous data streams.
2. All of your clients can consume data from APIs using HTTP requests.

Traditionally, these conditions were true for most API use cases at most organizations. That’s why the standard approach to API management is to deploy a gateway, configure some load balancing and security policies and call it a day.

Next-Generation API Management

Increasingly, however, not all data is shared synchronously, and not all applications can consume data using HTTP requests. Instead, businesses want to be able to stream data in real-time to multiple types of applications. They might use an open source event broker like Kafka to stream monitoring data from across their IT assets to the tools that IT engineers use to observe and analyze systems, for example. Or, they might need to stream sales transaction data to a fraud detection engine so it can catch fraudulent transactions in real-time.

For use cases like these, legacy API management doesn’t work well at all. Traditional API gateways aren’t very compatible with event brokers because the gateways are not designed to manage real-time data streams. Nor do they typically support protocols other than HTTP, which is a problem because event brokers like Kafka use their own protocols, not HTTP, to enable communication between servers and clients.

Next-generation API management solutions fill this gap. They operate as gateways that can not only mediate between traditional REST-based client and API endpoints, but also support event brokers, asynchronous APIs, and even apply policies at the message level to support message-based communication.

In addition, next-gen API management tools offer the ability to translate between different front and backend protocols. That means that you could have, for example, a Kafka stream on the backend that is exposed to clients using HTTP – effectively allowing apps that can only make requests via HTTP to connect to Kafka data streams, without having to build support for protocol translation into the applications themselves.

Thinking Beyond Internal APIs

At this point, you may be thinking, “We don’t expose data streams externally, so why would we need a next-gen API management solution to support them?”

That’s a fair point. The most basic use cases for data streaming platforms like Kafka involve streaming data internally – from one set of applications to another within the same IT environment, for example.

But this is certainly not always the case. There are plenty of circumstances where you might want to expose a data stream externally and require the security and reliability protections that next-gen API management tools deliver. Maybe you’re a manufacturer who wants to stream data to regulatory authorities so they can monitor your operations remotely, for example. Perhaps you use third-party monitoring or cybersecurity tools and need to stream data from your own systems to that software. Or, even if you are only exposing APIs and data to internal consumers, perhaps that data contains sensitive PII and additional security mechanisms are still necessary. And, for almost all use cases, being able to centrally expose, catalog, and make APIs discoverable is a major leg up.

Even if your business doesn’t currently expose data streams externally, by the way, there’s a good chance it will in the future. As Gartner notes, “Organizations are improving their decision intelligence and real-time applications by tapping the growing availability of streaming data.” The future for most organizations is likely to include more data streams and more complex use cases for those streams, making the functionality of next-gen API management increasingly critical to business success.

Conclusion

In short, it’s time to move beyond the confines of legacy API management – which means no longer being limited to supporting only REST API management. To capitalize fully on the opportunities enabled by streaming data, businesses need a more flexible and dynamic approach to API management, which requires a new breed of API management software.

October 5, 2023
IBM LLM AI: COBOL to Java ASAP ¦ ARM IPO is GO!

Welcome to The Long View—where we peruse the news of the week and strip it to the essentials. Let’s work out what really matters.

This week: Translating legacy COBOL code to a slightly more modern language, and Arm will go public (again) next month. (more…)

August 23, 2023
Four Ways to Modernize Your Enterprise Monoliths

In a perfect world, every application would run like Netflix–by which I mean every application would be cloud-native, microservices-based, containerized and, in all other senses of the word, modern. Unfortunately, not every application can be transformed in these ways. Some of the most important enterprise applications–like those powered by SAP or Oracle–don’t lend themselves to architectural transformations simply because the vendors don’t support that strategy. As much as organizations might wish that they could re-architect legacy apps from the ground up, the reality is that they just can’t.

Fortunately, though, all hope is not lost if you have an enterprise app that you can’t containerize or “microservices-ize.” Even if you can’t “break down your monolith”–to quote the standard advice you’ll find if you read articles about how to modernize legacy apps–there are a variety of practical steps you can take to make legacy, monolithic applications look and feel (in most respects) like modern, cloud-native apps.

Here are four ways to make your enterprise monolithic apps more cloud-friendly, even if they’re not technically cloud-native.

Use Cloud Automation to Modernize Legacy Application Management

Most enterprise apps are compatible with the cloud in the sense that you can host them on cloud-based virtual machine (VM) instances using services like Amazon EC2.

By taking advantage of cloud hosting for enterprise apps, organizations can also leverage cloud automation tools, such as autoscaling policies and rules that automatically start or stop VMs. In this way, it becomes possible to deploy enterprise apps in a highly scalable, automated way. You get something that resembles Kubernetes-based orchestration, except your apps don’t actually run in containers on top of Kubernetes. They’re just monoliths that benefit from cloud automation tools.

Deploy Load Balancers to Optimize Performance

Load balancers, which automatically manage network traffic flowing to and from applications, are another critical tool that organizations can easily leverage when they migrate legacy applications to the cloud.

Although configuring load balancers for enterprise apps hosted in the cloud is not a strict requirement, doing so can substantially improve application performance and reliability because load balancers help to ensure that requests are directed at application instances best able to handle them. They prevent situations where one instance of your application is overwhelmed with traffic while another sits idle.

Again, this is something that platforms like Kubernetes also do for microservices apps. But, as I’ve explained, you don’t actually need your app to run as microservices to take advantage of load balancing. You can do it with monolithic legacy apps, too.

Adopt an Everything-as-Code Strategy

When your app lives in the cloud and is managed by cloud automation tools, it becomes possible to manage most aspects of application operations via code. Specifically, you can:

● Use infrastructure-as-code (IaC) tools to automate infrastructure provisioning and setup.
● Automate application configuration management using code rather than applying configuration changes manually.
● Codify operational processes, such as application updates and backups, instead of relying on manual operations.

Practices like these, which enable a strategy known as everything-as-code, significantly reduce the amount of time and manual effort that engineers spend managing applications. They also provide consistency and minimize the risk of errors due to human oversight. And there’s no reason why you can’t leverage these practices for legacy apps as well as modern ones.

Integrate Legacy Apps With Modern Toolchains

If your enterprise app is hosted on-premises, you probably manage it using tools and processes designed solely for it. But when you move an enterprise app to the cloud, it becomes possible to manage the application using the same integrated toolchain you deploy for other applications and processes.

You can, for example, take advantage of cloud-based monitoring tools to continuously collect data about application performance and generate automated alerts when something is wrong. Likewise, you can regularly create image-based backups of your application.

This toolchain-based operational strategy is another way to streamline enterprise application management, even in situations where you can’t rewrite or containerize the application itself.

Conclusion: Getting More From Legacy Apps

Just because you have a legacy enterprise app that you can’t transform into microservices doesn’t mean you have to settle for legacy management and operational practices. Modern automation tools combined with modern operational strategies like everything-as-code make it possible to achieve an enterprise application management experience that closely parallels that of fully modern, cloud-native apps, regardless of which architecture your app actually uses.

June 15, 2023
Why Legacy Application Development is a Challenge

Enterprises seeking to modernize their legacy mainframe applications often cite the key reason as not being able to evolve quickly enough to support the changing needs of the business. This is also a driving force behind the increasing trend toward moving these legacy applications onto other platforms, either by attempting to rewrite or re-platform them.

If this was a simple process however, the transition would have happened years ago. So, why is it so difficult to modernize mainframe applications? It is surely not the programming language. COBOL and other traditional languages are simply programming languages. They may not be considered “cool,” but neither are many of the other scripting or development languages, or configuration syntax programmers deal with day in and day out.

So, what actually makes legacy application modernization challenging? After countless conversations with several outsourcers, system integrators and mainframe customers, four major issues came up time and time again.

Development Tooling–Low Productivity on Mainframe

The traditional environment for a programmer, working with COBOL or PL/1 on the mainframe, has few of the productivity features today’s programmers expect: smart cross-references, on-the-fly syntax checking, autocomplete for variables etc. Attempts have been made to bring the experience closer to that of a modern JAVA programmer, but there is no escaping the fact a mainframe exists in the background with all its quirks. This divergence in programmer productivity unfairly tends to stick to the programming languages on the mainframe. With the result that language choice gets conflated with developer tooling in decisions to make mainframe development more agile.

Open-Source Availability–Limited Access to Innovation

Looking for solutions to business requirements among the many popular open-source projects is one of the fundamental reasons why development processes in general have become more agile. Clearly, if less code is needed to support any requirement, the faster that requirement will be satisfied. Very few open-source projects have been implemented and tested on mainframes. Perhaps even more troubling is the difficulty of integrating a Linux-oriented solution with legacy applications in a natural way.

Pipeline Integration–Separate Pipeline and Processes for Mainframe Development

In a similar vein to developer tooling, modern development pipelines do not fit naturally with the way in which mainframe applications move from development into production. Build processes, testing, source-code management and promotion are integrated and streamlined within a modern development pipeline. Specifically, modern tools enable developers to promote locally-tested changes and have those changes flow through an automated testing pipeline, with steadily increased levels of integration with existing applications and data. The pipeline is facilitated by the tight integration of tools such as GIT, Eclipse, Jenkins, Containers, etc. to leverage the best practices of CI/CD. All working in harmony with an agile development organization to get application updates to production faster. The mainframe is not as efficient, mainly due to nascent support for many of these technologies, alongside the costs of running many of the automated processes repeatedly.

Testing–Limited Autonomy and Automation

Testing, as part of a modern development pipeline is a highly automated and scalable process. The ability to launch containers with a single click is a fundamental reason why development is faster today than in the mainframe era.

Development expectations for testing extend to being able to launch a complete testing environment on the developer workstation, without recourse to systems administrators, for the recurring small unit tests intertwined with the development itself. Due to the physical hardware dependency of mainframe development, this testing autonomy can be highly impractical for legacy applications.

Furthermore, large scale-tests, which can be regularly, automatically and cost-effectively scheduled in modern, container-based cloud environments, require extensive budgeting and planning for legacy applications.

Consequently, all testing of legacy application developments is extended by the need to involve mainframe systems administrators, and schedule expensive mainframe resources.

Solving the Issues

To address the issues above, a solution is required which breaks the software lock-in that ties a legacy application to the mainframe. It is possible to rewrite or re-platform existing applications, although this can be time-consuming, costly and risky. A more beneficial approach is to rehost the mainframe applications by migrating them to the cloud, without recompilation or conversion of data types. This method replaces the proprietary, mainframe hardware and software dependencies with a set of Linux libraries, requiring less time and fewer resources to execute.

Linux libraries can be created to include features that recreate the behaviors of the operating system, databases, security and language runtime that previously tied the application to the mainframe hardware. In addition, the incompatibility of the mainframe instruction-set with x86 hardware can be managed via a virtual machine.

If the mainframe is represented as a set of libraries in the same way as any other more conventional application dependency, then the entire problem of legacy application development is reduced. Any workstation or commodity server can fully virtualize the mainframe environment, in several instances in parallel–if needed–thanks to containers. And with total elasticity when on the cloud.

Migrating to the cloud makes it possible for a developer to spin up a full test environment for legacy applications on their workstation, in the same way as they would for a Java or C# program. The entire development cycle for legacy applications can follow the same pipeline as for other applications. The same agile models, and CI/CD DevOps policies with strong emphasis on shift-left testing can be employed. The same automated scale testing is possible. The same exploitation of open-source projects can be considered.

Once rehosted, the word “legacy” can be dropped from the description of these applications. They simply become applications written in COBOL or PL/1. Everything else about their ongoing enhancement is identical to those applications written in modern languages.

The highlighted issues that businesses seeking agility are facing, are only issues that exist on the traditional mainframe. Migrating legacy applications to a software defined mainframe, running on an x86 architecture provides a modern dynamic environment in which to develop, innovate, integrate and test applications.

— Dale Vecchio

October 3, 2019
How to Modernize Mainframe Applications
It’s easy to understand the current disillusionment around the term “digital transformation,” as it has been thrown around vaguely as the ultimate answer to business innovation without much recognition of the nuances of technological change.

When it comes to mainframe systems, the issue here most likely can be traced back to the word “transformation.” The perceived challenge inherent in the transformation of a vital business process is that it involves a paradigm-shifting, instantaneous “big-bang” overhaul of IT, with no recognition of the complications involved in modernizing a complex web of intertwined systems and data.

It is this perceived challenge that has seen large enterprises reluctant to change and modernize legacy technology, as the pressure to throw out the old and bring in the new is seen as simply too risky. Horror stories, such as the infamous failed IT upgrade at UK retail bank TSB in 2018, have dampened the spirit of many enterprises to modernize with the perception that a big transformative leap may be required.

Many businesses seek a solution to this “all or nothing” conundrum and would benefit from modernizing legacy IT in small steps—carefully considering the modernization of infrastructure and applications as separate processes. This more graceful approach delivers gradual benefits while reducing risk and acknowledges that modernization is a continuum, not a destination.

What is the Reality of a Modernization Process?

Applications developed to support business requirements are never static; they change along with the business. Modern development environments, infrastructure and DevOps processes have benefited from tremendous innovation in recent years, enabling IT to more rapidly evolve applications to support these changing business requirements.

There is mounting pressure to integrate mainframe applications, which house critical business processes with these approaches. The aim of this modernization is to enable legacy applications to take advantage of the innovations spearheaded by the modern world of open source—and, therefore, respond more rapidly to the needs of the business. To achieve this, organizations would benefit from a two-phase approach:
1. Modernizing IT infrastructure: Moving legacy applications, in their original formats, to an environment where they can benefit from the scalability benefits of cloud deployments, exploit open source solutions and significantly reduce cost once legacy systems are replaced.
2. Modernizing mainframe applications: Rewriting or transcoding into modern languages only those legacy applications or programs that are most in need of modernization, while enabling them to communicate with those still in legacy languages.
Infrastructure Modernization

The foundation of infrastructure modernization is the movement of legacy applications, written and compiled to run on the IBM Z mainframe, to the Linux operating environment running on x86 architecture computer, either on-premises or in the cloud. Using conventional methods to re-host these legacy applications into x86/Linux requires all programs that make up a legacy application to be recompiled and, in many cases, changed. This approach has often resulted in budget overruns and a high risk of failure.

If an organization is able to move these applications and data to a modern environment without the requirement for recompilation or reformatting, this migration risk is significantly reduced.

Once moved to a Linux environment, and despite remaining in original languages, these applications can now benefit from a vast array of tooling to optimize their ongoing exploitation. They can take advantage of cost-effective scaling options, modern monitoring, high-availability tools and many other facilities available on Linux on x86. For those legacy applications that continue to be maintained, modern development environments are now available, meaning their management is no longer restricted to a diminishing mainframe workforce.

Application Modernization

The goal of many companies is to have their entire application portfolio based on modern languages or packages. Where legacy applications are concerned, attaining this means converting the individual programs from COBOL or PL/1 (and, in some cases, even more arcane languages) into modern languages such as Java. In so doing, an organization can tap into much larger pools of development talent and tooling. Furthermore, the door to an entire world of open source innovation is opened, which alters entirely the trajectory of application development for these legacy applications.

Yet modernizing mainframe applications in this way, using conventional methods, is not a process to be taken lightly.

Infrastructure vs. Application: The Chicken or the Egg?

When looking to modernize an application, many organizations question whether to start with the infrastructure or the application itself. The answer is infrastructure.

Of course, the organization could attempt to first modernize the applications in place. At a glance this method may seem the least disruptive, but very quickly the effort would run into many of the challenges previously highlighted that have caused the mainframe to be abandoned as a contemporary platform for application modernization. Using a modern programming language such as Java is one way to enable modernization with those programs being completely rewritten. With the whole process conducted on a legacy mainframe, costs and resource contention become practical issues. Organizations that run mainframes carefully control access to mainframe resources due to the punitive costs of exceeding licensed capacity; indeed, this resource contention point is another reason for companies to move ongoing development off the mainframe.

When taken collectively, it is easy to understand how these factors have conspired to thwart most modernization initiatives that choose to modernize applications first. By modernizing the infrastructure first, the issues identified above are circumvented.

Once mainframe applications and data are re-hosted on modern, open environments on x86, Linux and the cloud, the real process of application modernization, as outlined above, can begin.

The modernization journey is not easy, but continuing to depend on a declining skill set and diminishing vendor community in the mainframe world is no longer an acceptable alternative. By breaking up the modernization journey into manageable pieces, moving forward over time to the desired end state is the most realistic approach. After all, to quote a Chinese proverb, “A journey of a thousand miles begins with a single step.”

— Dale Vecchio
April 8, 2019
DevOps Chat: App Modernization and Service Virtualization with IBM’s Matt Tarnawsky

While everyone wants to focus on building a new app, the fact is there are many, many apps already out there. They need to be maintained, updated and modernized. How do you upgrade these legacy apps to new infrastructure, frameworks, languages and more? App modernization is a thing.

In this DevOps Chat we speak with Matt Tarnawsky of IBM about app modernization and service virtualization, testing and more.

As usual, the streaming audio is immediately below, followed by the transcript of our conversation.

Transcript

Alan Shimel: Hey everyone. Alan Shimel, devopsy.kinsta.cloud, and you’re listening to another DevOps Chat. In this episode of DevOps Chat we’re going to spend time talking about app modernization, testing, service virtualization. We will see what else kind of gets dragged in within our time frame.

Our guest, you know, who’s going to serve as the expert today, Matt Tarnawsky. Matt, hopefully I didn’t mess that up too bad. Matt is with IBM. And Matt, welcome to DevOps Chat.

Matt Tarnawsky: Thank you, Alan. Good to be here.

Shimel: Absolutely. Pleasure to have you here. So Matt, first of all, I did not mangle your name too badly, I hope.

Tarnawsky: No, you got it pretty good.

Shimel: Fantastic. So Matt, following our tradition here on DevOps Chat, we like our guests to introduce themselves. So why don’t you tell us a little bit about your current role at IBM and also a little bit about your own personal journey that led you to be here today.

Tarnawsky: Sure thing. So I am the product manager for IBM’s Rational Test Workbench and Rational Test Virtualization Server, specializing in API testing and service specialization. In terms of how I got here, I have had an interesting journey, starting off with working with a lot of the circuit boards and SPGAs, working with those and then testing those as well.

I ended up migrating across to working with a small company called Green Hat Software building, AGI testing and service specialization tools. We were acquired by IBM several years ago and I’ve been enjoying the journey ever since.

Shimel: Good for you. That’s a great story met. And you know, it’s fairly – it’s more usual than unusual in the IT space when you see someone – when we look at the arc of our careers, five years from now who knows what we will be working on. It may not even exist or we even know about it today. So it’s kind of right par for the course.

So Matt, as I mentioned earlier, I wanted to talk a little bit about app modernization. In my mind, anyway, it’s a loaded – that phrase is a loaded phrase. We hear it bandied out about a lot but when you talk to people – to some people they are talking about, “Well, I am modernizing my existing app. So I’m taking my legacy applications and modernizing them for new infrastructures, new delivery methods, new imports, what have you.” So really about modernizing your existing legacy applications.

To other people, though, it’s more about modernizing the way you develop, deploy, and manage applications. So modern – application modernization. Of course, no matter how we are talking about it, testing plays a vital role in this. So why don’t we first do a little level set. You know, Matt, what you – when you hear app modernization, what does it mean to you?

Tarnawsky: So to me, it encompasses a lot of things you mentioned _____ _____ _____ _____ people talk about is, how to do I _____ _____ my app to new markets, new interfaces. That can be things like mobile. It can be things like presenting apps _____ _____ _____. And that often does require _____ _____ _____ _____ their interfaces to actually bring them into the _____ away from our systems that they may have had in the past.

But often, it also means they want to look at the faster methods of delivery, new development styles. These things often go hand in hand. You often can’t get those faster methods of development with rapid feedback and everything else without looking at, how do we change our entire development life cycle; trying to be stuck in a waterfall method doesn’t really get you that same acceleration that you can get when you start looking at, how do we automate everything and moved to DevOps.

Shimel: Got it. Got it. Now you broke up there a little bit back on – and I’m hoping people listening will be able to catch all of that. But let’s – so we may fill in. But let’s talk a little bit about the role of testing and that framework of app modernization. Obviously, testing is important, but how so and in what context?

Tarnawsky: So I actually see – you know, if we are taking older, monolithic applications and looking to migrate them towards micro-services, it’s a great opportunity for testing. You are putting things into small units that can be tested on their own. You get things like – let’s say you’re migrating to a _____ based _____ based service, you get the opportunity to describe it with _____ Swiger or _____ or open API. And you’ve got a contract there that you can then test against.This means that things now have the parameters, the what needs to be tested, how to test it, and it can be imported into many different test tools to help them start testing, but it’s smaller bits of functionality. So it’s a great opportunity to break things up and really look at how those can be worked with at a smaller level.

At the same time though, it also introduces new opportunities for more complex interactions that we didn’t previously anticipate. So let’s just say you’ve taken that big monolith. You want to expose micro-services to the rest of the world or indeed to other teams in your organization well, suddenly the integration that you had relied on in the past might not be the same ones that you are looking at using in the future. You need to start looking at how we test not just those smaller levels of the micro-services, but also looking at how we test each of those integrations over time.

One of my customers has a great analogy that they like to use in terms of the testing. They said when you’re testing a car, you don’t just test the car. You test little pieces of it. So you might start off by testing the spark plug. Then you might say we will integrate some pieces and see how those work together. We test the engine out. Then at some point in the future, we will put the engine in the car and we will test that as – you know taking it for a test drive, taking it for a spin, and make sure the entire end-to-end system works as well.

Shimel: Excellent. Excellent. Actually, that’s a great analogy. Great analogy. So Matt, another phrase we hear, and we’ve all been hearing it, a lot of us, in IT for a long time now, is service virtualization. I think people have a better handle on what we mean when we talk about service virtualization. But again, let’s lay a foundation. When you talk about service virtualization, especially in terms of app modernization and testing, what does that mean to you?

Tarnawsky: So service virtualization is where we take some sort of interface that we might have had that could be something silent rest. It could be something like Mqueue. It could be a legacy mainframe interface like Kicks. We provide some sort of mock that is going to handle the interactions with the interface. That can be coupled on the rest of the application. It starts to be providing us – I’m sorry.

It starts providing us with the ability to look around and – I’m sorry about that. Got myself a little bit jumbled there.

Shimel: No problem.

Tarnawsky: Basically, it gives us the ability to remove restrictions from ourselves. So if we had legacy items in our environment that we might not have in the cloud yet, well actually, we can do is we can say, well, look, that mainframe connection, that SAP connection, whatever it might be that’s in our current on-prem environment, we can virtualize that, we can house something that sits in the cloud. And in fact, our developers can have an entire environment in the cloud that is no longer linked to those legacy systems.

Shimel: Fair enough. So how does this all play out then, Matt? Right? So now we’ve got our little parts here and we’ve defined what we mean by this. In your world, where are you seeing sort of rubber meets the road, if you will, in terms of organizations deploying, employing, doing this kind of stuff? Make sense?

Tarnawsky: So we see all sorts of things. We do see people looking at that – we want to do a migration. So we are going to record various pieces of environment and have them available to replay. Just very, very basic style of creating virtual services, but it can be effective. And indeed, they can do the same sort of thing for their API testing as well and just say, “Have we migrated this correctly? Does it still have the same functionality we had last week when we had it on-prem as we got on the cloud now?”

That’s a very effective way to look at things. But equally, we see all sorts of interesting stories come up from customers, whether that’s looking to virtualize the way a third party – so we’ve seen customers look at things like Apple Pay or Experian or whatever, where they have to pay every time, they had that third-party service. Or indeed, that might just don’t have control over it and they want to simulate what happens if that third-party service responds or throws an error code or whatever.

We see customers who do some really – one of my favorite stories comes from a customer who was looking at testing out their credit card issuing systems. Now, you can imagine that as you’re looking at issuing a credit card, at some point you’re going to print a credit card. Now, just imagine test automation for that sort of thing. Do you really want credit cards popping out of the machine? No you want to be able to test things effectively, but not have these physical dependencies.

So what they did what they looked at the traffic that was going through that credit card machine, the issuing machine. They said, here we got a point where we can virtualize some things in our environment and remove this dependency that we currently have. We don’t want to print real credit cards. It’s a waste of time, waste of resources, and in fact, it slows that test automation down. So if we can virtualize that out, we can get faster feedback back to the development team. So that’s exactly what they did.

Shimel: So Matt, we recently did a webinar over on devopsy.kinsta.cloud you are probably aware of it, modernizing testing as apps re-architect. Is this the kind of stuff we are talking about?

Tarnawsky: Very much so. You want to make sure that your testing stays in pace with that re-architecting of the apps. A lot of that does come down to developers delivering fast from the cloud. And tests more and more need to be automated. That doesn’t remove the value of exploratory manual testing or anything like that. But we do see that need for testing to become more and more automated in the future as well, as this change happens.

Shimel: Absolutely. You know, another thing, and I get this a lot, Matt, when I’m traveling around the world. Is this a sort of – not to use the – abuse the term, but first world problems, right? Is this where… In terms of the market, is it sophisticated, large enterprises who already have legacy infrastructures, a history of service virtualization and so forth? Or is this something that’s really kind of more grassroots that’s happening across the board, across the globe, as organizations are starting to move to app modernization and dealing with the realities of having to test around this stuff?

Tarnawsky: So we see all sorts. Actually, the example I gave you earlier, the bank that was looking at the credit cards, was a fairly new bank here in the UK.

Shimel: It was? Okay.

Tarnawsky: They were – that they are very much based on looking at how they can do things in the cloud, trying to keep things as modern as they can. They are looking for all the latest techniques, all the newest tools that can help speed up their development and their time-to-market. So yes, we do have a number of customers using this stuff for legacy enterprise systems. They are looking to virtualize the way a mainframe, for example.

But equally, we see other customers come to us and say, “I want to accelerate my development and my testing as fast as we can. What can you do to help us?”

Shimel: Good. Interesting. You know, Matt, I mentioned when we started that the time goes pretty quick. We are coming up on the end of our time frame here. But for people who maybe want to get more information on this, where can they – where can we dig? I mean, I should mention that the webinar I referenced earlier is available on devopsy.kinsta.cloud and YouTube, a recording of it along with the slides. So people who are interested there should absolutely head on over and get that. We will try to include the URL in our show notes for today’s podcast.

But where else should they go, Matt? How do people get smart on this?

Tarnawsky: So I would invite them to come on to IBM.com, searching for Rational Test Workbench or Rational Test Virtualization Server, depending on whether they are interested in test automation or service virtualization, as a starting point. And we’ve got loads of white papers. We’ve got customer stories. We’ve got all sorts of things you can look at for more information just to see how other people have been doing things.

Shimel: Fantastic. Matt Tarnawsky, thanks for being our guest on DevOps Chat today. Pleasure to have you on. Hopefully we will have you on again soon.

Tarnawsky: And thank you very much for having me.

Shimel: All right Matt. Hey, this is Alan Shimel for devopsy.kinsta.cloud. You just listen to another DevOps Chat. Have a great day, everyone.

— Alan Shimel

November 30, 2018
Are Legacy Apps the Antithesis to DevOps?
Do you have an application marked with cryptic warning signs and a wealth of cobwebs that is running on legacy hardware hidden away in the back corner of your data center? If you’re in enterprise IT, chances are high that you do. These old platforms are often considered a bane to IT. More importantly, legacy applications can present a real headache when attempting to uplift the people, processes and tooling needed to embrace a hybrid cloud model.

Providing self-service, orchestrated workflows and delivery pipelines are just a few signature attributes to constructing a hybrid cloud. Imaging trying to apply these ideas to legacy technology, where the interface requires using an archaic console and software that looks like it was written for Windows 3.1. It’s not fun and often derails any efforts to deliver services living in an on-premises software-defined data center (SDDC) or public cloud environment. It also splits up a team focused on working within the realm of a DevOps cultural model, because concepts such as infrastructure as code, stateless configuration and collaborative code check-ins crumble. For most folks I speak with, legacy applications remain with a small skeleton crew that keeps the lights running and the old hardware humming.

Hybridity is a reality for most infrastructures rather than being purely in the public cloud or purely on-premises. This is where the importance of DevOps and embracing an API-first mentality is multiplied. Having an API-first approach isn’t necessarily about having a single code repository or a single application that does “all the things.” It is about leveraging APIs across multiple repositories and multiple applications to weave together a single, programmatic software fabric in which everything can communicate and integrate regardless of whether legacy or cloud-native.

The way to end chaos induced by the complexity with hybridity is a progressive, policy-driven state with proper implementation.

Here are four tips on how to integrate your legacy crew with your DevOps team to create a high-functioning hybrid cloud model:
- Find tools that play well with others: Legacy applications do not have to be the anchor holding you back from adopting a DevOps strategy. Find solutions that treat legacy applications, modern applications and cloud-native applications as if they were all first-class citizens. Legacy applications tend to run on old OSes, including some that are no longer supported. Eventually this configuration becomes increasingly fragile, requiring manual care and feeding. Adopting an automation strategy can reduce risk associated with build, testing, deployment, remediation and monitoring.
- Strive for simplicity; don’t create silos: Make sure the workflows that deliver services are applicable to the vast majority of use cases. Historically there has been little to no incentive to building mechanisms that are shared across the enterprise. Systems thinking advises that there are dependencies, but if you only improve one application in the cluster, there will be no benefit. The entire cluster must be addressed for successful progress. The use of RESTful APIs is turning tables, allowing for a single set of tooling across many applications, platforms and services. Share data across the organization; leverage APIs to streamline how the team interacts with legacy workloads by extracting data from the legacy architecture.
- Embrace a one-team mentality: Don’t form multiple teams or tiers of teams. If the decision is made to adopt DevOps, then it must be embraced by the entire organization. As the old adage goes: “There are no legacy systems, just legacy thinking.” DevOps isn’t about developers and operations folks collaborating; rather, it’s about two separate silos becoming one team. Focus on improving communication and set aside time for learning.
- Avoid infrastructure-specific solutions: Abstract storage and compute; work instead on the application layer and how to deliver services for those applications. Legacy applications often are coupled tightly to the underlying hardware, making it challenging and risky to manage any application component individually. This means maintenance and upgrades are incredibly time-consuming, difficult and even expensive. The adoption of infrastructure-as-code, whether on-premises or in the cloud, gives teams the permissions and tooling provision infrastructure on-demand. As you can imagine, not every legacy application can be easily migrated to this type of infrastructure, but many can.
Using automation increases the organization’s agility by reducing human interaction and orchestrating dependencies across the organization. Self-service consumption of infrastructure further eliminates silos. DevOps is not incompatible with legacy applications, but it requires an organization to evaluate what this implementation actually means and how to embark upon this transformation. Never fear: You can still depend on a legacy application and embrace DevOps. It requires additional design considerations, elbow grease and a little bit of creativity. The fundamental element is to apply these principles in a consistent and efficient manner in the context of all applications, both legacy and modern.

— Rebecca Fitzhugh
October 9, 2018
DevOps Chat: Modern Apps in DevOps w/ Andrea Martinez Crawford, IBM

Andrea Martinez, now Andrea Crawford, is the CTO for DevOps for IBM Hybrid Cloud. In this DevOps Chat we had a great discussion on what is “modern apps in DevOps,” how to upgrade your legacy apps, what to do if your legacy apps are not upgradeable and what this all means for DevOps.

Great discussion and as usual, the streaming media audio of our conversation is immediately below, followed by a transcript of our conversation.

Alan Shimel: Hey everyone, it’s Alan Shimel and you’re listening to another DevOps Chat. DevOps Chat today is going to be a good one. We have with us Andrea C. Martinez who is the CTO for DevOps for IBM’s hybrid cloud.

Andrea, welcome to DevOps Chat.

Andrea Martinez: Hey, thanks for having me, Alan.

Shimel: Thank you. Did I get that all right?

Martinez: So, I’m mostly known as Andrea C. Martinez but I’ve recently gotten married so my new last name is Andrea Crawford. Different last name but the same cowboy hat. So I always wear my cowboy hat in my profile pics. If you’ve got the hat you’ve got the right gal.

Shimel: And it’s the right Andrea. Okay. No hyphen – Martinez-Crawford or something like that?

Martinez: Nah, I’m not that fancy. It’s just Crawford.

Shimel: I’ve got to tell you I just got back from China last week and I was with my partner Jayne Groll, one of the co-founders of the DevOps Institute, and one of my co-founders there, Jayne Groll, is actually now Jayne Groll-Ray because she’s also recently married and what a mess on the – hats off to you – getting your passports done and redone and what name to use because you know it has to match exactly and all your other IDs – I didn’t realize what a –

Martinez: Alan, that’s why we get the diamonds in our ring.

Shimel: I guess so. Yeah, and well-deserved. Anyway, congratulations on your recent marriage but we’re going to talk DevOps and we’re going to talk a little bit about modern applications here today, okay?

Martinez: Awesome. Sounds good.

Shimel: Great. All right. So I should mention that we have a webinar coming up on June 28 at 1 p.m. Eastern time on this topic and really this chat is just sort of a lead-in to the full hour-long webinar and that webinar is on DevOps for application modernization and we’ll put the link to it in our show notes here but if you find what we’re talking about today interesting by all means check out the webinar.

But let’s dive in. So Andrea, let’s talk a little bit about modern applications versus non-modern applications: Do we have archaic applications, are they obsolete applications? What do we mean when we say a modern application?

Martinez: Yeah, I think to – and by the way, I like your archaic reference there. I think to really appreciate the modern application we really have to understand how applications use to be designed and built not all that long ago.

So, if I were to go back to the N-1 generation of applications I could make some pretty broad characterizations of how they were designed, built and delivered. So, if we were to like take it back to around the late-Clinton era of post-dot.com boom, world wide web and the internet, you know, that was really becoming the network platform where enterprises were exposing business function and really more of nascent digital economy.

So some of the characteristics of what I’ll call “heritage applications” were, No. 1, they were monolithic so the applications lived on these things called “application servers,” typically running on maybe like higher end commodity infrastructure – think like X86, power hardware with large amounts of disk and memory, we had big centralized databases and these memory hog application servers.

So the applications ran inside these app servers and they were fairly monolithic – that means they basically bundled the presentation or the view, the business logic and the data model components all together.

The second characterization I would say for heritage applications is that they lived on these app servers that resided on these sort of heavyweight operating systems and although at the time Linux was very much on the scene overall. The trust in Linux was not yet solidified so you had these operating systems that were sort of these enterprise, server, big and bold editions of this OS – and it was pretty prevalent, you know? Windows and proprietary Unix based operating systems pretty much ruled the day.

And, No. 3, the – in terms of the infrastructure on-premises data centers ruled the roost. I mean virtualization technology’s around, we had this notion of logical partitions, L-pars, VMware, virtualization, but it was mostly a hardware OS story, okay?

And then we had, in terms of looking at the actual heritage applications themselves, we had the rise and dominance of function-rich languages like Java and dot-net because the application server middleware pretty much dictated the language of what an application was coded in, right? So we still have a lot of these skills around today.

And I think the last characterization of a heritage app was in the way that they were delivered. So we had the waterfall delivery – very sequential, big-bang releases with siloed organizations, dev teams didn’t really talk to the operations teams and having that kind of silo was pretty much the norm.

So, that’s how I would sort of characterize the heritage applications.

Shimel: Yeah, dead on for me Andrea. We didn’t talk about this but in my background I had started a web hosting company in those dot-com days and I sold it to a company and started to work with them and we were one of the first big, what they called ASPs, if you remember those days – application service provider – and so this is – we wound up doing the whole IPO and all of that – this is 1997, ’98, ’99, 2000, into 2001 –

Martinez: Absolutely. Yep.

Shimel: Oh, they were glory days. We were probably one of the largest Lotus notes, if you – you know,

Martinez: Yeah, all right.

Shimel: So, picture this: We’re hosting – we are offering hosted Lotus notes as a service – it’s not even IBM WebSphere yet, I think it was still notes server –

Martinez: Domino?

Shimel: No, it wasn’t even Domino. What came before Domino? I mean this is old. And then Domino came out and it kind of rocked our world because it was really a big thing for us. But there’s no – it was on dedicated hardware, there was no cloud, T1 lines were about $1,200 a month, and so you had to deliver it over a T1 line oftentimes – you know, a dedicated T1 line point to point – and then we expanded to PeopleSoft and Oracle apps and stuff like that, and what a disaster. I’ve got to be honest. I mean we went through about $500 million – a half-a-billion dollars in monies raised between the IPO and so forth – but it was hard. It was really, really hard.

And then we tried to get into the whole custom app kind of hosting with ISVs and so forth.

Martinez: Oh boy.

Shimel: So, what you just described I had a déjà vu to my bones and, yeah, I wouldn’t call those modern applications but I think we’ve done a good job.

Now contrast that, though, with what modern applications are like today, Andrea.

Martinez: Yeah. So there’s sort of been an inflection point where some pretty emergent things have happened since those days, right? And you know we dealt with all of those issues around the heritage environments and applications. And now all of the sudden if we take a look at what I call “modern applications” today – Alan, you mentioned the cloud – No. 1, modern applications live in the cloud. So these applications leverage the elastic, infrastructure and platform that comes with it.

When we develop modern applications – you also may have heard them referred to as “cloud-native applications” or “born-on-the-cloud applications” – and really what has happened here is to leverage that type of virtualized infrastructure we’ve started to breakdown the monolith and the applications that run on them.

So, now we’re taking our bundled model-view controller applications and frameworks and we’re deploying them separately as components and hopefully we’re using architectural styles like microservices so that the monolith becomes broken down into choreographed microservices that can be managed separately, okay?

So, we have technologies that actually support this today. And so Docker, Kubernetes are some of the technologies that have come on the scene since that have allowed us to have a control plane and an environment to manage smaller-granularity applications which are hopefully services.

So, the other thing about modern applications that I think is interesting and has fostered the adoption of these microservices and smaller components is the fact that the solution stack has become more and more commoditized, right? And so, when you think about the solution stack of network hardware, operating system, middleware layer and so on, we’re seeing a commoditization of the solutions stack from the bottom up, okay? And that is – the whole notion of cloud computing, right? – We’re no longer concerned so much about CPUs and CPU architecture, you know, X86 versus Power P Series, this or that – it sort of doesn’t matter as much anymore, right?

And because of that, I think that has also fostered in some cases where critical proprietary software was not necessarily needed for enterprise applications. And so, when you start to break down the monolith and deploy smaller services, you’re isolating your risk because you’re isolating business functions and smaller chunks that are lighter-weight and it’s really analogous to the difference between deploying a 10-ton gorilla and deploying a colony of ants. If one ant dies it’s not a big deal, right?

But commoditization has even spread to the way that these little ants or microservices talk to each other with the advent of API interfaces and as long as you use rest APIs over HTTP or a lightweight messaging protocol that’s pretty much the lingua franca of modern applications today.

So, I think there’s probably this notion of componentization and that is actually fed into the notion of polyglot programming, and so enabling modern apps to be programmed in just about any language. So, you’re no longer bounded to this big, proprietary application server; you can program in just about any language, and as long as you can talk – rest over HTTP or send an A-synch message over a lightweight message hub – it sort of doesn’t matter.

You know, you can be a Java or a dot-net bigot but, hey, the new guy who knows Python and is really good at it, he can get in on the action, too. It’s all good.

Shimel: Absolutely. So, Andrea a lot of what you – you know, it was more than a mouthful there – and a lot of what you’re talking about really describes DevOps, right? This idea of chunking down stuff so it’s not the monolith, right, and doing small – you know, lots of smaller, little things rather than one large, bigger thing and learning and feedback looping and all of that – I mean this is – it’s DevOps 101, right?

And to me – and this is – you know, I speak to a lot of people obviously around the world about this stuff – but to me there’s a chicken-and-egg relationship between DevOps and modern application infrastructure, development and deployment as you’ve described. I don’t know if you could have one without the other. I just don’t know which one comes first. But they’re intrinsically linked, right?

And this is why, in my mind anyway, we need – you know, there’s a special relationship, if you will, between DevOps and modern application and how we do modern applications today.

Martinez: Spot on, Alan. I mean, when you think about the gorilla versus colony of ants analogy. Think about having to deploy the 10-ton gorilla – it’s typically pretty complicated but there’s a lot of dependencies and release management is of utmost concern – but deploying the colony of ants you’re deploying a lot of little things at different times – so you’re achieving the multispeed IT – if you don’t have DevOps, if you don’t have that kind of automation to do the build, compile, package, test, deploy and you’re doing it intra daily – if you don’t have that automated then your cost case is going to fly out the window.

DevOps is essential for modern applications. Period.

Shimel: Dead on. Dead on. Amen. You know, you’re preaching to the choir here, Andrea – we’re devopsy.kinsta.cloud – but it really is essential and there is this sort of hand-in-glove type of relationship.

We are, as I promised you when we started – the time goes so quickly and there’s more I want to cover. So we talk about how DevOps fits into the story on the how and why of modern applications.

You know IBM has this history now – and as I mentioned to you four or four-and-a-half years ago when I launched devopsy.kinsta.cloud IBM was one of the leading advocates for what was kind of the rise of the DevOps community and culture. And then you guys had the garage method, which was really kind of intrinsically again all about DevOps and DevOps ways, if you will – how does all of that fit in to modernizing applications and the way we’re doing it now?

Martinez: So the IBM cloud garage method is IBM’s opinionated view on how to deliver especially modern applications: It combines Agile, DevOps, Lean and our enterprise-design thinking and sort of rolls it all up into this wonderful burrito of reference architectures, practices and techniques – you know, it’s lunchtime, so I’ve got food on the mind – but it’s basically a way to help our clients act like a startup so that they can achieve the kind of agility that they need.

So, DevOps is also about organizational and behavioral changes because that’s really paramount. At the heart of our method is culture – how to organize our teams, the collaborative practices, social coding, pair programming – breaking down the silos, right – that’s what DevOps is all about, having product owners sit with the squads, right? – sit with them. You know, DevOps is a behavior story – a tool automation story – and it’s all in the name of enabling business agility.

Shimel: Absolutely. Andrea, I’ve got another final area I want to click on if you don’t mind. And that is for our folks listening out here – and it’s a worldwide audience at devopsy.kinsta.cloud – and I’ve traveled the world doing this, and though we all want to do DevOps we move at different speeds and we’re at different stages of our journey.

What advice, if we can kind of give blanket advice – what advice around DevOps would you give to our listeners who – and again, Andrea, this is something you’ve probably learned, right? – I wish we all had a greenfield to play in, but we don’t, right? We have brownfields and there’s heritage apps, as you call them, and there’s modern apps, there’s old and new, fat and skinny, ants and gorillas – even burritos – what advice can you give our listeners on how to deal with all of these things?

Martinez: I think just recognizing that you may have to approach DevOps a little bit differently depending upon whether you’re modern or heritage or even somewhere in-between. Typically DevOps for heritage applications, you’re retrofitting automation in situ, right? So, you’re sort of bringing the automation to the app. Whereas for modern applications, you definitely want to have a DevOps Day One approach, right?

Because the ability to create a cloud-native application and have the automation in your mind right on day one is critical. But just know that you may have instances where you may not be able to bring a heritage application all the way to modern – you might be somewhere in between Fred Flintstone and George Jetson, right?

Shimel: Two of my favorites.

Martinez: But that’s okay, right? I mean that’s real-world. Because it may not be cost-effective to automate everything but that’s all right, you know?

So I think we can engineer our way through some of this – and there are products out there – you know, you mentioned before that hybrid cloud was really sort of the area that I was focused in terms of DevOps – and we have proprietary products, open source products, we have our Urban Code Deploy, our IBM Cloud Private – that really sort of helps bridge the gap between applications living in this cloud or that cloud – whether they’re modern or heritage there is an answer.

It’s not cookie-cutter, but we have the piece parts and the emerging technologies on the scene to help navigate.

Shimel: Yep. Excellent. So Andrea that is important for people to know but where do we go – where can people go to find out more about this?

Martinez: So I think the first place I would point them to is the IBM Cloud Garage method website – and I’m sure we’ll be providing links as part of the podcast references – but at that site you can definitely find reference architectures for DevOps, reference architectures for microservices in cloud-native development – and one of my favorites – the practices and techniques that we employ with our garage method for automation, tooling and social collaboration throughout the software delivery life cycle.

Shimel: Perfect. So, Andrea, as I mentioned, the time here it goes so quick. It was delightful, refreshing and fantastic having you as a guest here. I’d love to have you – you know, where else are we going to learn about gorillas, ants and everything else? I need to have you back on the show. We’ll talk off-mic about it, but if it’s not too much – and we’re Florida neighbors – so I’d love to have you back here. But we’re going to wrap it up for this episode of DevOps Chat.

Andrea Martinez-Crawford, welcome and thank you for being our guest on DevOps Chat today. We will have links to a lot of what you’re speaking about in our show notes, so folks, please take a look at that. But for now, we’re going to call it a wrap on this episode of DevOps Chat.

You’ve just listened to another DevOps Chat and hope to see you soon everyone. Have a great day. Bye-bye.

— Alan Shimel

June 1, 2018
Moving That Big Project to CI/CD

Any organization that has been around for more than a few years (and many who haven’t) have a massive system that would cost a fortune to replace. Most midsized enterprises have several of them. Those systems are one of the reasons that enterprises don’t adopt CI/CD at the same rate as startups. Sure, any new project or small things go there, but the elephant in the room just hangs out.

So what happens when you get called upon to move one of these systems into CI/CD?

Lets’ start with the easy case: the rewrite. In that case, you’re almost greenfield from the start and can build CI/CD into your development and test processes as you would for any brand new project.

The harder case is when the org has seen the benefits of CI/CD and decides it wants to apply them to the monster that slows every project it touches to a crawl. These big systems are generally mission-critical, so there’s an added layer there. But this is doable.

Yes, It’s Daunting, But…

First, get a copy of the app building in a separate environment. Containers are good for this, since it’s just a development environment anyway. Replicate the entirety of the system (except unscrubbed real data, whether you use that in testing or not is a per-org question) to this new environment, make sure it builds and passes the existing test environment.

Next, configure Dev/CI/CD tools (for a project of the size we’re talking, I recommend configuring application release automation [ARA] also to help track dependencies), and get the system building and testing under them. This will take time—time to configure the CI/CD tools, time to get the build environment to work reliably under them and time to figure out how to integrate existing test structures into the CI tool. That’s okay, the point is investing now to save later. Note that this is also the time to upgrade your dev tools. If you’re on J2EE 5, for example, it is time to upgrade. It’s been a decade, and you’re working on a copy of the project.

Once that reliably builds and passes tests, the next step is to move those pieces into a more CI/CD-friendly format. Traditional test scripts can be called out in CI, but consciously building the tests within the CI environment is more resilient. A close look at tests, what’s missing, what could be better, what needs to stay as it is and just report back are the biggest parts of this test.

Note that change requests will have to be copied to the new environment’s change request system, too. Some projects are lucky enough to lock out changes to the source application while the new one is brought up to speed, but for most, changes made in the original will need to be replicated. Once you start modifying code, they can no longer be just pulled from the source tree.

Then start fragmenting the app. Apps of that size inevitably have massive functions/methods that do a lot of things. Not a terrible plan in single-processing days, but if you’re moving to CI/CD and upgrading dev tools, you may as well optimize code, too. Breaking a 5,000-line subroutine into a master and four child subroutines, each of 1,000 lines (yeah, it’s never that easy, but you get the idea) might offer room for parallelization, and definitely reduces your change footprint when you have to go in there and modify code.

Fear Pundits Bearing Advice

It is relatively common in DevOps for someone to offer the advice, “Get the newest of X every time.” Don’t follow that advice. Pick a version and hardcode it. It is anathema to hardcode versions of tools/OS/libraries in DevOps, but the point of DevOps is to control change, and anyone offering that advice missed the havoc that their advice would have created with the release of Python 3 or RHOS/CentOS 7.0, both cases where backward compatibility was left in favor of progress. Plan updates; don’t come in to, “The entire build failed last night,” and have to fix it then.

By this point, you’ll have a better understanding of the app, and will know (or at least suspect) where monitoring points will be best placed. Add them in to do active monitoring instead of logging. Your first set of points won’t be perfect, and you’ll want to review them to see what improvements you can make, but the weak points of the system should be monitored at a minimum, and alerting or corrective action automatically take place.

Test, Test and More Test

As soon as things are stable in the new CI/CD environment, bring in beta testers from the user community to help you find issues where it doesn’t behave like the original. The point here is to copy the original first, most likely to enable faster changes in the future, so just ask users to do what they would normally do when interacting with the app, and see what happens. Anywhere there are issues is a good place to add testing (I don’t know any monolithic app that has 100 percent code coverage, and honestly have only heard rumors of new apps that do, so there’s always room for more testing at known weak points).

Before you know it, the system will be moved into CI/CD, and you’ll have a list of things you want to improve under the new environment. At that point, you’re covered, and can move the process to replace your old build process.

Voilá!

It is work to move a large app into CI/CD, and more work to keep from letting it get out of hand. In the new environment, it is easy to want to change everything at once. But don’t make that part of the move; build change requests for after the move to CI/CD is done, to keep project lines clean and evaluate those desired improvements in the light of all the other change requests building up.

— Don Macvittie

May 29, 2017
Step-By-Step: How DataArt Processes Application Backlogs, Converting Legacy Apps with DevOps

DataArt designs and custom builds software for industries such as travel & hospitality, media, IoT, financial services, and healthcare. “We build new products and services, improving existing solutions, re-engineering legacy systems, and providing expertise in technological and industrial domains,” says Anton Krasikov, Project Manager, DataArt.

How DataArt Uses DevOps

DataArt quickly delivers high quality software in support of operations performance. DataArt’s DevOps process enlists version control systems like Git and build systems such as Ant, Maven, and Gradle. “Almost every project relies on using these build systems inside CI servers like Jenkins, TeamCity, or TFS,” says Krasikov.

DataArt’s development process also requires a lot of unit and integration testing using frameworks like TestNG and Junit. “There are also projects where we prefer TDD or BDD approaches, with the use of specific frameworks such as Selenium for UI testing,” says Krasikov. By using Puppet, Vagrant, and Chef, DataArt can swiftly deploy the development environment in the CD pipeline. DataArt applies these select DevOps development tools using methodologies, which could include Kanban or Scrum (Scrum was most frequently mentioned by DataArt for this story).

Previously, DataArt relied on manual processes or scripting tools for build systems, source code, and unit tests. “It took a lot of management effort to tie everything including development, quality assurance, and operations together,” says Krasikov.

Challenges to Adding Backlogged Applications to Those You Develop Using DevOps Approaches

In the instance where DataArt returns to work on existing systems that include some legacy applications, it is possible to use agile development in maintaining these as they are, but excessive software dependencies could make it hard to align the software releases / release cycles.

“Legacy software may not have good test coverage or a proper build system, so we usually take steps in a DevOps direction, if not to achieve continuous delivery, at least to implement proper continuous integration,” says Krasikov.

There are challenges to adding backlogged apps to a DevOps development process. “These kinds of non-DevOps applications usually suffer from several issues, or in worst case scenarios from all of them,” says Krasikov. Backlogged apps waiting in a pre-DevOps state and hungry for transformation may have any or all of the following symptoms, according to Krasikov:

*many dependencies between system modules
*no automated environments
*no continuous integration
*no test automation
*no build system or inefficient usage

Transforming an Application, Step-By-Step

To add apps to DevOps approaches, says Krasikov, start by analyzing the existing source code, system dependencies, and system environments. The steps that follow come from an example with a recent Java project that DataArt did.

A) Select a source code control system such as Git.

B) Do refactoring and perform unit tests. “These go hand-in-hand, so the main business logic was covered with tests. It does not make sense to rush to 100% test coverage, but we tested the main bits. This also allowed us to reduce system complexity and decouple system components,” says Krasikov.

C) Build system – “The original application used huge and confusing Ant scripts to produce deployable packages. After we completed the refactoring, we split the application into modules with Maven as the build and dependency management system,” says Krasikov.

D) Integration tests – “We introduced these as a separate test automation suite to ensure that application modules work correctly as a whole,” says Krasikov.

E) Continuous Integration – DataArt set up a Jenkins server for multiple build jobs, building with each commit and targeting deployments to their intended environments with overnight integration testing. “CI allowed us to identify non-working builds early to save time on investigations,” says Krasikov.

F) Environment setup – Finally, DataArt implemented infrastructure as code. The new development environment uses Puppet with Vagrant to check out source code, spin up a VM box / template, and deploy fast, according to Krasikov.

DataArt runs some steps in parallel for additional time savings, for example moving to virtualized provisioned environments along with source code migration, according to Krasikov.

DevOps Helps DataArt Speed Projects to Their Customers’ Delight

By increasingly converting backlogged apps to DevOps development approaches, the enterprise benefits by achieving project goals and delivering a working software iteration with each pass through the DevOps processes. “The high level of automation reduces the risk of human errors and helps us to fail early in the development cycle. With continuous delivery, it is much easier for our clients to see the immediate impact of their ideas,” says Krasikov.

May 6, 2015