devopsy.kinsta.cloud

Tag: applications

  • Connectivity as Code: Making Network Security DevOps-Friendly

    Connectivity as Code: Making Network Security DevOps-Friendly

    DevOps is all about agility, with fast, short delivery cycles and automation for software development and applications. Enabled by recently introduced technologies such as virtualization, cloud and software-defined networking (SDN), spinning up new servers, provisioning storage in a public or private cloud or even launching whole environments can take just minutes or even seconds. But if that new application, service or environment needs a change in network connectivity or firewall rules to enable it to work, then the pace of delivery can often slow to a crawl.

    Unlike the typical DevOps scenario, in which developers create a CI/CD pipeline that includes running build processes and tests and delivering the new application completely automatically, provisioning new network connectivity usually involves a slow, laborious process. The application developer needs to manually open a change request and then wait for approvals and implementation of the new connectivity flow before proceeding with the DevOps flow.

    The Application Development Security Disconnect

    As part of this, the developer is usually required to provide information about firewalls, zones, subnets and other information related to the underlying network infrastructure in these change requests—information that is not always known or clear to the application developer. On the other hand, the application requirements and context are not always understood by the network security team assigned to implement the changes. This can result in back-and-forth communication between the developer and network security, making the entire process extremely long, error-prone, inefficient and frustrating to both teams.

    The challenge here is building in the necessary checks and balances to ensure that DevOps processes are not rapidly introducing security problems, while equally ensuring that security is not stepping on the brakes and slowing down the organization’s agility. So, how can this be achieved?

    The Solution: Connectivity as Code

    The answer is connectivity as code, a methodology that supports not only automation and agility in the application delivery process but also bridges the gap between application developers and network security teams on an ongoing basis, even after the application is deployed into production.

    Connectivity as code creates an abstraction layer that translates between the two worlds, giving application developers more control over their applications while helping network security understand the business impact of their day-to-day tasks, and thus ensuring business continuity.

    Establishing Connectivity Requirements

    In the first stage of this approach, the application developer describes the application connectivity requirements in a simple machine-readable text file, listing all the logical flows that represent these requirements, including those for the different application environments (e.g. test, development, production, etc.). It is presented as a list of abstract flows that typically include the source (who initiates the connection), destination (who accepts the connection), service and additional information.

    There is no need to know where the servers are located or what the underlying network topology is—such as whether there is a firewall between them or cloud security controls—or even their IP addresses or subnets. Furthermore, there also is no need to know whether connectivity is already available, because it’s also required by other applications.

    The Connectivity Phase

    Once this list is created, a new phase in the DevOps process, the connectivity phase, will automatically take care of things with no further involvement required from the application developer. The connectivity phase is where the actual connectivity provisioning and validation happens, leveraging a network security policy management (NSPM) solution to automate the network connectivity provisioning as part of the DevOps process just like any other configuration or provisioning step in the DevOps pipeline.

    The NSPM solution will read the file before automatically checking whether the connectivity requirements have changed. If no changes were made, the solution will verify that the required connectivity remains in place, and that the application will work correctly. This provides the application developer with the assurance that the necessary connectivity is in place and connectivity-related failures are not anticipated when moving into production.

    Automating Connectivity Changes

    If the connectivity requirements have changed, the NSPM solution will use the connectivity file to provision the required connectivity while retaining it for future reference, so that the application’s connectivity remains intact in the event of future network architecture changes or application or server migrations.

    If one or more of the required flows is currently blocked, a security policy change request process is triggered. Through this process a NSPM will verify that the new flows comply with the organization’s preapproved security policy, as well as security best practices and industry regulations, and will then design and implement the required changes directly on the different security devices on the network, automatically and within minutes.

    If, however, a change request is non-compliant, it will be escalated for approval and, once approved, it will be implemented automatically, thereby saving the developer the need to manually open an out-of-band change request.

    Furthermore, whenever a new version of the application is developed, this list of connectivity requirements is updated by the developer and the connectivity phase will ensure that the new flows are provisioned. With the connectivity phase completed, the DevOps process can securely proceed onto the next phase.

    The Benefits of Connectivity as Code

    Introducing the connectivity-as-code methodology into the DevOps process achieves a range of benefits. These include:

    • Seamless management of network connectivity: This makes DevOps process far faster and more agile, and enables problem-free application delivery, rather than as an external out-of-band issue that requires separate—and manual—handling.
    • Continuous compliance and auditability: Throughout the application delivery process organizations will remain fully compliant with all relevant guidelines, while all changes are fully logged for auditing purposes.
    • Business continuity: Application connectivity requirements are clearly documented and up to date, ensuring minimal disruption to the business even during network, infrastructure or architecture changes.

    Ultimately, connectivity as code bridges the gap between application developers and network security throughout the entire application life cycle, from planning and development through to deployment, production and decommissioning.  It ensures both developer and security teams get what they want: Dev teams don’t have to worry about security slowing them down, and security teams know that risk and compliance checks have been built in to the continuous delivery process, complete with a full audit trail of changes. By weaving network connectivity into the DevOps process, organizations can ensure fast delivery without any compromise in security or compliance.

    — Anner Kushnir

  • DevOps Chat: Modern Apps in DevOps w/ Andrea Martinez Crawford, IBM

    DevOps Chat: Modern Apps in DevOps w/ Andrea Martinez Crawford, IBM

    Andrea Martinez, now Andrea Crawford, is the CTO for DevOps for IBM Hybrid Cloud. In this DevOps Chat we had a great discussion on what is “modern apps in DevOps,” how to upgrade your legacy apps, what to do if your legacy apps are not upgradeable and what this all means for DevOps.

    Great discussion and as usual, the streaming media audio of our conversation is immediately below, followed by a transcript of our conversation.

    Alan Shimel: Hey everyone, it’s Alan Shimel and you’re listening to another DevOps Chat. DevOps Chat today is going to be a good one. We have with us Andrea C. Martinez who is the CTO for DevOps for IBM’s hybrid cloud.

    Andrea, welcome to DevOps Chat.

    Andrea Martinez: Hey, thanks for having me, Alan.

    Shimel: Thank you. Did I get that all right?

    Martinez: So, I’m mostly known as Andrea C. Martinez but I’ve recently gotten married so my new last name is Andrea Crawford. Different last name but the same cowboy hat. So I always wear my cowboy hat in my profile pics. If you’ve got the hat you’ve got the right gal.

    Shimel: And it’s the right Andrea. Okay. No hyphen – Martinez-Crawford or something like that?

    Martinez: Nah, I’m not that fancy. It’s just Crawford.

    Shimel: I’ve got to tell you I just got back from China last week and I was with my partner Jayne Groll, one of the co-founders of the DevOps Institute, and one of my co-founders there, Jayne Groll, is actually now Jayne Groll-Ray because she’s also recently married and what a mess on the – hats off to you – getting your passports done and redone and what name to use because you know it has to match exactly and all your other IDs – I didn’t realize what a –

    Martinez: Alan, that’s why we get the diamonds in our ring.

    Shimel: I guess so. Yeah, and well-deserved. Anyway, congratulations on your recent marriage but we’re going to talk DevOps and we’re going to talk a little bit about modern applications here today, okay?

    Martinez: Awesome. Sounds good.

    Shimel: Great. All right. So I should mention that we have a webinar coming up on June 28 at 1 p.m. Eastern time on this topic and really this chat is just sort of a lead-in to the full hour-long webinar and that webinar is on DevOps for application modernization and we’ll put the link to it in our show notes here but if you find what we’re talking about today interesting by all means check out the webinar.

    But let’s dive in. So Andrea, let’s talk a little bit about modern applications versus non-modern applications: Do we have archaic applications, are they obsolete applications? What do we mean when we say a modern application?

    Martinez: Yeah, I think to – and by the way, I like your archaic reference there. I think to really appreciate the modern application we really have to understand how applications use to be designed and built not all that long ago.

    So, if I were to go back to the N-1 generation of applications I could make some pretty broad characterizations of how they were designed, built and delivered. So, if we were to like take it back to around the late-Clinton era of post-dot.com boom, world wide web and the internet, you know, that was really becoming the network platform where enterprises were exposing business function and really more of nascent digital economy.

    So some of the characteristics of what I’ll call “heritage applications” were, No. 1, they were monolithic so the applications lived on these things called “application servers,” typically running on maybe like higher end commodity infrastructure – think like X86, power hardware with large amounts of disk and memory, we had big centralized databases and these memory hog application servers.

    So the applications ran inside these app servers and they were fairly monolithic – that means they basically bundled the presentation or the view, the business logic and the data model components all together.

    The second characterization I would say for heritage applications is that they lived on these app servers that resided on these sort of heavyweight operating systems and although at the time Linux was very much on the scene overall. The trust in Linux was not yet solidified so you had these operating systems that were sort of these enterprise, server, big and bold editions of this OS – and it was pretty prevalent, you know? Windows and proprietary Unix based operating systems pretty much ruled the day.

    And, No. 3, the – in terms of the infrastructure on-premises data centers ruled the roost. I mean virtualization technology’s around, we had this notion of logical partitions, L-pars, VMware, virtualization, but it was mostly a hardware OS story, okay?

    And then we had, in terms of looking at the actual heritage applications themselves, we had the rise and dominance of function-rich languages like Java and dot-net because the application server middleware pretty much dictated the language of what an application was coded in, right? So we still have a lot of these skills around today.

    And I think the last characterization of a heritage app was in the way that they were delivered. So we had the waterfall delivery – very sequential, big-bang releases with siloed organizations, dev teams didn’t really talk to the operations teams and having that kind of silo was pretty much the norm.

    So, that’s how I would sort of characterize the heritage applications.

    Shimel: Yeah, dead on for me Andrea. We didn’t talk about this but in my background I had started a web hosting company in those dot-com days and I sold it to a company and started to work with them and we were one of the first big, what they called ASPs, if you remember those days – application service provider – and so this is – we wound up doing the whole IPO and all of that – this is 1997, ’98, ’99, 2000, into 2001 –

    Martinez: Absolutely. Yep.

    Shimel: Oh, they were glory days. We were probably one of the largest Lotus notes, if you – you know,

    Martinez: Yeah, all right.

    Shimel: So, picture this: We’re hosting – we are offering hosted Lotus notes as a service – it’s not even IBM WebSphere yet, I think it was still notes server –

    Martinez: Domino?

    Shimel: No, it wasn’t even Domino. What came before Domino? I mean this is old. And then Domino came out and it kind of rocked our world because it was really a big thing for us. But there’s no – it was on dedicated hardware, there was no cloud, T1 lines were about $1,200 a month, and so you had to deliver it over a T1 line oftentimes – you know, a dedicated T1 line point to point – and then we expanded to PeopleSoft and Oracle apps and stuff like that, and what a disaster. I’ve got to be honest. I mean we went through about $500 million – a half-a-billion dollars in monies raised between the IPO and so forth – but it was hard. It was really, really hard.

    And then we tried to get into the whole custom app kind of hosting with ISVs and so forth.

    Martinez: Oh boy.

    Shimel: So, what you just described I had a déjà vu to my bones and, yeah, I wouldn’t call those modern applications but I think we’ve done a good job.

    Now contrast that, though, with what modern applications are like today, Andrea.

    Martinez: Yeah. So there’s sort of been an inflection point where some pretty emergent things have happened since those days, right? And you know we dealt with all of those issues around the heritage environments and applications. And now all of the sudden if we take a look at what I call “modern applications” today – Alan, you mentioned the cloud – No. 1, modern applications live in the cloud. So these applications leverage the elastic, infrastructure and platform that comes with it.

    When we develop modern applications – you also may have heard them referred to as “cloud-native applications” or “born-on-the-cloud applications” – and really what has happened here is to leverage that type of virtualized infrastructure we’ve started to breakdown the monolith and the applications that run on them.

    So, now we’re taking our bundled model-view controller applications and frameworks and we’re deploying them separately as components and hopefully we’re using architectural styles like microservices so that the monolith becomes broken down into choreographed microservices that can be managed separately, okay?

    So, we have technologies that actually support this today. And so Docker, Kubernetes are some of the technologies that have come on the scene since that have allowed us to have a control plane and an environment to manage smaller-granularity applications which are hopefully services.

    So, the other thing about modern applications that I think is interesting and has fostered the adoption of these microservices and smaller components is the fact that the solution stack has become more and more commoditized, right? And so, when you think about the solution stack of network hardware, operating system, middleware layer and so on, we’re seeing a commoditization of the solutions stack from the bottom up, okay? And that is – the whole notion of cloud computing, right? – We’re no longer concerned so much about CPUs and CPU architecture, you know, X86 versus Power P Series, this or that – it sort of doesn’t matter as much anymore, right?

    And because of that, I think that has also fostered in some cases where critical proprietary software was not necessarily needed for enterprise applications. And so, when you start to break down the monolith and deploy smaller services, you’re isolating your risk because you’re isolating business functions and smaller chunks that are lighter-weight and it’s really analogous to the difference between deploying a 10-ton gorilla and deploying a colony of ants. If one ant dies it’s not a big deal, right?

    But commoditization has even spread to the way that these little ants or microservices talk to each other with the advent of API interfaces and as long as you use rest APIs over HTTP or a lightweight messaging protocol that’s pretty much the lingua franca of modern applications today.

    So, I think there’s probably this notion of componentization and that is actually fed into the notion of polyglot programming, and so enabling modern apps to be programmed in just about any language. So, you’re no longer bounded to this big, proprietary application server; you can program in just about any language, and as long as you can talk – rest over HTTP or send an A-synch message over a lightweight message hub – it sort of doesn’t matter.

    You know, you can be a Java or a dot-net bigot but, hey, the new guy who knows Python and is really good at it, he can get in on the action, too. It’s all good.

    Shimel: Absolutely. So, Andrea a lot of what you – you know, it was more than a mouthful there – and a lot of what you’re talking about really describes DevOps, right? This idea of chunking down stuff so it’s not the monolith, right, and doing small – you know, lots of smaller, little things rather than one large, bigger thing and learning and feedback looping and all of that – I mean this is – it’s DevOps 101, right?

    And to me – and this is – you know, I speak to a lot of people obviously around the world about this stuff – but to me there’s a chicken-and-egg relationship between DevOps and modern application infrastructure, development and deployment as you’ve described. I don’t know if you could have one without the other. I just don’t know which one comes first. But they’re intrinsically linked, right?

    And this is why, in my mind anyway, we need – you know, there’s a special relationship, if you will, between DevOps and modern application and how we do modern applications today.

    Martinez: Spot on, Alan. I mean, when you think about the gorilla versus colony of ants analogy. Think about having to deploy the 10-ton gorilla – it’s typically pretty complicated but there’s a lot of dependencies and release management is of utmost concern – but deploying the colony of ants you’re deploying a lot of little things at different times – so you’re achieving the multispeed IT – if you don’t have DevOps, if you don’t have that kind of automation to do the build, compile, package, test, deploy and you’re doing it intra daily – if you don’t have that automated then your cost case is going to fly out the window.

    DevOps is essential for modern applications. Period.

    Shimel: Dead on. Dead on. Amen. You know, you’re preaching to the choir here, Andrea – we’re devopsy.kinsta.cloud – but it really is essential and there is this sort of hand-in-glove type of relationship.

    We are, as I promised you when we started – the time goes so quickly and there’s more I want to cover. So we talk about how DevOps fits into the story on the how and why of modern applications.

    You know IBM has this history now – and as I mentioned to you four or four-and-a-half years ago when I launched devopsy.kinsta.cloud IBM was one of the leading advocates for what was kind of the rise of the DevOps community and culture. And then you guys had the garage method, which was really kind of intrinsically again all about DevOps and DevOps ways, if you will – how does all of that fit in to modernizing applications and the way we’re doing it now?

    Martinez: So the IBM cloud garage method is IBM’s opinionated view on how to deliver especially modern applications: It combines Agile, DevOps, Lean and our enterprise-design thinking and sort of rolls it all up into this wonderful burrito of reference architectures, practices and techniques – you know, it’s lunchtime, so I’ve got food on the mind – but it’s basically a way to help our clients act like a startup so that they can achieve the kind of agility that they need.

    So, DevOps is also about organizational and behavioral changes because that’s really paramount. At the heart of our method is culture – how to organize our teams, the collaborative practices, social coding, pair programming – breaking down the silos, right – that’s what DevOps is all about, having product owners sit with the squads, right? – sit with them. You know, DevOps is a behavior story – a tool automation story – and it’s all in the name of enabling business agility.

    Shimel: Absolutely. Andrea, I’ve got another final area I want to click on if you don’t mind. And that is for our folks listening out here – and it’s a worldwide audience at devopsy.kinsta.cloud – and I’ve traveled the world doing this, and though we all want to do DevOps we move at different speeds and we’re at different stages of our journey.

    What advice, if we can kind of give blanket advice – what advice around DevOps would you give to our listeners who – and again, Andrea, this is something you’ve probably learned, right? – I wish we all had a greenfield to play in, but we don’t, right? We have brownfields and there’s heritage apps, as you call them, and there’s modern apps, there’s old and new, fat and skinny, ants and gorillas – even burritos – what advice can you give our listeners on how to deal with all of these things?

    Martinez: I think just recognizing that you may have to approach DevOps a little bit differently depending upon whether you’re modern or heritage or even somewhere in-between. Typically DevOps for heritage applications, you’re retrofitting automation in situ, right? So, you’re sort of bringing the automation to the app. Whereas for modern applications, you definitely want to have a DevOps Day One approach, right?

    Because the ability to create a cloud-native application and have the automation in your mind right on day one is critical. But just know that you may have instances where you may not be able to bring a heritage application all the way to modern – you might be somewhere in between Fred Flintstone and George Jetson, right?

    Shimel: Two of my favorites.

    Martinez: But that’s okay, right? I mean that’s real-world. Because it may not be cost-effective to automate everything but that’s all right, you know?

    So I think we can engineer our way through some of this – and there are products out there – you know, you mentioned before that hybrid cloud was really sort of the area that I was focused in terms of DevOps – and we have proprietary products, open source products, we have our Urban Code Deploy, our IBM Cloud Private – that really sort of helps bridge the gap between applications living in this cloud or that cloud – whether they’re modern or heritage there is an answer.

    It’s not cookie-cutter, but we have the piece parts and the emerging technologies on the scene to help navigate.

    Shimel: Yep. Excellent. So Andrea that is important for people to know but where do we go – where can people go to find out more about this?

    Martinez: So I think the first place I would point them to is the IBM Cloud Garage method website – and I’m sure we’ll be providing links as part of the podcast references – but at that site you can definitely find reference architectures for DevOps, reference architectures for microservices in cloud-native development – and one of my favorites – the practices and techniques that we employ with our garage method for automation, tooling and social collaboration throughout the software delivery life cycle.

    Shimel: Perfect. So, Andrea, as I mentioned, the time here it goes so quick. It was delightful, refreshing and fantastic having you as a guest here. I’d love to have you – you know, where else are we going to learn about gorillas, ants and everything else? I need to have you back on the show. We’ll talk off-mic about it, but if it’s not too much – and we’re Florida neighbors – so I’d love to have you back here. But we’re going to wrap it up for this episode of DevOps Chat.

    Andrea Martinez-Crawford, welcome and thank you for being our guest on DevOps Chat today. We will have links to a lot of what you’re speaking about in our show notes, so folks, please take a look at that. But for now, we’re going to call it a wrap on this episode of DevOps Chat.

    You’ve just listened to another DevOps Chat and hope to see you soon everyone. Have a great day. Bye-bye.

    — Alan Shimel

  • AppDynamics Taps into SAP Applications

    AppDynamics Taps into SAP Applications

    Application performance management (APM) is starting to be  employed more widely as more organizations embrace digital business transformation. As the number of digital business processes spanning multiple applications increases, so does the need to understand what’s occurring inside both greenfield and legacy applications. In fact, at the heart of many of those digital business processes are enterprise resource planning (ERP) and customer relationship management (CRM) applications from SAP. To make it possible to gain insights into those applications AppDynamics, a unit of Cisco Systems, announced today it is making available AppDynamics for SAP.

    Jonah Kowall, vice president of market development and insights at AppDynamics, said his company is now able to support ABAP, the proprietary language SAP developed to build its applications. That language also has been employed extensively by IT organizations to customize those applications.

    SAP currently is trying to transition its customers to building custom applications on a cloud framework based on the Cloud Foundry platform-as-a-service (PaaS) environment. Most of those applications are be written in Java or the new Jakarta platform that replaces Java. SAP has also provided a software development kit and application programming interfaces (APIs) for tapping into S/4 HANA ERP applications.

    But there is a massive installed based of ABAP applications running in enterprise IT environments. Much of that code is now being integrated with other applications as part of a larger digital business transformation initiative.

    Kowall said those extensions are now driving organizations to apply an agent for the AppDynamics APM to custom applications written in ABAP. Historically, enterprise IT organizations only applied APM to their most critical applications. But as organizations move to digitize business processes, they now need to able to track transactions spanning multiple applications. Given the reliance many organizations have on SAP applications, it’s inevitable those transactions will involve SAP applications that were customized using ABAP.

    In addition, Kowall noted that being able to discover precisely where those custom applications are involved in any transaction is pivotal when it comes to upgrading to the SAP S/4 HANA applications, which are based on the SAP in-memory database, or lifting and shifting a legacy SAP application into the cloud.

    Kowall said organizations that have used ABAP to customize SAP applications now will have visibility down to the code level, including queries being made to the underlying database. From a DevOps perspective, that capability will go a long way to breaking down yet another silo in the enterprise. AppDynamics expects the first order of business for many organizations will be simply to create a performance baseline for their ABAP applications, which in many cases are likely to be an eye-opening experience, he said.

    The focus on digital business transformation is likely to spur adoption of DevOps processes over a much broader swath of the enterprise. Rather than simply apply DevOps processes to new applications, many organizations soon will extend DevOps into the realm of legacy application environments as well. That may take some time to play out, given the scope of the cultural changes involved. But at this point, those changes are now all but inevitable.

    — Mike Vizard

  • Eclipse Foundation Charts Cloud-Native Path Forward for Jakarta

    Eclipse Foundation Charts Cloud-Native Path Forward for Jakarta

    A new era begins for the Java community today with a commitment to a Jakarta EE specification that the Eclipse Foundation expects will become cloud-native in short order.

    Jakarta EE formally marks the end of a Java era. While the term Java will still apply to any technologies developed prior to Oracle transferring stewardship of the venerable programming language to the Eclipse Foundation, all future innovations will be delivered under the Jakarta brand.

    A survey of 1,800 Java developers shows that in addition to increasing the overall pace of innovation, the top two priorities of the developer community is to see more emphasis being placed on microservices and integration with the emerging de facto Kubernetes container orchestration platform.

    A full 95 percent of the respondents building microservices said they are employing Java as part of those efforts, while nearly one-third of respondents are already working with Kubernetes. Mike Milinkovich, executive director of the Eclipse Foundation, said that although it’s yet to be determined precisely how Java and Kubernetes will be more tightly coupled, it’s clear the Jakarta community wants to be able to deploy cloud-native applications based on Jakarta. Half of survey respondents said they ran only one-fifth of their Java applications today in a cloud. But more than 30 percent said they expect to be running 60 percent of more of their applications in the cloud within the next two years.

    Milinkovich said with Jakarta under the auspices of the Eclipse Foundation, many more developers from different vendors will be contributing code to advance it. Previous advances to Java have been slow in coming over the years because all contributions to the code base were controlled by Oracle. One of the first things to change under the stewardship of the Eclipse Foundation, he noted, is that Oracle has committed to open-sourcing the Test Compatibility Kits (TCKs), which vendors previously were required to license at significant expense.

    Also expected to witness a faster innovation cycle, Milinkovich said, is Eclipse MicroProfile, a Java framework developed by IBM and Red Hat optimized for microservices, which the Eclipse Foundation expects to be as widely embraced as the Spring framework based on Java that Pivotal Software helped popularize. In fact, there will be a lot more cross-pollination of ideas and concepts across all the major frameworks from writing applications, he added.

    Tighter integration between Jakarta and Kubernetes and a general simplification of the effort required to develop microservices is likely to have a significant impact on DevOps for years to come. Many organizations today are contemplating the degree to which they may want to lift-and-shift existing Java application into the cloud versus rewriting them completely in a new language. With the Eclipse Foundation’s commitment to a cloud-native implementation of Jakarta EE, it’s clear that organizations can have greater confidence that existing investments in Java applications likely will remain relevant in the age of the cloud. In fact, the only decision left for many of them to make is the degree to which they might want to turn monolithic applications written in Java into highly portable microservices based on Jakarta EE.

    — Mike Vizard

  • A Unified Route to Bring Mainframes to DevOps Speed

    A Unified Route to Bring Mainframes to DevOps Speed

    Most businesses have mainframes where their key applications and data reside. In today’s digital environment, customer-facing mobile and web systems retrieve critical data from back-end mainframe systems. It is critical to adapt mainframe applications and data to today’s rapidly changing market demands if businesses want to remain competitive.

    This is the question IT and businesses are facing today. How can they integrate their well-established mainframe environments and have a unified cross-platform across the enterprise for DevOps to manage releases across mainframe and non-mainframe applications?

    An Innovative Digital Solution

    The answer lies in mainframe digital DevOps. This helps enterprises remain competitive in an app-centric economy. Through a unified cross platform, DevOps is integrating mainframe tools with open source DevOps tools.

    This approach allows organizations to retain their legacy code and does not require legacy code migration—especially when the back-end core applications are on mainframe. As veteran COBOL programmers retire out of the skill orbit, this becomes even more important. There is an imperative and an expectation for agility and higher productivity from IT—without jeopardizing performance, customer experience, security and other compliance factors.

    Transforming the Developer Environment

    Most mainframe development, test and ops work is executed in antique “green screen” SPF environments, which calls for specialist knowledge that limit productivity among programmers of Agile and DevOps transformation. Turnaround times of monolithic mainframe applications are painfully slow compared with the new-age agile technology of mobile and web applications. This can be frustrating for developers who wish to make frequent changes to mainframe code to fulfill the urgent needs of business.

    Therefore, it’s important to make the shift to modern tools and integrate mainframe with non-mainframe DevOps processes to deliver cross-platform services. This will improve staff productivity across development, testing and production operations and enable inexperienced mainframe professionals to perform application development and enhancements with ease and efficiency.

    Large mainframe clients embark on transformation by modernizing the workspace to work on both mainframe and non-mainframe applications. Open system developers are familiar with new-age IDEs such as Eclipse, which have user-friendly interfaces. Similarly, other mainframe tools are available that provide source code editing, debugging, fault diagnosis and browse data to enable developers to write, compile and run code from a modern Eclipse-based IDE. Developers can drag and drop files between LPARs and perform other tasks to improve productivity metrics. Collaboration between the two teams will motivate mainframe developers to use DevOps principles to integrate mainframe with distributed and cloud applications.

    Once quick wins are established, adoption levels of Agile and DevOps will improve among mainframe developers and business.

    Eliminating Barriers to Unit Testing

    Historically, mainframe developers have not been accustomed to unit testing. Automated unit testing is common in Java but not in the mainframe environment. Agile coaches, familiar with mainframe tools, will need to enable collaboration in interactively testing smaller pieces of code.

    Unit testing can improve throughput by increasing the frequent code drops in production environment and help to accomplish DevOps key performance indicators (KPIs) with fewer errors in the later stages of the software development lifecycle process. Some tools can facilitate this process by automatically breaking the mainframe code into discrete logical units, and creating tests for such logical units.

    Tracking and Monitoring Progress for High Quality

    Visualization is an important element of DevOps for improved visibility to track and monitor progress across the enterprise. As most mainframe applications and data structures are large, complex and not well-documented, enterprise IT is highly dependent on the personal and institutional knowledge of senior mainframe staff. Graphical visualization can help quickly “read” existing application logic, program interdependencies, data structures and data relationships. Business rule extraction tools are available in the market to reveal logic and data relationships through graphically intuitive diagrams. This helps developers work independently on unfamiliar programs to better estimate project work, meet delivery deadlines and commit timelines to business.

    Most mainframe applications are often associated with core business processes with zero tolerance for error. Any attempt with inadequate mainframe experience can lead to catastrophic non-recoverable error. Continuous integration (CI) is critical in determining application quality issues early and establishing KPIs to deliver continuous quality code in production. Enterprises have started embracing modern tools and techniques to transform from the Waterfall delivery model to Agile. Modern DevOps tools embrace CI, and certain tools can quickly detect application quality issues (such as unbalanced or unmatched working storage and data items, and out-of-range nested conditions and branches).

    Training for Effective Enablement

    Successful implementation of cross-platform DevOps calls for the right training on Agile tools, processes and methodologies. A single Scrum team can be formed to collaborate with developers across mobile, web and mainframe. The Agile mainframe team will need to focus on stories and epics that capture specific units of business value instead of technical tasks in a project plan. Open system tools such as JIRA and Confluence that can support Scrum and Kanban can be used to manage tasks and improve collaboration through centralized and well-organized web tools. This will incorporate Agile values such as transparency, knowledge-sharing and ideation to improve the developer’s performance.

    The move to Agile scrumming represents a transformative shift of mainframe work culture, and herein lies the importance of the right coaching in Agile processes. A “2- n a box” partnering model of mainframe developers and experienced Agile developers from other platforms can create the required collaboration.

    Operational Data Analysis to Close the ‘Last Mile’ Loop

    Good quality code alone is not enough. Operational data analysis is required to reduce costs by discovering and mitigating avoidable CPU consumption caused by code inefficiency.

    Mere monitoring and reporting by triage teams is not enough to support and maintain applications in the world of DevOps.  Production data analyses by enabling an auto-feedback loop is imperative to improve productivity of the dev, test and production teams. Tools are available that can quickly pinpoint bad statements and pinpoint application inefficiencies such as bad SQL statements. Tools also help identify the causes of excessive CPU consumption and slow data retrieval.

    Automation for Speedy Efficiency

    Agile-enabled core source code management (SCM) is a fundamental requirement of DevOps. Traditional SCM cannot support parallel development work on different user stories. SCM migration is therefore essential to increase the speed and frequency of mainframe code drops. Automating the change management process and defining the proper workflows can eliminate manual steps and reduce code approval delays.

    Again, tools are available that can fulfill business expectations by seamless SCM migration without disrupting the current work in progress.

    Automated release and deployment is pivotal to keeping pace with today’s fast-moving markets. Business needs to quickly see reliable new code in production.

    Toward a Unified IT Service Management

    Cross-platform continuous delivery is vital for efficient use of mainframe applications and their data for multiplatform customer-facing applications, including mobile, web and cloud components. Mainframe DevOps process should fully synchronize the code delivery process across all platforms. This will establish a centralized deployment control center to provide a unified cross-platform and accurately report progress. Tools can be the single point of control for all changes across z/OS, Windows, Unix and other platforms. REST APIs are flexible to mix and match best-in-class tools and avoid vendor lock-in.

    The ultimate destination is to create a de-siloed environment where mainframe is just another platform with high scalability, reliability, performance, security and cost-efficiency. A unified mainframe digital DevOps platform may be the answer to modifying traditional process that fulfills this business demand.

    — Ramesh Ganapathy

  • Testing’s Role in Delivering Digital Transformation

    Testing’s Role in Delivering Digital Transformation

    “We have a quality philosophy now…but it took the rise of the middle child to make it happen.”

    • Ann Lewis, ExxonMobil

    Ann Lewis’ metaphor of testers being the “middle child” between software development and IT operations was present throughout Accelerate 2017, Tricentis’ annual user conference. Nearly every head in the room nodded as Lewis described how testers, like middle children, tend to get less attention than their professional or familial siblings, but she insisted that there’s never been a better opportunity for testers to receive the respect they deserve. Digital transformation, no matter how you define it, requires new commitments to quality, and it’s up to testers to claim be the change agents in the transformations that CIOs are tasked with delivering.

    Tricentis CEO Sandeep Johri and other CxOs who spoke at Accelerate echoed this sentiment. Johri pointed out that speed continues to be a primary focus of maintaining or gaining competitive advantage, and manual testing processes that prevent agility or DevOps success at the enterprise level must be automated going forward.

    What was also made clear is that scaling investments in automation is not the single key to success. The pressure CIOs are under today, especially around digital transformation, requires new investments in a number of different areas, many of which are only just being realized. While CIOs may understand the necessity of quality, it’s up to testers to get the buy-in required for the continuous testing required to deliver it at scale.

    Testers with an oft-ignored middle child mindset may find the concept of being recognized as a “champion” by their CIO or CEO unlikely. But, Todd Pierce’s impassioned session on “What CxOs really think about testing” was designed to inspire testers to change the way testing was viewed too often in the past.

    As a current Rock Health and Dignity Health board member, and former Bill & Melinda Gates Foundation chief digital officer, Pierce shared his peers’ all-too-common beliefs that testing takes too long, costs too much and constrains developer productivity. And, while it may hurt to admit, with manual testing that should be automated and insufficient testing time that results in bugs making it to production, it’s not surprising that CIOs would feel this way.

    Pierce explained that testers who adopt innovative, scriptless “high-precision testing,” are able to shorten their test cycle, reduce defects that make it to production and eliminate costly rework for themselves and developers alike. When high-precision testing is performed continuously throughout the software development life cycle (SDLC), those beliefs that testing takes too long, costs too much and impedes innovation lose all validity. “Everyone in this room should be the drivers of agility in the enterprise,” remarked Pierce.

    Emmet Keeffe at Insight Venture Partners hosted a CIO roundtable later in the day where four CIOs detailed their personal experiences with taking enterprises through digital transformations. Each CIO admitted that in the past they had underestimated the the time being spent on testing, and one somewhat bravely stated that he’d always found testing somewhat “boring.” One might expect a room of testers to storm the stage with charges of heresy, but, given that many testers have suspected that management felt this way for years, I got the feeling that many were simply impressed that someone who felt that way had admitted it.

    But the panel, Pierce, Lewis and countless other speakers at Accelerate also made sure that the attendees of their sessions understood one thing: As more organizations undergo digital transformation—and no matter what stage of digital transformation they’re in—testing has never been in a better position to drive positive changes that will have monumental impacts on the business.

    Whether CIOs have been tasked with delivering greater agility, DevOps shifts, increased security, enhanced user experience, reliability and performance or legacy app modernization, testing may not be the first organization that CIOs turn to for help. But, as Keeffe explained, testing leaders who truly understand the incredible opportunity they have to make those CIOs heroes will make themselves heroes in the process.

    About the Author / Noel Wurst

    Noel Wurst leads influencer relations for Tricentis, and collects stories around cultural shifts and evolving technology’s impact on the software testing industry. By attending global conferences, interviewing passionate software professionals, and working closely with product, engineering and customer success teams, Noel looks for stories that matter and innovative ways to share them with audiences that lead to business growth. You can follow Noel on Twitter and LinkedIn.

  • The Need for Speed Demands New-Era APM

    The Need for Speed Demands New-Era APM

    Applications have become the primary conduit by which customers experience a brand, making the application experience critical to business success. As such, continuous delivery and highly available systems that deliver optimal end user experiences at lightning speed have become top priorities for organizations. Modern IT is all about speed, efficiency and customer experience. Simply put, companies that develop new applications and services the fastest will win.

    That’s where containers and microservices come in. Containerized microservice applications enable organizations of all sizes to approach the ultimate goal of continuous integration and continuous delivery. This accelerates everything within the application life cycle, from initial development to component updates and new functionality.

    To match the development speed that microservices and containers enable, Ops teams need tools that can also operate hyper-fast, providing an immediate understanding of the impact of changes made, real-time analysis of vast amounts of data across applications and technologies, visibility across a range of languages and infrastructures, and knowledge of how the application is performing for end users.

    In fact, we are at a critical inflection point in application performance management (APM), where a generational shift is needed to keep pace with the new hyper-dynamic application architectures consisting of microservices, containers and hybrid cloud environments. To fully understand the impact of this shift, we can examine the genealogy of APM.

    APM Evolution

    The first generation of APM, established at the end of the last century, collected statistics and metrics out of system and application components, providing a first generation of user-defined transaction profiling by using instrumentation technology. APM 1.0 required a lot of effort to implement and maintain, but helped to monitor and analyze the performance of newly developed 3-tier applications.

    The second generation of APM was driven by more distributed, service-oriented architectures (SOA) and the rise of more elastic, virtualized or cloud-based infrastructure. APM 2.0 added application topology discovery and could analyze distributed applications. User experience monitoring of web and mobile apps quickly became part of APM 2.0 as more businesses depended on the service quality of their applications. With a primary focus on application monitoring, not management, APM 2.0 required an even broader set of subject matter experts to configure and interpret monitoring data from dashboards and reports. Similar to APM 1.0, the second generation still required way too much human analysis and setup to fully understand application problems and derive actions.

    The continuous change in today’s modern dynamic applications makes it difficult, if not impossible, for those older tools to keep up. Throw in additional architectural challenges such as hybrid clouds, containers and orchestration (such as Kubernetes and Mesosphere) and the picture gets murkier, all in the name of high-speed delivery. Meanwhile, the microservice architecture and development strategy spawns wide technology diversity—each engineer can choose the programing language, middleware and databases best suited to code their individual microservice. Scale, complexity and constant change is the new reality for today’s dynamic applications.

    Wanted: A New Breed of APM

    In this environment, traditional IT monitoring tools won’t work. These new applications use a vastly larger set of technologies, must scale dynamically and are in a constant state of flux.

    In this effort to engineer for speed and efficiency, complexity and scalability have increased exponentially, requiring continuous monitoring and more visibility than ever before. As organizations embrace agile methodologies for development and containerized microservices for deployment, developers have shifted their thinking in technology choices. Developers now continually select the most optimal technology for their individual goals regardless of DevOps’s ability to configure, deploy and manage the specific technology.

    With several teams across the enterprise constantly publishing new software releases, understanding the impact of these new releases and changes to the overall environment has become a significant challenge. Today there is no single source of truth assisting communication between various organizations, resulting in what seems a wild west–like environment for IT. With so many groups and technologies in play, everybody seems to be speaking a different language, especially when it comes to delivering high-quality applications. In this environment, a new breed of APM  is needed that can provide a “Truth Machine”—complete with deep visibility across all application components in the technology stack.

    This new breed of APM must be automatic and dynamic to deal with the ever-changing world of applications running in containerized microservices. To deliver these critical capabilities, the tool must provide real-time visibility into dynamic, containerized, microservice applications, regardless of language or infrastructure. New thinking is required to meet the challenges of these complex, large and dynamic application environments; a new monitoring and management approach that can handle dynamism by:

    • Delivering real-time analysis of vast amounts of data across all applications and technologies.
    • Conducting continuous discovery and automatically mapping every component, host and container, (hosted on-prem, private cloud, public cloud), to the exact architecture of complete technology stack.
    • Visualizing the application architecture and delivering actionable suggestions that assure the operational health of systems and applications.
    • Providing real-time accuracy to visualize, structure and identify problems immediately.
    • Monitoring the complete set of diverse technologies that run in microservice environments.
    • Operating in a heterogeneous environment and automating deployment across nearly all programming languages.

    Artificial Intelligence for Next-Generation APM

    DevOps and IT organizations are struggling to match the speed of agile development, and this is creating exponentially more challenges around new application architectures. The only constant in these modern application environments is change. Whether change is due to microservice architectures, container orchestration or simply more agile delivery execution to effectively manage microservice applications, DevOps needs up-to-date visibility, a precise picture of the application’s structure and an immediate understanding of its health. To accomplish that, DevOps must be able to apply artificial intelligence to the act of monitoring modern dynamic applications.

    Rather than spending time configuring and reconfiguring monitoring tools, organizations must automate management steps with as little human setup and configuration as possible. To have complete control and visibility of their environments, regardless of language and infrastructure, organizations must use a modern application management solution that is automatic and AI-powered to deal with the ever-changing world of dynamic containerized microservice applications.

    About the Author / Pete Abrams

    Pete Abrams is a co-founder and the chief operating officer at Instana, the leader in application performance management for containerized microservices applications. Formerly vice president of customer centric innovation at AppDynamics, Pete has spent more than two decades in the technology industry focused on introducing new products to market and ensuring customer feedback and desired capabilities are incorporated into future product innovation. Connect with him on LinkedIn.

     

  • Building a Services and IoT Platform with DevOps, Part 1

    Building a Services and IoT Platform with DevOps, Part 1

    With the growth of the internet of things (IoT), expected to connect more than 50 billion devices (things) in next three years, building intelligent systems in a more connected world is presenting many challenges. Especially in building software for IoT, beyond hardware and network connectivity challenges there are numerous challenges in development, test, release, managing the infrastructure, monitoring the infrastructure and devices (things) and security.

    This article focuses on key components in IoT platform, how to select right set of DevOps tools for building those components, and creating a DevOps pipeline for the IoT platform.

    Building Your Own Services and IoT Platform with DevOps

    The main components in IoT solution comprises of device registration and identity, data ingestion, command and control, rules and actions for hot data analytics and cold data analytics, data storage, alerts and reporting.

    Device Registration and Identity: The devices/edge need to connect to each other and to a cloud gateway to send/receive the data, for which the devices have to be uniquely identified and registered at the gateway so that only authenticated and authorized devices/users can communicate to the cloud. Every IoT platform vendors has its own way of device registration and identity methods for storing device metadata such as AWS Things Shadow and AWS Gateway Rules Engine, Device twin and Azure IOT hub in Azure, GE Predix, etc.

    Data Ingestion: The devices communicate to the cloud/field gateway, which further ingests the data (device info and telemetry info, such as temperature data from sensors) into the cloud. You can use multiple options of PAAS services from Azure/AWS/GE predix, etc. for this component.

    Command and Control: These components cause the device to take an action and send messages to a device from the cloud, which can be built with PaaS services such as AWS Kinesis, Lambda functions, Azure functions, Azure Logic Apps, etc. or open-source technologies.

    Rules and Actions: These components act on specific device-to-cloud data, using the rules defined for processing hot data (where the data needs to be queried and analyzed within specified time interval) and cold data (where the data can be stored for delayed querying and further analysis). These components can also be built with existing PaaS services or using open-source tools such as Apache Storm, Map Reduce, Hive, Data Lake, etc. for data streaming and analytics.

    Data Storage: You need to store multiple types of data ranging from device metadata, telemetry data from specified set of devices, hot data and cold raw data, which needs to be stored and scaled up based on demand.

    Alerts/Notifications: The analyzed data needs to be visualized to derive more useful business metrics and outcomes, which can be integrated to external systems or other business applications to take preventive and predictive actions. Customized dashboards and data monitoring tools needs to be integrated to the solution to enable monitoring, alerts and notifications.

    With increased complexity in IoT development, the large number of devices to be monitored and large volumes of data to be analyzed and to manage complex infrastructure, DevOps becomes essential to the IoT development life cycle.

    Selecting the Right DevOps Tools for an IoT Platform

    Selecting the right DevOps tools for an IoT platform can help overcome challenges in creating/pushing applications and firmware updates to large scale of devices, as well as monitoring and infrastructure provisioning on-demand.

    With PaaS services used for IoT development, there is less code development but more configurations and rules-based development for developing key IoT components.

    Also, the teams developing device software and application development may be distributed, but the releases need to be streamlined with complete traceability and auditability. You need to manage a scalable infrastructure to support an IoT platform.

    Given below are some of the DevOps Tools used in most of IoT developments to help DevOps teams develop, build and release an IoT platform seamlessly.

    • Version Control: Git, for distributed teams and maintaining repositories for device development, application development and infrastructure automation development. Other tools are Clear case, SVN or TFVC.
    • Continuous Integration: Jenkins, VSTS (Visual Studio Team Services) and Bamboo to create jobs to do build, unit test and code quality analysis for the application, device software and infrastructure scripts, with the support of large collection of tools plugins and usage of different build tools such as gradle, maven, msbuild, etc., as well as supporting multiple technology stacks.
    • Infrastructure Provisioning and Automation: Ansible, ARM (Azure Resource Manager) and AWS cloud formation templates to automate the infrastructure in multiple cloud vendors and use tools such as Terraform or Spinnaker to automate the provisioning of infrastructure across hybrid clouds and maintain a common infrastructure blueprint, which enables the porting/migration of infrastructure workloads across clouds.
    • Infrastructure Testing: When PaaS services are used for developing certain components in an IoT platform, testing those PaaS components is a bit different—PaaS services have less code but more configurations and endpoints from dev/test team perspectives. Testing these components will involve endpoint testing, whether rules are created properly, how data flows between the different components and robust infrastructure verification. These infrastructure testing scripts can be written using Python/groovy or Vagrant to simulate the environment and test, or by using testing frameworks such as Pester for PowerShell scripts. You can extend your test solutions to leverage the REST APIs/CLI commands, exposed by different PaaS services, and these test scripts can test the PaaS services too, with simulated test data and generate the test result reports.
    • Continuous Deployment: VSTS Release manager, Jenkins, Capistrano, CA Nolio, etc. can be used to create the deployment workflows for deploying device firmware updates, application updates separately with control to release on multiple environments, gated checks and streamlined deployment process, with feedback enabled.
    • Test Automation: Selenium, Jmeter, security testing tools such as Veracode, EMMA, CA Lisa, Inspec, Robot framework and performance test tools are used for test automation. Testing in IiT is complex, as it needs an end-to-end testing of multiple devices interconnected. Most of the other device development may not be in your control, except your own device development, so to test an end-to-end functionality you need to have robust test cases and simulated test devices and test environments. This is where containers can help in test automation—a simulated device as a Docker container can run in one test environment and testers can use service virtualization tools such as IBM Rational Test Virtualization Server, CA Service Virtualization, Parasoft Virtualize or HP Service Virtualization to test their end-to end functionality when other dependent components/devices are being developed or not available.
    • Security and Code Quality Analysis and Security Testing is key in IoT development. As more and more devices are interconnected, the risk of data breach/data privacy is high; hence, a shift left of security testing during the development phase as part of the CI pipeline is inevitable. Tools such as Veracode, EMMA and Inspec can be used to run security analysis and identify security vulnerabilities and compliance issues. Unit testing tools such as junit, xunit, nunit or mstest can be used based on the technology stack. Code quality analysis tools such as SonarQube, coverity, cobertura and Fxcop are widely used for code quality analysis in IoT application components development.
    • Containerization and Container Orchestration: You can deploy your application as Docker containers, which have less footprint in the physical/hosted infrastructure and runs only with required application framework, libraries and dependencies. These containers/cluster of containers are managed with container orchestration tools such as Kubernetes, Docker Swarm, AWS Container Services, Azure Container Services, etc. As mentioned above, simulated test devices can run in docker containers, which enables shipping/moving the containers across any cloud and making the test environments ready within minutes.
    • Monitoring: Monitoring in IoT involves monitoring the devices; infrastructure VMs/clusters; telemetry data; the build, test and releases; and billing, among others. Metrics such as cluster usage, CPU usage, network availability, response time, number of devices connected and devices updated/not updated can be configured and viewed in monitoring tools including AWS Cloudwatch, Azure Application Insights & LogAnalytics, Stream Analytics, New Relic, Prometheus (open source), Nagios, Coscale, SysDig, Dynatrace and GE Predix Monitoring. The tools can be selected depending on the target platform on different clouds or on-premises, technology stack and other factors.
    • Cloud and Cloud Management: Cloud providers including AWS, GE Predix, Azure and GCP provide out-of-the-box features and support for IoT platforms with starter kits, IoT device SDKs, protocols, gateways customization and more. Cloud management tools such as Right Scale cloud management platform, OneOps, etc., can be used for managing the infrastructure workloads; bringing in central governance, access control and security; monitoring billing and optimization; managing security and standards; and streamlining and orchestration of IoT platform development.

    Depending on the technology stack, the DevOps requirements and business needs, the right tool needs to be selected to streamline IoT platform development with feedback flowing between the different stream of teams involved in IoT development, and to bring in standardization in security, governance and tools, metrics to measure and to better manage IoT platform.

    Release, Rollout Firmware/Application Updates

    Rollout of firmware updates for an IoT platform is more challenging compared to application releases; with device updates, we have to deal with network latency, issues with connectivity of devices, security issues, different sets of devices, different deployment channels, deployments done over the air and different sets of protocols used by different manufacturers.

    There is additional complexity involved when there are new additions to devices or when supporting legacy devices, and this needs incorporating integrated end-to-end testing as part of deployment pipeline, to test the devices and applications from a user-centric approach.

    In IoT development, there are multiple development and deployment pipelines for firmware updates, server-side code, mobile apps and multiple devices delivering to multiple endpoints. Hence, the release cadence of IoT components varies, as application code updates can be released monthly, whereas firmware updates to devices can happen every three to six months—hence, a well-defined process must be followed for regularly integrating the code.

    Continuous testing involves infrastructure testing, performance testing, end user testing and functional testing, all of which must be integrated with the release pipeline, where deployments can happen to simulated test environments, provisioned as environments as code (environment with all runtime dependencies needed to run the application like Docker containers), rather than simply infrastructure as code.

    DevOps teams need to keep track of releases and monitor all these deployments, ensure environments are stable, trace back the releases with the device firmware updates and track the defects. The DevOps teams should pull the release packages of different components from multiple pipelines and prepare main and customized builds to integrate and deploy the components together.

    The following are the main steps in CI/CD pipeline for IoT platform development.

    1. Code checked into version control: Check in the code for application development, device/firmware development, PaaS services configurations, infra automation scripts and templates to the version control, pointing to the defined repositories.
    2. Continuous Integration: Build jobs created in a CI server such as Jenkins/VSTS/Bamboo executes build compilation, unit testing, code quality analysis, automated reviews, environment configuration preparation, running security testing analysis, creating deployment packages or baked-in container images, for deployment.
    3. Triggers Deployment: The build agents generate the packages for firmware updates application updates and notifies release agent to trigger deployment.
    4. The deployment workflow can call a service/worker job, which in turn can connect to the cloud gateway services, to notify firmware updates.
    5. The cloud gateway picks up the firmware updates and notifies devices for devices updates. The devices will the pull the updates and installs the updates and further send telemetry data.
    6. Multiple testing environments in cloud can be integrated to the release pipeline to simulate the firmware updates.
    7. In case of application rollouts, there will be canary releases to gradually roll out to different environments in different regions after functional testing/regression testing.
    8. Testing: Security testing, firmware updates targeting to specific devices, performance of the device during the updates, device identity and memory usage, etc., needs to be addressed during deployments in IoT. Once the application is deployed to the test environment, functional/regression testing occurs and generates the test reports.
    9. Monitoring metrics such as memory usage, performance of the device, network traffic, latency, devices connected, reliability, etc., needs to be monitored, as these are of prime importance to decide on how to implement remote updates and monitoring without affecting the performance of the devices and these metrics has to be notified to the Dev/QA/DevOps teams as part of release pipeline execution.
    10. On-demand environment provisioning scripts/templates must be integrated with deployment pipeline to provision the test environments, add more devices as per the scaling metrics.

    Establishing a CI CD pipeline well ahead at the start of development and following mature DevOps practices such as security testing, infrastructure testing, containerization, monitoring, cloud and DevOps governance and feedback-driven development are the keys to help address the challenges in building your own services and IoT solutions.

    About the Author / Lavanya Subbarayalu

    Lavanya Subbarayalu is Senior Architect working with Technology Office in HCL Technologies. She has expertise in IOT, Azure, DevOps consulting & Microsoft technologies. She is associated with DevOps COE, working on design and Development of DevOps solutions and consulting tools. Connect with her on LinkedIn.

  • With a Multicloud Infrastructure, Control is Key

    With a Multicloud Infrastructure, Control is Key

    Ask any developer or DevOps manager about their first experiences with the public cloud and it’s likely they’ll happily share some memories of quickly provisioning some compute instances for a small project or new app. For (seemingly) a few pennies, you could take full advantage of the full suite of public cloud services—as well as the scalability, elasticity, security and pay-as-you-go pricing model. All of this made it easy for teams to get started with the cloud, saving on both the IT budgets and infrastructure setup time. Public cloud providers AWS, Azure, Google Cloud, Rackspace and others made it easy to innovate.

    Fast forward several years and the early promise of the cloud is still relevant: Services have expanded, costs have (in some cases) been reduced and DevOps teams have adapted to the needs of their team by spinning up compute instances whenever they’re needed. But for many companies, the realities of their hybrid-IT infrastructure necessitates support for more than one public cloud provider. To make this work, IT Ops needs a control layer that sits on top of their infrastructure, and can deliver applications to customers over any architecture, including multicloud. This is true, no matter what the reason teams need to support multi-cloud environments.

    Prepare for the Worst

    As any IT Ops manager, or anyone who has lost access to their web app knows, outages and cloud service degradation happen. Modern Ops teams need a plan in place for when they do. Many companies choose to utilize multiple public cloud providers to ensure their application is always available to worldwide customers, even during an outage. The process of manually re-routing traffic to a second public cloud in the event of an outage is cumbersome, to say the least. Adding an app delivery control plan on top of your infrastructure allows companies to seamlessly and automatically deliver applications over multiple clouds, factoring in real-time availability and performance.

    Support Cloud-driven Innovation

    Ops teams often support many different agile development teams, often in multiple countries or from new acquisitions. When this is the case, it’s likely that the various teams are using many architectures on more than one public cloud. Asking some dev teams to switch cloud vendors is not very DevOps-y. A better option is to control app delivery automation with a cloud-agnostic control plane that sits on top of any cloud, data center or CDN architectures. This allows dev teams to work in their preferred cloud environment, without worrying about delivery.

    Avoid Cloud Vendor Lock-in

    Public cloud vendors such as Amazon Web Services or Microsoft Azure aren’t just infrastructure-as-a-service (IaaS) vendors, they sell (or resell) products and services that could very well compete with your company’s offering. In the beginning, using a few cloud instances didn’t seem like such a big deal. But now that you’re in full production and depend on one of these cloud providers for your mission-critical app, this no longer feels like a great strategy. Adding a second public cloud to your infrastructure lessens your dependence on a single cloud vendor who you may be in “coopetition” with.

    Multiple-vendor sourcing is a proven business strategy in many other areas of IT, giving you more options during price and SLA negotiations. The same is true for IaaS. Cloud services change often, as new services are added or removed, and price structures change. Taking control over these changes in public cloud service offerings, pricing models and SLAs is another powerful motivator for Ops teams to move to a multi-cloud architecture. An application delivery automation platform that can ingest and act on cloud service pricing data is essential.

    Apps (and How They’re Delivered) Have Changed

    Monolithic apps are out. Modern, distributed apps that are powered by microservices are in. Similarly, older application delivery controllers (ADCs) were built for a static infrastructure world, before the cloud (and SaaS) were commonly used by businesses. Using an ADC for application delivery requires a significant upfront capital expense, limits your ability to rapidly scale and hinders the flexibility to support dynamic (i.e. cloud) infrastructure. Using ADCs for multiple cloud environments compounds these issues exponentially. A software-defined application delivery control layer eliminates the need for older ADC technology and scales directly with your business and infrastructure.

    Regain Control

    Full support for multicloud in product may sound daunting. After all, Ops teams already have plenty to worry about daily. Adding a second cloud vendor requires a significant ramp-up period to get ready for production-level delivery, and the new protocols, alerts, competencies and other things you need to think about. You can’t be knee-deep in the details of each cloud and still manage infrastructure. Adding in the complexity of application delivery over multiple clouds can be a challenge, but much less so if you use a SaaS-based application delivery platform. With multicloud infrastructure, control is key.

    About the Author / Andrew Marshall

    Andrew Marshall is the Director of Product Marketing for Cedexis, and has driven product marketing efforts at New Relic, Amazon Web Services and Microsoft. Connect with him on LinkedIn.

     

  • Software Security is an Engineering Problem

    Software Security is an Engineering Problem

    We live in a world where those who sell products are generally held accountable for defects. Automobile manufacturers, pharmaceutical companies and toy makers are among the businesses that have been handed fines and tarnished reputations for distributing items that were later deemed ineffective or dangerous.

    One notable exception to this “consumer first” ethos is the traditional shrink-wrapped software world, where responsibility for product problems is hazy. The vendors that develop and sell software and the customers who purchase and install that software are in a constant battle to shift security liability. The vendors strive as best they can to reduce and remove product vulnerabilities and offer secure configurations while customers deploy, configure and operate the software in very different environments, with very different configurations, which can produce very different security outcomes. Because no customer can (or should) assume receipt of 100 percent vulnerability-free software, they must always take on the primary burden of protecting the environments where that software lives to offset the risks—which typically falls to the customer’s IT and security teams.

    Although there are excellent arguments for both sides, the current situation leaves both parties wanting—and needing—a higher level of assurance in both the development and runtime settings.

    Software Delivery Has Changed (Along with Security Responsibility)

    The good news for software customers is that software delivery is rapidly moving away from an on-premises install approach to a cloud-based subscription model where applications are delivered via the internet. Upfront costs for hardware and software, as well as the expense of personnel to install and manage that software, are replaced with business-friendly, predictable, subscription-based operational costs. Software is easy to launch, upgrades are uncomplicated and scaling to meet changing business needs is simple.

    Customer expectations are high for software-as-a-service (SaaS) solutions. Purchasers demand an elevated level of innovation and want new features more often. They assume built-in support of global information security and privacy regulations such as the European Union’s General Data Protection Regulation (GDPR) and various U.S. regulations, including the Health Insurance Portability and Accountability Act of 1996 (HIPAA) and PCI Digital Security Standard (PCI DSS). SaaS customers also expect the vendor to take on larger responsibility for any potential software vulnerabilities that might negatively impact the customer’s data and operation.

    The new reality is that deploying software in the cloud significantly changes the paradigm of how software security should be treated at the vendor level. This means companies now have to think critically about security as an engineering problem as the balance of the liability burden shifts focus to the cloud provider’s engineering practices to ensure constant delivery of a secure and trusted cloud service, rather than just secure software.

    Now is the Time to Rethink Software Security

    Cloud-based application delivery brings together under a single company umbrella the ones that make the software and the ones who run the software. Informing software development of runtime behavior and vice versa can have significant advantages in enhancing the overall security of the software by offering a closed-loop system. In such a perspective, the rapid pace of software releases work in favor of enhancing security.

    Shifting security to an all-around engineering responsibility requires the elimination of traditional work silos. Geared-for-speed developers must receive precise information early, allowing them to correct security problems during the initial coding process. DevOps team members also must assume security responsibilities. And security engineers who understand the attacker mindset, security risks and compliance requirements must provide guidance at all points in the software development process.

    Change is never easy in any organization, so transforming disconnected teams into one in which security is an across-the-board responsibility is difficult. Managers must be careful to not alienate team members who already have full plates. And although we’re starting to see some shifts in the educational model to one in which professionals are taught early on how to seamlessly integrate coding and security, we have a long way to go.

    While we wait, the most effective way to overcome the chronic problem of heavily siloed teams is to provide the entire software engineering team with a means to quickly identify issues between software development, its deployment and runtime across all the components. These tools must give users the ability to:

    • Scan and provide real-time analysis that integrates source code analysis and its runtime behavior. After all, they are two sides of the same coin;
    • Obtain accurate and full visibility into the security impact of changes from build-to-build;
    • Ease the audit analysis process;
    • Reduce the number of false positives; and
    • Allow developers, DevOps, and Security to collaborate to enhance the security of the environment.

    The goal? An ability to fix software mistakes, weaknesses and vulnerabilities at the beginning of the development process instead of post-deployment, when liability, revenue and reputational risks are the greatest.

    Consolidating today’s siloed security practices into an integrated engineering effort is the only way to ensure the appropriate level of security at the appropriate time in today’s fast-moving cloud-based software industry. This sort of shift in thinking and processes will give customers less security worries and better products. Development teams will become more effective and efficient. DevOps will have real-time visibility into how each release impacts the infrastructure. And Security will know how each release impacts both security and compliance requirements. SaaS companies will reap the financial benefits that come with strengthened industry reputations for rapid innovation capabilities and high levels of software security. It’s a win-win-win for everyone.

    About the Authors

     Manish Gupta is founder and CEO of ShiftLeft Inc. He is a 20-year veteran of the security industry and has led product development, management and strategy at FireEye, Cisco Systems, Intel, McAfee and Redback Networks.

     

     

    Craig Rosen is vice president and CISO at AppDynamics. He has held leadership roles for more than two decades in the security industry, including at FireEye, Pacific Gas & Electric, TDI and CGI.