devopsy.kinsta.cloud

Tag: CI/CD

  • Rookout Tool now Debugs Third-Party Code

    Rookout Tool now Debugs Third-Party Code

    Rookout revealed today it has extended its eponymous debugging tools to include third-party applications.

    Shahar Fogel, Rookout’s CEO, says most organizations today are building applications that rely heavily on code or modules developed by a third party. Rookout now makes it possible to analyze and debug that code before it is incorporated within a larger application environment, Fogel said.

    That’s crucial, because many of the operational issues a DevOps team is likely to encounter can be traced to flawed code written by someone outside of the IT organization, Fogel said. This new capability will reduce the amount of time DevOps teams spend trying to deduce the root cause of an issue, he said.

    Many IT organizations erroneously assume that all the code running in their environments has passed through a continuous integration/continuous deployment (CI/CD) environment, Fogel said. In reality, it’s not uncommon for developers to deploy code directly into a production environment in the name of expediency. Not all of that code, however, may have been written by the developer. Many developers wrongly assume the code they download from a repository was properly vetted and debugged.

    Rookout now makes it possible to debug both open source code and proprietary code written by a third party or present on an application deployment platform, said Fogel. Rookout collects the local variables, function arguments, tracing data, call stack and other relevant data, and can identify issues down to specific lines of code. Support for third-party code has been extended to the call stack level to enable developers to see how code execution flows across their own and third-party code.

    As more responsibility for applications shifts left toward developers, organizations need to provide them with tools to analyze applications and the platforms they run on. Beside building applications faster, one of the primary theoretical benefits of DevOps is the notion that developers can build more resilient applications, because it’s ultimately their responsibility to remediate any issues that arise after an application is deployed in production. In practice, that’s difficult to achieve when developers have no visibility into code that they didn’t write themselves.

    Going forward, there’s going to be a lot more third-party code running in production, as developers reuse application code encapsulated in containers downloaded directly from registries, Fogel said.

    The degree of accountability developers have for their appliations’ code currently varies widely by organization. However, given the right tools, and enough time, most developers would prefer to debug both the code they write and any they have incorporated, even if it were written by someone else. That desire, however, requires more time to build and test applications. Of course, it’s worth the tradeoff if developers encounter fewer incidents that require developers to debug applications after they are deployed in a production environment.

  • A Security Vulnerability Management Guide

    A Security Vulnerability Management Guide

    Living in a container-native world is not easy. Containers have a reputation for being a point of entry for security vulnerabilities for many organizations. In 2015, according to a research paper, over 40% of Docker images distributed through Docker Hub had high-risk vulnerabilities; at that time there were more than 95,000 container images hosted on Docker Hub. Today, there are over 3.5 million container images on Docker, and container security is of even greater concern. In a more recent 2020 study by a team of researchers at the Norwegian University of Science and Technology found container image vulnerabilities in certified and popular packages.

    This is just one example of one kind of security vulnerability. This article will share an introduction to security vulnerabilities and the role of vulnerability management for containers and other artifacts in a continuous integration and continuous delivery (CI/CD) pipeline.

    What is a Security Vulnerability?

    A vulnerability is a weakness or flaw present in software. Security vulnerabilities can be present in application dependencies or operating system (OS) packages. Common vulnerabilities include missing data encryption, buffer overflows, missing authentication for critical functions and insecure interactions between software components.

    There are different risks associated with different types of vulnerabilities. With critical or high-risk vulnerabilities, an attacker who exploits your software has the potential to seriously impact your organization. Risks associated with data breaches impact not only an organization, but also its end users and customers.

    Vulnerability Management Techniques

    Vulnerability assessment practices and tools exist to make detecting vulnerabilities simple, accurate and fast. Some of these practices include:

    • Penetration testing – Penetration testing, or pen testing, allows you to identify security vulnerabilities by attempting to break into your own systems to identify weaknesses.
    • Configuration management – Configuration management involves managing infrastructure configurations and identifying missing patches that leave your software service vulnerable to errors and risks. There are tools that provide infrastructure scanning as-a-service to help detect outdated or misconfigured instances.
    • Container and application Scanning – Allows you to detect vulnerabilities in deployable artifacts and running applications. Some available tools for container scanning include Twistlock, Clair and Trivy.

    These practices help build solid, repeatable vulnerability management techniques across your security and delivery teams. One way to scale these security practices is through security automation. Security automation is the use of technology that performs tasks with reduced manual assistance. It ultimately enables users to apply security decisions automatically and secure processes to deliver more robust applications and infrastructure.

    Security Practices in your CI/CD process

    Security automation is a core tenant of DevSecOps. DevSecOps is short for development security and operations, and it describes how organizations deliver and make security decisions and actions within the development life cycle of their valued deliverables.

    DevSecOps is a way of continuously integrating security in the software development life cycle. It’s a way of working and thinking so that security is at the forefront of how security teams deliver business value.

    One way to enable DevSecOps is through your CI/CD pipeline. Today’s enterprises can use CI/CD to accelerate better software delivery, but with that speed it becomes even more important to enable robust security. Vulnerability scanning improves software security while giving individuals across engineering and product teams accountability for each of the processes these teams own.

    Don’t let Security Vulnerabilities Ruin your DevSecOps

    This is a simple, basic introduction to security vulnerabilities and the risks they pose for your organization. There are practices and tools for detecting and reducing security vulnerabilities, and finding ways to incorporate those processes into a CI/CD pipeline is a great way to accelerate your DevSecOps.

  • What Does the Future Hold for DevOps and CI/CD?

    What Does the Future Hold for DevOps and CI/CD?

    After such a challenging year, we have finally made it to 2021. Although a new year has started, we are still trying to figure out what the “new normal” will look like once the COVID-19 pandemic is finally over.

    While the pandemic has affected many businesses and our daily lives in general, it also had a major impact on software development. This past year we have seen rapid innovation and acceleration of digital transformation as a result of the pandemic. So, what does 2021 have in store for DevOps? How can DevOps teams improve productivity and become high performers in 2021?

    In this interview, CircleCI CEO Jim Rose joins Rebecca Auguste to talk about what 2021 has in store for DevOps and CI/CD, how the pandemic impacted software teams and what they can do to succeed in this new year.

    Jim is one of the distinguished speakers that will be sharing his knowledge and predictions for 2021 at this year’s Predict Virtual Summit. During his session, Jim will discuss what’s in store for engineering teams in a post-pandemic world, backed by anonymized team data from millions of DevOps workflows. Attendees will leave with insights and benchmarks to help teams build better software faster.

    Jim has led category-defining companies through economic crises, with COVID-19 being no exception. He is on a mission to provide everyone the ability to build and deliver software at the speed of imagination.

    To find out what the future of software looks like, attend Jim’s session at Predict Virtual Summit on Jan. 21. To check out the full agenda or to register for free, visit the Predict 2021 website.

  • DevOps for the Development and Delivery of High-Performance Applications

    DevOps for the Development and Delivery of High-Performance Applications

    Engineering teams that successfully implement DevOps practices often deliver software with higher business value. These organizations can easily identify areas where problematic code exists, quickly eliminate issues and deliver high-performance applications more quickly than their competitors that use traditional delivery methodologies. DevOps practitioners understand that creating an efficient model for performance testing goes hand-in-hand with four key objectives of digital transformation: extending performance testing to new roles; integration into continuous integration/continuous delivery (CI/CD) processes; end-to-end performance monitoring; and continuous optimization.

    Reduce Cycle Time and Time to Market

    In order to meet the requirements of rapid, high-quality application delivery, modern software teams need an approach that goes beyond conventional performance testing. This is where DevOps-centric approaches can provide faster, more efficient development and help ensure the delivery of high-quality software. Building the components of an application using agile approaches and validating features as they are created ensures that applications can be tested and verified more quickly, with no increase in associated costs or resources.

    By adopting DevOps practices, individual steps within the application delivery process can be automated as part of a DevOps tool-chain. For example, automating quality assurance test runs can eliminate – or significantly reduce – time-consuming human or analog searches for defects in code. Test automation can lead to an increased volume and expanded scope of testing, which, in turn, leads to a reduction in the defect escape rate. That saves resources, time and money.

    Shift Left to Increase Agility

    To mitigate performance errors and to address issues as early as possible in the application delivery cycle, tests should be executed during the build stage of the development process. By shifting left – running the tests as early as possible, ideally as part of a code commit – enterprises reduce the risk of discovering defects or issues with the product after significant amounts of development work have been completed. It is widely accepted that addressing defects later in development cycles takes significantly more time and money versus identifying and remediating defects early in the development cycle.

    As developers perform rapid, high quality, incremental changes, business analysts or product owners can more easily share changes, concepts or suggestions with their development teams. This continuous flow of communication can significantly reduce feedback time, increasing an organization’s ability to react to changing market requirements and allowing a level of flexibility within the development process that drives competitive advantage.

    DevOps in a Hybrid Landscape

    By supporting a wide range of tools and technologies, organizations that adopt DevOps practices can support applications that run in a local network environment, in cloud-based environments or, more commonly, in hybrid IT environments. As companies continue cloud migrations and transformations, well-designed DevOps tools can help simplify this transition while continuing to provide support for the legacy applications that provide core capabilities and processing for many enterprises. These hybrid tools are fully integrated into existing application development processes, allowing all application delivery teams to successfully transition to DevOps-centric practices without leaving any teams behind.

    One particularly important DevOps practice is performance optimization through continuous analysis. By using a centralized store and a common approach for quality-centric data collation, this practice helps analyze, enforce and improve the connections between software developer, CI Build and end-to-end performance testing to ensure a “quality first” mentality and higher-value delivery outcomes. The application of data visualization techniques allows teams to access code – build – test results in real-time and make decisions based on up-to-date insights.

    If DevOps practices are successfully established within a team or organization, they not only support more efficient software development and delivery but also are cost-effective. However, in order to successfully implement DevOps practices, enterprises need to understand and implement both technological and cultural changes. Through a combination of process, tooling and cultural simplification, and aligning teams toward common business objectives, organizations can deliver their end goal of creating high-quality, high-value software that satisfies the needs of the end user. By focusing on the common goal, and understanding the journey necessary to get there, enterprises can do so more quickly and at less cost – a win-win!

  • What Is a CI/CD Pipeline? How to Improve CI/CD

    What Is a CI/CD Pipeline? How to Improve CI/CD

    The most effective way to ensure that your CI/CD pipeline is efficient is to use the right tool, like a static code analyzer. Here we explain what is a CI/CD pipeline and how a static analysis tool can help.

    What You Need to Know About CI/CD Pipelines

    The easiest way to understand a CI/CD pipeline is by understanding its individual components. Continuous integration (CI) is the practice of merging all of the working copies of code together into a shared mainline, while continuous delivery (CD) is the regular delivery of software functionalities.

    With a CI/CD pipeline, you can expect the following benefits:

    • Identifying software issues, errors, and bugs earlier in the software development process. This enables you and your team to fix these issues before they become too damaging and costly.
    • Reducing the amount of potential risk to your development process and your software.
    • Improving the quality and reliability of your software.

    Without having to run the program, a static analysis tool inspects your source code to identify defects, vulnerabilities, and compliance issues.

    This makes it invaluable to any CI/CD pipeline, as it helps with:

    • Detecting security vulnerabilities, runtime errors, and other software issues.
    • Enforcing compliance with safety-related coding standards as well as with company- or project-wide coding guidelines.

    How You Can Improve Your CI/CD Pipeline With a Static Analysis Tool

    With a static analysis tool, you are able to:

    • Ensure that your software is secure, reliable, and high quality.
    • Reduce the cost of identifying and fixing software defects and vulnerabilities earlier in your development process.
    • Enforce software coding standards to help you more easily prove compliance.
    • Improve the productivity, testing efforts, and pace of your software delivery efforts.
    • Report on the quality of your software over time and across multiple product versions.

    To read more, please visit: https://www.perforce.com/blog/kw/static-analysis-ci-cd-pipelines

  • How Healthcare Companies Can Optimize Software for Improved Patient Outcomes

    How Healthcare Companies Can Optimize Software for Improved Patient Outcomes

    Healthcare companies are more reliant than ever on software to help them achieve good patient outcomes. Recent research reveals that the use of software as a service (SaaS) solutions is growing exponentially in health care, covering a wide range of areas including:

    • Patient diagnosis
    • Medical imaging
    • Therapy and patient self-care
    • Data collection and analysis
    • Hospital management

    Due to the increased use of digital products in patient care, healthcare providers recognize the need to speed innovation and fundamentally rethink how they build software. This has led many engineering teams to move toward continuous integration and continuous delivery (CI/CD) when building and deploying software.

    Considering the industry’s emphasis on improving patient experiences, it makes sense that healthcare providers are keener than ever to optimize their systems. In fact, around 90% of healthcare providers are implementing or already using customized software solutions and development.

    As teams look to improve these critical processes in software development, feature flagging is one way to speed up development and optimize software performance in the healthcare industry.

    CI/CD Challenges for Healthcare Companies

    The healthcare industry faces unique challenges when applying DevOps strategies such as CI/CD.

    Intense Regulation

    The main problem is that, for obvious reasons, health care faces stringent regulation and oversight. A high level of scrutiny is necessary in an industry in which one small mistake or misjudgment can cost patient lives; however, in terms of DevOps, waiting for regulatory evaluation and approval can slow down the entire software development process, which defeats the core purpose of CI/CD.

    Security

    Data security poses another challenge for healthcare providers, especially when dealing with cloud-based software solutions. Cloud-based systems will always have potential vulnerabilities, as they rely on infrastructure that connects a variety of devices and smart technology equipment. There will always be inherent weaknesses in remote systems, as opposed to on-premises, too. A recent survey revealed that 42% of respondents said their major security concern was unauthorized access to cloud systems through insecure APIs. Network configuration problems were a close second, with 40% of respondents citing those as a concern.

    In a healthcare setting, another issue is the high volume and high velocity of data. Even a small, specialist clinic that may only see 150 patients per day will still generate vast amounts of data over the course of a month or year. Most of the data is private and sensitive, which means that data privacy and cybersecurity are top priorities for healthcare companies.

    The Impact on DevOps

    At first glance, because of the high level of regulatory oversight and cybersecurity concerns, it might seem that health care is a less-than-ideal environment for agile DevOps processes. How can you adopt fast-paced CI/CD cadences and innovate rapidly when there are necessary, opposing forces?

    The answer is to leverage innovative software development solutions such as on-premises feature flagging.

    Feature Flagging and Its Impact on Software Innovation

    Feature flags are, essentially, toggles or switches that allow software development teams to turn features on or off. The main advantage of feature flagging is that it allows for faster software deployment. One or more features can be coded into the software during development and made ready to go, but it is turned off for the user and/or patient on initial deployment.

    The ability to embed features within software that can be toggled on or off is perfectly suited for the healthcare industry since that approach allows for the necessary regulatory approval(s) before full deployment.

    For example, look at the rapid rise of telemedicine during the COVID-19 pandemic. Doctors, hospitals and patients alike were forced, along with the rest of the world, into the “new normal” of virtual interaction to mitigate the spread of disease while still providing quality treatment.

    For most industries, adding video chat/conferencing capabilities to existing applications was relatively easy. But for health care, the approval process, privacy and security concerns slowed down the adoption of these features. In our experience, typical healthcare teams are, best-case scenario, running on monthly—or, most likely, quarterly—deployment cycles to align their deployment efforts with compliance and governance review boards. Product and engineering teams work to get their updates and improvements in line with this cadence, which can make innovation difficult.

    If teams wanted to deliver this capability quickly, they could adopt feature flagging to decouple the concepts of “deploy,” “approve” and “release.” To use the video chat example, here are the steps they’d be able to take:

    1) Build the videoconferencing capability into their app and deploy the feature to production with the capability switched on for internal users (IP targeting, specific email addresses, etc.) only.

    2) Now that the feature is in live production, they can perform actual field testing to ensure that the functionality is working well, while simultaneously moving through internal and external approval processes.

    3) Once the review is complete, move to the release stage by switching the feature(s) on and allowing (external) end users to access the capability.

    All this can be achieved without needing to redeploy the software, and the developers are still free to make iterative improvements to the application.

    Now that the regulatory hurdles are addressed, it’s important to acknowledge that security is still an issue for healthcare companies that employ a SaaS feature flagging platform.

    While most providers focus solely on SaaS, the ability to deploy feature flagging software on proprietary cloud infrastructure is important in the healthcare space. This model enables both extra security and HIPAA compliance, and also opens up important capabilities and performance results that SaaS-only providers have a hard time matching. Since this technology typically becomes deeply embedded in the fabric of an organization’s product and technology, feature flagging is well-suited to an on-premises implementation.

    The implementation of on-premises feature flagging is a great way for healthcare companies to increase deployment velocity from monthly to daily. It allows their software development teams to adopt CI/CD cadences and innovate more rapidly, without the risk of breaching regulations or making sensitive data easily available to hackers.

  • Unlocking Your DevOps Automation Mindset

    Unlocking Your DevOps Automation Mindset

    In the new digital normal, where remote working has upended conventional software development cycles, the role of DevOps is central. Many enterprises have an accelerated digital agenda, but are often underinvested in rebooting DevOps for several reasons, including a lack of experience and insight into how it can help businesses recover from the new challenges created by the pandemic. This can be a strategic mistake, as real acceleration is driven at the developer level. Here are three ways DevOps automation can help enterprises overcome challenges in product life cycles and excel during this time of flux.

    DevOps Can Accelerate Product Life Cycles to Meet Demand

    In 2015, Adidas released a new line of Yeezy sneakers, which became an instant hit. The company quickly sold more than 2,600 shoes every five seconds. But Adidas’ IT infrastructure could not keep up with demand and crashed, and software developers were unable to find a quick fix.

    Impacted by customer and employee dissatisfaction, Adidas needed a solution to recover from the chaos and prevent future challenges. That’s when the company embraced cloud-native architecture, Kubernetes and DevOps. Today, Adidas’s IT infrastructure has accelerated its software product delivery cycle from once every six weeks to five times a day, while building a more resilient system.

    It is not that Adidas wasn’t aware of DevOps in 2015. The company hadn’t yet invested in educating its teams about the DevOps automation mindset, nor had it made DevOps automation a priority. Adidas needed a shift in culture, one that it brought about by rebooting its DevOps practice. This DevOps success story underscores how an accelerating go-to-market demand needs to be supported by underlying structural and systemic change.

    Today, as the pandemic continues to disrupt software development workflows, I urge enterprises to use this opportunity to reboot DevOps and gain speed, agility and resilience. DevOps can change the mechanics of dependencies between development and operations to reduce friction. By leveraging extreme programming and design thinking in conjunction with DevOps, enterprises can accelerate every segment of the software development life cycle independently, while maintaining resilience and adequate levels of governance. It provides the opportunity to invest in security-as-code (aka DevSecOps) so it is sustainable and scales easily. This also enables enterprises to reimagine software architecture and governance by implementing advanced digital architectures.

    For enterprises on the path to such transformation, there are four key DevOps Research and Assessments (DORA) metrics indicative of their DevOps practice’s success. These are: lead time for changes, deployment frequency, time to restore service and change failure rate. The first two of these could be seen as an “offensive strategy” and the latter two as a “defensive strategy” to measure speed. This means to accelerate delivery, organizations must introduce a faster lead time for changes to production, increase the frequency of new software deployment, shorten the time it takes to restore services to production and reduce the change failure rate to production. As you can see, Adidas needed improvements in all four of these areas to successfully reinvent its performance.

    DevOps Automation Can Reduce Complexities in the Software Development Process

    It took more than four years for BMW, the premium automobile manufacturer, to move from a waterfall software development methodology to a 100% agile, DevOps methodology. Today, 20,000 employees in the company use an agile software development toolchain, allowing them to streamline and synchronize their end-to-end software development. BMW has reduced its time-to-market by 25%, minimize incidents by 30% and gain a 360-degree view of IT.

    Today, enterprises are shifting from waterfall to agile weekly and daily releases. My belief is that every enterprise needs to adopt a 100% agile methodology, just like BMW did. Testing and continuous improvement/continuous development (CI/CD) is key for deploying code in small chunks and reducing merge issues and refactor efforts. Ultimately, this increases developer velocity and decreases lead time.

    The shift from a partial to a 100% agile model requires more than simply senior leadership’s resolve. It needs a dedicated pool of certified DevOps automation consultants, coaches and subject matter experts with experience in SAFE, LESS, Scrum and Kanban frameworks. Best-in-class enterprises and OSS toolchains that cater to DevSecOps, service meshes and omnichannel apps are essential. Simultaneously, agile-based delivery coaching, audits and continuous support to existing and new delivery teams are a must.

    While DORA metrics can serve as a good measure of an enterprise’s DevOps performance, businesses will need tools to assess DevOps maturity, improve developer productivity and provide specific recommendations for improvement. Data will play an important role in decision making and aid every developer’s performance, more than at any time in the past.

    Automation Makes Room for New Product Development

    Many software development teams struggle to keep up with the onslaught of new technologies entering the market. Take, for instance, artificial intelligence (AI); the number of applications built using AI is rapidly increasing. However, it continues to be new to many development teams. A similar story unfolds with Internet of Things (IoT), which creates complex systems with information pouring in from thousands of sensors and appliances every second with multiple software and hardware configurations. Testing these devices, configurations and performance cannot be done manually. This is where a DevOps automation mindset can help. It can ensure quality assurance processes have considerable test coverage across these different technologies and configurations by introducing automation.

    Newer technologies, such as Kubernetes, serverless apps, 5G, edge and cloud computing are increasingly used in product development, making DevOps even more important today. DevOps also plays a key role in transforming the developer experience. Today, developers need the foundation of agility, decision-making power and speed to deliver solutions on-demand. To facilitate this, enterprises must automate the management of development, testing, quality assurance and IT operations to successfully build continuous delivery processes. Further, given COVID-19 and the shift to working remotely, DevOps can help product teams collaborate remotely, leveraging well-established platforms available with any of the major cloud providers.

    As enterprises accelerate and adapt to work in this new normal, leveraging both the cloud and a remote workforce, a DevOps Automation mindset can speed up this transition. Enterprises must assess performance and enhance capability at the developer level to truly accelerate the journey to digital transformation.

  • Developer Tooling for Emerging Technology

    Developer Tooling for Emerging Technology

    2020 has brought a seemingly endless amount of problems, but software developers have responded with solutions. Despite all of this year’s change, one thing remains true: Software is being used in a multitude of ways to tackle some of the world’s most pressing challenges. And developer creativity is helping to progress exciting new areas of technology innovation; specifically, internet of things (IoT) and edge computing, 5G and autonomous vehicles.

    The barriers to entry for development in these new areas has lowered in recent years. It’s now just as cost-effective to get started on an IoT hardware project as it is to build a web or mobile application and learning how to get started with the hardware is as easy as searching for “how to raspberry pi” on YouTube. The issue is, once you get started and begin shipping units, you’ll come to realize that developer tooling for emerging areas of technology is lagging behind the curve. The market for tools that monitor and test web and mobile applications is healthy and diverse, but tooling for IoT/edge, 5G and autonomous vehicles has yet to see the same surge. And as projects become more complex, how do you keep up with both automated and manual testing? How do you ensure that firmware updates don’t “brick” devices out in the field? Fixing a bug in a website can be deployed in a few minutes, and in some cases, customers won’t even notice. Shipping new versions of software to devices is a little more tricky.

    One of the biggest areas of change in this sector is innovation in CPUs. For the longest time, engineers could write software that worked on either Intel- or AMD-based CPUs—and the majority of software that exists today can run on either of those platforms. However, IoT devices make heavy use of the newest player in the CPU space: Arm. Most people don’t realize it, but anyone with either an Android or iPhone smartphone in their pocket already has a device running an Arm processor. Arm processors work especially well in low-power environments, such as those you find in IoT. The biggest problem that engineers face with this new CPU is that the majority of developer tooling isn’t suited for them. It’s all been built with either Intel or AMD CPUs in mind.

    We are only beginning to realize the possibilities of these emerging technologies—and developers will be the ones paving the way, building the solutions that will power our future. As they continue to evolve and create new unstructured data sources, companies will need to invest in the tools that enable developer teams to not only keep pace but also innovate.

    The tools I’m referring to will be the ones that let developers seamlessly deliver software across any environment—from IoT devices to private or public cloud to embedded systems to 5G. CI/CD platforms are one way to accomplish this. But there’s a misconception when it comes to selecting CI/CD providers: that developers must choose between the security of on-premises environments and the scale of public cloud. A hybrid cloud approach to CI/CD is an effective way to test and deploy software at scale without compromising the security of a company’s most valuable resource: its source code.

    For so many teams and companies, this was the year of throwing out the playbook and rewriting the road map. The window of opportunity for true software innovation has never been more open. Whether it be in IoT, 5G or autonomous vehicles, developers are moving fast to change the world. They need developer tooling that will help them deliver on those ambitions.

  • Continuous Delivery Without Exception

    Continuous Delivery Without Exception

    Continuous delivery (CD) and all the other continuous things—continuous integration (CI), continuous testing, continuous deployment—have become the new mantra of DevOps. CI/CD adoption typically starts with agile development teams working on new or well-funded systems. But what about the rest of the world and all of those “other” applications, under-funded IT teams are responsible for supporting? Can or should continuous delivery be a reality for all applications?

    The answer is a resounding yes! No application is an island. Every application is subject to change, whether it be code changes, patches to the OS, changes to the supporting databases or changes to the infrastructure it is running on. Without a regulated pipeline to deliver changes quickly across all systems, organizations expose themselves to unforeseen risks and escalating maintenance costs.

    No matter how new or well-funded an application is today, it will soon become one of many other applications that have to be secured, patched, updated and maintained as part of an organization’s IT estate. For example, consider all those “hot-new” Windows IIS sites built in the early 2000s now sitting on Windows Server 2008/R2, which reached end of life Jan. 14, 2020. Unsupported operating systems are not only costly but they also provide prime targets for cyberattacks, (the Bluekeep Vulnerability being just one of many examples).

    The term “technical debt” was introduced in 1992 by Ward Cunningham to describe what happens when we fail to adequately maintain what he calls “immature” code:

    “Although immature code may work fine and be completely acceptable to the customer, excess quantities will make a program un-masterable, leading to extreme specialization of programmers and finally an inflexible product. Shipping first-time code is like going into debt. A little debt speeds development so long as it is paid back promptly with a rewrite … The danger occurs when the debt is not repaid. Every minute spent on not-quite-right code counts as interest on that debt. Entire engineering organizations can be brought to a standstill under the debt load of anunconsolidated implementation.”

    Organizations Must Navigate Complexity

    The “J-Curve of Transformation” included in the “The State of DevOps, Dora Report” visualizes the challenges large, complex organizations face when automating delivery pipelines. Teams first target high-profile projects and quick wins and then see a drop-off as they’re faced with mounting complexity and technical debt.

    Nowhere is complexity and technical debt more relevant than when it comes to mission-critical commercial-off-the-shelf (COTS) applications Microsoft SQL. Manual controls, layers of process and siloed support teams are all commonly associated with these types of apps. The term “bi-modal” IT is commonly used to describe the variance in delivery processes used for these types of apps versus custom apps being released as part of continuous delivery pipelines.

    Not only are COTS apps handled in this “bi-modal” fashion, but they are also the “forgotten” and the “hidden” applications—the forgotten being legacy applications still in use but no longer being actively developed, and the hidden being all the applications needed by IT to run IT operations. To succeed at continuous delivery, organizations need to apply standardization across delivery pipelines for all teams and environments including complex, forgotten and hidden applications.

    Complex: Mission-Critical COTS Applications

    Large, mission-critical applications such as Microsoft SQL present unique challenges to teams focused on enterprise DevOps, but successful CD pipelines are possible. Teams must first recognize that packaged applications are fundamentally different than custom-built applications. Similar to older applications (which many packaged applications also are), the fundamental challenge is creating an automated way to build packages that can then be consumed by the pipeline. Success here also requires additional intelligence about the underpinning proprietary services and technologies supporting the packaged application.

    Forgotten: Legacy Applications

    There are myriad reasons why older applications present major challenges to organizations trying to achieve continuous delivery: technical debt, lack of resources, required refactoring, operating platform dependencies and a fear of business disruption, to name a few. Updating these apps many times requires individual acts of heroism, involving an abundance of manual efforts in which runbooks are individually updated, shadow sessions are invoked and the impact on dependent systems is typically not known until the update hits production.

    Getting started with continuous delivery requires creating an automated build package that can then be deployed via the pipeline to the preferred destination—cloud, on-premises or hybrid. If the application and all of its dependencies have been successfully abstracted from the underlying operating system and infrastructure and packaged accordingly, it can be consumed into the pipeline and deployed anywhere without having to refactor or rewrite the application.

    Hidden: IT ‘Owned’ Applications

    The Digital Age has led to an exponential growth of applications used by the business (websites, distribution, POS, etc.) as well as third-party applications/tools used by IT teams to build, test, deploy, manage, secure and monitor those applications. In many cases, these applications have become just as mission-critical as the business applications they’re supporting, but they’re not typically treated as “applications” by the business.

    In many cases, they remain completely hidden from the business, with little understanding of the value they provide. Often there is no clear owner for the application, different teams use the application in different ways, updates and patches are handled ad hoc, and expertise is limited to a few individuals within the organization (who take the knowledge about the application with them when they leave).

    Ultimately, third-party applications used by IT need to be managed like any other application. If not, delays in release and production failures caused by changes not properly tested in pre-prod will happen.

  • Buildkite Tightens CI/CD Integration With GitHub

    Buildkite Tightens CI/CD Integration With GitHub

    Buildkite has tightened the integration between its namesake continuous integration/continuous delivery platform and the GitHub source code repository.

    Fred Showell, a senior product manager for Buildkite, said the integration will accelerate application development efforts using the Buildkite application in GitHub. Any connected GitHub account can be selected as a source when creating Buildkite pipelines, he noted.

    Repository webhooks can also be created automatically, making it faster to kick off builds on a Buildkite pipeline from pull requests and then feed build information back into GitHub using the Status application programming interface (API).

    The bulk of organizations that employ Buildkite are software vendors that are continuously updating software-as-a-service (SaaS) applications. Most of those organizations routinely employ software artifacts that are stored in the GitHub repository.

    While GitHub continues to expand its DevOps capabilities in lockstep with its parent company Microsoft, Showell said there’s plenty of opportunity to extend capabilities of frameworks such as GitHub Actions, a set of APIs that DevOps tools can invoke to automate workflows.

    In general, Showell noted the number of SaaS-based platforms continues to expand as organizations embrace cloud applications to drive digital business transformation initiatives. Some of those efforts are now also expanding to include use cases involving edge computing initiatives, which is expanding the reach and scope of the Buildkite platform, he said.

    Of course, many enterprise IT organizations are now applying many of the same best DevOps practices employed by SaaS platform providers to their own application development efforts. Many of those organizations are realizing they are essentially software companies that also manufacture something or provide a service within the context of a specific vertical industry. As such, the number of organizations embracing best DevOps practices should substantially increase in the months ahead.

    Most of those organizations are already trying to manage a large portfolio of applications. Digital business transformation initiatives may allow them to in time retire some of those applications, but chances are good legacy applications will need to be maintained for some time until the digital process that has been put in place proves effective. It’s likely then that sound software development lifecycle management practices will be more relevant than ever.

    Competition among providers of CI/CD platforms to provide those capabilities is already fierce. Fueled by massive amounts of capital, providers of these platforms are expanding the scope of their offering to address a market that is expected to be valued in the tens of billions. Buildkite itself raised $28 million in additional funding last summer.

    It’s too early to say how the next phase of the DevOps platform wars might play out. However, it’s apparent that when it comes to DevOps platforms the level of investment required just to keep pace is substantial.