Prolific advancements in IT, specifically in agile methods and culture over the years, has led to burgeoning needs for a more holistic end-to-end software delivery approach. Agile development is an umbrella term that encompasses numerous incremental and iterative software development practices — including Kanban, SAFe (Scaled Agile Framework), Scrum, Extreme Programming (XP), Lean Development, among others.
Each methodology works on its own unique approach. However, the core commonalities they share lie mainly in their values and vision. The methodologies integrate continuous feedback from iteration to refine and deliver a successful software system. They include various forms of continuous evolution, such as continuous integration, planning and testing, of both software and project. Furthermore, one of the most significant aspects of agile methodologies is their keen focus on empowering workforces to work together and collaborate more efficiently, thus simplifying and speeding up decision-making processes.
Advancement in agile development is what has birthed the DevOps culture. The technology is born from the rising need to adapt to increasing software velocity and throughputs generated through agile methods.
Understanding DevOps
The DevOps market consists of a set of practices, designed to integrate and automate processes between IT and software development teams, enabling them to create, test and deploy software in a quicker and more efficient manner.
The term DevOps, which was coined in 2009 by Andrew Clay Shafer and Patrick Debois, is a combination of the words “development” and “operations.” It is also indicative of a cultural shift in the IT sphere, which bridges the gap between historically siloed teams, such as operations and development.
The DevOps framework essentially adopts a culture where the critical focus is on facilitating collaboration between the business, operations and development teams. Although tools are considered as pillars of the technology, DevOps is designed to deliver continuous value for customers, empower organizations to provide solutions of the highest quality at swift rates, and bring about the automation of the entire business process spectrum, from building to deployment of a product or application.
Shifting Trends Toward Automation in Software Development Processes and the Subsequent Impact on DevOps Adoption in the IT Space
The software development process consists of several integral parts, among which testing is a crucial task. It allows developers to monitor the functionality of the software, monitor and register performance, and detect any aspects that require remediation. However, with software projects becoming increasingly more complex as the years progress, and with project development life cycles accelerating, conventional methods of manual QA (quality assurance) testing fall short in meeting testing objectives within the desired time frames.
Consequently, software developers are shifting their focus toward automated workflows and testing tools, in order to expedite testing routines without compromising on consistency and completeness in QA processes.
DevOps practices are significantly dependent on automation to ensure frequent and timely deliveries across platforms. Studies have shown that nearly 85% of IT leaders claim that automation is a critical aspect of their DevOps strategies. This is attributed largely to the fact that automation empowers infrastructures to successfully field constant changes in codes that are associated with DevOps tools.
Furthermore, automation allows the effortless scaling of environments and unburdens skilled workforces from mundane and repetitive tasks, enabling them to focus on more integral aspects.
Considering a process perspective, automation in testing involves the addition of test automation actions and tools to regular software development regimens. For instance, new builds delivered to a repository can be automatically subjected to automated testing procedures using a number of prescribed tools, with little to no need of developer intervention. The results are recorded with care, then compared to previous test runs and eventually submitted for review to developers.
Depending on the outcome, the software can then be cycled back to the developers for further remediation or approved for deployment. This is especially relevant in terms of the DevOps framework, which relies on continuous integration/continuous delivery (CI/CD) pipelines.
CI/CD pipelines are a prominent outcome of developer operations, allowing developers to validate the quality of the software with minimal intervention, and deploy apps and software to customers in a quicker manner.
CI/CD platform CircleCI, for example, has recently raised over $100 million in Series E funding, in an effort to boost the development of their DevOps product suite. This move is indicative of CircleCI’s proficiency in delivering an automated approach to DevOps practices, including tools for companies to create, test and deliver software.
How Is the Current Pandemic Impacting Digital Development?
The COVID-19 pandemic has asserted a massive influence on the industrial landscape, driving workforces across the globe into remote-only work set-ups. With millions of people working from home for a seemingly indefinite period, enterprises across the globe are rallying to accelerate automation efforts in order to more efficiently deliver digital business applications to engage their employees from remote workspaces.
According to various studies, nearly 73% of IT operations and DevOps teams are working toward maintaining and expediting initiatives for digital transformation throughout the current crisis, holding the belief that the value of digital offerings is set to witness a positive growth trajectory in the social distancing era.
DevOps industry players such as JFrog are working toward providing enterprises and teams with access to cloud-based DevOps technologies, free of cost, particularly those which are undertaking active research and response efforts to combat the COVID-19 pandemic. These DevOps technologies include Pipelines, Xray and Artifactory, among others.
While the exact impact of the global crisis on the DevOps environment in particular and the IT space in general is yet unclear, it is becoming more prominent that the strong focus on the prevention of such crises in the future — both in and out of the IT realm — is likely to set a great precedent for the growth of the DevOps industry in the years ahead.
GitLab today announced it has acquired Peach Tech, a provider of protocol fuzz testing and dynamic application security testing (DAST) API testing tools, and Fuzzit, a continuous fuzz testing tool, as part of its efforts to advance the adoption of best DevSecOps practices.
David DeSanto, director of product for GitLab Secure & Defend, said the two acquisitions should make it easier for DevOps teams to incorporate both whitebox and blackbox fuzz testing techniques for security testing much earlier in the application development and deployment process.
Fuzz testing is an automated technique that involves providing invalid, unexpected or random data as inputs to software, which is then monitored to see what happens—if it, for example, crashes or a memory leak is created.
Peach Tech adds Peach Fuzzer, an automated security testing platform that employs definition files known as Peach Pits to generate the fuzzed data consumed by the test target along with a framework for automating the web application programming interface (API) security testing process.
Once Peach Tech and Fuzzit technologies are fully integrated with each other and the GitLab platform, DeSanto said GitLab Secure customers will be able to automate myriad tasks, from security testing to vulnerability management and remediation.
GitLab will also employ technologies from both companies to further its ambitions to drive the adoption of interactive application security testing (IAST). The goal is to not only make it easier for developers to employ DevSecOps tools but also understand what issues are being created as applications are developed, said DeSanto. That approach should reduce the number of instances where development teams are continuously making the same mistakes across multiple application development projects, he noted.
The further left cybersecurity testing is shifted the less stress there will be on chronically short-staffed cybersecurity teams. The challenge is that while most organizations recognize the potential benefits of DevSecOps there hasn’t been much progress in educating developers on what issues to look for and providing them the tools needed to discover and remediate vulnerabilities.
There is, of course, no silver bullet when it comes to cybersecurity. As long as humans write code there always will be potential issues. However, many more routine cybersecurity issues should be addressed long before an application is deployed within a production environment. To achieve that goal organizations are providing developers with tools that identify cybersecurity issues as they write code in addition to incorporating more security testing into their CI/CD workflows.
Of course, GitLab is not the only provider of a CI/CD platform focusing on how to foster the adoption of best DevSecOps practices. There’s clearly a race to embed security testing tools within CI/CD platforms that should advance adoption for DevSecOps, if for no other reason than they are becoming easier for developers to discover and employ.
CI/CD pipelines are at the core of daily operations for many businesses today. These processes, when set up correctly, help to keep the delivery process consistent by automating many manual tasks and providing visibility into how the software is being worked on.
DevOps is also the place in your technology stack where your infrastructure has access to many different resources, from development and production environment to analytics keys and code signing credentials. The more resources your pipelines have access to (secure secrets, proprietary code, databases, etc.), the more important it is to keep your CI/CD system secure.
In this article, we talk about the different parts of DevOps security for CI/CD pipelines and highlight a number of ways to improve on these aspects. We usually divide security practices into three parts that you can address with different kinds of solutions:
Secure pipeline configuration
Code and Git history analysis
Security policy enforcement
All three aspects are equally important. Let’s take a deeper look at these categories and ways to address them.
Secure Pipeline Configuration
It is possible to use your CI/CD pipeline configuration to make security issues less likely to happen.
First, safely store secrets that you use in your pipelines for connecting to databases and third-party services. Depending on the CI platform you use, you may have the option to use encrypted-at-rest environment variables, or to use the contexts feature. Contexts are used to provide access to environment variables across projects. Their use can also be restricted to specific security group members as defined by the organization’s administrator.
Another option is to use a third-party solution to dynamically fetch secrets from their secure storage for your jobs. To ensure that none of your credentials become exposed to those who shouldn’t see them, you should never check these into your repository in plain text, even if the repository is private.
Second, for sensitive files like code signing keys, I recommend adding an additional layer of isolation between your encrypted files and your repository for added security. Keep them encrypted in your repository and store the decryption key in the environment variables or contexts, only decrypting them inside your CI/CD jobs when they are needed. This makes it even more difficult to expose or leak these highly sensitive files.
Third, you never want to leave your CI/CD environment running without monitoring it. Therefore, make sure that the containers and VMs used for your pipelines are destroyed after jobs that require sensitive information finish running.
Fourth, pay extra attention to how your CI/CD system handles builds for pull requests that come from forks of your repository. If your build stages require secrets to work, forked pull requests might gain access to them through your pipelines.
By addressing these four aspects you can significantly reduce the risk of security incidents due to your CI/CD pipeline configuration. Next up, code and Git history analysis.
Code and Git History Analysis
We all love Git for its comprehensive history of changes, being able to look through the history of the project with a simple CLI command and see who has most recently changed a line of code, among other benefits. This also means that any sensitive information that makes it into the Git history can be accessible to attackers if a Git repository is exposed, even if the most recent state of the repository doesn’t contain the secrets anymore.
Trufflehog and GitLeaks are two tools that can help you identify secrets that have been committed to the codebase so that you can deactivate and replace them. These two tools will also scan your Git history for traces of secrets that your team might have added to the repository in the past.
Once your Git history is clear of secrets, you can proceed to the next level: making sure that your current revision doesn’t contain any vulnerable dependencies.
Static Application Security Testing (SAST) techniques can look through the application in your commit and analyze its dependencies. If any dependencies contain any issues or known security vulnerabilities, your commit will be marked as insecure and won’t be allowed to proceed to deployment.
Dynamic Application Security Testing (DAST) techniques go one step further and spin up a copy of your production environment inside your CI job in order to scan the resulting containers and executables. The dynamic aspect helps the system catch dependencies that are being loaded at launch time, for example, as those won’t be caught by SAST.
Security Policy Enforcement
Some security aspects can’t be statically checked based on known vulnerabilities, but rather are specific to your particular company, and therefore need to be codified as policies. These can take the form of automated or manual compliance checks.
Certain tasks such as reviewing the list of all accounts that have access to your repositories, or making sure your onboarding and offboarding processes are in sync, will be manual. We recommend that you set reminders to perform these regularly.
Some tasks actually can be automated quite easily. For example, third-party services can provide a convenient way to codify a set of rules that will be matched against in your CI pipeline, e.g., proving compliance with the regulations that govern your data. If there is a mismatch, the build will fail.
Some of the Important Parts Can’t Be Automated
Keeping your CI/CD pipelines secure is more important today than ever before as automating your CI/CD practices requires more access to development and production environments, analytics keys and code signing credentials than in the past. The hard thing about security is that if you only get one of the three categories right, you won’t be 33% secure. You will be 0% secure until you get all the bases covered.
With the approaches outlined above, you’ll be in a much more secure position than you would be without them. However, some of the aspects that matter the most for CI/CD security can’t be automated or detected with an automated policy. These aspects are shaped by the behavior and culture within your team.
A DevSecOps Mindset
Your team is your most valuable set of security researchers because they know your applications best and constantly work on them. Have you implemented a clear process for your teammates to report security issues if they find any, and do you dedicate time to fix the security flaws they find? Do you celebrate those who report issues openly? In many companies, the answer isn’t an immediate “yes” to these, and it should be.
How long does it take for someone on your team to report suspicious activity? Do they need to post in multiple Slack channels to get attention, or is there a streamlined process to get an issue like this looked into? How do you treat those who report false positives — do you blame them for wasting the team’s time, or do you hold a blameless post-mortem so that everyone on your team learns from the mistakes? All these questions can make or break your security-minded development culture.
Companies today are under heavy pressure to deliver software faster and with more quality than their competitors. A set of good features and their reach to customers can increase only when the company releases software updates frequently. As a result, organizations are embracing DevOps and CI/CD approaches to improve their ability to plan, build, test and release applications and features to market quickly with quality and at scale. IDC predicts the worldwide DevOps software market will reach $8 billion by 2022, up from $3.9 billion in 2017.
The buzz around continuous integration, continuous deployment and continuous delivery is more than ever today, but in practice, the actual usage of CI/CD seems to be less. It can be because of the shortage of skill set required in DevOps or companies still practicing traditional software development methodologies. Enterprises are not understanding the amount of money they are leaving on the table by missing CI/CD and DevOps automation practices.
For many companies, DevOps is something they have to try because their competition is doing so or because it is a trend. This is a desperate attempt without understanding the consequences and basic understanding of DevOps principles.
For the most part, it is a disparate set of tools that customers have to connect together for their software delivery that creates chaos around. On the other hand, it is difficult to choose the right tools and understand what are the best practices. That’s why we are starting to see a trend of DevOps companies delivering a unified toolset so that customers have a complete solution out of the box —without having to worry about finding many different solutions from different vendors.
What Is CI/CD?
Whenever a developer commits and pushes changes to the repository, the CI system will rebuild the branch and run all related test cases to verify that new changes won’t break the existing application and functionalities.
Continuous delivery is an extension of continuous integration that helps you deploy your application with a click of a button. It means deploying changes every time they pass CI tests. With this, you can easily automate your build and testing phase and most of your release process as well.
With continuous deployment, developers will be able to release or deploy software automatically into the production environment. In this stage, no manual code check-ins happen into the app. Code is automatically tested through the CI process for issues, and if none are found, then the code is deployed.
Why CI/CD?
CI/CD serves as the best practice for testing and deployment of applications. For the purpose of quality assurance, making CI/CD the single channel to deploy code is the best practice. DevOps teams should ensure that everything is going through the CI/CD pipeline so that the code in testing and production are the same. This will also reduce human error and avoid deploying other versions not being tested by mistake.
One of the biggest reasons for companies to practice CI/CD today is faster software delivery, which leads to more frequent updates to customers. It makes them feel happy and a constant improvement with frequent releases will ensure enhanced customer satisfaction. This is a competitive advantage because it allows companies to deliver new features more often. It can help companies stay ahead of competitors.
Passing code through the CI/CD pipeline ensures that any changes made adhere to the established standards before moving to the next stage in the delivery pipeline. Any failure or discrepancy prevents the advancement of such changes to the next step until it is rectified. This simply means this approach of CI/CD ensures that bad code does not get into the production. CI/CD promotes the fail fast, learn fast and deliver fast approach. When you have automated CI/CD in place, the code is run through the CI/CD process constantly with each commit by developers which allows bugs to be caught quickly — it is easier to isolate the bug since each small change is tested separately.
What Is the CI/CD Pipeline?
A CI/CD pipeline is a deployment pipeline workflow that is integrated with automation. When conducted well, it will minimize manual handoffs and errors to enhance the feedback loops throughout the software delivery life cycle, allowing development teams to deliver smaller chunks of releases within a shorter time.
The CI/CD pipeline is now mainstream among software companies, and it has become the backbone of the modern DevOps environment. While DevOps lays principles and standards related to the culture aspect, CI/CD focuses on the automation, flow, process and tools. CI/CD scenario has changed with the introduction of containerization and microservices. Now, every cloud vendor has support and integration with tools such as Docker and Kubernetes that help the CI/CD pipeline to scale.
Stages of the Continuous Delivery Pipeline
Test Automation and CI: This is the important stage where the new features and prototypes are prepared, coding is done and then the code is integrated with the base. With the use of a CI server, the unit tests and packaging is done.
Build Automation: This is the stage where the artifacts are pushed by developers (i.e., Dependencies, Docker image) to the registry (i.e., Amazon ECR or Docker hub or any container registry).
Alpha Deployments: In this stage, developer checks-in changes. It is the place for developers to carefully see the changes they’ve made and how they interact with each other and affect other systems. Also, here developers can be able to see if they broke anything in the master branch.
Beta Deployments: This is the stage where testers carry out manual testing. Manual testing is still required to see the perfection of CI tools. This gives the DevOps team the guarantee that the condition of the application is good and will work definitely as desired when deployed.
Production Deployments: Here comes the final stage, where the app goes live.
What Is the Market Saying About CI/CD?
The search volume has gradually increased in the world for the keyword “CI CD Pipeline” on Google.
When Mabl, a test automation company surveyed 500 software testers, 53% of the people said their team uses continuous integration, while 38% embrace continuous delivery and only 29% have adopted continuous deployment.
A DevOps related survey from Codefresh showed that a third of companies had automated more than half of their workloads, but only 1% were all the way there. Almost no one has automated the entire CI/CD process. A plurality of engineering teams have automated less than 10% of the process.
The iOS Developer Community Survey is the largest survey of Apple platform developers ever undertaken, where 2,290 people filled in the questionnaire in which CI/CD questions were also a part of the survey.
Below are the results that showed a majority of developers are practicing CI/CD when it comes to business apps.
Figure 1: Do you practice continuous delivery for your personal/hobby apps?Figure 2: Do you practice continuous delivery for your business’s apps?
The way forward: Companies will not be able to fully take advantage of containers and DevOps unless they embrace the use of a proper deployment pipeline. Companies will start offering pre-built pipelines to customers, something like JFrog pipelines. There is a need for a unified platform that has all the tools needed to support a CI/CD system — automation (Pipelines), binary repository management (Artifactory), security (Xray), release/deployment (Distribution), etc. JFrog provides a unified platform.
Codefresh conducted a survey and published the results of the survey of 130 developers at the KubeCon + CloudNativeCon Europe 2019 conference — and the results were a bit shocking. It showed that nearly a third (32%) don’t use any continuous integration or continuous delivery (CI/CD) platform at all. Sixty percent of the survey respondents said their organization has yet to take full advantage of automation to accelerate software development and deployment.
The Market Radar Report by GigaOm is a recent research on the enterprise CI/CD solutions where the author compares the leading CI/CD solutions in the industry and rates them depending on different valuation criteria. According to the results, JFrog, Codefresh and GitLab are outperformers as shown below.
The CI/CD solutions considered were AWS, CircleCI, CloudBees, CodeFresh, GitLab, IBM, JFrog, Micro Focus, Microsoft, Red Hat and Xebia Labs. You can download the free analyst comparison report here: GigaOm Radar For Enterprise CI/CD.
CI/CD is one of the DevOps best practices because it ensures proper alignment between developers and operations. While it enhances automation, developers can push code changes more reliably and frequently. At the same time, the operations team can see greater stability because environments have standard configurations, and rollback procedures are automated. To win the CI/CD game, both dev and ops teams need to develop consensus on the right tools and approaches that are beneficial for the business growth in the long run.
Cloud and DevOps Go Hand in Hand
You can’t do DevOps without the cloud. DevOps won’t have much value without it, and the cloud won’t do much without DevOps.
Cloud computing has many definitions, the fundamental explanation of the cloud can be defined as a system that enables and supports the provisioning of infrastructure. Thus, it allows it to be described as code or templates, facilitating the creation of repeatable processes. That is also the basic principle of DevOps, automating the processes/tasks in the software development life cycle to the fullest extent.
One of the biggest benefits of cloud for CI/CD is the dynamic nature of cloud infrastructure. This is perfect for the ephemeral and bursty nature of CI/CD workloads. Cloud resources can automatically scale up and down based on CI/CD workloads. This allows huge management and cost benefits for enterprises. Enterprise companies don’t need to have their own servers to maintain, they need to increase these servers as CI/CD workloads increase or waste server resources when they are idle.
Regardless of how your organization defines the cloud technologies and, despite which DevOps routes you travel to achieve digital transformation with the cloud integration, the aesthetics of cloud technologies and services can be easily amalgamated in the DevOps dictionary because they radically complement DevOps principles and processes.
For DevOps implementation, Agile principles serve as a driver, as there are shorter release cycles and automation to deliver fast and achieve the deadlines. To implement continuous practices, cloud computing is a great facilitator, since resources are available quickly for all the activities involved in development and application lifecycle management. Agile principles, along with cloud computing and DevOps practices, enable high developer productivity and faster growth in organizations.
With secured cloud gateways, users get the ability to access enterprise resources easily from anywhere and with any device, without compromising on the security of the network. Letting users access your network from their devices might sometimes seem scary, but the modern cloud platforms have much more to offer, they come with built-in security stacks to help provide administrators with the tools they need to manage and support security.
DevOps coupled with cloud tech has enormous benefits for businesses employing them:
Increased developer productivity and collaboration.
Faster time to market through quick and easy access to development environments.
Help to automate different repetitive tasks so that developers can focus on the tasks that matter.
Cloud complexity and system maintenance can be incredibly reduced by automation and infrastructure as code approaches.
Increase security with modern cloud platforms and principles that serve to eliminate the manual error.
Help to eliminate downtime through cloud-based continuous business operations. Developers can build stateless applications with automation, which will increase the availability and failover ability, along with increasing business reliability and improved customer satisfaction.
Scale to new heights — one of the main reasons businesses employ cloud and DevOps is to scale their infrastructure and applications as and when needed.
Cloud Deployment Models
When the cloud provider entirely hosts organization’s applications and data, it is a public cloud. With just having an internet connection, the employees of the company employing public cloud can access the applications.
A private cloud is when the resources of an organization such as storage, compute and networking resources exist in a provider’s data center. Companies prefer private cloud mainly because of security and for storing highly sensitive data when it comes to running applications.
One more modern approach is the hybrid cloud; it is a combination of private and public cloud services. According to the traffic and demand considerations, organizations can make the change. Some customers may have on-prem CI/CD systems. Some of these CI/CD systems can expand their workloads to the cloud in a hybrid setup. This can allow them to realize the benefits of cloud now and not have to do a complete migration.
Mainly, when there is high demand, workloads can be run in the public cloud, and then when things get back to normal, return to the private cloud. This approach reduces the overhead cost spent on cloud resources. Additionally, the key point is, sensitive information, data and critical applications can be kept in the private cloud. In contrast, less-sensitive data and applications can be stored and run in the public cloud.
Choosing DevOps Cloud Solutions
If cloud is like an instrument, then DevOps is the musician who plays it.
Cloud computing and DevOps play a vital role and are interlocking parts of the IT strategy. Together, cloud and DevOps help IT shift its emphasis from “How far and long can we go without an outage?” to “How often we can deliver new and stable functionality/feature?” or “How fast/quickly can we deploy a new service?”
Businesses today are looking for ways to digital transformation with the help of the cloud, DevOps and automation practices. Considering the competition in the cloud space, it is overwhelming to see so many vendors and get confused on what to choose.
Here, in the State of DevOps Cloud Solutions 2020, you can see a high-level overview of the cloud DevOps platform solutions available today, and what you should expect from vendors in your cloud DevOps journey.
Clyde Seepersad, general manager of training at the Linux Foundation, said that while the foundation already provides access to some higher-level conceptual DevOps courses, the launch of the CDF last year provides a knowledge base from which the Linux Foundation can now provide online courses specific to DevOps tools. The CDF is an arm of The Linux Foundation.
The Linux Foundation will also provide courses on Spinnaker, a CD platform, and other DevOps tools that, along with Jenkins, are now being developed under the auspices of the CDF, he added.
In general, Seepersad said demand for DevOps expertise is increasing as organizations more aggressively embrace digital business transformation initiatives in the wake of the COVID-19 pandemic. At the same time, IT teams are adopting a wider range of cloud-native technologies to build more flexible and resilient applications. DevOps platforms such as Jenkins make it possible for IT teams to build those applications faster.
The challenge many organizations underestimate is the extent of changes to their internal IT culture required to make that transition, Seepersad said—too many organizations still put tools into the hands of developers without really thinking through the workflow issues between developers and IT operations teams.
The most important thing for any organization to do, he said, is to make sure there is an open DevOps dialogue versus tiptoeing around issues that will tend to increase resistance only further.
Ultimately, free training should help expand the overall size of the DevOps community at a time when many IT professionals are trying to increase their skills. DevOps professionals not only command higher salaries, but they are also less likely to be laid off. DevOps teams oversee the development and deployment of applications and can also programmatically manage levels of infrastructure that IT administrators relying on traditional graphical user interface (GUI) tools can’t match. That’s a critical issue when many IT organizations these days are trying to do a lot more with even less.
There’s nothing quite like an economic crisis to focus everyone’s mind. The biggest DevOps challenge any organization typically faces is overcoming inertia. Too many IT professionals have relatively narrow skillsets that conspire to increase the total size of the IT organization. In addition to relying on increased levels of automation to reduce costs, IT leaders want to be able to scale IT environments without having to increase headcount.
It may take a little while longer for the transition to DevOps to fully play out. However, it’s now more of a question of existential survival rather than if or even to what degree to embrace DevOps. Organizations that don’t embrace DevOps processes to ruthlessly automate the management of IT now might very well never get another chance.
Despite most developers and managers being well aware of the concept of DevSecOps, it is still often confused with a number of related processes and concepts. This is particularly true for the Department of Defense (DoD) contractors because they have long been encouraged to use a related process known as agile development.
Since agile development has been around for more than a decade, there is a tendency for organizations to regard DevSecOps as an extension, or even a synonym, for it. In reality, though, the two approaches are distinct. The objectives of both approaches are similar: balancing speed and agility, detecting risk early and both focus on cloud-native security and performance.
However, while DevSecOps does build on some agile development principles, such as the continuous integration and delivery of software systems in cycles, its key emphasis from the beginning of the process is to integrate security features, while Agile only focuses on delivering software.
In this article, we’ll look at the key differences between agile development and genuine DevSecOps processes, and how each complements the other.
DevSecOps vs. Agile Development
The difference between DevSecOps and agile development methodologies can be understood in reference to one aspect of software development: security. When, where and who implements security in software development varies between the two approaches.
Agile development methodologies focus on iterative development cycles, in which feedback is continuously reintegrated into ongoing software development. However, even in mature agile development processes, security is still often added to software as an afterthought. This should not be read as blaming software developers for often underestimating the potential harm from malware or overlooking the importance of cybersecurity.
Rather, in many firms, it is simply not the responsibility of developers to think about the security implications of their code, because software will be passed to the security team before release.
DevSecOps takes security and puts it on the same level as continuous integration and delivery. DevSecOps methodologies emphasize security at the very earliest stages of development and make security an important part of overall software quality.
In essence, these approaches shift the program manager’s perspective away from making sure that software is in compliance or meets a specification or audit, to ensuring that the code is written correctly and securely and that it’s deployed in a repeatable manner.
The Changing Culture of the DoD
The adoption of DevSecOps methodologies is critical for DoD contractors because these methodologies reflect DoD’s own modernization strategy. Following high-profile vulnerabilities being discovered in all types of systems, DoD seems to have released that cybersecurity weaknesses are not just a potential threat to critical defense data, but also impose a significant operational bottleneck on the continuous delivery of functionality.
At the moment, even those firms who possess a mature agile development framework can find that the multiple security checks and compliance processes that must be performed on software, prior to shipping, can detrimentally impact on their ability to deliver iterative software improvements. This is the issue that the transition to DevSecOps seeks to overcome, by attempting to shift organizations’ approach to cybersecurity away from compliance, and toward genuine security consciousness.
At the same time, analysts are aware that shifting responsibility for cybersecurity onto developers will be uncomfortable, and potentially dangerous, for many DoD contractors. Developers working on the base code for military software do not have – and are not expected to have – an exhaustive appreciation of all of the contexts in which this will be deployed, and exactly which systems their software will be required to interface with.
Automation and New Models
In response to these concerns, DoD contractors are reappraising the DevSecOps model and thinking seriously about how it can be deployed in contexts where continuous service delivery is key. There are three key ways in which this is happening.
The first is the rise of automation. The types of large-scale, multicloud infrastructures that most contemporary DoD projects rely on can often necessitate repetitive, ongoing maintenance and security assessment across both development and security teams. DevSecOps replaces these human checks with automated systems: Instead of requiring a person to go through checklists of hundreds of controls, this is done automatically as part of the software development and supply process pipeline.
The second key element in the transition to DevSecOps is endpoint security. The types of monolithic development processes that DoD contractors traditionally worked with are not suited to the contemporary deployment environment of many DoD projects. In these traditional methods, systems were built as discrete holes, with no expectation that data would be exposed as it moved between components.
This is now an obsolete approach: In DevSecOps processes, virtual private networks are used to provide security and encrypt data as it moves within a network, and the principle of least privilege reduces staff access to small portions of It environments.
Thirdly, a new generation of younger military and civilian personnel in the DoD, many of them trained in the commercial sector rather than entering the military directly from college, are bringing corporate approaches to the development of military software. This is seen in the exponential increase in the utilization of cloud services in DoD applications, and more dynamic and distributed managerial frameworks that distribute security responsibility across entire organizations, rather than having this as the sole remit of a dedicated security team.
Get Support
Moving to a truly DevSecOps methodology is likely to be a challenge for many contractors. However, just as in the last decade many firms have had to redesign their development lifecycles in order to be agile, now they will need to achieve secure development lifecycles in order to stay competitive.
If you are looking to make this shift, you should also recognize that you are not alone. DoD is working actively with its partners in order to facilitate the roll-out of DevSecOps processes and to make commercial tools and techniques available and useful for military contractors because the department recognizes the value of this methodology for the software they commission.
This is seen most clearly in the recent updates to Microsoft Azure Government, though you should also be aware that many of the resources recently released by the Defense Logistics Agency are also focused on the transition to DevSecOps.
In short, whilst the transition to DevSecOps may be challenging, it will be no more so than the earlier shift to agile development. To make this shift, however, firms should draw on the resources available to them, recognize the value of integrating security into their development workflows and build on existing agile frameworks.
Suzie Prince, head of product for Bitbucket at Atlassian, said the latest updates are intended to enable DevOps teams to streamline workflows without requiring everyone in the organization to standardize on the same set of tools.
Updates to the Atlassian DevOps portfolio include DevOps Automation Triggers, which makes it possible to create automation to keep development activities synchronized, such as reassigning issues surfaced in the Jira project management application from Atlassian to code review or sending a message to a Slack channel.
The Bitbucket repository from Atlassian has also been updated to make it easier and faster to review code changes after a pull request. New features such as a consolidated list of tasks, integrated Jira issue creation and activity feed filters make it possible for developers to complete their code reviews faster and juggle multiple pull requests at once, said Prince. A Your Work dashboard in Bitbucket Cloud has also been expanded to include assigned Jira issues.
The Atlassian VS Code integration is also being made available to make it easier for developers who use the Microsoft integrated development environment to more easily track and visualize Bitbucket Pipelines.
A Code Insights in Bitbucket Cloud has also been added to bring scanning, testing and analysis tools into the code review process, including Mabl for test automation and Sentry for automated monitoring. Bitbucket is also now providing integration with Snyk scanning tools to make it easier to discover vulnerabilities.
Available in a preview mode is an automated change management capability across Jira Service Desk Cloud and Bitbucket Pipelines that can be used to pause CI/CD processes, create a change request in Jira Service Desk and trigger deployment once it is approved.
A Risk Assessment Engine in Jira Service Desk Cloud also allows IT teams to score the risk of a change and automatically approve and deploy low-risk changes. A Change Management View has also been added to Jira Service Desk Cloud to streamline approvals for high-risk changes by pulling all of the relevant information together in traceable change requests.
Atlassian has also tightened integration between its Opsgenie incident management software and Bitbucket Cloud to centralize alerts and filters. Via an Incident Investigation Dashboard, members of a DevOps team can look at the deployment that happened right before the incident, add it as a potential cause, and contact the developer who made the change if it needs to be rolled back.
The company has also extended integrations with Slack and Statuspage to facilitate both members of DevOps teams and external stakeholders.
Finally, Atlassian is making available a resources hub to make it easier to share DevOps best practices, templates and guides.
Prince said it’s not ever going to be feasible for most organizations to standardize on one set of DevOps tools. The goal should be to enable organizations to construct DevOps processes in ways that allow developers to employ whatever tool is their favorite at the moment. Only then will organizations get developers to buy into a DevOps process that doesn’t necessarily change the way they are most comfortable working, she noted.
The challenge, of course, is often getting IT leaders to be as equally comfortable with what, from their perspective, might be an experiment in controlled chaos.
CircleCI today in partnership with Plandek launched an analytics plugin dubbed Buildtrakr for its namesake continuous integration and continuous delivery (CI/CD) platform. Buildtrakr integrates with CircleCI and Git to synthesize commit and build performance data, with critical metrics being surfaced via a dashboard.
In addition, CircleCI has revamped its user interface and reconfigured a new fleet of Docker images, also known as CircleCI convenience images, to optimize them for a continuous integration environment.
CircleCI CEO Jim Rose said Docker images made available by CircleCI are faster, easier to download and provide a more efficient way to build custom images. CircleCI also released a new category of convenience images available for languages such as Python, Ruby, Rust, Go and several others, that come with improved documentation, more stability and more efficient infrastructure. Those convenience images are individual open source projects and can be used in IT environments other than the CI/CD platform from CircleCI.
Finally, the company also announced that its insights endpoints now offer start and end date parameters for workflows and jobs. These parameters restrict the range of executions returned based on their start times in addition to providing a new filter to use for testing analysis.
Rose said the new user interface is significant because it makes it possible to view the status of entire pipelines more easily while making it easier for more advanced DevOps teams to leverage workflows.
The redesigned interface also adds a filterable view with nested workflow and job information, making it easier to scan the status of recent builds. Pages were also overhauled to deliver more digestible job details, full-screen step outputs for easier debugging and a cleaner configuration-as-code view. There’s also been a 65% reduction in page load times, noted Rose.
In general, Rose said activity on the CircleCI platforms in cloud environments is on the rise in the wake of the COVID-19 pandemic. Organizations that are working remotely due to stay-at-home directives and can’t access a local data center have come to realize that having a build box located under someone’s desk is no longer a viable option.
Of course, it remains to be seen how much of that initial burst of activity will result in new applications being deployed. However, with more organizations starting to build and deploy digital business applications to engage customers, it may be only a question of when IT operations teams will be able to catch up to a building backlog of applications. In fact, many of those applications are now likely to be deployed on the same cloud where they were constructed.
Whatever the path forward, there most certainly will be a greater reliance on metrics. Naturally, the hope is that analytics will be used to coach DevOps teams to become more efficient versus simply riding roughshod over developers. After all, the goal isn’t to write more code faster but to deliver better code at the right time.
In addition, GitLab announced it is making generally available Gitaly Clusters, which enable DevOps teams to create a warm replica of a Git repository.
Brendan O’Leary, senior developer evangelist at GitLab, said with release 13 GitLab continues to build on a single code base that allows the platform to be managed and accessed as an integrated set of applications rather than disparate components that DevOps teams need to pull together and then manage.
In terms of core DevOps capabilities, the latest release adds the ability to customize the Value Stream Analytics module to specific workflows. GitLab is also planning to make it possible to visualize stages of a workflow.
GitLab is also improving traceability between requirements and has added the ability to visualize the workflow associated with a specific release. A future release will enable DevOps teams to employ feature flags to conduct both A/B testing based as well as create feature flags from merge requests and to filter feature flags by status, according to the company.
Other capabilities added include more customizable dashboards, with security dashboards now being shareable outside of GitLab to foster collaboration with IT security teams. Future releases will add Kubernetes Clusters to the dashboard to show all clusters and pods in use.
DevOps teams now can route existing monitoring alerts to GitLab for triage and incident resolution. At the same time, GitLab has moved design management to the core platform.
GitLab is also now making AutoDevOps, which provides pre-defined CI/CD pipelines, available on Amazon Web Services (AWS).
In terms of cybersecurity, GitLab has revamped the way the platform manages vulnerabilities to make it easier in a future update to prioritize remediation efforts.
GitLab also has static application security testing (SAST) support for .NET frameworks and has expanded support for offline environments. There’s also now support for dynamic application security testing (DAST) for REST application programming interfaces (APIs) and access to full commit history scan for secrets. Fuzz testing support will be introduced later this year, according to the company, along with an ability to secure secrets across services. GitLab also plans to add integration with security tools that track container behavior analytics, as well as an out-of-the-box container network policy.
Thanks to GitLab’s collaboration with Okta, there is now also tighter integration with Okta’s System for Cross-domain Identity Management, which can be provisioned for a GitLab group.
In terms of compliance, GitLab also plans to enable DevOps teams to establish a compliance framework, adopt its regulatory controls and simplify audit reporting. The company is also working on an initial security policy user interface to simplify setting up security guardrails.
Finally, GitLab has become a Common Vulnerabilities and Exposures (CVE) ID Numbering Authority (CNA). GitLab users can request a CVE from GitLab, either for GitLab itself or for any project hosted on GitLab.com. In the future, users can request CVE IDs directly from within GitLab.
O’Leary said GitLab is working toward increasing developer confidence in their security skills. By making it easier to embrace best DevOps practices, developers no longer will have to tiptoe around security issues that they today largely have little to no familiarity with, he said.
In general, O’Leary said GitLab’s biggest challenge is getting organizations to move beyond simply using it to manage source code. Once organizations do make their first foray beyond source code management, however, it’s not too long before they are employing a wide range of modules, he noted.
Of course, encouraging organizations to dive deeper DevOps has as much to do with the culture of the organization as it does the capabilities of the CI/CD platform employed.
Moritz Plassnig, senior vice president cloud and general manager for SDM at CloudBees, said these integrations with Google Cloud Build and Tekton pipelines are being provided as part of a long-standing alliance between the two companies to advance the adoption of best DevOps practices. The CloudBees SDM service runs on the Google Cloud Platform (GCP).
As part of that effort, CloudBees has been making investments in an SDM platform that enables DevOps teams to pull metrics from any tool or platform. That data is then fed into an analytics cloud service through which CloudBees surfaces insights that identify, for example, how delays to a project might impact overall revenue goals.
Interest in software delivery analytics has risen in the wake of the COVID-19 pandemic as organizations accelerate the building and deploying of digital business applications, said Plassnig. With employees and customers alike working from home to combat the pandemic, organizations are investing more in creating applications that provide digital alternatives to engaging end users, he noted, adding delays to any of those projects are likely to have a much bigger impact on business continuity strategies today than they did just a few short months ago.
The CloudBees SDM service provides DevOps teams and business managers alike a real-time view of all product and feature development occurring across the organization, as well as a policy engine to trigger actions to better optimize the software delivery process. Organizations can also model and visualize the software delivery process while measuring overall DevOps performance.
Rather than requiring organizations to standardize on a single platform to achieve that goal, Plassnig said CloudBees is leveraging application programming interfaces (APIs) to integrate metrics across a broad range of DevOps tools. That approach, he said, makes it possible to meet organizations wherever they are in terms of DevOps maturity.
Support for Tekton pipelines is significant, Plassnig noted, because it’s expected that most continuous integration/continuous delivery (CI/CD) platforms in the future will be built using Tekton pipelines, which provide an open source framework for building platforms that are designed to be deployed on Kubernetes clusters. Jenkins X, originally developed by CloudBees, is one of the first CI/CD platforms to support Tekton pipelines, which were originally developed by Google. Jenkins X and Tekton pipelines are now being advanced under the auspices of the Continuous Delivery Foundation, an arm of the Linux Foundation.
As more CI/CD platforms embrace Tekton, the platforms themselves will become less of a black box from an analytics perspective, noted Plassnig.
In the meantime, CloudBees also expects Google to continue to gain momentum as a cloud service provider, he said. While Google trails Amazon Web Services (AWS) and Microsoft in terms of overall market share by a wide margin, Plassnig noted that with the arrival of Anthos, a cloud operating system capable of being deployed on multiple cloud and on-premises IT environments, it’s only a matter of time before services such as Google Cloud Build are employed more widely.
Whatever the platform used, the one thing for certain is the rate at which software is being generated and deployed on various platforms will be watched much more closely.
Fuzz testing is an automated technique that involves providing invalid, unexpected or random data as inputs to software, which is then monitored to see what happens—if it, for example, crashes or a memory leak is created.
