devopsy.kinsta.cloud

Tag: CI/CD

Relyance AI Shifts Compliance Left Using ML Algorithms

Relyance AI emerged from stealth this week to unveil a namesake platform for managing privacy and data governance in real-time within the context of a larger DevOps workflow.

Fresh off raising $30 million in seed capital, Relyance AI co-CEO Leila Golchehreh said organizations can now quickly and accurately answer essential privacy questions and ensure compliance and governance by mapping requirements directly to their codebase. The platform provides transparency and visibility all the way down to the source code in real-time using machine learning algorithms, she added.

Relyance AI co-CEO Abhi Sharma also noted most legal and data protection teams today have limited visibility into digital operations. Developers and data scientists are constantly creating and updating applications. By the time compliance teams gather the data required, it’s obsolete, said Sharma.

The Relyance AI platform analyzes contractual requirements against the operational reality as determined by data processing in real-time. Compliance teams can then address privacy issues as they arise, added Sharma.

At the core of the Relyance AI platform are five integrated components that first inventory assets by connecting to application programming interfaces (APIs); generate universal records of processing activities (ROPAs) by scanning code and understanding how data is being processed; share alerts when issues are detected; provide views into data life cycles to highlight data subject access request (DSAR) exceptions and extensions and address vendor life cycle management to ensure third-parties are complying with policies.

Sharma said compliance, much like security, is shifting further left toward application development as organizations look to eliminate issues before they arise in a production environment. The Relyance AI platform is designed to be integrated with continuous integration/continuous delivery (CI/CD) platforms that are the core of modern software development life cycle processes, he added.

Undoubtedly, as the worlds of compliance and application development collide, there will be cultural challenges similar to what is now occurring with the rest of DevSecOps best practices that require more collaboration between developers and security teams. The challenge is compliance officers tend to be even further removed from application development than their IT security counterparts.

However, it’s now only a matter of time before compliance becomes code. Modern applications already make use of the Open Policy Agent, for example, to embed compliance rules in applications. AI platforms should make it simpler to identify compliance issues as they arise so that they can be addressed before any infraction ever occurs.

DevOps teams, of course, have a vested interest in managing compliance as code. It’s not uncommon for DevOps teams to race to build and deploy an application only to encounter a wave of compliance issues that need to be addressed by recoding elements of the application. Not surprisingly, those compliance issues often result in application delivery deadlines being missed. Machine learning algorithms provide an alternative to the current raft of documents that compliance officers create, but not enough other members of the IT team read those, much less understand them.

September 23, 2021
DevOps World: Forget Doc, it’s All About Dev

This year for DevOps World, we’re heading back to the future. Doc Brown won’t be there with his souped-up time machine, but plenty of devs will. As the world is coming to realize, developers are keeping everything running, from the apps we use on a daily basis to check the weather, bank online or order food to the self-parking features on autonomous cars and artificial intelligence that’s augmenting human intelligence. Software and technology are the future, which means the future starts with the creative mind of the developer.

The theme of this year’s DevOps World show is “Building the Future of Software, Together.” For developers, this cannot ring more true: Across the globe, dev teams are working to code new software pipelines that will power the world’s next great innovations. We’ve already seen voyages into space and breakthrough life-saving medicines this year—what more do developers have in store for us?

As we look ahead to DevOps World 2021 on Sept. 28 to 30, we are putting the focus on developers by offering a number of sessions to help sharpen their skills and prepare them to build the future of software. In particular, we are looking at some of the recent challenges that have been affecting code, including cybersecurity, scalability and CI/CD best practices.

On the security front, we’ve seen a number of cyberattacks dominate news headlines this year. As a result, companies are hyper-focused on securing their software pipelines and ensuring business and customer data are safe. Developers now need to double as the first line of defense for their organizations, ensuring their code and pipelines are as secure as possible. We’ve geared a number of sessions towards helping developers sharpen their security chops—from CloudBees customer success stories to tips for building more secure code.

As DevOps professionals know, software today is about continuous everything. We’ve designed a number of sessions to share best practices for CI/CD, progressive delivery and balancing speed, innovation and safety. Imagine if the DeLorean didn’t need the plutonium and just ran continuously—that’s what we’re hoping developers will learn and take away from these sessions!

Lastly, scalability has always been a hot topic in tech. Startups continue to pop up everywhere and as these companies grow, they need to scale accordingly. We want developers to be scaling pros when they need to be, so we have a number of sessions (including keynote presentations) on scalability.

DevOps World is the event for developers looking to sharpen their skills, learn new abilities and network with other dev professionals. As we look to build the future of software together, the people making the magic behind the code will be driving changes and innovation, and we want to celebrate that. While Doc Brown may be a fun character who goes back to the future, it’s dev where the future truly is. Re gister for DevOps World now!

September 17, 2021
3 Building Blocks for Enterprise Digital Transformation

DevOps has gone from a new idea to a buzzword in every boardroom over the last decade. With society increasing its reliance on digitization and pushing their preferences for speed and convenience, enterprises of all types have had to change their business culture to help their teams to work together with greater speed and autonomy.

While the pandemic has undoubtedly heightened the importance of DevOps in the workplace, its widespread use predates the last few years. HGS Digital research from as far back as 2018 found that only 3% of software development businesses had never heard of DevOps. The 2021: Future of the Atlassian Ecosystem report revealed that 54% of organizations have implemented a DevOps strategy—up from 48% in 2020. A further 27% of respondents are hoping to adopt a DevOps strategy in the next three years.

Indeed, DevOps is now mainstream, and the uptick in job listings for ‘DevOps engineers’—as well as enterprises’ quickness to tout their ‘experienced DevOps teams’—is perhaps further evidence of this. However, some of these postings and sentiments can also be considered red flags as they suggest that organizations are applying DevOps practices to isolated teams. Or worse, they are simply labeling traditional engineering roles with the DevOps label rather than implementing a complete mindset shift that applies to the entire organization and enables it to release reliable products at a much faster pace.

With this in mind, here are three key building blocks for enterprises looking to avoid common downfalls and effectively scale their DevOps strategies. These will help them change the way they work and think across all aspects of development and operations.

Changing Culture and Empowering Employees

Everyone needs to be on board for DevOps to be successful, and buy-in is strongly dependent on the business culture they are working within. For example, Gartner predicts that through 2022, 75% of DevOps initiatives will fail to meet expectations. And the reasons why have to do with issues around organizational learning and change.

To put it another way, if employees don’t understand DevOps’ deeper thinking and benefits both to them personally and to the wider group, they will naturally be resistant to the change. It is, after all, an entirely new way of thinking for many departments, especially operations teams who traditionally reduce risk by minimizing change and taking things slowly. DevOps encourages them to speed up with minor changes delivered through continuous integration operating in a fail-fast and often fail-forward pipeline.

Therefore, educating employees is vital for success. In particular, organizations should give staff a clear idea of how it will feel to work differently. Shared definitions of success, shared vocabulary and shared visibility of the work to be done is key. With that—and along with sharing actionable insights that teams can apply to their work and then observe the results—no matter where they are in the pipeline, the real value of DevOps can be achieved.

The reality is that people are the most critical component of any DevOps transformation, making their commitment to the cause non-negotiable for success. Despite the tech-heavy nature of software development, soft skills like communication, collaboration, flexibility and decision-making must be prioritized as much as security, coding and infrastructure knowledge. It’s imperative that all businesses look to adopt a DevOps mindset and implement these traits into their culture in the future.

Applying Four Fundamental Processes to the Foundation of DevOps

That said, a successful DevOps transformation can’t happen unless enterprises step away from traditional IT’s ‘trust but verify’ methodology. Instead, they must incorporate a set of essential processes that empower everyone to work more efficiently and effectively toward achieving faster, high-quality software delivery to suit the end-user’s needs. These fundamental processes include collaboration, automation, continuous integration and continuous delivery (CI/CD) and monitoring.

While collaboration has been one of the most significant rallying cries for enabling globally distributed IT teams to align their goals and meet their objectives, automation has also been highly crucial. Many of those ingrained in DevOps environments will tell you that automation is what makes DevOps highly valuable. It helps teams achieve high levels of efficiency and allows employees to focus more on complex problems and less on repetitive tasks. When a company’s automation strategy is aligned with its broader business goals, it can help deliver value at scale.

But as is the nature of software development, problems can arise. And this is where continuous integration and delivery come into play; it forces developers to merge their source code updates with others far more frequently than they might otherwise choose to. Operations teams can ensure they’re not creating a bottleneck with manual testing to get releases out to users. The approach means smaller changesets; helping to identify, target and resolve bugs more efficiently. This increases the flow of work, the quality of releases and helps to avoid nasty problems at the end of the development cycle.

But there’s no point in having a speedy, heavily automated CI/CD process with a genuinely collaborative culture if enterprises don’t take the time to measure what’s working and assess which actions need to be improved. This is why an effective DevOps plan needs to be built on the mantra ‘implement, measure and improve’. After all, DevOps is about maximizing value and efficiency through increased visibility of work and constant feedback. Measuring real-time metrics becomes vital to ensure business’ transformations are in line with their objectives at all times. If enterprises stick to these fundamentals, they’re well on their way to a successful DevOps strategy with complete buy-in.

Adopting the Right Tools for Transformation

As teams grow and DevOps processes scale, predictable performance is imperative. Therefore, the enterprise’s tools must be available around the clock and should perform at a high level to support their collaborative pipeline.

But using DevOps tools doesn’t automatically mean organizations are applying DevOps principles. The value of tooling is limited by businesses’ commitment to using them holistically with the sort of culture and processes mentioned above. It is this combination that allows them to transition how they develop and deliver their products. With all that said, it’s important to understand how important it is to adopt the proper toolchain—and the wide variety of choices and combinations available—this is why it’s easy to understand why decision-makers might become overwhelmed.

After all, there’s a lot to keep in mind. They need to choose tools that step up their collaboration, keep context-switching to a minimum, aid automation, encourage visibility and allow for effective monitoring. But there’s also the risk that they’ll pick the solutions that they feel most comfortable with, which can be counterintuitive to transformation. This approach can lead to a tool binge, bringing on too much technology where integration isn’t a priority, and leaving enterprises with nothing that does the job they need.

With this in mind, enterprises need to take the time to consult on which tools are right for them while constantly keeping the fundamentals of DevOps in mind. To illustrate just how valuable this is, DORA and Google Cloud’s 2019 State of DevOps report found that the highest performing engineers are 1.5 times more likely to leverage easy-to-use tools. Meanwhile, failure to do this is perhaps a primary reason why a separate study from the DevOps Institute uncovered that the DevOps transformation journey is still arduous for more than 50% of respondents.

Taking the Next Step Toward DevOps

A successful DevOps transformation rests on in-depth understanding, mass buy-in and an organization-wide adoption of fundamental principles and practices. When this agile mindset is applied to the people, processes and tools that make up an organization, enterprises can drive significant change across their teams while increasing their products’ reliability, stability and security.

It’s always important to keep in mind who DevOps is designed to benefit the most—the customer. This mindset is particularly valuable in today’s landscape, where businesses and consumers alike are flush with more choice than ever. With speed and convenience now at the forefront of most people’s purchasing decisions, DevOps can play a huge role in helping organizations strive for greatness and offer the customer something they didn’t even know they needed. Using the above building blocks, organizations of all sizes can dramatically improve their customer experiences and work more efficiently to quickly get the right products into their hands.

September 10, 2021
CloudBolt Survey Shows DevOps Frustration in the Enterprise

A survey of more than 200 senior IT leaders at organizations with more than 1,000 employees published today finds only 4% consider their organizations to be experts when it comes to continuous integration/continuous delivery (CI/CD).

Conducted by CloudBolt Software, a provider of a platform for many IT environments, the survey also found only 11% consider their CI/CD infrastructure to be reliable. Another 69% said their CI/CD infrastructure was somewhat reliable, while 21% said their platforms were somewhat unreliable or not reliable at all. A full 85% also report they regularly test the infrastructure they rely on to drive DevOps processes.

Overall, the CloudBolt survey finds only 5% of respondents report their organization is deploying code multiple times a day. Most respondents (85%) said their organization only deploys code on a weekly or monthly basis.

A full 70% said that, on average, it also takes multiple days or even weeks to set up a pipeline. Nearly two-thirds (63%) said there are too many manual processes involved when setting up pipeline infrastructure. More than half (56%) also noted they have no automated processes for post-deployment tasks.

More than half (53%) said it is also challenging to proactively detect infrastructure issues, while 55% said it is difficult to create a consistent CI/CD pipeline environment across multiple development teams. Improving those pipeline environments requires speeding up the process of provisioning infrastructure through automated processes (70%); continuously detecting infrastructure issues to reduce testing challenges and failures (62%) and simplifying remediation of infrastructure issues proactively (56%), according to survey respondents.

The CloudBolt Software survey specifically identifies issues organizations encounter when employing Terraform to manage infrastructure-as-code (IaC). Only 5% of respondents said they were “very satisfied” with using Terraform and another 75% reported they are somewhat satisfied. Top challenges include difficulty using it in on-premises infrastructure deployments (62%), lack of Day 2 management for deployed resources (53%) and governance (42%). Only 2% of respondents said they were employing Terraform to deploy 75% or more of their infrastructure. A full 90% noted that Terraform requires more custom-coded integrations with other critical tools and systems to work properly.

Finally, 29% said their organization lacks specific Terraform expertise. The most sought-after Terraform capabilities are enabling non-Terraform experts to easily deploy infrastructure using Terraform; ensuring that only Terraform plans with IT “guardrails” are executed and the ability to visualize and cost-optimize infrastructure deployed by Terraform. More than a third (35%) want to simplify deployment of on-premises infrastructure when using Terraform.

Jeff Kulkowski, CloudBolt CEO, said the survey makes it clear there simply isn’t enough expertise available to implement DevOps best practices at scale without enabling IT administrators—that often lack programming expertise—to manage infrastructure and DevOps platforms. That requirement creates the need for an IT management platform based on a set of visual tools that enables DevOps professionals and traditional IT managers to work more collaboratively, he added. DevOps teams can manage IaC whenever they desire, but it’s also crucial to enable IT operations staff to manage IT infrastructure alongside those teams.

Obviously, DevOps as an IT discipline still has a long way to go before it is pervasively adopted across enterprise IT organizations. The challenge now is to remove many of the low-level barriers that conspire to block IT modernization initiatives.

August 25, 2021
How CIOs Can Measure Digital Transformation ROI

To fully grasp how to measure success and gauge progress toward goals in any digital transformation, it’s important to understand the shifting expectations of CIOs who must report these objectives and outcomes. The pandemic has substantially changed the role of the CIO, giving them heftier goals and a shift away from IT.

Today’s CIO is no longer solely responsible for the IT department. Instead, the expectation is for them to have a greater influence in solving strategic business challenges. CIOs are now held to higher standards and have great responsibilities, with the assumption that they excel at communication and facilitation. Today’s CIO role also includes business planning and monitoring, transformational development, testing and security and compliance.

Gartner states that, in a technology-driven workplace, 41% of tech producers are within other business units, not in IT. Solutions are being created from those in charge of the bottom line, who make organizational and business decisions. Further, the rise of low-code platforms has accelerated the shift from IT to the business. Finally, as Generation Z moves into the workforce, there will be a significant shift in what this workforce wants and needs to be their most productive and fulfilled. Typically, this generation of workers are masters of autonomy and they strive to impact everything they touch and have greater control over what they handle and take on.

Within this shifting landscape, it can be difficult to understand how to measure and set goals.

Acquire the Right Tools and Frameworks

The first thing CIOs need to provide to this decentralized organization is the right tools and frameworks. These tools, such as value stream delivery platforms (inclusive of CI/CD, testing, artifact management, etc.) coupled with agile frameworks like SAFe provide a consistent method to deliver apps within the organization.

CIOs need platforms that allow them to visualize, manage and monitor progress on an ongoing basis. While this was largely a manual process before, solutions have vastly improved and, luckily, we no longer need to sit in a room and manually whiteboard progress.

Leverage Value Stream Management

This is often new territory for CIOs and can be daunting. CIOs need to be able to protect the organization, facilitate the quality of its solutions and take accountability for the tools being used and sold to its customers. Fortunately, this new power and influence can be managed with value stream management. It is what allows CIOs to keep track and monitor the bottlenecks in the development process. Further, the CIO can provide collaboration tools to help connect users around common processes.

Value stream management can be a powerful improvement tool for organizations. It can serve as a material and information flow map that helps workers and executives continually view their performance and improvements. “Value” emphasizes that the purpose of the work process can prove the value and worth to the customer, while the term “stream” means that the flow of this value is being measured throughout the process.

Set Business Goals, Not Just IT Goals

Digital transformation success for the CIO not only means that technology works, but that it works for the business. That means the goals CIOs set for technology need to meet whatever bottom-line goals are set for the business, too.

Define KPI objectives that align specifically with the business’ goals and focus on relevant metrics that matter. There are a number of types of metrics that CIOs should consider, but each organization will have to choose which are most relevant to their unique business, industry and their business goals: User engagement, customer experience, reliability, availability, active usage (think: Conversion rates, repeat customers/users, daily active users) are all examples.

When choosing your metrics, find those that you can accurately benchmark against past performance. A baseline will prove your worth; without it, it’s difficult to understand value gained. Ultimately, the CIO must ensure that these metrics are valuable to everyone, and stakeholder buy-in is critical. Finally, everyone in the organization needs to be on board with these goals and metrics.

August 17, 2021
Report: Lots of Room for Continuous Delivery Improvement

A report published by the Continuous Delivery (CD) Foundation suggests that, in terms of achieving that goal, most organizations still have a long way to go. Based on analysis of the Developer Nation survey data collected by SlashData, nearly two-thirds of developers said it takes at least one week to go from code committed to code successfully running in production. Only one in 10 developers are deploying code in a production environment multiple times per day.

On the plus side, half of developers reported that they restore service from an unplanned outage in less than a day.

The survey identified the top programming languages being employed as shell scripting languages, Go/Golang, JavaScript, PHP, Scala, Dart, C#, Ruby, Python and Java.

Surprisingly, the vertical industry furthest along in terms of achieving continuous delivery is the retail sector that has embraced digital business transformation, while the furthest behind is the telecommunications sector.

Tracy Miranda, executive director for the CD Foundation, noted that even after more than 10 years of advocacy, DevOps adoption of CD best practices remains nascent. Many organizations have adopted so-called continuous integration/continuous delivery (CI/CD) platforms, but they are mainly used to drive a set of CI best processes. CD, in comparison, has proven to be a more elusive goal to achieve mainly because each platform on which an application is deployed tends to have unique attributes. Automating the delivery of applications across multiple platforms has, as a result, been difficult to achieve and maintain, she noted.

Nevertheless, with the arrival of cloud-native container-based applications destined to be deployed on Kubernetes clusters that expose a consistent set of application programming interfaces (APIs), the prospects for more widely achieving CD are improving. Each platform that has a Kubernetes cluster running will expose the same API to a CD platform.

In the short term, however, Miranda noted that many IT professionals are still somewhat intimated by both Kubernetes and CD best practices. Too many organizations still tend to think of CD as a platform to acquire rather than a set of processes and skills to be defined and nurtured, she added.

One way or another, however, the CD challenge will need to be addressed as more organizations realize how dependent they are on software to deliver a differentiated customer experience. The rate at which that customer experience improves will determine the level of customer affinity that organizations will be able to attain and retain. That’s doesn’t mean every organization has to be as proficient as, for example, Netflix when it comes to implementing DevOps best practices, Miranda added. However, she said, it’s already apparent most organizations will need to improve how they deliver software to drive multiple digital business transformation initiatives. The opportunity lies in reviewing those processes now with an eye toward creating a culture squarely focused on continually improving the customer experience, she said.

It may still be a while before most organizations truly master CD, but, as is the case with most journeys, the most important thing is to simply take that first big step.

August 2, 2021
CircleCI Survey Shows the Need to Prioritize DevOps

A CircleCI survey of 2,013 business leaders in the U.S. and UK published today finds nearly all respondents (97%) recognize that the success of their business within the next year depends on their software development teams. Despite those findings, well over one-third of respondents (39%) admit their organization is challenged when it comes to innovation and acquiring and maintaining developer skills.

A full 89% of respondents said they also have a good understanding of how to measure the performance of their engineering teams. However, 40% of companies are still using antiquated metrics such as the number of lines of code written, while another 37% measure story points.

More troubling still, less than one-third of respondents (30%) said they plan to make adoption of DevOps best practices a priority. On the plus side, only 15% of respondents said they are implementing a continuous integration/continuous delivery (CI/CD) platform for the first time.

The CircleCI survey suggests that improvements in how software is delivered could be worth, on average, up to $126 million per year. Nearly two-thirds of respondents (65%) said their business is involved in digital business transformation initiatives of one kind or another. Another 20% said they have completed one.

Jim Rose, CircleCI CEO, said it appears that while more organizations now realize how dependent they are on software, many of them have yet to address issues that fundamentally impact developer productivity. For example, more than half (52%) of respondents don’t allow their software developers to choose their own tools, noted Rose. While there is a clear need for a consistent approach to building and deploying applications, Rose noted that DevOps best practices should not require developers to employ a specific set of tools. Developers are only able to truly innovate when they are able to experiment with different tools, said Rose.

In general, Rose said DevOps adoption tends to go through various cycles. Many organizations today are looking to reduce the number of tools that make up a toolchain. However, those efforts shouldn’t curtail the ability of developers to innovate. A CI/CD platform that provides the foundation for DevOps workflows needs to be easily extensible, noted Rose.

It’s not clear to what degree adoption of DevOps best practices have stagnated, but the number of organizations that have implemented DevOps across their entire enterprise remains limited. Most organizations have individual teams that might have adopted DevOps to one degree or another. In many cases, those teams are using disparate tools and platforms.

Most organizations clearly want to be able to build and deploy applications faster. The challenge isn’t so much mastering DevOps platforms as overcoming many of the cultural issues that conspire to limit DevOps adoption. In time, however, the need to rely more on digital processes will force the DevOps issue. As one organization makes advances—thanks in part to their ability to effectively employ DevOps processes—the more keen rival organizations will be to make sure they don’t lose ground. It may be a while before business executives connect the dots between their ability to compete and the degree to which organizations have mastered DevOps, but it’s increasingly apparent that day is not too far off.

July 28, 2021
From One-Click Deployment to One-Click Feedback
What makes a group of individual contributors a team?

You can look at today’s product development teams as an ensemble of craftspeople from different disciplines. Code builders, design artists, storytellers, and an orchestrator. Similar to an orchestra, their output is based on the ability of each and every one of them to work together in harmony, continuously and collaboratively.

Let’s dig a bit deeper into today’s development methodology to see what this means in a practical sense.

At a high level, we see three major steps: planning, development and release.
1. Planning = Empower design craft
2. Development = Empower code building craft
3. Release = Optimize time-to-market (since there is no more time for nonsense!)
Despite the fact that teams are supposed to work together and be very in tune with each other, when we look at the reality, it’s very different. We actually work in silos, nurturing black-box thinking. Instead of collaboration, often there are three separate units that comprise a whole, solely with interfacing endpoints.

In such a configuration, it’s unavoidable that one unit’s priorities will be disruptive to another’s work, or flow. Without better integration of the actual collaborative parts of the work, in the form of frictionless and ongoing feedback throughout and during the end-to-end planning, development and release processes, eventually, your own unit’s priorities will take precedence over another’s.

Engineering IRL

A typical engineering process, whether it’s a new feature being developed, a bug fix or any other kind of collaborative work, usually looks like this:
- A developer receives a design handoff and, from that moment on, is expected to deliver a working manifestation of the design specifications.
- The developer gets to work (whether solo or on the dev team; independent of product or design, at this point) based on the specifications and starts writing the code.
- Once completed, the code is deployed to a pre-release staging environment, where the designer actually experiences the result of their craft for the very first time.
If you’ve been through similar processes, you’ll agree that, by this point, there is rarely time or tolerance in the process to iterate or provide feedback, nor correct much at all. More often than not, the mindset in fast-moving organizations is to stick with what we have ready to ship and just move forward. The actual “collaborative” part of the work is essentially bypassed. It’s really just a relay race of passing the baton downstream.

But this engineering paradigm has its consequences. Ever tried to dissuade an artist from providing their feedback? Expect a pretty bad vibe in the room (or virtual room in today’s hybrid and remote team structure).

This example, of course, is actually a virtual utopia of the development process. In reality, most organizations today have a far more destructive culture of continuous disruptions.

Now, let’s spice up the scenario with some real-world interruptions that often break the ongoing individual flow.

When a craftsperson is in the zone and creating, don’t interrupt! But what can you do when there are so many things happening in parallel? In small organizations that are just ramping up, most people are juggling much more work simultaneously.

What often happens looks a lot like this:

Jolene from sales requires a specific version for customizing a demo for a super important customer; Marko from support has opened 31 tickets you have yet to handle since it is a production issue that affects end users; your beloved single staging environment broke and you can’t review any new feature—oh, and don’t forget to attend the three meetings scheduled for today.

Frustration, anyone? I like to call it the ‘multi-context bad habit.’

Okay, I can already sense some eyebrows being raised, and I can almost hear you saying, “Yeah?! So what? We’ve always worked this way—what’s the issue??” Allow me to show you some numbers that may surprise you.

“Finding flow is key, and interruptions are a drag. Minimal or no interruptions give developers an 82% chance of having a good day, but interruptions throughout the day decrease the chance of a good day to just 7%.”

Eirini Kalliamvakou

“Meetings are both awesome and terrible. Collaboration improves our work, but too many meetings can be a blocker; going from two to three meetings per day lowered the chances of making progress toward their goals by 60%.”

Dr. Nicole Forsgren

To summarize, these data points basically indicate that disruptions, interruptions and the “multi-context bad habit” ultimately have a tangible, detrimental impact on results and outcomes. They diminish successful outcomes and delivery times by orders of magnitude.

Zero-Click Deployments as a Culture

Fortunately, there are multiple practices and tools today that can be leveraged to dissipate the friction, break down the silos and improve the flow.

This can be achieved by evolving from one-click deployment to zero-click deployments. Allow me to explain.

Today, we have enough working examples that could (and should) be replicated more widely in engineering practices. Here are some examples.

Preview Environments

Preview environments (sometimes referred to as ephemeral or PR environments) provide, well, the opportunity to preview products. By allowing multiple stakeholders to review and experience the product throughout the development process, you shift-left the way you manage your development processes and are able to provide the relevant feedback and fixes earlier in the process. This basically means that for any PR you get a live environment that is shareable and independent from your staging environment. (I like to call this an in-development feedback loop).

The JAMStack

Another good example can be taken from JAMstack. When JAMstacking, consider using Netlify, Vercel or AWS Amplify to gain this capability for your team.

Infrastructure-as-Code (IaC)

If your project is non-JAMstack, you can consider building a similar preview environment through IaC solutions like Terraform or Pulumi. The downside here is that this comes with quite a bit of hassle, as you have to build automation for every single PR to have a dedicated shareable URL of the environment for review.

This is great, though, because it democratizes the PR approval cycle by being more inclusive. Project managers, designers—practically anyone can review and approve the PR in development without having to wait until the end of the process to provide input. It’s easy to manage as it is isolated and independent of staging, which also provides always-on issue reproduction ability in the event that some kind of bug was discovered or tests need to be run during the development phase.

But it also isn’t all roses and sunshine.

There are some gaps and challenges that still need to be overcome for this to work. Some examples:

How do we streamline feedback?

Developers are used to getting comments from peers during code review in a language they can understand easily (for the most part) and quickly merge the input into their project. They don’t lose context by navigating away from the tool they’re working in (e.g. to Slack or Acme Corp. Task Manager), as it keeps them in their source code management environment (SCM). We need to think about how can we offer the same developer experience for everyone in the process to minimize PR turnover.

How do we provide actionable and quality feedback?

For example, UI and behavioral issue reporting are time-consuming, since you have to document and explain what, exactly, needs to be changed in order for it to be effective. Any wanted change likely involves the developer, which adds yet another disruption and interruption to their day.

Taking Zero-Click Deployments to Single-Click Feedback

If we want to optimize the developers’ flow, the next phase after simplifying the deployment aspects is to see how we add in the feedback layer in a way that isn’t intrusive.

We need to now take this mindset and empower what I like to call one-click feedback.

This isn’t a new concept, of course. Let’s take some inspiration from collaboration tools we all love like Figma, Miro and Notion.

What’s magical about each and every one of these tools is the ability to provide contextual feedback. This is important as it enables contextual collaboration. Any user is basically one click away from sharing their feedback while remaining in-context, increasing the potential for feedback to be dealt with frictionlessly.

This makes possible the following core values:
1. Any person can easily point out something which requires the creator’s attention.
2. The creator can instantly understand the feedback within the context of the project and what they are working on, and reply or tag another relevant peer for their point of view.
3. The creator doesn’t leave their main work environment to prioritize the new task or give it the attention it requires. Just as designers do not leave Figma to apply feedback, PMs don’t need to leave Miro to apply theirs, and so on.
These tools welcome feedback in an inclusive manner while enforcing a frictionless workflow. No one is messing with the creator’s flow as it is also happening asynchronously—no need for meetings, syncs or coordination of multiple stakeholders. This is also much more inclusive and relevant for today’s remote work culture, which requires investment in team communication through asynchronous channels and processes to be effective. (Here’s a great talk to watch on this topic; yes, the irony of interrupting your flow to watch this video is not lost. No, Really, Don’t Chuck Everything in Slack: Communications for Distributed Teams – from Florian Haas)

Assuming you find some truth in this, I’d like to address the now apparent elephant in the room: Why don’t we introduce the same abilities for in-development collaboration?

One-Click Feedback-as-Code

We’re starting to see a trend where framework-coupled companies like Netlify and Vercel, for example, are adding collaboration aspects to their offering. Netlify, in fact, is acquiring a company, FeaturePeek, to embed collaboration features on top of their preview environments, and Next.js starting an early access rollout of their Next.js Live product.

Imagine developers getting inclusive feedback during the PR approval cycle; not only comments about their code from peers but also UI, design and behavior comments with all of the needed assets and context to move forward without leaving their work environment! Such a shift will have a measurable impact on time-to-market, quality and velocity when employing such a workflow.

The main premise here is that to achieve better metrics associated with flow (as noted above by Dr. Nicole Forsgren), we should not mess with the flow. The benefits of reducing the friction and interruptions in creators’ flow are exponential and will enable us to get closer to working in tune and in harmony (if we stick with the orchestral references) or simply more collaboratively, as a team where each individual is always maximizing the time they are able to be in creation mode, without silos or black boxes.

What do you think? What’s your view of how teams should work together? How do you handle flow within your organization?
July 27, 2021
7 (More) Security Considerations for CI/CD

CI/CD orchestration has become increasingly popular in DevOps practices. It facilitates faster deployment of software and product delivery. Through the use of automation when building, testing and deploying applications, CI/CD bridges the gap between development, operation activities and teams.

However, there are many security considerations to keep in mind with frequent deployment. From quality automation to code history analysis, security must be baked into your CI/CD pipeline to ensure your product and processes remain safe from hackers.

To gain insights on the top security considerations for CI/CD, I asked the speakers and sponsors for the upcoming SKILup Day as well as several DevOps Institute Ambassadors for their thoughts. In this two-part series, I will be sharing their insights. Did you miss part one? You can read it here. Here’s what they had to say in part two of this series:

Speaker, Ravi Lachhman, Evangelist, Harness

“CI/CD platforms are core to an organization’s engineering efficiency and typically have elevated privileges to sensitive environments such as production. Since CI/CD platforms typically will be deploying, this makes sense. Delving into the SolarWinds hack with binary artifact signing, etc., can separate security concerns into two parts. One concern would be the security around the actual CI/CD platform and one around the DevSecOps goals inside a CI/CD pipeline. With so many items shifting left, having proper role-based access control (RBAC) is crucial. With the separation of duties for regulated environments and the need for engineers to be efficient, controls that ensure only certain entities can deploy and certain entities can view pipelines is important. Around the DevSecOps movement, container/OSS scanning is still top of mind for many firms.”

Sponsor, Basis Technologies, Eran Kriegshauser, Basis Technologies’ SAP DevOps Architect

“The question you should be asking is ‘How do I maintain quality with the frequent deployment tempo demanded by business today?’ We need automation of quality and guaranteed rigor. For example, take promoting security configuration changes in systems, applications and products (SAP). Today, this is done by SAP transports where the change is created in an SAP development system, then imported into QA and, ultimately, into production. How do you ensure that a developer can’t give themselves extra security access through a role modification? Automated analysis and approval workflows must support segregation of duties requirements. Users can’t approve changes they create–the appropriate CI/CD process will not allow it.

Another area is automatic peer review notification on changes to sensitive enterprise resource planning (ERP) objects. Once identified, the promotion of those object changes are halted until approved by a team lead or change and release manager.”

Savinder Puri, DevOps Evangelist, Zensar Technologies

“Infuse as many aspects of security as possible into your CI pipeline. CD is already too late for security—you’ve already deployed the application into an environment.

Start from the basic static code analysis, security scanners, etc., and then keep maturing. Whatever tools you integrate into CI Server, ensure the corresponding integrated development environment (IDE) plugins are made available to developers. That way, devs can do the first-level validation within their IDE itself, and do it as they code, even before compiling. That’s the practical implementation of the often overused term ‘shift left.’”

Vishnu Vasudevan, Head of Product Engineering and Development, Opsera

“Now with cloud migration, emerging technologies and open source technology, companies will have to make sure they have the right tooling to measure and take action based on the security challenges born out of different development activities.

The focus will need to be on managing the risk that is tied to the business criticality. Each industry, domain, technology, etc., all have their own independent security goals, and you cannot have one security goal across the enterprise for different technologies and different domains. It is important to evaluate the technology being used and weigh the security requirements accordingly.”

Stephen Walters, Sales Engineer, Everbridge

“Inclusion! This is why the term DevSecOps came to be—because, erroneously, security was not included as part of a DevOps transformation, and the same is true for transformation of an SDLC to incorporate security.

Security has still been treated as a non-functional requirement; an afterthought. If there is one thing we should learn, there is no such thing as a non-functional requirement. Security needs must be considered on an equal footing with all other requirements and built into the design, test, deployment and delivery of any feature.”

Mark Peters, Technical Lead, Novetta

“CI/CD pipelines should implement security at every step. Tests can be automated in tooling to conduct tests of various functions at multiple levels for code accuracy, unit tests through fuzzing and integration or vulnerability testing. Most vulnerability testing in a pipeline is actually patch management, ensuring all commercial and proprietary holes are filled for release software. The first step should be using coding checks to make sure there are not any common problems with the code. While this may seem easy, if you are not managing that basic step, you cannot move ahead to the more complicated areas.”

Pawel Piwosz, Lead Systems Engineer, EPAM Systems

“At a high level, I would say that security is still outside the main scope of CI/CD. We should consider a few areas and different angles, here. Security should be not only implemented by CI/CD; CI/CD should be implemented in a secure way. Everything-as-code is a solution here: policy-as-code, security-as-code, drift-as-code, and if we talk about infrastructure, remediation-as-code.”

If you’d like to learn more about the latest in CI/CD and similar topics, check out the upcoming SKILup Day schedule. Or, learn more about the benefits of becoming a DevOps Institute Member.

July 8, 2021
Copado Acquires Qentinel to Gain AI Testing Tools

Copado has acquired Qentinel as part of an effort to incorporate AI-infused application testing within a DevOps platform and build applications using low-code tools across multiple software-as-a-service (SaaS) platforms.

The Copado continuous integration/continuous delivery (CI/CD) platform was originally created to accelerate the development of low-code applications on top of the Salesforce platform. Now, however, Copado is expanding the scope of its DevOps reach to include other SaaS platforms.

Andrew Leigh, chief marketing officer for Copado, said as organizations increase the number of SaaS platforms they employ in the wake of the COVID-19 pandemic, it quickly became apparent there would soon be a need to build applications that enable organizations to unify business processes spanning multiple SaaS applications.

As part of that effort, Copado’s acquisition of Qentinel aims to advance that goal by making it simpler for both developers and end users to participate in the application testing process, Leigh said.

Qentinel is accessed mainly via the cloud to automate the creation of application tests. Rechristened Copado Robotic Testing, the core of the Qentinel platform is based on the open source Robot Framework that understands scripts, executes them and generates logs and reports. A set of built-in libraries comes packaged with Robot Framework and addresses a wide range of testing scenarios. Qentinel customers include Brunswick, Elisa and Konecranes.

Gartner is forecasting that, by 2024, 75% of large enterprises will be using AI-enabled test automation tools to advance continuous testing across different stages of the DevOps life cycle. Much of the interest in AI-infused application testing tools is being driven by necessity as the pace of low-code application development exceeds the pace at which manual testing processes can work. As application testing becomes a bottleneck, organizations will naturally look toward automation platforms that employ machine learning algorithms capable of testing applications at scale.

It’s not clear to what degree those platforms might eliminate the need for humans to test applications. The expectation is humans will be able to test application functionality at a deeper level versus, for example, merely checking to see if compliance mandates have been met. Many smaller organizations building low-code applications don’t have any meaningful application testing capabilities at all. Testing platforms infused with AI will prove to be especially crucial for citizen developers that require guidance to build a secure application that can actually scale, noted Leigh. The more SaaS application environments those applications span, the more critical testing becomes, added Leigh.

One way or another AI is coming to application testing. After all, machine learning algorithms don’t ever need to take a day off or suddenly leave to take a better offer elsewhere. In fact, soon, most organizations soon won’t even consider a test automation platform that doesn’t have AI capabilities as they attempt to reduce the level of manual, monotonous and repetitive testing tasks. It may take a while for everyone to gain confidence in the algorithms, but it’s now more a question of the degree to which AI will be employed rather than if.

July 7, 2021