devopsy.kinsta.cloud

Category: DevOps Practice

Are Developers Responsible for Open Source Governance?

There are lots of factors in the open source software world converging to make it a big year for “shift left” in software development. Heightened security concerns, an increasing need for software supply chain visibility and the growth and complexity of open source ecosystems will continue to push the responsibility for ensuring code is secure earlier in the engineering process as far left as the request and design phases.

This means 2021 will be yet another year in which developers are challenged to take on more responsibility, and pushed further onto the front lines of open source governance. Developers will need access to the right training, resources and tools to do this job. But they also can’t be successful if they’re expected to do it alone. Shift left can’t mean shifting all responsibility to the developer as the sole protector of secure code and the only go-to open source expert. The reality is that the depth, breadth and complexity of open source governance in 2021 demands a cross-functional team and layered approach.

Equipping Developers With Governance Knowledge

Let’s start with how to better equip developers and other stakeholders in the governance process, including legal, security, IP compliance and product people. These teams need enough knowledge to be aware of the top compliance issues, and their corresponding remediation options, to be active contributors in this rapidly evolving space. Research from Forrester shows that of the top 40 computer science programs in the United States and the top five programs internationally, none include open source licensing and security coding practices in the curriculum.

Instead, it has fallen to the industry itself to impart this expertise on the job. Leading companies are packaging open source training programs into something that resembles standard HR-led and managed training, including check-ins and quizzes to test and solidify knowledge, and regularly updating content and scheduling trainings to make sure people stay abreast of changes and challenges. In the spirit of open source, many industries work collaboratively to share best practices on governance in this regard, with industry experts speaking on the topic at other companies. Others are leveraging existing management training courses, such as the ones offered by the Linux Foundation.

The growing importance of the Software Bill of Materials (SBoM) also points to the need for cross-functional collaboration. Software supply chain visibility will become even more crucial in 2021 so that companies can build and sell secure and safe products, as well as ensure compliance. More people across the organization will need to access and view the SBoM in ways relevant to their functional needs. As a result, its assembly is transforming from being assembly-line-esque (passed along and viewed in the context of siloed business functions) to becoming much more collaborative. The SBoM is becoming akin to a business intelligence tool, providing the ability for different functions and stakeholders to filter for relevant information.

Depending on your function, you may be interested in a top-down view of the open source and third-party components across your portfolio of products in order to understand risks in the organization. For instance, as an engineering leader, you may want to understand if you are impacted by a new security vulnerability, where in your portfolio of products you may be exposed and the current status of remediation work for that vulnerability. If you are in legal, you may be interested in a significant licensing change for a component in use at your organization. This allows you to be prepared with changes to legal policies for automated reviews and obligation fulfillment guidance for engineering when the next build occurs and a more recent version of a dependency is pulled in. If you are in the export control function, and are required to provide your application’s encryption capabilities as part of the certification process, understanding which open source components are part of the application, and which provide cryptographic capabilities is essential. Finally, don’t overlook your customers, who today are requiring open source disclosures as part of commercial software contracts to protect themselves from insecure software.

Collaboration has also been made simpler by the fact that new industry standards now enable an asset portfolio view of open source security. Open Chain obtained ISO certification in late 2020, ISO/IEC 5230:2020, providing a clear and effective process management standard for open source license compliance. It allows companies of all sizes and in all sectors to adopt the key requirements of a quality open source compliance program. Coupled with ISO/IEC 27001–the de facto standard for managing security in information security management systems (ISMS) – organizations can now take a more holistic view of all of their software assets in an effort to make the information they hold more secure.

Open Source Program Office

The best way to ensure developers aren’t left footing the bill for open source governance is to create a centralized team dedicated to it. Savvy organizations could form an Open Source Program Office (OSPO), which brings together key stakeholders in legal, engineering, product and security to develop, communicate and operationalize open source strategy for their entire organization. This team works to define a framework and process for consuming open source software as part of their development work (inbound use case) and/or contributing code to the community via open source projects (outbound use case). The mission and main charter of these teams is to help their organization create and manage an open source strategy in terms of the adoption, use, support, participation and development of open source software. However, they’ve also found their role has evolved to become the experts on open source internally and evangelists of the technology and its use.

The bottom line is that the developer’s job is to develop software – and every piece put in place for stronger open source governance means less time spent focusing on the problems that could be in the open source code and more time on the promise of using it to rapidly innovate for the benefit of their customers.

April 27, 2021
Reducing Friction in Developer Collaboration

Given how the individual unit of deployment has gotten increasingly smaller, such that the unit of deployment is usually a microservice, it would be tempting to think that individual developers are more able to control how well the code they write (the service they deploy) will work in production. In fact, though, almost the opposite is true. Instead, as the number of services in an environment increases, whether or not one service will work correctly in a production environment has more to do with how well it works with other services rather than the quality of the code.

This makes collaboration between developers even more critical than ever before, but the packaging of code in microservices actually makes collaboration more challenging. Developers don’t collaborate by working together on a shared codebase — instead they work completely independently on services that could easily be written in different languages, providing the illusion of complete independence.

In most organizations, ensuring that services work together is a largely informal process. It’s managed by walking over to a colleague’s work station and asking what updates they’re working on or what version libraries they’re using. In a remote work environment, teams rely on Slack messages or Zoom calls to solve issues relating to services interaction. This creates a lot of friction for something that is a critical part of the software development workflow. Failures in service integration can and do cause problems in production.

Pay Attention to Service Communications

A perfectly crafted microservice is not going to run in isolation in the production environment. Compatibility issues between each service and upstream and downstream dependencies are just as likely to cause bugs, downtime and poor performance as problems with the code deployed in a container.

The only way to catch compatibility issues before production is for developers to work together and communicate about the updates they’re working on and how those might impact interdependent services. In the current system, however, this level of collaboration introduces a lot of friction in the development workflow, reducing the individual’s and the team’s velocity. Since developers are generally evaluated based on velocity and code quality — but not necessarily on how well their services work with other services — if the friction remains, developers will be tempted to deploy without fully understanding how dependencies will be impacted.

Since CI/CD pipelines are generally not set up to test how well services communicate with each other, problems with service fit that aren’t addressed at the development stage generally aren’t found until either a canary deployment goes wrong or there are problems in production.

Replacing Tribal Knowledge

One of the other reasons companies need to focus on developer collaboration is that remote work has made it even harder to communicate tribal knowledge. In my experience, no development team has been able to completely wean themselves off of tribal knowledge. Even teams with extensive technical documentation often had or have experienced team members who maintain a unique understanding of the application’s business logic and can share that knowledge when needed.

Tribal knowledge also relates to service integration and communication — often, insights about how services work together isn’t well-documented, even if each individual service is well-documented. Often, the easiest way to see how they would work together is not to have the two developers responsible for each service work together, but rather to have a senior developer take a look. Especially as development teams work remotely, that type of review isn’t feasible.

How to Help Developers Collaborate

Most teams have ways for teams to communicate — they have a Slack setup, they have internal forums and video conferencing solutions. But they often don’t realize that these communication platforms aren’t really collaboration platforms — they are not providing a way for developers to actually work together on interdependent services the way they might if working on a shared codebase. Communication isn’t enough for true collaboration to happen — developers need a way to work together and see how colleagues’ changes impact their own services.

Facilitating true collaboration, the kind you would get from two people looking at the same screen and coming up with solutions together, is more challenging than making it easier for developers to talk to each other. Without true collaboration, teams will continue to have trouble with communication between services — and there’s no Slack for that.

April 27, 2021
Is Low-Code Development a Security Risk?
According to Gartner, the worldwide low-code development market is projected to be $13.8 billion in 2021, a whopping increase of 22.6% from 2020. Gartner also predicts that the surge in remote development during COVID-19 will continue to boost low-code adoption, despite budget cuts and efforts to optimize cost.

Low-Code’s Popularity Among Professional and Citizen Developers

Low-code is no longer the new kid on the block. From simple dashboards to complex applications, low-code applications are becoming more and more diverse, with mainstream adoption in the enterprise world. In essence, low-code is a visual paradigm of application development that involves drag-and-drop of pre-built components and integrations resulting in fast, easy and less-error-prone development.

The advantage of low-code is that it enables users from a non-traditional development background to participate in the app development process, thus democratizing development and accelerating project timelines. It also narrows the demand-supply gap that arose as organizations catered to ever-rising digitization needs with limited resources and developer scarcity. Many low-code platforms make it easy for citizen developers (business analysts, line-of-business users, junior developers, for instance) to build applications while drastically reducing delivery time compared to traditional coding methods.

But citizen developer productivity and speed are not the only benefits of low-code development. Today, low-code platforms also come with capabilities purpose-built for professional developers from enterprise IT teams to ISVs. These platforms are hardened for enterprise use, are able to leverage the scalability and security needs of a complex application and have integration capabilities mature enough to seamlessly fit in with existing tools and technologies. During the COVID-19 pandemic, many businesses reinvented themselves through adoption of low-code platforms and applications that helped them adapt to the sudden shift to a remote work scenario and enable the necessary modern application needs that came with it.

Security Challenges with Low-Code and Remote Work

Compared to traditional development, low-code involves a variety of personas working together to build applications while dealing with automatically generated code, ready-made components and built-in default configurations. This shift in environment revealed some unique challenges that need to be addressed. There are a few common security challenges with remote teams building on low-code.

Application Development and Remote Team Collaboration
- Lack of security awareness: Low-code users come from both business and tech backgrounds. Some practitioners aren’t familiar with application security best practices and lack awareness and understanding of potential vulnerabilities and security holes.
- Platform access and admin controls: Low-code is deployed centrally and available for users across an enterprise via browser access. This introduces a risk of network intrusion; providing access to unauthorized developers and opening up greater permissions to users that do not need it when accessing the platform remotely.
- Code repository and team collaboration: Low-code platforms must ensure that the automatically generated code can committed to enterprise-sanctioned repositories. This code access shouldn’t be misused, and ought to have sufficient protocols in place for code control and upgrades.
Code Generation and DevSecOps Best Practices
- Securing custom code: Low-code tools allow for writing custom code to extend and enforce platform coding guidelines and design patterns to protect sensitive data from unauthorized access.
- Adhering to secure release practices: Integration with the existing enterprise CI/CD pipeline is important so that dev teams can extend the same release governance protocol to the auto-generated code before going to production.
End User Application Access and Data Protection
- Preventing malicious attacks: Today, both web and mobile apps are becoming constant targets for security breaches. Automatically generated code, and citizen developers working remotely, can make low-code applications more prone to vulnerabilities. Platforms should generate apps that are fully protected against phishing attacks, SQL injections, brute force attacks and DOS attacks.
- Secure data and application access: Low-code platforms should provide a comprehensive access control mechanism preventing unauthorized access to data and app functionality. With remote teams accessing apps from anywhere, at any time and from any device, data breaches can be prevented with the right controls in place.
Low-Code and Security – The Future of Development

As enterprises and ISVs turn to low-code for more serious use cases, the larger question still remains. Can low-code platforms enable modern development teams to deliver applications faster, while still enabling a secure and tightly governed environment?
The answer is, yes, they can! However, development teams must consider the following security checklist when considering low-code platforms:
1. The chosen low-code platform must be set up within an enterprise secure DMZ or secure private cloud, and must effortlessly pass cybersecurity clearances.
2. The platform must enforce best practices in programming (coding conventions, design patterns and data encryption) for automatically generated code as well as custom code written by the developer, facilitating easier integration with existing CI/CD processes and tools.
3. The platform must offer full protection against the top 10 OWASP vulnerabilities for web and mobile apps, and have third-party certifications to guarantee code quality and security. Also, organizations should ensure that there are no vulnerabilities in their chosen platform’s binaries, as well as all third-party dependencies (including open source libraries) as listed in the CVE library.
4. The solution must support multiple authentication providers (database, LDAP, AD, SSO, SAML, Open-ID, multifactor, biometric) to build applications with strong user security. For user authorization, ensure support for both coarse-grained as well as fine-grained access control policies to protect various aspects of the application based on RBAC.
Development teams can embrace low-code technology while ensuring security best practices are in place. Low-code is here to stay, and with the right platform, enterprises and ISVs can make sure security is shifted left and addressed by developers much earlier in the development process. The result is a highly productive, remote workforce churning out applications that are modern, scalable and secure.
April 26, 2021
How Shifting Testing Left Can Boost DevOps Velocity
DevOps is all about balance: How can speed and quality be balanced to best meet the demands of customers who want exceptional digital experiences as fast as possible? DevOps looks to meet these expectations head-on by helping to deliver a faster cadence of new builds to customers while keeping quality high.

What’s more, quality should continuously improve, build-over-build, so that fewer fixes are needed with each iteration. While inserting more testing earlier in the development life cycle may at first seem like a hindrance to the speed DevOps promises, it will soon empower teams to release higher-quality software, faster, as shift-left testing becomes a routine part of the team’s processes.

Many organizations wait until the very end of their software development life cycle (SDLC) before executing testing. With this structure, testing can and does find unexpected bugs late in the SDLC, where issues are the most difficult and expensive to resolve. With this approach, testing becomes a bottleneck, impacting that ever-important goal of efficient and Agile software development and deployment. An unexpected outcome: it also negatively affects team morale, as developers who spent plenty of time and effort to build and ship code quickly now have to delay their deployment process in order to correct issues.

There is a more efficient approach that organizations can put into place, however.

Shifting Testing Left

Testing can be implemented during and throughout the critical stages of design and development of software, instead of after.

This is true regardless of how a team approaches development methodologies: Agile, waterfall, continuous or a combination of these methods. When testing is shifted left, it becomes the responsibility of the entire organization, not just dedicated QA testers. That means SDETs, developers and others should all be active in testing along with QA. When these groups are involved in testing early on in the SDLC, there are plenty of benefits, including:
- A list of user workflows and test validation conditions can be created to streamline testing during development processes.
- Testers can catch potential issues much earlier in the process so they can be fixed during earlier stages, instead of after code is done being developed.
- Developers and testers can work with code they are familiar with to edit existing code and fix issues themselves.
Building a Continuous Feedback Loop

Another major advantage of shifting testing left in the SDLC is that it creates a continuous feedback loop. Software can be updated and improved continuously as it is being built, instead of when it is “complete” and ready to ship. This model of a feedback loop saves time and ultimately ensures no show-stopping issues make their way into the hands of customers.

Test Automation in the SDLC

One more key element to shifting testing left is test automation. Unfortunately, automation is often not leveraged completely or to its fullest potential. A recent report found that 74% of QA testers see test automation taking place outside the regular test cycle. Moreover, only 45% of organizations use test automation, and only 50% use it for continuous integration/continuous delivery (CI/CD).

For test automation to succeed, it must involve developers creating the automation code and testers, then editing, maintaining and executing that code. When test development, including automated testing, is moved into the SDLC properly, it leads to automated and integrated unit testing, which helps achieve that goal of increased efficiency for DevOps teams.

Automation is obviously crucial to the success of DevOps, but manual processes must also be considered. Some types of testing – like exploratory, payments, usability and accessibility – cannot be automated (at least not fully), but they should also be accounted for as part of a shift-left strategy.

The origin of DevOps methodology and the focus of developers involves the goal of building and deploying code as quickly as possible. But testing software to make sure it works as intended is an important step that should be included throughout the SDLC. When testing only happens at the end of the process, it can be an inhibitor to the overall goal of increased development velocity. But when testing is shifted left and involved earlier on in the SDLC, it can do the opposite – by creating a continuous feedback loop and implementing automation to drive improved DevOps velocity.

Ultimately, shifting testing left in the SDLC leads to features getting to users and customers faster, an overall improvement of software quality and a reduction of QA testing time and bug fixes.
April 23, 2021
ADOC: Assessing DevOps Culture

The Assessment of DevOps Capabilities (ADOC) is crowdsourced, vendor-neutral and designed for individuals, teams and organizations who want to baseline their current DevOps state, identify the next target state, gain insights into how to improve their organization and team capabilities and measure and accelerate continuous improvement during their DevOps journey.

The assessment models five DevOps dimensions – the human aspects, process and frameworks, functional composition, intelligent automation and technology ecosystems – to help you gauge your DevOps maturity and identify areas for improvement. In this TechStrongTV interview, Alan Shimel talks with Garima Bajpai, Louise Cermak and Brendan O’Reilly about reshaping culture to get the most out of any DevOps journey. The video is below, followed by a complete transcript.

Announcer: This is Digital Anarchist.

Alan Shimel: Hi, everyone. Thanks for joining us on another segment here for TechStrong TV. This is a special segment panel for TechStrong that I’m gonna be conducting, here. I’m happy to have you join us. I’m also really happy to have three interesting people join us. Let me introduce you to them. And if I mispronounce people’s names, I apologize; we do the best we can here on TechStrong TV, right?

But we’re gonna start with Garima, is it Bajpai? Fair? Good? Okay. Garima, why don’t you introduce yourself?

Garima Bajpai: Thank you, Alan. I’m Garima Bajpai from Ottawa, Canada. I am the Founder for the DevOps Community of practice here in Canada, which has three chapters, Toronto, Edmonton, and Ottawa. I’m also the organizer for the DevOps Summit which happens in Canada November 8th and 9th, and I am Ambassador with DevOps Institute, as well as an Ambassador with the CD Foundation. So, today, we are going to have a very interesting dialogue on DevOps assessments. So, I can’t wait to get started, right?

Shimel: Yeah, thank you, thank you. Thank you for all you’re doing for the DevOps community. Next up, let me introduce you to Brendan O’Reilly. Brendan, welcome—a little bit about yourself?

Brendan O’Reilly: Sure. So, my company is called Daysha DevOps. We’ve been working with the DevOps Institute now for about five or six years, predominantly delivering their training. And I myself am a consultant, and like the other speakers here today, really looking forward and excited about what’s happening with ADOC, because I see a big opportunity for our customers to get deeper into their DevOps journeys, to progress those journeys.

Shimel: Fantastic, and thank you. And then last, but certainly not least is, we have Louise Cermak—Louise, did I get that right?

Louise Cermak: Almost, yeah.

Shimel: Well, say it right, because you’re entitled to have your name said right, it’s my fault. So, please, how do we pronounce your name?

Cermak: Thanks, Alan, it’s “Chermock.”

Shimel: “Chermock,” okay.

Cermak: Yeah, so, I’m Louise Cermak. I’m a Principal Consultant with Catapult CX. We build software ourselves, and we also consult in agile and DevOps and have been doing so for sort of 10, 15 years. And we’re excited for ADOC, because actually, having the combination of being engineers as well, we can really understand the capability and how to apply it.

Shimel: Absolutely. Well, all three of you, thanks for joining us. So, a couple of you have mentioned ADOC, right, and this is something that our friends at the DevOps Institute announced, I guess it was last week or the week before now. And ADOC, for our audience who may not be familiar, stands for Assessment of DevOps Capabilities. And this is a new assessment model that the folks at DevOps Institute have been working on.

Look, you know, I know they’ve been working on it for over a year already. It’s been primarily driven by Helen Beal, who is a—Helen is a very well-known DevOps person in the community, but she’s also the Founder of the DevOps Institute Ambassadors, and has been a DevOps consultant herself for many, many years, as long as I’m doing DevOps.com.

And then Helen was really helped with this by Eveline Oehrlich, who’s a DevOps Research Director at DevOps Institute, a former Forrester VP. And then I also know that she’s been helped a lot with people like the three of you, who are real world DevOps consultants who are helping organizations every single day with their DevOps transformations and capabilities.

So, before we jump into ADOC, though, let’s take it up a notch and talk about DevOps assessments in general, right? Are they—why are they important? Why are they important, and if anyone wants to give us sort of a brief history of evolution or time around these assessments and how they’re used, let’s say, by consultants and/or end user organizations.

Garima, I know you have a lot of experience, so if you don’t mind, I’m gonna ask you to kick it off, and then have the rest of the panel chime in.

Bajpai: Right, and I actually term DevOps transformations as a journey towards remaking culture. So, it has been largely a complex journey, because of two factors, actually. The first factor is that the complexity brings along with, you know, a lot of focus on how you can upskill people, how can you actually transform your knowledge based and skill set to a journey which is basically based on experimentation, on learning, and learning from failures, right?

And the second aspect is that there is no such literature available. You know, there is a lot of talk about DevOps transformation, but the literature is in the making. So, there is limited academic support as well as support from a best practices perspective how to do this transformation, right? So, these assessments come very—it is very crucial, because it will help you assess where you are, and every organization will have unique challenges, right?

So, it is important to assess where you are in your journey and what you can do. You know, it’s not like a traditional transformation journey of three to five years where you are relying on a lot of targets ________ models which were already defined. It’s continuously evolving, right? So, you will have to have a view on what you can do today and how you can incrementally, in short cycles, transform.

So, what happens in this process is that, you know, assessment becomes more crucial and essential, because there are discrete efforts. There are efforts in short cycles, and this assessment and measurement will help you course correct, will help you understand where you are heading to in your journey and what is needed as next steps to ensure that you incrementally build value, right?

So, that’s why, I think, these assessments are extremely important. We will talk along this conversation how ADOC can actually help you on this journey as well as what are the major impediments and challenges organizations have and what should they look out for.

Shimel: Fair.

Cermak: I think, over the last few years where we’ve been helping organizations, what we’ve noticed is that many of them are trying to do their own assessments, and I’ve been into so many organizations where people have got Excel spreadsheets and they’re trying to measure how they’re performing.

Because, you know, Garima’s right—there’s a lot of learning and experimentation. A lot of it is self-learning, which is what we encourage, right, from a DevOps culture perspective. But people wanna know that they’re going in the right direction, so they start with their own spreadsheets, fill them in, try and measure their capabilities. And a tool like ADOC helps organizations have a much more structured way of doing that, and obviously, it’s more scalable than an Excel spreadsheet, but people wanna know that they’re doing the right things.

Shimel: Agreed.

O’Reilly: Just a little bit about the history, perhaps Alan—you know, ADOC is brand new, but there was a progenitor in the form of DORA, if you recall.

Shimel: Sure, sure.

O’Reilly: Yeah, so, that was with Jez Humble and Nicole Forsgren. So, when they produced their State of DevOps report, it was a means by which organizations could form views about what good looked like, so you have your release and then so on.

And it definitely raised people’s curiosity about—can we do this, can we measure? And of course, one of the principles with any DevOps transformation is to be data-driven. And so, the 180-odd questions that are included in the enterprise edition are very much about trying to capture and turn subjective opinion into something quantitative, a good first guess. And I think DORA has been successful, it’s now part of the Google organization, I believe, but this is a nice independent offer from the DevOps Institute.

We’re really looking forward, here at Daysha, to taking that to our clients. I see some really interesting use cases. Some of the people that I’m talking to right now are just starting in a CTO role, and they want to baseline what they’re doing.

Shimel: Absolutely.

O’Reilly: So, they can look—you know, six months from now, they can say, “How much have we done?” They wanna get an understanding of what have they got in-house right now and then look at it again to see if they’ve been effective in those changes that Garima was—really well-made point by Garima—nice, short, sharp, Plan Do Check Act type changes that are gonna take place over the next six months.

Shimel: I mean, for me [Cross talk]—

Bajpai: Right, I mean, I also want to focus on two specific aspects of ADOC, which stands out. So, a lot of people have actually understood that you cannot remake culture without people-centricity, right? So, that’s a given. And we also understand that emerging technology can be a tool in your toolbox when you’re trying to kind of transition from X state to Y state. And a lot of focus has been placed on processes and frameworks.

What ADOC has been standing out for me is, they have actually reflected to other different things, which are equally essential. The first thing is functional silos. And if you look at large enterprises, what they struggle with, like, what has been in the past is that we have large, functional organizations and units with their own business goals and objectives. And when these people are trying to [Audio skips] towards ________-centricity, towards DevOps capabilities. That’s a large challenge, because you know, it’s not easy to actually break down the functional silos, burn all the ships so that there is no, you know, way back. It has to be carefully done so that we have a possibility of course correction. That’s the first important point.

The second important point which stands out for me is automation. And when I look at automation, I see that data driven approach towards DevOps and how that will ensure that we automate good processes, we automate the capabilities which are creating value, not only end to end, but also within the practices, right? So, that’s—those two things, I think, put together with people, technology, and process would actually make a kind of, a full house for, you know, doing this assessment in one way rather than looking at it in pockets.

Shimel: Agreed. In my mind, you know, Brendan, you mentioned the DORA folks, and of course, Nicole has moved on from Google, she’s over at GitHub, and I don’t know if Jez is even with Google anymore, frankly. But really, with DORA, it wasn’t necessarily a consultant’s tool, though consultants used it.

I think one of the interesting things with ADOC and what I know about how it came about is, it really was designed for consultants to use as part of their consulting practice to help you collectively help your clients get a baseline, a roadmap to show where you’re strong, where you’re weak and going forward so that that consultant can be more helpful, more impactful and, frankly, more efficient, which translates to less dollars, right, that clients waste in kind of circling around to figure out where the ground zero is.

So, I would just point out that, now, that’s one use of ADOC is as a consultant’s tool. There’s also the community version for individual organizations and large enterprises can do it on their own, they don’t necessarily need a consultant. But it really was designed with that in mind, and I think it’s gonna be interesting to see, you know, DevOps especially, it was one of the knocks on DevOps, right? And I’m DevOps.com, I’ve heard this from day one—it’s squishy, there’s no manifesto, like in agile. There’s no real definition. It’s, how do you put your finger on it? You know, it seems like a unicorn or a mythical beast or something.

Well, with ADOC, a lesson I learned early in my career in technology was, everything can be measured, and we measure everything, right? And ADOC kinda gives us a little bit of that certainty, at least as opposed to where you are in DevOps. I’m wondering, have you guys actually had a chance yet to try ADOC out with real world clients and what some of the results or findings were?

Cermak: Yeah, we’re doing that right now with a client and we found that it was much easier to prioritize the activities that they needed to do, so we could align the activities with their roadmap and make sur that they were incrementing in the right direction to address some of the challenges that they had with their own customers.

So, we took a very customer-based approach to prioritization of the activities, and they could really see a difference in the way they were able to engage the customer, deliver the features much faster, get the features agreed properly and through improving their continuous delivery pipeline, for example, and their lead time to change would improve and stability and reliability. So, we were able to prioritize exactly the things that were important right then to them with their customer base.

Shimel: That’s great. Anybody else?

O’Reilly: Well, just to echo, I personally haven’t had experience with the product just yet, but I have had that experience of walking the corridors with a spreadsheet, meeting people, capturing data—yeah, and that’s a really slow, inefficient way to capture a picture that evolves very, very quickly. So, I’m excited to do it this way, which is just way more efficient and lower cost to the client.

And I think in terms of the consulting add-on, it’s that experience of having been through several transformations that allows us to analyze the results once they’re produced. And then, exactly like Louis has pointed out—prioritize, right? So, what changes can we make that will fit in with the natural rhythm of this company if they’ve got, if they’re not yet delivering on a regular basis? So, that kinda stuff I think is where the consulting value add is.

Shimel: Interesting.

Cermak: And the other thing that’s quite nice is, you can get the marginal gains as well, because you don’t have to tackle everything at once. You can improve one metric and then go and do the next one and the next one and continually improve, which I think the teams that we’re working with have really appreciated that, because they can see an uptick quite quickly, they get motivated, they wanna change more, they want to get better, and they keep going and keep going and getting those marginal gains.

Bajpai: Right, and this is actually an extremely good point, because it gives us a perspective and this is something which I actually have experienced myself. I went to experience ADOC myself and, you know, I wanted to bring my team in as well. But what has happened in this journey, I have realized that organizations have overestimated the potential of short-term gains. And when I say that, often, people do not realize that whatever transition or transformation from a DevOps perspective they are doing, they are looking at short-term gains and probably losing out on the long-term potential of, you know, larger, creating larger value, long-term value, right?

So, this is also a reflection that when you take this kind of an assessment, you would potentially have two aspects—what you can do in short-term, and what you achieve through several initiatives put together incrementally and how that outcome would look like in long-term, right? So, that is where these assessments would become very crucial for organizations, at especially big organizations.

Shimel: Agreed, agreed. With all three of you being involved in consulting, I’m gonna ask you to take off your consultant hats for a second and for end users, end user organizations out here watching this, is ADOC of use to them without a consultant?

Cermak: Oh, definitely, yeah. You know, we do build software, and we use those metrics ourselves, and we’re constantly monitoring those dials and—okay, we do some consulting around this stuff as well, but actually, we’ve got customers that we need to deliver to, and for us, being on top of all of that all the time is really helpful.

So, anyone taking the assessment would be able to identify changes that they could make in their own engineering. That’s straightforward. It’s really set out very easily to understand.

Shimel: Very good.

O’Reilly: Alan—

Shimel: Go ahead, Brendan.

O’Reilly: Yeah, so, there’s a friend of mine who’s a lawyer, and pretty much any time I ask him a question, the answer is, “It depends.” [Laughter] So, in answer to your question about whether end using organizations can benefit from this or that consultant, I would say it depends. It depends on the mindset in the organization, the appetite for learning, the appetite for change, and so forth.

But for sure, even just to kind of get started and to understand where they’re at, there’s huge value here without a consultant just to get a picture of where you’re at.

Shimel: Agreed. And I think that’s part of this, too, and I think that’s also where there’s the—I forget if they call it the enterprise edition, which is the full, I forget how many questions there are already, hundreds of questions, versus the community edition, which I think is just, like, 30 questions or something like that. But even the community edition, you know, I think brings value to an end user organization who may just want to—you know, let me stick a flag in the ground as a marker and then see where we go from there.

Guys, we’re about out of time, but I want to thank you, all three of you, for joining us. I should mention that if you’re interested in ADOC, right, Assessment of DevOps Capabilities, you can find out a lot more about it at the DevOps Institute site, which is DevOpsInstitute.com, and I know that there’s licenses for consultants, end user organizations. We’ve also launched the membership association, so as a member of the association, I think you get some use of it as well.

But for now—Louis, Brendan, Garima—thank you so much for joining us on TechStrong TV. Good luck, and keep us posted with ADOC.

Cermak: Thanks, Alan—pleasure.

Bajpai: Thank you.

O’Reilly: Thanks, Alan.

Shimel: Thank you. Alright, we’re gonna take a break and we’ll be right back with our next guest.

[End of Audio]

April 22, 2021
Survey Shows Acceleration of Centralized IT Management

An OpsRamp survey published this week suggests large enterprises are looking to finally unify IT management at at time when it is becoming increasingly complex.

OpsRamp, a provider of an IT management platform based on machine learning algorithms, polled 132 IT operations directors working for organizations in the U.S. with more than 500 employees that spend more than $5 million a year on IT.

According to the results, the survey found nearly 73% of respondents plan to roll out a digital operations management platform in 2021. That platform will, hopefully, enable IT teams to unify hybrid cloud computing environments made up of monolithic and microservices-based applications via a single console. Another 21%of respondents expressed interest in such a platform, if budgetary constraints allowed.

More than half (51%) also noted they expect to employ automation and AIOps (51%) in the form of machine learning algorithms and process automation to reliably cut down on the number of alerts being generated, identify the root cause of issues and eliminate repetitive manual tasks.

The two main goals identified in the survey are to enable greater agility and resolve issues faster (46%) and reduce costs by consolidating tools (41%).

While the rise of cloud computing has made IT more agile, each cloud service that is added to the IT firmament tends to increase the total cost of IT. Each IT platform in use today requires a dedicated team to operate, which conspires to increase both the number of tools required and the overall size of the IT staff required to master them.

In the wake of the economic downturn during the COVID-19 pandemic, the amount of pressure being applied to reduce those costs has increased significantly. Security and compliance concerns are also starting to push organizations toward centralizing the management of IT as much as possible. The OpsRamp survey notes that platform security (59%) is the most critical attribute of a next-generation IT platform, followed by hybrid infrastructure management capabilities (52%) and flexible pricing (37%).

Deepak Jannu, senior director for product marketing at OpsRamp, said achieving those goals across what has become an extended enterprise requires a platform that has the ability to first capture telemetry data in real-time and then aggregate that data in way that enables artificial intelligence (AI) models to learn how the IT environment actually operates.

Armed with that data, it then becomes possible to employ AI to both troubleshoot IT issues more easily and surface recommendations that address capacity utilization, noted Jannu.

It may be a while before AI-enabled IT management platforms are commonplace in the enterprise. However, it’s also apparent a number of factors are driving IT leaders to reevaluate their existing platforms as IT environments become increasingly more complex.

Regardless of what platform an organization eventually opts to standardize on, the rate at which IT management tasks are being automated is only going to increase as machine learning algorithms become more familiar with the IT environment in which they are deployed. That doesn’t necessarily mean those algorithms will remove the need for IT administrators and DevOps teams, however. It does mean that the tasks the individuals on IT teams are focused on will, hopefully, provide a lot more value to the business.

April 21, 2021
DevOps Trends to Watch in 2021

Engineers at fast-paced startups often wear both Dev and DevOps hats. Early in my career, as a developer at a sentiment-analysis startup, I was tasked with writing scripts to spin up servers, configure them, install necessary software and deploy the code we wrote. My custom shell scripts, Python scripts and cron jobs did all the heavy lifting so that we could reliably deploy software and scale resources on AWS. Fast forward to today, and that problem of confidently building and securely deploying software has become an even more enormous challenge at scale.

The Cloud-Native Application Stack

A simple looking e-commerce app today can easily be a big set of coordinating microservices that are powered by containers, orchestrated using Kubernetes and hosted on multi-cloud/hybrid cloud. This combination enables faster delivery of code, cheaper hosting bills and more efficient use of server resources.

While the application components certainly became smaller over time, organizations today are tasked with managing tens of thousands of containers in a production environment. In a 2020 Cloud Native Computing Foundation (CNCF) survey, 92% of respondents said they use containers in production, a 300% increase from just 23% in the first survey from March 2016. Such coordination complexity at scale is being managed by software delivery pipelines. Just as Ford revolutionized the automobile industry with its assembly line, these integrated DevOps products are massively impacting verticals like finance and insurance.

DevOps Trends

Trend: Developers are now Decision Makers and Influencers

We are excited to see the increasing impact and influence of developers when it comes to the selection of tools. The top-down sales motion largely doesn’t work in the cloud-native world, and needs to be supplemented by bottom-up community adoption. The best developers pick peer-recommended tools to solve these tech challenges. Developers today leverage large communities, self-organize and pick the best open source tools to kickstart small projects that have potential to impact the growth curve of any organization. For vendors, developer evangelism becomes an important strategy for customer acquisition.

Trend: Integrated Software Delivery Pipelines

CIOs typically pick category-leading products for each category of the DevOps toolchain, like CI, CD, SAST and SCA. Historically speaking, integrating these tools to maintain an effective CI/CD pipeline has been painful and requires stitching together tools using custom scripts. We hear the frustration when we speak with maintainers of such scripts. To address this challenge, we think end-to-end integrated DevOps platforms will become the norm. Such platforms will offer a few best-in-class components with built-in automation, and will enable organizations to integrate other DevOps tools in a plug-and-play manner.

Trend: Shifting Left with Code

The shift left phenomena, which enables fixing of bugs/security issues early in the software development life cycle when it is cheapest to do so, is accelerating. By implementing a shared security responsibility model and enabling developers to take more ownership related to infrastructure, security and other critical production issues with code, organizations are shipping code faster and cheaper with less bugs.

Trend: Low-Code DevOps Automation

Many workflows are pretty standard in the DevOps world. Go look at an alert in production; prioritize it, look at the knowledge base for a fix, create incident, apply fix, resolve ticket and then repeat. This example is an oversimplification, but there are many repeat use cases that can be automatically managed. These workflows can and should be automated, which can make DevOps teams more effective. We think that customizable low-code automation interfaces will evolve to act as secret weapons for effective DevOps engineers.

Trend: Security at the Speed of Dev

Software release cycles a decade ago used to be four to six months long — if not longer — but today, the software delivery life cycle and release cadence is accelerating. The 2019 State of DevOps report suggests that the top performers deploy software 1,460 times per year, which is effectively four deployments per day. Security, compliance, vulnerability management, authentication and authorization all have to move at this speed, otherwise delivery of secure code is just not possible. We see the most innovative CISOs adopt a shared ownership model of security that enables application component owners to detect and fix vulnerabilities. In this shared model, a good balance between agility (so that development is not blocked) and security (so that risk can be managed) needs to be struck.

Are You Building the Next Big Thing?

Cloud infrastructure is an enormous opportunity that Gartner expects to grow to $412 billion by 2024 and many enterprises are just getting started. If you haven’t been paying attention, Satya Nadella now runs Microsoft and Andy Jassy will soon run Amazon. Both these leaders are pioneers in cloud, having led Azure at Microsoft and AWS at Amazon, respectively. This shows how much the largest companies, as measured by global market cap, believe in the opportunity related to the cloud. We couldn’t agree more.

April 21, 2021
Languages and DevOps: Mainframe

In true DevOps fashion, I nearly forgot to cover the mainframe after saying it warranted its own blog!

There is a lot going on right now in the mainframe space, so the question of mainframe language choices has several different answers. We’ve got Linux running on the mainframe, we’ve got cross-compiled open source capable of running on the mainframe natively, we’ve got mainframe-based containers and we even have traditional mainframe tools being ported to other platforms (hello, COBOL on Linux!) so that development can happen on smaller platforms but target the mainframe.

Taken in the order they came available, we have traditional mainframe development, in languages like COBOL and RPG, right on the mainframe in standard LPARs. This has gotten easier over the years, but is still basically the same development environment that was used in the 1980s, with prettier UIs.

Then we have Linux LPARs available that allow for development right on the mainframe. There are issues and differences from other Linux development environments, but not as many as you’d think. This approach gives a broad selection of development tools on a Linux platform, making it an appealing choice.

Next came containers, which can then run Linux in a container with the container offering the interfaces that Linux expects to find. This approach has the benefit of being more like development on other Linux platforms, but interaction with core services can still be platform-specific at times. This approach offers the breadth of open-source languages that came naturally with containers plus the few that have been ported over to z/OS.

Then came speed porting. The biggest weakness of the above options was that it took too long for vulnerabilities to be addressed and new functionality to be moved over to the mainframe. The fact is that porting a language took time, and wasn’t as rapid as the response for the core Linux kernel. There has been a movement to improve the dev tools that create the ports, and now response time to CVEs or improvements in the kernel is much better. A side effect of this movement has been increased availability of languages and tools that are used by DevOps to get things done.

Finally, we have recently received support for traditional languages on other Linux platforms. Most notably, given the number of lines of code out there, COBOL is now officially supported on Linux, meaning you can develop on the desktop without a ton of overhead, and then deploy to the mainframe. This has the effect of enabling DevOps processes and mentality for more traditional languages. This is still new, and the differences between developing on one platform and deploying on another will have to be worked out, but it is a step in the right direction.

So what should you use?

Mainframes are generally a more controlled environment than most DevOps teams are used to. The work required to support and update a given language, keeping it and its libraries up-to-date is more complex than just a generic Linux system, and that gives rise to very focused shops in the mainframe space. Generally speaking, you will be told what to use, or more to the point, which languages are supported. If you get input into the process, recommend Python, as it has the biggest pool of developers and libraries out there at the moment. But chances are, the mainframe team already has a language in place that they want DevOps teams to focus on, and in that case, use what’s available.

No matter which language you end up using, API-wrapping mainframe functionality is a time honored tradition at this point, make the mainframe do the heavy lifting and return just what your application needs. Think of it as a big, smart database that can pre-process results for you at high speed, in high volumes.

And keep rolling.

Mainframes have been around forever. They’ve been pronounced dead dozens of times, and yet the security, stability and locality of a mainframe keeps them in the mix. Data gravity helps too – the data is there, the routines to do processing on the data are there, handing that data off to another platform for presentation isn’t that big a deal, and the mainframe just keeps ticking along. Because (at this point) generations of developers have made it tick. Keep that tradition alive if you’re in a mainframe shop; they are literally the core of systems in the places they are in heavy use.

April 21, 2021
How the DevOps Skill Journey Fosters a Learning Culture
Human skills, among others, have always been important; now more than ever. Empathy, diversity, creativity, personal commitment have been put to the test. The increase in remote work tends to challenge individuals in unique ways, and these human skills are becoming critical.

Top 5 Skill Domains for 2021

This year, 69% of respondents to the Upskilling 2021: Enterprise DevOps Skills Report told DevOps Institute they consider human skills a “must-have or very important” skill for the DevOps human, among other priorities, around skill domains as seen below:

It is no surprise that challenges around the DevOps journey continue around people, process and technology with all respondents giving them equal weight (33%).

To continue thriving within DevOps or to initiate a DevOps journey, individuals, leaders and organizations should take action in developing skills across these areas. Right?

Why, then, are only 39% of respondents reporting they have a formal upskilling program? What is the challenge for the other 61%? Recognizing the importance of these skill domains is one thing – doing something about it seems quite another. Why is so little done about upskilling?

Can these skills be taught? Well, maybe, or maybe not.

Research has shown that knowledge and skills can be developed by most people, while abilities, personal capabilities and characteristics are rather innate. Furthermore, existing human capabilities are constructs that facilitate individual learning, adaptability and development and maintain their value even when drastic environmental changes occur.

When we asked how organizations are prioritizing human skills during our research, we found that the human capability or domain is among the top five must-have skill domains. We also found that organizations have adopted both the concept of the learning organization and safety culture as two important constructs for DevOps.

3 Things to Understand about Human or “Soft” Skills
1. The human skill or domain is fundamentally different. While technical skills – such as container orchestration, CI/CD automation technologies and cloud infrastructures – can be developed through training (and these skills can be certified), the human skill domain is somewhat different. First, some human skills are shaped through experience and practice in both our personal and work life. For example, collaboration and communication skills are shaped during projects and can be adjusted according to the culture of the organization. Other skills, such as curiosity, might have been developed during early life stages or are somewhat innate.
2. Take inventory of your top human skills. Our survey respondents offered great insight on the top must-have human skills within DevOps. Every situation, however, is different, but take a look at these and do a self-check on your human skills.
3. Human skill development requires coaching and continuous feedback. No matter what methodologies organizations leverage, one key aspect is that individuals and leaders alike should have a coach or mentor who provides guidance and support. And, just as important is continuous feedback around an individual’s personal development. Today’s human resources teams might be able to assist. If not, network with others which might already model the skills needed and that can provide support and guidance.
Must-Have Human Skills According to the C-Suite

The graph below shows executives, management and individual contributors all have similar priorities when it comes to human skills. (Blue is executives and management, and orange are individual contributors)

There are some differences in opinion when it comes to multitasking, customer skills, thought leadership, business understanding, creativity, diversity and inclusion, empathy, personal value, communication, interpersonal skills, sharing and knowledge transfer.

How does one identify an individual’s potential for soft skills and other skill development? Unfortunately, this is not an exact science. One possibility is to understand what DevOps capabilities are needed, first, and then perform a skills inventory check. Then, the question becomes, how can you encourage individuals to develop their skills?

The Skill Journey Toward a Learning Organization

A learning organization is composed of individuals and leaders all interested in increasing their capabilities and improving their skills to achieve results they care about. Learning how to do something is extremely rewarding, and, at the same time, achieves specific organizational goals if the learning is aligned to the needed capabilities.

First, assessing what DevOps capabilities are missing, paired with a skill path for the DevOps team members, is a win-win.

5 Tips for Your DevOps Journey
1. Make continuous learning part of the DevOps journey. Continuous learning and education must be a fundamental best practice within your DevOps journey. A first step is always education, providing learning resources and allowing engineers and individuals to participate with and learn from others in the community.
2. Don’t hide the experts. Another critical step is determining if subject matter experts are being sheltered or siloed away. Instead, learn how to embed those people within teams, so they can spread their subject matter expertise by osmosis to others on the team.
3. Growth of skills requires time and budget. According to Jayne Groll, “Enterprises should take intentional steps to support the growth of their DevOps humans.” Some may assume that engineers learn organically, but in truth, engineers need time and budgeted resources to develop new skills, hone existing skills, experiment and improve their capabilities. This could be in the form of time to attend a virtual conference, go to a training class, practice in a sandbox and/or doing peer-to-peer mentoring. Just take a look at site reliability engineers, for example, which are tasked with spending half of their time “making tomorrow better than today” by reducing toil through automation. DevOps engineers should be allotted a similar amount of time to do the same.
4. Adopt a buddy system. The adoption of a coach or buddy system to further aid individual learning and growth is an excellent idea; pair less-skilled DevOps humans and engineers with highly skilled and experienced ones, and watch growth happen. This helps growth on both sides – both for the experienced buddy and the less-experienced person, in learning and teaching.
5. Equip your leaders. Leaders are expected to know how to do things, right? To have all the answers? Well, not really. Some research has shown that the largest group of leaders today are “accidental leaders,” and 60% of frontline leaders say they have never received any training for their new role. Leaders need to learn and upskill, as well.
The challenges of the past year showcased the importance of human skills in transformational efforts. Without the spirit, willingness, collaboration, cooperation and competencies of IT professionals around the world, many enterprises would not have been able to adapt to rapidly changing business models in a pandemic-fueled economy. As we continue into this next decade, organizations should plan to invest in upskilling their humans as much (or even more) than their technology.
April 21, 2021
Redis Labs Extends Reach of In-Memory Database

At its online RedisConf 2021 event, Redis Labs today detailed how the company’s in-memory database will be employed to drive real-time applications that are at the core of most digital business transformation initiatives.

Yiftach Shoolman, Redis Labs CTO and co-founder, said there are four initiatives at the core of the Redis Labs effort to replace legacy databases that organizations have historically employed to drive batch-oriented application processes.

RedisRaft, which will become available with release 7.0 of the Redis database in the third quarter, provides a module based on the Raft consensus algorithm that enable multiple Redis servers to be consumed by an application as if they were one single, fault-tolerant, logical cluster. That capability will provide IT teams with the ability to distribute Redis servers closer to the point where data is being consumed by a distributed application without incurring any conflicts between databases, said Shoolman.

At the same time, Redis is extending its Active-Active technology to enable integrated data models to be deployed in a way that allows distributed applications to run as close as possible to the point where data is being created and consumed.

Finally, RediSearch is being updated to include support for indexing capabilities in JSON applications using a nested document model, while a RedisAI capability will add an inference engine for artificial intelligence (AI) applications that can be employed as a feature store through which the deployment of AI models within applications can be managed. That approach also promises to significantly increase the overall performance of AI applications, noted Shoolman.

Historically, the Redis in-memory database has been best known for the caching capabilities it provides developers. However, Shoolman noted 66% of organizations that employ the Redis database are using it for additional use cases. As microservices that are often configured with their own database become more prevalent, the adoption of an in-memory database platform should only increase, added Shoolman.

Many of those microservices are being employed within the context of a digital business transformation project. The issue that organizations are coming to terms with is most of those processes need to run in near real-time. That requirement will eventually push more organizations toward deploying a database based on a distributed database running in-memory that appears as one logical entity to an application, said Shoolman.

At its core, Redis a data store that supports structures such as strings, hashes, lists, sets, sorted sets with range queries, bitmaps, hyperloglogs, geospatial indexes, and streams. Among other capabilities, it includes support for built-in replication, Lua scripting, transactions and varying levels of persistence. The company claims the core Redis datastore has been downloaded 1.5 million times in the last nine years.

On top of that installed base, Redis Labs then makes available tools for deploying and managing Redis within large-scale application environments. Most recently, Redis Labs raised an additional $100 million round of financing to bring its valuation to above $2 billion. The company now claims to have more than 8,000 paying customers, with a 54% compound annual growth rate in sales for the past three years.

It’s hard to say to what degree Redis data stores might supplant legacy database platforms. However, the one thing that is certain is there are lot more databases running in memory that, in many cases, obviate the need for a traditional relational database.

April 20, 2021