devopsy.kinsta.cloud

Category: DevOps Culture

Key Success Factors for Your Nearshore Initiative

The IT talent shortage is becoming a growing problem. The U.S Bureau of Labor Statistics projects the industry demand for software developers will grow 22% between 2019–2029. This is significantly higher than the average demand growth rate of 4% for all other occupations. As the IT talent gap grows, companies across industries are looking to nearshore partners to help fill the void. Unlike offshoring, nearshoring relies on partners geographically located in the Americas, as opposed to Europe or Asia, for example. It is a growing segment of the market. Case in point, Brazil’s software development services exports have grown 1000% in the last 10 years, gaining an increasing foothold from in the United States for innovative software development and digital transformation initiatives.

There are several advantages to outsourcing your software development needs to a nearshore partner, including access to high quality IT talent, lower costs, faster time-to-market and, most importantly, the productivity increase that comes with full-time collaboration. With a growing list of global providers in the market, how do you go about selecting the right partner? Here are three important factors to consider.

Time Zone Proximity

There are many benefits to being in close geographic proximity and in a similar time zone to your development partner. Being able to have regular iteration reviews during your regular business hours can make or break any project. Everyone is already taxed with a full day of video meetings during the pandemic – why make it more difficult? I’ve heard horror stories from corporate leaders trying to manage partners on the other side of the world with a 12-hour time difference. Time and distance challenges can be disruptive, lower productivity and hurt team building. The importance of real-time communication, feedback and collaboration cannot be underestimated during the software development process. We are all looking forward to getting back to face-to-face meetings, and being in closer geographic proximity will make travel easier once we can gather together again.

Cultural Alignment

Teams built on collaborative partnerships are the most successful. The first step to ensuring your teams can successfully tackle your software projects is to make certain the team knows what problems you are trying to solve. That can be more difficult than it seems when working with global partners. My own experience has taught me that different cultures have different work styles and can approach projects in different ways. The culture similarities between North and South America for example, make it easy to stay culturally aligned and approach projects in a more collaborative way. Creativity, innovation and collaboration need to be guiding principles, and your partners should be recognized for their technological capabilities, results and business acumen. Strong teams offer professional support and recommendations during the co-creation process, adding value across all deployment cycles. When projects require collaboration and coordination, cultural alignment becomes more important than ever.

Ability to Scale

Most companies start out outsourcing small projects to their partners to gain confidence and trust in the relationship. As that initial project comes to a close, companies often turn to their partners for larger, more complex projects only to find out their partner does not have the right resources to handle that next project.

As a result, they are forced to restart their search and onboard a new partner. You can avoid this all together by ensuring your nearshore partner can successfully scale their teams and capabilities based on both your current and future needs. Doing your due diligence and verifying a partner’s track record and their ability to rapidly build teams can head off any growing pains. Can they double or triple the team’s size and hire a specific type of developer or engineer? If yes, where are these professionals coming from – in-house or via an outside agency? Does the team have exceptional qualifications and the right skills? How are your partner’s retention rates? It is important to understand how and where the resources will come from when you are ready to expand your relationship. You want companies that are built as nearshoring partners and who are able to scale as you scale.

Nearshore partnerships are not a one-size-fits-all relationship. To be successful, organizations need to ensure their teams can work cohesively together. The benefits of proximity and time zone overlap, as well as cultural alignment, can ensure the partnership adds value. Collaboration and teamwork will be easier with a team whose culture is similar to yours. Partners who can meet that criteria, and who can demonstrate a similar set of work values, can help you reach your software development and digital transformation goals.

April 26, 2021
How Shifting Testing Left Can Boost DevOps Velocity
DevOps is all about balance: How can speed and quality be balanced to best meet the demands of customers who want exceptional digital experiences as fast as possible? DevOps looks to meet these expectations head-on by helping to deliver a faster cadence of new builds to customers while keeping quality high.

What’s more, quality should continuously improve, build-over-build, so that fewer fixes are needed with each iteration. While inserting more testing earlier in the development life cycle may at first seem like a hindrance to the speed DevOps promises, it will soon empower teams to release higher-quality software, faster, as shift-left testing becomes a routine part of the team’s processes.

Many organizations wait until the very end of their software development life cycle (SDLC) before executing testing. With this structure, testing can and does find unexpected bugs late in the SDLC, where issues are the most difficult and expensive to resolve. With this approach, testing becomes a bottleneck, impacting that ever-important goal of efficient and Agile software development and deployment. An unexpected outcome: it also negatively affects team morale, as developers who spent plenty of time and effort to build and ship code quickly now have to delay their deployment process in order to correct issues.

There is a more efficient approach that organizations can put into place, however.

Shifting Testing Left

Testing can be implemented during and throughout the critical stages of design and development of software, instead of after.

This is true regardless of how a team approaches development methodologies: Agile, waterfall, continuous or a combination of these methods. When testing is shifted left, it becomes the responsibility of the entire organization, not just dedicated QA testers. That means SDETs, developers and others should all be active in testing along with QA. When these groups are involved in testing early on in the SDLC, there are plenty of benefits, including:
- A list of user workflows and test validation conditions can be created to streamline testing during development processes.
- Testers can catch potential issues much earlier in the process so they can be fixed during earlier stages, instead of after code is done being developed.
- Developers and testers can work with code they are familiar with to edit existing code and fix issues themselves.
Building a Continuous Feedback Loop

Another major advantage of shifting testing left in the SDLC is that it creates a continuous feedback loop. Software can be updated and improved continuously as it is being built, instead of when it is “complete” and ready to ship. This model of a feedback loop saves time and ultimately ensures no show-stopping issues make their way into the hands of customers.

Test Automation in the SDLC

One more key element to shifting testing left is test automation. Unfortunately, automation is often not leveraged completely or to its fullest potential. A recent report found that 74% of QA testers see test automation taking place outside the regular test cycle. Moreover, only 45% of organizations use test automation, and only 50% use it for continuous integration/continuous delivery (CI/CD).

For test automation to succeed, it must involve developers creating the automation code and testers, then editing, maintaining and executing that code. When test development, including automated testing, is moved into the SDLC properly, it leads to automated and integrated unit testing, which helps achieve that goal of increased efficiency for DevOps teams.

Automation is obviously crucial to the success of DevOps, but manual processes must also be considered. Some types of testing – like exploratory, payments, usability and accessibility – cannot be automated (at least not fully), but they should also be accounted for as part of a shift-left strategy.

The origin of DevOps methodology and the focus of developers involves the goal of building and deploying code as quickly as possible. But testing software to make sure it works as intended is an important step that should be included throughout the SDLC. When testing only happens at the end of the process, it can be an inhibitor to the overall goal of increased development velocity. But when testing is shifted left and involved earlier on in the SDLC, it can do the opposite – by creating a continuous feedback loop and implementing automation to drive improved DevOps velocity.

Ultimately, shifting testing left in the SDLC leads to features getting to users and customers faster, an overall improvement of software quality and a reduction of QA testing time and bug fixes.
April 23, 2021
Earth Day: The Green Side of Computing

When we think of climate change and the existential threat of global warming, our minds often conjure images of dirty factories pumping out carbon dioxide or fuel-guzzling cars sitting idle with their engines rumbling. We think of ice sheets tumbling into the ocean and sea levels rising, and we know that grand, sweeping changes are needed. However, while it’s always good to have one eye on the elephant in the room, it’s equally important to step back and look at the picture as a whole.

Changing Climate Perceptions

Have you ever thought about your own impact on the environment when settling down to stream something on Netflix? Or how your Zoom meeting tomorrow might contribute to the climate emergency? A recent study found that, despite a record drop in global emissions in 2020 due to the pandemic, the switch to remote working and at-home entertainment still presented a significant environmental risk. One hour of streaming or video conferencing, for instance, can emit 150 – 1,000 grams of carbon dioxide, so perhaps you should think twice about switching on your camera during your next virtual meeting. That’s because everything you do online is facilitated by a data center somewhere in the world. That data center will need to be powered, cooled and maintained in order to operate effectively. The majority of countries reported around a 20% rise in internet traffic from the beginning of the pandemic, meaning those data centers have to work even harder.

Now, let’s apply this to the world of business. Do organizations consider their cloud activity or data center usage when assessing their environmental footprint? If the latest figures are anything to go by, simply turning down the thermostat by a few degrees may not be enough. Today, the technology business sector consumes roughly 7% of the world’s energy. By 2025, Greenpeace estimates that figure could skyrocket to more than 20%, mainly due to innovations in cloud computing designed to drive efficiency, automation and, of course, facilitate remote working.

Data center workloads increased by 500% between 2010 and 2020 as more of the working world moved online. In the U.S. alone, data centers use roughly 660 billion liters of water per year to generate electricity and cool their facilities. In recent years, greenhouse gas emissions from data centers have eclipsed that of the entire global airline industry. That means your company’s use of technology could contribute more to global warming than a transatlantic flight. It’s clear that our perceptions around climate change and our relationship to the environment need to broaden considerably.

Turning the Tide

Fortunately, it seems the tide is turning. There are initiatives popping up all around the world to try and counter the impact our data centers are having on the environment. The European Union, for instance, has launched the 2030 Climate and Energy Framework – which aims to reduce greenhouse gas emissions by 40% by 2030, with all sectors doing their part. While governments have traditionally focussed on the more obvious issues, such as the burning of fossil fuels by cars, it seems their scope is now broadening in the knowledge that every action – large or small – helps. If we can solve the problem of polluting data centers now, before our dependence on them grows exponentially, we can prevent the next big ‘elephant in the room’ from emerging. Google knows this, and vowed to power all of its data centers using carbon-free energy by 2030. Other companies, like Equinix, are even further along with their green initiatives to protect and power a more sustainable digital world. In 2019, over 90% of Equinix’s global data center energy consumption was covered by renewable energy sources, pushing it one step closer to its long-term goal of using 100% clean and renewable energy.

Computing has always had an impact on the environment, but it was so negligible that it barely registered. Those impacts are still negligible when looked at in isolation, but when multiplied by billions, they quickly add up. By changing the way we think about cloud computing and accepting it as a net contributor to the climate crisis, we can start to steer our future to safer shores. Making sure you work with data centers who share your passion for a green future is a great first step.

April 22, 2021
ADOC: Assessing DevOps Culture

The Assessment of DevOps Capabilities (ADOC) is crowdsourced, vendor-neutral and designed for individuals, teams and organizations who want to baseline their current DevOps state, identify the next target state, gain insights into how to improve their organization and team capabilities and measure and accelerate continuous improvement during their DevOps journey.

The assessment models five DevOps dimensions – the human aspects, process and frameworks, functional composition, intelligent automation and technology ecosystems – to help you gauge your DevOps maturity and identify areas for improvement. In this TechStrongTV interview, Alan Shimel talks with Garima Bajpai, Louise Cermak and Brendan O’Reilly about reshaping culture to get the most out of any DevOps journey. The video is below, followed by a complete transcript.

Announcer: This is Digital Anarchist.

Alan Shimel: Hi, everyone. Thanks for joining us on another segment here for TechStrong TV. This is a special segment panel for TechStrong that I’m gonna be conducting, here. I’m happy to have you join us. I’m also really happy to have three interesting people join us. Let me introduce you to them. And if I mispronounce people’s names, I apologize; we do the best we can here on TechStrong TV, right?

But we’re gonna start with Garima, is it Bajpai? Fair? Good? Okay. Garima, why don’t you introduce yourself?

Garima Bajpai: Thank you, Alan. I’m Garima Bajpai from Ottawa, Canada. I am the Founder for the DevOps Community of practice here in Canada, which has three chapters, Toronto, Edmonton, and Ottawa. I’m also the organizer for the DevOps Summit which happens in Canada November 8th and 9th, and I am Ambassador with DevOps Institute, as well as an Ambassador with the CD Foundation. So, today, we are going to have a very interesting dialogue on DevOps assessments. So, I can’t wait to get started, right?

Shimel: Yeah, thank you, thank you. Thank you for all you’re doing for the DevOps community. Next up, let me introduce you to Brendan O’Reilly. Brendan, welcome—a little bit about yourself?

Brendan O’Reilly: Sure. So, my company is called Daysha DevOps. We’ve been working with the DevOps Institute now for about five or six years, predominantly delivering their training. And I myself am a consultant, and like the other speakers here today, really looking forward and excited about what’s happening with ADOC, because I see a big opportunity for our customers to get deeper into their DevOps journeys, to progress those journeys.

Shimel: Fantastic, and thank you. And then last, but certainly not least is, we have Louise Cermak—Louise, did I get that right?

Louise Cermak: Almost, yeah.

Shimel: Well, say it right, because you’re entitled to have your name said right, it’s my fault. So, please, how do we pronounce your name?

Cermak: Thanks, Alan, it’s “Chermock.”

Shimel: “Chermock,” okay.

Cermak: Yeah, so, I’m Louise Cermak. I’m a Principal Consultant with Catapult CX. We build software ourselves, and we also consult in agile and DevOps and have been doing so for sort of 10, 15 years. And we’re excited for ADOC, because actually, having the combination of being engineers as well, we can really understand the capability and how to apply it.

Shimel: Absolutely. Well, all three of you, thanks for joining us. So, a couple of you have mentioned ADOC, right, and this is something that our friends at the DevOps Institute announced, I guess it was last week or the week before now. And ADOC, for our audience who may not be familiar, stands for Assessment of DevOps Capabilities. And this is a new assessment model that the folks at DevOps Institute have been working on.

Look, you know, I know they’ve been working on it for over a year already. It’s been primarily driven by Helen Beal, who is a—Helen is a very well-known DevOps person in the community, but she’s also the Founder of the DevOps Institute Ambassadors, and has been a DevOps consultant herself for many, many years, as long as I’m doing DevOps.com.

And then Helen was really helped with this by Eveline Oehrlich, who’s a DevOps Research Director at DevOps Institute, a former Forrester VP. And then I also know that she’s been helped a lot with people like the three of you, who are real world DevOps consultants who are helping organizations every single day with their DevOps transformations and capabilities.

So, before we jump into ADOC, though, let’s take it up a notch and talk about DevOps assessments in general, right? Are they—why are they important? Why are they important, and if anyone wants to give us sort of a brief history of evolution or time around these assessments and how they’re used, let’s say, by consultants and/or end user organizations.

Garima, I know you have a lot of experience, so if you don’t mind, I’m gonna ask you to kick it off, and then have the rest of the panel chime in.

Bajpai: Right, and I actually term DevOps transformations as a journey towards remaking culture. So, it has been largely a complex journey, because of two factors, actually. The first factor is that the complexity brings along with, you know, a lot of focus on how you can upskill people, how can you actually transform your knowledge based and skill set to a journey which is basically based on experimentation, on learning, and learning from failures, right?

And the second aspect is that there is no such literature available. You know, there is a lot of talk about DevOps transformation, but the literature is in the making. So, there is limited academic support as well as support from a best practices perspective how to do this transformation, right? So, these assessments come very—it is very crucial, because it will help you assess where you are, and every organization will have unique challenges, right?

So, it is important to assess where you are in your journey and what you can do. You know, it’s not like a traditional transformation journey of three to five years where you are relying on a lot of targets ________ models which were already defined. It’s continuously evolving, right? So, you will have to have a view on what you can do today and how you can incrementally, in short cycles, transform.

So, what happens in this process is that, you know, assessment becomes more crucial and essential, because there are discrete efforts. There are efforts in short cycles, and this assessment and measurement will help you course correct, will help you understand where you are heading to in your journey and what is needed as next steps to ensure that you incrementally build value, right?

So, that’s why, I think, these assessments are extremely important. We will talk along this conversation how ADOC can actually help you on this journey as well as what are the major impediments and challenges organizations have and what should they look out for.

Shimel: Fair.

Cermak: I think, over the last few years where we’ve been helping organizations, what we’ve noticed is that many of them are trying to do their own assessments, and I’ve been into so many organizations where people have got Excel spreadsheets and they’re trying to measure how they’re performing.

Because, you know, Garima’s right—there’s a lot of learning and experimentation. A lot of it is self-learning, which is what we encourage, right, from a DevOps culture perspective. But people wanna know that they’re going in the right direction, so they start with their own spreadsheets, fill them in, try and measure their capabilities. And a tool like ADOC helps organizations have a much more structured way of doing that, and obviously, it’s more scalable than an Excel spreadsheet, but people wanna know that they’re doing the right things.

Shimel: Agreed.

O’Reilly: Just a little bit about the history, perhaps Alan—you know, ADOC is brand new, but there was a progenitor in the form of DORA, if you recall.

Shimel: Sure, sure.

O’Reilly: Yeah, so, that was with Jez Humble and Nicole Forsgren. So, when they produced their State of DevOps report, it was a means by which organizations could form views about what good looked like, so you have your release and then so on.

And it definitely raised people’s curiosity about—can we do this, can we measure? And of course, one of the principles with any DevOps transformation is to be data-driven. And so, the 180-odd questions that are included in the enterprise edition are very much about trying to capture and turn subjective opinion into something quantitative, a good first guess. And I think DORA has been successful, it’s now part of the Google organization, I believe, but this is a nice independent offer from the DevOps Institute.

We’re really looking forward, here at Daysha, to taking that to our clients. I see some really interesting use cases. Some of the people that I’m talking to right now are just starting in a CTO role, and they want to baseline what they’re doing.

Shimel: Absolutely.

O’Reilly: So, they can look—you know, six months from now, they can say, “How much have we done?” They wanna get an understanding of what have they got in-house right now and then look at it again to see if they’ve been effective in those changes that Garima was—really well-made point by Garima—nice, short, sharp, Plan Do Check Act type changes that are gonna take place over the next six months.

Shimel: I mean, for me [Cross talk]—

Bajpai: Right, I mean, I also want to focus on two specific aspects of ADOC, which stands out. So, a lot of people have actually understood that you cannot remake culture without people-centricity, right? So, that’s a given. And we also understand that emerging technology can be a tool in your toolbox when you’re trying to kind of transition from X state to Y state. And a lot of focus has been placed on processes and frameworks.

What ADOC has been standing out for me is, they have actually reflected to other different things, which are equally essential. The first thing is functional silos. And if you look at large enterprises, what they struggle with, like, what has been in the past is that we have large, functional organizations and units with their own business goals and objectives. And when these people are trying to [Audio skips] towards ________-centricity, towards DevOps capabilities. That’s a large challenge, because you know, it’s not easy to actually break down the functional silos, burn all the ships so that there is no, you know, way back. It has to be carefully done so that we have a possibility of course correction. That’s the first important point.

The second important point which stands out for me is automation. And when I look at automation, I see that data driven approach towards DevOps and how that will ensure that we automate good processes, we automate the capabilities which are creating value, not only end to end, but also within the practices, right? So, that’s—those two things, I think, put together with people, technology, and process would actually make a kind of, a full house for, you know, doing this assessment in one way rather than looking at it in pockets.

Shimel: Agreed. In my mind, you know, Brendan, you mentioned the DORA folks, and of course, Nicole has moved on from Google, she’s over at GitHub, and I don’t know if Jez is even with Google anymore, frankly. But really, with DORA, it wasn’t necessarily a consultant’s tool, though consultants used it.

I think one of the interesting things with ADOC and what I know about how it came about is, it really was designed for consultants to use as part of their consulting practice to help you collectively help your clients get a baseline, a roadmap to show where you’re strong, where you’re weak and going forward so that that consultant can be more helpful, more impactful and, frankly, more efficient, which translates to less dollars, right, that clients waste in kind of circling around to figure out where the ground zero is.

So, I would just point out that, now, that’s one use of ADOC is as a consultant’s tool. There’s also the community version for individual organizations and large enterprises can do it on their own, they don’t necessarily need a consultant. But it really was designed with that in mind, and I think it’s gonna be interesting to see, you know, DevOps especially, it was one of the knocks on DevOps, right? And I’m DevOps.com, I’ve heard this from day one—it’s squishy, there’s no manifesto, like in agile. There’s no real definition. It’s, how do you put your finger on it? You know, it seems like a unicorn or a mythical beast or something.

Well, with ADOC, a lesson I learned early in my career in technology was, everything can be measured, and we measure everything, right? And ADOC kinda gives us a little bit of that certainty, at least as opposed to where you are in DevOps. I’m wondering, have you guys actually had a chance yet to try ADOC out with real world clients and what some of the results or findings were?

Cermak: Yeah, we’re doing that right now with a client and we found that it was much easier to prioritize the activities that they needed to do, so we could align the activities with their roadmap and make sur that they were incrementing in the right direction to address some of the challenges that they had with their own customers.

So, we took a very customer-based approach to prioritization of the activities, and they could really see a difference in the way they were able to engage the customer, deliver the features much faster, get the features agreed properly and through improving their continuous delivery pipeline, for example, and their lead time to change would improve and stability and reliability. So, we were able to prioritize exactly the things that were important right then to them with their customer base.

Shimel: That’s great. Anybody else?

O’Reilly: Well, just to echo, I personally haven’t had experience with the product just yet, but I have had that experience of walking the corridors with a spreadsheet, meeting people, capturing data—yeah, and that’s a really slow, inefficient way to capture a picture that evolves very, very quickly. So, I’m excited to do it this way, which is just way more efficient and lower cost to the client.

And I think in terms of the consulting add-on, it’s that experience of having been through several transformations that allows us to analyze the results once they’re produced. And then, exactly like Louis has pointed out—prioritize, right? So, what changes can we make that will fit in with the natural rhythm of this company if they’ve got, if they’re not yet delivering on a regular basis? So, that kinda stuff I think is where the consulting value add is.

Shimel: Interesting.

Cermak: And the other thing that’s quite nice is, you can get the marginal gains as well, because you don’t have to tackle everything at once. You can improve one metric and then go and do the next one and the next one and continually improve, which I think the teams that we’re working with have really appreciated that, because they can see an uptick quite quickly, they get motivated, they wanna change more, they want to get better, and they keep going and keep going and getting those marginal gains.

Bajpai: Right, and this is actually an extremely good point, because it gives us a perspective and this is something which I actually have experienced myself. I went to experience ADOC myself and, you know, I wanted to bring my team in as well. But what has happened in this journey, I have realized that organizations have overestimated the potential of short-term gains. And when I say that, often, people do not realize that whatever transition or transformation from a DevOps perspective they are doing, they are looking at short-term gains and probably losing out on the long-term potential of, you know, larger, creating larger value, long-term value, right?

So, this is also a reflection that when you take this kind of an assessment, you would potentially have two aspects—what you can do in short-term, and what you achieve through several initiatives put together incrementally and how that outcome would look like in long-term, right? So, that is where these assessments would become very crucial for organizations, at especially big organizations.

Shimel: Agreed, agreed. With all three of you being involved in consulting, I’m gonna ask you to take off your consultant hats for a second and for end users, end user organizations out here watching this, is ADOC of use to them without a consultant?

Cermak: Oh, definitely, yeah. You know, we do build software, and we use those metrics ourselves, and we’re constantly monitoring those dials and—okay, we do some consulting around this stuff as well, but actually, we’ve got customers that we need to deliver to, and for us, being on top of all of that all the time is really helpful.

So, anyone taking the assessment would be able to identify changes that they could make in their own engineering. That’s straightforward. It’s really set out very easily to understand.

Shimel: Very good.

O’Reilly: Alan—

Shimel: Go ahead, Brendan.

O’Reilly: Yeah, so, there’s a friend of mine who’s a lawyer, and pretty much any time I ask him a question, the answer is, “It depends.” [Laughter] So, in answer to your question about whether end using organizations can benefit from this or that consultant, I would say it depends. It depends on the mindset in the organization, the appetite for learning, the appetite for change, and so forth.

But for sure, even just to kind of get started and to understand where they’re at, there’s huge value here without a consultant just to get a picture of where you’re at.

Shimel: Agreed. And I think that’s part of this, too, and I think that’s also where there’s the—I forget if they call it the enterprise edition, which is the full, I forget how many questions there are already, hundreds of questions, versus the community edition, which I think is just, like, 30 questions or something like that. But even the community edition, you know, I think brings value to an end user organization who may just want to—you know, let me stick a flag in the ground as a marker and then see where we go from there.

Guys, we’re about out of time, but I want to thank you, all three of you, for joining us. I should mention that if you’re interested in ADOC, right, Assessment of DevOps Capabilities, you can find out a lot more about it at the DevOps Institute site, which is DevOpsInstitute.com, and I know that there’s licenses for consultants, end user organizations. We’ve also launched the membership association, so as a member of the association, I think you get some use of it as well.

But for now—Louis, Brendan, Garima—thank you so much for joining us on TechStrong TV. Good luck, and keep us posted with ADOC.

Cermak: Thanks, Alan—pleasure.

Bajpai: Thank you.

O’Reilly: Thanks, Alan.

Shimel: Thank you. Alright, we’re gonna take a break and we’ll be right back with our next guest.

[End of Audio]

April 22, 2021
DevOps Trends to Watch in 2021

Engineers at fast-paced startups often wear both Dev and DevOps hats. Early in my career, as a developer at a sentiment-analysis startup, I was tasked with writing scripts to spin up servers, configure them, install necessary software and deploy the code we wrote. My custom shell scripts, Python scripts and cron jobs did all the heavy lifting so that we could reliably deploy software and scale resources on AWS. Fast forward to today, and that problem of confidently building and securely deploying software has become an even more enormous challenge at scale.

The Cloud-Native Application Stack

A simple looking e-commerce app today can easily be a big set of coordinating microservices that are powered by containers, orchestrated using Kubernetes and hosted on multi-cloud/hybrid cloud. This combination enables faster delivery of code, cheaper hosting bills and more efficient use of server resources.

While the application components certainly became smaller over time, organizations today are tasked with managing tens of thousands of containers in a production environment. In a 2020 Cloud Native Computing Foundation (CNCF) survey, 92% of respondents said they use containers in production, a 300% increase from just 23% in the first survey from March 2016. Such coordination complexity at scale is being managed by software delivery pipelines. Just as Ford revolutionized the automobile industry with its assembly line, these integrated DevOps products are massively impacting verticals like finance and insurance.

DevOps Trends

Trend: Developers are now Decision Makers and Influencers

We are excited to see the increasing impact and influence of developers when it comes to the selection of tools. The top-down sales motion largely doesn’t work in the cloud-native world, and needs to be supplemented by bottom-up community adoption. The best developers pick peer-recommended tools to solve these tech challenges. Developers today leverage large communities, self-organize and pick the best open source tools to kickstart small projects that have potential to impact the growth curve of any organization. For vendors, developer evangelism becomes an important strategy for customer acquisition.

Trend: Integrated Software Delivery Pipelines

CIOs typically pick category-leading products for each category of the DevOps toolchain, like CI, CD, SAST and SCA. Historically speaking, integrating these tools to maintain an effective CI/CD pipeline has been painful and requires stitching together tools using custom scripts. We hear the frustration when we speak with maintainers of such scripts. To address this challenge, we think end-to-end integrated DevOps platforms will become the norm. Such platforms will offer a few best-in-class components with built-in automation, and will enable organizations to integrate other DevOps tools in a plug-and-play manner.

Trend: Shifting Left with Code

The shift left phenomena, which enables fixing of bugs/security issues early in the software development life cycle when it is cheapest to do so, is accelerating. By implementing a shared security responsibility model and enabling developers to take more ownership related to infrastructure, security and other critical production issues with code, organizations are shipping code faster and cheaper with less bugs.

Trend: Low-Code DevOps Automation

Many workflows are pretty standard in the DevOps world. Go look at an alert in production; prioritize it, look at the knowledge base for a fix, create incident, apply fix, resolve ticket and then repeat. This example is an oversimplification, but there are many repeat use cases that can be automatically managed. These workflows can and should be automated, which can make DevOps teams more effective. We think that customizable low-code automation interfaces will evolve to act as secret weapons for effective DevOps engineers.

Trend: Security at the Speed of Dev

Software release cycles a decade ago used to be four to six months long — if not longer — but today, the software delivery life cycle and release cadence is accelerating. The 2019 State of DevOps report suggests that the top performers deploy software 1,460 times per year, which is effectively four deployments per day. Security, compliance, vulnerability management, authentication and authorization all have to move at this speed, otherwise delivery of secure code is just not possible. We see the most innovative CISOs adopt a shared ownership model of security that enables application component owners to detect and fix vulnerabilities. In this shared model, a good balance between agility (so that development is not blocked) and security (so that risk can be managed) needs to be struck.

Are You Building the Next Big Thing?

Cloud infrastructure is an enormous opportunity that Gartner expects to grow to $412 billion by 2024 and many enterprises are just getting started. If you haven’t been paying attention, Satya Nadella now runs Microsoft and Andy Jassy will soon run Amazon. Both these leaders are pioneers in cloud, having led Azure at Microsoft and AWS at Amazon, respectively. This shows how much the largest companies, as measured by global market cap, believe in the opportunity related to the cloud. We couldn’t agree more.

April 21, 2021
How the DevOps Skill Journey Fosters a Learning Culture
Human skills, among others, have always been important; now more than ever. Empathy, diversity, creativity, personal commitment have been put to the test. The increase in remote work tends to challenge individuals in unique ways, and these human skills are becoming critical.

Top 5 Skill Domains for 2021

This year, 69% of respondents to the Upskilling 2021: Enterprise DevOps Skills Report told DevOps Institute they consider human skills a “must-have or very important” skill for the DevOps human, among other priorities, around skill domains as seen below:

It is no surprise that challenges around the DevOps journey continue around people, process and technology with all respondents giving them equal weight (33%).

To continue thriving within DevOps or to initiate a DevOps journey, individuals, leaders and organizations should take action in developing skills across these areas. Right?

Why, then, are only 39% of respondents reporting they have a formal upskilling program? What is the challenge for the other 61%? Recognizing the importance of these skill domains is one thing – doing something about it seems quite another. Why is so little done about upskilling?

Can these skills be taught? Well, maybe, or maybe not.

Research has shown that knowledge and skills can be developed by most people, while abilities, personal capabilities and characteristics are rather innate. Furthermore, existing human capabilities are constructs that facilitate individual learning, adaptability and development and maintain their value even when drastic environmental changes occur.

When we asked how organizations are prioritizing human skills during our research, we found that the human capability or domain is among the top five must-have skill domains. We also found that organizations have adopted both the concept of the learning organization and safety culture as two important constructs for DevOps.

3 Things to Understand about Human or “Soft” Skills
1. The human skill or domain is fundamentally different. While technical skills – such as container orchestration, CI/CD automation technologies and cloud infrastructures – can be developed through training (and these skills can be certified), the human skill domain is somewhat different. First, some human skills are shaped through experience and practice in both our personal and work life. For example, collaboration and communication skills are shaped during projects and can be adjusted according to the culture of the organization. Other skills, such as curiosity, might have been developed during early life stages or are somewhat innate.
2. Take inventory of your top human skills. Our survey respondents offered great insight on the top must-have human skills within DevOps. Every situation, however, is different, but take a look at these and do a self-check on your human skills.
3. Human skill development requires coaching and continuous feedback. No matter what methodologies organizations leverage, one key aspect is that individuals and leaders alike should have a coach or mentor who provides guidance and support. And, just as important is continuous feedback around an individual’s personal development. Today’s human resources teams might be able to assist. If not, network with others which might already model the skills needed and that can provide support and guidance.
Must-Have Human Skills According to the C-Suite

The graph below shows executives, management and individual contributors all have similar priorities when it comes to human skills. (Blue is executives and management, and orange are individual contributors)

There are some differences in opinion when it comes to multitasking, customer skills, thought leadership, business understanding, creativity, diversity and inclusion, empathy, personal value, communication, interpersonal skills, sharing and knowledge transfer.

How does one identify an individual’s potential for soft skills and other skill development? Unfortunately, this is not an exact science. One possibility is to understand what DevOps capabilities are needed, first, and then perform a skills inventory check. Then, the question becomes, how can you encourage individuals to develop their skills?

The Skill Journey Toward a Learning Organization

A learning organization is composed of individuals and leaders all interested in increasing their capabilities and improving their skills to achieve results they care about. Learning how to do something is extremely rewarding, and, at the same time, achieves specific organizational goals if the learning is aligned to the needed capabilities.

First, assessing what DevOps capabilities are missing, paired with a skill path for the DevOps team members, is a win-win.

5 Tips for Your DevOps Journey
1. Make continuous learning part of the DevOps journey. Continuous learning and education must be a fundamental best practice within your DevOps journey. A first step is always education, providing learning resources and allowing engineers and individuals to participate with and learn from others in the community.
2. Don’t hide the experts. Another critical step is determining if subject matter experts are being sheltered or siloed away. Instead, learn how to embed those people within teams, so they can spread their subject matter expertise by osmosis to others on the team.
3. Growth of skills requires time and budget. According to Jayne Groll, “Enterprises should take intentional steps to support the growth of their DevOps humans.” Some may assume that engineers learn organically, but in truth, engineers need time and budgeted resources to develop new skills, hone existing skills, experiment and improve their capabilities. This could be in the form of time to attend a virtual conference, go to a training class, practice in a sandbox and/or doing peer-to-peer mentoring. Just take a look at site reliability engineers, for example, which are tasked with spending half of their time “making tomorrow better than today” by reducing toil through automation. DevOps engineers should be allotted a similar amount of time to do the same.
4. Adopt a buddy system. The adoption of a coach or buddy system to further aid individual learning and growth is an excellent idea; pair less-skilled DevOps humans and engineers with highly skilled and experienced ones, and watch growth happen. This helps growth on both sides – both for the experienced buddy and the less-experienced person, in learning and teaching.
5. Equip your leaders. Leaders are expected to know how to do things, right? To have all the answers? Well, not really. Some research has shown that the largest group of leaders today are “accidental leaders,” and 60% of frontline leaders say they have never received any training for their new role. Leaders need to learn and upskill, as well.
The challenges of the past year showcased the importance of human skills in transformational efforts. Without the spirit, willingness, collaboration, cooperation and competencies of IT professionals around the world, many enterprises would not have been able to adapt to rapidly changing business models in a pandemic-fueled economy. As we continue into this next decade, organizations should plan to invest in upskilling their humans as much (or even more) than their technology.
April 21, 2021
Solving Cloud-Native Challenges in the Rush to the Cloud
The shift to cloud-native environments away from traditional data center infrastructures continues unabated, but security and complexity challenges remain a struggle for DevOps teams.

These were two of the main takeaways in a survey and analyst report published by the IT consultancy firm Flexera.

The survey, based on the responses of 750 IT professionals, revealed that 92% of the organizations represented in the survey have a multi-cloud strategy, 80% have a hybrid cloud strategy and only a minority (45%) have integrated data between cloud environments.

The top challenges, according to the survey results, are:
- security (81%)
- managing cloud spending (79%)
- governance, lack of resources/expertise and compliance (75%)
- comprehending cost implications of software licenses (55%).
The solution for these challenges largely involves — in addition to implementing the right DevOps culture and practices — adoption of the right tools and platforms, especially for security. However, first-generation cloud-cost management tools “simply are not cutting it” for modern architectures, said Asim Razzaq, CEO of Yotascale, which offers a cloud resource optimization platform.

“Today’s modern architectures are dynamic, and ownership is transient and not always well-defined. The greatest strength of containers and Kubernetes is the ability to scale on-demand and share cloud resources among multiple applications,” said Razzaq. “Tracking cloud costs in modern architecture requires a strong ability to automate cost attribution — or tag management — for those dynamic resources, and the ability to understand usage at the application level, even for containers that are sharing resource instances.”

First-generation cloud management tools are, in this way, unable to provide the information DevOps teams require to understand how to automate cost attribution for modern architectures, said Razzaq.

“[DevOps teams] can’t break down costs into any level of meaningful detail for engineering and finance teams to digest and act upon,” said Razzaq. “Conversely, newer approaches that are focused on Kubernetes cost management lack the ability to understand and break down the costs of non-cloud-native workloads.”

Indeed, tools are often frustrating to engineering teams because of their static custom reporting, broad and generalized recommendations and no effective workflow for effective collaboration between finance and engineering, Razzaq said.

“The biggest source of frustration is the inability to drill down and manage costs in a way that matches the way that these teams do business. There may be 10 teams, each with five production applications running in the cloud, for a particular account,” Razzaq said. “Engineers need to see, at the application level, what their resource utilization, costs and cost anomalies are in order to make the best cost/performance decisions for their applications. Without that, they’re flying blind; having to defend their cloud usage without any supporting data.”

As an example, a misconfigured Kubernetes cluster can “cause a massive spike in costs,” Razzaq explained. “Without detailed cost and utilization reporting at the team and application level, an engineer can spend weeks digging through spreadsheets and logs trying to track down the source of the anomaly. That’s weeks of their time that they aren’t [using to] be productive on new features.”

Razzaq said there are a number of solutions DevOps teams can adopt to address these issues, including automated cost attribution and tag management, actionable recommendations and comprehensive observability capabilities.

Automated cost attribution and tag management “is the foundation of accurate and detailed cloud cost management,” said Razzaq. “Engineers need tools to help them set tag management policies, identify untagged instances and update tags quickly and easily in their cloud environment,” he said. “With this in place, cloud-native workloads can be automatically tagged as they are dynamically created, ensuring that cloud costs are always clearly attributed to the right teams and applications.”

Actionable recommendations at the cloud-instance level with closed-loop feedback also are an important process “so that engineers can see exactly why the recommendation is being made, and what the cost impact will be of taking that action,” Razzaq said. “The ability to provide feedback on the recommendation and dismiss recommendations that don’t align with business needs ensures that new recommendations are relevant,” Razzaq added.

Finally, tools that provide a unified view of both cloud-native and traditional workloads that have been shifted to the cloud help to ensure “that engineering and finance have a single source of truth for their cloud costs that enables them to make smarter decisions for their business,” Razzaq said.
April 19, 2021
Do You Need a Cloud AI Engineer?

For all the technological advancements we are witnessing in AI and machine learning, the practical benefits remain elusive in the non-tech business world, frustrating decision-makers as AI investment climbs. For example, over the past several years, many Fortune 500 companies have invested in strong data science teams and AI labs, but still struggle when it comes to producing and scaling their models. Moreover, a recent report by Algorithmia found that, while enterprise budgets for AI and ML are rapidly increasing for 83% of organizations studied, deployment challenges persist.

These deployment issues often begin when leadership fails to exert enough upfront engineering focus to ensure their proof of concepts (POCs) are “real-world ready.” The challenges are compounded when businesses put these models into action using insufficient, outdated or cobbled-together IT infrastructures. This combination results in POCs that either fall apart under real-world conditions or are rife with inefficiencies. Fortunately, cloud tech today provides the opportunity to overcome these challenges.

Businesses can enable development, experimentation and scaling of data science and AI solutions with more flexibility and speed than ever through cloud platforms. A design-led AI project that uses the tools available in the cloud can help organizations create business impact and realize ROI much faster. So while the role of data scientist may have been the ‘sexiest job’ in recent years, we may be seeing a new ‘sexiest job’ title emerging: the cloud AI engineer.

Why do we need a Cloud AI Engineer?

Traditionally, a successful AI project in the cloud requires a team with different skills and titles, including data scientists, big data engineers, cloud engineers and full stack developers. This has led businesses to look at ways to merge these specialties into one specific title. Enter the cloud AI engineer who, theoretically, would be able to cover all these duties — including knowledge of and experience with the tools and accelerators that cloud platforms now offer, as well as demonstrated aptitude for fast-paced development in AutoML. These are skills and tasks previously spread out across other areas and roles within the organization.

What is the Job Description?

So what would this job actually look like? The cloud AI engineers of the future will focus on the deployment of AI and ML models at scale in the cloud, and on integrating them with existing products and IT systems. Additionally, cloud AI engineers would need knowledge of the different types of machine-learning algorithms; familiarity with frameworks like Tensorflow and PyTorch; hands-on experience with data processing; analysis and feature selection using analytic tools and ETL/ELT frameworks (Apache Spark, Hive, Amazon, Athena, etc.); and model deployment experience on cloud platforms like AWS, Azure or Google Cloud.

Obviously, tackling all of these would be a huge undertaking for one individual; given the vast workloads on both the cloud and engineering fronts, it isn’t possible to replace these workers and their skills with just a handful of cloud AI engineers. Instead, cloud AI engineers will sit between the two disciplines and the workforces and will serve as a key bridge to easily deploy models and integrate them with enterprise systems and tools for consumption.

Where are We Going to Find Cloud AI Engineers?

Given that cloud AI engineers will have a robust level of responsibility, they will quickly become some of the most sought-after team members — but that also means talent for the role will likely be in short supply.

Additionally, individuals with this broad skill set will not appear overnight — at least not en masse. Therefore, to build the cloud architecture capable of delivering the real-world AI results that businesses need today, tech companies need to find ways to break down the internal divides between data scientists and cloud engineers so that they can cross-train, learn and develop skills in each area. If this is done correctly, not only will tech companies have a more well-rounded workforce, but they will be able to deliver the practical AI success that non-tech businesses are looking for today.

Despite excitement about AI and its benefits, operational challenges remain. By focusing on the cloud and building up specialists in this space, AI can begin to drive the practical results the business world needs far more quickly.

April 15, 2021
Survey Shows Mounting DevOps Frustration and Costs

A survey of 400 developers and DevOps team members in the U.S. and Europe found, on average, 15 hours a week are being spent managing tasks such as debugging pipelines and waiting for tests and builds to complete.

Conducted by the market research firm Vanson Bourne on behalf of Garden, a provider of a platform for automating application development and deployment tasks in Kubernetes environments, the survey finds more than three-quarters of respondents (76%) said the time spent on these and other manual DevOps tasks is generally being wasted.

Garden CEO Jon Edvald said, all told, 39% of a 40-hour workweek is typically devoted to these tasks. When the number of developers and DevOps specialists that are employed today are factored into that equation along with their salaries, as much as $61 billion is being wasted on these tasks in the U.S. alone, he noted.

The top sources of DevOps frustration cited by survey respondents are waiting for pipelines to run (76%), waiting for builds and tests (74%) and setting up, maintaining and debugging pipelines/automation (71%). In fact, only 11% of all respondents are completely happy with their development setups and workflows.

Those hidden costs may even increase as organizations bring on additional DevOps expertise. A full 83% of respondents said their organizations are planning to hire more staff or increase their use of freelancers/outsourcing into DevOps roles in 2021. Three quarters (75%) said DevOps budgets have increased or will increase compared to 2020.

In addition, the adoption of more complex platforms, such as Kubernetes, may also conspire to increase costs, noted Edvald. Respondents at organizations not yet using Kubernetes spend an average of 14.3 hours per week on low-level DevOps tasks, as compared to 16.5 hours a week for those who are already using Kubernetes. Nearly two-thirds of respondents (62%) said their organizations are already using Kubernetes, while 31% are currently evaluating it, to varying degrees. A full 95% of respondents using Kubernetes have faced challenges after rolling it out, including training and onboarding (37%) or complexity of setup/configuration (35%).

Of course, not all the time that might be saved is going to be applied to tasks that increase overall productivity; nearly half of respondents (49%) said if they had it, they would spend more time developing new products and services.

Specifically, 46% said they would improve speed and delivery of existing products and services, while 44% said they would focus on improving security for existing products and services.

Edvald said the survey makes it clear there’s still plenty of opportunity to automate repetitive DevOps tasks. Organizations are spending too much time on the care and feeding of DevOps platforms that takes time away from building and deploying applications, noted Edvald.

Less clear is when organizations will reach some type of DevOps breaking point. Garden is betting that the transition to cloud-native applications based on microservices will eventually force the DevOps automation issue. However, in organizations that only know the DevOps processes they have used for years, change may be a long time in coming.

April 14, 2021
1Password Extends Encryption to Automate Secrets Management

1Password this week added a Secrets Automation platform to its offerings that enables IT organizations to more easily encrypt, manage and orchestrate credentials, application programming interface (API) tokens, keys and certificates.

In addition, the company revealed it has acquired SecretHub, a provider of a separate secrets management tool. Terms of the acquisition were not disclosed.

Carson Brown, senior product manager for 1Password, said the SecretHub team will now focus on the development of the 1Password Secrets Automation platform based on encryption technology previously created by 1Password.

Finally, 1Password this week also announced an alliance with GitHub under which it will, at some future date, enable DevOps teams to employ SecretsHub to encrypt secrets used to access the source code repository. 1Password already provides integrations with HashiCorp Vault, Terraform, Kubernetes and Ansible, in addition to client libraries written in Go, Node and Python.

The 1Password platform for managing and securing passwords is already in use by more than 80,000 businesses worldwide. The Secrets Automation platform now extends the reach of the company’s core encryption technology into the realm of DevSecOps best practices, said Brown.

It’s still early in the cycle of DevSecOps best practices adoption, but it’s clear an initial focus will be secrets management. In the wake of recent high-profile breaches involving software supply chains, there’s now increased scrutiny of secrets management. Cybercriminals have become more adept at scanning for secrets left exposed as plain text. The challenge organizations face is that developers, while building applications, tend to copy secrets for the sake of convenience, which they often forget to delete after an application is deployed in a production environment.

In response, many IT teams vacillate between overly restrictive and overly permissive access controls that are rarely implemented consistently.

Of course, it’s now also only a matter of time before auditors start citing all those unencrypted secrets as compliance violations, Brown noted. A set of DevSecOps best practices based on automatic encryption of secrets eliminates those compliance concerns in a way that is transparent to application developers, noted Brown.

It’s not immediately clear whether it will be DevOps teams that lead the charge to automate the encryption of secrets, or whether cybersecurity teams will take the lead. Regardless of which team assumes responsibility, the number of secrets that need to be tracked keeps expanding as the number of platforms employed increases. Manually keeping track of all those secrets is no longer feasible. Centralizing secrets management is the first step toward regaining control over an IT environment, said Brown.

Secrets management may not always be the first thing that comes to mind when organizations begin their DevSecOps journey. As it turns out, however, securing secrets may very well turn out to be the first order of business as senior business and IT leaders review their entire software supply chain. After all, once it becomes apparent just how dependent organizations are on software, they quickly realize how much of their intellectual property is one compromised credential away from being stolen.

April 14, 2021