devopsy.kinsta.cloud

Author: Benjamin Brial

Empowering Self-Service Users in the Digital Age

We live in an increasingly self-serviced world where digital natives have become the dominant generation in the workplace. Most individuals today are not only more comfortable with self-service, but they also actively choose it. Therefore, creating and evolving an intuitive yet secure self-service portal for users consuming IT resources has become a key undertaking for many organizations.

Portals can take platform teams years to deliver, and maintenance will remain ongoing. Still, if done well, they can massively lighten the load on IT departments for end users by improving the overall experience and reducing the cognitive load, and for platform teams by optimizing operational efficiency and reducing the number of open tickets. Therefore, it speeds up business processes and places autonomy and control in the hands of a happier workforce.

Self-service IT is about centralizing and giving access to important items such as applications, automation, deployment, documentations, cloud, security, FinOps and GreenOps simply and comprehensively. To use a metaphor, it is not about building a hugely over-engineered Boeing 747 with every feature for a quick city break. Quite the opposite. It is about creating a streamlined glider equipped with the right amount of ability to manage the operationally intensive tasks it needs to.

But, of course, self-service has advantages and disadvantages, and achieving a balance is essential when empowering users in the digital era.

Managing Shadow IT

There has already been a creeping level of unauthorized self-service in many organizations. Therefore, shadow IT, the use of IT-related hardware or software by an individual or department without the knowledge or approval of the IT department has become a major consideration for CIOs, and while not all bad, it is far from ideal.

Shadow IT is a clear signifier of the demand for self-service, it also points to a challenge that IT in isolation cannot overcome — people. On traditional infrastructure, organizations are already starting to develop interfaces on top of virtualizations because they have identified the need to provide self-service guardrails to users. However, when it comes to handling multi-cloud deployments at scale, with security, governance, FinOps and GreenOps considerations, this becomes challenging.

People are unpredictable, they can be set in their ways and/or willing to experiment. They might use a new service for a while and then migrate to another without much thought for security or privacy. They can deploy a new service for proof of concept and forget to delete it. They can become attached to processes or systems with which they’re familiar. It is all well and good to declare an intention to build a self-service portal, but there is no guarantee that individuals will flock to use it.

Assuming that any new self-service portal has all the functionality and features to cover the existing demands of the people who will use it — there are (broadly speaking) two options in change management. Force people to do the new thing or make the new thing easier to do than the old thing. Spoiler alert — one of these options tends to be more effective than the other.

Providing comprehensive support to users as they navigate through a self-service portal is, therefore, essential. Organizations should supply clear and accessible guides, conduct informative training sessions and establish a reliable support system. When developers feel empowered and well-supported, they won’t simply use the portal, they will integrate it as a vital component of their toolkit.

In the long-term, this will help foster loyalty and retention among top talent. It is also critical when it comes to building the best possible developer experience and helping the organization and people focus on what matters — improving and maintaining automation and infrastructure on one side and developing their applications on the other.

Maximizing User Experience

Developer experience, or DevX (as I have written about before), is of paramount importance for the successful uptake of any self-service portal.

It might sound obvious but when creating a portal an organization needs to consider the individuals who will use it, rather than creating something that makes people work differently. A well-designed self-service portal is one characterized by its ability to be used without the need for outside assistance, with both the sophistication suited to a skilled developer community and intuitiveness to encourage engagement with a less technically savvy user group. A successful portal therefore is straightforward, intuitive and enriched with the functionality to enhance developer workflows.

On the topic of workflow, the portal must integrate seamlessly with existing tools and systems. Leveraging everything-as-code serves as an effective foundation to ensure smooth integration across all systems, extending beyond just the self-service portal, with minimal error or delay. Incorporating automation features and orchestration can streamline operations, freeing up time from mundane repetitive tasks to improve DevX, allowing for a greater focus on innovation and problem-solving.

Freedom vs Control

Ultimately, portals must strike the balance between freedom and control, which can be achieved by ensuring flexibility with role-based access control. Granting end users the freedom to deploy within a secure framework of predefined permissions creates an environment ripe for innovation within a robustly protected environment. This means users can explore, experiment and innovate without concerns about security boundaries or unnecessary hurdles.

But of course, as with any project, organizations can’t afford to build something and consider that job done. Measuring success is ongoing. Metrics such as how often the portal is accessed, who uses what, which service catalogs are used and how the portal usage should be tracked, along with other relevant data will help point to any areas that need improvement.

It is also important to remember that it is collaborative work between the platform team and end users. And in technology, there is always room for improvement. For instance, recent advances in AI/ML could soon be leveraged to analyze previously inaccessible datasets and generate smarter and faster decision-making.

A well-designed self-service portal should deliver an experience that demystifies and democratizes access to services and data and enhances the project lifecycle process between teams. To achieve this, they should simplify usability, strive to assist developers in leveraging its features effectively, ensure alignment with existing workflows, enhance its functionality and regularly assess its impact on productivity. This will ensure that self-service users survive and thrive in the digital age.

July 9, 2024
How to Get Platform Engineering Just Right

If you want your organization to start moving toward a more efficient and streamlined digital transformation, platform engineering is the only way to go. However, without a defined roadmap, most organizations will pretty quickly feel like they’ve wandered into the tech wilderness without a GPS. According to the State of DevOps Report 2023, it takes, on average, three years for platform engineering to start showing tangible results—but, in reality, the majority of companies will be struggling for a lot longer.

Getting platform engineering right will help organizations create an environment that energizes their teams to develop solutions more efficiently and with greater reliability, ultimately driving business growth. Below, I have set out some of the critical steps that organizations need to take to ensure that the right guardrails are put in place to achieve all of the benefits of platform engineering whilst avoiding some of the common pitfalls.

Step 1: Define Governance – Creating Order Amidst the Chaos

Before embarking on your platform engineering journey, it is crucial to define an effective governance framework. So many organizations attempt to implement new strategies without having a clear understanding of the whos, hows and whats of their processes, which has the net result of creating further complexity.

Start the process by determining your team’s roles and permissions and establish a center of excellence. In doing so, every team member will know their responsibilities and areas of expertise with a clear and comprehensive access control and permission structure. Follow this by creating clear governance policies and procedures which will serve as guidelines for how the platform will be managed and maintained to ensure consistency and security throughout the platform’s life cycle.

Step 2: Set up Your Service Catalog – Building a Digital Buffet

By seeking to improve the developer experience first, organizations will achieve better platform engineering results, which is why one of the most important parts is defining what goes on the digital menu. Once you have established an effective approach to governance, choose which services you want to integrate into your service catalog and self-service portal in a full GitOps approach, which is maintained by the platform team.

An effective platform lets anyone interact with tools and the cloud without needing to be a technical expert. It’s like having a tech buffet where anyone can choose their meal, but only if the self-service portal is intuitive and has a user-centric interface. Access controls need to be built into the platform with predefined governance so that users can only request services for which they have the right permissions.

Step 3: Improve Observability – Shedding Light on the Cloud

In the land of digital transformation, seeing is believing, which is where observability has a role to play. Improving observability is crucial for gaining insights into the platform’s performance and behavior, which involves integrating tools like event and project monitoring, cloud cost transparency, application performance, infrastructure health and user interactions.

In a rapidly growing cloud environment, observability enables teams to keep track of what is happening in terms of cost, usage, availability, performance and security across a constantly transforming cloud infrastructure. Once a project has been deployed, it needs to be managed and maintained across all cloud providers, something which is critical for keeping costs to a minimum but is often a huge and messy task.

Managing this effectively requires monitoring key performance indicators (KPIs) and setting up alerts for critical events, and using logs and analysis tools to gain visibility into application behavior, track errors, and troubleshoot issues more effectively. Finally, implementing tracing systems that can track the flow of requests across various microservices and components helps to identify performance bottlenecks, understand latency issues and optimize system behavior.

Step 4: Plug CI/CD Pipelines – All Aboard the Automation Express

The next step is to turbocharge the platform by plugging in the CI/CD pipelines. These pipelines, connected to your Git, will generate templates with embedded automation, which allow automatic testing of code changes whenever developers commit or merge code into the repository. In doing so, this drastically reduces human error and ensures consistency throughout.

Following this with continuous monitoring and feedback mechanisms in the CI/CD pipelines ensures that only reliable, well-tested code makes it into production. These pipelines are orchestrated through APIs, which will make the digital journey faster and smoother.

Step 5: Set up FinOps & GreenOps – A Balancing Act for a Brighter Tomorrow

Finally, getting platform engineering right is as much about getting the technical basics right as it is about taking responsibility. By combining FinOps and GreenOps approaches and placing sustainability at the orchestration layer to empower users across the organization to consume less infrastructure, organizations can reduce software delivery/cloud costs and even their carbon emissions.

As I’ve written before in these pages, FinOps and GreenOps strategies go hand in hand – integrating cloud cost management and carbon footprint management is the most effective way to ensure that your journey is not just efficient but also environmentally conscious.

And that’s it. By defining governance, setting up an effective self-service portal, improving observability, plugging in CI/CD pipelines and embracing FinOps and GreenOps, organizations will be well on their way to creating a platform that’s not only efficient but also environmentally responsible.

February 2, 2024
A Manifesto for a More Sustainable Cloud Future

I’ve talked about cloud sustainability extensively in these pages previously, but until we start taking tangible steps to collectively reduce our cloud carbon footprint, things are not going to change. Achieving real change comes down to a smarter, more efficient use of resources and establishing a culture of sobriety—both financial and environmental—that flows through an organization. Below, I present the steps necessary for a more sustainable cloud future.

1. Shift the Mindset

First, and arguably most important, there needs to be a shift in mindset that sees IT managers, CIOs and other IT leaders become part of a much broader conversation about how to bake sustainability into the overall IT strategy from the start. In doing so, this offers a path to bring development, finance, ESR and business teams together to ensure greater financial and environmental accountability.

2. Right-Size to Optimize

Regular sustainability assessments can help to identify oversized resources and underused services, which paves the way for smarter allocation. Eliminating unnecessary services and right-sizing instances will help to identify recurring problem areas and serve to reduce carbon emissions. This will create a best practice of optimal resource usage and, consequently, contribute to ongoing savings made to an organization’s bottom line.

3. Embrace Dynamic Scaling

Most cloud providers now offer dynamic scaling capabilities that allow resources to be adjusted according to demand. By embracing these sorts of features, organizations can ensure that resources are used more efficiently while minimizing emissions. This sort of real-time optimization can strike an important balance between performance and environmental responsibility.

4. Fewer Servers Means Less Power Needed

Server virtualization—essentially the process of dividing a physical server into multiple unique virtual servers—offers a clear way of contributing to carbon reduction efforts. By optimizing resources and reducing the physical server count, the consumption of energy can be minimized. Fewer servers mean less power consumption, which directly translates into lower carbon emissions.

5. Only Energy-Efficient Architecture

Platforms such as serverless computing architectures have emerged as a greener alternative to traditional infrastructure, allocating resources only when needed. This creates greater energy efficiency and reduces total carbon emissions. Leveraging these sorts of energy-efficient instance types and hardware, such as ARM-based processors that consume less power than x86-based alternatives, provides a clear way to minimize power consumption and environmental impact.

6. Localization Matters

While the steps above are effective, their impact will be amplified by choosing cloud regions that are based on low-carbon energy sources. For example, regions that are reliant on coal or other high-carbon energy sources should be avoided or used sparingly to minimize environmental impact, but this is not as straightforward as it sounds due to seasonal variations, which might mean that renewable energy sources are subject to fluctuations; the costs associated with storing excess energy from intermittent sources which are both challenging and expensive and the cost of energy trading between regions which can impact carbon footprint.

The truth is that there is no one-size-fits-all solution when it comes to evaluating localization. Assess the energy mix, consider the availability and reliability of renewable energy sources and factor in any seasonal variations or dependencies on fossil fuel-based backup systems. This way, you can make a more informed decision that aligns with your sustainability goals and minimizes the environmental impact of your cloud infrastructure.

7. Align FinOps and GreenOps Strategies

Closely linked to more established terms like FinOps—another framework for managing operational expenditure across an organization, often in conjunction with the cloud and cloud computing—GreenOps is an emerging term that essentially means providing greater cost transparency while promoting a culture of environmental responsibility. The challenge is that making an entire IT ecosystem sustainable requires a step back and all areas of IT to consider their impact on the environment.

This requires collaboration right the way across an organization and with other lines of business which includes areas like HR, marketing, procurement and the accounting departments, all of whom must set business requirements, determine software development and DevOps processes, and adhere to operational principles. Aligning GreenOps and FinOps strategies will allow organizations to start monitoring cloud computing emissions by connecting emissions data directly to their cloud expenses so they can see both the costs to budget and the cost to the environment in the same place.

8. Remember to Turn Off the Tap

Finally, think of efficiency as being the cornerstone of any sustainable initiative, and cloud computing is no different. Optimize cloud instances and you will, in turn, reduce carbon emissions. By evaluating running service instances continuously and strategically and turning off those that are not needed during periods of low demand, organizations can significantly cut down on unnecessary energy consumption. This will not only reduce carbon footprint but also optimize operational costs.

By addressing these complexities head-on, I believe we, collectively, as an industry, have a once-in-a-lifetime opportunity to reduce our carbon footprint and pave the way for a greener, more efficient cloud computing landscape.

October 27, 2023
How Platform Engineering Can Drive Digital Sobriety

If we’re being honest, the “green issue” currently isn’t the priority it needs to be for a lot of companies. At best, they are ticking the box marked ‘sustainability’ and paying lip service to a trendy topic. At worst, they are engaging in nothing more than greenwashing.

Considering carbon emissions from cloud computing have now overtaken those from commercial aviation, this is disheartening for the IT sector. Unsurprisingly, most of these emissions come from inefficient use of the cloud due to overly complex processes and a lack of centralized oversight over cloud infrastructures. And if “emissions” sounds a bit vague, here’s a translation into dollars–$147 billion was wasted in 2022 in cloud spend.

So, when it comes to reducing an organization’s cloud carbon footprint, there is clearly a financial incentive. Even for those who don’t believe in individual responsibility or worse, simply don’t care what kind of world we’ll leave future (and some present) humans to live in, there is money to be saved.

The truth is that companies need to put sustainability first across the whole enterprise, and they need to do it now.

A New Kind of Business

Tackling departments one-by-one or singling out individual processes to make them more sustainable is not going to work. What organizations need is to stop thinking of sustainability as an afterthought and start incorporating it in the decision-making process from the get-go, maybe even as part of their very business model.

This is the sort of business that places its people and the planet above profits. Progressive, modern companies that want to stay in business longer need to invest in their people and take care of the planet instead of chasing endless growth and ever-increasing profit because that, simply put, cannot last.

As a culture, we’re currently at a turning point–late-stage capitalism is running its course, and more people are realizing that constant growth at the expense of well-being and the planet’s dwindling resources is not sustainable (in all senses of the word).

For this to work, it’s about looking at business through a new prism.

Driving Digital Sustainability From the Ground Up

It’s not too late for businesses to become more sustainable. In fact, it is the best time to make life-saving changes. When it comes to tech specifically, there are a couple of things you can do, starting with two key things that need to change.

First, cloud waste has been costing companies billions of dollars for years now. Organizations are set to spend nearly $600 billion on the cloud in 2023, but it seems that most people are still in the dark about the real environmental impact of cloud usage. As with most big corporate changes, it’s all about a change of mindset. We need to accelerate awareness and promote digital sobriety to inspire transformation and decision-making with GreenOps on the enterprise level.

A cloud-first approach has long been the holy grail of IT. However, what we couldn’t anticipate is the sheer volume of resource usage and complexity that comes with it. We simply do not have the planetary resources to support this growth much longer. It’s time to become smarter, leaner and more environmentally conscious by using the “consume better” approach.

Second, think about the KPIs. Change without success parameters will not work. While organizations have a key role in decreasing global cloud spend, they must have the right tools and references. They need to be able to assess the environmental impact of their digital choices at all decision levels.

For the IT sector, this is where platform engineering comes into play. An internal developer platform can act as the critical piece that connects DevX, deployments, governance, security and observability.

With green best practices and cloud carbon footprint visibility at every stage–from the dev deploying a new environment on the cloud to the CFO checking the monthly cloud bill–a platform engineering approach ensures not only organizational but also individual responsibility. When all tools are accessible and processes transparent, nothing gets lost in translation. This means less room for error and cloud waste.

Ultimately, it’s About Consuming Better and Consuming Less

Right now, these KPIs will be financially motivated, and rightly so. But there will come a time when progressive organizations will put their environmental impact at the forefront of their mission statement.

The B-Corp certification, which evaluates companies in terms of their social and environmental performance, grew 63% in 2022. This shows the trend toward moving to a more inclusive and sustainable economy. Partly because it’s in the interest of the businesses’ long-term success and reducing costs, but partly because it is simply the right thing to do.

August 25, 2023
How Platform Engineering Protects Profit, People and our Planet

Platform engineering is emerging as a way of evolving DevOps, with 94% of respondents to the recent State of DevOps report agreeing that the practice helps their organization to reap the benefits of DevOps. The business arguments for platform engineering are clear: In that same report, respondents stated that platform engineering improves efficiency (59%), speeds up delivery time (58%) and improves workflow and process standards (57%).

But in addition to the huge benefits organizations are now realizing in terms of business value, platform engineering is also pretty crucial for a business’s long-term viability. And by that, I mean that platform engineering can be inherently sustainable.

How Can Platform Engineering Be Sustainable?

First, it’s important to note that ‘sustainability’ here doesn’t just mean ‘green.’ Rather, sustainability means that an organization is able to meet the needs of the present without compromising the ability of future generations to meet their needs, too. These are needs that come in three different varieties: People, planet and profits.

This means that to be truly sustainable, platform engineering needs to deliver lasting and stable returns on things like return on investment year-over-year, improved awareness of and reduction in an organization’s carbon footprint and better governance and optimization through monitoring and analytics.

Any approach which supports business growth while simultaneously empowering employees and being kind to the planet is the dream for pretty much every organization. But how exactly can departments and wider organizations unlock all of these benefits? The answer is: There’s no magic bullet.

Sustainable platform engineering is about treating your platform as an iterative cycle: It isn’t the actual creation of the tool that matters, as much as it is the maintenance, upkeep and usability of the platform as your business moves forward. This, naturally, will take time, effort and money—all of which, it’s safe to say in the current climate, are in short supply. But it is crucial that businesses act strategically, looking for areas for improvement and paths that will all add up to a more sustainable business in the long term.

The Three Paths to Success

These paths can be divided into three main areas of focus, and the first is GitOps or infrastructure-as-code (IaC).

A tweak that every organization should start making is using Git as a single source of truth for deployments, enabling developers to work faster by empowering them to use their own code without hands-on involvement from ops teams. This could be combined with optimized infrastructure, moving toward IaC to enable faster and more efficient deployments.

These principles form the foundation of sustainable platform engineering as they protect profits, allowing the business to keep moving forward and investing in the other areas crucial for a sustainable operation. Combining IaC with GitOps ensures speed and simplicity while maintaining a high-quality end product.

Creating an optimized infrastructure is important, but just as important is empowering employees to feel good about the work that they are doing. That’s why implementing a self-service tool alongside an optimized UX is the second area of focus. This removes the pressure on those in the tech department to be infrastructure experts while erasing the need for endless tickets to be raised for issues.

By freeing up DevOps engineers and platform experts to spend more time doing what organizations need them to do, businesses can keep innovating to maintain an edge over their competitors. Self-service capabilities combined with an accessible UX are a useful combination for a much-improved developer experience (DevX).

Finally, it’s important to keep in mind that self-service isn’t just going to benefit end-users; it also has major upsides for improved governance, as well as keeping control of cloud spending and carbon footprint impact. Optimizing and iterating your platform engineering tool means that a business can keep up with the fast pace of the global tech market while also continuing to implement the latest tweaks and upgrades. It all adds up and builds on the organization’s platform engineering solution.

Reaping the Rewards of Sustainability

If an organization follows the above three paths, it is well on its way to a more sustainable business powered by platform engineering. By focusing on targeted, achievable projects to build a solid foundation, organizations can achieve big benefits for people, the planet and profit.

From an economic perspective, sustainable platform engineering ensures legacy infrastructure is modernized, silos are broken down and production is accelerated, all ensuring the business doesn’t risk lagging behind competitors. A stable and secure return on investment is key to sustainability in the medium to long term, and sustainable platform engineering ensures a business is able to weather future challenges and keep a healthy profit margin.

As we all know, underpinning any great business is the people who work there. Techs get stressed by having to interact with infrastructure they know nothing about, DevOps and specialists get stressed when those techs desperately raise tickets in an attempt to get help, and platform engineers get stressed when the weight of providing a solution to these problems rests squarely on their shoulders. Implementing a UX-strong self-service portal relieves the pressure, allowing employees to focus on delivering real value to the business through innovation. A better DevX ultimately leads to better business outcomes.

And, of course, it’s ultimately no good having a successful business if the planet we all share continues to suffer from the effects of the climate emergency. Just like a business that hemorrhages money is unsustainable, so is a planet that wastes its finite resources. By getting control of cloud usage and controlling cloud climate impact, businesses will not only be empowering their people and improving their bottom line, but they will also be making a tangible difference to our shared environment, supporting our planet as it recovers from years of climate neglect.

Platform engineering offers lots of tangible benefits for businesses, but sustainable platform engineering can be even better, not just for a business, but employees and our shared environment. It might just be time to upgrade DevOps and start thinking about building a sustainable organization that supports profit, people and the planet.

June 29, 2023
DevOps and Hybrid Cloud: Life in the Fast Lane, Part Two

In the early years of cloud computing, cost savings were the major driver of adoption. These days, however, organizations are increasingly faced with spiraling expenditures, waste and a lack of control. As cloud usage bills and inefficiencies start to balloon, these challenges are beginning to undermine the benefits of migrating in the first place.

The truth is that optimizing costs in the cloud is more taxing than many organizations first expect. Identifying cost savings that can be made isn’t generally the hard part; effectively implementing those savings is where the real challenge lies.

A lack of control over infrastructure means that many organizations are failing to make the changes needed to truly realize these gains. So common are these challenges that there is even a word for the process to deal with them: FinOps. FinOps, at its core, is the culture and practice of creating visibility and accountability to manage cloud penetration throughout an organization, and it’s an area that is growing rapidly.

This is the second article in my series that looks at what needs to change so that organizations can really start embracing the opportunities provided by hybrid cloud and DevOps. In this installment, I’m going to take a look at the areas of FinOps, cloud waste and automation. In doing so, I hope to show that there is indeed life in the hybrid cloud and DevOps fast lane if handled the right way.

Get Ready for FinOps

If you think hybrid cloud and the topics of FinOps and, more specifically, cloud waste, haven’t come up during discussions, you’ve probably been living under a rock.

It naturally follows that the more agile an organization becomes the harder it needs to work to keep track of waste. Exponential growth is great, but without the right controls a cloud budget will disappear before there has been a chance to see any returns; there are even examples of companies migrating and then deciding to roll back—such are the concerns around lack of control and rising costs.

Based on survey responses from over 750 cloud decision-makers, Flexera found earlier this year in its 2022 State of the Cloud Report that organizations continue to waste significant amounts of money in the cloud. A full 66% of respondents said that cloud usage is “higher than initially planned this year”, and they estimated that they are wasting 32% of their overall cloud spend. A good five years into full cloud migration and most organizations still haven’t got a handle on costs.

The tricky part is that waste can rear its head at many stages over the course of a project, which is why it’s so important to maintain control from day one.

Establishing a baseline of an environment that is deemed to be the known state means that what is versioned can become a source of trust and the desired state of an organization’s cloud infrastructure. How is this possible? Simple: By creating a culture and practice of visibility and accountability throughout an organization.

How do you do that? Yes, you guessed it: FinOps.

Tackling Cloud Waste Head-On

The ideal time and place to start establishing robust FinOps policies and reducing waste come when the move has been made to start embracing a dual DevOps/hybrid cloud approach. The good news is that the principles of FinOps are easy to follow and can be applied at various stages of an application management life cycle and DevOps and hybrid cloud journey.

First, teams need to estimate financial and environmental costs ahead of deployment, something which can be particularly challenging when infrastructure-as-code (IaC) tools such as Terraform, Ansible or Puppet are being used, but which is essential to successful FinOps. Once a team has decided which cloud provider they will be using, how much they plan to spend and, more generally, what criteria will be governing these decisions, effective tag management comes in.

Tagging is the basis for achieving a structured cloud infrastructure, especially in multi-cloud environments that feature different resources being used for different purposes. Tagging means that users can add descriptive metadata (tags) to cloud infrastructure to identify key resources or values such as staging, development or production.

In a rapidly growing cloud environment, consistent and comprehensive tag management helps to keep track of what is happening in terms of cost, usage, availability, performance and security across a constantly transforming cloud infrastructure. Once a project has been deployed, these tags need to be managed and maintained across all cloud providers, something which is critical for the management of waste, but is often a huge and messy task.

Good tags help drive good decision-making and the more options there are for categorizing data, the better. To truly get a handle on tag management, finding a platform that can enable users to visualize and study multi-cloud costs in a central place makes it possible to establish that all-important baseline and start to reverse engineer invoices, optimize monthly costs and tackle waste head-on.

One Cloud, One Strategy

There was a time a few years ago that we used to hear people say this a lot: “I’ll migrate all my assets to a single cloud provider.” Cloud providers were pushing migration to a one-cloud strategy; considered completely normal at the time, but which has since been shown to be flawed, especially in the world of enterprise.

Segregation of responsibility is perhaps the best example of why—there are simply too many responsibilities that keep a modern business afloat to risk putting all your eggs in one (cloud) basket. Consider on-premises legacy apps, sensitive data, critical workloads, the production life cycle, ongoing applications without too many peaks and even backend apps. Then consider the cloud providers, new cloud-first applications, public data, non-critical workloads, the development life cycle and frontend applications dealing with high traffic.

Now imagine all of that suddenly disappears.

Hybrid cloud means eggs in multiple baskets. Yes, it adds a layer of complexity, but when so much is at stake, it’s worth it. And when you have defined where you want to host everything for each application, it becomes much easier to gain clarity. The challenge is that all cloud providers have their own ecosystem, which will either be private or public.

These ecosystems make it possible for everything to be integrated, but what is often not immediately obvious is that even though they are integrated, they are not always aligned. It’s in the interests of the cloud provider to provide the best tools to use their platform but not necessarily the best user or developer experience which is something that requires specialist expertise. That usually leads to a certain degree of lock-in as you generally cannot use their tools for managing other applications that aren’t part of the infrastructure.

This lack of transparency and visibility into other cloud providers can get in the way and impact the ability to get the most out of a dual DevOps and hybrid cloud strategy. However, by adopting a hybrid cloud approach, teams can choose the best cloud provider for each application, putting the team in the best position possible to provide the best services possible.

And finally, Always Consider the DevX Factor

For DevOps to thrive, it is important for companies to stay independent both for their own sovereignty and for creating the best developer experience, or DevX (a topic I have explored in a previous article for devopsy.kinsta.cloud).

It’s easy to fall into the trap of using all the automation capabilities available from each cloud provider, but that locks an organization into a situation where it needs deep expertise across a wide variety of technologies. Adopting a one-cloud, one-automation strategy means that it would never be possible to migrate automation or reuse it in different application deployment processes across various clouds.

This is where automation of configuration management, IaC, CI/CD pipelines and the DevOps and hybrid cloud platform has a significant role to play. Automation on an organization’s own terms is arguably the most critical element in the race to reduce time to market and improve quality where concepts such as shift left are adding complexity and increasing the responsibilities piled on top of developers.

Using automation technologies to maintain autonomy and favor cross-subject expertise and team mobility gives organizations the freedom and flexibility to save time, money and roll out better DevX. Ultimately, it also helps to create the best possible environment for organizations to truly start enjoying a DevOps and hybrid cloud life in the fast lane.

September 19, 2022
DevOps and Hybrid Cloud: Life in the Fast Lane?

Scream if you want to go faster! When it comes to a dual hybrid-cloud-and-DevOps rollout, most organizations would admit they’re eager to get going a little bit faster. But what started so enthusiastically can turn to frustration (and a different kind of screaming starts) when they’ve accelerated into this major transformation without considering all the necessary elements.

The benefits are clear. Hybrid cloud backed up by DevOps is an efficient vehicle for change. A robust DevOps platform can be a make-or-break factor in developing a hybrid cloud strategy that goes the distance. Together, they are a force for transformation, and if organizations aren’t already implementing both DevOps and hybrid cloud, they should be asking themselves why not.

It is true that the move to hybrid cloud is a different journey for cloud-enabled organizations than it is for cloud-native ones. The trip is harder for cloud-enabled, particularly for larger enterprises. Increased release frequency requires tighter alignment and collaboration than is traditionally seen between lines of business, development and IT operations, and this, in turn, drives the need for enhanced collaboration, automation and information transparency.

But although the transition might be harder, it must be seen as inevitable. Organizations could wait another year, maybe a little more, but the transformation of technology and business is going to happen—so why not get started? In the first of two articles, I’m going to take a look at what needs to change so that organizations can start living life in the hybrid cloud and DevOps fast lane.

A Cultural Phenomenon

As has been covered in these pages in some detail already, the move to hybrid cloud backed by DevOps is doomed to fail if organizations think all they need to worry about is tooling and hiring. It’s not. Underlying both of these transformations is a sweeping cultural change. DevOps is about agility, trust and autonomy, and so is hybrid cloud.

Most of all, though, DevOps and hybrid cloud are a commitment to enhancing an organization’s developer experience (DevX), and that’s not something that can be achieved with simply a ticketing tool or by adopting a GitOps approach. DevX means autonomous, unbound movement. It means empowerment, user experience and self-service. These are changes that can only happen with a successful change in business culture, not a tool. And this is an important starting point that many organizations are still struggling to understand.

The People Problem

People are another well-worn topic, but one that continues to present a major challenge. The cultural transition required to successfully implement DevOps can, theoretically, start with just one person. But for a successful outcome, it actually requires long-term change as deeply-ingrained beliefs are disbanded and tribal loyalties are broken down.

For instance, many believe that implementing DevOps requires hiring new people. That belief embraces two fallacies: One, that organizations will be able to hire the teams they need and two, that DevOps is just a job title. As we know, a good DevOps team is not hired, it is developed through the melding of development and operations. The old silos, where devs developed and ops operated, is a relic from less-enlightened times.

Today, it requires learning how to build automation, often open source, while managing on-premises, private and public cloud infrastructure. DevOps is a philosophy that seeks to help others, and with that comes the need for collaboration, encouragement, persuasion and other soft skills.

Security and Privacy Are not Straightforward

Moving to a hybrid cloud means that more care needs to be taken with regard to security and privacy. One of the main problems is that it’s no longer enough for security teams to reign from on high. With control moving from the server room to the cloud and from highly-siloed ops engineers to the whole team, security policies also need to change. Competent developers will make the right decisions, but they need the right bottom-up tooling that takes their decisions from theory to reality and safeguards those who aren’t as aware.

These must be a catalyst to make organizations sit up and take a long look at what security in the age of cloud, containers and microservices really means. If the simultaneous move to DevOps and hybrid cloud is that catalyst, then so be it. Just as DevOps is revolutionizing the way teams approach their work, DevSecOps is going to revolutionize the way they look at security.

Security shows its face at various stages of infrastructure: When a cloud organization is created and structured; when a vault is integrated to ensure that key passwords are secured and even when a pipeline is launched that triggers dedicated security software. This isn’t an exhaustive list, but it illustrates how organizations need to start taking a methodical, step-by-step look at security in their hybrid cloud.

What About GitOps?

Gartner recently found that the average company uses 28 infrastructure tools. That’s simply too many to be handled and understood by one person. Even specialist DevOps engineers can’t be experts in all the cloud providers, OpenStack and VMware. Other studies showed that a developer generally only uses 5% of their Git repositories or an average of five repositories.

The mistake lots of companies make is thinking that suddenly shifting to using APIs with a GitOps approach is adequate to handle the change to hybrid cloud. It’s not, and can actually be detrimental to DevX. Why? There’s too big a gap—it’s like just doing a long weekly run the day before a marathon. You might start, but you’ll be severely compromising your ability to meet your goal. Even DevOps engineers struggle to maintain a single source of truth and well-maintained reusable repositories.

The solution is to plan for the long term. It’s not realistic to believe that one or two DevOps engineers can be the saviors of a hybrid cloud approach. There are far too many tools, best practices and options—between Ansible, Terraform, Kubernetes and Concourse CI/CD to name just a few—to be managed by a handful of devs, no matter how talented they are.

That’s why the true way to hybrid cloud mastery lies in the ability to get the whole tech team pulling in the same direction, collaborating, upskilling and leveraging their strengths to support the parts that make the whole. This might sound like a pipe dream, but there are companies out there talking the talk and walking the walk.

Start Your Engines

It is futile to tell an organization exactly how to get started since many are already taking the first steps and every organization’s situation is unique. There is more to cover; in the next installment, I’ll be taking a look at areas like FinOps, automation and cloud waste. Hopefully, you’ll see the fast lane is clear up ahead.

May 23, 2022
Bridging the Developer Experience (DevX) Gap

They say our past experiences have an impact on how we view life and our general outlook. Routine behaviours and patterns become a program that we follow day in, day out. We stop using our minds to their fullest potential and become trapped in unquestioning, continual cycles of process and emotion.

The same could be said of our experiences in the present—our worlds shaped by the environments around us. The people, practices, tools and culture. Only by challenging these accepted patterns can we start to think about changing our mindsets.

DevOps is a mindset; a way of forming an agile relationship between development and operations teams to speed up delivery of software to support business operations and improve customer experience. By and large, it has been successful. Companies react to change more effectively; software is developed and deployed swiftly. It has become a necessity across most organizations.

There are challenges, as we all know, however. The State of DevOps Report 2021 found that 83% of organizations are implementing DevOps practices, but most are still failing to scale adoption. It’s a situation that hasn’t changed for four years. Why? Among other things, the report states there are too many silos, not enough skilled people, a saturated market with too many tools—and a culture that prohibits scale.

Given that the end goal of DevOps is to generate untrammelled innovation across an organization, getting the culture right is critical. Developers need an environment where they have the confidence to innovate and contribute back to their community, help grow the business, improve their skill sets and further their careers without being limited by the people, practices and tools around them.

Above all, it’s about creating the best possible developer experience (DevX) in which to thrive.

The Developer Experience Gap

Stephen O’Grady at Red Monk, the developer-focused analyst firm, originally wrote about a DevX (developer experience) gap in this post back in October 2020.

Stephen likened the experience you get from an iPhone to the expectations of developers. The iPhone’s closed SDK created an environment that allows all developers to work in a consistent way. Functions are tightly integrated so that, for example, it would be easy to ensure the music volume lowers when you receive a call. A basic function, but one that we expect to work seamlessly with any app we install.

Since then, developers have been served a vast array of tools and services with more and more software arriving on a daily basis. The challenge is that, where the phone and music player teams were controlled in a way that would make them work well together, modern development infrastructure doesn’t have that capability. In most cases, the toolchain is a “patchwork quilt” of products and services from different suppliers. And it is up to developers to build and maintain that quilt.

This means that developers are being forced to take time away from writing code—the reason why they are employed in the first place—to dedicate valuable time and resources toward understanding and managing issues associated with highly complex, multi-cloud toolchains that are often being held together by nothing more than sticky tape and string.

In other words, the very same market that is telling developers they can have anything they want to do their jobs more efficiently is simultaneously telling them that piecing the whole thing together is their problem. This is the gap for developers: It’s a gap between expectations and current reality. And a bridge across that gap needs to be built for organizations to get the most out of DevOps.

Delivering the DevX Factor

Attracting the best people is about showing that the experience they will have in your organization is both rewarding and liberating, so expecting developers to spend the bulk of their time not developing is clearly not an optimal experience. Creating better DevX will not only influence your ability to attract new people but enhance your ability to hold on to, empower and upskill the teams you already have.

Where to start? Well, contrary to popular belief, you don’t need an army of DevOps engineers to reach your goals. Instead, you need to empower your existing teams to reach their full DevOps potential, supporting their progress with the right tools and training for their experience and role.

As with any work culture (remote or hybrid), the importance of knowing where to find what you need and how to collaborate with other teams has never been so important. Tooling is a good place to start. DevX is affected by the tools they use, so organizing them in a way that helps with the culture is a step forward. In a multi-cloud, multi-vendor environment, there isn’t going to be one tool that does it all so the key is to allow teams to use the tools they love and automate as much as possible to remove the repetitive tasks.

Automation is also a critical element in the race to reduce time-to-market and improve quality where concepts such as shift left are adding complexity and increasing the responsibilities piled on developers. Having access to testing automation in a consistent way will improve the overall experience for DevOps teams.

And finally, think about starting your DevX journey by using a self-service platform, where information, tooling and automation can be centralized and made available to developers on demand. For overworked specialists, access to a platform that connects to any tool and creates an on demand environment that automates much of the repetitive work will create more time for them to do what they were originally employed to do: Develop.

December 2, 2021