devopsy.kinsta.cloud

Tag: devops

Automation Challenges Holding DevOps Back

A survey of 500 IT and engineering leaders in the U.S. published today finds organizations build an average of 6.7 internal applications per year themselves in addition to investing in 7.7 separate platforms to build applications.

Commissioned by Internal, a provider of platforms for building application development environments, the survey also found organizations are spending 40% of their time building and maintaining these internal tools and workflows. While more than three-quarters (76%) of respondents said that this work is necessary, it distracts from building applications that drive business revenue. A total of 82% said a lack of true flexibility and customization is the top reason so much time is spent on building and managing internal tools and workflows.

A full 94% of respondents said they want a more collaborative platform that enables technical and non-technical users to build internal tools/workflows faster together. The survey also found 91% of respondents believe a platform that encompasses application development and workflow automation would save development time.

Internal CEO Arisa Amano said one thing that holds IT teams back is that managing the platforms used to build applications requires too much programming expertise. Internal is making a case for a graphical approach to building application development environments so IT administrators can easily construct them using a drag-and-drop interface on behalf of an application development team. That approach makes it possible to allocate more programming resources to building applications, she said.

More organizations than ever are revisiting how applications are built as the number of projects increases despite an uncertain global economy, noted Amano. Organizations of all sizes are reluctant to hire additional full-time engineers even as digital transformation initiatives demand a growing number of custom applications, she added. The only way to effectively address those conflicting goals is to adopt a platform that automates much of the workflow associated with building an application development environment, said Amano.

It’s not clear how willing DevOps teams are to replace the platforms they use today to build those environments, but many organizations are still relying on bespoke platforms they built and continue to maintain. Many of those platforms are made up of a collection of tools that DevOps teams cobbled together over the years. However, many of the software engineers that initially built those integrations may have since moved on and the documentation left behind is likely to be scant, noted Amano.

Most DevOps teams in 2023 will continue investing in higher automation levels to accelerate application development. The quandary they all face is whether to continue to invest in the platforms they have already or migrate to a platform that makes it simpler to build and maintain an application environment using automated workflows built and maintained by a vendor. Regardless of what they choose, it’s certain the days when organizations could afford to throw engineers at every integration challenge have come to an end.

February 1, 2023
Stream Big, Think Bigger: Analyze Streaming Data at Scale

Data streams are all the rage. Once a niche element of data engineering, streaming data is the new normal—more than 80% of Fortune 100 companies have adopted Apache Kafka, the most common streaming platform, and every major cloud provider (AWS, Google Cloud Platform and Microsoft Azure) has launched its own streaming service.

This shift is fueled by the growing demand to deliver events quickly, reliably and at scale to support a wide range of internal and external applications. From always-on, customer-facing applications to mission-critical operations, a growing number of use cases depend on subsecond updates—further upping the ante for real-time data processing and dissemination.

There’s no doubt 2023 will be a big year for streaming technology.

Meanwhile, as streaming moves toward ubiquity, we’re seeing another shift in the way businesses use the data: Analyzing events as they’re created for real-time insights. With the right tools in place, people can instantly compare what’s happening now to what happened in the past, triage problems as they occur and make time-sensitive decisions in the moment.

With the proliferation of data streams, a new set of use cases and requirements for real-time analytics has emerged. To unlock the power of streaming in 2023, data teams will need to look beyond the traditional batch-oriented stack and adopt a streaming-native approach to the analytics architecture.

The Streaming Paradigm Shift

Data has always been at the center of how businesses operate. Until recently, we lived in a very batch-dominant world of business and data. The goal of data infrastructure was to identify data at a fixed point in time and store it for eventual use. The evolution from daily batch operations on mainframes to the internet-driven always-on world replaces fixed, static “data at rest” with fast-moving data in motion, as information flows freely between applications and data systems within and between organizations.

Data at rest still exists, and batch systems continue to serve a variety of reporting purposes. But reality isn’t fixed or static, and to meet the demands for seamless, authentic data experiences, the systems we build must be designed for data in motion.

Hence, the rise of streaming technology—and with it, a new mindset for how we think about data. As streaming increases, streaming data platforms increasingly act as an organization’s central nervous system, connecting the different functions and driving mission-critical operations. Evolving technologies are purpose-built to support data in motion systems: Stream processors and event databases.

Apache Druid fits into the purpose-built category; as a real-time analytics database, it is designed to enable users to query every event as it joins the data stream and do it at a tremendous scale while enabling subsecond queries on a mix of stream and batch data.

Thousands of businesses are already using streaming processors like Kafka and Amazon Kinesis with Druid to build cutting-edge systems that make terabytes to petabytes of streaming data accessible to applications and humans in milliseconds. Some of those businesses—including Reddit, Citrix and Expedia—were highlighted at Current 2022, the annual streaming event organized by Confluent.

The next evolution of data intelligence is here, and it’s based on the ability to react to events as they are happening. But on the growth chart of streaming adoption, we are only at the beginning of an upward curve—a critical inflection point where streaming (and the technology that is built for it) starts to become the bedrock of everyone’s data architecture.

So, when it comes to enabling scalable, subsecond analytics on streaming data, many developers and data innovators are wondering, ‘How?’

We’re Streaming! Now What?

At Current, we spoke with hundreds of Kafka users who were turning their attention to building out the next phase of their data streaming pipelines and embedding them more deeply into their organizations.

While streaming adoption is becoming more widespread, most companies still have just one or two use cases they’re solving with a streaming platform. At Current, many people spoke about how Kafka effectively set their data in motion. But when it came time to analyze or put those streams to work in a user-facing application, their “data in motion” turned into “data in waiting” because their analytics system was designed for batch data, not streaming.

To solve this problem, a new type of database is needed—and that’s why Druid was created. Turning billions or trillions of events into streams that can then be immediately queried by thousands of users simultaneously is where stream processors like Kafka, in combination with Druid, have unlocked a new set of capabilities that are enabling new developer-built analytics applications.

For example, Reddit generates tens of gigabytes of event data per hour from advertisements on its platform. To let advertisers understand their impact and decide how to target their spending, Reddit needs to enable interactive queries across the last six months of data—hundreds of billions of raw events to slice and dice. But they also needed to empower advertisers to see user groups and sizes in real-time, adjusting based on interests (sports, art, movies, technology…) and locations to find how many Reddit users fit their target. To achieve this, they built a Druid-powered application that ingests data from Kafka and enables Reddit’s advertising partners to make real-time decisions and get the best possible ROI on their campaigns.

Stream-to-Batch Vs. Stream-to-Stream

Reddit chose Druid as the database layer of their application in large part because of its close integration with Kafka, as Druid was designed to ingest and analyze streaming data. This sets Druid apart from virtually every other analytics database, the rest of which were built for batch ingestion.

Batch ingestion works exactly the way it sounds—it takes chunks of stream data, dumps them into a file, processes the file, then loads it into the database. Using a batch-based system to analyze streams introduces a bottleneck to your streaming pipeline. By contrast, Druid provides a connector-free integration with the top streaming platforms and handles the latency, consistency and scale requirements for high-performance stream analytics cost-effectively.

In addition to out-of-the-box integrations, Druid has built-in indexing services that provide event-by-event ingestion. This means streaming data is ingested into memory and made immediately available for use. No waiting for events to be processed through batch processes before they can be queried. This capability, coupled with exactly-once semantics, guarantees data is always fresh and consistent. If there’s a failure during stream ingestion, Druid will automatically continue to ingest every event exactly once (including events that enter the stream during the outage) to prevent any duplicates or data loss.

However, perhaps the most powerful reason to use Druid for streaming data analytics is its near-infinite scalability. For even the largest, most complex ingestion jobs, Druid can easily scale into the tens of millions of events per second. With variable ingestion patterns, Druid avoids resource lag (as well as overprovisioning) by enabling dynamic scaling. It combines the query and ingestion performance of shared-nothing architecture with a cloud data warehouse’s flexibility and non-stop reliability. This means you can add computing power and scale out without downtime or rebalancing, which is handled automatically.

In essence, while using a cloud data warehouse to serve both batch-oriented and real-time use cases might sound efficient, doing so defeats the purpose of a streaming pipeline.

“The weakest link in the chain sort of dominates everything,” Databricks CEO Ali Ghodsi told the audience at Current. “If you have one step of your pipeline that is batch-processed and really slow, then it doesn’t matter how fast you are in the rest of the pipeline.”

That’s why when Confluent’s data engineering team needed to build a real-time observability platform, they chose to use Kafka and Druid together. Today, the platform ingests over five million events per second from Kafka and handles hundreds of concurrent queries from a large team of analysts.

“We do a lot of things today that we didn’t imagine doing three or four years ago,” Matt Armstrong, who leads the observability and data platform team at Confluent, said in a Druid Summit 2022 interview. “From a [product] perspective, you could think of Druid as being able to power a great product [Confluent Cloud] with a great support experience—but perhaps more importantly—an unconstrained future.”

It’s Time to Think Big

We’re already moving to this entirely new way of thinking about moving, analyzing, and sharing data. Streaming technology has opened the door for a plethora of new use cases and products to come to fruition. We believe organizations that view streaming data as a force multiplier and implement a streaming-native approach to analytics will gain a competitive advantage in 2023 and beyond.

Druid is purpose-built for these new streaming use cases, but it’s also built for a new type of application—one that draws characteristics from both the analytics and transactional database worlds. Industry-leading teams have built mission-critical applications that enable operational visibility, customer-facing analytics, drill-down exploration and real-time decision-making using streaming data and Druid. This trend is now starting to take hold and the next wave of analytics applications is already being built.

January 27, 2023
Don’t Hire for Product Expertise

We all know the jokes about talent recruitment that “requires a century of experience in AI/ML.” With the number of layoffs, no doubt this type of approach will get worse. But that is not generally a tech problem; it is an HR problem–the problem of having people hire in tech that aren’t active in or knowledgeable about tech.

The real concern that is very much a tech problem is hiring for product and/or technology. I will say outright: Stop doing that except in very special circumstances. Hire for the ability to find information. One of the other things we all joke about is increasingly convoluted interviews where the hiring company wants you to solve their problems in a fraction of the time it would actually take them to do it. Again, if you’re a hiring company doing this, knock it off.

IT is an ever-evolving field, more so once Agile and DevOps are taken into account. Indeed, we would not be far off if we renamed the field Continuous Evolution, because tomorrow is guaranteed to hold different solutions, no matter what. The idea that requiring experience in Grafana or mastery of Python is an indicator of long-term success is downright farcical. There are positions where this is tolerable–if a position is for maintenance of a traditional app that is not going anywhere and is not getting massive updates, then a list of what tools are desired for the role is easy to build and will be lasting. But I would argue that hiring specifically for that application’s needs is not in the best interest of the organization as a whole, because that person is not portable and adaptable. For modern development, particularly of new products that are just now being architected, the needs are going to change in the process, so claiming you want X years of experience in Spring (when it may use a different framework by deployment time) is not productive.

As we move into belt-tightening yet again, employees who can do the job and know how to look up new information will be worth their weight in gold. Having a person who can work on existing applications and drive innovation in future applications all in one body is important. Giving them a chance to exercise that drive to learn is also important, but is a separate issue. None of the areas I cover–applications, operations and security–are standing still or even coming close to standing still. The need for people who can go find out what is needed is huge.

And while the internet is a horrendous mix of good and bad, the ability to find information—and even solutions to problems—is undeniably better than it ever has been. Having people skilled at using that information resource is the key to moving forward.

So, find people that can solve problems given a modern context. Hiring for technology will solve today’s problems, but problems are not restricted to today, so solve ongoing problems. Gate for basic tech knowledge and skills, then evaluate for the ability to find answers. A far better use of everyone’s time during interviews would be—using email as an example—“Hosted Exchange just went down. Walk me through your troubleshooting steps. You have a couple minutes.” Or—using development as an example—“We just discovered that an OSS library we use has unacceptable vulnerabilities. Walk me through how you would find instances of it and what steps you would take to find replacements. Compliance is breathing down our neck, you have a couple of minutes to show us.” You get to see how they approach likely problems, they get to ask about the environment. Far better than “Show us all the steps to get BGP routing configured,” or “Develop an entire B-Tree while we watch,” questions. Certainly better than any question that ends with “And don’t use the internet.” Are you kidding? Can we talk about the field and what year it is?

As I’ve said off and on over the years, you want the best coworkers you can get or the best employees you can get. Either way, hiring for expertise with a given tech product is not the best route to that goal. You rock—so get people with you who do, too. It’s not going to get less complex, so have peeps that can keep up and carry a load, no matter what tomorrow’s technology looks like.

January 25, 2023
Digital Experience and the Future of Observability

Over the past two years, there’s been significant hype around observability in the IT operations and DevOps world. As shorthand for providing better insights, observability has seen a rise in lockstep with distributed architectures, cloud-native applications and the increasingly complex IT environments associated with them.

As we round out 2022 and move into the new year, there’s evidence observability is gaining mainstream adoption by IT operations and has begun to take off, with the market reaching a better understanding of what observability is and why it matters. This broader adoption offers a moment to reflect on the impact of observability so far and how it will continue to evolve in the year ahead.

What is Observability?

Though there’s ongoing debate in the industry about what observability actually means and what an observability platform should include, most people in the industry will argue observability is the evolution of monitoring and promises to offer more complete answers than standalone monitoring. Gartner, for example, first asserted in 2020 observability is “the evolution of monitoring.” But the two aren’t mutually exclusive, and observability and monitoring will continue to coexist and fill distinct needs for different teams based on what information they need about their environments and how they need to act on it.

So far, observability has had the most considerable impact on application performance and digital experience monitoring.

The days of monolithic applications are over. Instead, modern applications are highly distributed, are built natively in the cloud using container technology, are organized around microservices and leverage databases in multiple cloud environments. These increasingly distributed software stacks have introduced enormous flexibility for developers but huge complexity for operations teams, making it harder to deliver an optimal experience to the end user. The DevOps teams tasked with building and managing custom web applications are under pressure, and the result is, as one recent article so concisely put it, “Devs don’t want to do ops.”

Observability has provided a welcome respite for DevOps teams who need intelligent visibility into their environments to determine how applications are performing. This has made application performance a key focus for many observability providers.

An Intolerance of Bad Digital Experience

Ensuring external customer-facing applications are performing optimally is the key driver behind this. Digital businesses with external applications core to the company’s success need to ensure those applications perform optimally—customers are fickle. They’re intolerant of bad digital experiences, and alternatives are plentiful. Any e-commerce vendor knows no website equals no sales.

Traditional application performance monitoring (APM) tools look at telemetry to flag application issues or outages needing to be addressed. But understanding the health of your applications from this narrow standpoint isn’t the same as understanding how the applications perform from a user perspective. In other words, if you aren’t considering how applications are performing (or not!) for customers, employees and other users, you’re not seeing the forest for the trees. Observability helps solve this by elevating APM into a more holistic digital experience approach by adding visibility into how the end user engages with your application.

By providing a view into how an application is performing with the context of what’s happening throughout the entire stack—including networks, databases, systems, and more—and how issues are impacting the user experience, DevOps teams can shed light on previous blind spots.

Elevating the importance of digital experience through observability enables teams to understand and respond to issues more efficiently and provides critical data to help DevOps teams further improve their apps beyond just debugging and issue remediation. By understanding how users engage with and experience the application, you can gather critical data to improve or simplify the digital user experience. In our e-commerce example above, this could mean fewer clicks or an easier checkout experience (better digital experience = more sales).

Cloud migration efforts, coupled with ongoing hybrid and remote work and understaffed tech teams, make prioritizing what matters most even more important. Focusing on the experience of the end user—but with end-to-end visibility across the environment—allows stretched DevOps teams “do more with less.”

2023 Will be the Year of Digital Experience Observability

An observability approach prioritizing digital experience is the first step toward building a complete customer-centric environment. This eliminates the potential risks of missing issues directly impacting application performance and creates an enhanced, complete experience for every user. This will make 2023 the year of digital experience observability.

January 20, 2023
Optimizing the Developer Experience in 2023

2022 was a banner year for innovation, but not only in the traditional “churning out new features” way. Many organizations started thinking about their processes and how to amplify them to maximize efficiency and employee retention. And when they started digging, companies realized that the developer experience was a major pain point that could be rectified–and a ripple effect of positive gains would result.

Companies started looking more closely at how the tools and processes developers use support and enhance their work. Great tools and thoughtful processes boost developer happiness and satisfaction. And when those levels rise, guess what else increased? Productivity, efficiency and employee retention rates.

We saw this trend of developer support take hold last year, but with more work to be done across many companies, we’ll see it continue in 2023 and beyond, impacting just about every facet of tech in new ways. Many companies have realized what does work by trying what doesn’t work and are making big changes from there.

Shifting Results

A major player in the changing developer experience has been the shift left approach that’s taken hold in recent years. About 80% of organizations have said they are adopting this idea of moving more of the responsibility for security and compliance to the front of the pipeline to avoid a compliance scramble at the point of delivery. The result, however, is a compliance process that ends up being more complex and less efficient, putting additional burdens on developers.

Involving more people creates more opportunities for security breaches and slows down the entire software delivery life cycle. It also shifts security responsibilities to developers and requires them to do significantly more than is listed in their job description. In fact, 60% of those shifting left said their development team is bearing the burden of the change. Rather than focusing on writing code, developers have been called on to understand regulatory policies, build regulatory controls into their applications or pipelines, inspect massive log files generated by more and more diagnostic tools for security info and spend more time in meetings to discuss audits. That’s not a recipe for fulfilled developers; it’s the path to a loss of innovation.

Stop Paying the Compliance Tax

When developers–or anyone on your team–are tied up in compliance tasks, they aren’t spending that time in profit-earning efforts. This “compliance tax” costs companies an estimated 15% each year–that’s 15% of their time spent managing compliance instead of innovating or providing services that add value. That’s not to say that compliance isn’t important–it’s essential, in fact–but limiting the burden of compliance improves the overall experience of developers.

When developers aren’t innovating, they’re not happy. They want to be writing great code, not translating regulatory governance policies. In today’s culture of quiet quitting, companies simply can’t afford to turn over compliance tasks to developers. It’s a waste of their talent and a surefire way to lose great developers and the chance to attract new ones.

Shift Left, but Do it Right

The good news is the compliance tax is largely avoidable. Instead of shifting responsibilities left, shift compliance everywhere. No one person or group, including developers, should be in charge of these tasks–that approach just isn’t comprehensive enough to ensure security and compliance throughout the software delivery life cycle.

Instead, security and compliance should be moved away from a point-in-time part of the cycle to a continuous process built into every step of the life cycle. CI/CD reduces the burden on any one person or group and is proven to be more efficient at catching anything that is non-compliant. When the tools are doing the heavy lifting, the people aren’t making as many mistakes and non-compliant code isn’t slipping through.

Baking in compliance allows developers to spend more time doing what they do best: Creating amazing features. This, more than anything, will boost the developer experience. Developers want to create and, when they experience the right environment and the right resources, innovation tends to happen.

Beyond the Developer Experience

Focusing on the developer experience pays off in more ways than just creating happy employees (though that alone is reason enough to make a change). Innovative developers are the backbone of leading tech companies. When they’re churning out new features that are getting public recognition and are touting a culture with a positive developer experience, companies attract more talented developers and the innovation loop continues.

So, if you’re looking for more innovation in 2023, start where you’ve always started: Your developers. Set them up for success by automating what can be automated and baking in the mundane – yet essential – tasks to truly optimize the developer experience. Then sit back and watch what incredible capabilities your developers can generate.

January 19, 2023
Complexity is Still With Us

We have come a long way and had a lot of improvements over the years, but we still struggle with the ability to simplify things faster than we create complexity. It isn’t enough that each new wave of Really Important Bits™ comes with its own issues and layers and complexity; we also haven’t been cleaning up our previous Really Important Bits™ projects. We haven’t cleaned up the successful ones and we haven’t cleaned up the failures.

It is time to start. Honestly, we are looking at an extended stretch of ‘Do More With Less,’ and we need to get our houses in order. Want to do more with less? Start by standardizing. Yes, that one project that used Really Important Bits™ (or its companion product Really Cool Tech™) was neat and all, but keeping a technology and its layers and pieces laying around for a single project is not great management. In fact, it isn’t management at all; it is letting things happen. I’d call it chaos, but then we’d have to have a whole conversation about Dungeons & Dragons and, frankly, it is happening in the forum of the overall enterprise architecture so it’s not actually chaos, it is simply excess technical debt.

DevOps was never meant to be a destination, but on this cross-country road trip, our IT car will get filled with garbage if we don’t take the time to clear out the Really Important Things™ of last year or the last management team or the latest merger. We cannot continue to layer more on without relieving pressure somewhere. AI has the potential to offer some pressure relief in some areas, but we were already behind, so for a variety of reasons, it isn’t the silver bullet we need. Those reasons are simple–it is less secure to leave 50 different ways to do the same thing laying around. AI is resolving some problems and freeing up time, but you’re still wasting time and money on redundant or unused Really Important Things™.

Some things simply cannot be cleaned up without a massive investment. That’s fine—I hope after years of using DevOps methodologies you have them on a list of things that you’d like to upgrade/replace, but it just isn’t feasible to do so. Focus on the things that you can simplify. The famous low-hanging fruits are fine, just like simplifying things in the early days of DevOps meant that each piece of low-hanging fruit freed up more time to focus on other issues, the same is true now that the low-hanging fruit is generated from DevOps. Or DevOops, in some cases.

How did I get here, you ask? I was writing a simple overview of Git, CI/CD, DevOps, DevSecOps, GitOps, etc., etc. for a newer technologist, and realized that in between those important parts are a whole lot of tools and technologies we thought were important, but which are no longer terribly useful or necessary. I did her the favor of not including the by-the-wayside technology spaces and product markets. I removed the complexity, you could say.

And keep rocking it. No matter how far along you are in your IT career, you can look back at all of the things you have learned. Apply that learning to simplifying and improving. Reduce overhead by reducing redundancy and look forward to the next Really Important Thing™ you’ll get to learn about. Keep improving—both your knowledge and the environment. And we’ll keep cheering you on.

January 19, 2023
State of DevOps Report Finds a Rise in Platform Engineering

The speed of software development velocity is increasing, and to enable this, platform engineering is emerging to evolve the DevOps practice. Interest in platform engineering is rising—so much so that Puppet emphasized platform engineering in its 2023 State of DevOps Report.

A whopping 94% of respondents agreed that platform engineering is helping their organizations better realize the benefits of DevOps. The move toward platform engineering has increased over time, bringing benefits such as increased release velocity, reliability and efficiency.

Below, we’ll highlight some of the key takeaways from the report. We’ll define what platform engineering is and highlight its benefits. We’ll also consider obstacles on the path to embracing a platform-based development culture.

Platform Engineering Supplants DevOps

DevOps has been in use for a decade or more at most institutions, maturing steadily with new automation and infrastructure abstraction. The latest evolution of DevOps is a practice known as platform engineering—51% of organizations have already adopted this practice within the last three years and 93% said it’s a step in the right direction.

Platform engineering is the discipline of designing and building self-service capabilities to minimize cognitive load for developers and to enable fast-flow software delivery.

In essence, platform engineering is the process of creating standardized, self-service infrastructure that helps developers get up and running quickly. These self-service platforms typically handle capabilities like deployment (43%), operation of infrastructure (43%), monitoring production applications (42%) and security and compliance (42%).

These shared internal platforms are maintained by a central platform team that embraces a product mindset to iterate the technology based on user feedback.

State of Platform Engineering

Platform engineering is relatively new— 27% of platform teams have formed within the last two to three years. Notably, big tech companies hopped on the bandwagon years ago, and non-tech companies are now following. Regardless, 70% believed their timing was “just right.”

The goals of a platform engineering team are numerous. According to the Puppet report, 54% said problem-solving is the top goal. This is followed by educating and empowering developer teams (47%), setting best practices (46%) and optimizing iteration (44%). A platform engineering team’s scope of work is primarily focused on automating workflows and building and managing infrastructure.

The ‘correct’ number of internal platforms is a moving target. The majority of organizations (57%) have between two and four platforms, and 30% have five or more internal self-service platforms. This demonstrates that although platform engineering is somewhat centralized, it’s also flexible to the needs of an organization.

Benefits and Hurdles

So, what are some benefits as opposed to traditional DevOps? Well, the most common benefit is improved system reliability, at 60%. This tracks, as a standardized platform could reduce human error and missteps in the development workflow. Following this benefit is improving efficiency (59%), speeding up delivery time (58%) and improving workflow and process standards (57%).

The most common driver of adoption is increasing the speed of delivery. Slight more than half of respondents (51%) said development speed has somewhat increased since the inception of the platform team, and 42% say it’s increased by “a great deal.” Other top drivers include the need to scale, free up engineering time, control infrastructure spending and avoid duplication across teams. Those with more mature adoption tend to be more satisfied with the result, indicating that results increased over time.

Of course, no new technological introduction comes without its hurdles, and platform engineering is not immune from certain pain points. For example, 34% said that cycle time is slower than expected. This might be due to nascent industry adoption and a lack of best practices.

Just under one-third (32%) also noticed resistance to platform team adoption, and 32% reported a lack of communication around changes. Organizations will likely have to increase transparency and reduce siloing to encourage productive platform adoption that satisfies the needs of various stakeholders.

According to the report, product management skills can help realize these goals. Some of the most fundamental product management skills here include strong communication skills (61%), problem-solving expertise (60%) and the ability to foster collaboration (54%). Many companies will also have to convince senior management to invest in platform engineering—nearly half agreed that senior management doesn’t yet understand its value.

Future Outlook

Confidence around platform engineering is increasing, especially for teams with mature DevOps approaches. As a result, the practice will likely gain more traction in the coming year—71% of respondents reported their firm plans to hire people with platform engineering experience over the next 12 months.

Over the next year, organizations that adopt the practice will want to increase awareness of platform capabilities and set expectations across the organization. This will also include setting governance policies and satisfying the needs of specific developers, departments and the organization as a whole.

Above, we’ve scratched the surface of the current state of platform engineering, its benefits and its pain points. For plenty of other insights, you can pick up a copy of The State of DevOps Report 2023: Platform Engineering Edition from Puppet by Perforce here.

January 18, 2023
Predict 2023: Stand and Deliver

It’s that time of year again—the New Year’s ritual: Time to predict the big trends in your world for the new year. By now you have probably seen dozens of headlines, articles, videos and interviews telling you what big trends will shape the coming year. Here at Techstrong Group, our Predict virtual event is built around this New Year’s ritual. This year, Predict is January 12, 2023 and, not to blow our own horn, but I think it is our best Predict event yet.

What sets this event apart? Well, we have a real Techstrong Research team now headed up by Mike Rothman and Mitchell Ashley who are leading this year’s Predict event. We will have four tracks: DevOps, cybersecurity, cloud-native and digital transformation. Each track will kick off with either Mitchell or Mike presenting a Techstrong Research trends report for that market.

After the trends report, there will be a keynote panel featuring some of the top thought leaders in each of these fields. This will be followed by other sessions that explore what this upcoming year has in store. Many of these sessions will be streamed live on January 12, 2023 and allow for audience questions and input.

But wait—there’s more! One session I am really looking forward to features John Willis and Damon Edwards live at Techstrong Group Studios giving us their personal predictions for 2023. John and Damon need no introduction to the DevOps community. This should be a great session, so don’t miss it. I am also moderating two great panels; one is with C-level executives from CloudBees, JFrog and GitLab and Kohsuke Kawaguchi (KK), the creator of Jenkins and co-CEO of Launchable. The other panel features my friend Emmet Keefe of Insight Ignite who has his finger on the pulse of the digital transformation market.

The best part? We’ll have a live chat available so you can interact with the speakers and network with your fellow attendees. And you can register and attend Predict 2023 for free. That’s right—this analyst-led, daylong virtual interactive event is free. Don’t wait! Head over right now and register here.

Another highlight will be the announcement of our DevOps Dozen winners at the event. This year, the DevOps Dozen broke all of our previous records for both nominations and community votes. It has been quite a tough job selecting the winners from such incredible submissions and nominations.

I haven’t even mentioned the amazing cybersecurity sessions hosted by Mike Rothman or the sessions covering the cloud-native vision with Mitchell Ashley and digital transformation with Mike Vizard. There is just so much great insight you won’t want to miss.

A big thank you to all of the sponsors, panelists and speakers who help us make this event a true standout. If you want to get a head start on learning about big trends for 2023 and how they’ll impact cloud-native, cybersecurity, DevOps and digital transformation, check out these articles and register for Predict 2023 on January 12, 2023. We hope to see you there!

January 10, 2023
Why GraphQL Will Reign King

After years of development, GraphQL is set to reign as the go-to API for data fetching. But what is GraphQL? Simply put, it’s a query language that allows you to fetch data from any server, and it’s a noteworthy alternative to the likes of other web service development styles like representational state transfer (REST) and simple object access protocol (SOAP). There are a few reasons why GraphQL will dominate the DevOps world, including its overall efficiency, ability to work with any server and overall ease of use.

Understanding GraphQL

GraphQL is a query language for APIs and a runtime for fulfilling queries with existing data. GraphQL provides a complete and digestible description of the data in an API, gives clients the power to ask for exactly what they need, makes it easier to evolve APIs over time and enables powerful developer tools.

Top Benefits of GraphQL

GraphQL addresses many pain points that often slow down developers, who prioritize adaptability and working quickly and efficiently when building applications.

For one, it is much more flexible than other data query languages, allowing users to specify exactly what data they want and in which format. Other data query languages require users to retrieve all data in a specific format, regardless of whether they actually need the data. That excess, unnecessary intake can cause delays and confusion.

GraphQL’s ease of use is also invaluable, thanks to its simple and intuitive syntax. The language is strongly typed, which means developers can catch errors early on in the development process before code goes into production. This can save much-needed time and money in the long run.

Another benefit is that it can be used to fetch data from multiple sources because it uses a schema that defines the different types of data available. This means if a developer has a field in a schema that could potentially return data from multiple sources, they can specify which source to fetch the data from. Developers don’t get this benefit from other data query languages, which often allow data fetching from only a single source.

Advantages for DevOps Teams Over Other Data Query Languages

As the world of software development continues maturing, so do the tools and technologies DevOps teams use to build and operate their applications.

REST and SOAP are popular web service technologies, but they come with some major drawbacks. Notably, both REST and SOAP rely heavily on URL structures and verbose XML payloads, which makes it difficult to evolve the API without breaking existing clients. And because each endpoint is a unique URL, providing a consistent experience across different parts of the API can be challenging. REST and SOAP are also quite chatty and can make it hard to optimize performance.

In contrast to REST, GraphQL provides a single endpoint for all data requests and is much faster than making many HTTP requests. GraphQL’s declarative query language specifies precisely the data needed – no more, no less – increasing bandwidth and improving performance. Applications can run faster and use fewer resources, which is especially important for mobile apps that need to be lightweight and fast.

Understanding the data is crucial for DevOps teams, and GraphQL allows developers to visualize data in a graph. Teams can see the relationships between different points more easily, leading to better-informed decisions about the best ways to manipulate the data.

The increased efficiency when conducting API calls is also a plus with GraphQL, which helps teams retrieve all information they need with a single API call. Traditional REST APIs often require multiple API calls, mainly because each resource in a REST API is represented by its own unique URL, along with its own set of permissions and data that needs to be retrieved. GraphQL can eliminate this bottleneck.

Overall, dealing with complexity is significantly reduced when developers use GraphQL. When using traditional REST APIs, writing a lot of code to handle different edge cases and error conditions is the norm. But GraphQL streamlines this – much of the complexity can be handled by the server itself, making way for simpler and more maintainable codebases. DevOps teams are also able to decouple their front-end and back-end development efforts by using a schema-based approach. GraphQL enables frontend and backend components to be developed separately without worrying about breaking changes, which means faster development cycles and fewer errors.

The amount of data that organizations need to understand, organize and query will only increase in the coming years. DevOps teams can get ahead of the ongoing data and information onslaught by taking advantage of GraphQL and its unmatched ability to provide an efficient, powerful and flexible API for data fetching and manipulation.

January 10, 2023
How DevOps Helps Banks Optimize Their Services

A world-renowned bank’s CEO was recently quoted as saying that while there are thousands of manual banks now, the future will see only a handful of digital banks. Along with the rest of the world’s critical business sectors, an overwhelming number of banks are also shifting to virtual platforms. This migration is being orchestrated by some of the best DevOps resources that the global FinTech sector has to offer.

The convenience of personalized services that are always on-demand has incentivized the banking sector’s reliance on DevOps workflows. When the COVID-19 pandemic struck and limited physical banking services, the migration of enterprise banking systems onto the cloud had been in full swing. Most bank employees had to shift to a work-from-home setup, accelerating the use of virtual service delivery.

As per an estimate by the IBM Institute for Business Value, about 59% of the companies surveyed are in various phases of their digital migration and the banking sector is naturally following suit. DevOps teams are tasked with writing software extensions for connecting digital banking services to the end customer in an agile manner.

This article explores why DevOps is slowly becoming an essential foundation for any successful banking institution. We will also discuss how traditional institutions cope with these changes and what benefits they can look forward to once they make the leap to DevOps.

Agile Processes Facilitate Digital Transformation

Most banks these days have been adopting agile processes to stay ahead of the competitive curve. The main motive behind this is to keep up with frequent business requirements that come with efforts to improve clients’ banking experiences. Fine-tuning the project planning and execution processes that drive this improvement are what the DevOps teams would be mainly doing.

Under the Agile methodology, banking applications would be created in small, controllable steps known as sprints. Each step is tested meticulously before it can be considered executable or complete. Before continuing to build on the delivered component, proper channels are followed for real-time feedback and course correction. The end goal is that the customer must get a product that is of value in a much shorter timeframe than is possible through any other development process.

More than even the process and the rules governing it, the DevOps teams involved and their collaboration is a top priority as well as the eventual outcomes. Agile projects behind the most optimal banking services and products must involve continuous planning, testing and integration of finished components.

Faster Success for DevOps Teams in Banking

To successfully drive new digital banking services, large banks need billions in assets and need to gain millions of new customers along the way. It all boils down to shifting their processes to a more granular process and continuously improving upon it.

In situations like these where DevOps practices need to be adopted proactively, the client’s line of business (LOB) teams move to Agile planning strategies fairly quickly. They use a variety of tools to implement Agile and DevOps practices to tackle growing customer demands for their improved digital banking services.

But for this to work seamlessly, the developer experience for both the LOB and IT teams needs to be streamlined. The LOB and IT teams’ efforts can be integrated if the DevOps platform is automated with built-in Agile capabilities and bi-directional integration. This ensures seamless collaboration, faster DevOps implementation and the speedy delivery of banking services and products.

Focused Insights and Better Predictability

Large banks can standardize the internal IT development team on an engineering life cycle management platform, simultaneously providing support to the LOB team via the Agile issue tracking dashboard. Agile life cycle management tools extend an integrated solution for banks. Using these tools, the DevOps teams can be provided with full transparency and traceability of developer data.

Developer-specific capabilities such as source code management, build automation, continuous integration and delivery and Agile planning are more accessible this way. The IT developer community considers Agile workflow management platforms as the benchmark for efficient and continuous business flow. Workflow management platforms and issue tracking portals are the places where most of the Agile planning takes place.

Agile life cycle integration adaptors then link the LOB-defined Agile stories with specific developer tasks in easy-to-manage packets. The developer changes to the application code are tracked within the workforce management tool to draw useful and quantifiable insights from the data. The speed of application delivery is accelerated further because of the connections established between the projects, teams and tools.

Minimizing Service Delivery Mishaps

Using workflow management tools, the team can provide continuous build and integration functionalities. They can deploy automated integration builds on dedicated cloud or on-premises environments which minimizes errors in service delivery. A software bill of materials (SBOM) is generated for each build in the final audit trail which lets developers resolve errors retroactively as well.

A broad picture is created of which new tasks and pieces of code went into the build and what software assets were created. DevOps implementation ensures continuous delivery of the banking application across test and production environments. Errors are reduced throughout the production life cycles of each banking service or product. This reduces the time taken from development to production. A fully automated and auditable delivery pipeline is created to make future deployments even faster.

Reduce Banking Product Release Cycles

DevOps provides a very robust and reiterable foundational environment for the bank’s business to thrive while reducing release cycle time with every iteration. This means every time a new banking service is delivered, the development life cycle is improved bit by bit. In some cases, the release cycle can be reduced by more than half the original time taken.

While service delivery is improved, this also frees up release engineers for urgent tasks crucial to the deployment window. More deployments occur while still keeping the bank or financial institution in accordance with regulatory guidelines.

January 6, 2023