devopsy.kinsta.cloud

Category: Continuous Delivery

Continuous delivery is a software engineering discipline that allows organizations to consistently develop and release quality software at a rapid pace. It is achieved by automating the software development lifecycle, enabling tasks such as development, testing and release to occur concurrently and continuously.

  • The Role of APIs In Digital Transformation

    The Role of APIs In Digital Transformation

    Digital transformation is increasingly propelled by rapidly changing user expectations. Release frequency is increasing, as is the need to connect a growing number of applications and digital experiences. As a result, 83% of IT specialists consider API integration critical to their business, found the State of API Integration Report.

    Application programming interfaces (APIs) hold powerful sway in tech discussions of late. Building API-first could increase internal reusability. APIs have become a standardized machine-readable mechanism for connecting partners. There is also a hot market for API startups, meaning APIs are becoming products in their own right.

    APIs are also enabling change for large enterprises. I recently connected with Bernd Gross, CTO at Software AG, to get his perspective on the role APIs and integrations are playing within the new wave of enterprise digital transformation. Below, we’ll see how connected customer experiences, digital business excellence and a new ecosystem-driven economy are driving the emergence of more API-based strategies.

    Three Drivers of the Connected Enterprise

    Businesses continue to transform into more connected enterprises. Once relegated solely to improving back-office efficiency, digitalization efforts are being applied across the board — to employee management, assets, machines, partners and the supply chain at large, said Gross.

    Amid rising digitalization, connectivity is the glue that unites disparate elements and tools. A fully connected world is “the new flavor of digitalization,” said Gross. To his point, ‘connected experiences’ is a crucial driver for 68% of platform providers, The State of API Integration Report found.

    So, how are APIs fueling the new connected enterprise? Gross identified three key reasons why integration is now fundamental to enterprise digitalization initiatives.

    Software AG Marketecture, used with permission.

    1. Connected Customer Experience

    Many organizations “want to transform into a technology and software company,” said Gross. To participate in the connected digital economy, organizations are striving to create advanced 360-degree customer experiences. Today’s technology needs to “look beyond a PC application,” Gross said. These experiences must cater not only to desktops and smartphones but also to IoT devices and physical assets.

    For example, he explains how one large elevator company architected an embedded connected experience, incorporating digital touchpoints within elevators to showcase signage, entertainment and ads, opening new monetization possibilities. This change was so radical that the mechanical engineering company pivoted its mission statement, reimagining itself as a “vertical transportation company.” Soon, their AI-enabled elevators will predict behaviors and optimize travel times, so they are always on the right floor, waiting for human riders.

    Intelligent building technology is similarly being applied by digital-savvy hotels, Gross explained, which is especially important for contactless visits during the pandemic. A fully digitalized customer experience could involve automated check-in, a room access code sent to a smartphone, an online billing system, physical sensors and more. Enabling a seamless digital hotel experience sounds easy, but it’s technically quite challenging — there are endless integration requirements to connect. In some cases, it would take 24 integration points, noted Gross.

    2. Digital Business Excellence

    Gross recognized the high expectations for modern digital experiences. He described a “new trend of business excellence in delivering more of an end-to-end digital experience.” This means data can no longer be locked up; even traditional companies must transition to platforms to offer software and data to their end users. In the process, interesting things may occur — your customers could become partners, Gross adds.

    He described how one medical company, a leading centrifuge producer, sought to build more digital experiences to complement their hardware. Though the machine functioned seamlessly, there was no easy way to port data results to other applications. This inhibited initiating the fully digitalized workflow required for a smart lab experience.

    In this case, APIs were imperative to opening up back end cloud services to enable smooth user interfaces and end-to-end workflow automation. “Intuitive user interfaces must integrate all services, including cloud and on-premises services, to create and automate workflows,” Gross explained. “To do this, you need integration.”

    3. Ecosystem-Driven Economy

    The network effect of platforms is that they enable an ecosystem to form. And, according to Gross, involving partners in this new digital experience matrix is imperative for success. For example, the API gateway is one area where partnerships help an integration strategy; “You need an API gateway for security and governance reasons,” he stated.

    Throughout his experience overseeing APIs on many projects, Gross often noticed a naïveté associated with defining the ecosystem. “When [clients] engage in the first phase of these projects, they think they know how to do it, but they typically don’t,” he said. Potential API providers often don’t fully realize their monetization strategy, the security implications or the complementary software providers necessary to support a working model. “Partnerships often facilitate a value-add to customers — they enrich the total offering through APIs and ecosystems,” said Gross.

    Similar to how a machinery company may embed a bill of materials into the product they sell, digital partnership ecosystems can open up data so partners can access and innovate with it, Gross said. Frequently, ecosystem partnerships make a lot of sense, as they can better meet consumer expectations. For example, Gross suggested that integrating a ticketing system into a conditional monitoring system would bring compelling value.

    The Next Stage: Self-Service Integration With APIs

    According to the report, 40% of companies said that digital transformation is driving their need for app integration. Truly connected enterprises must embrace digitalization and integration across all aspects of the business. As a result, this is forcing a business shift toward more open ecosystems.

    But building these systems will require newfound knowledge dissemination around developer experience and self-service capabilities. “Self-service has become fundamentally important,” said Gross. It’s not enough to simply open interfaces — you have to also add self-service capabilities to integration tools, he added. This could mean offering low-code options alongside APIs — such drag-and-drop UIs would help facilitate integrations for more citizen developers. A self-service approach to digital platformification is an emerging trend, yet one that’s “not fully at mass market,” described Gross; it’s still in its early stages.

    Open ecosystems are guided by consumers, who often desire connections between your tool and external partners. But, opening up data isn’t just helpful for partner integration; it could also benefit internal use. As Gross described, time-series industrial analytics could empower operators on the shop floor or production facility to make sense of local data. “You need self-service analytics tools to give them that ability,” Gross said.

    Building new customer experiences to cater to various devices and platforms is a big driver for API adoption. As a result, headless styles are emerging to address multi-platform requirements. For these reasons and many more, APIs are fueling new digital transformations across enterprises, powering innovative customer experiences and hiding complexity in the back end. With APIs, smart processes, smart machines and smart partners can all come together.

  • Appian Extends Low-Code Approach to Aggregating Data

    Appian Extends Low-Code Approach to Aggregating Data

    At an online Appian World conference, Appian today unveiled an update to its low-code platform that adds a set of visual tools that enables developers to aggregate data within an application with the help of a database administrator (DBA) or data engineering team.

    Malcolm Ross, vice president of product at Appian, said one of the things that holds up application development the most is the need to wait for data to be massaged in a way that makes sense within the context of an application. The latest edition of the Appian Low-code Automation Platform aggregates multiple data sources into a single data vector columnar data store that resides within the platform.

    In effect, Appian is applying a low-code approach to data engineering processes that, today, require dedicated expertise to set up and manage.

    Appian

    The latest version of Appian also adds an Intelligent Document Processing (IDP) tool enabled by optical character recognitions and AI models, application design guidance, developer collaboration tools and enhanced DevSecOps capabilities that enable software packages to move between development, test and production environments with a single click in a way that is governed by the Appian platform.

    Appian is also extending robotic processing automation (RPA) tools to centrally manage bots within an application to Windows platforms.

    Finally, Appian is making a formal, less-robust community edition of its platform available to a limited number of developers as part of an effort to expand the overall size of its developer community.

    Ross said as both professional and so-called citizen developers embrace low-code applications it’s critical to have guardrails in place that ensure applications are deployed in a way that adheres to DevSecOps best practices. The Appian Low-code Automation Platform includes the software development life cycle (SDLC) tools needed to set those guardrails regardless of the level of experience any developer has, noted Ross. In fact, Appian guarantees, for a flat services fee, that an organization’s first project will go live in just eight weeks.

    In general, Ross said the number of individuals capable of building applications continues to expand as more “business technologists” learn how to build applications. Previously, these individuals might have used 4GL tools or macros to build applications based on Microsoft Excel spreadsheets. The Appian low-code platform makes it possible for this class of power end users to build more robust enterprise applications using a set of tools that compare their efforts to known best practices and offer suggestions to make improvements.

    At the same time, professional developers are opting to build applications using low-code tools to reduce the size of application development backlogs. Not every application that needs to be developed requires developers to employ procedural code.

    The net effect is more applications than ever are being simultaneously built and delivered at a time when organizations are launching multiple digital business transformation initiatives.

    It may be a while before the effects of low-code platforms are fully realized within IT environments. However, the one thing that is certain is IT teams of all sizes should expect to be managing an exponentially higher number of applications.

  • CircleCI Acquires Vamp to Apply AI to CD

    CircleCI Acquires Vamp to Apply AI to CD

    CircleCI today announced it has acquired Vamp, a provider of an orchestration platform for automating application releases. CircleCI at the same time revealed it has garnered an additional $100 million in financing to attain a $1.7 billion valuation.

    Jim Rose, CircleCI CEO, said Vamp will play a significant role in advancing the capabilities of the CircleCI continuous integration/continuous delivery (CI/CD) platform at a time when the rate at which applications are being deployed continues to steadily increase. In fact, CircleCI claims its platform is now being employed to process 2.5 million jobs processed per day.

    The Vamp platform makes it easier to construct and orchestrate a multi-stage release cycle involving, for example, canary releases of an application. It then employs machine learning algorithms to validate the impact, performance and quality of releases based on the service level objectives defined by an IT team. If anything is determined to be amiss, the platform will then automatically roll back a release.

    CircleCI

    While many organizations have successfully implemented CI best practices, the ability to continuously deploy application releases has proven to be a more elusive goal. Each platform on which an application is deployed has unique attributes. The addition of machine learning algorithms that can provide continuous validation of deployments should help drive a sea change in the way applications are being developed and deployed to improve the overall state of CD, noted Rose.

    Those machine learning algorithms, along with other forms of artificial intelligence (AI) won’t replace the need for DevOps teams as much augment them at a time when IT environments, thanks in part to the rise of microservices, are becoming more complex. In fact, with the arrival of containers, Kubernetes and serverless computing frameworks, it’s becoming next to impossible to manage an IT environment without the use of AI.

    Rose added that, as the line between pre-production and production environments continues to blur, it’s critical for IT teams to more closely align CI and CD workflows. Rather than isolating CD from CI within the context of a GitOps workflow, the goal should be to integrate those processes more closely. It is becoming more apparent that the elements of what constitutes an application are more empirical thanks to, for example, configuration files that are now stored in a Git repository along with application code. That tighter integration ultimately makes it easier to test and validate applications before they are deployed. Metadata that can be analyzed to optimize application development and deployment can also be more easily captured as part of a continuous feedback loop.

    That approach is more critical than ever, because the days of the “Big Bang” deployment of an application update are over. In its place are now a steady stream of updates that deliver new capabilities, which need to be continuously vetted, noted Rose.

    Each successive era of DevOps is, naturally, going to be more automated than the previous one. The challenge is recognizing when those new eras arrive before rivals gain an advantage in speed and agility that becomes too big to overcome.

  • Linux Foundation Embraces Lightweight API Gateway

    Linux Foundation Embraces Lightweight API Gateway

    The Linux Foundation announced today that an open source application programming interface (API) gateway known as KrakenD has become the latest project it will oversee.

    Rechristened the Lura Project, the lightweight stateless API gateway was originally developed by KrakenD, which plans to continue to offer an enterprise edition of the platform.

    KrakenD CEO Albert Lombarte said the API gateway it developed provides a more streamlined approach to managing APIs that doesn’t require an organization to install a separate proxy or database. All the reverse proxy capabilities most organizations require is already embedded within the API gateway. That approach also makes it easier for IT teams to decouple back-end server applications from client applications because now an API can be more easily updated without impacting client applications, Lombarte added.

    KrakenD Linux Foundation

    Organizations that already contribute to the ongoing development of the API gateway include 99P, Hepsiburada, Openroom, Postman and Stayforlong. The API gateway also is embedded in a variety of other platforms, said Lombarte; he noted that the number of servers running the gateway now exceeds more than one million per month.

    API gateways have been available in various forms for years, but with the rise of microservices the number of APIs organizations need to deploy and manage continues to rise steadily. Each microservice added to an environment adds yet another API. At a time when microservices-based applications are at the core of many digital business transformation initiatives, IT teams must employ a more flexible approach to managing the interactions between APIs.

    Thanks in part to the role they play in enabling digital business transformations, APIs are also being targeted more frequently by cybercriminals that have come to view them as an efficient way to exfiltrate data or inject malware into an application environment.

    It’s not clear what impact the Linux Foundation’s support might have on organizations’ selection of one API gateway versus another. In fact, many organizations are still trying to navigate when to employ an API gateway versus relying on proxy software to manage a handful of APIs versus a service mesh to manage thousands. The issue comes down to not just how many APIs there are, but also the overall weight of the platform being employed, said Lombarte. A service mesh typically requires more time and effort to deploy and manage, he noted.

    As the number of APIs in an IT environment continues to increase, there may come a day when organizations will have to revisit how APIs are created and maintained. All too often, APIs are still created as an afterthought once an application is created. They tend to be brittle and, as a consequence, still break often. A more holistic approach to managing APIs based on a set of DevOps best practices is required. The challenge is getting DevOps teams to prioritize APIs when there are so many other issues that require their attention. Nevertheless, it’s become apparent APIs are, in many cases, becoming too strategic to the business for their management to be left in the hands of a single development team.

  • ServiceNow Acquires Lightstep to Gain Observability Platform

    ServiceNow Acquires Lightstep to Gain Observability Platform

    ServiceNow today announced it has acquired Lightstep as part of an effort to add an observability platform to its IT operations portfolio. Terms of the deal, expected to close this quarter, were not disclosed.

    The Change Intelligence observability platform from Lightstep is based on a time-series database that is capable of processing a trillion events each day. It was built by the same team that created Monarch, the globally distributed in-memory time series database system that Google employs to monitor its applications and systems. Lightstep team members also co-created both the OpenTracing and OpenTelemetry open source projects for collecting traces, metrics and logs.

    Pablo Stern, senior vice president and general manager for IT workflow products at ServiceNow, said ServiceNow will employ Lightstep to enable IT organizations to observe both monolithic and microservices-based applications as IT operations and DevOps teams continue to collaborate more closely with one another. The goal is to provide a single pane of glass through which IT teams can observe their entire IT environment via the Now platform that ServiceNow makes available via a software-as-service (SaaS) delivery model.

    That approach will enable DevOps teams to programmatically manage IT environments alongside IT administrators employing graphical tools, noted Stern.

    Lightstep CEO Ben Sigelman added that, in time, the number of users of that observability will only increase as more application stakeholders start to track software development projects that are at the heart of various digital business transformation initiatives.

    ServiceNow

    Thanks mainly to the rise of microservices that are now being deployed across highly distributed computing environments, it’s become more difficult to discover the root cause of an IT issue. Microservices-based applications have lots of dependencies that, while serving to make the overall application more resilient, conspire to make it difficult to determine which specific microservices might be adversely impacting the performance of an application.

    The time-series database that lies underneath the Change Intelligence observability platform makes it feasible to go beyond logs and metrics to process distributed traces that provide more insight into application behavior. Ultimately, the goal is to enable organizations to innovate faster with a higher degree of confidence in the IT environment, said Sigelman.

    As far as the march to observability in the enterprise is concerned, it’s still early days. While observability may be a core tenet of any DevOps practice, most IT organizations rely on myriad tools that surface data the teams then attempt to correlate. Whenever there is a major issue, those IT teams typically convene in a “war room” to compare reports generated by the various IT monitoring tools employed. Observability platforms that collect and process distributed traces used to instrument an application using a consistent set of specifications, frameworks and libraries. The OpenTracing and OpenTelemetry tools are significant because they provide an open source approach to instrumenting applications that reduces the total cost of observability. In comparison, many legacy application performance management (APM) platforms employ proprietary agents to instrument applications.

    Naturally, each IT organizations will be moving down the path toward an observability platform that provides greater application context. At this point, however, it’s not clear to what degree observability platforms will need to be installed before making the transition to microservices versus being a capability that only gets appreciated once the trouble with microservices finally becomes apparent.

  • Empowering Developers Through Lean Code

    Empowering Developers Through Lean Code

    Digital transformation is widely understood as a critical component for increasing user adoption, fueling workplace productivity and innovation, and driving business growth. Software has become increasingly important across countless industries, teams, and functionalities, and as a result, companies are beginning to understand that being an industry leader and gaining competitive advantages starts with software development.

    Easier said than done. Modern software apps are a complex orchestration of infrastructure, deployment pipelines, notifications, user experience, data filtering and processing, and error handling. A good engineering team will be able to account for each of these items and integrate them into a larger, comprehensive infrastructure.

    So what happens when you don’t have a solid, in-house team guiding your software maintenance and development? Consider a lean code development platform; a developer service that pares down the explicit requirements of standard functions, interactions, errors and software infrastructure into implicit scripts, templates, configurations and API requests. Using a lean code solution will give your team the power to build higher-quality features more quickly than they otherwise would on their own.

    There are several ways that lean code platforms can simplify processes for developers. Primarily, these platforms turn explicit requirements into implicit interactions, eliminating the need for developers to pull elaborate infrastructure to solve complex problems. Lean code also gives developers the freedom to customize everything to fit where it is needed, while limiting the amount of code necessary to do so. Lastly, lean code platforms provide tools that assist developers at all levels of the software stack, significantly increasing productivity while cutting down on time and workload.

    How Does it Differ from Low-Code?

    While many people might be unfamiliar with the term “low-code,” they’d probably be surprised to learn that they use it all the time. Low-code describes platforms that require minimal code in order to build applications and processes.

    Essentially, these programs enable users with little to no coding or software development experience to create and use applications. The idea of low-code is to introduce audiences to intermediate level software development concepts such as object-oriented programming, interface design, conditional logic and code syntax.

    However, no matter how helpful these low-code approaches might be, many users still run into the problem of wanting more customization than the low-code system is designed to give. In fact, many users become frustrated with the term “customization” because it’s viewed as the enemy, indicating a need for more advanced software development services than the consumer is able to provide.

    Lean code platforms, on the other hand, are designed to specifically meet developers’ needs at all stages of the software development life cycle. In essence, lean-code prioritizes simplicity, efficiency, flexibility and accessibility with the goal of making software development accessible on a larger scale.

    Instead of encouraging you to reduce your development team, lean code platforms allow users to expand development through exploring new avenues and abstracting away the explicit requirements of traditional software development.

    What Does Lean Code Have to do With Productivity Infrastructure?

    Many forward-thinking companies have gone beyond simply building technical solutions to specific development problems. Instead, they’ve taken the advancement of lean code platforms to a new level by providing fixes that meet developers at the central stages of the integration process.

    Because lean code offers such integrated solutions to developer challenges, the approach is increasingly becoming the norm across the bulk of the productivity infrastructure category — an emerging market category that combines elements of cloud communications services, API platforms, low-code development tools, business process automation and AI software development kits.

    If you’re looking to implement a lean-code platform, consider the following features:

    • Intelligence: By making it easy to pull information from unstructured data sources and flip it into more meaningful material, machine learning (ML) replaces the manual effort to build, implement and maintain models.
    • Automation: Workflows, triggers, templates and notifications can be abstracted away for common use cases.
    • Security: When handling user data, security is a top priority. From protecting health records to pet names, data security is crucial.
    • Connectivity: The ability to connect to a relevant data source can make or break an infrastructure.
    • User experience: User interfaces should follow expected conventions for whichever task the user is trying to accomplish. Maintaining a successful user experience requires constant evaluation.

    How will Lean Code Become Market-Leading Software?

    Lean code development platforms have emerged as market-leading software development solutions in three major areas:

    • Productivity Infrastructure: More companies have pivoted to providing full-stack development services that assist developers in building out native communications, search and indexing, scheduling and financial experiences.
    • Software development and deployment workflows: Continuous integration/ continuous deployment (CI/CD) pair down complex build and deployment pipelines to basic configuration files.
    • Infrastructure deployment: Infrastructure as a service (IaaS) products assist in deploying scalable web services via API requests and simple configurations.

    Where are lean code development platforms headed?

    It’s safe to assume that, given their ability to provide efficient and effective infrastructure to development teams, lean code has a bright future ahead. The number of developers in the space is only growing. A Slash Data survey from 2019 estimated there to be just under 19 million active developers across the globe and expected the number to grow to 45 million by 2030.

    As the market for software development rapidly expands, it would make sense to assume that the need for a platform that solves problems across the entire software ecosystem would, too. Lean code development platforms are emerging as a top choice for companies that want to build out a leading software platform into the future.

  • Lessons in User Engagement From Game Development

    Lessons in User Engagement From Game Development

    Quarantine measures forced most of the world to stay physically distant — a shift that has forever changed consumer behaviors, and especially how humans interact and spend their free time. A clear change over the past year was the rapid boom of gaming, which saw a massive increase in both console and mobile games — estimated to be as much as 50%. With this sudden increase in new players, how can developers keep gaming experience fresh and exciting? What can game developers do to keep players coming back after they hit ‘Close’? And are there lessons developers in other industries can learn from those game developers to make their applications more engaging?

    While this article is geared toward game developers specifically, there are multiple lessons that could apply to any application with regard to user experience and engagement. It’s especially applicable if you have a gamified corporate application or continuing education app for your enterprise users.

    Much of the answer to the engagement question comes down to users’ journey before, during and after they play a game. Effectively communicating why someone should download a mobile game is the first step, but communicating with players during and after their experience is equally important for long-term engagement and overall experience.

    A vast majority of mobile games still don’t implement basic messaging best practices for improving user engagement and retention. At the same time, COVID-19 has raised the bar for digital experiences. More than ever, game developers need to improve their communication strategies to survive and succeed.

    One difference between established gamers and the large number of new gamers from the pandemic is that they are increasingly diverse. This wave of new gamers are also more likely to play on their mobile device, or even across multiple devices like their smartphone and tablet, which will likely require a friendlier, more streamlined experience than their more experienced peers.

    Hence, creating the right communication strategy, whether through in-game events, updates on exciting new content, or relevant promotions, will all go a long way in keeping gamers coming back after they hit close.

    User Onboarding is Crucial

    There’s been over a 700% increase in game titles available on the Apple and Google app stores in the past decade, with nearly a million titles to choose from. With so much more competition, developers struggle to find and keep players; the top 25% of mobile games have an average 28 day retention rate of just 4%.

    Mobile games have minutes (or even seconds) to convey value and engage users. How a game onboards a player can mean the difference between a loyal customer and a missed opportunity. If you drop a player into a game with no support or context, they might fumble through the rules and features, and could get frustrated enough to abandon the game.

    Here are some ways to onboard players more effectively:

    Function-Oriented Onboarding – This can take the form of a deck-of-cards tutorial immediately after a user opens your game. Through full-screen in-app messages, it educates the player on game mechanics, app navigation and enables players to swipe through the information at their own pace.

    Benefit-Oriented Onboarding – This method reveals features that improve the player’s experience in addition to the game’s basic mechanics. For example, highlighting how a user can resume from a checkpoint instead of starting over.

    Progressive Onboarding – This method helps players interact with the game interface and guides them through different actions when they reach milestones in the game. It’s well-suited to games that have more complex or varied mechanics.

    Enhancing the Post-Game Messaging Experience

    Re-engaging users outside of the game through push notifications, email messages or even SMS can be incredibly powerful. At the same time, if your message’s quality does not match the quality of your in-game experience, users are much more likely to block your messages or uninstall your game entirely.

    The differences come down to personalization, timeliness and progression.

    Players want to get recognition for their achievements and milestones, plus get instantly notified about making their next move, attack or strategy. Generic messages like, “You haven’t played in a while. Come back today!” are unlikely to have much of an impact on user retention. Instead, enhancing the in-game experience with personalized messages that provide incentives tend to perform far better: “Congrats, Matilda! Your hero is nearly at level 10 – get double XP for every level you complete today!” is a much more engaging option.

    In Zynga’s popular “Words with Friends” game, the granular personalization of notifications tells users when a friend made a move, which word they played and its point value. This helps increase messaging utility and encourage reengagement by promoting friendly competition.

    Another way to enhance the experience both within and outside of the game is by leveraging achievements and recognition. These can turn a mundane milestone into a moment of celebration, or convince a player to return to the game to get another reward. Displaying the daily high score of top players can also motivate players. With set goals, players strive to achieve a higher score, push harder to win a level, and play the game more.

    Developers who only focus on building great game mechanics often overlook basic (and critical) communication features to engage and retain users. By employing these best practices for engagement, game developers can build a solid, growing community of happy players that keep coming back for more.

  • Using RPA for DevOps Process Automation

    Using RPA for DevOps Process Automation

    Modern operations teams today are at the cutting edge of automation. DevOps and SRE practices are entirely based on an automation-first philosophy. CI/CD is fully automated and the next generation of automated testing is now being adopted.

    However, these same teams are still running their human processes based on hastily thrown-together wiki pages. The increase in remote work over the last year has amplified the problem. Engineering teams that were used to quickly getting the right people physically into a war room now struggle to find the people with the right knowledge to troubleshoot or resolve a problem.

    Teams with ample resources have built tools and bots for DevOps process automation and orchestration. These tools can help automate things like getting communications channels set up quickly, with the right people, the moment an incident is identified. Or, they may automate actions across the DevOps toolchain that would otherwise live as static documentation for an on-call engineer to try to decipher and execute. But only a handful of organizations have the resources to build this kind of automation technology in-house.

    These higher-level DevOps processes are ripe for innovation similar to robotic process automation (RPA). Companies like UiPath and Automation Anywhere have shown that there’s an enormous market for this in every business unit and industry. Process automation for DevOps needs to be another layer of automation available to any engineering team. Automated remediation of incidents and resolution of service desk tickets has to be the future of this space. However, there are important differences in how automation is optimized for DevOps versus the application of RPA for business functions.

    Where RPA Fits on the Automation Spectrum

    RPA is great for solving deterministic problems – those that can be resolved in the same way every time you run into them – which are fully automatable. An example of this type of end-to-end automation is any back-office task, which are consistently good candidates for the application of automation. It’s the reason why fields like HR and finance have made some of the greatest progress in automating business processes to date. Every time you close the books, for example, the steps are pretty similar.

    Process automation for DevOps and SRE, however, is on the other end of the spectrum. Although teams may benefit from RPA for simplistic, repeatable tasks, many DevOps processes are dynamic and unpredictable, making full end-to-end automation too often fallible. The complexity of modern technology stacks and infrastructure requires processes to be agile and adaptable to change, rather than deterministic. This is where it’s important to enable humans to step into the spectrum of automation. Human-in-the-loop automation – the idea that lets humans be in the cockpit while letting machines provide guidance and support – is the future of DevOps automation.

    Applying Process Automation to DevOps

    To use automation in a meaningful way, DevOps teams need a mix of what RPA provides with the added element of human judgment. Human-in-the-loop automation can be applied to numerous DevOps processes including incident management, service requests and continuous deployment and delivery.

    Rather than automating from end-to-end right from the start, DevOps teams should use incremental automation to strengthen and build trust in automation. This starts by codifying free-form processes so teams can more easily identify predictable steps that can be automated. Then teams can mature to more complex workflows, automating pieces within processes and incorporating human checkpoints for areas that will need judgment or approvals.

    As teams incrementally automate, they build up sustainability – in their systems and their roles – in high-stress environments and are able to orchestrate processes across the continuum. The goal is to find the right balance between leveraging machines to make lives easier while creating room for human judgment and adaptability.

    Level-Up DevOps Automation

    Looking ahead to the next few years, what the RPA movement did for back office tasks will also happen for DevOps and SRE. Technology leaders can take notes from the RPA movement and apply them to eliminating the toil of modern production operations. The issue is that, in DevOps, many problems are unique.

    The next evolution of DevOps automation will force us to look at how we can model the real world in a more holistic way, which includes incorporating human judgement and manual execution of non-repeatable tasks while automating repetitive steps.

  • Splunk Unveils Unified Observability Platform

    Splunk Unveils Unified Observability Platform

    Splunk today unveiled an observability platform delivered as a cloud service that captures all metrics, traces and logs in real-time without relying on sampling.

    Spiros Xanthos, vice president of product management for observability and IT Ops at Splunk, said Splunk Observability Cloud collects all that data via a single agent based on open source OpenTelemetry software to instrument an application.

    Historically, application performance management platforms (APMs) have only analyzed samples of data to identify a potential anomaly. Splunk Observability Cloud analyzes all the streaming data captured by its agent software to enable IT teams to pinpoint the root cause of an issue faster, said Xanthos.

    As a result, Xanthos said the signal-to-noise ratio dramatically improves compared to legacy monitoring platforms. It will then be up to each IT team to decide how much of that data they want to store after it’s been analyzed, added Xanthos. Pricing for Splunk Observability Cloud is based on the number of hosts employed rather than the amount of data collected.

    The Splunk Observability Cloud unifies a Splunk analytics portfolio by including capabilities provided via Splunk Log Observer, Splunk Real User Monitoring (RUM), Splunk Synthetic Monitoring, Splunk Infrastructure Monitoring, Splunk APM and Splunk On-Call. Splunk Synthetics Monitoring is based on technology Splunk gained with the acquisition of Rigor last year.

    Splunk

    The Splunk Observability Cloud is one of the first such platforms to rely solely on OpenTelemetry agent software being developed under the auspices of the Cloud Native Computing Foundation (CNCF) to collect data. That project not only reduces the cost of instrumenting applications, it also reduces the number of agents an IT team might have to deploy to collect metrics, traces and log data.

    Many rival observability platforms can collect data from OpenTelemetry agent software, but also tend to encourage DevOps teams to deploy proprietary agent software that they claim is easier to install and collects data at a more granular level. Splunk has decided OpenTelemetry has matured to the point where those proprietary agents are no longer required, said Xanthos.

    The rise of observability platforms presents an opportunity for IT teams to rationalize a raft of monitoring tools that often surface conflicting analyses. IT teams today spend an inordinate amount of time convening so-called “war room” meetings to determine the actual source of an issue. Those meetings can go on for hours, only to eventually discover an issue that only takes a few minutes to fix.

    While observability has always been a core DevOps principle, the level of visibility most IT teams have into their application environments is limited. There is no shortage of monitoring tools, but because there is no unified approach to observing an IT environment, there simply isn’t enough context being provided. As IT environments become more complex – thanks primarily to the rise of microservices-based applications that have lots of dependencies – that lack of context is rapidly becoming a major issue. A microservices-based application is designed to degrade gracefully if one microservice suddenly become unavailable. The issue is that it can take a very long time to discover the root cause of performance degradation.

    It may be a while before most IT teams achieve true observability, but once they do, many will wonder how they previously managed IT without it.

  • Debugging is Broken – Here’s How To Fix It

    Debugging is Broken – Here’s How To Fix It

    Let’s face it, debugging is broken.

    Whether you wholeheartedly agree or vehemently disagree with that statement, I think we can all agree that debugging, at the very least, has room for improvement. In fact, the way software developers debug their code hasn’t changed much. Of course, there are now a wide array of observability and monitoring tools available to make a developer’s life easier, but when we look at the process developers typically follow when debugging code, it’s stayed relatively unchanged over the years.

    Having the right tools available definitely improved the overall debugging experience, but they’re not the only factor. Internal organizational processes are equally as important, and should be adapted to make developers’ lives as smooth and stress-free as possible while still maintaining the proper levels of organizational security and compliance. Inefficient or highly restrictive processes, while great for the compliance team, can adversely affect the velocity of getting to the bottom of issues quickly. The right solution is typically a combination of adapting those processes to fit the company culture, along with a combination of tools that allow for compliance with industry standards and best practices.

    How Developers Debug Code

    When you talk to developers today about how they debug their applications, the process is typically not all that different than it was many years ago. The standard approach to debugging is still through the use of logging frameworks. Of course, teams are taking advantage of things like application performance management (APM) tools and other tools in the monitoring and observability space, but when you look at the primary method of debugging issues that require developers to dive into the code, logging is typically the main approach.

    The process typically looks something like this:

     

    debugging process

    When a defect is found, developers will first go to the log files on the production or pre-production servers in the environment where they are debugging. If they can’t find information in the logs to help them get to the root cause of the issue, they go back to their code and add additional log lines, finally sending their code through the process shown above. For highly advanced organizations, the process can take minutes or hours, but more often than not, this process drags on for days – or even weeks. It becomes especially time-consuming when the defect developers are trying to find and remediate is in production. In these cases, the cost to the business can be staggering; there’s got to be a better and more efficient approach to debugging.

    Where Tools Can Help

    While new and shiny tools don’t always solve all debugging problems, they can often ease some of the pain developers face and, in the process, make their lives a little bit easier. Debugging, observability and monitoring tools can get developers the data they need to solve problems faster than the more traditional method of digging through logs and using trial and error.

    The place many organizations start when considering diagnostic tools is with APM. These types of tools focus specifically on monitoring and management of the application availability, and can help to collect metrics and other relevant application information that can aid in pinpointing infrastructure issues. Most often, these tools are not actually used by developers themselves, but rather operations or production support teams to ensure availability and performance thresholds of applications.

    Another critical category of telemetry are tracing tools, which can help developers better understand how requests flow through multiple tiers of applications and even help identify error rates and latency. There are many vendors selling this category of tools, but there are also open source options which allow developers to integrate tracing into their applications via a vendor-neutral approach.

    After diagnosing an issue with an APM solution, there is often a handoff to development to remediate the issue. When developers need to debug, they often use their local IDEs or, even better, a remote debugging solution. Remote debuggers are used directly by developers to simplify debugging of applications in remote environments. These tools helps developers get to the bottom of customer issues faster by giving them direct access to the underlying application data in a safe and secure way while the application is running without stopping it or having to redeploy it. A major benefit of remote debuggers is that they can give developers the data they need from production environments without compromising security or requiring developers to have direct access to sensitive environments. While remote debuggers and other debugging tools are not a silver bullet, they can be a critical component that empowers developers to better address critical issues.

    Fixing Organizational Problems

    If solving customer issues is taking longer than expected or is potentially hurting your brand, it’s critical to sit down with your development team and use a data-driven approach to identify where the problems lie. Do a deep dive into your issue tracking or ITSM system to identify trends or recurring issues within particular application components. Development teams can often become so overwhelmed tackling issues and building out new features that valuable information in these internal systems can go untapped. Customer issues and defects should be reviewed periodically to assess things like business impact, cost to fix and alignment with the product roadmap. There is valuable information that can help focus the team’s effort on time-saving activities as well as improving the product, making for happier customers.

    Another area to look at is your software delivery process. Are there bottlenecks in your delivery pipeline that could be made easier or more efficient but have been overlooked? The speed and efficiency of your software delivery pipeline has a direct impact on developers’ ability to debug effectively and can contribute to a broken or slow debugging process. Developers should have the ability to push code into new environments as quickly as possible to remove slowdowns from too much context switching. This means automating as much of the delivery pipeline as possible, removing unnecessary manual deployment approvals and shifting left as many activities as possible.

    When it comes to empowering developers and fixing broken debugging processes, organizations should focus on fixing organizational bottlenecks as well as equipping developers with tools that can save them time and improve their daily workflows. Focusing on these areas can increase development team velocity and eliminate costly day-to-day slowdowns which add up over time and negatively impact productivity. With a few key changes, debugging can be a much more manageable and predictable process.