devopsy.kinsta.cloud

Category: DevOps Toolbox

  • Micronaut Taps Moderne to Automate Java Framework Updates

    Micronaut Taps Moderne to Automate Java Framework Updates

    Micronaut, a provider of a framework for building applications based on microservices and serverless computing platforms, today revealed it is allying with Moderne, a provider of a tool for refactoring code, to create a “recipe” that will automatically update the Micronaut framework.

    Moderne CEO Jonathan Schneider said the open source OpenRewrite refactoring tool for code, originally developed at Netflix, makes it possible to automate updates to codebases, remediate vulnerabilities and provide a foundation for high-level building blocks that will make authoring custom transformations simpler. That approach will eventually be embraced by other providers of application development frameworks.

    Fresh from raising $4.7 million in initial funding, Moderne makes OpenRewrite available via a software-as-a-service (SaaS) platform to enable IT teams to search and transform millions of lines of code in minutes. OpenRewrite currently supports Java, but the company is working toward adding support for additional languages, including infrastructure-as-code (IaC) tools such as Kubernetes, YAML and XML.

    Moderne has also developed “recipes” for common search and refactoring operations that can be assembled into more complex recipes for framework migrations involving, for example, an upgrade from Java 8 to Java 11. Micronaut is the first provider of a development framework to create a recipe for its own framework using OpenRewrite.

    At a time when applications are more distributed than ever, updating codebases is a major challenge. In fact, the manual effort involved in making those migrations is one of the primary reasons many organizations continue to employ older versions of frameworks such as Java 8. OpenRewrite is designed to automate that process at scale in a way that leaves it up to the individual developer to determine how involved they want or be or if they simply prefer to have a DevOps team manage the upgrade process on their behalf, said Schneider.

    In the longer term, Schneider said he expects recipes will be created to address everything from mass migrations to remediating specific types of vulnerabilities across an entire codebase.

    Too much reliance on legacy development frameworks generally stymies innovation. The latest versions of Java, for example, are designed to make it simpler for developers to build microservices-based applications.

    It remains to be seen to what degree DevOps teams will embrace automation to upgrade code. Many developers are hesitant to rely on automated tools of any kind. However, given the need for most developers to spend more time writing new code, it’s only a matter of time before most of them come to terms with automated approaches to updating existing code.

    In the meantime, the cost of supporting legacy frameworks should decline as organizations begin to migrate their codebases faster. One of the reasons there is so much technical debt in the enterprise is because different development teams tend to adopt different framework versions at different times. DevOps teams can find themselves being asked to simultaneously support multiple versions of the same framework. There may always be multiple versions of those frameworks, but, over time, DevOps teams should eventually see the bulk of deployed applications all using the same framework as the upgrade cycle becomes more automated.

  • The Future of IT: Smart Tools, Intelligent Technology

    The Future of IT: Smart Tools, Intelligent Technology

    In the last few years, the lines between information technology and development have been blurred. Developers need to know about infrastructure, and administrators need to know how to code. Advances like containerization, orchestration, continuous integration and infrastructure-as-code (IaC) happened in tandem with DevOps’ rise. The advantage of DevOps is that you can quickly build and deploy products. That comes at a cost, though, and that cost is infrastructure complexity. The answer to managing that complexity lies in technologies such as artificial intelligence (AI) and machine learning (ML).

    How can AI Help IT?

    Intelligent technology has come a long way in recent years. Machine learning and artificial intelligence are finding their way into more and more technology. New tools are beginning to emerge that can monitor and manage complex systems with relative ease. There are several clear upsides to using these systems in IT:

    • Instant notifications when there is a problem.
    • The automation of tedious IT tasks.
    • Dynamic infrastructure adjustments to help with traffic spikes and other changes.

    Training an AI to keep systems running optimally, apply updates and alert IT staff to problems in real-time is entirely possible today. By automating away a lot of tedious, manual work like digging through logs, IT is free to focus on mission-critical objectives instead of babysitting servers and monitoring dashboards.

    What Does That Mean for IT Jobs?

    Some people worry that advances in AI will lead to a world where robots take human jobs. While there will undoubtedly be some domains in which AI and ML make more sense than a person, those same technologies will also create new jobs. As IT teams are unburdened from tedious tasks, they can expand into working with ML and AI technology. IT teams are ideal for guiding business decisions about what tasks benefit the most from intelligence technology. So while IT jobs may be transformed by AI, there is no reason to believe that they will disappear. It is more helpful to think of AI as a new tool for IT instead.

    Where can IT Implement AI Tools?

    AI is best used for tasks where there is a lot of data. Having a human comb through log files, databases and dashboards is a slow process. An AI could continuously monitor everything, all the time, freeing up the IT staff to focus on the solution instead of tracking down which piece of the infrastructure is experiencing a problem.

    One area in particular that could benefit from AI/ML is fraud detection. Machine learning can analyze data to find fraudulent transactions much faster than a person ever could. And over time, it will get faster and more accurate at detection.

    Another area with potential for AI is the customer-facing helpdesk. Intelligent systems that have access to ticket resolution histories and device data may be able to handle a majority of requests without needing to use the time of an actual technician. If the AI fails to solve the problem, it can escalate the issue to a technician. As these tools mature, they’re impacting customer service processes, too; let the AI go as far as it can and then escalate less common and/or more complicated issues to a technician. This is an excellent example of how AI and IT can work together to provide the best possible outcome.

    AI and ML tools are the future for IT teams. These tools will be able to handle a lot of the tedious work that is currently done by people such as parsing logs, monitoring traffic and providing some measure of technical support. That leaves IT to focus on mission-critical issues and problems that AI and ML  cannot easily solve.

  • Foundation Proposes Advancing eBPF Adoption Across Multiple OSes

    Foundation Proposes Advancing eBPF Adoption Across Multiple OSes

    The eBPF Foundation, which proposes to advance adoption of an approach that enables sandbox programs to run faster at the kernel level, is being launched today as an arm of the Linux Foundation.

    The technology known as eBPF was originally developed for Linux. The eBPF Foundation is now committed to expanding use of eBPF across all operating systems. Members of the eBPF Foundation include Facebook, Google, Isovalent, Microsoft and Netflix.

    Thomas Graf, CTO and co-founder of Isovalent, a provider of networking and security tools that leverage eBPF, said IT organizations should expect to see the technology become part of Windows, along with other operating systems, as the core technology is tested and validated across a much wider range of platforms.

    In effect, eBPF changes the way operating systems are designed. It bridges the boundary between kernel and user space by enabling developers to combine and apply logic across multiple subsystems that, historically, were completely independent of one another. That approach enables, for example, a security tool to scale to the point where it can identify threats at much higher levels of throughput to improve overall scale at a time when the volume of cybersecurity attacks being simultaneously launched continues to increase.

    Currently, eBPF is widely employed by web-scale companies such as cloud service providers. Facebook is using it as the primary software-defined load balancer in its data centers while Google is using open source Cilium networking software within its managed Kubernetes offerings.

    However, adoption within on-premises IT environments that have deployed Linux has been more limited simply because the number of vendors that have optimized networking, security and storage offerings to take advantage of eBPF is still fairly limited.

    Sysdig, for example, recently launched made available an instance of the open source Falco container security platform available that employs eBPF. Tigera has also made available an instance of its container networking platform that leverages eBPF at the kernel level of Linux.

    Ultimately, Graf notes the biggest benefit of eBPF is efficiency. The total cost of processing for security, networking and storage platforms should decline as more vendors take advantage of a capability that will one day be widely employed across multiple operating systems.

    In the meantime, IT organizations would be well-advised to ask their vendors when they plan to support eBPF. Cloud service providers generally require it to deliver their own managed services more efficiently, so any vendor that hopes to sell a platform to those providers will need to support eBPF to drive performance at scale. The issue then becomes determining which platforms that have embraced the technology are also available in on-premises IT environments looking for similar benefits.

    Regardless of approach, it’s clear there will soon be a reason to upgrade not just networking, storage and security platforms but also instances of operating systems that don’t yet support eBPF. DevOps teams would be well-advised to plan accordingly because the number of platforms that will ultimately benefit from eBPF spans the entire enterprise.

  • Choosing an Incident Management Platform

    Choosing an Incident Management Platform

    When you’re feeling the stress and pain of manually managing incidents and incident response, making the decision to find an incident management tool is a no-brainer. But how do you choose the one that will work best for you, your team and your business?

    You might be asking yourself, “Where do I start? What do I need to know? What questions do I ask? What are the options? How can I be sure we’re choosing the right tool?”

    To help identify the most appropriate solution for your organization, here are the top four things to consider.

    1. Do you have the culture and processes you need to support an incident management solution?

    Different business models require different approaches to incident management. You should be looking for a platform to fit the unique needs of your organization. More importantly, even if you find a solution that seems like a fit, do you have the people, culture and processes necessary to support a new tool? And if not, do you have a plan to put them in place?

    Implementing solutions successfully takes planning, so you’ll want to make sure that you and your team can handle the workload.

    2. What are your pain points and goals?

    You’re looking for an incident management solution for a reason, but do you know exactly which pain points you hope to solve? Your top goal may be to “make it easier,” but every solution is likely to tout that they make things easier. Determine what you’re hoping to accomplish. Do you want to automate processes? Bring your team together more quickly? Track every action from declaration to retro? All of the above? If you prioritize the problems you want to solve, it’ll be easier to check off boxes when you investigate your top choices.

    3. Who should be involved in this search?

    You may be the one taking the lead on searching for a better way to manage incidents, but knowing who else should be part of the decision-making process will make the search more efficient. You’ll need to know if you have support from your team and from leadership; who’ll authorize the budget for your new tool and who has to sign off on your decision.

    And don’t forget other stakeholders outside of your team! Incidents impact all parts of an organization, so consider who needs to be a part of the selection process—from your client success team to your sales team.

    4. How will you make a decision?

    Finding and implementing an incident management solution requires time and resources. It’s especially important to account for the time it will take to onboard your team and configure your solution, including assigning individual roles, setting up integrations with your existing tools and test-driving your incident management process.

    The worst-case scenario is that you purchase a platform but don’t have time to implement and use it. You won’t be happy, nor will your team or executive stakeholders.

    So have a clear idea of what you need to make a final decision—whether it’s considering who’s involved, how much budget you need to secure, what your timeline is or how you’ll address any legal or security issues.

  • Google Unveils Tool to Better Secure GitHub Repos

    Google Unveils Tool to Better Secure GitHub Repos

    Google today launched a GitHub app that provides automated continuous enforcement of security best practices for GitHub projects.

    Kim Lewandowski, a product manager for open source software security at Google, said the Allstar application enables IT teams to assess any project on GitHub to check for security policy adherence. In addition, Allstar sets desired enforcement actions and automatically applies those rules when triggered by a setting or file change in a repository.

    The goal is to provide the open source community with a tool that makes it possible for organizations to have more confidence in the open source software that is being employed within a software supply chain, said Lewandowski.

    Allstar is intended as a companion application for Security Scorecards, a tool that Google made available last year to assess whether, for example, an open source project employs branch protection to ensure that malware isn’t inadvertently committed to a project. Allstar continuously checks expected GitHub API states and repository file contents against security policies and the enforcement actions defined by an IT organization. After it detects a policy violation, the tool can be configured to simply send an alert or to automatically enforce a specific policy to remediate the issue, said Lewandoski. Specifically, she said the options available are to log the security policy violation without taking any additional action, open a GitHub issue or modify the GitHub setting to match the original Allstar configuration.

    At present, Allstar provides a limited number of security policy checks, but more will be made available over the coming months, said Lewandowski. Current security policy checks include branch protection, vulnerability disclosures, access controls and detection of binary artifacts that can’t be scanned. Additional checks that will be added include automatic dependency updates and frozen dependencies.

    Lewandowksi said Allstar and Security Scorecards are part of an ongoing Google effort to give back to the open source community that is being targeted by cybercriminals attempting to compromise software supply chains. Just about every application is now dependent on open source components, to some degree. The maintainers of those projects, however, don’t always have the tools or expertise required to check for malware that has been injected into a codebase, she noted.

    In general, Lewandoski noted that achieving DevSecOps best practices will require a lot more automation of application security. It’s not possible for each developer to become a world-class cybersecurity expert. The challenge is finding a way to introduce that automation into the application development process as early as possible, which, she noted, in many cases starts with open source components that are incorporated in applications.

    It may take a while for every maintainer of an open source project to revamp the processes through which code is added to their project. However, the easier it becomes to continuously review that software, the more confidence organizations that use it will have in its security. In fact, the maintainers of those projects should expect the downstream users of that software to be asking some pointed questions about how security is being managed in the wake of a series of recent high-profile breaches of software supply chains.

  • DataOps Vs. DevOps: What’s the Difference?

    DataOps Vs. DevOps: What’s the Difference?

    There is a mindboggling amount of data today; to even measure it requires using a byte measurement called a zettabyte, which is one sextillion bytes (that’s 21 zeros). Currently, because such a ridiculous amount of data exists, there is a growing urgency to end wasteful data processes. From this environment, DataOps was born. Similar to the way that enterprises adopted DevOps to formalize and streamline wasteful development practices in the past, today, many large organizations turn to DataOps to formalize modern data management practices.

    What, Exactly, is DataOps?

    Primarily, many companies are adopting these principles because they are either trying to avoid or rectify data debt, which is the amount of money required to fix data problems due to the mismanagement of data processes. Data debt is a strong motivator for revamping outdated processes and policies, particularly when decision-makers and stakeholders require metrics before implementing change. Unpaid data debt can be detrimental to a business; the longer it remains unpaid, the more it costs to maintain a data landscape.

    By implementing DataOps principles and data governance, an organization can effectively reduce its data debt and prevent it from growing any larger. Moreover, DataOps practices and software engineering can be used to detect inefficiencies, minimize knowledge loss and capitalize on missed opportunities related to data usage.

    Similarities Between DataOps and DevOps

    Much of the processes that enable DataOps were borrowed initially from the same foundations that built DevOps. Likewise, just as companies need DevOps to provide a high-quality, consistent framework for software and feature development, data enterprises also rely on these same features to realize rapid data engineering and analytics development. For organizations that already have a DevOps framework in place, leveraging DataOps is relativity straightforward. Several important DevOps concepts adopted by DataOps include:

    • Agile development
    • Focus on delivering business value
    • Continuous integration and continuous delivery (CI/CD)
    • Automated testing and code promotion
    • Reuse and automation

    The Differences

    Despite the similarities between the underpinnings of DevOps and DataOps, there are several major differences.

    The human factor: The people using DataOps and DevOps have divergent personalities and skillsets. DataOps participants may be tech-savvy, but often their knowledge is theoretical. DataOps professionals can include data engineers, data scientists and analysts who focus on creating models and visual aids. DevOps, however, was made for software developers and engineers–coding is in their DNA.

    The process: The life cycles of DataOps and DevOps do share similar interactive properties. But, the former deviates in that it consists of a data pipeline and an analytics development process, both active and intersecting. While conceptually, the pipelines of DataOps resemble the development processes of DevOps, typically, experts note that the DataOps process is more challenging.

    Orchestration: In the DevOps process, application code does not require complex orchestration. However, for DataOps, both the data pipeline and analytics development orchestration is an essential component. Although orchestration in the DataOps pipelines occurs frequently and drives data flows, there is usually no such coordination of pipelines in application development and DevOps processes.

    Testing: Again, the two pipelines of DataOps create a significant difference from DevOps; testing in DataOps occurs during both the data pipeline and the analytic development process. These tests attempt to catch anomalies, flag abnormal data values and–unlike DevOps–validate new analytics before deployment. Likewise, these tests get embedded into a data quality framework for continual monitoring.

    Test data management: In most DevOps environments, test data management hardly takes priority; with DataOps, it’s vital to accelerate analytics development so that innovation keeps pace with agile iterations.

    Tools: DevOps is the ‘father’ of DataOps, and as such, the tools needed to support the latter are still in their infancy. While testing in DevOps is primarily automated, DataOps doesn’t have the same luxury – most users modify testing automation tools or build their own from scratch.

    Exploratory environment management: Generally, data teams use more tools than software development teams. Moreover, exploratory environments in data analytics are more challenging from a tools and data perspective; data teams also naturally depart from data islands across the enterprise.

    While foundationally, the concepts of DevOps serve as a starting point for DataOps, the latter involves additional considerations to maximize efficiency when operating data and analytical products. Nevertheless, both serve their intended audiences, reducing data debt and evolving data products or shortening systems development life cycles or providing continuous delivery. For businesses looking to make internal data-related processes more efficient, they should start by examining best practices associated with DevOps.

  • A New DevOps Test Environment Platform is Needed

    A New DevOps Test Environment Platform is Needed

    Test environment management is bottlenecking DevOps transformations and is driving the need for a new test environment management platform solution. On-demand test environments in which developers and testers do not have to wait for a shared environment are sorely needed.

    The evidence is clear. In my book Engineering DevOps I indicated that finding and setting up test environments is a primary source of bottlenecks in DevOps value streams.

    In my February 2021 webinar with Plutora, I presented Best Practices for Developing Test Management Environments. I indicated that orchestrating and managing test environments is often a primary source of delays and waste within value streams.

    The State of Testing Report 2020 by PractiTest indicates that DevOps practices increased 57%; from 26% in 2017 to 41% in 2020. The report revealed that 46% of testing has shifted to using CI/CD processes, 55% are actively involved in shifting testing left and 36% are testing in production. This is clear evidence that DevOps testing is on the rise.

    Wolfgang Platz’s December 2019 article Why Software Testing Remains a Bottleneck indicates that testing is responsible for more delays than any other part of the development process – accounting for 52% of the bottlenecks in the development process. And  53% of respondents to the World Quality Report 2020-21 indicated that test environments are not available when needed, 67% lack the capability to book and manage test environments and 71% lack visibility of available test environments.

    An article by Velocity echoes the problems with pre-production environments. “With the rise of microservices and cloud-native applications, environments are becoming more complex. Often you need to wait or compete to get access to develop on a specific microservice, or can spend time waiting to deploy to a shared staging environment only to realize something already changed, and you’re back to square one.”

    The Continuous Testing Report – 2020 by Capgemini defines test environment management (TEM) as managing infrastructures, test data and applications to support testing on demand. These environments are one of the greatest impediments to continuous testing. In this report, 36% of respondents said they are spending more than half their time building and managing test environments; 44% of respondents indicate they spend weeks of their time searching, managing and generating test data.

    Test environments should be regarded as functions to be spun up or destroyed at will, with virtualization, infrastructure-as-code (IaC)  practices, service virtualization, containerization and improved test data management.

    Given all the above evidence, it is clear that a new approach to DevOps test environment management is needed.

    The following are examples of benefits that DevOps test environment management platforms need to offer:

    • Compatible with current tools:
      • Automatically map and configure application infrastructure to develop and test multiple services in an environment that runs just like it would in production. This needs to be fully agnostic to cloud and tech stacks, such as Kubernetes and other environments.
      • The platform needs to work on applications in parallel without disrupting others, with full access to data and third-party dependencies.
    • Automatic configuration and maintenance:
      • The platform needs to automatically configure and update any changes within the production infrastructure’s mapping in real-time so ephemeral environments will always be synced with what is in production. Instead of having to spend time making sure any changes are configured, the platform will ensure the right environment that replicates production, without having to wait.
    • Data and third-party dependencies:
      • The platform needs to load the relevant data and third-party resources that are vital parts of the environment to test, work and debug against, so test environments will, as closely as possible, mimic working in production.
    • Team collaboration:
      • The platform needs the capability to share ephemeral test environments with other stakeholders so they get visibility and can collaborate for faster feedback cycles. This is also important for collaborating for debugging.

    What This Means

    DevOps testing is on the rise, and test environments are a major bottleneck for DevOps testing. A new DevOps test environment management platform is needed to reduce the bottlenecks. On-demand test environments in which developers and testers do not have to wait for a shared test environment are needed. Such a solution would improve agility and reduce wasteful orchestration bottlenecks in test, improve quality and risk management through comprehensive and automated testing and enable continuous improvement by providing more intelligent insights and feedback.

  • Cycloid Tool Can Reverse Engineer Provisioned IaC

    Cycloid Tool Can Reverse Engineer Provisioned IaC

    Cycloid this week unveiled a tool that makes it possible to reverse engineer code used to manually provision cloud infrastructure. The Infra Import tool is part of an effort to create a more consistent and reliable version of that code using open source Terraform software.

    Benjamin Brial, Cycloid CEO, said Infra Import is based on Cycloid’s open source TerraCognita software, which enables IT teams to reverse engineer infrastructure-as-code (IaC) created using open source Terraform tools.

    Infra Import extends that capability to cloud platforms such as Amazon Web Services (AWS), Microsoft Azure, Google Cloud Platform or an IT environment employing the open source OpenStack cloud platform that may have been manually provisioned. IT teams can then create a configuration based on that Terraform code that could be employed across multiple clouds.

    Infra Import is designed to connect to a cloud provider and then automatically reverse engineer Terraform files and IaC stacks based on the existing deployment. It also ensures that the latest edition of Terraform is being used to create that configuration code, noted Brial. That capability eliminates the need for DevOps teams to keep track of what version of Terraform might need to be updated as new versions of the IaC tool become available.

    Just as significantly, Infra Import also makes it a lot easier to onboard new members to a DevOps teams because the tools uses valid secure configurations they know can be relied on. Misconfiguration of cloud services is one of the biggest security issues that IT teams today are wrestling with, simply because most developers lack the security expertise required to configure cloud services properly.

    Cycloid is in the process of rolling out a comprehensive suite of management tools that complement a lightweight framework it created to make DevOps best practices more accessible. The challenge is many IT organizations have already manually provisioned cloud infrastructure in a way that is, at the very least, often suboptimal if not downright insecure.

    Misconfigurations are rife across cloud computing environments, but most organizations are not prepared to manually address the issue. Automating the reverse engineering of those configurations is the first step toward enabling IT teams to address the problem at scale.

    It’s not clear to what degree configuration concerns have impacted the number of applications that would otherwise have been deployed in the cloud. Security is always listed as the top concern IT leaders have when employing cloud platforms. The issue, however, is not the security of the platforms themselves but rather the processes employed to deploy applications on the platforms. Cloud service providers have promoted a shared responsibility model for security under which developers are accountable for both securely configuring services and ensuring the overall security of the software environment. The trouble is, not every developer understands the implications of that shared responsibility model much less how to actually secure an application workload.

    It’s not likely security concerns will slow down the rate at which applications are deployed in the cloud. However, as more cloud security issues are encountered, more scrutiny will be applied.

  • EP 12: DevOps Unicorns

    EP 12: DevOps Unicorns

    What should you look for when choosing a DevOps vendor? What criteria are others in your industry using to inform their decisions? How do you choose a vendor that is a good fit for your organization? And how do you know if you made the right choice? In this episode of DevOps Unbound, host Alan Shimel welcomes Larry Maccherone of Comcast, John Willis of Red Hat, Ravyn Manuel of the Smithsonian, Garima Bajpai of Capital Carbon Consulting and Mitch Ashley of Accelerated Strategies Group for a panel on DevOps unicorns. The video is below, followed by a transcript of the conversation.

    Alan Shimel: Hello, everyone. Welcome to another edition of DevOps Unbound, where we talk about DevOps, hence the title. Thanks for joining us. This episode is titled DevOps Unicorns: The Quest for End-to-End DevOps.

    You know, I’ve been in IT 30-plus years and there’s always been this tension or this pendulum that swings between the one throat to choke versus the best of breed solutions in any area. Today it’s DevOps and software delivery and deployment. But in anything, in the security world, in everything, in hosting, in infrastructure it’s always been a common theme. But it’s really, really becoming acute within the DevOps sector now. And we’ve assembled what we think is a great panel to discuss this issue. Let me introduce you to them and we will go from there.

    First of all I want to welcome, well, to a panel for the first time in a long time, my friend Larry Maccherone. Larry, welcome to DevOps Unbound. Give people a little background, if you don’t mind.

    Larry Maccherone: Sure. Thanks for having me here, Alan. First of all I want to say so I launched the Dev PsychOps Transformation Program at Comcast and grew that for five years. And I’m a developer at heart, though, so I was sort of the ideal person to take what was actually becoming more common outside of big enterprises, Dev PsychOps, and try to actually do a diverse approach inside of a large enterprise like Comcast. And so all of my experience is around that recently, the last five years or so, around that.

    Shimel: Excellent. Next let me introduce you, she’s a repeat visitor to DevOps Unbound. Garima, if I mess up your last name I apologize in advance, but Garima Bajpai. Close?

    Garima Bajpai: Almost. Yeah. So I’m Garima Bajpai. I’m based in _____, Canada and I’m the founder for the DevOps Community of Practice here in Canada, which has three chapters: Toronto, Ottawa, and Edmonton. I’m also the organizer of the DevOps Summit in Canada, which happens in November 8th and 9th. And I’m also associated with organizations like DevOps Institute as an ambassador, as well as CD Foundation as an ambassador.

    Shimel: Fantastic. And, Garima, I apologize on your last name. My bad. I’m trying to read it and the monitor’s too far away from me without my glasses. But thank you for correcting me, and welcome.

    Next up, another good friend of mine, my actually business partner on Digital Anarchist. And though that’s not his full-time gig these days; he’s very busy with his red hat career. The one and only Botchagalupe, John Willis. John, welcome.

    John Willis: Thank you, sir. Thank you, Alan. Hey, yeah, John Willis, Botchagalupe on Twitter. My current day job is in a team called Global Transformation Office at Red Hat. I’m sort of blessed to be on a team with Andrew Clay Shafer; Kevin Behr, the co-author of The Phoenix Project; and Jabe Bloom. Just quickly, you know, I’ve been in the industry for 40-plus years, 10 or 11 startups, 12 books, if somebody’s counting. Probably the most notable is the co-author of DevOps Handbook.

    Also was involved in basically the starting and building of the original DevOps Days. Was the only American at first DevOps Days and Damon Edwards and I created the first DevOps Days event in the USA.

    Shimel: Yep. And that’s only a partial list, John. Thanks for joining us today, man.

    And then I want to introduce – she’s off-camera right now, but hopefully when you’re watching this at least you’ll get to see her picture, but she was having some video difficulties, but happy to have her on audio, my friend, Ravyn Manuel. Ravyn, welcome. And if you wouldn’t mind, give people a little background.

    Ravyn Manuel: Thank you, Alan. I do appreciate being on your show. It is awesome. It makes me laugh. I am the senior application developer for the National Museum of African-American History and Culture. And I’m also the DevOps engineer for that particular unit. As a developer I implement some of the hands-on interactive that you play with in the museum, and as a DevOps engineer I am in charge of just about everything that’s DevOps, from the cloud solutions and how we – from the application development side, how we’re using cloud and the tools and writing strategies and keeping my leadership informed of all the good things that are out there.

    Shimel: Fantastic. Ravyn, welcome and thanks for joining us today.

    Last but not least, my co-host here on DevOps Unbound, Mitch Ashley. Hey, Mitchell, welcome. You want to just introduce yourself?

    Mitchell Ashley: Great. Great to be here. Man, what a panel. I am so excited to be with these folks on today. Yeah, I head up Accelerated Strategies Group, analysts from focusing on, of course, DevOps and digital transformation, full cybersecurity and cloud. And I also serve as CTO with MediaOps, so heading up the platform strategy and technology, so we used to deliver programs like this.

    Like John, I think we’ve been probably in the industry about the same amount of time and have zero books to my credits, so he’s made a lot more progress in those areas, but also a startup person and run a lot of software projects, product, as well as IT groups, as well as on the vendor side. So this will be a fun discussion.

    Shimel: Absolutely. I should mention DevOps Unbound is sponsored by our friends at Tricentis. To give you an idea of what a great sponsor they are, they don’t even have anyone on our panel today. But they do enable this show to go forward every other week, so many, many thanks to Tricentis.

    So, John, I laid it out, right, it’s the one throat to choke versus best of breed, how many vendors can we support, how do I package solutions that are more than just point solutions, or maybe I just want point solutions. What’s the answer here, John?

    Willis: The answer is, Alan, I would say if I had the right answer I’d be on a Grady-White right now in the Gulf of Mexico.

    Shimel: Exactly. And I’d be right with you.

    Willis: But no, I think this is a great panel. I agree with Mitch; I’m excited about just the discussions we can all have. I think the question – you know, I was thinking just that sort of Chinese curse translation, “May you live in interesting times,” right? And that’s clearly where we are right now. And I think your question starts with a shift, right? Like there was this – talking about you, Mitchell, some of us have been here forever, and like back in the day there was this question of one throat to choke versus should I do an ELA with computer associates versus piecemeal things, right? And probably less than one-percent of the enterprises were do-it-yourselfs.

    Now today you will find probably – I’m just going to swag the number, but it’s probably north of 10, maybe 15-percent, maybe even higher of companies that are now being challenged to do it yourself. You know, Kubernetes is a great example. That’s a can opener for companies to make a decision of are my vendors keeping up with what comes from basically sort of upstream technology that I should build myself.

    So I think that’s the sort of question we move into, is more about do you build internally to solve your problems, to keep up with the changes in the market and the technology, or do you try to find a pair of vendors or a single vendor – and there really is very few single vendors that can give you the whole scope of what’s going on in modern technology today. But I think that’s the question that I think we’ll have fun just talking about today. _____.

    Shimel: I don’t disagree. Panel?

    Manuel: So, Alan, this is Ravyn. I agree with you, John. I think one of the things – because that’s a struggle for me and my organization, is we need help with implementing DevOps, but I think the vendors are very tool-focused and the tools are like the third priority in the stack of there. And what I’ve been doing when I’m building the strategy for how we’re going to implement DevOps, I looked, and we’re talking about tools. I look at the standards, because if they’re making standards and the tool has to follow that standard then it’s plug-and-play, you don’t get – like if Docker decides that they’re going to go out of business then you can have another container-type technology to do that.

    So when we are looking at benders we actually kind of do a piecemeal, because we’re really unicorns. We’re not a software company, we don’t have software products; we have digital products, and they’re all one-offs. And so it really just depends on what fits for us. So we don’t really go for the one vendor solution; we actually go for who can help us get to where we need based on our business problem.

    Bajpai: Excellent. And I wanted to add a few things, because when we drill deeper into this conversation we will find out that there are two sides of this coin. One part is where an organization from a maturity perspective is. And the second thing is what are the vendors and key capabilities you’re looking at. But if you look at both sides of the coin there are three things which are needed to calibrate these DevOps tools in the overall context. The first thing, and this is – I want to reiterate the fact, because I come from a very large complex organization background, that it is very essential to have an enterprise architecture view. So I would suggest and recommend that before even, you know, taking this approach of single vendor or multivendor discussion, we need to kind of break down what are the key capabilities which are needed. So whether it’s a single vendor which can provide you that enterprise IT architecture capability, or it will be your organization who will be doing it. So that’s essential.

    The second thing I think which is also essential is orchestration layer. You know, you will have to have that governance of total cost of ownership, compliance, security, identity access management, and that orchestration layer has to be there to stitch all you calibrated DevOps tools. So it’s a question of whether you want to kind of build it in house in that organization or you kind of have a vendor-centric approach where this capability should be or could be brought in.

    So these are like two things, and then you start calibrating these DevOps tools. So you don’t have to look at the foundation aspects of compliance, security, governance, total cost of ownership, and the job becomes much more easier. The question is that, you know, how to bring these DevOps tools and enable business value.

    Maccherone: So I want to riff off of that orchestration layer idea, but I want to backtrack a little bit on sort of the – we have a binary sort of difference here between unicorn and best of breed. But I think within the unicorn bucket there’s really two major flavors of unicorn that are wildly different, and so I want to sort of bring that out. You have these Frankenstein vendors that are piecing together acquisitions into this monster and those are not pleasant experiences. The only benefit you get out of one vendor in that case is really the contracting benefit of having a single master service level agreement. You’re not getting an integrated product that really allows you to actually smoothly do the workflow, insist in the workflow that developers are using to produce value for your company.

    And then you have this other class of unicorns in the DevOps world, but less so in the security world, and I’ll talk about that in a second, which have taken a marketplace approach. And this is not a DevOps vendor, but I think the first vendor to really pull off this marketplace approach was Atlassian. You know, they had one product that was really great, but they said, “Okay, plug into this product” and they created APIs and they created toolkits and they created a whole community. Most of those people did it for free and they just sort of did it for themselves and then they shared it with the community. But some of those people became vendors and sold their product in the Atlassian marketplace, and then Atlassian acquired those. They were already pre-integrated with Atlassian’s other offerings when they got acquired, and so their acquisition unicorn approach doesn’t look like a Frankenstein’s Monster in the end, because they were built with the APIs of the vendor in place.

    And I think Github is definitely taking that marketplace approach and pulling it off really well, even into the security space. And so I think that kind of unicorn – and Github to a lesser degree, that kind of unicorn doesn’t suffer from that Frankenstein Monster’s sort of problem. No one in the security space has really pulled off the marketplace approach yet, though. We have some Frankensteins and Frankenstein Monsters over there, but we don’t have a full sort of marketplace approach. And I’m waiting for somebody to do that. I think that’s going to be the big win, somebody _____.

    Willis: I’d like to do a whole podcast on that, because that’s all I’ve been thinking about right now, is what is the end-to-end on security. But like if I got into it now it would take up way too much time. So I did want to comment on some of the things. You know, the first, Ravyn, I think the interestingly about standards, I agree, right? The problem we have right now is figuring out which standards are actually going to last, are malleable. I mean that’s one of the things I’ve really been working hard on and looking at like OSCAL versus OVAL versus S-CAT versus, you know, just NIST _____ – in order for – I mean there’s just so many things and they all sort of have a little bit of different approach.

    So I agree, the question we have to deal with now in these sort of, you know, may we live in interesting times is it’s like picking sort of an early vendor, like I like this vendor technology, but are they going to be around? Are they going to be acquired by a company that I don’t really want to do business with? I think the standards is just it’s such complicated.

    Just the other thing I wanted to say is I think that for vendors, you know, and I’m not _____ ’cause I work for Red Hat. Because I will say right now the thing that like in the grand scheme of technology, if you’re going to go all in a vendor, look around. You know, I’m going to say this. I’d love to be corrected and I’m not saying, you know, Red Hat because I work at Red Hat. Anybody that knows me, I’m not a shill for any vendor I’ve ever worked for. But it’s VMware and Red Hat, right? They can keep up with – because you’ve got to – in this world you have to have people on staff, their full-time job are contributors to an open source project.

    And if you think about what’s going on with like Kubernetes and clusters and server list, you know, you think of the two companies that are positioned really well at that sort of scale version to be able to keep up with the complexity and change of technology because they’ve put the resources in. I mean a longer list that you can probably make after that. And I’m not including the cloud providers either, ’cause that’s a different story.

    But anyway, those are the two comments I wanted to make on some of the things people talked about.

    Ashley: I just wanted to jump in. What’s fascinating about this conversation is how diverse and complex it is. There are a lot of aspects to consider, whether it’s an enterprise architecture like Garima is talking about, John’s talking about, who are the vendors that can pull it off. Larry is too, if you’re talking about this sort of roll-up strategy, but retro-ally integrated strategy.

    I think another dimension to consider is every organization has a DNA about how they think and operate when it comes to vendors and software and tools. And while I’m not always an advocate of go with the flow, sometimes when you’re thinking about how do we have something adopted, think about how the organization operates. If you are kind of a large tool vendor, let’s work with, you know, Red Hat or VMware or Broadcom or whoever might be – think about that. That’s oftentimes, you know, save yourself some brain cells and live a little longer.

    There’s also the innovation curve, and some of the best ways to ride the innovation curve is the open source route and the vendors that are built around that kind of a strategy. There’s combinations of all those.

    So I guess the thing I would add to the consideration of the conversation here is think about how you match up your strategy with the DNA of the organization so you can accomplish as much without killing as few brain cells as possible.

    Bajpai: And thank you, Mitch, for bringing up this point of DNA of an organization. I think that this is a pathway of dialogue which we are having. I do believe that the mindset is shifting, and even I see in large legacy enterprises that they have started to realize that, you know, this vendor-based approach is something that needs to shift, and they need to bring people as partners in this context. Because when you think about DevOps it’s basically exploring your opportunities, breaking the barrier for, you know, your technology onboarding, the gaps – the alignment and outcome gaps you have created in the past.

    I think this is the opportunity where the organizations can shift and bring more access to innovation through these partners. So I truly believe that there are softer aspects of this conversation as well which we need to kind of consider that traditionally how contracts are _____, you know, how we look at these DevOps tools from a vendor perspective, how can we bring them as partners, what we need to change if we start doing that.

    It’s not kind of done overnight, right? So we have to kind of set what kind of modeling practices we need to bring in. And this opens up another door of possibilities. And probably during this conversation we will have some opportunity to discuss these SLA constraints which we have built in the organizations which steer us to a specific vendor or steer us to like a specific window through which we can outsource a lot of capabilities and just, you know, a choke the throat approach, right?

    So let’s maybe in the upcoming discussions we also can bring these points and highlight.

    Shimel: Ravyn, I know Smithsonian is sort of quasi-government, or it’s not true government. But to Garima’s point about SLA and being able to only deal with chosen vendors like that, it is; it is a real governor on your ability to choose best of breed, on your ability to have choice in how you want to do things. Interested in your experiences around that.

    Manuel: It’s been difficult. I’ve been listening to this conversation and it just hit – I don’t know why it just hit me, but the vendors – and I love Red Hat, so I’m not trying to talk Red Hat or anything, and open-ship. But when you get a vendor that you have worked with and you trust, and Smithsonian, we use Red Hat a lot in our OCIO. Red Hat is doing open shift and Kubernetes and they are driving their tools to these specific tools. And for me, as a unit within the Smithsonian, I may not want to use open shift and I may not want to use Kubernetes, but I don’t actually have a choice, right? Because that vendor is a great vendor and they’ve got great tools, but they have chosen for me what I am going to be using, and that makes it really hard for me to say “Does that really fit what I really need? Is that too much?”

    Because I can tell you that a lot of the vendors that I talk to, they’re expecting us to use this like suite of products, and it’s too much for what we want to do, and so I’m left with I either go with this vendor and overspend and use tools that I don’t really need, or I have to do it myself because I can’t really find somebody that can customize that for me. So it’s a challenge.

    Shimel: Yeah, it sounds like it.

    Willis: You know, one of the things I struggle with, you know, I sort of love committees and hate committees, right? Because like I so – HashiCorp has this cluster manager called Nomad, and I’ve actually seen people use it effectively with like incredibly less toil, 80-percent of the features of it, but it’s never going to get any escape velocity. It hasn’t and it won’t.

    And I fear for our industry that we’re like so focused on there’s just only one way to do cluster management today where like – so to Ravyn’s point, like whether it’s Red Hat or whatever, it seems like you’re not even able to explore possible emerging containerized cluster management solutions because all of the oxygen in our industry right now is consumed by Kubernetes.

    Manuel: Amen.

    Shimel: Thoughts on that? I’d like to jump in with some thoughts on that. I mean this is sort of a dream/nightmare scenario. Right? I’ll take it – let’s take it off Kubernetes, let’s go back to PC operating systems. It was great that we had only Windows to develop on, and this way if you were an inept developer you only had to worry about developing on Windows. You had one operating system, you could make your app on it and it worked and you got 90-percent market share with it. What a fantastic thing for app developers. Of course, it gave Microsoft the inside track on Office and everything they did. And from a security point of view what a big, fat juicy payload that was, right, to go attack.

    So then we started seeing, for as long as I’m in IT, “This is going to be the year of the Linux desktop.” Right? How long have we heard that? And so, you know, and then of course Mac resurfaced.

    But this is we strive for some uniformity. We strive for Kubernetes to be the standard, ’cause it’s open source and it’s run by a foundation, and we want a standard. But then we chafe that, “Hey, we all can’t fit into that same standard. We want to be our own unicorn.” And that’s really what we’re talking about here today. If you go back to early, John, you sold the company to Docker before Kubernetes dominance, right? And there were several cluster management solutions vying. Right? There was Mesosphere and Rancher out there _____.

    Willis: Mesosphere and _____, yeah. [Laughs]

    Shimel: You remember. Yeah, _____ of these, exactly. So, you know, this is – but this is the world we – you know, to quote Hyman Roth in the Godfather II, “This is the world we chose to live in,” right? We strive for these single kind of standards that one-size-fits-all. And maybe that’s a huge mistake; I don’t know. Maybe it is a mistake. Larry, what do you think?

    Maccherone: Yeah, I do see that problem, John, you’re sort of focused on here, that sometimes the de facto standard is not the best technical choice. You can go all the way back to Betamax versus VHS, for those of you who are old enough to remember that, right? VHS won out, but Betamax was considered by everyone to be a superior technology. Why?

    You know, so I think if you’re going to take a best-of-breed approach you’re going to have to – when you have a true de facto standard like Kubernetes you don’t have to isolate yourself from it, but when you’re making a bet on which one is going to win out – and John is really good at this; John talks about this all the time. John sort of like doesn’t want to put his chips down on something that’s not going to survive long-term, so he wants to think about what’s going to survive.

    So in order to hedge your bet there I think you need this orchestration layer that was talked about earlier. You need to sort of isolate the tool vendor from the way it’s consumed to a certain degree, so you can plug-and-play a little bit as time goes on. And this is what we did at Comcast, we built a layer between all of the tool vendors for security tools, because they don’t have a good unicorn solution there. There is no good unicorn solution there anyway, even if I wanted to. And so we had more than one vendor in every tool category competing internally, and sometimes we had three tool vendors. But from the developer’s perspective it was really easy for them to switch out from one to the other, and they got the same feedback channels, they got the same visualization, it aggregated and scored the same way across all these different vendors.

    I think that orchestration layer is really key when you have a best-of-breed approach, and I think you have to build it yourself. I don’t think you can buy it today. I really don’t think you can do that.

    Willis: I’ll try to shut up quickly. I just want to make a comment. I think there’s two books that help us understand our world. One is Guns, Germs, and Steel by Jared Diamond, which actually shows you why we get into these predicaments. If you haven’t read it, it’s a fabulous book. And then Start With Why by Simon Sinek is the sort of anti-pattern to that, which I would say like there’s clusters – atomic computed things like sort of serverless or functions or containers will be around for 10, 15 years. Clusters of those things will be around for 10 years. So if you think in those terms instead of thinking about Kubernetes and Docker – and Docker is already slowly dying, then you get that sort of start with why isolation against these sort of Guns, Germs, and Steel trends.

    Bajpai: Right. Then I also wanted to kind of bring another aspect of this, which is connected, but I think companies might start looking at it from a scaling perspective. We talk about a lot of tools and how to calibrate these tools in our DevOps pipeline. Of course when you have a large organization you have multiple product lines, and these product lines will choose to be flexible, so they need to have that flexibility to deploy their DevOps pipeline in the way they choose. And that’s the whole power of DevOps, right?

    But I also feel that the conversation in this discussion is leading us to a point where we start looking at our technical oversight committee from a governance perspective, where we can stitch the dots, we can actually lay that common foundation which is required in order for people not to kind of rewind the _____ over and over again. So I think that’s the key here.

    And of course when we talk about a specific random _____ with huge capabilities it has pros and cons, right? So one pro would be like they can mutually enforce new capabilities. So when they bring one feature in one product it’s mutually enforcing another product, which is a beautiful advantage they bring in. But that also limits our possibility to look beyond what the vendor is bringing in. So we are kind of essentially not exploiting our own innovation cycles to introduce more capabilities around how we want to experiment, how we want to kind of build innovative new features into that life cycle.

    So these are kind of good considerations to look at and I truly believe that the technical oversight committee is something that you will see in large organizations a wave kind of starts in a bigger way when it comes to DevOps.

    Ashley: And there’s interesting too, when you think about different sizes of companies that are in the market for technologies, you know, Larry mentioned what he did at Comcast or what he was part of that was created as sort of an architecture where tools can be plugged in, unplugged, replaced, etcetera. It sounds simple, but I know that they have an architecture to do that. And some organizations have the wherewithal to create those things. What we see in the data, looking at the market, is in the small business, the kind of sub-200 people or so companies, they have really good high-performance teams that can work to create software or set up their own environments, kind of create a very productive setup for themselves, tool chains, etcetera.

    Then on the high end or the large end, the enterprises, you know, they have a lot of resources available to them that they can potentially marshal to figure out how to do some of those things. The mid-tier market, the medium-sized company is often where they’re most challenged to – they want to do what the enterprise can do, but they just don’t have the resources. They don’t have the money or the vendor relationships across the board to do all those things and they’re expected to deliver at almost an enterprise-class level, but they don’t have all of the resources to be able to do that. And that’s often where somebody – the non-Frankenstein vendors that do a good job of kind of delivering a suite of solutions can work in those situations. Not that they can’t work in the others; they can, too.

    But depending where you are in that kind of a company, that size of a company, makes a big difference on what strategy you can take.

    Maccherone: I think that’s a great point and my recent experience last five years has really been enterprise. But prior to that I was talking to lots of companies, and you’re right, the mid-sized companies are the ones that are sort of in that no-man’s land sometimes, where they need some standardization, they need some consistency, but they don’t have the wherewithal to put it all together. And a vendor unicorn approach might actually be better in those circumstances.

    Shimel: Hey, guys, I want to – Ravyn, I’m going to let you in a second. John, the name of the first book that you mentioned – I want to put these books in the notes somewhere.

    Willis: Oh yeah. I did a presentation with Josh Corman way back with Guns, Germs, and Steel. Anyway, it’s an amazing book about how civilizations move from sort of – you know, how they go from sort of agriculture and they move into these clusters. If you can see the forest beyond the trees you can see how we get into these like Kubernetes is the only word we can use. Anyway, Jared Diamond, Guns, Germs, and Steel, it’s incredible.

    Shimel: All right, we’ll try to get it in there. Ravyn, did you want to say something and I cut you off?

    Manuel: No, I don’t think that was me.

    Shimel: Garima?

    Bajpai: Yeah. I wanted to actually enhance what Larry was saying about midsized companies. This is totally based on my experience and exposure with the Canada DevOps Community of Practice. We are seeing more and more of these midsized companies, not only the consulting companies, as well as the companies who want to build a DevOps capability coming to the community of practice and starting to steer this discussion of evaluation and assessment of tools. And I’m also seeing that the winter side. You know, specifically we are talking about tier two companies here. The winters and coming and exposing the tool capabilities in the community of practice, and they want some support and help for us to kind of steer this conversation “Is this a good tool? Is this a good capability where they are in their context?”

    And it’s a very interesting discussion because the technical side is one side of the story. I would also like to kind of emphasize the fact that the questions we ask these vendors and these companies are like, “Okay, do you have a product owner for this product?” Because essentially, you know, what it means is that how accountability is built in a specific tool perspective. I also kind of ensure that we ask them what kind of themes they have, like what kind of cross-functional themes they have, so that we do not miss out on the point of interoperability, integration, automation, all these like are important points.

    So I just wanted to mention that how we take that approach from the community of practice perspective when we’re asked to kind of assess.

    Willis: You know, I totally agree with all of that. One of the things I was thinking about, I’ve had this luxury of being involved in these different communities. Like I’m in Alan’s community, you know, so I’ve gotten hooked into usually what’s interesting and going on in Alan’s world, which is a pretty big world. I’ve been hooked into Gene’s world pretty heavily, you know, DevOps Enterprise Forum/Committee. And then, you know, I’m one of the co-founders and advisors for the DevOps Days.

    And one of the things that I get to see is true leaders. And I don’t want to put all this sort of weight and gravity on like you have to have a great leader to be successful, but like as I think about the people – Larry is a great example. Like, you know, you can talk all day long about what Comcast does. I’ve seen Larry in action. Like that’s the kind of guy you want in your enterprise.

    Courtney Kissler now, you know, was at Nike and now she’s moved on to a startup. Ross Clanton now at American Airlines, Heather Mickman. I mean I can go on on the list, but these are people that impact change in larger organizations that where it’s incredibly hard. And there’s just something about that.

    Maccherone: There is an immune system at large enterprises, though, that makes it hard for someone like me to actually be hired and be successful. You know, the idea of disruptive change is not something that enterprises are sort of seeking out ever. And so if you’re sort of like grassroots and you’re sort of like ground up and you get a few pilots going, that’s easy; there’s not a lot of resistance. It’s when you get enough momentum that somebody feels like their domain is threatened that you get the political pushback in an enterprise. And, you know, that’s really hard to overcome. Companies have to be very conscious about that. And Comcast has done a great job of that for me in the past, but I see it in the nature of the enterprise immune system essentially.

    Ashley: We probably could take any one of those last three comments and turn that into a whole other dialogue. Maybe we can get some help from the folks in the production department for pulling us back together.

    Alan, do you want to close this out?

    Shimel: One thing’s for sure, is we’ve got a lot more to talk about on this subject. I’d love to – I invite you all back to continue the conversation. But for now, Ravyn, I appreciate you soldiering through this on audio only. If we do it again we’ll make sure to get that video working. Garima, always a pleasure to have you on here; you bring so much. Larry and John, what can I say, it’s a pleasure to see you guys. I can’t wait till we can see each other in person. I know a lot of us have gotten our shots; it won’t be long. But thank you so much for adding so much. Mitchell, fantastic job.

    Most of all, hey, thank you people that are watching this, because that’s why we do this, is for you guys to get something out of it, join in the conversation, let us hear what you’re thinking. Many thanks to Tricentis for sponsoring DevOps Unbound.

    Until next time, though, this is Alan Shimel, Media Ops for DevOps Unbound. Have a great day, everyone. Bye-bye.

    [End of Audio]

  • Logz.io Allies with Microsoft on Open Source Observability

    Logz.io Allies with Microsoft on Open Source Observability

    Logz.io this week announced an alliance with Microsoft that enables DevOps teams to deploy its observability platform—based on open source platforms such as ElasticSearch, Logstash, Kibana (ELK), Prometheus and Jaeger—on the Azure public cloud.

    Jonah Kowall, Logz.io’s CTO, said the multi-year effort enables those teams to deploy the Logz.io platform via the Azure Console with the click of a button. The Azure platform has a data collection capability that aggregates operating system events in a way that makes it simpler for the Prometheus monitoring solution to consume, he added.

    DevOps teams, however, will still have to collect data from the applications they instrument on the Azure platform, he added.

    Logz.io integrates multiple open source monitoring tools based on metrics, logs and traces within a single observability platform that otherwise would require considerable time and effort from internal IT teams on their own.

    Interest in observability is rising as organizations attempt to provide internal IT teams with more context when troubleshooting issues. Whenever there is an IT issue, most teams convene a “war room” where they attempt to discover the root cause of an issue by comparing analytics generated by a wide range of monitoring tools. Observability platforms promise to enable a DevOps team to replace many of those tools in a way that provides more context by integrating log data, metrics and distributed traces within a single platform.

    In many instances, existing application performance management (APM) platforms are being transformed into observability platforms by adding tools to monitor IT infrastructure. Conversely, platforms for managing IT infrastructure are adding support for tools to monitor applications. Elsewhere, a range of startups have emerged to offer observability platforms they have developed from the ground up.

    Regardless of the platform employed, the rise of microservices-based applications within many organizations is forcing the observability issue. The dependencies that exist within microservices-based applications are too complex to manage without a platform that provides some level of increased observability. Of course, most existing monolithic applications are not being retired any time soon, so the need for an observability framework for both legacy and modern cloud-native applications has only become more pressing.

    The Logz.io platform can, of course, be deployed on other cloud platforms as well as within on-premises IT environments. As hybrid cloud computing continues to evolve, it is apparent that observability platforms will need to extend their reach from the cloud to the network edge. Over time, artificial intelligence (AI) capabilities will be added to make it easier to, for example, streamline the number of alerts that are generated across complex IT environments.

    In the meantime, the most important thing with regard to observability may be simply getting started. Most IT teams start to analyze logs before moving on to metrics and then traces using the open source Jaeger agent software. After all, the only thing more difficult to instrument than an application before it is deployed is, arguably, one that is already running in a production environment.