devopsy.kinsta.cloud

Category: Features

  • CNCF Takes LitmusChaos Platform to the Incubation Level

    CNCF Takes LitmusChaos Platform to the Incubation Level

    The technical oversight committee (TOC) for the Cloud Native Computing Foundation (CNCF) announced today it is elevating the open source LitmusChaos application testing platform to the incubation level.

    LitmusChaos is a chaos engineering platform donated to the CNCF by ChaosNative in 2020. Since then, it has seen adoption within production environments by more than 25 organizations, including Intuit, Lenskart, Orange, Red Hat and VMware.

    ChaosNative CEO Uma Mukkara says LitmusChaos was developed to provide DevOps teams with a chaos engineering platform built using containers that makes it easier to scale up and down than existing monolithic alternatives. The platform itself, however, can be used to test both monolithic and microservices-based applications.

    The platform consists of a Chaos Operator based on a software development kit (SDK); a ChaosHub for hosting and sharing experiments; Litmus Workflows that declaratively chains experiments (either in sequence or parallel) to build a chaos scenario; a centralized control plane, dubbed ChaosCenter, to design, schedule and monitor Litmus Workflows; Litmus Probes for creating chaos scenarios that automate steady-state validation and remediation actions and a Chaos Observability tool to export Prometheus metrics.

    In total, there are now more than 400 contributors to the LitmusChaos project and more than 4,000 pull requests have been made. Mukkara said since the beginning of this year, Litmus operator installations have grown to more than 2,000 per day. Going forward, the goal for 2022 is to significantly increase the number of LitmusChaos integrations there are for a wider range of DevOps platforms, added Mukkara. Additional sets of experiments for Kubernetes and non-Kubernetes targets, improved observability and integration with other platforms—via the open source OpenTelemetry agent software also being advanced by the CNCF—is also planned.

    In general, it’s still early days as far as the adoption of chaos engineering with DevOps workflows is concerned. However, Mukkara said that as more organizations embrace observability to better understand IT processes, the number of organizations that also use chaos engineering to test IT resiliency should also increase. In fact, as IT environments become more complex—thanks mainly to the rise of microservices-based applications—chaos engineering approaches to testing may someday be required, noted Mukkara.

    In the meantime, there are still a lot of IT professionals that are uncomfortable with chaos engineering approaches to testing that deliberately break underlying services to test the resiliency of an application. The core idea, however, is that no application should have a single point of failure that could lead to it becoming unavailable. In theory, a microservices-based application won’t fail because when a service becomes unavailable calls are automatically rerouted to another service. The performance of that application will degrade, but it won’t go offline. The challenge is that building an application that achieves that level of resiliency using microservices requires a significant amount of DevOps expertise.

    Regardless of how an application is constructed, the one thing that is clear is how dependent organizations are on applications to drive revenue. As such, the tolerance for application failures—or performance degradation—has never been lower.

  • GitLab Revamps Observability Strategy After Acquiring Opstrace

    GitLab Revamps Observability Strategy After Acquiring Opstrace

    GitLab plans to make observability a more integrated element of its continuous integration/continuous delivery (CI/CD) platform after acquiring Opstrace.

    Opstrace’s open source project exposes a horizontally scalable application programming interface (API) created by Cortex for Prometheus, an open source monitoring platform being advanced under the auspices of the Cloud Native Computing Foundation (CNCF). It then makes use of a Loki API to collect log data.

    DevOps teams can also point an existing instance of Prometheus to Opstrace or any instance of Fluentd or Promtail software they are using to collect log data, with support for a wide variety of other APIs, such as the Datadog agent, planned. A command line interface (CLI) is provided to set an instance of Opstrace that creates a controller through which the collecting of log data is managed.

    Kenny Johnston, senior director for product management for GitLab, said as part of an ongoing effort to make observability more accessible, GitLab plans to integrate Opstrace by default with GitLab Monitor for organizations that use either the self-managed version of GitLab or the software-as-a-service (SaaS) platform.

    Opstrace currently collects logs and metrics and support for distributed traces is planned. The goal is to extend the overall scope of observability capabilities that GitLab already provides within its CI/CD platform, noted Johnston. The goal is to make it simpler for DevOps teams to employ observability capabilities and interrogate data to optimize workflows without acquiring a separate observability platform, he added.

    GitLab is betting that, as Prometheus continues to gain traction, more DevOps teams will prefer to extend its reach into the realm of observability rather than having to manage a separate platform.

    Although observability has always been a core tenet of DevOps best practices, achieving it has proven to be a challenge. At best, most organizations can continuously monitor IT environments based on predefined metrics that identify when a specific platform or application is performing within expectations. Observability combines metrics, logs and traces—a specialized form of logging—to instrument applications in a way that makes it simpler to troubleshot issues without having to rely solely on a limited set of pre-defined metrics that have been created to monitor a specific process or function.

    The rate at which DevOps teams will embrace observability will naturally vary. In 2022, the biggest obstacle might not be the technology as much as it is simply understanding what queries can help DevOps teams better understand the root cause of an IT issue before there is a major disruption. In the long term, it’s expected that machine learning algorithms will leverage the data collected by observability platforms to automatically identify issues that might lead to a disruption long before it actually occurs.

    One way or another, the ability to discern events even within the most complex IT environment is steadily improving. Whether that’s accomplished as an extension of monitoring or by embracing true observability may not be all that relevant—as long as the ultimate goal is achieved.

  • Intel Expands Scope of OneAPI Toolkit Initiative

    Intel Expands Scope of OneAPI Toolkit Initiative

    Intel has made available a 2022 edition of its oneAPI toolkit through which it makes processor extensions accessible to DevOps teams.

    James Reinders, chief evangelist for Intel software products, said the goal is to make it simpler for DevOps teams to expose extensions to any company’s processors in the runtimes developers use.

    At its core, the oneAPI toolkit is a collection of APIs that Intel created for x86, graphics processing units (GPUs), application-specific integrated chips (ASICs), field programmable gate arrays (FPGAs) and other classes of processors.

    Reinders noted that developers have never had as much ready access to so many different processor classes before. The challenge organizations face, however, is making it simpler for developers to take advantage of those capabilities in a way that doesn’t require them to write low-level code, he added.

    The oneAPI 2022 toolkit adds additional capabilities including the world’s first unified compiler implementing C++, SYCL and Fortran, data-parallel Python for CPUs and GPUs, advanced accelerator performance modeling and tuning and performance acceleration for artificial intelligence (AI) and ray tracing visualization workloads. Collectively, Intel said it has made 900 new and enhanced features available via the oneAPI toolkit in the last year, including deeper Microsoft Visual Studio Code integration, support for Microsoft Visual Studio 2022 and Microsoft WSL2 for Linux development on Windows.

    As IT continues to evolve it will become more routine for developers to invoke heterogeneous classes of processors within the same application, said Reinders. Those capabilities will drive an explosion of software development innovation through the rest of the decade and beyond, he added.

    Intel is especially focused on making it easier for developers to invoke inference engines that take advantage of processor extensions to run AI workloads, Reinders said. Going forward, most applications will take advantage of machine learning algorithms that run inference engines optimized by Intel via extensions to its processors. As those workloads are constructed, integrating AI models within DevOps workflows will become the next major challenge, noted Reinders.

    It’s too early to say how simplified access to heterogeneous processors, also known as XPUs, might change the application development landscape. The one thing that is clear is IT organizations now have more options than ever. The issue now is finding a way to invoke all those capabilities in a way that doesn’t overwhelm developers. DevOps teams will play a key role in deciding which processors and associated extensions to make available to their development teams.

    In the meantime, the battle for processor dominance will only intensify in the months ahead. DevOps teams will need to decide how comfortable they are using processors manufactured by a single company like Intel or whether they want to employ a mix of processors provided by multiple vendors. Regardless of what they decide, it appears APIs will play a critical role in making it easier to benefit from advances in processor technologies.

  • Theranos’s Holmes VC Reverb | 5G C-Band Marches On | Mozilla Jumps Shark

    Theranos’s Holmes VC Reverb | 5G C-Band Marches On | Mozilla Jumps Shark

    Welcome to The Long View—where we peruse the news of the week and strip it to the essentials. Let’s work out what really matters.

    This week: The conviction of Theranos CEO Elizabeth Holmes leads to greater scrutiny of startups, higher-frequency 5G NR is creeping towards reality, and Mozilla gets criticized for promoting climate-harming scams.

    1. Holmes Case Causes Valley Rethink?

    First up this week: Elizabeth Holmes has been found guilty of defrauding Theranos investors. But how will the affair change Silicon Valley’s startup-and-funding culture?

    Analysis: Don’t fake it—just make it.

    This could be a seminal moment, causing investors to look more closely at startup founders and their outlandish promises. On the other hand, key Theranos investors—e.g., Murdoch, Cox, DeVoss—hardly fit the mold of Silicon Valley VCs.

    David Streitfeld: The Epic Rise and Fall

    F. Scott Fitzgerald’s creation … Jay Gatsby … was a bootlegger [and] sold fake bonds. Ms. Holmes chose Silicon Valley, the last and greatest of all human dreams. [She] was a natural salesperson, as good at bending reality as Steve Jobs himself.
    …
    She and her deputy and boyfriend, Ramesh Balwani … thought they could brazen it out in the Silicon Valley tradition until they had something that actually worked. [But] the verdict signaled the end of an era. In Silicon Valley, where the line between talk and achievement is often vague, there is finally a limit. … It is an epic rise and fall that will be chewed over in the coffee shops and juice bars of Palo Alto, Calif., until the tech industry departs [for] off-world colonies.
    …
    So where does this conviction leave … her investors and former fans? Ripe for the next huckster that comes along, probably. Some Silicon Valley promises are so sweet we just can’t get enough of them. Immortality. Crypto. Flying cars. Mars. Digital harmony. Wealth beyond compare.


    Think different, thinks DeanonymizedCoward:

    [There’s a] fine line between bombastic marketing spam, “fake it ’til you make it” pitches, and outright fraud. … The first is repugnant and sometimes unethical, but generally legal.
    …
    [But] If your product requires bull**** to get funded—even if it ultimately works—you’re a fraud. … Regulatory reforms … and a sort of corporate death penalty … can fix a lot of this by making it less financially practical to engage in ****ty behavior.


    Who else is to blame? Ask DS999:

    I am sure some of her investors didn’t really care whether it worked. They just wanted to get in early, ride the hype train up, and sell before actual results mattered.

    2. FAA and 5G Carriers Kiss and Make Up

    Higher-frequency 5G NR is creeping towards reality in the U.S., thanks to the FAA backing down. After the aviation regulator had vetoed C-band deployment—based on precious little evidence—it looks like as much as 280 MHz of extra spectrum is now ready to switch on.

    Analysis: Nomadic DevOps gets more bandwidth

    While band 77 is nowhere near the much-vaunted “FR2” extremely high frequency bands, this is a useful chunk of extra spectrum for mobile and nomadic apps. Next stop: ubiquitous Ka-band.

    Jon Brodkin: FAA agrees not to seek any more 5G delays

    The Federal Aviation Administration tentatively agreed not to seek any more 5G delays from AT&T and Verizon, potentially ending a battle over the aviation industry’s unproven claim that 5G … will interfere with airplane altimeters. [Their] C-Band spectrum licenses are for the frequencies from 3.7 GHz to 3.98 GHz.
    …
    The deal incorporates voluntary commitments that AT&T and Verizon previously made, including “C-Band radio exclusion zones” around airports for six months. [They] modeled the exclusion zones after those used in France, which are 910×2100 meters.
    …
    Assuming there are no further problems, AT&T and Verizon would be able to use their spectrum licenses without extra restrictions after July 5. … President Biden applauded the agreement … calling it “a significant step in the right direction.”


    In summary? Here’s Dieter Bohn:

    The agency in charge of airplanes is mad about wireless things. But the agency in charge of wireless things and the companies that make them lost a game of chicken.
    …
    GREAT JOB AMERICA. … I always said I wanted Internet Service Providers to be Dumb Pipes, but I never expected them to be this dumb.


    Has anyone read the FAA’s detailed report? bburdge has:

    I can see why the FAA is backing down here now, it only makes sense when your key report could be paraphrased as: “We got a couple grad students to run some basic model comparisons … using altimeter data from the aviation industry (with a proven track record of lying to us) and then went golfing for eight months before publishing … but didn’t bother … to actually test anything.”

    3. Mozilla Forgets its Values

    Firefox custodian, the Mozilla Foundation, has angered many by promoting a “grossly irresponsible” “Ponzi scheme” that harms the environment. What is this awful thing? I hear you ask. It’s cryptocurrency.

    Analysis: Imaginary money is bad—is this the last straw for Mozilla?

    Less than a year ago, Mozilla said it wants to be carbon-neutral, but it’s already reneged on that promise: Distributed proof-of-work systems require … well, work. And that work uses ridiculous amounts of energy per DeFi transaction. Electricity production is a zero-sum game, so cryptocurrencies and NFTs cause climate change by emitting ridiculous amounts of CO₂e.

    Liam Proven: Mozilla founder blasts browser maker

    A few days ago, Mozilla Foundation invited netizens on Twitter to send in cryptocurrency donations. … This move by the Firefox browser maker rapidly drew criticism.
    …
    As recently as September, Moz’s web-standards lead Tantek Çelik slammed cryptocurrencies as being unsustainable, stating: “[They] are harmful for sustainability. … Their energy requirements grow without any discernible upper bound, which is grossly irresponsible given the global environmental crisis.”
    …
    Mozilla has accepted crypto-coin donations for years, perhaps as part of its long-ongoing efforts to alienate its own users: It’s only the payment provider that changed.


    Chief among critics was Jamie “jwz” Zawinski:

    I founded Mozilla and I’m here to say **** you and **** this. Everyone involved in the project should be witheringly ashamed of this decision to partner with planet-incinerating Ponzi grifters.
    …
    Stop saying you were promised flying cars. Unless you were born in 1935, you weren’t promised flying cars, you were promised a cyberpunk corporate dystopia.


    It’s a shark-jump moment thinks Jeremy Keith:

    The most convincing argument I’ve heard yet for switching away from Firefox as my main browser comes from Mozilla. I wasn’t expecting that plot twist.

    The Moral of the Story: What is safety to one is not always safety to another.


    You have been reading The Long View by Richi Jennings. You can contact him at @RiCHi or tlv@richi.uk.

    Image: Benjamin Hayward (via Unsplash)

  • DevSecOps Will Cross the Chasm in 2022

    DevSecOps Will Cross the Chasm in 2022

    We’ve been talking about DevSecOps and shift-left security for years. Although this approach probably didn’t “cross the chasm” in 2021, we did see some very telling milestones. Cybersecurity VC funding surged to record heights with a focus on DevOps and cloud security and the need for DevSecOps became glaringly evident with the Log4j vulnerability. On a more personal note, our own open source security project Checkov has surpassed three million downloads.

    Much of this momentum was inevitable. Those in infrastructure and reliability circles have already been adopting and touting the benefits of shifting left and automating as much as possible, so in many ways, it’s expected that security would follow suit. Undoubtedly, the pandemic has also accelerated the need for developers to work more autonomously without having security personnel and processes acting as barriers to productivity and velocity.

    We’re confident that 2022 will see the silos between development and security teams continue to crumble as developer-led security practices become the norm for cloud-native organizations.

    DevSecOps finally crossing the chasm doesn’t mean that every enterprise and traditional organization will shift security left and adopt security best practices. It will, however, give them competitive advantages over those who don’t—both in decreased security costs and increased developer productivity and, thus, time-to-market.

    What else is in store for the future of DevSecOps?

    Rise of the DevSecOps Job Title

    Fewer than 5,000 people on LinkedIn currently have “DevSecOps” in their job title, yet there are over 20,000 current openings for DevSecOps roles. In 2022, expect to see more of those positions filled. This means security teams across industries will conduct fewer manual security audits, there will be a considerable influx of homegrown DevSecOps tooling and point solutions will likely begin to consolidate into single platforms. 

    Blurred Lines Between Application and Infrastructure Security

    Until recently, application security was a very well-defined (albeit fractured) space focusing on securing the custom code and open source packages that make up applications. However, with the rapid adoption of cloud-native applications, the lines between application and infrastructure security are blurring. We expect to see this trend continue as more engineers take on more infrastructure-related projects, vendors start catering to use cases outside of their core competencies (through acquisitions and in-house development) and the role of DevSecOps continues to expand within organizations.

    Infrastructure-as-Code: The Great Cloud Migration’s Next Chapter

    We’ve been talking about the great migration to the cloud for years. At this point, many companies are ready to move on to the next chapter: Infrastructure-as-code (IaC). As a result of this trend, DevSecOps will become much more important, as security needs to be baked earlier in development phases or risk being left behind. In addition, security teams will need to become more well-versed in development technologies and practices to provide the proper guidance for the new way applications are built and deployed. 

    More Software Supply Chain Attacks

    Hackers have been targeting retailers and security vendors for years, exploiting a minor weakness to gain access and move laterally into sensitive data. In the past year, however, software supply chain attacks stepped into the spotlight due to multiple supply chain attacks. We expect this trend to continue, which will, in turn, put more focus on securing supply chains. DevSecOps, which has focused mostly on tools and practices for securing the code and infrastructure, will expand to include the supply chain mechanism. 

    If 2021 was the year of hype for DevSecOps, we believe (and hope) that these best practices are embraced en masse across industries. The benefits of shift-left security are well documented: The number of high severity incidents is significantly reduced, the potential attack surface is minimized, compliance efforts are simplified and the time to remediation is lowered. Organizations also save money by catching misconfigurations and vulnerabilities earlier in the software development life cycle while at the same time gaining time back with tools, both open source and commercial, that are empowering developers to move fast and build applications that are more secure and reliable.

  • LogDNA Survey Surfaces Observability ROI Concerns

    LogDNA Survey Surfaces Observability ROI Concerns

    A Harris Poll of 200 senior engineering professionals suggests it may be a while before most DevOps teams are able to achieve true observability in their IT environments.

    The survey was conducted on behalf of LogDNA, a provider of an observability platform, and found nearly three-quarters of respondents (74%) are struggling to achieve their observability goals despite investing hundreds of thousands of dollars. In fact, well over a third (38%) said they are already investing $300,000 annually on observability tools.

    Despite those investments, less than half of respondents are very satisfied with their ability to use log data. Key areas of frustration include the challenges of collaboration with colleagues spanning multiple teams (67%), the fact that the tools are not easy to use (66%) and the routing of security events (58%).

    Nevertheless, the bulk of survey respondents said they are optimistic about observability. A full 85% of participants said they believe true observability is possible.

    LogDNA CEO Tucker Callaway said the core issue that many organizations fail to recognize is that observability is really a data literacy and management challenge. Organizations are collecting logs, metrics and traces from a wide range of tools and platforms without thinking through optimal data pipeline construction. The result is an aggregation of data that has not been normalized to simplify surfacing actionable intelligence, he said. In fact, many organizations quickly discover that all they’ve really accomplished is increase their data storage costs, said Callaway.

    In the absence of that data normalization, Callaway noted it becomes exceedingly difficult for cross-functional teams to query all the data being collected within a repository.

    At its core, observability promised to make it easier to identify IT issues before they cause a disruption. Observability in one form or another has always been a core tenet of any DevOps best practice. Initially, DevOps teams focused on continuous monitoring as the most effective way to proactively manage application environments. However, it can still take days, sometimes weeks, to discover the root cause of an issue. Observability platforms make it possible to correlate events in a way that makes it easier for analytics tools to identify anomalous behavior that might be indicative of the root cause of an IT issue.

    In contrast, traditional IT monitoring focuses on predefined metrics to identify when a specific platform or application is performing within expectations. Observability platforms combine metrics, logs and traces—a specialized form of logging—to make it easier for IT professionals to interrogate data being generated by a wide range of DevOps tools and platforms.

    Of course, the assumption is that DevOps teams actually know what queries to launch to determine what the root cause of an IT issue might be. In the longer term, machine learning algorithms should make it easier to identify anomalies within IT environments. In the meantime, however, achieving observability still requires a fair amount of DevOps expertise that is not always readily available.

    Eventually, observability will become much more automated than it generally is today. As that transition occurs, IT teams should find the ROI of observability is steadily improving.

  • Predict 2022: Back to the Future

    Predict 2022: Back to the Future

    After a year of innovation, rapid digital transformation efforts, new challenges and settling into a new normal, it is time to look back at all the lessons learned and look forward to the future. What does 2022 has in store for us? Will 2022 be a breakthrough year in terms of progress and opportunity? At Predict 2022 Virtual Summit, taking place Jan. 19-20, top industry leaders will share their speculations on what the future holds in the realms of DevOps, cybersecurity, digital transformation, cloud-native and IT.

    Predict 2022 is a free virtual event featuring a conference lobby, presentation theater, expo hall and more. This event is a great opportunity for networking and learning about the trends, technologies and looming topics that could have the most impact on technology in this new year.

    Predict 2022 features a stellar lineup of industry experts, including:

    • Tracy Ragan, CEO and co-founder at DeployHub, CDF board member
    • Mark Miller, senior storyteller at JupiterOne
    • Mike Jones, H4unt3d Hacker Podcast, content creator at Knight Ink
    • Caroline Wong, chief strategy officer at Cobalt
    • John Willis, senior director of the Global Transformation Office at Red Hat
    • Donovan Farrow, CEO at Alias
    • Nicole Dove, CIA, CRMA, business information security officer at WarnerMedia
    • Niladri Choudhuri, founder and CEO at Xellentro Consulting Services LLP
    • Emmet B. Keefe III, operating partner and founder at Insight Partners
    • Matthew Guarini, VP, senior research director at Forrester
    • Mike Vizard, chief content officer, GM/editorial director at Techstrong Group/Digital CxO
    • Mitch Ashley, CTO at Techstrong Group and principal at Techstrong Research
    • Ori Inbar, CEO at Augmented World Expo
    • Thomas Krane, principal at Insight Partners
    • Ali Khan, partner, US Consulting Solutions at PwC
    • Jim Walker, principal product evangelist at Cockroach Labs
    • Keri Pearlson, executive director at MIT (CAMS Research Consortium)
    • Russ Finney, lecturer and director of MIS Undergraduate Programs at The University of Texas at Austin McCombs School of Business
    • Marnie (Huss) Wilking, CISO, advisor, board member at Wayfair
    • Patricia Titus, chief privacy and information security officer at Markel Corporation
    • Rob Zuber, CTO at CircleCI
    • Steve Chin, author and VP of developer relations at JFrog
    • Whurley, founder and CEO at Strangeworks, Inc.
    • Chris Hughes, co-founder and CISO at Aquia
    • Tracy Bannon, senior principal, software architect and DevOps advisor at MITRE Corporation
    • Christina M. Rhylander, acquisition and policy specialist at MITRE Corporation
    • Daniel Elkabes, vulnerability research team leader at WhiteSource
    • Vanessa Pegueros, board member and cybersecurity leader at OneLogin
    • Alan Shimel, founder and CEO at Techstrong Group
    • Charlene O’Hanlon, CCO at Techstrong Group

    Attendees will be able to interact with sponsors and peers, download information and participate in raffles, giveaways and other interactive activities.

    MediaOps CEO and DevOps.com Editor-in-Chief Alan Shimel will also be announcing the 2021 DevOps Dozen² Awards winners during the event.

    To check out the full agenda and to register, please visit the Predict 2022 website.

  • New Relic: Observability Tops 2022 DevOps Agenda

    New Relic: Observability Tops 2022 DevOps Agenda

    If 2021 was the year when IT organizations became aware of the need for observability, then the coming year should see observability aspirations realized.

    A report published by observability platform provider New Relic postulates that there are no less than 11 factors converging in a way that will make observability an absolute requirement for organizations.

    At the top of that list is a simple fact: Observability is now required to drive digital business transformation initiatives that typically span multiple processes. In fact, a global survey of 1,300 IT leaders, software engineers and developers conducted by New Relic found 90% of respondents view observability as critical to the business.

    Second, as a result, observability is now starting to be applied in both pre- and post-production environments as organizations take a more holistic approach to managing the software development life cycle, the report noted.

    Third, the report suggested that IT teams are looking to make better-informed decisions based more on facts than intuition. To do this, teams are using data collected from across the enterprise application environment. In fact, the fourth factor advancing observability in the enterprise is that the cost of instrumenting application environments is dropping, which the report notes is making it easier to collect relevant data.

    The fifth and sixth factors reflect a desire to rationalize the number of monitoring tools that IT teams currently use and the fact that usage-based pricing has simultaneously made it less expensive to leverage observability platforms as needed.

    The seventh factor is the overall shift toward open source software that enables observability platform providers to collectively leverage innovations faster. The eighth factor, closely linked to the open source trend, is the rise of more complex open source platforms such as Kubernetes that require much higher visibility to successfully deploy and manage.

    The ninth trend identified in the report is increased adoption of DevOps best practices that encourage IT teams to view observability as a core tenet of any IT management strategy. The tenth factor cited by the report is the need to drive increased collaboration among IT professionals.

    Finally, the eleventh factor driving further adoption of observability is the need for increased application reliability as more organizations appreciate their dependence on software to deliver essential services.

    Buddy Brewer, a group vice president and general manager for New Relic, said now that observability platforms are becoming more accessible, the next big challenge will be teaching IT professionals what questions to ask as they troubleshoot both legacy monolithic applications and a wide range of emerging microservices-based applications. The results of those queries then need to be easily shared with both developers and internal IT operations teams that must work together to maximize application performance and availability, he added. The latter requirement is why New Relic decided to acquire CodeStream to make telemetry data available with integrated development environments (IDEs), noted Brewer.

    There is, of course, no shortage of observability platforms for IT teams to adopt. In fact, the biggest challenge now is getting every application stakeholder to agree to standardize on an observability platform—no small feat, as many DevOps teams may already have a preference for one approach over another.

  • Best of 2021 – From Agile to DevOps to DevSecOps: The Next Evolution

    Best of 2021 – From Agile to DevOps to DevSecOps: The Next Evolution

    As we close out 2021, we at DevOps.com wanted to highlight the most popular articles of the year. Following is the ninth in our series of the Best of 2021.

    As the workforce shifts to remote locations, networks are becoming more diffuse as the edge gets farther away. At the same time, the number of devices and applications on those networks continues to rise. Some estimates suggest that, by 2030, there will be 15 connected devices for every person on the planet.

    The rapid expansion of the remote workforce and their associated application needs has created a significant challenge for both application developers and the IT professionals tasked with deployment and maintenance of applications. Finding the right balance between speed of release and stability and reliability of applications – never an easy task – is paramount for the success of any organization.

    Agile and DevOps evolved to help organizations strike this balance, providing the best possible customer experience in the shortest realistic time frame. But have Agile and DevOps fulfilled that promise?

    The Promise of the Agile Manifesto

    While Agile as a methodology has been known and followed for nearly two decades, the ways in which organizations have applied Agile have been as varied as the organizations themselves.

    The Agile manifesto set out four basic values when it was first written in 2001:

    • Individuals and interactions take precedence over processes and tools.
    • Working software takes precedence over comprehensive documentation.
    • Customer collaboration takes precedence over contract negotiation.
    • Response to change takes precedence over mindlessly following plans.

    Agile processes are applied across a range of development activities, from development of standalone applications to development of cloud-native business applications. The Agile Manifesto also contains several more principles that further explain what the values mean when put in practice.

    These Agile principles can be condensed into a few areas of focus:

    Prioritize the customer over the product

    Agile development requires continuous customer engagement throughout the entire development process, with the end goal being frequent product delivery based on specific customer feedback. While this can present challenges for an Agile team, particularly when customers demand changes late in the development cycle, Agile maintains that the best way to elevate customer experience is to make the customer part of the development process.

    Trust Agile Teams to Self-Organize and Self-Manage

    Agile focuses on smaller teams that remain in constant communication with customers, as well as with each other. These teams should include both developers and those with knowledge of the larger business requirements being met through development. For these teams to work effectively, trust is key. Agile posits that self-organized teams, given leeway to pursue their projects in the way they see fit, result in the shortest development cycles.

    Keep it Simple

    Agile teams require simplicity. This correlates with Agile’s disdain for overuse of process. Agile seeks to minimize the total amount of work required, thus minimizing overall development time.

    Agility Leads to Agility

    Just as Agile teams continuously improve their product and the customer experience, they also constantly improve themselves. Constant evolution of teams and the processes they employ (remember, teams before processes) improves team agility itself.

    While the Agile values and principles seem simple, effective application of the values and principles is another story entirely. Indeed, at times, the implementation of Agile development has violated the very first value of the manifesto by focusing on which tools and processes should be used. But, in an environment where more users are clamoring for more products and services, implementation of Agile methodologies is not just important, it is imperative.

    Building on Agile with DevOps

    The precise meaning of the term itself and the scope of DevOps remain a source of debate. At a high level, DevOps is the marriage of the development cycle with post-development activities, such as deployment and maintenance.

    According to open source contributor Barbara Ericson of Cloud Defense, “Instead of IT operations and software development being siloed off from each other, DevOps breaks down the traditional boundaries that previously existed between them in order to achieve continuous integration and continuous delivery (CI/CD) of quality software features and applications to end users.”

    Just as with Agile, the effective implementation of DevOps requires cultivating a DevOps culture within the organization. However, Agile and DevOps are also not mutually exclusive. That is to say, DevOps is not merely a replacement for or an evolution of Agile. Although Agile and DevOps differ in their structures and approaches, they can work in unison to achieve the ultimate goals of speedy deployment and enhanced customer experience.

    While Agile and DevOps share common goals, they have not always agreed on how to achieve those goals. DevOps differs in many respects from Agile, but, at its best, DevOps applies Agile methodologies, along with lean manufacturing principles, to speed up software deployment.

    One area of particular tension between Agile and DevOps is that the latter relies heavily on tools; in particular, when it comes to the automation of testing and deployment processes. But DevOps can overcome the resistance of Agile developers to tool usage simply by applying Agile principles themselves.

    Effectively, DevOps proponents must convince Agile teams that dogmatic adherence to the underlying principles of Agile is actually inconsistent with Agile in the first place. Ironically, Agile developers who insist that the process is always bad actually violate Agile principles by refusing to acknowledge the benefits offered through change, another Agile principle.

    The challenge is to have the Agile development teams trust in the automation efforts of DevOps, while at the same time encouraging the DevOps team to consider the business goals of deployment rather than pursuing speed of deployment for its own sake.

    With constant communication between the Agile team and DevOps team (another Agile principle), development can achieve a degree of comfort with DevOps tasks and processes. This means that testing and deployment automation can proceed quickly, resulting in a decreased time to market, overall.

    Ultimately, Agile and DevOps can work together most effectively when they recall their shared goals: smaller change increments based on feedback resulting in faster delivery cycles.

    Continued Evolution: DevSecOps

    As networks expand and businesses continue to integrate cloud-based services (in many cases, building multi-cloud environments), security issues become more and more complex. One area in which Agile is lacking is in integrating security into the development process. Unfortunately, in an Agile environment, application security is often an issue addressed after development, rather than as part of the process. Enter the next iteration of DevOps: DevSecOps.

    DevSecOps revolves around three basic principles: ensuring data security while minimizing inconvenience for users in accessing data, using development tools that identify risks as early as possible in the development process and ensuring data encryption using technologies such as secure sockets layer (SSL) and virtual private networks (VPNs).

    While DevSecOps and Agile may also disagree about the priority of tools, security must become an integral part of any development project. This is why Agile, DevOps and DevSecOps must all work together to ensure not only rapid deployment but secure deployment.

    The Next Evolution

    Agile and DevOps were both important evolutions of the software development process, but DevSecOps is the next evolutionary step. Despite a few disparate characteristics, Agile, DevOps and DevSecOps can and should work in unison to achieve their shared goal: the best possible customer experience and short deployment cycles.

  • Best of 2021 – 11 Open Source DevOps Tools We Love For 2021

    Best of 2021 – 11 Open Source DevOps Tools We Love For 2021

    As we close out 2021, we at DevOps.com wanted to highlight the most popular articles of the year. Following is the eighth in our series of the Best of 2021.

    DevOps isn’t just a cultural shift — it requires great tools to come to fruition. Below, we’ve pulled together a list of some of the most well-loved DevOps tools available today. But, throwing loads of money into fancy SaaS solutions can quickly gobble up the cloud budget. These DevOps tools all are open source, and enable everything from container builds and orchestration to microservices networking, configuration management, CI/CD automation, full-stack monitoring and more. Here are some of our favorite open source DevOps tools for 2021.

    1. Kubernetes

    With the ubiquity of microservices and container-based software, it’s no surprise that Kubernetes tops this year’s list of open source DevOps tools. Kubernetes, adoption of which rose by 48% in 2020, is used to orchestrate containers. Instead of releasing microservices manually, Kubernetes can automate deployment, maintenance and scaling of groups of containers in production. Kubernetes, sometimes written as K8s, is hosted by the Cloud-Native Computing Foundation (CNCF).

    2. Docker

    Docker, the software, is a free and open source platform used to build, ship and run an application as a lightweight container. Containers package up the binaries, libraries, configuration files and dependencies required for a program to run. Over the past decade, containers have played a key role in agile development, and Docker containers led the revolution. At its core is the Docker Engine. Docker Hub is also an excellent resource for finding and sharing prepackaged functions as containers. Also, to plug container vulnerabilities, it may be helpful to use open source container auditing tools like Docker Bench or Anchore.

    3. Istio

    Microservices are a handy development style, yet they bring new development and architecture concerns. Namely, how do we apply networking policies like security, encryption, observability and telemetry elements consistently across all our services? Well, service mesh is one answer. Service mesh places a sidecar proxy next to each container and abstracts these networking capabilities to a control plane. Istio is one such open source service mesh that has seen widespread adoption. Istio is built on Envoy, opening it up to plugins and extensibility options. We should also mention Linkerd and Kuma as viable open source service mesh alternatives.

    4. GitHub Actions

    GitHub is arguably the most popular source control and software collaboration platform on the planet. The GitHub platform itself, based on Git, has seen some significant updates in the past few years. Most notable is the GitHub Actions capability. GitHub Actions enable software packages hosted on GitHub to accept inputs and trigger other processes. This could help automate some cool DevOps workflows within GitHub, such as code reviews, branch management or CI/CD processes — the combinations of possibilities here are endless. GitHub Actions are, essentially, YAML files hosted in GitHub repositories that leverage GitHub webhooks. Though this is more of a feature than an open-source tool, we feel it’s important to include here. Actions is free for public repositories with a limit of 100 actions.

    5. Jenkins

    A big part of the DevOps philosophy is finding ways to automate and deploy new iterations more efficiently. Part of this goal is creating a streamlined continuous integration and continuous delivery (CI/CD) pipeline. Jenkins is an open source automation server with hundreds of plugins to automate the building, deployment and testing of software projects. Although GitHub Actions could theoretically replace a CI server in the future, CI tools like Jenkins, CircleCI, TravisCI and GitLab Community Edition still are preferred by many DevOps teams.

    6. Prometheus

    Metrics and alerting systems are crucial for site reliability engineers to visualize applications and react to issues. Prometheus, a graduated CNCF project, is a well-loved open source monitoring solution. A Prometheus server collects time-series metrics by scraping HTTP endpoints and generates a system to interact with this data, offering deep querying, visualization, storage and other capabilities. Check out this Awesome Prometheus list for Prometheus introductions and additional resources.

    7. Ansible

    Ansible is all about automation. Ansible, an open source project sponsored by Red Hat, can be used to automate things like cloud provisioning, networking, deployment, configuration management and other tasks. Ansible has a simple yet effective architecture that is relatively easy to assemble — all you need is a text editor and command line. You describe your infrastructure in a text document and organize your desired states in a playbook. For an example in practice, see how OpenIO uses Ansible. “Ansible is our standard tool not only to deploy the OpenIO core, but also our WebUI, OIO-FS and all upcoming options,” writes Cédric Delgehier, Ops at OpenIO.

    8. Chef

    Chef is another infrastructure-as-code (IaC) solution for automating your configuration management. Chef uses Ruby to automate server configurations and works well with all major cloud service providers (CSPs). This can be very useful when creating and provisioning large quantities of machines. Like other automation tools on this list, the user describes their components and their states in a declarative format. In Chef, these are known as “recipes,” which can be grouped together into “cookbooks.” You can’t diss Chef for not being on-theme!

    9. Terraform

    Terraform is another IaC tool that can be used for initiation of building, versioning and further automation using configuration files. “Terraform is a tool for building, changing and versioning infrastructure safely and efficiently,” as it is described on GitHub. Terraform follows an “execution plan” that a user creates with high-level syntax. One unique aspect of Terraform is its emphasis on versioning — this allows you to version the blueprint of your service just as you would your software.

    10. JAMStack

    As I’ve covered before, JAMStack combines JavaScript, APIs and markdown for constructing web-based applications. While more of a “headless development” methodology than a single open source tool, JAMStack projects are often built using open source components. For example, JAMStack often leverages open source headless content management systems such as Ghost, Strapi and/or Netlify CMS.

    11. ELK Stack

    ELK Stack is the union of three open source projects maintained by Elastic: Elasticsearch, Logstash and Kibana. With these three components, developers can take in and log data from any source and create helpful visualizations. This centralized logging is enabled with a NoSQL database for storage with Elasticsearch, processing and data collection with Logstash and visualization with Kibana. Increased visibility is vital for data analysis and helps identify errors to reduce mean time to recovery (MTTR).

    Rounding Out The DevOps Toolchain for 2021

    Programming isn’t only about shipping quality code, it’s about efficient execution. This drive to improve operations across the board is truly putting Ops into everything. With awesome open source DevOps tooling, it means more and more architects can adopt a DevOps approach within their deployment models.

    DevOps as a practice, as well as its underlying technology, is always evolving. In 2021, a lot of effort is being placed on wrangling the effects of introducing a microservices development style. We’ve noticed the maturation of container orchestrators and service mesh, as a result. Blueprinting infrastructure-as-code and creating automated, repeatable configurations is another must-have for implementing automated build and release pipelines. Not to mention, many no-code and low-code platforms are opening up DevOps capabilities to non-programmers, albeit from a proprietary perspective.

    It’s also helpful to note that some popular open source DevOps tools have been acquired — like Docker and Chef — blurring the lines between their business and their open source roots. When it comes to open source tools, it’s a good practice to adopt vendor-neutral tools with solid community support representing diverse stakeholders. This will help future-proof your project. Even though open source is “free,” users must ensure the benefits outweigh the time-to-onboard as well as the operational overhead of self-hosting.

    DevOps tools are beneficial to automate software deployment. Here, we’ve attempted to list some of the most exciting tools to help automate that process. Of course, many new tools are emerging in the DevOps space, and we have hardly scratched the surface. Do you have a favorite DevOps tool that’s not mentioned above? Let us know in the comments below!