devopsy.kinsta.cloud

Category: DevOps in the Cloud

  • Cosmonic Contributes WebAssembly Runtime to CNCF

    Cosmonic Contributes WebAssembly Runtime to CNCF

    Cosmonic today revealed it has donated its universal application runtime for building cloud-native applications using the WebAssembly (Wasm) development framework to the Cloud Native Computing Foundation (CNCF).

    The Cosmonic wasmCloud runtime makes it possible to run applications built using WebAssembly on any platform. In comparison, applications built using containers are not as portable because IT teams can’t take a container application developed for Linux and run it on Windows. The container technologies for each operating system are implemented differently.

    The CNCF will make wasmCloud available as a sandbox project and will invite other organizations to start making contributions as part of an effort to eventually put wasmCloud on par with other projects such as Kubernetes. There are already nearly 80 contributors to a wasmCloud runtime that was launched last November.

    Cosmonic

    The CNCF is trying to advance other WebAssembly technologies including Krustlet, a tool for Kubernetes environments that listens for event streams from new pods so that the scheduler can then assign workloads.

    However, the CNCF is now not the only organization looking to promote adoption of WebAssembly technologies. The Bytecode Alliance has launched WebAssembly (Wasm) and WebAssembly System Interface (WASI) initiatives as part of an effort to promote standards that would make it easier to run Wasm applications anywhere. Originally founded by Fastly, Intel, Mozilla and Microsoft, the Bytecode Alliance now also includes Arm, DFINITY Foundation, Embark Studios, Google, Shopify and University of California at San Diego.

    At its core, Wasm is a portable binary instruction format for building software that describes a memory-safe, sandboxed execution environment. That approach might one day replace the current predominant method for building software that relies on the aggregation of software components that tend to lack distinct security boundaries between them. As a result, it becomes relatively simple for malware to infect all the components of an application.

    Liam Randall, CEO of Cosmonic, said even though Wasm, in effect, provides another type of virtualization for applications, IT teams should not view it as a set of technologies that will one day obviate the need for containers. Applications developed using Wasm will be layered on top of various container platforms, he said.

    There is, for example, already a Wasm project that creates a server-side Wasm runtime that enables applications to be deployed on a secure container architecture that is more efficient than Docker containers.

    The World Wide Web Consortium (W3C) drove the development of Wasm as part of an effort to create a common format for browsers executing JavaScript code. Wasm is now being extended beyond browsers and JavaScript to enable developers to ultimately create a set of universal binaries that could work on any platform without modification.

    It may be a while before the fruits of those labors are realized. However, it’s apparent the walls that have separated one IT platform from another are starting to crumble. Once applications become much more portable, the reasons why one organization may choose one platform over another will have a lot less to do with what developers prefer.

  • Edge Computing: Tech’s Next Trillion-Dollar Opportunity 

    Edge Computing: Tech’s Next Trillion-Dollar Opportunity 

    Even as technology continues to move into the cloud, a simultaneous shift is already underway in the opposite direction as computing increasingly moves to the edge. One key reason: by 2025, the number of connected devices in use is expected to exceed 56 billion units, according to IDC. Edge computing is not a successor to the cloud, but another expansion of technology’s reach that represents an exceptional opportunity for investment and growth.

    As edge-device and computing adoption continues to skyrocket, organizations need to process the proliferating data from these devices locally so they can act on it in real-time. This will require investments in edge-adapted infrastructure and related tools and platforms. Enterprise technology is about to be transformed yet again — the need to process and analyze some data at the point where it is both created and consumed is becoming a business requirement as mandatory as moving other parts of the technology stack into central locations in the cloud.

    Organizations already must deliver compelling customer experiences with edge devices based on data that must be continuously refreshed. The alternative—uploading and downloading data to and from a cloud service or local data center—often takes too long. Whether the data is on a consumer’s smartphone or a manufacturing floor, organizations need to process and analyze data as near as possible to the point where a real-time data-driven response is required.

    The competition to provide these capabilities is already intense. Many vendors—ranging from cloud service providers to telecommunications carriers to longtime providers of server and storage infrastructure—view edge computing as their next trillion-dollar opportunity.

    Why We Compute at the Edge

    Edge computing has been around for decades in various forms. There are countless examples of standalone software being deployed on, for example, devices used on factory floors that automate various aspects of a manufacturing process. In recent years, many of those systems have been connected to the Internet to enable organizations to collect and share data more easily. The challenge now is that the amount of data being generated at the edge often exceeds the point where it is practical to transfer that volume of raw data across a wide area network (WAN). Platforms that can process and analyze large amounts of data at the network edge are required. The aggregated results of data processed on an edge platform can then be shared more efficiently with other applications running in the cloud, in an on-premises IT environment or even on the network itself.

    Low-latency requirements are today’s biggest challenge. In many scenarios, an application running on a gateway at the edge of a telecommunications network needs to be able to respond in a fraction of a second to a mobile computing application’s request for data. In other circumstances, an edge application is driving an automated factory operation that needs to dynamically adjust to analytics processed at the edge of an extended enterprise.

    There are even more advanced use cases, some of which are described below. These include in-car computers that process computer-vision data in real-time for driverless vehicles and robotics applications that promise to transform manufacturing.

    Hardware Advances Enable Edge Computing

    The often competing demands of low-latency network connections, high-power computing and large storage requirements are driving innovations in edge hardware.  Electric carmaker Tesla, for example, has added custom-made devices to its processors capable of pre-processing massive amounts of data, performing machine learning inference to make quick driving decisions and predictions and even do some deep learning model training on the vehicle itself–before sending a subset of the data to central systems to do even more training.

    Earlier this year, Tesla unveiled a processor with six billion transistors. According to the company, its device boosts performance by 21 times compared to the Nvidia GPU it replaced in its Tesla Model S, Model 3 and Model X models. These powerful devices are used in addition to CPUs from traditional providers such as AMD and Intel for workhorse applications like infotainment.
    With this computing power at the edge, a Tesla car can process sensor data that allow it to recognize and self-pilot around pedestrians, other cars on the road, emergency roadside signage and potentially hazardous moving objects. This sensor data is also processed with pre-loaded mapping data and GPS connectivity. Additionally, Tesla’s in-car computers relay information over Tesla’s network to massive storage facilities where the data is analyzed to improve Tesla vehicles’ self-driving and other functions. Improvements to the vehicle’s software stack are then downloaded to it over the network.

    Healthcare is an example of a sector with similar needs for advanced machine learning-taught devices and software. Images from MRI and other scanning devices with inference-computing capabilities deployed at numerous locations would also require computing power and network infrastructure similar to that which Tesla’s cars need. Most hospitals, however, don’t have the interest or wherewithal to build their own powerful edge computing hardware and will need to consume this capability from technology vendors.

    As advances in semiconductor circuits approach seven nanometers and smaller, the ability to run applications at the edge in an energy-efficient manner becomes more realistic for a wider range of organizations. With each new iteration of processors, the total cost of edge platforms will continue to decline while the amount of horsepower available to run applications steadily increases.

    Networking at the Edge

    Regardless of how much data is processed and analyzed at the edge, the sheer volume of data creates more demand for network bandwidth and, consequently, the potential for bottlenecks. One main bottleneck is at the backhaul—the stretch between the edge towers and central servers and network. Whether tomorrow’s edge network infrastructure will consist of 5G, more ultra-powerful WiFi connections and devices, low-latency daisy chain systems, fiber systems such as those that Google is developing, satellite connectivity, yet-to-be-created technology or a combination of these is still unclear–but one thing is certain: Data volumes are increasing faster than network bandwidth.

    Today, organizations have a choice of connecting edge platforms among several available wired and wireless network alternatives. The number of use cases involving edge computing is expanding in part because wireless 5G networks make it possible to share data across multiple interconnected edge platforms. For example, manufacturing employees wearing augmented reality headsets that can share data and analytics in near-real-time not only with their coworkers on the floor but with colleagues hundreds of miles away.

    As telecom carriers continue to virtualize the network infrastructure they employ to deliver 5G services, the cost of delivering them should steadily decline across a widening range of geographic areas. Today, a 4G network can support roughly 4,000 devices per square mile. A 5G network could pack up to three million devices in the same area.

    It’s still early days as far as 5G is concerned. A Capgemini survey of 1,000 industrial organizations that plan to incorporate 5G into their operations finds less than a third (30%) have progressed to trials and real-world implementations. Yet, carriers are already at work defining the next generation of 6G wireless standards that promise to deliver another giant leap for wireless networks in another decade.

    Regardless of the network employed, organizations will still need to strike a balance between how much data needs to be processed, stored and eventually disposed of at the edge versus the amount of aggregated data that needs to be transferred over a WAN and incorporated within various other platforms that process data centrally or function as systems of record.

    Factoring in Security

    Arguably, the biggest obstacle to edge computing is security. Each time an organization deploys an edge platform, the attack surface they need to defend grows. Many of these edge platforms are also managed by operational technology (OT) teams that don’t have as much cybersecurity expertise as does, say, the data center security team at Google. Most organizations are trying to meld their OT and traditional information technology (IT) teams in an effort to deploy, manage and secure edge platforms. However, the cultures of these two teams are significantly different. It takes time for organizations to harmonize the efforts of these two groups.

    More challenging still, cybersecurity expertise is hard to find and retain. Cybercriminals have discovered how vulnerable OT environments are. Attackers affiliated with various nation-states have already demonstrated that industrial control systems controlling electricity grids and other critical infrastructure systems are primary targets should war erupt. From their perspective, each edge computing platform connected to a network is yet another vehicle through which malware can be introduced into an IT environment. It only takes a few minutes for malware to propagate laterally across an entire platform.

    Ultimately, organizations will soon spend billions of dollars securing edge platforms. Once the edge systems are locked down and secure, edge processing can improve overall cyberdefense—processing more data at the edge without moving it around as much can dramatically reduce the attack surface.

    How the Edge Computing Ecosystem is Evolving

    Once the right level of infrastructure is in place, the next challenge is building applications. The widespread adoption of container technologies has allowed developers to build applications using artifacts that require much less memory and storage. Most container environments today, like those based on Docker and Kubernetes, are decentralized, in a sense, but are run in centralized environments like data centers. But work is progressing on even smaller container formats that make it even easier to deploy application software on, for example, an oil rig hundreds of miles from any server or gateway.

    Lighter-weight instances of the open source Kubernetes container orchestration engine are starting to gain traction. Kubernetes, by surfacing a consistent set of application programming interfaces (APIs) regardless of what platform it is deployed on, provides an opportunity to centralize the deployment and management of modern microservices-based applications across highly distributed computing environments.

    Industry API specifications and frameworks are also beginning to emerge for machine learning, data science and other technologies that empower edge computing. An example is oneAPI, supported by Intel and others, which provides a framework for developing edge applications.

    Providers of development platforms are also racing to create frameworks that make it simpler to build applications within edge architectures that seamlessly invoke backend application services running in the cloud. Distributed event-streaming platforms like Apache Airflow and Kafka can play a role in allowing edge computing platforms to share data with those backend platforms at scale. Some organizations will also invoke the services of content delivery networks (CDNs) to improve the performance of applications deployed across a global network of points-of-presence (PoPs) that have already been constructed.

    Finally, stateful applications deployed at the edge can require local databases that provide access to persistent forms of data. The bulk of container applications deployed today are stateless—they store data on an external storage system. Building stateful applications that access data in containers on a local Kubernetes cluster is more challenging.

    There are already millions of developers familiar with container platforms who are standing by until improved hardware and software simplifies and secures deployment of highly portable applications as far out on the edge as possible.

    The Coming AI and Data Management Challenge

    The most exciting edge innovations ahead will likely be driven by the rise of machine learning and AI. Today, AI models are typically trained within centrally located cloud computing platforms. An AI and machine learning inference engine can then be created and deployed in a production environment, where a trained AI engine deduces what actions to take as new data is discovered.

    Over time, those inference engines are subject to drift as the amount of new data collected exceeds the parameters on which the original AI model was based. Data science teams working in collaboration with IT and OT teams then need to train a new AI model to replace the inference engines at the edge. Advances in new AI and machine learning capabilities, in conjunction with lower-power and higher-performance processors and other components to help lower costs, can improve the accuracy and performance of prediction systems, but typically doesn’t reduce the degree to which this drift happens.

    If AI models can be at least partially retrained at the edge, this can help to reduce prediction accuracy drift. However the software and hardware infrastructure required to retrain an AI model efficiently within an edge platform is still very early-stage. That said, as the cost of increasingly high-powered hardware declines and edge machine learning software systems improve, the ability to train at the edge will become more feasible.

    The Economic Impact and Investment Opportunity

    The global edge computing market was valued at approximately $3.6 billion in 2020 by Markets and Markets. The research firm forecasts this market will reach $15.7 billion by 2025, representing an astounding 34.1% compound annual growth rate (CAGR).
    McKinsey forecasts that the economic value generated by use cases involving internet of things (IoT) applications alone will be somewhere between $3.9 trillion and $11.1 trillion per year by 2025. Add in all the other use cases for edge platforms and the potential for positive economic impact becomes enormous.

    Increased computing at the edge will be a huge technology market in its own right and will also drive increased demand for centralized cloud computing. Gartner notes that by year-end 2023, 20% of installed edge computing platforms will be delivered and managed by hyperscale cloud providers. The other 80% of this massive emerging market opportunity is up for grabs.

    Lonne Jaffe co-authored this article.

  • Infrastructure-as-Code Security will Improve Your Cloud Security

    Infrastructure-as-Code Security will Improve Your Cloud Security

    Moving to the cloud creates a tremendous opportunity to get security right and reduce the risk of a data breach. Most organizations start their cloud security efforts by ‘shifting security left’ thereby addressing issues in the CI/CD pipeline. Fixing known vulnerabilities and risky configurations before pushing images to production makes sense. But there is an opportunity to shift security even further left—as you set up your cloud infrastructure—to reduce risk. Infrastructure-as-code (IaC) security allows you to define security controls as you configure your cloud infrastructure.

    Getting Started on Your Cloud Journey

    Before we dive into IaC security, let’s talk about why platform teams want to deploy infrastructure-as-code as they move through their cloud journey. Most organizations have not yet tackled IaC. They start their journey by experimenting with cloud configurations until they have defined a stack that works for their teams. In this early stage, application teams may be defining their own cloud stacks. They are manually configuring infrastructure or using scripts. Configurations may not be standardized across teams.

    Standardization of Infrastructure Saves Time

    Although experimentation is reasonable early on in the journey, standardization of infrastructure saves time. Cloud services are no different than applications that run on premises. When your teams are using different versions of tools in their stack, getting an application up and running can be a nightmare. Interoperability between services is challenging, and it can feel like you are fixing a different flavor of the same problem over and over.

    Mature cloud platform teams typically prefer to create a ‘golden image’ with a baseline set of services for development teams. They may create a menu of a few standard options to meet different teams’ needs. Because they are standardized, the platform team can take time to validate that they interoperate and are secure. For example, these golden images will be regularly scanned for vulnerabilities and updated to ensure they meet internal security standards and any relevant regulatory requirements.

    IaC security cloud

    Automating Infrastructure Deployment Reduces Errors

    Manually configuring the environment in the cloud for each team is tedious. Once these configurations have been standardized, the cloud infrastructure can be provisioned as code using templates. A declarative approach defines the desired end state, whereas an imperative approach defines the detailed steps to achieve the end state. The templates are normally stored in a Git repository and version controlled. Automating the configuration of infrastructure not only saves time but also reduces errors that can disrupt operations.

    IaC Security Shifts Security Further Left

    Once you have templates that define your infrastructure, standardizing permissions and access rights for services-as-code is a logical next step. The security team typically defines the guardrails or policies for accessing cloud services. These policies can then be defined as code along with the infrastructure configurations. This IaC security is a further ‘shift left’, allowing security to be embedded into infrastructure, even before an application can be deployed. Implementing policy-as-code minimizes the opportunity for human error in configuring security services. By defining these policies and enforcement guidelines together, the DevOps and security teams can establish a trusted partnership.

    Although some organizations may start their journey by implementing IaC security, for most organizations it happens later in the cloud journey. The order is less important than ensuring you take a DevSecOps approach and embed security at each stage to manage risk. Be sure to address the following areas as part of your cloud and container security program:

    Continuously Check Cloud Permissions and Configurations

    Even if you have IaC templates, you need a program to regularly take inventory of all cloud assets and confirm permissions and configurations. The Center for Internet Security (CIS) benchmarks are commonly used to check passwords and access rights. Beyond enforcing these best practices for security on cloud services, continuously checking for changes to permissions and suspicious activity is critical. The cloud vendors capture logs across all of their services, and your security tool should continually monitor and alert using those logs.

    Scan Images for Vulnerabilities and Risky Configurations

    Scanning for vulnerabilities and ensuring configurations use best practices allows you to shift security left into the development pipeline. Scan everywhere you have images—including within registries, pipelines and also check for new vulnerabilities identified for images running in production. Be sure to scan hosts as well as containers. Defining scanning policies and enforcing through the development process creates guardrails for the development team and clarifies expectations. Most regulatory compliance standards have image scanning requirements as well.

    Detect Threats and Respond to Alerts

    Prevention is fundamental but must be supplemented with threat detection and response. Alerting on unexpected behavior can identify the presence of a threat. Initially, teams may use out-of-the-box detection rules as a way to get started and then customize rules over time. Checking configs against dockerfile best practices is a good way to get started. Overly permissive access rights for containers is common in Kubernetes environments and should be allowed only when required.

    As your team develops more expertise with containers and Kubernetes, you can also implement zero-trust network security policies to block unexpected communication between pods, services and applications.

    Some DevOps teams handle triage of alerts, as they can quickly determine if the issue is a configuration issue or an indicator of compromise. Teams often map detection rules to the MITRE ATT&CK framework, allowing their security teams to manage alerts consistently across their environment. For any team to effectively respond to a security alert, it is important to capture a detailed record of exactly what happened before, during and after that event. This visibility can be particularly challenging with containers but it is critical because containers have short lives. The 2021 Sysdig Container Security and Usage Report revealed that about half of containers live less than five minutes.

    Complying with Regulatory Standards

    Complying with regulatory standards can be painful. Some standards were written with cloud computing in mind, like NIST 800-53 and SOC2, and others were not. With those broader standards, it is important to understand the controls that are relevant to containers and the cloud.

    As you consider your compliance program, be proactive—don’t wait until you are ready to roll into production. Start early by engaging your compliance team and helping them get up to speed. Your role is to provide evidence of compliance to pass an audit at any time using the documentation from your security platform.

    Getting Cloud and Container Security Right

    The move to the cloud creates a tremendous opportunity to get security right and reduce risk. The fact that physical security and a chunk of operations are delegated to cloud vendors frees up security and development teams to focus on their more narrow scope within the shared security model. By taking a secure DevOps approach and inserting security into every stage of the application life cycle, your team can confidently and securely run cloud and containers.

  • The SolarWinds Query Report 2021: Database Priorities and Pitfalls

    The SolarWinds Query Report 2021: Database Priorities and Pitfalls

    The rise of hybrid IT and the accelerated adoption of cloud-native apps have evolved traditional database design, management and monitoring. Workloads are rapidly shifting to the cloud and application performance is heavily dependent on the performance of the underlying database. Meanwhile, the global COVID-19 pandemic has forced companies to make significant changes in the last year. Although it was and is a necessity, the transition to remote work and many organizations’ rapid pivots in strategy posed significant challenges for the technology professionals tasked with supporting new employee demands and shifting business priorities.

    As a result, organizations are increasingly acknowledging the importance of database performance management and strategy to ensure their most valuable asset: Data. SolarWinds recently revealed the findings of The SolarWinds Query Report 2021: Database Priorities and Pitfalls, which examines the current state of database management, evaluates the primary challenges database professionals face today and outlines how database professionals can best overcome these challenges.

    The North American results are based on responses to an online survey fielded in February 2021, which included participation from 331 technology professionals responsible for database performance management in the U.S. and Canada from public and private sector small, mid-size and enterprise organizations.

    Among many other key findings, this year’s report revealed the following:

    Increasing Complexity

    Database design and management are evolving in tandem with increased complexity, and IT professionals are on the verge of a significant shift in strategy that embraces cloud database-as-a-service (DBaaS) platforms as critical database performance success factors.

    • Though nearly 80% of the respondents said Microsoft SQL Server is the most critical platform in use today, there’s a relatively even split between the second, third, and fourth most-used platforms: cloud DBaaS (54%), Oracle (47%) and MySQL (43%).
    • As tech pros keep pace with advances in database platforms and indicate an intended shift toward cloud DBaaS adoption (33% of the respondents ranked cloud DBaaS as the number-one priority database platform to adopt over the next three years), the new mix of traditional/cloud DBaaS platforms will also require new skills.
    • Complexity increased in response to COVID-19 and the transition to remote work over the past year, according to 31% of respondents, who cited increased data demands from users, increased volume of data and security/compliance issues (such as GDPR) as the top three drivers of added complexity.

    The industry’s acknowledgment of cloud DBaaS platforms as critical success factors for database performance is now underway. When it comes to cloud migration (especially amidst a pandemic dramatically accelerating planning and execution of cloud computing and digital transformation), database professionals should start with a basic data and code cleanse and ensure only what’s truly needed is being migrated to reduce complexity and overall cost (we know cloud computing costs can add up). A comprehensive, successful migration strategy should also include a detailed assessment of the data estate—what dependencies or complex coding features exist? Remember, everything interconnected with a database has to go with it to the cloud, so taking these preliminary steps will help organizations avoid surprises down the road.

    Database Security/Compliance Influence Priorities

    The cloud and security/compliance are key factors influencing priorities and resource allocation in 2021 and beyond.

    • Nearly 60% of tech pros surveyed stated that of the data/systems currently managed within their organization, less than 25% are in the cloud. But this is expected to change, as 36% of respondents reported integrating cloud DBaaS is one of their organization’s top priorities in 2021.
    • Two-fifths of the respondents anticipated bigger budget allocations in 2021 (compared to 2020) for cloud services and security/compliance (42% and 39%, respectively).

    Managing database platforms is a big undertaking—a balancing act between proactive and reactive responses. But tech pros must make sure they’re covering all their bases, especially when it comes to security and compliance (and, on occasion, cost and licensing). Time and time again, we see these four big-ticket items pushed to the bottom of the priority list, and falling at one of these hurdles can do a great deal of damage. The new generation of legal compliance has real teeth if you fail to comply with the law. A simple cost-benefit analysis can be the proof point you need to highlight the potential impact of not upskilling in one of these areas and can give you a reason to move it back to the top of the priority list.

    Maintenance Constraints

    Database professionals devote more than a third of their day to database maintenance. This constrains both time and opportunity for activities such as innovation and upskilling in response to increased complexity and data-driven initiatives.

    • Maintenance work accounts for a greater portion of database professionals’ time when compared to other tasks. One-third of the tech pros surveyed stated one of the top areas where they spend the most time as a database/data/tech pro is maintenance (33%) and database development (22%).

    Daily database maintenance consumes time tech pros could otherwise be spending on things like upskilling or innovating new ways of working. For example, nearly half of those polled stated performance/query tuning was one of the top three skills/tools they believed they could develop/improve to better manage increasing database complexities, followed by security management and data engineering (45% and 41%, respectively). One of the best ways to work smarter, not harder: Mitigate downtime spent on maintenance and enable efficiency is to implement appropriate automation and monitoring tools. When implemented properly, automation and monitoring solutions can improve overall performance and eventually lead to reduced costs. Additionally, they can free up tech pros’ time, allowing them to focus on more proactive database performance management.

    The report reveals just how quickly the database architecture and operations tech pros are managing in today’s organizations are changing—and the web of management complexity accompanying this change. With application availability and performance as critical business drivers, the database has long been the linchpin of today’s IT environments. However, against the headwinds of increasingly diversified platforms and reduced resources, tech pros must be equipped to do more than reactive maintenance in environments that keep adding more databases with each passing day.

    Tech pros must capitalize on the opportunity ahead to maximize the performance potential of cloud-hosted databases and ensure the successful optimization of their organizations’ data.

  • Establishing Trust in Multi-Cloud Environments

    Establishing Trust in Multi-Cloud Environments

    Modern applications are transforming enterprises into digital innovation factories. However, the distributed nature and complexity of modern apps have made it extremely difficult for organizations to maintain trust and compliance across multi-platform, multi-cloud environments. 

    Although Kubernetes is the standard for application platforms today, each cloud service provider (CSP) has a different offering for infrastructure-as-a-service (IaaS) or their own platform-as-a-service (PaaS) with varying capabilities and APIs. Not only are these APIs incompatible (with the exception of Kubernetes if no vendor extensions are used), each infrastructure and platform are treated and managed in silos. These silos act as an isolated and non-overlapping administrative boundary,  preventing cross-boundary visibility and trust between microservices.  This makes it much harder for enterprises to know and consistently remediate their security posture across clouds and platforms, opening them up to attacks.

    To solve this problem, organizations are extending enterprise security controls to cloud environments. The problem is that application teams and security teams have traditionally had competing goals—agility versus risk and control—and security operations have not evolved at the same pace as application operations. This creates friction between the teams and forces application operation teams to make a choice: 

    1) Slow down application delivery and operations to reduce risk and miss out on the transformative benefits of multi-cloud or 

    2) Continue to develop and ship applications as quickly as possible without considering security and compliance. 

    Neither is a good option. To unlock the power of modern applications running in multi-platform and multi-cloud environments, organizations need to find a way to integrate security seamlessly throughout the software delivery life cycle at the speed users expect.

    Integrating Zero-Trust into the Application Delivery Cycle

    The key to securing applications in a multi-cloud world is zero-trust. Building zero-trust principles directly into modern applications allows organizations to identify threats early, reduce attack surfaces and ship security functions alongside the rest of the application stack—regardless of the underlying application platforms and cloud stacks underneath.

    In this use case, one of the key components of zero-trust is application segmentation; in other words, how to make access decisions between different applications or components of applications. Zero-trust must provide identity-aware, adaptative and on-demand access to any app from any user or app in a way that improves the flexibility and the agility of application access through a scalable model.

    Application segmentation must be provided to application operation teams so that they can embed it into their application quality processes as another gate. Application operation teams can do this by implementing application segmentation policies that identify workloads as they are created through rich, dynamic attributes. The more intrinsic/static attributes (IPs, digital certificates, namespaces, labels, etc.) and extrinsic/dynamic attributes (user behavior, workload behavior, network behavior, etc.) that are identified, the more fine-grained your access decisions can be. 

    Incredibly, application segmentation today continues to be done manually. But with millions of transactions happening across multiple cloud environments, there’s no way a person—or even a team of people—can manually assign policies to each workload. Organizations need a way to deliver zero-trust application segmentation in a scalable, transparent and automatic fashion. 

    A Connectivity and Security Platform 

    The way to do this is through the use of a modern application connectivity and security platform. Through a service mesh, an attribute-based access control (ABAC) model consolidates multiple third-party tools that provide visibility and security across application platforms and multi-cloud environments. This single view operational model gives application teams the ability to seamlessly and simply orchestrate security services across multi-cloud environments, automatically and at scale. 

    Unfortunately, granular policy management through microsegmentation increases complexity, which is the enemy of security. Suddenly, an organization may find itself with thousands of policies that dictate how different workloads access and interact with entities inside the network. These policies need to be continually updated and maintained, creating massive headaches for both application and security teams.

    When a workload is deployed, the associated policies are created along with it and move through the environment with the workload until the workload reaches the end of its life and is decommissioned. As applications run over time, they are being characterized by their unique behavior. This runtime security characterization allows policies to be automatically selected and applied directly to the workload.

     However, for a service mesh to work and enable secure applications in a multi-cloud environment, it needs to follow zero-trust principles without adding complexity and without slowing down the agile delivery cycle. Here’s how you do that:

    1. Baselining trust: A modern app connectivity and security platform should have native observability and self-discovery capabilities, such as being able to automatically discover APIs, catalog APIs and generate API documentation based on the OpenAPI standard. The modern app connectivity and security platform should be able to continuously baseline application behavior and detect anomalous application behaviors. This needs to alert on unknown and zero-day attacks, especially exfiltration of sensitive data.
    2. Establishing trust: It is critical to define the interconnectivity needs of different pieces of the application right off the bat. This explicit declaration of intent is typically done manually by application teams during the deployment of the application as part of a CI/CD pipeline. Another option is to automatically discover the connectivity intent during the testing phase of the application and allowing the service mesh to identify and record the resulting communication flows between microservices’ APIs. These are the basis of the application’s segmentation policies.
    3. Enforcing trust: There also must be a mechanism to automatically apply the appropriate application segmentation policies to enable the necessary communication for the application to run as intended. In the microservices model, different pieces of the application are spun up and down dynamically to enable the application to run as intended. It’s critical that the application segmentation policies that govern accessibility also adapt to these changes. This is done through the service mesh control plane that interacts with the application platform to gather the inventory of workloads.
    4. Dynamically adjust trust: As the applications run over time and clients interact with them, they are being characterized by a behavior. In this case, a security behavior. Observing this behavior can be accomplished through various runtime analysis tools that organizations already have in place that detect process, container, network and user behavior to provide security context. However, to maintain trust, there needs to be a way to integrate these tools through the service mesh to create a single source of truth. 
    5. Extending the trust model to the edge: Applications also interact with software-as-a-service (SaaS) platforms such as Salesforce or SAP, and other apps outside the data center. Ideally, the service mesh/zero-trust application segmentation model can integrate with other security solutions in the cloud such as secure access service edge (SASE), secure web gateway (SWG), cloud access security broker (CASB) and other zero-trust security components. This standardizes the processes for establishing trust across the data center, multiple cloud service providers, SaaS platforms and web apps—ensuring consistent security and compliance posture across the entire organization.

    Modern applications are transforming how business works by enabling agility and real-time decision making, but they will never reach their full potential unless application teams work with security teams to protect users, data and applications in multi-cloud environments. It’s clear that organizations need an evolved security model that application teams can use to seamlessly establish trust and orchestrate application segmentation across multi-cloud, multi-platform environments. Service meshes can provide the visibility and control needed to establish and continually evaluate trust.

  • 5 Steps to a Seamless Multi-Cloud Migration

    5 Steps to a Seamless Multi-Cloud Migration

    With digitization the topic of discussion in almost every boardroom since the onset of COVID-19, it’s no coincidence that migration to the cloud has skyrocketed in tandem. In fact, at the onset of the pandemic, 87% of IT decision-makers noted that the remote restructuring of the workplace would push their organizations to accelerate cloud migrations.

    That trend shows no signs of slowing down. In fact, Gartner predicts that cloud spending will increase by 23% in 2021 to reach $332B. Driving this historic shift is the adoption of multi-cloud infrastructures. Businesses are increasingly separating and distributing their data flows and critical functions across multiple clouds from different service providers. According to Flexera’s 2021 State of the Cloud Report, an overwhelming 92% of enterprises say they now have a multi-cloud strategy, and 82% say they have a hybrid cloud strategy.

    There are many reasons for this increase in adoption. Developers that use containers, for example, can test their applications across multiple systems in the multi-cloud environment to speed up innovation, due to containers’ agility and portability. This use of multiple clouds also allows businesses to tackle cloud migrations in a more measured and deliberate manner rather than a ‘big bang’ approach.

    What this also means, however, is that it multiplies IT’s stress as they deliberate the benefits, costs and risks of moving their business to a multi-cloud strategy. For example, they must consider whether or not to keep their data on-premises, migrate it to a public cloud or some combination of both. They must also ensure their application’s ironclad security, which has become a mission-critical consideration during the unrelenting cybersecurity emergency that is the pandemic. Meanwhile, they may worry that such a large-scale cloud migration will result in lost revenue and increased downtime.

    With these key considerations and challenges in mind, here are five tips to help enterprises seamlessly migrate their resources to a multi-cloud environment.

    Assess Your Application’s Ripeness

    Before any application is migrated, it’s necessary to do a complete audit of current applications to assess their readiness for the migration. Businesses should look at the application’s dependencies and the ecosystem it lives in. This ecosystem could be its physical and virtual network infrastructure, including configurations, policies, load balancers, security and compliance requirements.

    Plan and Prepare

    This is a particularly challenging phase, as IT teams often linger here; wasting valuable hours and productivity. Therefore, the entire organization must outline what their migration journey will look like, discuss which departments will be affected during the process, and delegate responsibilities to facilitate a smooth transition to the cloud. The IT operations team, the development and quality assurance teams, key stakeholders and leadership should all be involved in these discussions—and should share the goal of ensuring a predictable migration process.

    Carefully Weigh Your Cloud Options

    Choosing the right cloud depends entirely on the unique needs, challenges and opportunities for the organization. While there are a number of different providers available to every enterprise, the best option will be influenced by financial constraints, goals for cloud adoption and even the systems already in place.

    Of course, when a business is taking a multi-cloud approach, these decisions become that much more difficult—especially in light of the fact that the average cloud company uses 2.6 public clouds and 2.7 private clouds, according to the Flexera report. Connectivity between these providers is crucial, with 88% of businesses enabling their multi-cloud strategy with a private connection.

    With this in mind, it’s also absolutely vital that decision-makers consider the security risks associated with each provider, as well. The number of cyberattacks on enterprises’ cloud infrastructures has increased significantly since the onset of COVID-19. With the stakes so high, a migration partner can help businesses assess which cloud providers are the right fit for them; accounting for their cost-effectiveness as well as which options will pose the least friction for the migration.

    Construct Your Migration Checklist

    The migration of an enterprise’s workloads is the most critical step of its cloud journey, and every company’s objective is to make a move swiftly and seamlessly. Creating a checklist can help businesses achieve this goal, as it helps them to prioritize the most critical components of their journey and understand the repercussions of a stagnant or inefficient migration.

    For example, every business should aim to achieve minimum downtime during their migration process and little to no disruption of service. Both the loss of network access and system downtime can cause immediate pain in the form of lost productivity and opportunities and even more harmful damage to a company’s brand. In fact, according to Gartner, the average cost of IT downtime is $5,600 per minute, meaning a well-practiced and efficient migration strategy has a significant impact on a company’s financial health.

    Since each migration is unique to the business and, often, unforeseen challenges arise that are native to the environment, it’s beneficial for organizations to have an expert migration team or partner to help them develop their checklist and ensure their migrations are streamlined and airtight.

    Validate Your Cloud Journey

    As the final step in a business’ migration journey, companies must validate that all their resources from their previous environment are fully migrated and operational in their target cloud environments. This step is crucial, as it marks the completion of a successful multi-cloud migration journey! This is primarily a manual process ensuring that nothing is neglected or forgotten and is often followed by an entire testing lifecycle.

    Post-Migration Pointers

    Cloud migrations can be complex, especially for companies relying on a years-old traditional and highly manual approach. Thankfully, in the age of automation, enterprises today are experiencing substantial time and cost savings and deploying migration strategies that move their workloads across multiple clouds in seconds instead of days.

    But even when an organization’s migration journey is complete, the management of several clouds at once creates its own set of challenges and risks—especially true in today’s landscape of skyrocketing climate and cybersecurity risks. It’s therefore vital that business leaders heighten their focus on organizational resiliency and minimizing downtime through things like tightened security and effective disaster recovery strategies.

    After all, with hybrid work forcing C-suites to scale their digital transformation plans even further, it’s becoming increasingly clear that the future of work is in the cloud. It’s therefore imperative that as cloud spending soars, enterprises allocate substantial resources not only to migration and management but protection, too.

  • What It Will Take To Bring Cloud Gaming to Fruition?

    What It Will Take To Bring Cloud Gaming to Fruition?

    The gaming industry is booming. There are 211 million people gaming in the U.S., and it’s one of the fastest-growing forms of entertainment on the planet. In the age of Twitch streaming, gaming has even upstaged traditional sports viewership. On average, young gamers between the ages of 18 and 25 spend a full hour more watching online gaming than watching traditional sports.

    Gaming has also become a way of connecting with others—65% of players say they play with others online or in person. Some software development teams even encourage gaming in the workplace to increase productivity. For this reason, there has been a massive shift toward online cooperative games, RPGs and sandboxes like Minecraft. Also, games are becoming more cloud-based, with purchases, downloads and game saves occurring in the cloud. Still, many people lack the hardware necessary to run complex games locally. All these factors are harbingers of a coming shift away from PC and console-based processing.

    I recently met with Noam Salinger, product manager, Granulate, to learn about the future of gaming and how software teams in the gaming industry can prepare. To Salinger, “cloud gaming is the next step.” The move away from consoles and toward cloud-based, remote processing will have huge ripple effects. Lowering the barrier to gaming could introduce millions of more players into the ecosystem and further balloon the industry. However, cloud gaming presents significant hurdles. The transition will require a substantial reduction in latency, 5G networking and highly efficient computing optimizations.

    Roadblocks to Cloud Gaming

    “Cloud is an integral part of any gaming company,” says Salinger. More and more functionality associated with gaming is being pushed online. There are already online game marketplaces, cloud-based account management and remote game saves. This is especially prevalent with mobile gaming. But, what about the actual processing for computationally-intensive games—could these workloads also move to the cloud soon?

    This kind of cloud-based gaming could soon come to fruition. Yet, Salinger foresees some major roadblocks to cloud gaming:

    1. The Software Components

    A complete jump to cloud gaming will require having the proper software infrastructure in place and getting vendors on board to support it. This is perhaps the biggest obstacle to cloud gaming in the software industry, said Salinger, “It’s still not mature enough.”

    2. The Business Components

    Furthermore, gaming platform companies might refrain from cloud gaming due to fears of it cannibalizing their existing hardware sales channels. On the other hand, cloud gaming presents a big opportunity to take on more of a subscription-based model, such as allowing a user to play any game they want from an extensive suite of games. Akin to streaming services like Netflix, Hulu or Peacock, we could see similar platform styles emerging in gaming. “They will have no choice but to participate at some point,” said Salinger.

    3. The Network Components

    As most gamers can attest, online games can easily suffer from network issues that negatively affect gameplay. Salinger predicts that the ubiquity of 5G networks will be necessary to propel the cloud gaming market forward. 5G could be a universal method to enable real-time connections for a broader base of end-user gamers.

    Even if software developers optimize game processing speeds, a poor network could add enough latency to data throughput to cancel out those gains. 5G could expedite this process, but it’s taking a lot longer than expected. Some estimates place 5G finally arriving at full throttle sometime in 2023, after more antennas are constructed and pre-existing physical infrastructure is upgraded.

    4. The Latency Components

    Another significant roadblock to cloud gaming is figuring out real-time remote computing on the game vendor side. “This is where the latency aspect of things really comes into play,” said Salinger. Since game clients will be making many remote calls, optimizing cloud CPU utilization will be vital to reducing the response time to requests.

    Cloud gaming will require parallel running processes that anticipate user action. Similar to the way modern web applications simultaneously trigger multiple backend requests for authentication, geolocation, user databases and so on, real-time cloud games will require many different processes running in parallel. Servers could allocate resources before requests come in, using predictive analysis of usage patterns to trigger components in advance of user behaviors.

    To enable intelligent resource allocation, Salinger describes how an OS-agnostic kernel module could be in charge of resource allocation, continuously adapting to changes in the application workload. Low-level resource allocation at the CPU level, the allocation memory, I/O, and the sequence of execution must be optimal for each specific application, he described.

    This sort of process is already being applied in production by some ad tech companies to enable real-time ad bidding. For example, by optimizing cloud CPU utilization, Perion is able to perform faster, process more responses and use fewer machines to perform the same task—thereby reducing the roundtrip time for each request. Similar principles could be used to advance cloud gaming.

    Rendering sophisticated virtual environments is highly resource-intensive, and high-performance graphics cards are expensive. Video games are set to only increase in complexity with new AR/VR modes. Cloud gaming could help respond to rising computational needs and replace the need for local computing hardware. In the process, this move could bring in more gamers, disrupt the console wars, and nullify age-old PC-versus-Mac compatibility issues.

    Moving to a cloud-based approach is also arguably more aligned with today’s fast-paced software release cadences. Pushing more frequent patches can help resolve bugs faster. But it also could aid in retaining engagement by introducing new levels and objectives continually.

    The rise of the gaming ecosystem can’t be ignored. But as it stands now, there are still significant challenges to wider penetration of cloud gaming. Cloud computing optimizations to reduce latency, mainstream adoption of 5G and significant re-platforming of software infrastructure are just some of the many issues that will need to be tackled before truly cloud-based gaming experiences advance to the next level.

  • Harness Updates Cloud Cost Control Tool for DevOps Teams

    Harness Updates Cloud Cost Control Tool for DevOps Teams

    Harness today announced that it has updated its cloud cost management (CCM) module for its namesake continuous integration/continuous delivery (CI/CD) platform to add support for Kubernetes at the node level in addition to providing business intelligence tools to analyze overall cloud costs.

    Harish Doddala, senior director of product management for Harness, said the update surfaces more granular insights into cloud costs that DevOps teams are incurring because cloud resources are being wasted. The latest version of CCM enables DevOps teams to create their own customized views of those resources.

    As the overall economy continues to recover from the COVID-19 pandemic, more organizations are looking to exercise greater control over IT costs. In the immediate aftermath of the pandemic, many DevOps teams consumed cloud resources without much regard to cost to enable organizations to advance various digital business transformation initiatives. Now, finance teams are requiring DevOps teams to account for those cloud costs.

    The goal is not only to reduce costs whenever possible—by, for example, quickly identifying a sudden spike in cloud resources consumption—but to accurately forecast budget costs on a monthly basis so that both IT and finance teams can collaboratively track spending. The typical finance team is just as concerned about accurately predicting costs as they are the total amount spent.

    Harness Cloud Cost

    Ultimately, Doddala says Harness’ CCM enables organizations to better assess which software development projects to prioritize based on the cloud resources allocated. It’s not uncommon for organizations to suddenly have to shift software development priorities as business conditions change and evolve. A project that may be a top priority today can easily be downgraded because, for example, a rival has been able to launch some new capability that an organization now needs to quickly match.

    DevOps teams, unfortunately, don’t always pay as much attention to infrastructure costs. Harness’ CCM helps reduce those costs by automatically stopping idle resources running on Amazon Web Services (AWS), Microsoft Azure, Google Cloud Platform (GCP) or any Kubernetes cluster, said Doddala.

    There are, of course, any number of existing tools that cloud service providers make available to enable IT teams to reduce costs. CCM, however, provides that capability within the context of a DevOps platform that can span multiple cloud computing environments, noted Doddala. DevOps teams, for example, can create and enforce cost policies for cloud infrastructure into a CI/CD pipeline and prevent an organization from being surprised by a bill from their cloud service provider at the end of the month.

    It’s not clear the degree to which cost control modules will influence the selection of one CI/CD platform over another. However, it’s apparent the days when organizations consumed cloud resources with little regard to cost are ending. In fact, there may soon be more cost controls imposed on software development projects than there were prior to the pandemic. The challenge, of course, is finding a way to automate the implementation of those controls and minimize the impact on software development velocity.

  • Open Your Ears and Get Ready to Listen: Cloud-Native Days with Kubernetes

    Open Your Ears and Get Ready to Listen: Cloud-Native Days with Kubernetes

    If you think all the way back to primary and secondary school, you may remember hearing something like, “open your ears while your teacher is talking to you.” It’s true that someone tuned in to listening learns more than someone who doesn’t, but opening your ears is a lot more important than that. Open ears have the ability to open minds. And with the tech industry constantly changing it’s more important than ever to keep your ears—and your mind—open to change.

    With Cloud-Native Days with Kubernetes approaching, attendees have the perfect chance to open their ears and listen to the best of the best in the tech industry. Cloud-Native Days with Kubernetes is a two day virtual summit boasting over 50 speakers and sessions led by industry thought leaders and doers in the cloud-native space. Attendees will walk away with a better understanding of cloud native, its impact on the IT landscape and a lot more after listening to the key practices and methodologies of these tech titans.

    While there are a myriad of talented speakers at Cloud-Native Days with Kubernetes, there are two very special keynote speakers you’ll want to pay extra attention and listen extra closely to. Hosting keynote sessions Aug. 4 and 5, Tara Hernandez and Caroline Wong are prepared to talk about technology shifting and the future of cloud-native security. With loads of experience and sage advice, Tara and Caroline are without a doubt the top two tech experts you won’t want to miss.

    If you’re unfamiliar with Tara Hernandez, she’s a senior engineering manager at Google Cloud Platform, Associate Board member of Women Who Code and the host of the 12 episode podcast series, ‘Women Who Code’. A specialist in software engineering infrastructure and early backer of the Continuous Integration revolution, Tara has done everything from developing developer tools such as Tinderbox and Bugzilla, working at Pixar Animation studios to reinvent development infrastructure to handle the challenges of writing software in a film production environment, and helping create cloud developer platforms at Google. Beyond being a wizard in her field, Tara is equally impressive in her drive to share useful things about engineering management. 

    No-one’s journey goes without its challenges. But for Tara, “One issue [she] has faced in her rise to success is a case of imposter syndrome. While she has always had the skills to overcome any challenges that came her way, it took a lot of work for her to develop mental self defenses, and tame the anxiety that came when she was presented with a project […] Getting involved with [Women Who Code] has made her aware of the need for advocacy at a grassroots level, and she’s become more confident brainstorming with peers and management teams about increasing diversity in her own work place. At the same time it’s helped her find the confidence to attend more conferences, and even start giving talks.” A supporter, advocate, mentor, and “shining example of someone who has overcome doubt to achieve success,” Tara is the ultimate speaker to inspire change and innovation, especially in creators. 

    Check out Tara’s podcast ‘Women Who Code’ and listen to her talk about the current state of DevOps here to learn more before her talk, ‘Those Little Tectonic Shifts,’ Wednesday Aug. 4 at 12:45 p.m. EDT. 

    Cloud-Native Days with Kubernetes is also incredibly proud to host Caroline Wong, Chief Strategy Officer of COBALT. Caroline is also an author, teaches cybersecurity courses on LinkedIn Learning, and similar to her fellow keynote speaker, hosts the ‘Humans of InfoSec’ podcast. With a background in communications and cybersecurity, Caroline holds deep-rooted expertise every attendee will want access to. 

    As Caroline says, “Cloud native is here; it is scaling, and it is not going anywhere. The more we can see the reality and necessity of what security must become, the better we will all be in the long run.” In her talk, ‘The Future of Cloud Native Security,’ Thursday Aug. 5 at 11 a.m. EDT, Caroline will share her observations on how companies must change the way they build security into their cloud-native projects, forecast what’s next for cloud-native security, and how organizations must double down on people and process innovation to overcome the misconceptions, education gaps and common mistakes we see when it comes to the cloud.

    Be sure to check out Caroline’s podcast ‘Humans of InfoSec’ here before joining her keynote session next week.

    It’s always important to listen. But when listening means driving ideas forward, opening your ears can be the most beneficial tool to be better than we were yesterday.

    So open your ears, get ready to listen and join us Aug. 4 and 5 at 9 a.m. ET for a much needed active listening experience! Register today.

  • How to Build an Options-Based Observability Strategy

    How to Build an Options-Based Observability Strategy

    The global pandemic forced companies to accelerate their digital transformations, pushing them toward new technologies and architectures as they struggled to adapt to a changing world. Companies wanted fast development and deployment and were happy to trade off efficiency, reliability and sometimes, security, to pursue their goals. Now that the pandemic is receding in many parts of the world, the priorities are rebalancing as companies grapple with choices they made under fire.

    Observability has emerged as a way for enterprises to better understand sprawling infrastructure and application environments. According to Gartner, “Observability enables organizations to reduce the time it takes to identify the root cause of performance-impacting problems. In particular, and in contrast to traditional monitoring, operators can freely interrogate data post-hoc without the need to pre-program dashboards.”

    This sounds promising and the market responded quickly, but predictably. Large and monolithic monitoring platforms embraced the term and added new features, making them even larger and more monolithic. These monolithic platforms represent a failed choice-based strategy. They assume that all observability data is collected, centralized, integrated and only analyzed by a single tool. Enterprises following a choice-based observability strategy find themselves locked-in to multi-year investments without the flexibility to react to evolving technologies.

    While this has been good for vendors in the incumbent monitoring space, end users haven’t benefited from these enormous platform deployments because the choice-based strategy is built on the faulty assumption that a single pane of glass serves all of your monitoring and observability audiences, from SREs to DevOps and SecOps.

    A choice-based strategy also forces customers into limited options in the future as technology continues to evolve and requirements change. Defaulting to choice-based strategies also leaves little leverage for contract negotiations, data integrations and data access.

    Infrastructure and operations teams need to take a different approach to their observability strategy. Instead of focusing on buying products and making choices, teams need to shift their thinking to creating options. The choices you make today limit your options tomorrow. Tomorrow is unpredictable and uncertain. Taking an options-based approach allows you to be flexible and adaptable today and into the future.

    An options-based observability strategy is built on three pillars:

    • Establishing your observability strategy principles
    • Building composable environments
    • Staging your commitments

    Let’s look at each of these.

    Establish the Principles of Your Observability Strategy

    When asked about their observability strategy, many often leap into detailed plans covering every imaginable use case. They talk about the current state of their infrastructure and how things will be at the end of three or five years, including the skills and functions they’ll need in that time. If you’ve read the above, you’ll recognize this as a choice-based strategy: All the choices they need to make over the next several years are already laid out. This leaves little room for adaptation and options as the world evolves around you.

    Instead, think of your observability strategy as a single principle, or a collection of principles, guiding your observability strategy. Let’s say your core principle is that observability is a key enabler of operational efficiency across SREs and DevOps. We can build on that operational efficiency principle to define what key elements of our strategy has to include:

    • Data sources have utility-like accessibility; always on and available across tools.
    • Metrics are primarily driven by total cost of ownership and optimization.
    • Governance is centralized to the team distributing data to downstream systems.

    These elements are different if your core principle is something like customer intimacy or risk management.

    With principles and strategy in hand, we can focus on the next phase: Composing the observability environment.

    Compose your observability environment

    In an options-based strategy, the key success factor is avoiding large ecosystems that limit flexibility. Many vendors take a “give us everything” approach to observability. These approaches appear to start and end at the destination platform, but they begin even earlier, at data origination. It’s common to see agents only sending data to one destination, limiting the usefulness of the data by locking it into a silo.

    Staying modular means selecting tooling that abstracts the data sources from destinations and puts the power back into your hands. By using an abstraction like an observability pipeline, you can send data from one source to multiple destinations, while managing that data in any way you see fit. For example, this is where your centralized governance can occur.

    When new tools are introduced, upgraded or deprecated, an observability pipeline insulates the rest of your environment and users from those changes. A modular approach also gives you more power in contract negotiations with your other vendors because you’re less reliant on one vendor for all of your observability needs.

    Stage Your Commitments

    A choice-based observability strategy is a large commitment, often taking years. Once you start, stopping becomes impossible. That’s simply unacceptable in today’s environment of uncertainty. Choice-based strategies limit your ability to test, experiment, and learn. At the same time, our infrastructure is critical and it must be reliable.

    An options-preserving approach may involve experimenting with freemium products; only paying once the product’s value has been proven. Cloud-based options are another consideration since they are commonly subscription-based and offer easy try-before-you-buy purchasing styles. Even better, cloud-based options don’t require using your own infrastructure to test new options.

    Every IT team feels the need to react and respond to demands from the business. That urgency to make choices, to solve the immediate problem, can have long-term implications for future flexibility and agility. By structuring your observability strategy around a long-term vision based on preserving options, your team can remain adaptable as technologies and business requirements evolve.