devopsy.kinsta.cloud

Tag: CNCF

  • TOC Votes to Move OPA into CNCF Incubator

    Today, the Cloud Native Computing Foundation (CNCF) Technical Oversight Committee (TOC) voted to accept the Open Policy Agent (OPA) as an incubation-level hosted project.
    OPA, which entered the CNCF Sandbox in March 2018, is an open source, general-purpose policy engine that enables unified, context-aware policy enforcement across the entire stack. It
    provides greater flexibility and expressiveness than hard-coded service logic or ad-hoc domainspecific languages and comes with powerful tooling to help anyone get started.

    “Policy-based control is necessary as enterprise adoption of cloud native technology expands,” said Torin Sandall, Software Engineer at Styra and Technical Lead for OPA. “OPA has found an
    ideal home within CNCF. Other CNCF projects can optionally integrate with a consistent policy expression and enforcement mechanism, and common toolset around it. We are looking
    forward to continuing this work as we move into the Incubator.” Every organization has unique policies that affect the entire stack. These policies are vital to long term success because they codify important requirements around cost, performance, security, legal regulation, and more. At the same time, organizations often rely on tribal knowledge and documentation to ensure that policies are enforced correctly. While these approaches are known to be error prone, they exist because systems frequently lack the flexibility and expressiveness required to automate policy enforcement.
    With OPA, services offload policy decisions by executing queries, and OPA evaluates policies and data to produce query results (which are sent back to the client). Policies are written in a
    high-level declarative language, Rego, and can be loaded into OPA via the filesystem or welldefined APIs.

    “The cloud-native ecosystem must provide flexible solutions to control who can do what across modern, microservice deployments because legacy approaches to policy management do not
    satisfy the requirements of modern environments,” said Chris Aniszczyk, CTO/COO of the Cloud Native Computing Foundation. “OPA has made strides integrating with Kubernetes
    through the Gatekeeper project that integrates policy management. Moving OPA to the CNCF Incubator will raise awareness and encourage the development of OPA extensions in and
    outside the cloud native ecosystem.” OPA has been used to policy-enable software across several different domains across several layers of the stack: container management (Kubernetes), servers (Linux), public cloud infrastructure (Terraform), and microservice APIs (Istio, Linkerd, CloudFoundry). Starting in January 2019, Styra, Google, Microsoft, and others began jointly developing and contributing the OPA Gatekeeper sub-project. Gatekeeper integrates OPA with Kubernetes to help admins enforce admission control policies and audit clusters for existing policy violations.
    Gatekeeper also includes a standard library of policies for common use cases (e.g., registry whitelisting, ingress conflicts, label management, etc.).

    Netflix uses OPA as a method of enforcing access control in microservices across a variety of languages and frameworks for thousands of instances in their cloud infrastructure. At KubeCon
    Austin 2017, Netflix described how they architected access control around OPA. OPA is also used in production by companies like Intuit, Medallia, Chef, and others. “Applying access control in a diverse ecosystem is a challenge at scale. Ensuring that all of our applications spread across many applications and frameworks are applying authorization correctly and consistently would be impossible without the right tools. OPA gives us the power to define policies in a consistent manner, leverage complex data sources in those definitions, apply them everywhere in our environment, and easily test them for correctness.” said Ian Haken, Senior Security Engineer at Netflix.
    Main OPA Features:
    ● Decoupled – administrators can manage policies dynamically without requiring changes
    to services.
    ● Compatible – RESTful APIs use JSON over HTTP so you can integrate OPA with your
    service no matter which programming language you use.
    ● Responsive – designed from scratch with latency-sensitive applications in mind,
    enforcing policies with minimal performance impact.
    ● Interactive – anyone can use OPA’s interactive shell to quickly experiment with queries
    and data sets.
    ● Easy to Deploy – has zero deployment dependencies. It runs as a daemon side-by-side
    with your service and shares its fate for the purposes of high availability.
    ● Embeddable – services written with Go can use OPA as a library and do not need to run
    a separate daemon.
    Notable Milestones:
    ● 45 contributors
    ● 1,847 GitHub stars
    ● 52 releases
    ● 1,545 commits
    ● 151 forks
    As a CNCF hosted project, joining incubating technologies like OpenTracing, Fluentd, gRPC, rkt, CNI, Jaeger, Notary, TUF, Vitess, NATS, Linkerd, Helm, Rook, Harbor and etcd, OPA is part
    of a neutral foundation aligned with its technical interests, as well as the larger Linux Foundation, which provides governance, marketing support, and community outreach. Every CNCF project has an associated maturity level: sandbox, incubating, or graduated
    project. For more information on what qualifies a technology for each level, please visit the CNCF Graduation Criteria v.1.1.

    For more on OPA, please visit https://github.com/open-policy-agent.

     

  • NATS Messaging Entering Cloud Native Purview

    NATS Messaging Entering Cloud Native Purview

    The Cloud Native Computing Foundation (CNCF) has expanded the scope of its purview by adding open source NATS messaging software to an expanding list of technologies it now oversees.

    NATS messaging software has been commonly employed in environments such as platform-as-a-service (PaaS) environments. But with the rise of microservices, a campaign to make NATS a part of the portfolio of technologies that members of CNCF consortium will advance has begun in earnest, said Derek Collison, who lead the development of NATS and is now CEO of Synadia Communications, which is dedicated to enabling organizations to more easily deploy and manage messaging systems based on NATS.

    The ultimate goal, Collison said, is to not only make messaging ubiquitous, but also effectively invisible.

    Collison said he and the CNCF envision a world where messaging just comes baked into every application. PaaS environments may have driven mainstream adoption of NATS messaging to integrate various elements of an application, but microservices will make messaging a fundamental requirement for building applications that can efficiently scale, he said, adding too many organizations already run into these issues because they are depending on HTTP to integrate various microservices.

    NATS is already widely employed by a number of vendors and organizations including Apcera, Apporeto, Baidu, Bridgevine, Capital One, Clarifai, Cloud Foundry, Comcast, Ericsson, Faber, Fission, General Electric, Greta, HTC, Logimethods, Netlify, Pex, Pivotal, Platform9, Rapidloop, Samsung, Sendify, Sensay, StorageOS, VMware Acadiant, Weaveworks and Workiva.

    The core NATS server and NAT Streaming software are maintained by Synadia. There are clients for Python, Ruby, Node.js, Elixir, Java, NGINX, C and C#, as well as third-party contributions that add support for Arduino, Rust, Lua, PHP, Perl and others.

    Longer term, Collison sees NATS evolving to include functionality to support real-time applications, many of which today are using Apache Kafka software. Rather than having to deploy NATS and Kafka for different use cases, Collison said it will make more sense to have a common control plane to address multiple classes of use cases for messaging.

    Messaging systems in one form or another have been at the heart of integration platforms for decades now. But the assumption was always that integration would be addressed after the applications were deployed. In the future, the assumption is that all cloud-native applications need to be integrated, which is much easier to accomplish if they all come with some form of messaging capability baked in. Integrating and managing federated instances of messaging systems still will be required. But much of the lower-level functions now being centrally processed will be pushed out into the applications being integrated.

    Collison is the first to admit that messaging is not some new, emerging technology. But as microservices architectures continue, messaging will prove to be core to a new generation of cloud-native applications, he said. The CNCF is clearly betting that a consortium of vendors working on a common code base for messaging will accomplish that goal a whole lot sooner.

    — Mike Vizard

  • Q&A: CNCF on Instilling a DevOps Culture, Part 2

    Q&A: CNCF on Instilling a DevOps Culture, Part 2

    In part one of this two-part series, devopsy.kinsta.cloud introduced the Cloud Native Computing Foundation (CNCF), a nonprofit organization with 49 members engaged in promoting the development of cloud native applications. Let’s continue with the CNCF’s thoughts on instilling a DevOps culture.

    devopsy.kinsta.cloud: Why do developers need the freedom to choose how they build software? What kinds and levels of freedom? Is this about approaches, development tools or something else?

    Brian Grant: All of the above: approaches, tools, languages … Different solutions work better for different workloads, scenarios, environments and people. Even new applications developed from scratch to be cloud-native aren’t typically developed in a vacuum—their developers may be familiar with particular languages or tools, the new application may need to interact with legacy applications; it may need to be built with and/or deployed onto existing infrastructure, etc. Often developers need to be able to adopt new technologies or approaches incrementally.

    devopsy.kinsta.cloud: How do you recommend providing this freedom to developers? How do you help provide it?

    Grant: Containers provide an ideal mechanism for providing this freedom, since they can be used to deploy both new and legacy applications, written in any language, built by any tool, into any environment. It’s a goal of the CNCF to foster an ecosystem of composable technologies, such as Kubernetes that facilitate the management of container-packaged applications.

    devopsy.kinsta.cloud: How does this align with instilling a DevOps culture?

    Grant: Because containers are small and fast, one application can be packed in each container image. Consequently, immutable container images can be created at build/release time rather than deployment time, since each application doesn’t need to be composed with the rest of the application stack nor integrated into the production infrastructure environment. This empowers developers to deliver deployable artifacts to operations.

    devopsy.kinsta.cloud: How is code reuse a need of the developer?

    Craig McLuckie: The vast majority of the actual code that is run in any given deployment is written by others. Only a tiny fraction of code in any given instance is actually written by the developer. The rest is everything from common libraries, to full OSS subsystems, to commercial software that systems, to adjacent software projects that provide some business functions. Developers need a common way to integrate with existing systems (when it makes sense) or to find and reuse existing code in a way that is controlled and operations-friendly.

    devopsy.kinsta.cloud: How do you recommend giving developers the ability to reuse code? How does your organization help with this?

    McLuckie: CNCF stands behind two technologies that are critical to code reuse. The first is the use of containers, which represent an intrinsically easy and predictable way to reuse even a relatively complex piece of software. The second is microservices-based architecture that make it easy to wire together systems and reuse existing services versus having to build their own.

    devopsy.kinsta.cloud: How does this align with instilling a DevOps culture?

    McLuckie: Containers make operations far more predictable and reduce possible avenues of error when deploying and updating systems. Rather than run potentially error prone imperative configuration systems to reuse code, containers are simple, lightweight and predictable, and reduce the potential for errors.

    Microservices allow for better operations specialization and create systems that are more intrinsically agile and easy to evolve. They allow for smaller teams that are focused on delivering a specific function versus having to align around large system boundaries. This creates a more productive and operationally viable environment.

    devopsy.kinsta.cloud: What is cloud-native? What does that mean?

    McLuckie: For CNCF, ‘cloud native’ is about a new pattern of application development that was pioneered by the internet companies, but that is equally applicable to developers and operators running more traditional workloads.

    We see three common attributes that make systems ‘cloud-native’:

    Container-packaged: Applications and processes are run in software containers as a unit of application deployment, which facilitates reproducibility, transparency and resource isolation. This improves overall developer experience, fosters code and component reuse, and simplifies operations.

    Dynamically managed: Applications are actively scheduled and managed. This radically improves machine efficiency and resource utilization while reducing the costs associated with maintenance and operations.

    Microservices-oriented: Applications are loosely coupled, with dependencies explicitly specified. This significantly increases the overall agility and maintainability of applications.

    Together, these three capabilities allow developers to build systems that are intrinsically agile and scale well. They significantly reduce the operational complexity of running an application, allowing operators to be more focused on application-level concerns and relying heavily on smart systems to do the operations heavy lifting.

  • Q&A: CNCF on Instilling a DevOps Culture, Part 1

    Q&A: CNCF on Instilling a DevOps Culture, Part 1

    The Cloud Native Computing Foundation (CNCF) is a non-profit comprised of some 49 industry participants that have come together to drive development and adoption of cloud-native applications and services. The CNCF is uniting such giants as Google, IBM, Intel and Cisco Systems to distill a collection of container technologies that end-user companies deem praiseworthy to reach its goal.

    The CNCF intends its membership to represent end users (the CNCF website lists one end user, Goldman Sachs), cloud and container technologies, online and IT services, and academic institutions.

    On behalf of the CNCF, Craig McLuckie, its Governing Board chair, and Brian Grant, its Technical Oversight Committee representative, weigh in on leading the DevOps transformation and instilling a DevOps culture in this two-part Q&A.

    devopsy.kinsta.cloud: How is the CNCF specifically involved in leading the DevOps transformation by instilling the DevOps culture, in the industry or in member companies, for example?

    McLuckie: We believe that at the heart of the DevOps culture is the need to reduce the friction and, indeed, the barriers between teams that build an application and the teams that run an application. Developers learn to think about an application as a living service that evolves continuously and rapidly to meet the evolving business needs. Operations is there to deliver the tools and environment needed for them to successfully drive their application.

    devopsy.kinsta.cloud: Why do developers need an environment that is safe to change? What kind of change? Safe how?

    McLuckie: Safe change has many potential meanings: The first kind of “safe” is about creating deterministic behavior in operations. Does the system handle a change correctly every time? If I make a change to two systems, will they have the same state? How about to 1,000 systems? It’s important to focus on creating strong determinism by removing the moving parts in a production deployment.

    The second kind of “safe” is about being able to validate that a change is good—not just in pre-production environments, but in the actual production environment—with a simple and deterministic rollback. It is important to make it trivial to replicate environments, and offer a way to run a simple experiment in production (e.g. route 1 percent of my traffic to this new version) with the option to deterministically roll forward or backward without effort.

    A third kind of “safe” is about being able to ensure that every change is deliberate. One of the problems we often see in production systems is “configuration drift” that isn’t driven through any kind of formal process. This should be addressed by creating a strongly declarative toolset that makes every change deliberate and trackable.

    devopsy.kinsta.cloud: How do you recommend providing these things that developers need? How do you help provide them?

    McLuckie: There are multiple issues like data, security governance and applications that need to be addressed. Additionally, companies want to deliver apps with high velocity. How do you solve all of this? A vendor-neutral organization that can harmonize emerging technologies is important for facilitating cloud-native. Given the large number of players in the marketplace with competing approaches, having an open source way to solve this is also important.

    Grant: Cloud-native infrastructure enables self-service and “hands-off” operation, including provisioning, deployment, scaling, healing, discovery, monitoring and more, with relatively little marginal effort for specific applications. By supporting open source projects such as Kubernetes and Prometheus that provide such functionality, the CNCF hopes to democratize high-velocity application delivery, which was previously achievable mainly by web-scale companies.

    devopsy.kinsta.cloud: How does this align with instilling a DevOps culture?

    Grant: In a DevOps culture, operators are both partners and customers of the developers. Developers take responsibility for meeting operational requirements: security, reliability, availability, scalability, efficiency, actionable monitoring and logging output, necessary configurability, compatibility with the production environment, etc. Cloud-native infrastructure provides mechanisms developers and operators need to facilitate and capitalize on those capabilities.

    Stay tuned for part two of the CNCF interview.