devopsy.kinsta.cloud

Author: Ran Ilany

  • Microservices vs. Monoliths: Which is Right for Your Enterprise?

    Microservices vs. Monoliths: Which is Right for Your Enterprise?

    With microservices projected to grow globally at a 22.5% rate between 2019 and 2025, the choice between monolithic and microservices architectures needs to be carefully considered. If you are wondering what microservices are best suited for, or whether your business should stick to monolithic or switch to microservices, here are a few cardinal points to take into consideration.

    What Is Monolithic Architecture?

    A monolithic application is built on a single code base with a variable number of modules. The number of modules depends on the complexity of the business and its technical features. The whole application—and dependencies, when applicable—are built on a single system with a single executable binary for deployment.

    A monolithic architecture has distinct advantages and disadvantages that should be evaluated when deciding when selecting the optimal type of architecture for the application.

    Monolithic architecture pros:

    • Fewer cross-cutting concerns: The majority of applications typically have a large number of cross-cutting concerns. Running them all on a monolithic architecture facilitates hooking up components.
    • Less operational overheads: Avoids the additional costs stemming from microservices such as interservice communication, service discovery and registration, load balancing, decentralized data management or distributed logging, for example.
    • Complex testing: Testing for microservices is considerably more complex than for monoliths.
    • Better performance: As shared-memory access is faster than interprocess communication, a monolithic architecture typically has better throughput performance.

    Monolithic architecture cons:

    • Overly tight coupling: Entanglement increases the difficulty of isolating services for upgrades or independent scaling when the application evolves.
    • More difficult to understand: As the application grows and the number of layers and dependencies increases, understanding the system becomes increasingly difficult.
    • Slow build and test create releases: Upgrades imply recoding the entire application to factor in the dependencies and side effects.
    • Hard to distribute development: Tightly integrated applications are difficult to break into chunks, which complicates the creation of a distributed team effort.
    • Usually requires opting for a single programming language: Integrating more than one language in a monolithic application is fraught with challenges, typically preventing the integration of components written in a different language, which might limit the flexibility of adding features best suited to an external language.
    • Complicates the adoption of third-party tools: Adding self-contained third-party components, for example, cloud-native, to a single code base with multiple dependencies requires complicated hookups to different layers of the monolithic application.

    What Is Microservices Architecture?

    Microservices architecture is designed to accommodate the need for large application development by providing a suite of modular components and services.

    When considering what microservices are particularly well-suited for, the following advantages and disadvantages should be taken into account:

    Microservices pros:

    • Better organization: Microservices are organized around business and technological capabilities, in units that are independently deployable.
    • Decoupled: Each unit can be independently upgraded without requiring a complete overhaul of the application.
    • Performance: Flexibility in opting to privilege performance or flexibility as needed, including independently scaling heavily used services by isolating them from the remainder of the application.
    • Fewer mistakes: The hard-to-cross boundaries between microservice units limit the technical possibility of generating mistakes, notably cascading ones.
    • Easier onboarding of new developers: They only need to master the microservice unit they will work on instead of the entire application architecture.
    • Scalability: Each microservice can be scaled independently, while also adding new components.
    • Improved fault isolation: As modules work independently from each other, a failure in one module does not affect another.
    • Elimination of vendor or technology lock-in: As each microservice module can be operated independently, replacing underperforming modules with better ones from an alternate vendor is relatively easy.
    • Ease of understanding: As services are broken down into modules, understanding the functionality of a module is easy for developers.
    • Smaller and faster deployments: Each service can be scaled independently as needed, instead of scaling the entire application. This can lead to cost savings.

    Microservices cons:

    • Cross-cutting concerns: Microservices require handling cross-cutting concerns such as externalized configuration, logging, health checks, metrics, service registration and discovery and circuit breakers, as well as those specific to the technologies used by the microservices.
    • Higher operational overhead: Microservices deployment typically leads to an increase in the number of units and containers, increasing the management and operation cost overheads.
    • Security threats: Microservices architecture presents an expanded attack surface. Securing a containerized application requires managing privileged access at image, container, node, pod and namespace levels, ensuring encrypted tunneling from ingress to egress and other elements that require baking in security from the beginning.
    • Multiple complexities requiring management:
      • Communication between services on a technological level, and between teams on an operational level.
      • Selecting the optimal pattern from the several microservices patterns available.
      • Provisioning sufficient IPs.
      • Provisioning resources (memory, CPU, disk) to accommodate each cluster requirements.
      • Maintaining multiple programming languages and frameworks.
      • Handling the incompatibilities between existing tools and new service dependencies.
      • Accounting for automation tools for each service requiring individualized testing and monitoring.
      • Achieving data consistency between each service’s individual database and transaction management system.
      • Handling the number of processes that can grow exponentially when load balancing and messaging middleware are considered.
      • Provisioning for the increase in documentation overhead stemming from the need to keep schemas and interface documents up to date.
      • Refactoring of an application built on monolith architecture is a massive undertaking for large enterprise applications.
    • Complicated global testing: Global testing requires that each dependent service be tested first. This can be partly assuaged with automated testing.
    • Difficulty in debugging problems: Debugging requires inspecting each service log independently, increasing the workload.
    • Deployment challenges: Microservices are written in a variety of frameworks and languages and require distinct resources, scaling and monitoring, making orchestrating the deployment challenging.
    • Large versus small product companies: Though the benefits of microservices are invaluable for large companies, adopting a microservices architecture creates overwhelming complexities leading to slower implementation and unnecessary cost increases for small companies.
    • Higher costs: Without careful and strategic planning, cloud service bills may balloon due to the increase in data stemming from intraservices calls and the increase in resource requests to account for scalability and lack of friction.
    • Cultural changes: Microservices require the company to have an agile DevOps culture.

    As microservices architecture provides a better way to manage growing data sets, leverage cloud storage and automation and deploy solutions fast, microservices are usually preferable to monoliths. However, they might not always be the best choice for an organization. Before opting for microservices, a careful examination of your company culture and business objective is imperative.

  • A New Paradigm in Container Security with Service Mesh

    A New Paradigm in Container Security with Service Mesh

    Containers have been around for some time, but only recently has the technology become mature enough for large enterprises to adopt it both on-prem and in the cloud. The obvious benefits for developers include the ability to dramatically reduce the time to production and the ability to lower the operational burden. This is further supported by implementing a service mesh which provides fast, reliable and secure communication among containerized infrastructure.

    The Rise of Kubernetes and its Impact on Security

    Kubernetes (k8s), introduced late 2014, bring container orchestration at large scale to both cloud and on-prem environments. The benefits of using k8s is clear and stipulated mainly due to the ability to become agnostic to the infrastructure. An application based on k8s can easily scale out from your datacenter into multiple cloud providers, and back. However, the challenge with Kubernetes is it creates a huge problem for security operations. Trying to enforce micro-segmentation on a large k8s cluster using traditional firewall models is not feasible.

    As the move to containers rise, so will container-based cybersecurity attacks. Criminal hackers will always be on the hunt for new ways to exploit this area. As container operations are inherently dependent on the development pipeline (CI/CD) so should be their security attributes. Any other interaction point will be an afterthought resulting in slower operations or reduced security. Organizations moving to containers and k8s should act preemptively due to on-going and increasing complexity driven by scale.

    Enter Service-Mesh

    Implementing a service mesh layer ensures that communication among containerized infrastructure services is fast, reliable and secure. It is a configurable, low‑latency infrastructure layer that handles service-to-service communication. It is responsible for the reliable delivery of requests through the complex topology of services that make up a modern, cloud-native application. In practice, a service mesh is typically implemented as an array of lightweight network proxies called sidecars, that are deployed alongside application code without the application being aware. The mesh provides critical capabilities, including service discovery, load balancing, encryption, observability, traceability, authentication and authorization, and support for the circuit breaker pattern.

    Criteria and Security Best Practices

    • Pre-scanning of images isn’t enough. Enforcing runtime micro-segmentation for container services should be addressed at runtime as they introduce an easier, more prominent attack vector.
    • Easy integration with development pipeline. As noted, the alternative will impact business agility or security quality.
    • Micro-segmentation should be decoupled from the infra and network as otherwise, it will require some kind of application to IP/PORT/protocol mapping.

    Implementing security built for k8s environments and running both on-prem and in the public cloud that uses Istio, a well-known open-source service mesh developed by Google, is recommended. Vet solutions connect directly to the code pipeline with enhanced security capabilities, such as the signing of artifacts that will be deployed to the k8s cluster in the code stage with the ability to generate signed identities. Instead of looking at machines and IPs when workloads boot up, the administrator will see that services will be able to control where they run.

    This will also enable you to intuitively create identity-based micro-segmentation. This is an advantage since micro-segmentation was always a complicated task to achieve, requiring administrators to map an application to the network while keeping track of what is running in the environment.

    With identities and a service mesh layer, it is only necessary to focus on your business logic. The fact that identities were generated during the CI/CD pipeline, provides the ability to control the application without the need for service discovery. Identities are represented by their business-context names, originating in the CI/CD pipeline, so the operator can easily group them together and assign policies at different layers of granularity.

    K8s is a great technology that enables the harvesting of containerized environments. Adopting it can accelerate your organization’s digital transformation dramatically, but it can also require you to adopt new methods of application security. Contracting with a k8s-focused security organization will allow the business to accelerate its migration to cloud-native applications on-prem and in the cloud, in a most automated fashion.

    By providing the needed security and operational capabilities, k8s clusters admins are afforded an enterprise-level micro-segmentation solution with policy orchestration, single pane of glass to your clusters in a multi/hybrid cloud deployment and the ability to connect between containerized clusters and legacy applications, as well as other resources that are external to the cluster.

    — Ran Ilany

  • Common Cloud Security Mistakes and How to Avoid Them

    Common Cloud Security Mistakes and How to Avoid Them

    Over the last few years, it’s become apparent that traditional on-premise security policies are not a good fit for newer cloud-native environments. Even though the writing has been on the wall for a long time, the brewing security crisis still hasn’t been resolved. Far too many organizations continue to use security measures that are ineffective for cloud-native environments, along with inconsistent policy enforcement that leads to huge oversights and gaping organizational loopholes. The sad truth is that today, it doesn’t take a master hacker to infiltrate most cloud-native deployments.

    Many of the most infamous security breaches would have been extremely difficult to prevent even with an ideal security environment, but the weakness of the system is endemic, reaching far beyond a handful of vulnerable organizations. Even if some malicious actors could still get through, there’s no reason for organizations to leave their attack surface so blatantly unprotected.

    Here are increasingly common cloud security mistakes DevOps teams still make, and our suggestions for how you can stop them in your organization.

    Ignore Zombie Workloads

    Many organizations ignore zombie workloads that run on their architecture. When your team is stretched and you’re facing a number of serious security issues, it’s easy to dismiss zombie programs as nothing more than a harmless irritation.

    In actuality, having zombie resources scattered around your deployment blocks your ability to spot serious cryptojackers. Zombie workloads are often an indication of, and an invitation to, more significant invasions into your infrastructure. A report by SkyBoxSecurity revealed in 2018, cryptojacking was the leading attack vector. DevOps teams need to recognize the threat of hosting a program mining your resources for cryptocurrency, and take immediate steps to block them.

    Turn a Blind Eye to Leaky S3 Buckets

    AWS service buckets, particularly the S3 bucket, is one of the oldest cloud-native services. As a result, many S3 buckets are still governed by very old rules and regulations that leave many ways for someone to accidentally make the bucket public, open up access to unauthorized users and expose sensitive data.

    A study by HTTPCS reports 58% of AWS S3 buckets are accessible to the public, and 20% of them aren’t write-protected. Not only can malicious actors get to your sensitive customer data through S3 buckets, but often they can access your cloud credentials as well. Some of the most disastrous data breaches began from unrestricted access to S3 buckets, and it’s one of the most common causes of data breaches overall. It’s vital to regularly check your AWS console for public buckets.

    Allow Runtime Updates to Bypass the CI/CD Pipeline

    Every DevSecOps team agrees that your deployment is more secure when runtime updates are passed through the CI/CD pipeline, but that doesn’t mean the policy is always enforced. Developers will repeatedly ignore your security policies through practices, such as using open-source libraries, to avoid the security process of the CI/CD pipeline and speed up deployment.

    While this saves developers some time on releasing updates, it places a heavier burden on security teams, who have to run extra scans for these rogue workloads. What’s more, it builds complacency among DevOps teams, who assume there is no way to prevent unauthorized workloads and end up simply accepting them. Eventually, your security posture erodes to the extent that malicious actors can run harmful workloads without attracting your attention–until it’s too late.

    Permit Unrestricted Network Access

    Instead of spending hours on segmentation and separate access permissions, all too many DevOps teams fall back on a blanket set of network configurations that fall far short of the necessary access restrictions. They often place all their workloads into a single VPC, opening up access to third parties in the process.

    Without restrictions on public network access, it takes far longer for security teams to identify and isolate negligent and malicious activities. In a short space of time, DevSecOps teams find themselves overlooking serious oversights like unrestricted public root access, opening up enormous holes in their security profile.

    Faulty Rule Configurations for Micro-Segmentation

    When DevOps teams use micro-segmentation in containers, they face unique challenges. The more granular you get with your segmentation, the greater the risk you’ll miss faulty policy rules.

    Even your most familiar rule sets can create massive vulnerabilities. For example, if you allow your developers to use a specific IP to connect to the production runtime environment through SSH, you could unwittingly end up permitting unrestricted public access to sensitive areas. Sometimes, these faulty rule configurations go overlooked for as long as several months. To avoid supporting faulty rules, it’s important to audit them regularly with a tool such as Amazon Inspector’s Agentless Network Assessment.

    Avoiding AWS cloud security mistakes requires a shift in mindset. The harsh truth is a significant number of cloud-native deployments commit not just one, but several of the mistakes listed above. Cloud environments are deliberately decoupled from the network infrastructure in order to enable agile business practices, but DevSecOps teams continue to use network security techniques that aren’t capable of meeting the unique challenges of the cloud.

    The only way to fix these mistakes is to turn to new technologies designed for cloud security, such as using cloud workload identities to bridge the chasm between business logic and network infrastructure. Until ACLs and security groups are untethered from the network infrastructure and join cloud-agile business logic, they’ll fail to provide adequate cloud-native security.

    — Ran Ilany

  • Can DevSecOps Prevent a Zombie Apocalypse?

    Can DevSecOps Prevent a Zombie Apocalypse?

    Making consistent progress within any DevSecOps initiative often can be an overwhelming undertaking. Developers tend to outnumber operations dramatically, with little to no security accountability. This leaves DevSecOps doomed to work with the same network security stack that was originally designed decades ago for on-premises environments. Still, the expectation is to support shrinking deployment cycles on cloud-native and hybrid environments without compromising on security standards.

    To deliver on such a tall order, DevSecOps teams are armed with security groups and access lists (which are more of the same distributed firewall technologies) to secure a perimeter that essentially no longer exists. This article will help you figure out how much time you have left before your CI/CD pipeline is overrun by zombies (quite literally) and what you can do about it.

    Why Cloud-Native Environments Choke on Rule-Based Network Management

    Rule-based management of network infrastructure has worked for decades—and no doubt will continue to support many enterprises well into the future, unless your production environment includes a public cloud footprint that supports continuously deployed release cycles. That’s when things start to get a little weird. Instead of letting go of the binary perimeter premise (inside versus outside) and redefining it completely, security practitioners have essentially re-created multiple virtualized perimeters in the public cloud. In theory, you could spin up at least as many perimeters as you have running instances. Unfortunately, the same network rules that worked well for the one perimeter break down in a virtualized environment that has hundreds of instances supporting dynamically changing requirements. In fact, the very nature of the cloud is driven by business logic at the application layer, which is decoupled from network infrastructure by design. This creates a Catch-22 situation for cloud-native security pros, forcing DevSecOps teams to manually map network rules to highly agile and virtualized business logic. No human can realistically deliver results over time (and stay sane). The reality is, rule-based network management can never keep up with the requirements of cloud-native business logic.

    With Great Agility Comes Security by Bureaucracy

    Regardless of how skilled DevSecOps teams are, or how motivated they are to lead the brave new world of continuously secure deployment, the harsh truth tends to be disappointing. While DevSecOps presumably aspires to deliver agility without compromising on security, practitioners spend much of their time doing just that. Trapped by a security culture that has favored bureaucracy over automation for decades, DevSecOps tends to perpetuate the very dichotomy that triggered the emerging practice. In a cloud-native culture driven by innovation and automation, agility comes first. This inevitably becomes an exercise in futility—the more agile your deployment, the more vulnerable you become. That means giving up on the very idea that rule-based network management is a viable security methodology. The complexity of managing several dedicated security groups (up to five) for every instance rapidly spins out of control. Given the imperative to maintain the cloud’s agile environment, many teams consciously sacrifice security on the altar of agility, leaving a gaping attack surface vulnerable to unrestricted access.

    Welcome to Cloud-Native Multi-Perimeter Security

    The challenge is to prevent any unauthorized code (benign or otherwise) from infiltrating cloud deployments before any sensitive data or workloads are compromised. In the absence of robust automation to support this process, the challenge is often ignored. When that doesn’t work, the solutions often remain in the realm of bureaucracy, focusing on anecdotal symptoms without addressing the root cause.

    The inevitably grim question is not if your deployment is at risk, but how vulnerable you are and how soon is likely too late. The results are measurable and often disturbing. Below is the standard KPI checklist most DevSecOps are familiar with:

    1. Rogue workloads unauthorized by CI/CD pipeline. These are typically addressed by discovery tools and ad-hoc network scans but are all too often accepted as a necessary evil.
    2. Faulty network rules. Microsegmentation initiatives often lead to unexpected inter-workload communication.
      Workloads deployed via the CI/CD pipeline could fail to communicate effectively—or worse, communicate to support counterproductive or even malicious activity.
    3. Unexpected workload communication to Third-Party PaaS. A case in point is workload communication to a database such as S3, which could lead to a data breach. Well-documented cases include leaders, such as Accenture and Time Warner, that unwittingly enabled public access to S3 buckets storing sensitive customer data and access credentials.
    4. Network Security Configuration Mistakes. Vulnerabilities introduced by network security configuration often go unchecked for months. Such unrestricted access is commonly cited as the attack vector that enabled unauthorized actors to infiltrate a gaping cloud-native attack surface.
    5. Zombie Workloads Consuming Resources. At best, you’re just wasting resources. All too often, one of those zombie workloads is busy cryptojacking, mining cryptocurrency on your company’s buck unnoticed. Cryptojacking is so rampant, it has overtaken ransomware in 2018 as the most popular attack vector.

    The bad news is, no cloud deployment can eradicate the above issues entirely, using familiar technologies. However, you can mitigate some of the damage: Scan your network more regularly using your favorite open source tool (check out a great example implemented on Netflix Security Monkey presented by Ryan Hodgin). Never leave an entire network flat granting anyone unrestricted access. Ideally, take the time to evaluate new technologies designed to secure cloud-native deployments.

    DevSecOps teams are gradually accepting that cloud-native deployments require a fundamental paradigm shift to sustain a defensible security posture without sacrificing agility. Many believe (this author included) that we are witnessing a cloud-native security revolution, much like the one triggered by the internet in the 1990s. DevSecOps can certainly address the symptoms described above, but security policies will continue to fail miserably if they both remain entrenched at the network layer and rely on bureaucracy.

    — Ran Ilany