devopsy.kinsta.cloud

Category: DevOps in the Cloud

  • Pulumi Deepens Cloud Ties to DevOps

    Pulumi Deepens Cloud Ties to DevOps

    Pulumi has made available a framework for automating the management of multiple cloud computing platforms using familiar programming languages such as JavaScript, TypeScript, Python and Go.

    Company CEO Joe Duffy said Pulumi 1.0 provides a framework that enables developers to automate the deployment of applications on public clouds in a way that eliminates having to learn another programming tool or manually employ YAML files that are often at the root of misconfiguration issues. Pulumi also provides access to a software development kit (SDK) through which scripts or code developed for Terraform, CloudFormation, Azure Resource Manager, Kubernetes or Helm can be imported into the Pulumi framework.

    Pulumi also has extended the reach of its automation framework to provide integration with a variety of continuous integration/continuous deployment (CI/CD) platforms including Azure DevOps, AWS Code Services, CircleCI, Codefresh, GitHub, GitLab, Google Cloud Build, Jenkins and Travis. Pulumi also can be embedded with integrated development environments (IDEs) to provide statement completion, interactive documentation and rich error-checking.

    Duffy noted Pulumi also now can be extended to further adoption of best DevSecOps practices. A Pulumi Key Management Service is integrated with the key management tools from Amazon Web Services, Microsoft Azure, Google and HashiCorp. Pulumi also provides role-based access controls and integration with identity management tools spanning GitHub, GitLab, Atlassian, Active Directory, Okta and Google G Suite, among others.

    Pulumi comes in three flavors: A free Community Edition enables individuals to manage their cloud environments, a commercial Team Edition adds support for organizations and an Enterprise Edition provides additional identity options, policies and controls, as well as self-hosting capabilities.

    Duffy said frameworks for automating application deployments on multiple clouds are necessary because enterprise IT organizations now routinely embrace multiple clouds. Rather than requiring IT operations teams to master programming environments, Pulumi makes it possible to shift deployment responsibilities on to the developer, he said, noting that approach should significantly increase the rate of speed at which applications can be deployed using cloud platforms such as AWS, Azure, Google Cloud Platform, Digital Ocean or others.

    While a lot of progress has been made in terms of continuous integration, organizations are finding continuous delivery to be a challenge. Pulumi is betting that by directly involving more developers in the deployment process, organizations will find it easier to extend their DevOps processes in a way that doesn’t require anyone to learn new programming tools.

    In the meantime, the rate at which new applications are being built is only going to accelerate. As development teams embrace a wide range of cloud-native technologies, it’s only a matter of time before the application deployment backlog increases significantly. IT leaders are going to be under a lot of pressure to solve that issue in a way that results in more applications not only being deployed on a timely basis but also deployed reliably. Having more applications running in a production environment that winds up having issues later isn’t necessarily progress. In fact, it’s likely most IT leaders would find themselves in even hotter water for exacerbating cloud security issues that already plague most organizations.

    — Mike Vizard

  • Why ITOps Is Moving to the Cloud

    Why ITOps Is Moving to the Cloud

    There’s been a lot of movement in the IT operations management (ITOM) business lately, from the acquisition of SignalFx by Splunk to the PagerDuty IPO, and all signs point to a Datadog IPO in the future. What’s with all this consolidation? I believe we’re seeing the rise of a future-state of ITOM; that is to say, it’s the rise of SaaS-based ITOM. And it’s easy to see why.

    In my previous consulting career as lead enterprise systems architect, our team had an impeccable record in designing and implementing well-architected hybrid infrastructure solutions. We maintained an immaculate record and near-flawless customer satisfaction record. By project sign-off, our job was always done. And yet, returning to the same solutions six months later told a different story entirely.

    Well-architected solutions are similar to human bodies: They are perfect when they’re born but need constant care and feeding. These same solutions that satisfied SLAs, exceeded expectations and transformed organizational efficiency can easily degenerate, and just like our bodies have their nervous systems to monitor, brains to send alerts and tissue to self-heal, well-architected systems need operational maintenance to keep them humming.

    The traditional approach for solving this eternal need was and still is to design and implement well-architected IT operation management (ITOM) solutions, around a well-architected infrastructure. And yet, this is a self-fulfilling paradox because the ITOM solution itself needs the same care and feeding.

    There’s a Problem On-Premise

    ITOM is a broad term encompassing application and operating system (OS) performance, alerts, log management, notification, asset configuration, incident management and more. It typically involves the purchase of a suite of on-premise point tools addressing each need, and then to develop an internal framework to help those tools interoperate in a meaningful way. While that is possible conceptually, the facts on the ground reflect a very different reality:

    • Multi-vendor tools are often not designed to work together.
    • Creating an internal logical framework that orchestrates various teams and technologies can be very complex in large enterprises.
    • It’s near-impossible to create technical integrations flexible enough to adhere to inevitable organizational and technological changes that will affect this logical framework.
    • Predicting cost-of-ownership is nearly impossible since each tool is controlled by a different vendor, and the internal integration effort is often unknown.
    • Predicting the cost of the manpower required is also very difficult, as each tool requires its own set of specialists, in addition to integration specialists to make it all work together.
    • Upkeep is often overwhelming, as vendors offload software patches, upgrades and on-premises hardware costs to the customer.

    In the face of all these challenges, the end result is often unrealized value, overwhelmed operational teams, loss of service and inability to accommodate new technologies resulting business service disruptions.

    Why Cloud? Why Now?

    It’s historically been near-impossible to build a traditional ITOM platform on-premise. Vendors typically sell a collection of white-labeled tools cobbled together by acquisitions, and this is far from a platform. The complexity and the rate of technological change make it difficult to provide consistent quality and value across the various product lines. This puts the ITOps team squarely in the middle of a pickle: How can they ride the wave of a changing environment without relying on static tool suites?

    The Future is Flexible

    Enterprise ITOps has been stretched now more than ever. There is a serious skills gap, shortage in IT workforce and ever-increasing technical complexity. Time and resources are precious, and Enterprise ITOps need simplicity and predictability along with flexibility and control. Enter SaaS ITOM. By moving the ITOM function to a SaaS orientation, the responsibilities, workloads and daily tasks can transform according to the needs of the organization:

    Keeping Up with the Business

    SaaS ITOM can keep up with technological change, and keep pace with cloud, DevOps, artificial intelligence and more. In the world of SaaS, change is an accepted constant and not an inconvenience. What’s more, SaaS ITOM is infinitely more consumable than the tool suites of legacy past, and that reduces the learning curves associated with running ITOps.

    Keeping Up with Industry Needs

    A SaaS ITOM platform will be able to deliver a framework that’s both flexible and governed and can accommodate technical and organizational complexities. This agility is a feature of modern SaaS. SaaS ITOM can also integrate features running on a single code base supported completely by the SaaS vendor, who will absorb maintenance and upgrade cycles, freeing considerable and valuable time back to the operator. All of this results in a more predictable total cost of ownership, improved service quality and more value to the business user.

    It’s not news the world of IT is moving to the cloud. It is news, however, that cloud can offer such transformational benefits in ways we’ve never seen before.

    — Wael Altaqi

  • From DevOps to DevSecOps: Owning Cloud Security

    From DevOps to DevSecOps: Owning Cloud Security

    The cloud is a complex environment, with different groups managing different cloud services and environments. The most basic division is between the cloud provider, who provides the core infrastructure, and the customer, who leverages services or builds their applications using resources leased from the provider. Within larger organizations, different cloud resources, data and applications may be owned by various lines of business, each with their own budget and development resources.

    From a security perspective, this sort of divided ownership of the cloud can create severe challenges in terms of end-to-end visibility, control and compliance. Of course, the cloud is a shared infrastructure, and when it comes to security events, there will likely always be some shared responsibility. But the lack of a centralized security strategy and inconsistent security policies and standards leading to potential grey areas of responsibility can create serious security gaps, and put critical data and other digital resources at risk.

    Cloud Provider vs. Customer Responsibility

    When respondents to a recent survey by IHS Markit were asked, “who is truly responsible when there is a vulnerability or security event in their cloud deployment?” the answer reflected a lack of understanding of the basic division of responsibility. Even in the best-case scenario, where it should be clear as to who is responsible for a security event (such as a vulnerability in the cloud platform), only about half of respondents were able to identify the root cause and responsibility.

    Complicating things further, there is yet another set of tools as to which responsibility should be clear. The list of security vendors available as add-ons through a cloud provider’s marketplace is exhaustive. The cloud providers themselves also offer a range of security solutions (built into their platforms as services). And many of the cloud management platforms, hypervisors, orchestration tools and other bits of cloud infrastructure have various levels of security available.

    However, the truth is adhering to the shared responsibility model, whenever the customer applies any configuration to these tools, they become his responsibility. Nonetheless, divisions of responsibilities are not always clear for different events.

    The best course of action is to identify and apply best practices to any resource and service consumed in the cloud. A root cause analysis (RCA) playbook should identify possible threats and plot actions where possible, and plot strategies for determining the root cause of an event. This includes identifying all key stakeholders, determining which investigation tools are available for threat analysis, establishing processes for orchestrating responses and resources between all parties and digitize and map those details to actual cloud technologies. Further, any response should be automated as much as possible.

    Security Responsibilities Within the Customer Domain

    Once the primary divisions of responsibility for a dealing with a security event is worked out between providers, vendors and customers–such as determining if a security breach is the result of a corrupted service or a misconfigured system–most cloud customers still have to address the divisions of responsibility within their own organization.

    The challenge is, when it comes to cloud security, too many companies don’t have a good handle on who is responsible for what. Part of the challenge is many organizations have taken an ad hoc approach to cloud development. As a result, today’s organizations, with an average of five different cloud environments in place and two or more in development, may be hamstrung by the challenge that each cloud instance might be owned by different lines of business.

    For many reasons, these groups may be reluctant to turn over the security of their individual cloud environment to the central security team. For example, speed is a critical component of DevOps efforts, so any security implementation that interferes with their primary objective of delivering applications is going to be a problem.

    Part of the reason is traditional IT security teams rarely know much about cloud environments, and the security tools they recommend often bottleneck development. However, handing over to the DevOps teams building out cloud applications and environments is problematic as they have little to no expertise when it comes to security.

    For example, one of the biggest challenges of many cloud-based security tools is the time and skill required to properly configure and manage them, let alone adjusting those configurations to eliminate false positives, or to ensure they don’t miss critical events or drop legitimate traffic. All that requires expertise most DevOps teams don’t have.

    Shifting from DevOps to DevSecOps

    Resolving this challenge can be as simple as adding a cybersecurity specialist to each of the various DevOps teams. Once integrated, DevSecOps can help navigate the fine line of the shared responsibility model to ensure both development and security requirements are met. They can negotiate between the customer/vendor/provider responsibility model, provide strategies for ensuring consistent security across and between cloud instances, and balance protection with performance.

    Part of that is achieved through the selection, deployment and management of tools designed to meet the twin challenges of speed and security. For example, many SaaS-based security solutions, such as web application firewalls, are self-scalable, allowing web applications to grow as needed without compromising protection. Choosing the right tool can also ensure it can be easily stitched into an application transaction with minimal effort. Some of them even have the advantage of having already included deployment, maintenance, scaling and fine-tuning functions.

    The proper application of automation plays an equally critical role in this process. Checking configurations and scanning objects for malware are time-consuming activities that can get pushed to the side when other issues arise. Automated cloud security solution selected and designed by the DevSecOps team can provide comprehensive configuration assessments, dynamically secure stored data, automatically scan for public cloud vulnerabilities, identify misconfigurations that put data at risk, scan files stored in the cloud and prevent the unauthorized downloading of sensitive information.

    Building a DevSecOps Team Requires Executive Support

    DevOps has become part of C-suite and board-level discussions, attesting to the growing critical value of web applications and digital transformation as part of the broader business strategy. However, if the frequency of breaches and the growing concerns of CISOs are any indication, executives aggressively pushing for cloud solutions often have a mistaken understanding of the nature of the security risks that cloud adoption and careless DevOps programs can introduce into their organization.

    Therefore, it is critical executive leadership is educated on the risks associated with expanding an organization’s digital footprint, and as a result, its new potential attack surface. Expanding DevOps to DevSecOps enables the integration of security into new cloud environments on day one. It can also ensure RCA cloud security playbooks are developed and followed, and help introduce and manage security solutions designed to protect critical digital resources without introducing unreasonable risks. Finally, it can provide a much-needed buffer against fines and penalties associated with regulatory requirement violations should their cloud deployment become a conduit for a severe system breach.

    — Lior Cohen

  • Out of Date: Cloud Anti-Trends of 2019

    Out of Date: Cloud Anti-Trends of 2019

    Let’s talk about 2019 cloud tech anti-trends. A lot of those anti-trends are nothing but cloud myths, which are still widely spread and talked about.

    In this article, we’ll take a look at some anti-trends and try to understand where’s the myth part in them.

    Private Cloud Is Not Popular Anymore

    According to the IDC research organization, by 2021, companies from around the world will spend more than half a billion dollars on cloud tech. This is two times more than in 2016. At the same time, there is a clear tendency: The public cloud workloads are growing, while the situation is exactly opposite for the private cloud. Cisco said, after three years 73% of the company’s applications will move to the public cloud, while just a year ago this figure was around 50%.

    Such a shift is partly caused by cloud providers, who offer users of public clouds more and more features that simplify the development of services. For example, tools that allow creating machine learning models, manage software deployment processes and analyze logs.

    That’s why many felt the active development of a public cloud meant the final decline of the private cloud niche. During his speech at a conference last year, Anthony Lai, senior vice president of the cloud-based division of NetApp said private clouds would “die out.”

    I wouldn’t expect the private cloud to disappear completely. The statement that it will cease to exist is only partly true. The private cloud may disappear as an independent unit, but it will remain part of the hybrid IT infrastructure.

    As noted in Gartner, 72% of companies in the world use a hybrid strategy. And it is expected this figure will grow. According to analysts from Market Research, the market for cloud object storage systems will reach six billion dollars by 2023. This will definitely spur further development of the hybrid IT infrastructure.

    One reason to use private clouds is predictable business processes.

    Yes, the public cloud offers flexible scaling. The company does not have to urgently buy new hardware if the load on the services has increased dramatically (for example, the application became a success and tons of new users came in). In this case, you can simply rent additional computing power. But if the load can be predicted (say, it is static) a private cloud is often enough.

    Also, a private cloud is often used to create object storage. Sure, object storage services are also available in the public cloud and a number of companies use them. Just a couple of years ago, Netflix migrated its entire infrastructure to the public cloud.

    However, not all companies need to build a distributed infrastructure as Netflix did. There are cases when companies have object storage in a private cloud because it already has an appropriate low-cost automated infrastructure. CERN is one of them. All the data generated by the Large Hadron Collider is stored in a private cloud.

    The IT-Infrastructure Market Is Oversaturated with Specialists

    Today, companies are actively abandoning their own infrastructure in favor of cloud solutions. That is why the need for in-house specialists engaged in its maintenance, gradually disappears. The IaaS provider takes on many issues related to the management and administration of the infrastructure, and the specialists need only ensure that everything is set up correctly.

    That is also why companies often abandon their own IT departments. At first glance, it seems this will gradually lead to instability in the labor market. Experts in the field of IT infrastructure, system administrators and data center operators will have to compete fiercely for the remaining jobs. But will they really have to?

    This does not mean such specialists have become completely unnecessary. In reality, in-house specialists simply change the focus of their activities. Most of the IT infrastructure administration tasks are now on IaaS and IaaS providers are now the ones who need and are actively fighting for such specialists.

    Even when you do use third-party solution, that is what happens sometimes:

    When customers start using intermix.io for the first time, they can see the set-up and configuration of their Amazon Redshift cluster in the context of their queries and workflows.

    At that point, customers experience one common reaction: Knowing what we know now, how would we set up our Redshift cluster had we do it all over again?

    The answer is not that straightforward.

    However, sometimes it happens that migration to the cloud stimulates the company to strengthen and develop the IT department more, attracting new cloud employees. These people study the cloud ecosystem, analyze the trends and evaluate how the transfer of an application to the cloud will affect the company’s work. If the company is already working with the cloud, then cloud specialists are also exploring new features of the current platform and exploring services that will benefit a business in the future.

    Clouds Don’t Hold New Technologies

    Four years ago, IBM listed seven cloud technologies to help businesses. The list includes IaaS/PaaS, private/hybrid cloud, test and application development environments, big data processing and disaster recovery solutions, along with data storage and backup systems.

    These technologies served as a standard technology stack for a long time. It was all developed with the efforts of IaaS providers. Therefore, there was an opinion that the cloud ecosystem was finally formed and no other use cases will be foreseen in the near future.

    New Directions Still Appear: The blockchain technology was born in 2008, gained worldwide popularity 8 years later and came to the cloud about a year ago. Based on the virtual infrastructure, a new direction has started to develop: Blockchain-as-a-Service (BaaS). In this case, the client receives a virtual environment prepared for working with blockchain technology. Users can create their own applications on the blockchain (for example, for processing financial transactions) and run them in a secure virtual environment.

    Quantum Computers: the most recent, fresh cloud trend. The quantum machine uses qubits instead of bits we are used to. Unlike bits, qubits can take the value of zero and one at the same time. Thus, a quantum machine can be in several states at once. The more qubits a quantum computer has, the faster it performs the tasks assigned to it. Imagine how much faster it can brute-force passwords or simulate chemical reactions.

    Quantum computers already have practical applications. IBM has recently opened access to its 20-qubit developments via the IBM Q cloud platform. And this year, Alibaba partnered with the Chinese Academy of Sciences to deploy an 11-qubit quantum device in the cloud. Quantum Computing as-a-service (QCaaS) opened the way for experiments on the infrastructure of providers and proved the cloud industry can adapt to the emergence of new technologies.

    Another Trend Is Fogging or Fog Computing: IT-industry experts are trying to solve the problem of delay in transferring data to the cloud. In the case of fogging, server components are not located in the centralized data center, but on the periphery, close to the end-user devices.

    In such an architecture, a main, large cloud still remains but has additional networking and computing layers: cloud, fog, edge. The Cloud (cloud layer, primary) solves expensive tasks for periphery machines (edge layer, end-user): data analysis, system logging, etc. The fog layer does local calculations in real-time to generate reports and send them to the central hub. This architecture improves the performance of the entire infrastructure by significantly (up to 10 times) reducing the delay in data exchange.

    To Sum It Up on Cloud Myth-Busting

    So, as with everything, rushing into extremes doesn’t help. That’s what we figured out:

    1. Even though public cloud seems to be a mainstream option, private cloud infrastructure still makes sense for certain use cases.
    2. Most of the IT infrastructure administration tasks are now on IaaS and IaaS providers are now the ones who need and are actively fighting for Engineers i.e. market is more than far away from being oversaturated with specialists.
    3. Clouds don’t hold new technologies. Did anyone really believe this one anyway?

    — Dmitrii Borodin

  • Bringing 5G-Enabled Digital Services to Life Calls for Cloud-Native Operations

    Bringing 5G-Enabled Digital Services to Life Calls for Cloud-Native Operations

    Spotify, Netflix, IoT connectivity and cloud storage are a few examples of the digital services providing new options for fun and function to consumers and businesses alike, while representing a growing boon for communications service providers (CSPs).

    Gartner predicts by 2020, nearly one quarter (23%) of CSPs’ large enterprise business communication revenues will be driven by digital services—up a staggering 10 percentage points from 13% of revenues in 2017. This gain should come as no shock to developers who favor building in the cloud or using tools such as Microsoft Teams to collaborate, or who stream video from HBO Go or skim stories in Apple News during their commutes. Today’s customers (consumers or businesses) expect value-driven, digitally enabled services that inspire personalized and meaningful experiences with their brands of choice.

    The good news is companies are eager to meet demand: CSPs want to deploy services in the cloud to address the challenges of web-scale companies. In fact, 85% of companies surveyed say launching new digital services is critical to their business strategy.

    What’s the catch? To do it, they’ll need to be agile. Although telecom is not an industry known for its agility, and is often thought of as the opposite, it has been transforming—like many industries—through adopting more agile approaches in business strategy. CSPs are striving to maximize resources and reduce inefficiencies, while at the same time sparking innovation. Nothing has been more of a stimulus for agility than the cloud, especially for the web-scale companies whose services pose the greatest threat to the telco industry.

    A key requirement will be cloud-native applications that improve the agility of network functions. However, being cloud-native involves a lot more than just shoehorning code into a container previously ported from an appliance to a virtual machine.

    Rather, at Oracle, cloud-native means faster development and deployment through DevOps and continuous integration and continuous delivery (CI/CD) pipeline. CSPs can only achieve an agile cloud with an environment enabling developers to innovate true cloud-native solutions if they implement an operational framework that embraces a key set of principles. Specifically, we at Oracle have identified the principles for successful cloud-native operations.

    System Immutability

    Everything, including software and configuration, is code. All changes are made through CI/CD, where they are deployed as immutable artifacts. No manual configurations or customizations are allowed. This makes it easier to implement the principle of least privilege as there is no need to run scripts in the production environment. Also, any changes not coming though the delivery pipeline can be considered malicious.

    Automate Everything

    All aspects of build, test, verification and deployment are automated. This includes activities such as backup, recovery, password/key rotation, etc. Fully automating the DevOps pipeline, including verification and testing, removes much of the potential for human error. This automation also allows changes to be applied to the environment with confidence and provides for rapid repair.

    Disposability

    All services are transient and treated as short-lived. Instead of focusing on never failing, services are designed to go up and down quickly without service interruption. Regular repaving (re-deployment) of the environment ensures failed or failing services are removed and new ones deployed.

    Externalized Configuration

    Configuration (including passwords, credentials, location of backing stores, etc.) is decoupled from the software image, and like software, can be treated as a build artifact in a controlled and versioned manner. Versioned configuration enables development and production parity as an artifact and can eliminate costly operational errors.

    Logs as Event Streams and Constant Telemetry

    Everything needed to debug or diagnose any functional, operational or security issue is found in logs, traces or metrics data. These are treated as a stream of time-ordered events and stored in a centralized collector outside the system, enabling better threat monitoring, forensics and diagnostics through event correlation or analyzing the aggregated and holistic view. Constant telemetry is fundamentally enabled through logs as event streams and, thus, the two should always be considered together.

    Delegated Governance

    Some shared aspects of the environment are centrally managed like networking, identity management or infrastructure, but in a true DevOps fashion, teams delivering a service are responsible for operating the service. This is allowed only with strict governance, enforced through checks in the continuous delivery pipelines, giving greater control over rate of change. Business agility is improved as applications have better visibility into the operations of their service with tighter feedback loops that ultimately improve quality.

    Independent Lifecycle

    Independently upgrading, scaling and deploying each microservice is paramount for supporting other cloud-native principles as well as minimizing the amount of change in the system at a given time. Furthermore, such decoupling makes other principles, such as repaving, easier and promotes easier isolation of issues.

    As 5G introduces increased connectivity and speed, along with the need for more security and reliability, service providers will need the agility, flexibility and security of the cloud and cloud-native principles. These principles are enablers for better cloud operations. Together, they provide the ideal context in which to implement a broader view of the Cloud Native Computing Foundation (CNCF) landscape, giving developers a robust Platform-as-a-Service of backing services beyond just containers.

    As 5G moves from a fabled future to a reality, the market for digital services that reap its benefits is likely to continue to grow. By leveraging these principles, along with DevOps and the CI/CD pipeline, organizations will be able to evolve and prepare for 5G’s promise for innovative business models and new revenue streams—be they digital services or beyond.

    — Shirin Esfandiari

  • Autonomous Security in Containers

    Autonomous Security in Containers

    With the advent of DevOps, the development world has quickly moved to agile development practices and containerized applications. At Forum Systems, we have responded to this trend by putting our API security software, Forum Sentry, into virtual form factors such as Amazon Machine Image, Azure Image, VMware Image, Linux, Windows and Docker.

    Why do we feel it’s necessary to containerize API security in this way, and what benefits does it deliver over traditional software or even hardware-based API security?

    What Is API Security?

    Before we explain why we containerized our API security software, it is first necessary to define exactly what we mean by API security. As a result of the major IT trends of the last decade–in particular cloud, mobile and IoT–more people and applications are connecting to IT assets than ever before.

    What’s more, the majority of these interactions are from untrusted entities outside the organization’s network perimeter (and as you will read below, across container boundaries). Securing your organization’s systems, data and business-critical processes is harder than ever.

    Almost every interaction relies on an API to communicate to an application or system somewhere in the world. The simplicity of APIs makes it easy for developers to connect their projects to other systems to enhance their functionality, with data being easily shared between a myriad of external partners, cloud providers, virtualized data centers and on-premise applications.

    As a result, APIs have become the primary channel for business transactions and the traditional boundaries of data exchanges have become blurred. APIs provide many benefits but also present risks, as wherever there is innovation, there is also the dark side of threat and attack, and always someone who will aim to exploit weaknesses.

    In many environments, APIs and their underlying technologies are designed primarily to share data; they are not designed to thwart threat and attack. This is where API security gateways are necessary to validate both the data and the identity of users, systems and devices interacting with the API.

    API security goes far beyond just access control, it requires specialized technology to perform dynamic data security to inspect and ensure the specific and unique characteristics of each API communication are correct. The API security policies are designed to prevent intrusion, data leakage and other forms of data loss as a seamless and transparent part of the technology architecture.

    API security gateways differ greatly from API gateways; the key word “security” immediately distinguishes this technology from simple API gateways, which serve only to provide proxy and simplified access control points. API security gateways are purpose-built to protect against API threats, and the dynamic data security capabilities they offer provide centralized protection of bi-directional API communications. API security gateway technology provides essential risk mitigation at each layer of the API architecture, and as rising adoption of containers and micro-service architectures arise, it provide zero-trust concepts within and across the container boundaries to protect the assets and technologies therein.

    Why Containerize API Security?

    Container technology such as Docker has become a popular means to deploy API micro-services, a collection of loosely coupled services which are fine-grained and lightweight. The move toward virtual and cloud was initially driven by fully virtualized images with their own operating system, but the adoption of lightweight services and on-demand environments has led to widespread adoption of container technologies run on a shared operating system. These container architectures provide flexibility and ease of deployment, but come with the same set of API risks.

    The concept of containers also brings into focus the trust model that needs to be considered for communication among technology components within and across containers. Containers become a new API boundary layer and thus represent the same type of risk paradigms, where a rogue container application can gain access or otherwise wreak havoc if the proper security controls are not put into place (commonly known as an insider attack).

    By deploying API security directly into a container, organizations can automate their API security into their existing workflows and provide zero-trust security capability that ensures real-time enforcement and protection of container-based APIs. This means API security becomes autonomous and baked into the architecture, rather than tacked on at a later stage.

    As mentioned earlier, dynamic data security is essential wherever information is traversing. The traditional cyber umbrella approach no longer applies in the interconnected API world. It’s like using an umbrella in a water park, it may stop the water from the top, but not from the sides. By having automated API security at the container and virtual layers, the ability to visualize, connect and secure information becomes an integral (and necessary) aspect of the container security strategy and a fundamental baseline for secure API enablement and trusted data exchanges.

    API security gateways deployed as container nodes allow DevOps teams to harness the power of virtualization, cloud computing and containers while at the same time protecting the organization against API threats from both the inside and the outside. With API security baked into the container, developers are free to focus their time on building the best functionality in their apps.

    — Jason Macy

  • Virtual Instruments Acquires Metricly to Monitor Cloud Costs

    Virtual Instruments Acquires Metricly to Monitor Cloud Costs

    Virtual Instruments has acquired Metricly, a provider of a software-as-a-service (SaaS) application that help organizations to optimize the deployment of cloud application workloads.

    Tim Van Ash, senior vice president of products at Virtual Instruments, said Metricly employs machine learning algorithms to identify opportunities to improve performance and reduce costs. Those capabilities complement an existing VirtualWisdom platform from Virtual Instruments to apply machine learning algorithms to optimize IT operations, otherwise known as AIOps.

    Virtual Instruments focused exclusively on enhancing storage I/O performance before extending the reach of its portfolio to include server performance. With the acquisition of Metricly, the company is further extending the portfolio to span hybrid cloud computing environments, said Van Ash.

    Metricly, formerly known as Netuitive, provides analytics tools that are focused on helping organizations contain cloud costs.

    Metricly CEO Bob Farzami said the company’s tools increasingly are being incorporated via application programming interfaces (APIs) into DevOps processes to provide visibility into costs, as the number of applications deployed in the cloud expands and increases. The acquisition of Metricly adds more than 50 integrations to open source DevOps technologies to the Virtual Instruments monitoring portfolio, which spans databases, messaging platforms, containers and cloud infrastructure services such as AWS Lambda, EC2, ECS, ASG, EMR, Microsoft Azure VMs and Load Balancer.

    Those integrations are critical because as the number of services being made available on AWS increases, the visibility organizations have into their real cloud costs becomes more difficult to both discern and limit, said Farzami. There are already hundreds of cloud services on AWS. Cloud service providers often count on organizations to forget how many instances of, for example, virtual machines they have spun up, only to be presented with an unexpected bill at the end of the month. Those issues will be further exacerbated as organizations more aggressively employ multiple cloud platforms.

    Many organizations are also seeing cloud costs rise over time as the volume of data being dynamically accessed by stateful applications increases. Once those costs reach a certain level, those organizations are realizing it costs less to repatriate those applications back on into on-premises IT environments. Metricly provides a mechanism through which organizations gain more visibility into their cloud costs in real-time, but also identify when specific cost thresholds are about to be reached.

    It will be at least somewhere between five to 10 years before even the majority of application workloads are running in the cloud. Today, after more than 10 years, only about 20% of application workloads are running in public clouds. There are a whole host of reasons organizations still prefer on-premises IT environments, ranging from regulatory requirements and performance to cybersecurity and simple inertia. Wherever workloads wind up being deployed, however, the one thing DevOps teams can count on going forward is that someone in the finance organization soon will be asking more pointed questions about how much it costs to run that workload.

    — Mike Vizard

  • CloudBees Harnesses Additional Cloud Bursting

    CloudBees Harnesses Additional Cloud Bursting

    At the DevOps World | Jenkins World 2019 conference, CloudBees announced that the latest version of its CloudBees Accelerator platform now supports cloud bursting to all three of the major public cloud service providers.

    CloudBees Accelerator makes it possible to distribute the build, quality assurance and testing cycles across a large cluster of inexpensive servers. Version 11.1 further extends that capability into the public cloud by adding support for Microsoft Azure and Google Cloud Platform, alongside existing support for Amazon Web Services (AWS).

    Company CEO Sacha Labourey said as organizations embrace multiple clouds for application development and deployment, continuous integration/continuous deployment (CI/CD) platforms will need to span hybrid cloud computing environments. Most organizations are not yet making use of multiple public clouds to develop an application, but they are increasingly relying on external cloud resources to augment the infrastructure resources they have available in on-premises IT environments.

    In addition to providing cloud bursting support for Azure and GCP, version 11.1 of CloudBees Accelerator provides support for Android Q, a forthcoming update to the widely employed mobile operating system due out in the third quarter of this year. Other new capabilities include support for the SSL and TLS cryptographic protocols and improvements to out-of-the-box support for Windows, including the ability to distribute and cache Visual Studio projects.

    Version 11.1 of CloudBees Accelerator also now makes it possible to apply CloudBees Electrify, a framework for optimizing the use of all available infrastructure in parallel, to accelerate build tools such as MSBuild and Scons. In addition, it provides improvements for the open source Yocto Project, an initiative from the Linux Foundation that streamlines the creation of Linux distributions for embedded systems. CloudBees Accelerator enables developers to at least triple the speeds of builds using BitBake build tools or Buildroot files and patches.

    As expansion occurs in both the volume and types of applications that organizations rely on to build and deploy faster, the need to rely more on automation also is increasing. Many DevOps processes that organizations adopt initially have been crafted manually. As organizations become more adept, however, it becomes apparent there is an opportunity to automate a wide range of lower-level processes. Frameworks such as CloudBees Accelerator essentially provide a layer of abstraction that enables organizations to devote more time and energy to writing application code.

    In fact, the next wave of organizations adopting DevOps is likely to benefit greatly from all the processes that the first wave of adopters helped automate by identifying best DevOps practices. It remains to seen just how automated the entire CI/CD environment can become with the rise of, for example, artificial intelligence (AI), but with each passing upgrade cycle, the accessibility of DevOps increases thanks primarily to advances in automation. Those advances not only make it easier to share best DevOps practices across an enterprise, but they also make it feasible for more organizations to embrace DevOps because the level of expertise required to master those DevOps processes declines.

    — Mike Vizard

  • FaaS is Key to DevOps Efficiency

    FaaS is Key to DevOps Efficiency

    When AWS announced Lambda–a serverless Function-as-a-Service (FaaS) technology offering–the idea of encapsulating a specific task into a function that resides in the cloud was introduced into mainstream access.

    Now, with the rising popularity of microservices, DevOps is waking up the benefits of FaaS where it is used to build and maintain enterprise applications in agile environments. The move to serverless architecture using FaaS is one organizations are now increasingly likely to investigate in 2019.

    Given these industry realities, we need to ask ourselves what place will FaaS take in contemporary computing environments and how will its core efficiency proposition enable DevOps to flourish and improve experiences for users at every level?

    Back to Focus

    First let’s remind ourselves what any FaaS platform really does. It allows us to focus on developing and executing business logic because server provisioning and management is handled automatically at the backend. Developers can deploy applications into a space with zero server administration, yet with the ability to enjoy high availability and high scalability.

    The impact for DevOps is immediately apparent.

    Developers are able to work in a zone where they are essentially abstracted away from the servers their applications live and breath on. The services developers draw upon are event-driven and eminently scalable. Even if they have to scale by some massively parallel level of magnitude, server-side service optimization is handled at the backend by the FaaS vendor.

    At the same time, the cost structure is shifted and operations staff are able to oversee compute billing models based on actual execution of functions, as opposed to being dependent on the physical size of a pre-provisioned cloud server established at best guess size and core optimization parameters for compute, I/O, storage, analytics and so on.

    Speedier Implementation

    If we have so far looked at the FaaS efficiencies that reside at the backend server-side management element of the equation, let us also turn our attention toward the way FaaS allows developers to reduce the amount of time it takes to deploy code in a new environment.

    Adopting a FaaS-centric approach to application architectures allows the developer team to encapsulate specific software tasks into functions, hence the name. For a mortgage calculation application, for example, we can encapsulate functions such as interest rate calculations, payments, length of contract etc., and then place those functions in the cloud on a FaaS platform. The FaaS provider is responsible for the hosting of that function and also looks after the heavy lifting associated with availability and redundancy.

    The benefit to developers is a very rapid environment for deploying new code, getting feedback and iterating. The operations benefit is one of hosted happiness as the FaaS provider shoulders a significant administrative burden.

    FaaS and microservices are now increasingly viable options in any modern technology architecture and it’s important to distinguish between the two.

    Microservices are made up of self-contained elements of business logic designed to perform a very specific job. They are optimized for their single responsibility and can communicate with other services via application programming interfaces (APIs). Further, they can be developed, deployed and scaled independently without impacting other services.

    A FaaS-based approach is well-suited for transactional workloads where code is run periodically. This is also a technology that is quicker to establish, as and when needed. A FaaS function might also make a call to other application logic that (in the case of our mortgage example) calculates interest rate and so on.

    FaaS Challenges

    Like any still-nascent technology, FaaS has its functionality flaws and challenges. The automated nature of foundational provisioning in FaaS is great news, but it’s not a panacea.

    Why? For one, DevOps teams will have limits on the debugging capabilities for applications that run on FaaS DNA. Also, functions are instantiated and then de-instantiated as needed, which requires time, especially for a cold start. This process can cause unwanted delays in time-sensitive operations. The transient nature of FaaS also impedes state preservation among dependent functions, which requires careful design considerations.

    FaaS is a place where the burden of server resources control has been shifted. But there’s no such thing as a free lunch and this means DevOps won’t be able to interject some level of application monitoring function at the server level itself. You’ll need to know more about your FaaS-based application before you FaaS it up.

    We must also remember for any distributed software architecture, there’s an up-front responsibility to design to accommodate the limitations and increasing options available in modern services.

    The FaaS Road Ahead

    Examples of FaaS now extend beyond AWS Lambda, we can also look to Microsoft for Azure Functions, Google for Cloud Functions, IBM OpenWhisk, Oracle Functions, Fn Project and IronFunctions with its Iron.io platform.

    This sector of the software landscape is guaranteed to grow over the coming years. FaaS can be an appropriate option in an overall software architecture, but there are specific limitations, so choose wisely. Observing the evolution of FaaS over the coming years will be a fascinating process.

    — Mike Duensing

  • VMware Looks to Accelerate App Migration to the Cloud

    VMware Looks to Accelerate App Migration to the Cloud

    VMware this week expanded the scope of its cloud migration efforts to include workloads that aren’t running on its virtual machine. VMware HCX leverages replication and virtual machine mobility technology to make it easier to automate the migration of either VMware-based workloads or those running on rival platforms to the latest generation of VMware platforms.

    Previously, VMware only offered this migration capability as part of larger services offerings that were focused on only migrating workloads running on VMware platforms. Allwyn Sequeira, senior vice president and general manager of the Hybrid Services Business Unit at VMware, said VMware HCX not only extends that capability to orphaned platforms such as Open Stack and Kernel-based Virtual Machines (KVM), it also marks the first time VMware has offered this migration tool as a separate, standalone offering.

    Sequeira said VMware HCX provides an opportunity for organizations that have committed to a platform other than VMware to reconsider their options. VMware HCX provides a way to migrate thousands of virtual machines at a level of scale that significantly reduces the amount of time to make that transition, he said.

    In some cases, Sequeira said customers have already employed VMware application migration tools to lift and shift thousands of virtual machines into cloud computing environments such as Amazon Web Services (AWS) in a few weeks. Overall, the rate at which those migrations are occurring has doubled every quarter for the last six quarters, he added.

    VMware HCX is based on virtual machine mobility technology developed by VMware combined with a similar cross-platform workload migration technology the company gained in its acquisition of CloudVelox early last year. VMware HCX now provides organizations with the option of migrating virtual machines programmatically via an application programming interface (API) or via a graphical user interface (GUI).

    VMware HCX

    There exist multiple tools for migrating workloads into any number of cloud computing platforms. Many organizations are also taking advantage of Docker containers to encapsulate entire workloads so they can run on any platform. Deciding which path to pursue most often comes down to the level of integration required with the various services available on the targeted cloud migration platform.

    In the meantime, as workload migration tools become more sophisticated, decisions regarding where workloads need to run are becoming less concrete. Competition for new workloads between platform vendors has always been especially fierce, and it used to be that once an IT organization made its decision, it was unlikely it would change its mind, given the effort needed to swap one platform out for another. Now, however, as application workloads become more portable, competition among platform vendors will only increase as they seek to convince IT organizations to shift existing workloads onto a different platform.

    In general, that increased level of competition bodes well for DevOps teams, who are usually among the strongest advocates for change within any IT environment. The issue now is deciding which platforms within those already-cluttered IT environments make the best candidates for rationalization.

    — Mike Vizard