Category: DevOps in the Cloud

  • FinOps Foundation to Help Rein in Cloud Costs

    FinOps Foundation to Help Rein in Cloud Costs

    A new consortium has been formed to promote best practices for controlling cloud computing costs.

    Announced at the Cloud Economic Summit, the non-profit FinOps Foundation counts among its founding members Atlassian, Autodesk, Cloudability, HERE Technologies, Just Eat and Nationwide.

    Cloudability co-founder J.R. Storment said the goal is to bring financial accountability to variable cloud spending that can easily spiral out of control if not monitored carefully. The adoption of best DevOps practices makes IT organizations more agile, but not every organization has the financial controls in place to ensure spending on cloud services is aligned with business outcomes, said Storment.

    These days engineers can use their corporate credit card to spin up virtual machines in the cloud. The FinOps Foundation is making a case for more accountability over the spending.

    Gartner estimates worldwide cloud spending will reach $360 billion by 2022, which is five times the compound annual growth rate for overall IT spending. The unfortunate truth, however, is that most finance organizations don’t have a lot of visibility into how those dollars are being allocated.

    To coincide with the launch of FinOps, 451 Research also published a survey of 300 IT and finance executives in the United States and the U.K. that finds more than half (57 percent) of large enterprises worry about cloud costs daily, with 80 percent admitting poor cloud financial management has had a negative impact on their business.

    In fact, the 451 Research report makes it clear most finance departments have little visibility into cloud spending. Almost three-quarters (72 percent) of finance executives acknowledged they have no formal reporting capacity between departments. More than half (51 percent) of respondents admitted to occasionally overspending and 68 percent said they are only alerted to overspend too late. A full 82 percent of finance respondents rely on cloud vendors, spreadsheets and manual processes to track cloud costs.

    Despite much of the variable cost hype surround cloud computing, 73 percent of the finance respondents in the United States and 81 percent in the U.K. still treat cloud as a fixed capital expense rather than an operating expense.

    To help finance teams better understand how cloud spending is being allocated, the FinOps Foundation will provide access cloud economic research and related articles, FinOps training and meetup groups. Early access to the forthcoming “Cloud Financial Management Strategies, Creating a Culture of FinOps” book published by O’Reilly Media will also be provided.

    As IT organizations increasingly employ multiple cloud platforms to run different classes of workloads, understanding the true cost of choosing to use one cloud over another will become more important. At the same time, IT organizations have a greater appreciation for the need to move workloads between platforms as costs rise. Just because an application workload was created in the cloud doesn’t mean that workload should be deployed there. Many long-running applications are less expensive to deploy in an on-premises IT environment. The challenge IT organizations arguably face going forward will be integrating streams of real-time financial data into DevOps processes to make more economically informed decisions about how to allocate their cloud resources.

    — Mike Vizard

  • Can DevSecOps Prevent a Zombie Apocalypse?

    Can DevSecOps Prevent a Zombie Apocalypse?

    Making consistent progress within any DevSecOps initiative often can be an overwhelming undertaking. Developers tend to outnumber operations dramatically, with little to no security accountability. This leaves DevSecOps doomed to work with the same network security stack that was originally designed decades ago for on-premises environments. Still, the expectation is to support shrinking deployment cycles on cloud-native and hybrid environments without compromising on security standards.

    To deliver on such a tall order, DevSecOps teams are armed with security groups and access lists (which are more of the same distributed firewall technologies) to secure a perimeter that essentially no longer exists. This article will help you figure out how much time you have left before your CI/CD pipeline is overrun by zombies (quite literally) and what you can do about it.

    Why Cloud-Native Environments Choke on Rule-Based Network Management

    Rule-based management of network infrastructure has worked for decades—and no doubt will continue to support many enterprises well into the future, unless your production environment includes a public cloud footprint that supports continuously deployed release cycles. That’s when things start to get a little weird. Instead of letting go of the binary perimeter premise (inside versus outside) and redefining it completely, security practitioners have essentially re-created multiple virtualized perimeters in the public cloud. In theory, you could spin up at least as many perimeters as you have running instances. Unfortunately, the same network rules that worked well for the one perimeter break down in a virtualized environment that has hundreds of instances supporting dynamically changing requirements. In fact, the very nature of the cloud is driven by business logic at the application layer, which is decoupled from network infrastructure by design. This creates a Catch-22 situation for cloud-native security pros, forcing DevSecOps teams to manually map network rules to highly agile and virtualized business logic. No human can realistically deliver results over time (and stay sane). The reality is, rule-based network management can never keep up with the requirements of cloud-native business logic.

    With Great Agility Comes Security by Bureaucracy

    Regardless of how skilled DevSecOps teams are, or how motivated they are to lead the brave new world of continuously secure deployment, the harsh truth tends to be disappointing. While DevSecOps presumably aspires to deliver agility without compromising on security, practitioners spend much of their time doing just that. Trapped by a security culture that has favored bureaucracy over automation for decades, DevSecOps tends to perpetuate the very dichotomy that triggered the emerging practice. In a cloud-native culture driven by innovation and automation, agility comes first. This inevitably becomes an exercise in futility—the more agile your deployment, the more vulnerable you become. That means giving up on the very idea that rule-based network management is a viable security methodology. The complexity of managing several dedicated security groups (up to five) for every instance rapidly spins out of control. Given the imperative to maintain the cloud’s agile environment, many teams consciously sacrifice security on the altar of agility, leaving a gaping attack surface vulnerable to unrestricted access.

    Welcome to Cloud-Native Multi-Perimeter Security

    The challenge is to prevent any unauthorized code (benign or otherwise) from infiltrating cloud deployments before any sensitive data or workloads are compromised. In the absence of robust automation to support this process, the challenge is often ignored. When that doesn’t work, the solutions often remain in the realm of bureaucracy, focusing on anecdotal symptoms without addressing the root cause.

    The inevitably grim question is not if your deployment is at risk, but how vulnerable you are and how soon is likely too late. The results are measurable and often disturbing. Below is the standard KPI checklist most DevSecOps are familiar with:

    1. Rogue workloads unauthorized by CI/CD pipeline. These are typically addressed by discovery tools and ad-hoc network scans but are all too often accepted as a necessary evil.
    2. Faulty network rules. Microsegmentation initiatives often lead to unexpected inter-workload communication.
      Workloads deployed via the CI/CD pipeline could fail to communicate effectively—or worse, communicate to support counterproductive or even malicious activity.
    3. Unexpected workload communication to Third-Party PaaS. A case in point is workload communication to a database such as S3, which could lead to a data breach. Well-documented cases include leaders, such as Accenture and Time Warner, that unwittingly enabled public access to S3 buckets storing sensitive customer data and access credentials.
    4. Network Security Configuration Mistakes. Vulnerabilities introduced by network security configuration often go unchecked for months. Such unrestricted access is commonly cited as the attack vector that enabled unauthorized actors to infiltrate a gaping cloud-native attack surface.
    5. Zombie Workloads Consuming Resources. At best, you’re just wasting resources. All too often, one of those zombie workloads is busy cryptojacking, mining cryptocurrency on your company’s buck unnoticed. Cryptojacking is so rampant, it has overtaken ransomware in 2018 as the most popular attack vector.

    The bad news is, no cloud deployment can eradicate the above issues entirely, using familiar technologies. However, you can mitigate some of the damage: Scan your network more regularly using your favorite open source tool (check out a great example implemented on Netflix Security Monkey presented by Ryan Hodgin). Never leave an entire network flat granting anyone unrestricted access. Ideally, take the time to evaluate new technologies designed to secure cloud-native deployments.

    DevSecOps teams are gradually accepting that cloud-native deployments require a fundamental paradigm shift to sustain a defensible security posture without sacrificing agility. Many believe (this author included) that we are witnessing a cloud-native security revolution, much like the one triggered by the internet in the 1990s. DevSecOps can certainly address the symptoms described above, but security policies will continue to fail miserably if they both remain entrenched at the network layer and rely on bureaucracy.

    — Ran Ilany

  • Enhance DevOps Experience with AWS Smart Tools

    Enhance DevOps Experience with AWS Smart Tools

    DevOps has become the champion of business operations by fostering collaboration between disparate teams, providing a common work framework and paving way for continuous delivery that greatly boosted software development.

    Amazon Web Services (AWS) has become a leading provider of DevOps tools and services and has been instrumental in taking this experience to a higher level.

    So how does AWS elevate the DevOps experience?

    Automation

    DevOps not only eliminates the cultural disparities between two different environments but also eases the software development process by automating its tasks. This has made maintaining and looking after large-scale complex environments easier.

    Earlier, implementing automation in cloud environments was a painstaking process and was similar to implementation/usage in physical or virtual infrastructures. Today, however, leading cloud services providers offer DevOps automation tools as a part of their feature package, making automation a breeze.

    Continuous Integration and Delivery

    AWS CI/CD tools make building, testing and deploying application source—or just securely storing it—simple and organized.

    Continuous Integration and Delivery with AWS DevOps

    • AWS CodePipeline: This is a continuous integration/delivery tool that builds, tests and deploys code in the event of a code change. AWS CodePipeline helps developers introduce code quickly and reliably into the software life cycle.
    • AWS CodeBuild: AWS CodeBuild is a fully managed build service that compiles source code, runs tests and produces deployable software packages that are ready for use. CodeBuild is expandable and processes multiple builds simultaneously, propelling the purpose of continuous integration and delivery.
    • AWS CodeDeploy: AWS CodeDeploy automates code deployments to instances or on-premises servers. This tool helps users to release new features rapidly, avoid downtime during application deployment and manages updates easily.

      Special Features:

      • Automatic deployment
      • Integration with third-party tools
      • Centralized management
      • Integration with other AWS services
    • AWS CodeStar: AWS CodeStar helps users to rapidly develop, build and deploy applications while providing them with a common interface for managing software development activities. It allows users to set up their continuous delivery tool-chain in minutes for faster code release.

    Microservices

    Microservices allows users to build and deploy microservices architecture with the help of containers or serverless computing.

    • Amazon Elastic Container Service: Amazon EC2 services enables users to manage large groups of containerized applications with the help of Docker containers.
    • AWS Lambda: AWS Lambda relieves users from the burden of constantly administering and managing the system while running code for virtually any type of application or back-end service. This Amazon service just requires developers to upload code after, which the system takes care of running and scaling code with high availability.

    Platform as a Service

    • AWS Elastic Beanstalk: AWS Elastic Beanstalk is used to deploy and scale web applications and services developed with Docker, Java and Python on servers such as Nginx. All users have to do is upload the code and the service will handle deployment, including capacity provisioning, load balancing, auto-scaling and app health monitoring.Special Features:
      • Quick Deployment
      • Integration with other AWS Services
      • Application Health Monitoring
      • Easy Access to Application and System Logs
      • Customized Software and Applications

    Infrastructure as Code

    • Chef: This is an infrastructure automation framework that enables easy configuration, deploying and management of servers and applications on a cloud, physical or virtual environment. It sets up the infrastructure by using recipes.
    • Puppet: It can be tedious to tend to each server individually should you need to make configuration and content changes to multiple servers. In such a scenario, Puppet helps to set up each server as an agent that relays information to the AWS Puppet Master Machine about any changes to be executed on that server, paving way for uniformity across all servers.
    • Ansible: Ansible is another configuration management, software provisioning and application deployment tool. Designed for multi-tier deployments, Ansible enables continuous delivery. It designs your IT infrastructure by describing how all your systems can work as a network.

    Amazon Web Services has a number of DevOps tools, all designed to help professionals work more efficiently in the AWS cloud environment. Are you using the AWS tools? If so, what are your thoughts about them? Let us know your thoughts in the comments section below.

    — Veritis

  • Business Continuity in the Azure Cloud: Understanding the Options

    Business Continuity in the Azure Cloud: Understanding the Options

    There are numerous ways to assure business continuity for applications running within the Azure cloud via various high availability and disaster recovery provisions. But, selecting the best and most cost-effective provisions for each and every application can be extraordinarily difficult owing to the myriad choices available. At best, making poor choices during development can waste money. At worst, a wrong choice can cause failover provisions to fail when needed during operation.

    All business continuity provisions involve hardware and software redundancy, data replication and some means of failover and failback. Purpose-built failover clustering software has long been among the most popular choices based on its proven dependability and cost-effectiveness. The clusters are relatively easy to deploy in an enterprise data center using shared storage. But with no shared storage available in the public cloud, configuring failover clusters in Azure becomes considerably more challenging.

    This article examines the options available for high availability (HA) and disaster recovery (DR) provisions within and for the Azure cloud. Special emphasis is given to SQL Server as a particularly popular application for Azure.

    Options Available Within the Azure Cloud

    The Azure cloud offers redundancy at three layers: within data centers, within regions and across multiple regions. Within data centers, Availability Sets are used to distribute redundant servers across different fault domains located in different racks, which protects against failures at the server and rack levels. This affords some protection for some failures, but provides no protection during a sitewide failure, such as the one that occurred in September 2018 in Azure’s South Central US Region. The 99.95 percent Service Level Agreement (SLA) only guarantees that in an Availability Set with two or more servers, at least one will have external connectivity, but it does nothing to assure availability at the application level.

    To protect against sitewide failures, Azure is now offering Availability Zones (AZs). Regions with AZs have at least three data centers interconnected via a high-bandwidth, low-latency network that supports synchronous replication. Azure offers a 99.99 percent SLA for AZs, but again, only guarantees at least one server will have external connectivity—nothing less and nothing more.

    For redundancy during major disasters, Azure offers Region Pairs, in which each region is paired with another in the same geography (e.g. United States or Europe). The regions are separated by at least 300 miles to protect against widespread disasters that might impact an entire region, including across multiple AZs. By pairing regions, Microsoft is able to apply updates one at a time to prevent the “update gone bad” scenario and will prioritize the recovery of at least one region in each pair during an Azurewide outage. But again, Azure only guarantees “dial tone” for the servers, leaving it to the customers to ensure availability at the application level.

    Options Available in OS and SQL Server Software

    Windows Server Failover Clustering (WSFC) is a standard operating system feature that is utilized by many applications to provide HA protection in enterprise data centers. However, WSFC requires some form of shared storage, which historically has not been available in any public cloud, including Azure’s.

    Microsoft addressed this problem in the Datacenter Edition of Windows Server 2016 by adding Storage Spaces Direct (S2D), a software-defined, virtual storage area network. But because the cluster must reside entirely within a single data center, S2D is incompatible with Availability Zones. Applications that require multisite HA/DR protection will, therefore, need to use third-party failover clustering software, log shipping or some other additional provision(s).

    With no equivalent to WSFC or S2D for Linux, HA/DR protection requires either the use of open source software, such as Pacemaker, or a third-party failover clustering solution. Because supporting open source software requires a substantial and ongoing commitment, only the largest organizations have the wherewithal to even consider this do-it-yourself option.

    SQL Server, whether for Windows or Linux, offers two of its own HA/DR features: Failover Cluster Instances and Always On Availability Groups. FCIs afford two major advantages: inclusion in the Standard Edition; and protection for the entire SQL Server instance, including system databases. A notable disadvantage is the need for cluster-aware shared storage, including the virtual variety with S2D, which is only supported for SQL Server 2016 and later.

    Always On Availability Groups is SQL Server’s more robust HA/DR offering, capable of delivering recovery times of 5-10 seconds and recovery points of seconds or less. Its disadvantages include the lack of protection for the entire SQL instance and the need to license the more expensive Enterprise Edition, which can be cost-prohibitive for many applications.

    A significant disadvantage with all application-specific options is the need for DevOps staff to use different HA and/or DR solutions for different applications. Having multiple HA/DR solutions inevitably increases complexity and costs, making this another reason why application-agnostic third-party solutions are so popular.

    The Third-party Failover Clustering Software Option

    Being agnostic with respect to both applications and platforms enables purpose-built failover clustering software to provide a complete HA/DR solution for virtually all Windows and Linux applications. Application-agnosticism eliminates the need to have different HA/DR solutions for different applications. Platform-agnosticism makes it possible to leverage, while not being dependent upon, various capabilities and services within the Azure cloud, making this option suitable for use in private, public and hybrid cloud environments.

    Failover clustering solutions include, at a minimum, real-time data replication, continuous monitoring for detecting failures at the application level and configurable policies for failover/failback. All are designed to satisfy mission-critical recovery time and recovery point objectives, and most also offer a variety of value-added capabilities to simplify implementation and management.

    Clustering with Confidence

    Whether used individually or in various combinations, all of these options can have a role to play in making HA and DR protections more effective and more affordable for all applications—from those that can tolerate some downtime, to those that demand five 9s of uptime. But be sure that the options chosen afford protection at the application level for all likely failure scenarios.

    — Dave Bermingham

  • DevOps Chat: Serverless and Abstraction with Nimbella’s Anshu Agarwal

    DevOps Chat: Serverless and Abstraction with Nimbella’s Anshu Agarwal

    Anshu Agarwal has a tremendous record of working in startups that had successful exits. Now for the first time, along with her co-founders, she is actually founding the startup herself. Nimbella is a serverless platform that is cloud-agnostic and makes developing and deploying applications easy. Whether in a public cloud, private data center or both, Nimbella can help.

    In this DevOps Chat, we speak with Anshu about her own personal journey and where Nimbella is and what the plans are for going forward.

    As usual, the streaming audio is immediately below, followed by the transcript of our conversation.

    Transcript

    Alan Shimel: Hello, everyone, it’s Alan Shimel of devopsy.kinsta.cloud and you’re listening to another DevOps Chat. Today’s chat actually features a company that’s just starting out here, but it’s a guest who’s had a rather long career in the tech space. As long as mine, even, at some point, but I’m sure I’m much older. Anyway, I’d like to introduce you to Anshu Agarwal. And, Anshu, if I messed up your last name, I apologize. Was that good?

    Anshu Agarwal: Thank you, Alan. You said it perfectly right. Thank you for having me today.

    Shimel: Thank you. So, Anshu, let’s start off – I referenced that you’d been in tech a while. Let’s give people an idea of your personal journey that brings you here today, even going back to Speedera and beyond, if you’d like, but if you wanna start with Speedera, that’s fine. But let’s give folks an idea of who you are, what your background is.

    Agarwal: Sure. Thank you. So I am an engineer by background and I moved into product management and marketing after I finished my MBA. And, after that, I joined a startup and then the startup bug never left me. So I joined Speedera, which got acquired by Akamai. Then I joined another startup, Ankeena, which got acquired by Juniper. Another startup, ConteXtream, got acquired by HP. And my most recent startup, Cedexis got acquired by Citrix, so I have been acquired four times. And, in all of those companies, I kind of delivered the success for other people’s ideas because they were all successful startups, got acquired by very, very large companies, so they were indeed – from value standards, they’re considered successful, but, for myself, I had never really done a company of my own.

    So, when I got the opportunity of just starting and being a founder of a cloud infrastructure, because that’s my background – I’ve been doing cloud infrastructure for last 15 to 20 years – there was no question that I wouldn’t do it. And we got started on this company – the company name is “Nimbella.” And it is in the serverless framework or serverless platform space. And we are kind of disrupting what serverless is, so – or we are providing what serverless promises to the developers. So that’s what we are working on now.

    Shimel: Excellent. You know, you’re way too modest and humble because, for many people listening, whether they’re in the valley or Boulder or Austin or New York or Boston or London or Bangalore or anywhere else in this world, having four startups that were acquired by companies like Akamai, HP, Citrix, and I forgot the fourth one – I apologize – a lot of people would call that a career. Right? I mean, whether they founded the companies or not, just being associated with four companies that had successful exits to large companies like that is something to be proud of and it speaks volumes as to either you’re really, really lucky or you’re doing something right. Right? So congratulations to you for that.

    So you’ve mentioned Nimbella and I wanna kind of share with our audience up front that Nimbella is – you have raised some funds for the company and there’s a great startup – there’s a great founders group involved. You mentioned, I think, one of your cohorts from Speedera; I think there’s two other founders in the group. But the company’s sorta still – I don’t wanna say stealth, but pre-product, let’s call it. Pre-public-product. So I don’t wanna put you on the spot. I’d rather talk, I think, if it’s okay, Anshu, about some of the problems you see and potential solutions that we see.

    You know, we live in a great time for developers and engineers. Internet time, right? We’ve gone from servers to hypervisors to containers. Now serverless. And serverless seems to be all the rage, but, like any new technology, it’s not birthed mature, right? It doesn’t come out mature; it doesn’t start mature. And so there’s gonna be growing pains. Can you share with our audience a little bit about what you see as some of the growing pains around serverless or what you’re hearing from people around that?

    Agarwal: Absolutely. Well, thank you for – this is a great question because it is definitely the best thing that’s happening for developers. The reason I say that is because what developers really want is to develop their business logic or develop the code and not worry about the infrastructure. And serverless promise is all about that, that you don’t have to worry about the infrastructure. You don’t ever have to worry about how it scales, how it is secured. Anything that requires the knowledge of infrastructure should not be a developer’s worry.

    But, in today’s serverless frameworks – and there are a few serverless frameworks and some are very popular ones – there is so much complexity still for the developers because they have to combine many services. And, for instance, they have to think about storage, they have to think about identity management, and they have to learn all these services and integrate. Not only that, the management and debugging is still an issue. And these are growing pains, as you mentioned. The serverless computing is still in a nascent stage and, therefore, it’s lacking all the abstractions that are needed for the adoption to accelerate into new markets and application domains.

    Not only that, serverless today, if you look at it, is used only for stateless applications, so you’ll see it in IoT applications, you’ll see it in reactive kind of applications only. It is not suited for long-running or stateful workloads. The reason being is because serverless today is serverless compute; it is not serverless storage. So that’s another problem. We may consider it as a problem or we may consider it as the limited adoption of the technology, but that’s something we have to address if we want this to be adopted in general, across all workloads. Then the –

    Shimel: I’m sorry. We may also consider it as an opportunity, right? I mean –

    Agarwal: Oh, absolutely.

    Shimel: Yeah.

    Agarwal: Absolutely. And that’s what we’re doing. And I’m gonna talk about Nimbella and how Nimbella is addressing it.

    Shimel: Mm-hmm.

    Agarwal: Another important aspect of serverless today is, like any new technology environment, there is serious vendor lock-in because, if you are using AWS’s Lambda serverless framework, you are programming to the nuts and bolts of Lambda. If you are using Google, you are programming to GCP, and so it is with Microsoft Azure. So, in a sense, you are programming to proprietary frameworks and, therefore, your code is not cloud-portable.

    So, tomorrow, if your company decides that, “Now we are going to move from AWS to Google,” you gotta rewrite the whole thing again. So that is actually, again, the serverless promise, and, therefore, vendor lock-in is very real, especially for the companies, actually, that choose not to go to public cloud because they’re part of regulated industry or whatever _____. They want something to be running in their own private cloud; these serverless frameworks are not an option for them. And then, say, you were able to write code for all different frameworks, but then the complexity of operations is there because you can’t monitor and observe at each level since these are proprietary frameworks across these clouds.

    So there are serious issues for widespread adoption and that’s where we come in. That’s the opportunity we looked at when we decided to create this company, is we want to create a platform, a framework, that is providing all the necessary abstractions so the developer is not to worry about how the infrastructure works, how to scale the code, how to secure the code. The code is just nebulous logic that they write. Then it is also suited for long-running, stateful, streaming workloads, such as AI/ML workloads. I mean, we have a greater adoption of AI happening in every industry. If we are not going to be able to handle AI workloads, we are missing a big market.

    And those are real, real problems and we are addressing it with Nimbella, you know, the technical problems. Then there are operational issues where you want cloud portability, so we are also addressing that part because we are providing a layer of abstraction and it’s open-core-based, so a developer can program the way they program the business logic and not worry about which cloud they’re writing.

    Shimel: Got it. Got it, got it, got it. So, again, I don’t wanna be uncomfortable or make you uncomfortable with what we’re gonna talk about, but, you know, I was at KubeCon out in Seattle last month – seems like a year ago, but it was only last month – and, surprisingly or not surprisingly, “serverless” seemed to be on everyone’s lips. Right? And maybe some people listening to this are saying, “Wait a second, Kubernetes is for containers and containers can run on bare metal, yeah, but they run on VMware or hypervisor as well.” What do you think the role of Kubernetes is in the serverless future? Fair?

    Agarwal: Kubernetes is a great technology. I really think it has great future. And serverless frameworks use Kubernetes or something else like that. And we were also at KubeCon. It was actually our first conference as an Nimbella person attending – one of my cofounders, who actually is the creator of OpenWhisk Apache, a serverless framework, he attended it. And he mentioned the same thing that you mentioned, that serverless was on everybody’s slide deck. Okay? It is something everybody is looking at; it is something everybody is kind of they have it on their roadmap, that they wanna work on it. They wanna look at it and see if it works for their environment.

    And Kubernetes is perfect, actually. It provides the platform needed because it is widely accepted these days and it provides the necessary environment, but, again, it should not be the burden of a developer to know how to run, work with Kubernetes. It should be the infrastructure serverless framework that provides that to the developer and the developer’s still only programming the logic. It’s not for the developer to worry on what environment, what container environment they are working on and what VMs are being used and what bare metal is being used. That’s all abstracted away for the developer.

    Shimel: Got it. Got it, got it, got it. So, you know, I’m sure there are people listening to us today who are thinking to themselves, “This woman has been in four startups and has had tremendous success.” And now I’m sure you’re a CEO, right, of Nimbella and you’re cofounder –

    Agarwal: That is correct.

    Shimel: Right? And I’m hoping there’s a whole bunch of women engineers out here or women entrepreneurs who are listening to you, saying, “Hey, if she can do it, I can do it.” And we don’t have a lot of time left, but I wanna address those folks there, that are near and dear to me for a lot of reasons. What’s your advice to them?

    Agarwal: Absolutely. If I can do it, they can do it. That is a very true statement. And, I mean, you said some great words about me, but I consider myself average successful person and, yes, I have been lucky that the four companies I’ve worked for had great exits. But being a founder and CEO is still very different from what I did for the other companies before. So, basically, when you are part of the company, other companies, you’re not bothered by operational aspects of the company. You’re not bothered by hiring for every possible function in the company. So the skill sets that I’m developing now, there is no other experience that could have taught me that, so, even though – I say, “Everybody can do it and anybody can do it,” but they still have to learn the art of running a company on the job. It just – there is no course. There is no education that can be given because you have to experience it.

    So, to all the women engineers that are listening, I would say, “If I can do it, you can do it. You just have to just get into it. You just have to dive into it and it will come to you. You will develop your advisors. You will develop the people who help you on the way because you won’t have all the skill sets, but you will know who to go to for what skill set, as you take that role.”

    Shimel: Absolutely. Absolutely. And, you know, women or not women, entrepreneurs – I’ve also founded and helped start my fair share of companies – I don’t know if it’s courage or just pure moxie, if you will, right? You just put one foot in front of the other, right, and you just keep moving forward. And you can’t boil the ocean, right, but you can – these things are incremental, right, and they’re evolutionary. They’re not _____ –

    Agarwal: Absolutely. You’re absolutely right. These are such an incremental thing and you may be not finding the best solution, but you are optimizing your solution, at every step, because you’re trying to handle so many things and, if you wait for the best solution to arrive, it may be too late. So you’re kind of optimizing the solution for whatever issue or whatever problem you’re handling, and you move on with that. And it’s just only time will tell how successful I am in the role, but this is my experience six months into this company, six or seven months into this company, and I’m still learning the art.

    Shimel: Absolutely, it is. So what do you – short-term future, where can we find out more on Nimbella or when will we find out more, do you think?

    Agarwal: Oh, absolutely. So there is some information on the website – very little information, I would say – but it gives you a flavor of what we are doing. Then, in Q1, this quarter, we are going to be releasing our developer playground, which is actually – because the buyer of serverless technology is the developer and developer experience is the most important thing, a developer has to feel one with the programming experience. And that’s the true serverless promise and we wanna deliver on that promise, so, in Q1, we are going to release that.

    And, in later quarter, in Q2, since I have mentioned private cloud and hybrid cloud, there will be offering for private cloud in Q2. So, as this quarter progresses, we are going to populate the website more with the content and we’re going to host more meetings like this and make it available to the developer community in general, our solution, and then, later on, for enterprise sites.

    Shimel: Great. Great, great, great. Anshu, when I started, I said, “These are only about 15 minutes. It’s gonna go so quick, you won’t believe it.” If I told you we’re closer to 20 minutes already, you’d probably say, “Oh, my,” but we are and we’re gonna need to call an end to this DevOps Chat. You know what? It sounds like there’s a lot going on here and, if we can have you back on, you know –

    Agarwal: Oh, that would be great, Alan. I really hope we can –

    Shimel: Yeah?

    Agarwal: I really hope we can get back when we have our developer playground because I’d love to talk more about it.

    Shimel: I promise you we will.

    Agarwal: Thank you.

    Shimel: Promise you we will. Thank you. Anshu Agarwal, founder/CEO of Nimbella, our guest here on this DevOps Chat. This is Alan Shimel and you’ve just listened to another chat.

    — Alan Shimel

  • Spot Instances on Black Friday: A Case Study

    Spot Instances on Black Friday: A Case Study

    Businesses made the right choice using spot instances for Black Friday

    Black Friday is the unofficial kickoff to the holiday shopping season, when retailers announce exciting deals to attract as many customers as they can into their stores. With online shopping surging in recent years, e-commerce and AdTech businesses are also participating in Black Friday to increase their online presence and revenues. Today, millions of customers have the choice of shopping online or in brick-and-mortar stores. While these stores have their challenges managing massive amounts of shoppers, e-commerce and AdTech businesses are dealing with their own set of challenges when it comes to handling shopper traffic.

    Source: Business Insider

    During peak shopping days such as Black Friday, businesses use significantly more capacity to process online shopper requests while constantly trying to provide them optimal performance. In theory, as more companies utilize resources from a cloud provider, they may experience a capacity shortage when trying to scale. When a busy online shopping day finally approaches, this theory can become a reality as online retailers scale to meet their shoppers’ demands and cloud providers struggle to accommodate capacity. However, what many cloud users may not know is that about 50 percent of the major cloud vendors’ infrastructure is mostly idle.

    To be clear, this is not because AWS, Microsoft or Google are struggling to sell their capacity; it’s because they have to be ready for rapid increases in usage—just like the one businesses experience during Black Friday.

    The fear of insufficient capacity also has made users apprehensive to use cloud providers’ excess capacity such as spot instances to run their workloads, since many customers are using more cloud resources and are eating all of the cloud inventory. Businesses simply do not want to take a chance trying to compete for excess capacity and opt for something more predictable, such as on-demand instances. The fact that cloud providers are capable of handling the most aggressive computing demands has created a situation in which businesses that chose to make intelligent use of excess capacity during Black Friday saw great returns on their investments.

    Events such as Black Friday, national elections and the Super Bowl are pretty extreme stress tests for any consumer-facing business. We thought it would interesting to go through the data and see how the AWS spot market behaved during Black Friday, and see what we can learn from it. In this blog post, I will go over the spot market statistics and lessons learned from 2018’s Black Friday.

    Spot Markets and Statistics

    Spot markets are defined as the supply and demand for a specific instance type at a specific spot price in an availability zone. Spotinst Elastigroup ranks spot markets for availability and costs from current and historical data in a process called Spot Market Scoring. Spot Market Scoring enables us to predict an instance interruption in a market approximately 15 minutes before it happens (sometimes even more than an hour). This advance notification on upcoming interruptions is what triggers the spot instance replacement and ensures the continuous availability of compute workloads. Let’s see how the spot markets behaved before and after Black Friday.

    Now that you have a better understanding of the spot market, let’s delve into the statistics from Black Friday. In the image below, we can see the number of spot interruptions for each Friday three months prior to Black Friday in the us-west-1 and us-east-1 region. The us-east-1 region had significantly more spot interruptions during this period.

    The graph below shows the number of spot interruptions that occurred a week before and after Black Friday in us-east-1 and us-west-1 across all instance types. We can see that us-west-1 was more reliable for spot instances than us-east-1. Also, us-east-1 continued to experience more outages the week after Black Friday.

    We then took a look at popular instance types and sizes used before, on and after Black Friday to see how many interruptions there were. In the image below, we can see that c4.4xlarge had the highest number of interruptions and incrementally improved after.

    The section below shows metrics for Back Friday in comparison across different availability zones in us-east-1 and us-west-2:

    1. Total Spot Interruptions this Black Friday in the US: 462
    2. Breaking down the Interruptions by Availability Zones, some had more Spot interruptions than others
    Availability Zone us-east-1 us-west-2
    AZ 1 29 1
    AZ 2 3 7
    AZ 3 24 3
    1. Average Spot Uptime: 2675 Minutes (1.8 days)

    What Spot Interruptions and Autoscaling Events Look like

    Let’s examine what a spot market interruption looks like and how Elastigroup handles the situation. In the data below, you can see an actual event log from one of our customers.We can see that early on Black Friday, the day started with Elastigroup automatically replacing terminated or soon-to-be-terminated instances. A little later, as shoppers began scavenging the internet for deals, we see that the instances were automatically scaled up to meet demand. Elastigroup scaled down the instances the following day because demand was not as high as on Black Friday. With this type of autoscaling automation in Elastigroup, administrators can spend more time worrying about application development and other tasks.

    Takeaways

    1. DevOps Automation is Key – Elastigroup uses predictive algorithms to identify and drain instances that are about to be terminated. Prior to termination, Elastigroup will launch a new instance and replace it seamlessly. Elastigroup will also make sure to distribute your instances across different types to optimize costs and longevity.
    2. Spot Instance Availability – Having a stable uptime is crucial for peak days such as Black Friday. With millions of shoppers visiting e-commerce sites, an outage can result in revenue loss. On Black Friday, the small number of spot instances that experienced interruptions had an average uptime of 1.8 days. With that uptime, organizations can rely on spot instances managed by Elastigroup to reduce costs while not sacrificing service availability during peak times.
    3. Spot Market – Spots allow businesses to make better decisions without sacrificing costs and availability. Elastigroup reliably leverages the spot market to optimize your underlying infrastructure for cost, without compromising availability. Per the statistics above, we can see that some availability zones had more spot terminations than others. AZ 1 in us-east-1 had the highest number of spot interruptions.

    — Amiram Shachar

  • In AI Initiatives and Hybrid Clouds We Trust

    In AI Initiatives and Hybrid Clouds We Trust

    The 2019 State of the Union Address, delivered Feb. 5, was light on anything tech-related. But just a week later, Feb. 11, technology was the focus of an executive order that launched The American Artificial Intelligence (AI) Initiative. Why do we need a national AI initiative, and what does it have to do with DevOps? (Hint: You may need hardware and a hybrid cloud strategy for your AI initiative.)

    Investments in AI technology are exploding worldwide. Most analysts believe that while the total number of startups and equity deals to date still favors the United States, China is spawning startups rapidly, thanks to their recent AI initiative. According to just-released data from CB Insights:

    “While China accounted for only 10% of global AI deals in 2017, Chinese AI startups took 48% of all AI funding dollars that year, surpassing the US in AI funding for the first time.”

    The long tail doesn’t lie. A big global competition here is a good thing, though, and the scoreboard has spurred movement. The American AI Initiative lays out five principles, paraphrased in brief:

    1. We need to drive technological breakthroughs in AI.
    2. We need technical standards for development and testing.
    3. We need workforce training in skills and applications.
    4. We need to be able to trust AI solutions for privacy.
    5. We need both open markets and protection for “critical” technologies.

    Go, Team America! What, you expected more than motherhood and apple pie out of the White House? I’m not going there. I don’t think any administration could prepare the level of technical insight needed for something of this magnitude. It can, however, lay a course and task agencies for technology plans and funding proposals—and about two-thirds of this executive order is devoted to timelines for those.

    Let’s assume this AI initiative comes together with real dollars, and everyone including your competition is at work on AI. You may be thinking, “This is easy. I’m looking at a prominent cloud vendor’s machine learning webpage right now and it says, ‘No machine learning skills required.’ Its solution is based on its libraries of algorithms plus cloud-based workload acceleration hardware.”

    If that approach solves your specific problem, that’s great. Many use cases will fit those offerings. However, I’m thinking that a lot of you out there won’t turn over your data and your privacy control to a single vendor. Based on what I’ve seen, while the public cloud may be fast enough for some use cases, it’s woefully non-deterministic for others. You may be able to afford all kinds of time and elastic cloud resources for training, where time may be measured in weeks or months on bigger machine learning data sets.

    Real-time inference is a different game. There. Will. Be. Slowness. One missed result can be very costly—ask any autonomous vehicle developer, then look at all the hardware sitting in the trunk. Public cloud solutions are woefully non-deterministic, especially as everyone and their dog starts using them. The CEO of another big cloud vendor just said in an event keynote that most AI solutions will split 40/60 private/public cloud. She went on to say if you’re regulated, it’ll be more like 60/40.

    Most current startup investment is headed for AI chips. Machine learning is the ultimate workload optimization problem. By adding compute nodes with clusters of GPUs, FPGAs or dedicated system-on-chip (SoC) designs, inference speed-up is achievable. Whatever approach you chose, you’re probably going to want workload optimization safely in your private cloud instance within a hybrid cloud AI solution.

    You’re also going to want control over differentiation of your solution. Open source AI software solutions are great because they handle a fundamental base of value. However, AI-enabled applications are very likely to run into problems at scale. Open source will get you to prototyping quickly. Your innovation will be tested as real-world, real-time issues develop. If you can get a truly differentiated hybrid cloud AI solution that proves to be fast, reliable and secure, you’ll have a huge competitive advantage.

    Again, this looks like a perfect job for a DevOps team. Here’s where to start on your AI initiative, in increasing order of difficulty:

    • Have someone get familiar with machine learning and AI public cloud solution offerings—those vendors are easily found.
    • Also get familiar with open source frameworks such as TensorFlow, Caffe, Cognitive Toolkit, PyTorch, MXnet, DeepLearning4J and others.
    • Look at your preferred distributed computing framework for recent innovations in AI workload optimization; for instance, a lot of work is happening around Hadoop and Spark.
    • Inventory the problems you think AI might solve in your organization and see how they line up with those potential solutions.
    • Experiment and discover where the bottleneck in your preferred algorithms might be; those will be candidates for a hybrid cloud workflow optimized solution.
    • If there’s a profound bottleneck, you’ll need to evaluate the best hardware solution, meaning someone on your team (or a consultant) should understand GPU code or FPGA design.
    • For the most severe AI challenges, more companies will be considering either off-the-shelf AI chips or custom SoC designs—even companies that have never designed chips.

    My point here wasn’t to provide a primer on AI technology; that’s a huge topic. The American AI Initiative is a call that teams need to move on AI now or risk being left behind in the competitive prop wash. You also shouldn’t wait for someone to generate the exact solution you need out in the marketplace. With the feds driving a sea change in AI, investments and technology will be shifting even more rapidly, and it’s best to get in front of it.

    For the curious, here’s the entire executive order on the American AI Initiative.

    If you’ve started already, how’s your AI initiative going? Are there other areas in AI or machine learning you’re curious about that would be a good topic for future posts? Reach out if you’d like to discuss this.

    — Don Dingee

  • IBM Extends Cloud Reach to Include Integration

    IBM Extends Cloud Reach to Include Integration

    At the IBM Think 2019 conference this week, IBM unfurled an IBM Cloud Integration Platform that pulls together a range of tools in a single development platform that the company promises will reduce the amount of time it takes to deploy applications by making it easier to reuse integration code.

    Hillery Hunter, vice president and CTO for IBM Cloud Infrastructure, said by bringing all of its primary integration tools into a single cloud platform, IBM is effectively making an integration platform-as-a-service (iPaaS) environment available for the first time.

    IBM this week also formally rolled out IBM Services for Multicloud Management, a portal through which multiple clouds can be managed using software-as-a-service (SaaS) applications developed by ServiceNow, along with a set of IBM Services for Cloud Strategy and Design through which IBM will advise clients on how to architect the right holistic cloud strategy. The premise of both those services is that organizations soon will be rightsizing workloads to run on whatever cloud or on-premises IT platform that best suits them. The portal developed with ServiceNow provides access to dashboards and analytics designed to enable IT organizations to analyze cloud spend and manage workloads deployed on any platform, including Amazon Web Services (AWS) and Microsoft Azure.

    IBM is also extending the cloud services it provides to include the IBM Cloud Hyper Protect Crypto Service, which makes use of IBM LinuxONE mainframes to provide encryption key management built on the only FIPS 140-2 level 4-based technology.

    Hunter said IBM’s approach assumes organizations have already bought into best DevOps practices and API management to provide the means through which multiple clouds will be managed. The challenge many of those organizations are now wrestling with is implementing processes that determine where a workload not only should be developed and deployed but also moved as its characteristics evolve. To provide that analytics, IBM is employing machine learning algorithms running on distributed instances of Kubernetes to continuously track changes and updates to cloud platforms, said Hunter.

    IBM previously outlined a strategy under which it is trying to meld DevOps and ITIL-based processes to make it easier to manage workloads running on any cloud or on-premises IT environment. While the growth of cloud computing continues at an exponential rate, IBM executives this week took pains to remind conference attendees that 80 percent of enterprise-class workloads still run in on on-premises IT environment. In fact, IBM’s rationalization for spending $34 billion to acquire Red Hat is based on that fact. However, while many workloads will continue to run on-premises, IBM is clearly making a case for centralizing the management of all those platforms on IBM Cloud.

    It most likely will take the better part of the next decade to determine where different types of applications may wind up running. Predictions concerning the percentage of application workloads that will run in the cloud continues to be one of the most hotly debated topics in the IT industry. Whatever the outcome, however, it’s 100 percent certain more than one cloud platform will be involved.

    — Mike Vizard

  • Python Benefiting From Increase in DevOps Use

    Python Benefiting From Increase in DevOps Use

    DevOps and Python have developed a mutually beneficial relationship.

    In a recent survey released by the Python Software Foundation and software development company JetBrains, researchers found that more and more developers are using the program language for an array of versatile projects, including DevOps and machine learning.

    In fact, Python developers told the researchers that DevOps is increasingly part of their professional development work.

    “In 2018 we had significantly more respondents specifying they’re involved in DevOps (an increase of 8 percent compared to 2017). In terms of Python users using Python as their secondary language, DevOps has overtaken web development,” according to the “2018 Python Developers study,” which is based on responses from 18,000 developers in 150 countries.

    Over the last two years, Python has grown more important to developers thanks to its flexibility, especially when it comes to web development. It’s no surprise that as other projects have gained popularity in the enterprise—DevOps and machine learning—developers have started adopting the language to other disciplines, especially as businesses invest more in application development.

    In TIOBE’s February 2019 index, Python now ranks third behind favorites such as Java and C, edging out C++ for its current position.

    Digging into the numbers, the survey finds that 43 percent of developers are using Python for DevOps projects, as well as system administration and writing automation scripts. That’s an increase from the 35 percent who reported the same types of uses for the programming language in 2017.

    The report notes that these numbers reflect a combination of Python used as either the primary or secondary language for various projects. As the report notes: “DevOps/system administration/writing automation scripts has moved into first place among Python users using it as a secondary language.”

    Machine learning is another up-and-coming category, with 38 percent of developers using Python for these projects compared to 31 percent last year.

    Overall, data analysis remains the No. 1 use case for Python developers, with 58 percent reporting that it’s their main focus when it comes to development.

    In second place is web development, according to the report.

    In terms of the top cloud platforms for Python developers, Amazon Web Service is the top choice, with 36 percent of respondents reporting that they use AWS. From there, it’s Google Cloud Platform (29 percent) Heroku (26 percent), DigitalOcean (23 percent) and Microsoft Azure (16 percent).

    About a third of those surveyed responded that they don’t use any cloud platforms.

    When asked about running code in a cloud production environment, 47 percent responded that they use virtual machines, although 40 percent of Python developers reported that they prefer containers. Another 28 percent use platform as a service (PaaS) and 21 percent are taking the leap to serverless computing.

    Additionally, when developing for the cloud, about 35 percent of those in the survey report that they use Docker containers.

    This increase use of Python within DevOps, as well as other connected areas such as serverless computing, does not surprise Anshu Agarwal, the CEO and co-founder of Nimbella, a cloud-agnostic, serverless cloud platform for developers. She noted the language’s use within a wide range of projects, specifically machine learning and AI, are driving this change.

    “Python developers are driving serverless requirements in new and unique ways, primarily because of the interest in data analysis, ML and AI,” Agarwal wrote in an email. “These data analysis and ML/AI-centric serverless use cases are different than request/response style serverless functions which run for much shorter durations. Python functions in serverless may have much longer durations (10s of minutes or hours), and consume a lot more memory. For such functions a different kind of serverless framework needs to be adopted that can handle long running workloads.”

    When it comes to continuous integration systems, about a third of respondents don’t use any. However, when they do, Jenkins is the most popular at 25 percent, followed by Gitlab CI and Travis CI, with 18 percent each. For configuration management, 20 percent reported using Ansible, while 9 percent used custom tools. Puppet, Salt and Chef also appeared in the list.

    Finally, about two-thirds of Python developers (69 percent) use Linux as their main operating system of choice for development. Microsoft Windows comes in a distant second at 47 percent and Apple’s macOS in third at 32 percent, according to the study.

    — Scott Ferguson

  • Red Hat Tightens CloudForms Integration with Ansible

    Red Hat Tightens CloudForms Integration with Ansible

    Red Hat has tightened integration between CloudForms, its tool for managing IT infrastructure on-premises and in the cloud, and the its Ansible framework for automating IT operations.

    Pete Cruz, product marketing manager for management for Red Hat, said version 4.7 of Red Hat CloudForms and Ansible together promise to make it easier to manage multiple IT environments running on-premises and in the cloud using a common set of tools. Most IT organizations today are trying to manage at least one public cloud alongside their existing IT environment. Rather than having to master a separate framework for each IT platform, IT organizations can employ multiple Red Hat tools and frameworks to unify the management of multiple clouds and their internal IT operations, said Cruz.

    Version 4.7 of Red Hat CloudForms now makes it possible to execute Red Hat Ansible Tower workflows directly from CloudForms deployed on-premises. Those workflows can already be extended to include multiple clouds.

    With this release of Red Hat CloudForms, Red Hat is continuing its effort to extend its reach into the management of networks and servers. CloudForms is now integrated with Lenovo Clarity, a set of systems management tools, and Nuage Networks’ Virtualized Services Platform (VSP). Integration with networking platforms is becoming increasingly pivotal because internal IT organizations are now being asked to be as agile as a cloud service provider, said Cruz, noting it’s not especially useful if an IT organization can spin up a virtual machine in a matter of minutes but must wait weeks as network services are provisioned manually.

    Red Hat is trying to lay a foundation on top of which best DevOps practices can be extended across multiple IT environments. With each additional cloud service an organization adds, the less agile the internal IT organization becomes, unless it finds a way to centrally manage all the platforms being employed using a common set of tools, noted Cruz, and the only way to effectively manage IT at that level is to rely on automation frameworks.

    Unfortunately, the number of organizations that are haphazardly embracing multiple clouds seems to outnumber greatly the number of organizations defining a strategy for unifying the management of hybrid clouds. But as the operational costs of managing multiple clouds becomes more apparent, the pressure to unify the management of multiple public clouds and on-premises IT environments inevitably will rise.

    In the meantime, savvy IT leaders are trying to ensure their organizations don’t become overly dependent on any one cloud service provider. The deeper an IT organization dives into the management frameworks provided by cloud service providers, the more difficult it potentially becomes to move an application workload, should the need arise. It does not follow that because a workload was built on one platform it never will move off that platform.

    Naturally, Red Hat, which is scheduled to be acquired by IBM, is not the only vendor with an eye on the rising need to unify the management of hybrid clouds. But the one thing that is certain is that even before the deal closes, Red Hat is aligning its portfolio to specifically address that opportunity.

    — Mike Vizard