devopsy.kinsta.cloud

Category: DevOps in the Cloud

  • VMware Extends Scope of Alliance with IBM

    VMware Extends Scope of Alliance with IBM

    On the heels of IBM announcing its intent to acquire Red Hat, VMware announced at the VMworld Europe 2018 conference this week an extension of its longstanding cloud alliance with IBM.

    Red Hat competes directly with VMware; yet, there are also more than 1,600 customers running VMware on the IBM cloud. Ajay Patel, senior vice president of product development for cloud services, said VMware remains committed to the IBM cloud alliance even while its relationship with Amazon Web Services (AWS) continues to expand.

    VMware and IBM jointly announced this week they have opened Joint Innovation Lab and that IBM Cloud Private Hosted can now be installed on VMware vCenter Server on IBM Cloud. In addition, IBM Cloud for VMware Solutions are now integrated with the IBM Cloud Kubernetes Service. IBM is also adding support for VMware NSX-T Data Center to provide VMware customers with access to a network virtualization (NV) overlay on IBM Cloud.

    In addition, VMware vRealize Operations automation software is now available on IBM Power Systems, and VMware will use the IBM Watson cognitive computing platform within VMware customer support portals.

    Despite the obvious tension between VMware and Red Hat and IBM and AWS, Patel noted that VMware has more customers running on the IBM cloud service than any other cloud environment. In addition, IBM, VMware and Red Hat all are committed to advancing Kubernetes. Point of fact: VMware, separately at the conference, announced its plans to acquire Heptio, a provider of management tools for Kubernetes and added support for Kubernetes running on top of VMware virtual machines in version 3.5 of VMware Cloud Foundation.

    VMware, via its alliance with IBM, is making a case for enabling customers to run stateless components of an application on Kubernetes, while maintaining stateful components such as databases on virtual machines, which provide better support for persistent storage.

    IBM said it intends to treat Red Hat as an independent subsidiary capable of making independent alliances, in much the same way VMware operates as an independent unit of Dell Technologies. It may take a while for IT organizations to get used to navigating strange-bedfellow relationships in the era of the cloud; for example, Red Hat has an extensive relationship with Microsoft that spans both Microsoft Azure and on-premises IT environments.

    What remains to be seen is the degree to which DevOps teams will navigate conflicting vendor alliances in isolation from one another instead of trying to unify the management of all those clouds under a common framework. A recent IDC report finds that while most customers (64 percent) are employing multiple clouds, the degree to which those clouds are interoperate varies widely. IDC finds that only 24 percent of IT organizations have a high degree of interoperability between their cloud environments. Another 40 percent say they have achieved low interoperability between their clouds. Only 7 percent say they have a true hybrid cloud in place.

    As the number of organizations with hybrid cloud capabilities starts to increase, the next battle in the cloud won’t be focused on what platforms are employed as much as what management plane might be employed to unify them all.

    — Mike Vizard

  • DevOps and Cloud: A Symbiotic Relationship

    DevOps and Cloud: A Symbiotic Relationship

    Most companies want to increase their competitiveness in today’s swiftly changing world, and so they cannot ignore digital transformation. DevOps and cloud computing have become two of the ways companies can achieve this needed transformation, though the relationship between the two is not easily reconciled—DevOps is about the process and process improvement, while  cloud computing is about technology and services. It’s important to understand how the cloud and DevOps work together to help businesses achieve their transformation goals.

    DevOps and Cloud

    Different organizations outline DevOps in different ways. This article does not debate which definition is correct, but rather presents them both to focus on the cloud’s benefit to DevOps. That said, DevOps definitions generally fall into two terms:

    1. In organizations it is defined as developer-friendly operations—IT operations are run separately yet in a way that is much more friendly to developers (e.g., self-service catalogs are provided to developers for stipulating infrastructure or providing technology-enabled pipelines for deploying new code).
    2. DevOps as a single consolidated team is habituated in organizations—developers take on operations responsibilities and vice versa.

    While cloud computing itself has several definitions, the most basic explanation of the cloud is a system that allows the provisioning of infrastructure (e.g., VMs or routers). Thus, it allows to being defined as code or templates, enabling the creation of repeatable processes that are not possible without cloud technologies.

    The aesthetics of cloud technologies and services become so easily amalgamated in DevOps vocabulary because they complement DevOps processes, regardless of how your organization defines them—or, regardless of which DevOps routes you travel to digital transformation with the cloud integration.

    Companies that focus on developers for operations often use cloud computing to speed developer productivity and efficiency. Cloud computing permits developers more control over their own components, resulting in smaller wait times. This application-specific architecture makes it easy for developers to own more components. By using cloud tools and services to automate the process of building, managing and provisioning through the code, service teams speed up the development process, eliminate possible human error and establish repeatability.

    Cloud computing also enables users to create self-service methods for provisioning infrastructure through AWS Service Catalog. Developers are able to quickly try new things, fail fast and just easily succeed in getting new products to market faster, without having to wait for IT operations to provision services for them.

    DevOps as a single term is really a mixture of these approaches, in which developers and operations work together using the cloud as a single common language. In this way, both the DevOps and cloud are able to work together, as everyone is learning new definitions and approaches at the same time. Developers and operations are equally cushy with the new language of the cloud, as developers often teach operations about the code aspect and operations can teach developers about infrastructure and security, developing a meeting point that leads to strong team dynamics.

    No matter the definition, the importance of cloud computing to DevOps can’t be accentuated enough. Cloud computing advances IT transformation, and with advanced tools and automation, it can enable companies to double down on their work to streamline and embed DevOps processes for greater efficiencies that are truly transformative.

    Inherently, as process improvement, DevOps also requires a culture change. Cloud computing can play a role here, as it can help codify and automate new processes. For example, if there is a new way in which developers should cluster the components they need to deliver a service—such as code, configuration, libraries and pipeline definitions—advanced automation tools can streamline the process and make it distinctly repeatable.

    The business case for DevOps combined with cloud technology for a successful IT transformation is fivefold:

    1. Bring products to market faster through faster access to development environments and streamlined developer processes.
    2. Automation and architecture as code reduce cloud complexity and even system maintenance.
    3. Add high security with automated, repeatable processes that serve to eliminate inaccurate error and, even more importantly, develop security controls from the very beginning.
    4. Reduce downtime through cloud-based continuous operations. Moreover, in the process of applying automation, developers can build stateless cloud application development, which increases availability and failover ability, in the process increasing business reliability and customer satisfaction.
    5. Increase scalability. One of the main reasons organizations look to cloud computing in the first place is its scalability, which allows organizations to increase capacity with the click of a button. When combined with DevOps, scalability becomes an integral part of applications as they are developed, while reducing the cost of infrastructure and increasing global reach.

    Get Past the Changes

    Together, DevOps and cloud computing are a powerhouse. While each offers greater effectiveness and business impact, together they are able to drive meaningful IT transformation that directly impacts business goals, regardless of the DevOps definition.

    At the same time, culture within the enterprise and among developers must change around the nuances of DevOps and its role in driving cloud development. Finally, IT companies must invest in these technologies and do so without an immediate goal—which, of course, drives corporate leaders and shareholders crazy.

    However, the alternative is doing nothing, which means certain failure. Competitors are likely to pull ahead in terms of time-to-market with solutions and application services. When you can hold up applications and processes in near real-time and do so within an elastic and efficient environment, the market will reward you for putting that effort. If you haven’t commenced on the process yet, it’s time to get started.

    — Jaymin Vyas

  • DevOps Chat: Microclimate and IBM Cloud Private with Toby Corbin

    DevOps Chat: Microclimate and IBM Cloud Private with Toby Corbin

     IBM Cloud Private has cemented itself in the DevOps space as a personal cloud environment, which is great for developers who need the agility and computing power of the cloud but don’t want to step into a public cloud setting. Tony Corbin, technical lead for the Microclimate Project, describes IBM Cloud Private as “a cloud specifically targeted at individual customer.”

    In this DevOps Chat, Corbin and I talk more about IBM Cloud Private and its role in DevOps. As usual, the streaming audio is immediately below, followed by the transcript of our conversation.

    Transcript

    Alan Shimel: Hey, everyone. It’s Alan Shimel, DevOps Chat, and you’re listening to another DevOps Chat. I happen to be joined today by a first-time guest on a podcast. So this’ll be an experience for him. It’s Mr. Toby Corbin of IBM. Toby, welcome to DevOps Chat.

    Toby Corbin: Thanks, Alan. How are you doing?

    Shimel: Doing great, and happy to have you on today’s show, Toby. Rather than introduce you to the audience in terms of your title and role and everything, I thought it would be a good icebreaker for you to kind of introduce yourself. So why don’t you kind of share with the audience, if you will, what your title is and what that means and kind of what you do with IBM?

    Corbin: Sure. So at the moment I’m Technical Lead for one of the squads in the Microclimate Project. Microclimate is our developing experience, which we’re targeting to help onboard customers and developers into the IBM Cloud Private. So my role and my squad is – we sort of deal with the backbone of Microclimate. So we deal with making sure everything runs smoothly. We add in things like analytics and the sort of analysis application. But we make sure the whole thing sort of runs. So we’re kind of the backbone. So the squad that I lead, we’re the backbone of Microclimate.

    Shimel: Excellent. So Toby, this sounds sort of like an Ops role, if you will, right? When we talk about the Microclimate – and you know what? Let’s back up. Some of our audience out here may not even be familiar with IBM’s Cloud Private offering. And I don’t claim to be an expert. I’m going to assume you know much more than I.

    Corbin: A little more.

    Shimel: Yeah, a little bit. I would hope so, so you’re not collecting your check under false pretenses, correct?

    Corbin: [Laughs]. Correct.

    Shimel: So Toby, let’s start with that. How would you describe Cloud Private to our audience?

    Corbin: So we’ve all heard of public cloud – things like Amazon have a cloud and Google have a cloud and IBM also has their public cloud. And these are quite large deployments. They’re hidden in the ether somewhere, no one knows where they are, and that’s where all the really, really clever analysis goes, and that computing power that IBM can bring. Cloud Private is sort of the onboarding into that. So Cloud Private is a cloud specifically targeted at individual customer. So they won’t be sharing that with anybody else. It will be an instance just for them and they’ll have the very similar applications and things on there that the public cloud would have. The intention being that when our customers are happy using this Cloud Private installation – so they don’t have to have their own machines. They can have the Cloud Private stuff working for them, doing all of their processing. When they’re happy having that working, we can then take them up to the public cloud, where they get even more of IBM’s processing power and they have an even better experience.

    So it’s kind of taking them from a private cloud that’s just for them up to the bigger public cloud, where although it’s still for them and they don’t share with other people, they have bigger computing power available to them.

    Shimel: Got it. So is it sort of a way station along an upgrade path? Or can someone stay on Private Cloud indefinitely?

    Corbin: Oh, you can absolutely stay on Private Cloud. You’re not forced to go up. But obviously as we go up through the clouds, the capabilities IBM can offer you get a lot greater. So that’s why often you’d find that the customers would want to move up to the next level, because there is an even greater amount of processing and a greater amount of options available to them and to their customer deployments.

    Shimel: So they grow into it, I guess. So Toby, I was under the impression, and maybe I’m wrong, that in Private Cloud there was also – I thought it was a Kubernetes-based offering. Is that –

    Corbin: That is correct, yes. So IBM’s Private Cloud is all based around Kubernetes. It’s IBM’s flavor. So it’s Kubernetes under the covers and IBM have made it sort of into their ICP, their private cloud offering. But it is Kubernetes-based, that’s correct.

    Shimel: So it’s containers with Kubernetes and this whole sort of – you know, the _____ _____, obviously, but now we’re starting to see the domination almost of Kubernetes kinds of architectures.

    Corbin: Yeah, and that’s how IBM’s helping our customers. So you think of the old way of doing things, you’d have a _____ application that was a huge monolith, a giant application that couldn’t really go anywhere. With IBM’s Private Cloud what we’re looking at is, how can we take these applications, deploy them on the cloud, free up all of the hardware – in a customer’s site they don’t have to worry about those things – and take these large-scale applications and deploy them into a cloud environment? And then it’s not just make them work in a cloud environment. How can we work with customers to actually make them change their applications and sort of rebuild them, if you like? Strip them down, put them back together again, as a microservice opportunity?

    And that’s where Microclimate comes in. That’s what we’re doing for the developers. It’s how you create those microservices.

    Shimel: Perfect. And of course, Toby – I think a lot of our audiences are familiar with the term “microservices,” and we see microservices being mentioned a lot around Kubernetes, around containers, around DevOps and cloud and so forth. And you anticipated my next question, which was, when we talk about a microclimate, is that sort of the climate in which we do microservices or perform microservices.

    Corbin: Yes. Well let’s say for this, yes. Effectively what microclimate is, it’s the environment that we want to give to our developers to go from having absolutely nothing to being able to develop their very first microservice, test it and then deploy it up into production through things like the Jenkins Pipeline and using Travis, and get it running in their cloud, all without actually having to need any prior knowledge. We sort of do this all for you. So in the past there was always ways of being able to create an application and there was always ways of pushing an application into a cloud, but it was several steps, different technologies. You had to rely on the customer having various knowledge themselves. So Microclimate is aimed to make this really seamless. You don’t have to leave your web browser; it’s entirely web-based. But we give you the ways to easily create an application, develop it in the cloud in a web-based editor, test, adding in these performance issues that we’ll talk about in a second, and then when you’re really happy, then get it run in production environments, all with the under-the-covers work being done for you by Microclimates.

    Shimel: That’s fantastic. So this whole Microclimate, if you will – excuse me, microenvironment and Microclimate, this is all managed via web browser? All web-based, huh?

    Corbin: Exactly. Because this gives you the opportunity that if you are fully running in the cloud, you don’t actually have to have anything on your laptop at all. Everything can be based in your ICP installation. You need nothing on your laptop to run it, which means then you don’t have to have Swift installed or Node installed. But you can still go and create and generate Swift and Node and _____ applications without having anything local to your machine. It frees up your environment to do everything in the cloud, so you’re straightaway using the power of the cloud to do your development.

    Shimel: hat’s fantastic. And Toby, it sounds to me, even more than that – you know, so I’m old, right? So I’ve been around for a while. And wen we first saw microservices and containers, to me this was like Heroku – better than Heroku. It was truly moving from infrastructure as a service to platform as a service. So that if you’re a developer and I want to write stuff let’s say in Swift or JS or whatever, I didn’t have to worry about, “Was that installed? What was the infrastructure like? What was going on? Was it installed in the OS level?” I could work directly on my application. Do these sort of Microclimate, microenvironments – is that what this really represents?

    Corbin: Absolutely yes. So that’s one of the beauties. Before coming on to work on Microclimate, I was doing Swift developments and Node developments, and I would have to have the latest versions of Swift on my machine and I had to keep up with them to make sure that I would have the latest bug fixes done or whatever it was to make my application run smoothly. With Microclimate, we take care of that whole environment for you. So when you say, “I want to create myself a Swift application,” you get a container running with the latest version of Swift. All of the security patches for that run are already applied. You don’t, as a developer, have to worry about, “Do I have the right installer? Do I have the right version of he compiler?” All taken care of. As a developer you can now just write code and we will do everything else in the background for you, so it really frees up the developer to do their job and not have to worry about keeping their machines up to date with all of the latest patches and run-suns and everything else to do that.

    Shimel: That’s pretty nifty. Good stuff right there. So then, Toby, as part of your – you know, you’re a team lead with portal, you had mentioned, and – but you know, but your whole team and what you’re doing – would you say most of the people you’re serving are developers? Or do you kind of correspond with their Ops team, who are still servicing the developers within their organizations?

    Corbin: So this is where Microclimate has nicely evolved. We started off s being the developer experience. So we were the developer experience _____. And we were aimed at, how do we nicely onboard the developer from having nothing on their laptop to being able to write their first microservice and deploy it to the cloud? But then we quickly realized that we need to have DevOps. So we brought on a whole DevOps pipeline to the backend of Microclimate. So Microclimate is really _____ _____ as an end-to-end story of two either distinct parts, if you like, but they actually work in one. You start with, “I’m on my laptop, even though I’m running in a cloud environment.” So I’m coding on my laptop and I’m writing my first microservice. And then when I’m happy, I’ll use Microclimate to actually send it to my preconfigured Jenkins Pipeline, and the backend of Microclimate that deals with DevOps will take that and will build you on your Jenkins Pipeline, it will push towards your production and you’ll actually go the full end-to-end now.

    So Microclimate is not just really about writing your first piece of code now. It’s taking any application and pushing it right through to deployment at the other end and have it run in production.

    Shimel: It’s end-to-end. That’s beautiful, man. That’s really nice. So share with our audience, if you can a little bit, what are the kinds of things that you and your team are actually doing as part of maintaining this environment?

    Corbin: So one of the things that we brought – so as I said before, my background in this, I actually came from doing tools monitoring for things like Java and Node. We now bring that to Microclimate. So we can think of things like a performance issue that you would typically see. You know, my CPU is going mad and it’s at 100 percent, or my disk is thrashing or running out of memory. And often it’s very difficult to know why that’s happening or even if it’s supposed to be happening. So what we do, what we bring to Microclimate is, we bring to the table our analysis tooling that we have outside Microclimate. So when you use Microclimate, we actually build in our monitoring stuff. So that means that when you write your very first line of code, we actually start monitoring it straightaway.

    So you don’t even have to think about, “Okay, do I have to monitor my application for performance problems?” or, “I’ve got my application written. We normally have a performance team that does this.” We give it all to the developer. Because if a developer can see a problem with a line of code they just put in – for example, they’ve just coded a new function and when they run that function their fan goes mad on their MacBook and it’s thrashing away and they think, “What have I just done?” If we can tell them what they’ve just done at the point they do it, this speeds up development in the future and your testing gets quicker, because you don’t get a performance team two months down the line saying, “Hey, we’ve just run your stuff and it performs like a dog,” and then it comes back to the developer to have to re-find-out what they’ve done. We show you straight away. As you type your code in and hit Run for your first test, we can show you what performance was, so you can see, am I regressing? I can see where my CPU is going mad or I can see where my hard disk is lighting up or I can see where I’m running out of memory.

    We build that in for you. So hitting the developer experience right from day one means that hopefully when you get to the end and your application deploys, there’ll be no surprises around, “Well it’s not running anywhere near as well as I thought it would.” So that’s what we bring to Microclimate.

    Shimel: Got it. Like you’re the developer’s best friend.

    Corbin: We’d like to think so, yes.

    Shimel: So Toby, as I mentioned before, I’ve been around the block a little bit. You know, one of the problems with what you’re describing is territorial issues, let’s call them, right? “Hey man, I’m the Ops guy here from my org, and I’ll help my developer, thank you. Don’t be giving away our secrets, telling the developer some of these performance things.” You know, performance kinds of insights. Do you run into having to deal with pre-existing performance teams as part of the Ops at these organizations?

    Corbin: Well not so much, because we actually find now is that the sort of dynamic of a team is changing. As we go into this sort of more agile world where each team is intended to be self-directing, we find by having smaller teams with sort of jack of all trades, we get away from this developing, developing, developing, testing, testing, testing situation that we _____ _____ five or ten years ago. And this means that much smaller teams can deliver functionality that’s fit for purpose a lot quicker, because you’re not waiting for a longer test run at the end or other sort of testings that happen that would typically end up in this loop of sort of, “I deliver my code, someone tests it, it comes back to me to make changes, someone tests it and it comes back.” This is a sort of way development has now evolved. So with the agile world, smaller teams can do things a lot faster, and that’s why giving them the tools they need means they can do that.

    But this doesn’t mean we put testings out of business, as it were. Because we aim our stuff at the developer, for like the piece of function they’ve just written or for what that application is doing as a whole. There will always be a need for sort of larger-scale teams that need to oversee the entire application. So whereas before, if we take the example of the giant _____ monolith that was one giant application, you might find in today’s environment that that’s actually seven, eight, nine, ten small microservices. So a developer would be working on one microservice and he’d get the performance stats back from that microservice in isolation. But what about the bigger picture? What about when all ten of those are working together? How do you get the overall view of your microservice environment that’s 10, 20, 30 things all talking to each other? So that’s where you still have DevOps teams and other teams working on that. And there’s software for that as well.

    So as an example, there’s Prometheus, that’s an open source monitoring solution that’s very good at taking data from different sources. And then what we do with the stuff that we add into the applications at the developer level so that the monitoring solutions we put into Microclimate for the developer, that will actually talk to Prometheus. Which means that if the developer is happy seeing their code running and their single microservice view of the world all looks good – “Yes, my database function is performing as I expect it too.” I’ll now put it into the cloud where there’s 20 other microservices running, all talking. And Prometheus, that sits over and above all of this, can now take the data from each of those 20, and the data they’ll be taking will be the same data that the developer has already been seeing when running it locally or in the small view. We just pass it up through. So this again gives the developer an easy way of seeing – or actually in fact _____ ____, the SysOps guys, what their applications are doing without any additional changes. Because the code has been in there from day one to provide this information.

    Now that’s one of the hardest things about monitoring. It’s, how do we get at the data? If you can get at the data, there’s plenty of ways to visualize it. All sorts of fantastic tools, even the simplest of just writing out to the screen. It’s always the case of, how do we get to the information? So we build the information in from day one, and that’s what makes Microclimate so cool, as it were, because it’s just there for you. And no matter what sort of environment you run in, we’ll be able to make the data available to see the performance at any level.

    Shimel: You know, Toby, that’s excellent. Hey, when we started – before we started recording I told you, “You’re going to be surprised when I tell you, ‘Hey, we’re going over time.’”

    Corbin: [Laughs].

    Shimel: So we’re really way over. That was a difficult interview. It was pulling teeth to get you to talk to us.

    Corbin: [Laughs]. I’m sorry.

    Shimel: No, not a problem. Toby Corbin, IBM’s Microclimate Developer Experience Tribe, thanks for being our guest on DevOps Chat today, and continued success with the Cloud Private team at IBM.

    Corbin: Thank you very much for having me on, Alan.

    Shimel: My pleasure. This is Alan Shimel. devopsy.kinsta.cloud. You’ve just listened to another DevOps Chat.

    — Alan Shimel

  • How IBM’s $34B Bid for Red Hat Will Spur DevOps Adoption

    How IBM’s $34B Bid for Red Hat Will Spur DevOps Adoption

    IBM’s proposed acquisition of Red Hat for $34 billion will have an immediate impact in raising awareness for a programmatic approach to managing hybrid clouds based on best DevOps practices.

    A recent survey published by International Data Corp. (IDC) finds most organizations (64 percent) have already embraced multiple clouds to varying degrees. Only 7 percent of respondents said they have built a truly hybrid cloud. Another 24 percent, however, said they have achieved a high degree of interoperability across multiple clouds. IBM made it clear this week that its primary motivation for acquiring Red Hat was to tap into a trillion-dollar opportunity that will manifest itself as hybrid cloud computing continues to evolve. In fact, IBM is betting that Kubernetes clusters will play a key role in advancing the state of hybrid cloud computing.

    But for hybrid cloud computing to take hold, it’s also clear the level of mastery over DevOps within enterprise IT organizations will need to improve substantially. In fact, hybrid cloud computing might not even be the ultimate destination, said CloudBees CEO Sacha Labourey. Enterprise IT is increasingly being defined by programmatic services rather than platforms, he noted, and those services need to be weaved together to drive modern applications. But where each of those services resides is becoming less relevant as organizations become more DevOps proficient, Labourey said, noting the rise of hybrid cloud computing will drive more organizations to embrace DevOps to bring order to the services being consumed.

    “Within a hybrid cloud DevOps will be required,” said Labourey.

    While platforms are still relevant, they increasingly are being hidden behind services that expose an application programming interface (API), added Chip Childers, CTO of the Cloud Foundry Foundation, which oversees the development of the open source Cloud Foundry platform-as-a-service (PaaS) environment. A developer consuming a service might have little to no idea what platform that service runs on.

    “IT teams are building on the abstractions that have gone before,” Childers said.

    The ability of one organization to differentiate itself will come down to the maturity of the DevOps processes they rely on to most efficient consume a service, he added.

    Demand for that level of DevOps expertise is already starting to accelerate, said Tim Beerman, CTO of Ensono, a provider of managed IT services. As IT continues to transform into microservices enabled by containers and serverless computing frameworks, the complexity of the overall IT environment requires a more sophisticated approach to managing the overall environment, he said.

    But, he added, “A lot of organizations simply don’t have the talent.”

    Of course, IBM is betting that once it owns Red Hat, more organizations will look to IBM Services for that expertise. But as always, competition across the IT services space remains fierce. IBM is also betting that it will provide the management plane through which hybrid cloud services will be unified. But even there, competition from rivals small and large will be significant. Vendors such as Rackware are already providing frameworks for managing hybrid clouds.

    While hybrid cloud computing is still relatively immature, organizations that embrace DevOps will be much further down that path than others, said Todd Matters, chief architect at Rackware.

    “DevOps will absolutely be a critical part of hybrid clouds,” he said.

    IT has always been about striking a balance between people, process and products. It’s too early to say whether IBM will recoup its $34 billion investment to acquire Red Hat. But the deal inevitably will lead to a greater awareness for a more comprehensive approach to DevOps capable of spanning a hybrid cloud computing environment that includes multiple platforms.

    — Mike Vizard

  • Cost Per IT Ticket: Think Beyond Opex in Your Cloud Journey

    Cost Per IT Ticket: Think Beyond Opex in Your Cloud Journey

     Globally, organizations have embraced cloud computing and delivery models for their numerous advantages. Gartner predicts  the public cloud market is expected to grow 21.4 percent by the end of 2018, from $153.5 billion in 2017. Cloud computing services provide an opportunity for organizations to consume specific services with delivery models that are most appropriate for them. They help increase the business velocity and reduce the capital expenditure by converting it into operating expenditure.

    Capex to Opex Structure

    Capital expenditure refers to the money spent in purchasing hardware and in building and managing in-house IT infrastructure. With cloud computing, it is easy to access the entire storage and network infrastructure from a data center without any in-house infrastructure requirements. Cloud service providers also offer the required hardware infrastructure and resource provisioning as per business requirements. Resources can be consumed according to the need of the hour. Cloud computing also offers flexibility and scalability as per business demands.

    All these factors help organizations move from a fixed cost structure for capital expenditure to a variable cost structure for the operating expenditure.

    Cost of Assets and IT Service Management

    After moving to a variable cost structure, organizations must look at the components of its cost structure. They include the cost of assets and the cost of service or cost of IT service management. Cost of assets show a considerable reduction after moving the entire infrastructure or assets to cloud. The cost of service remains vital as it depends on the day-to-day IT operations and represents the day-after-cloud scenario. The leverage of cloud computing can only be realized if the cost of IT service management is brought down.

    Growing ITSM or ITOPs Market and High Stakes

    While IT service management (ITSM) has taken a new avatar as IT operations management (ITOM), incident management remains the critical IT support process in every organization. The incident response market is expanding rapidly as more enterprises are moving to cloud every year. According to Markets and Markets, the incident response market is expected to grow to $33.76 billion by the year 2023 from $ 13.38 billion in 2018. The key factors that drive the incident response market are heavy financial losses post incident occurrence, rise in security breaches targeting enterprises and compliance requirements such as the EU’s General Data Protection Regulation (GDPR).

    Service fallout or service degradation can impact key business operations. A survey conducted by ITIC indicates that 33 percent of enterprises reported that one hour of downtime could cost them $1 million to more than $5 million.

    Cost per IT Ticket: The Least Common Denominator of Cost of ITSM

    As organizations have high stakes in ensuring that business services run smooth, IT ops teams have additional responsibility in responding to incidents faster without compromising on the quality of service. The two important metrics for any incident management process are 1) cost per IT ticket and 2) mean time to resolution (MTTR). While cost per ticket impacts the overall operating expenditure, MTTR impacts customer satisfaction. The higher the MTTR, the more time it takes to resolve the tickets and, hence, the lower customer satisfaction.

    Cost per ticket is the total monthly operating expenditure of the IT ops team (IT service desk) divided by its monthly ticket volume. According to an HDI study, the average cost per service desk ticket in North America is $15.56. Cost per ticket increases as the ticket gets escalated and moves up the life cycle. For an L3 ticket, the average cost per ticket in North America is about $80-$100+.

    Severity vs. Volume of IT Tickets

    With our experience in managing the cloud IT ops for our clients, we understand that organizations normally look at the volume and severity of IT tickets. They target the High Severity and High Volume quadrants to reduce the cost of the tickets. However, with our experience, we strongly feel that organizations should start their journey with the low hanging fruits such as the Low Severity tickets, which are repeatable in nature and can be automated using bots.

    In the next blog, we will elaborate on this approach that can help organizations in measuring and reducing the cost of IT tickets.

    — Neeraj Deuskar

  • DevOps Chat: Security in the Cloud with Dome9’s Zolar Alon

    DevOps Chat: Security in the Cloud with Dome9’s Zolar Alon

    Dome9 is in the business of cloud security, so it’s no wonder some of the world’s top companies rely in it to secure their sensitive data and applications in Amazon Web Services, Microsoft Azure and Google Cloud environments. And with last week’s announcement of Check Point’s aquisition of Dome9, an even greater number of users will be able to take advantage of its SaaS solution.

    In a recent DevOps Chat, Dome9 co-founder and CEO Zohar Alon spoke with me about what he sees in the cloud security space as many organizations embark on their second generation of public cloud expansion.

    As usual, the streaming audio is immediately below, followed by the transcript of our conversation.

    Transcript

    Alan Shimel: Hello, everyone. It’s Alan Shimel, devopsy.kinsta.cloud, Security Boulevard, and we’re here for another DevOps Chat. Today’s DevOps Chat features and old friend of mine from the security world, Zohar Alon, CEO and co-founder of Dome9. Zohar, welcome to DevOps Chat.

    Zohar Alon: Alan, great to be here.

    Shimel: My pleasure to have you here, my friend. Zohar, let’s get this out of the way first. Some of our listeners are not necessarily from the security world. They’re from the DevOps world. They may not be familiar with Dome9. So let’s first let them know a little bit. You know, you live and breathe it, but tell them who Dome9 is.

    Alon: Sure. So Dome9 is a SaaS solution that helps security organizations and some DevOps and development organizations secure their public cloud environments as they grow and prosper from the single cloud to the multi-cloud focusing on AWS, Azure, and Google Cloud platform. A multitude of solutions.

    We’ve been in the market from the very beginning covering elements like network security in the cloud which is a bit different, governance and compliance, continuance compliance, and how our customers can tie continuance compliance to their development processes and their build processes and things like identity and access management protection for public cloud that prevent attackers from targeting the developers as the weak link as a vector of attack for compromising the credentials in real time by hacking into their laptops and pigging-backing on their authenticated and trusted sessions to create mayhem and havoc in the public cloud environment of the customer. Just celebrated joining our hundred employee and going strong in both our headquarters in Tel Aviv and Mountain View, California.

    Shimel: Absolutely. Very good. So Zohar, and I should mention, look, I know you and Dome9 pretty much from the year you started if not the day you started, right? I think it was an RSA event?

    Alon: Yes, yes, 2012, very early on.

    Shimel: I was going to say, you know, in a previous – you have a long history in the InfoSec world. Look, the topic for today is really have we seen, you know, the classic, you know, crossing the chasm model of early adopters, early mainstream, late mainstream, laggards, where are we with this in cloud adoption as it relates to cloud security? And at this point are we confident in saying that, look, it’s basically just the laggards at this point who haven’t already at least started their cloud migrations?

    Alon: I can say this for certain, that the level – at Dome9 we focus on the top segment of the enterprise customers, let’s say the global 5000 organizations and their cloud deployments; and there’s not a single industry, there’s not a single geography that we are not seeing a race to the cloud, in many cases, multi-cloud deployments very early on. What I’m sensing is that we have a lot of organizations that already completed their kind of version one of public cloud experience that was comprised of some lift and shift and some building of new systems in the cloud. They are now busy architecting their next generation, their second generation of public cloud expansion where elements of making not just the default but actively working to decommission the old infrastructure.

    And remember the motivation is you always have a software or a hardware refresh cycle happening somewhere, whether it’s for your switching fabric or for your compute fabric, in your various data centers or you need to upgrade those Windows 2008 and it’s all involves some interesting costs with it. So the drivers are already there. Once you already get the momentum and the velocity and the confidence, it’s something that is inevitable and it actually happens faster and is encouraged by kind of the known technical people, whether it’s the procurement or the finance people that realize that their support to this activity could save the organization actually a lot of money.

    We saw that AWS revenue run rate, just AWS is around $40 billion now, and we just saw that the big Cisco revenue run rate is $48 billion. So the reality that we have crossed the chasm and even kind of gotten to the late point. The late majority is already feeling it quite seriously. I can say this for certain with a lot of certainty.

    From our perspective at Dome9, we are also seeing organizations already architecting their second generation of what is to deliver continuous security and compliance and governance to their new architectures in the cloud. This is kind of it goes hand in hand. This is where we also see this kind of the maturity in the level of the people that we talk to and their requirements and their understanding that there’s not just a single or few parameters you need to take care of, but if you count them there are about 10 different areas of attention that you need to be very well covered with in order to provide to continue that trend and complete the migrations and sell those data centers as Intuit just did. I think they announced about two months ago that they sold their last data center. I believe we’re going to see a lot more of those announcements. We don’t need to worry. This real estate is not going away. Amazon buys it through Equinox and its partners on the other hand. So it’s not as if we need less compute. We just need a better, more resilient, more focused compute environment.

    Shimel: Understood. Couple of things you mentioned in there that I want to bring to up or I’d like to kind of dive a little bit. First of all, yes, I do agree with you, even the late majority has moved to the cloud; but I think one of the things we’ve discovered about cloud migration is no one up and picks their entire – or very few people but some do – up and pick the entire infrastructure en masse over to the cloud, right? It’s a phased sort of migration, if you will. Some faster than others. And I’m talking now for large enterprises. I’m not talking startups. Larger enterprises. How long a migration do you think that usually is?

    Alon: Look, so the key is the data. You know, when Amazon announced their Snowball device to lift and shift a couple of petabytes, I think it was three, four years ago at Korean event, people were in shock and like about the thinking of getting the data to AWS is, you know, FedEx is faster than the largest wipe you can find. And then they surprise last year with the full-blown semitrailer that has essentially storage on it that can take essentially any amount of data from your premises to the public cloud. So this is one thing.

    Once you get the sense of trust with some data, then trusting the public cloud with your entire dataset is not something that is inconceivable, right? Trust is very binary. If you have it and if it’s good for the backend of my mobile application with the replication of a lot of my banking stuff, for example, then it’s good for everything. Then it’s just the technicalities on how we get there.

    Now, I think one example I want to give is that other than data that can there are no barriers, sometimes the database can be a barrier and that’s why you’d see solutions like AWS will sell you migration services to move from your own – they’ll practically give it to you to move from your Oracle to their cloud-based RDS system whether it’s Aurora or whether it’s any of the other SQL versions that they offer as a service. You can say that probably what’s left is the big old mainframes.

    I was fortunate enough to see that in real time about a year ago where Vanguard – so essentially after we figured out how to migrate data and databases, the only thing you might say may prohibit us from flying at speeds to the cloud may be an old system of record that sits tucked in in a mainframe computers in some data center at an unknown location. We have been able to see, to experience, and to help also several financial institutions in the process of securing a lot of replication process where they showed how they create using serverless computing and the AWS DynamoDB, a replica of their system of record of their big old mainframe, in the public cloud to provide essentially zero latency backend to their entire application framework. And I’m certain that one of the examples is Vanguard. There’s a very famous video of their heard of architecture describing how they used I think seven steps to create that replica of their mainframe.

    It’s an amazing—and this was announced, this was shown a year ago, so you’d probably think that they’ve been working on it for two years before, and this is the reality, that when it started making sense, you know, the technological barriers to live in tandem until you are forced to make a decision about those old systems or those legacy systems is something that any organization should strive for.

    And, you know, you don’t need to work hard, you know? Just, your public cloud rep, whether you go with Google Cloud or Azure or AWS, they will help you with providing the perfect architecture and how you’re baking security into it and how you optimize it for cost—they will pull in the partners that will help you on those things to kind of tie them across the board. And the reality is that—and that’s why I can say that, unless there is real slowness and some legislative or some other reason that could prevent an organization from starting its exploration, most of them are already picking it.

    And sometimes it’s not advertised. I’m surprised to see, you know, industrial companies that come to us and, you know, I ask them how much is their public cloud spend, and they say, “We just crossed the $10 million-a-year mark.” And that’s a serious business. That means that the current way that things are going, they’re gonna get to ($50 million) or to ($100 million) in the next two years. And this means that you’re not investing in your on-prem systems any more. That’s what I’m seeing in the market.

    Shimel: You know, I’m not gonna argue with what you’re seeing. You’re out there talking to people doing it. Zohar, unfortunately, we’re way past our time, but you know what, I’d love to have you back on. I know we had a little bit of a technical glitch; hopefully, it’ll be ironed out where people are listening to.

    But let’s continue this discussion. I think it’s fascinating stuff, and I think people are interested to know. And we didn’t even—I really wanted to touch on what does this mean for cloud security? What does it mean for DevSecOps? Let’s do that in part two, if it’s okay with you.

    Alon: Absolutely. Looking forward.

    Shimel: Okay. Zohar Alon, Founder and CEO of Dome9—thanks for being our guest on DevOps Chat. This is Alan Shimel. You’ve just listened to another DevOps Chat.

    — Alan Shimel

  • SAP Embeds Integration Services in PaaS Environment

    SAP Embeds Integration Services in PaaS Environment

    SAP, at its SAP Tech-Ed Europe 2018 conference, expanded the scope of the integration services it provides within SAP Cloud Platform, the platform-as-a-service (PaaS) environment it provides for building custom applications based on the open source Cloud Foundry project.

    Dan Lahl, global vice president of product marketing for SAP Cloud Platform, said SAP is now adding a Content Advisor service that makes use of machine learning algorithms and other forms of artificial intelligence (AI) to recommend what integrations should be created next based on those that have been previously deployed on the platform.

    In addition, SAP announced the general availability of SAP Cloud Integration Automation Service, which provides deeper integration with a standard set of metadata SAP is in the process of defining for all its packaged application software, and SAP Cloud Platform Open Connectors, a set of application programming interfaces (APIs) based on code SAP has licensed from Cloud Elements.

    SAP also revealed that it will start making an on-premises edition of SAP Cloud Platform available via partners including IBM and Atos beginning in 2019. Lahl said that decision was driven by customers that for either regulatory or intellectual property concerns did not want to deploy custom code on any cloud service. Those efforts become more feasible once SAP finishes porting its distribution of Cloud Foundry to run on top of Kubernetes, which presents a more efficient approach to deploying an application development platform using a cloud-native platform that can run anywhere. While SAP remains committed to Cloud Foundry, the cost and effort of standing up instances of a Cloud Foundry PaaS in its original form are considerable.

    Finally, a new SAP Business Partner service on SAP Cloud Platform, planned for general availability in November, will provide access to a set of master data services that simplify access to and orchestration of enterprise business applications. The master data services can be shared and reused for any LOB-specific intelligent business processes to ensures a consistent and comprehensive view of customers, sales and products. In effect, SAP is now providing compliance as a service, said Lahl.

    The integration additions to the SAP Cloud Platform should improve performance and help drive down costs, he said, because  collectively they eliminate the need to rely on an external integration service when building and deploying applications on SAP Cloud Platform.

    SAP has put considerable effort into building out SAP Cloud Platform as part of a strategy that provides a layer of isolation between SAP applications running on the HANA database and custom applications running on Cloud Foundry. That approach enables SAP to deliver a quarterly cadence of updates to both its packaged applications and HANA running on a cloud service without having to worry about the impact those updates might have on custom applications. One of the reasons why so many SAP customers are several releases behind on SAP packaged applications is because they have all been customized extensively in on-premises IT environments.

    As part of that effort, SAP is now also making available a series of best practices for implementing DevOps processes across its platforms. It may be a while before organizations that have standardized on SAP platforms master those processes. But modernization of those platforms is now more a matter of when rather than if.

    — Mike Vizard

  • The Truth Behind DevOps on the Cloud

    The Truth Behind DevOps on the Cloud

    DevOps on the cloud may seem simple enough, but in reality it can be very complex

    The ultimate goal of a DevOps administrator is to make sure that everything is running properly and seamlessly—so much so that management is convinced that the DevOps administrator is playing solitaire all day. To that end, a successful DevOps team should have their systems automated such that the level of friction they impose on the organization is minimal.

    Leveraging a cloud environment makes it look easy—almost too easy, making it look as though all DevOps needs to do is press a button. IT can be deceived into thinking that the level of complexity in DevOps on the cloud is trivial, when in reality it is not.

    Running DevOps on the cloud is like renting an apartment in someone else’s building: Even though you don’t own the infrastructure, you’ll have to deal with any issues that may arise. Similarly to a neighbor who’s too noisy, you may experience different levels of performance from the same servers with the same capacity at different times of the day because someone else is running on the same hardware.

    There also are security threats that don’t come into play when running on your own hardware. If the same physical machine that runs your most important computations is also being used by Mr. Nefarious, he can use all sorts of tricks and hacks to gain access to valuable data. The Meltdown Inspector attacks is one example—it impacted all cloud providers with severe fleetwide performance impact, sometimes exceeding 10 percent, and required expensive mitigation steps.

    Making Sense of Your Gold Mine of Data

    The wealth of information provided to system administrators is awe-inspiring, terrifying and frustrating all at once. There is so much information available that, in theory, we should be able to troubleshoot and fix any and all issues. Unfortunately, making sense of the vast amount of information available and nailing down the critical pieces of information necessary can be extremely complex.

    The flat network topology typically expected of an on-premises data center is drastically more complex on the cloud. Beyond just VPN and DMZ, there are multiple isolated networks, containers running VMs and running on segmented networks going through NAT with multiple levels of routing and dispatch at play. Trying to troubleshoot in such an environment can be an exercise in frustration, because you don’t own the entire network.

    Certain tools and abilities you might take for granted in your own data center are not available, such as Tcpdump, a low-level tool to monitor network packets. For example, in VPN packet loss scenario, your cloud VPN server is shared with other clients, so  you won’t be able to capture packets to troubleshoot the connection issues.

    Also in managed services—in particular databases—the debug endpoints and tools that often are used in other environments to troubleshoot are not available, to avoid exposing data from other users. For example, you cannot get the full logs from the server to figure out what it is doing. That can hide critical goings that are too precious to ignore in production.

    Evaluating Cloud Providers

    Since it’s inevitable that problems will occur, one of the important considerations for selecting a cloud provider is the level and quality of support they offer. It’s important to go beyond simple SLAs and evaluate the actual quality and effectiveness of the support you can receive.

    Take, for instance, a situation where a machine would freeze, effectively at random, for long periods of time. All monitoring and metrics have told a simple story. The machine was up, had plenty of resources and capacity to spare, but simply would not respond every now and then for any connection attempt whatsoever. Troubleshooting involved stripping the problem to its core and being able to reproduce this in a minimal fashion, at which point one would contact the cloud provider support to try and understand what is going on. The actual problem turned out to be network port exhaustion in the NAT router, which caused a remote server to reset TCP connections, leading to long timeouts that froze the system. This particular problem took days to resolve and required a major change in the deployment topology.

    Running DevOps on the cloud is not an easy task, and the cloud itself is not a silver bullet. Rather, it’s a way to democratize hardware and operation resources and work effectively at scale. If you can manage the new challenges, you can enjoy a new level of efficiency.

    — Oren Eini

  • Building Great Cloud Security Guardrails

    Building Great Cloud Security Guardrails

    Security guardrails are an incredible way to keep our cloud deployments safer without slowing things down. Taking this structured approach will minimize friction while increasing protection.

    When I first started working hands-on in cloud (AWS) around nine years ago, I quickly realized both the power and risks of just how darn fast and agile cloud could be. Being, on occasion, an optimist, I decided this capability was a significant upgrade to our security capabilities. Sure, I could accidentally spin up internet-exposed servers, but in exchange I gained levels of central control and visibility unheard of in traditional infrastructure.

    One of the most powerful new capabilities is the ability to leverage cloud APIs and automation to build automated security guardrails without having to spend a gazillion dollars for a big box that actually delivers as promised, but also adds tremendous friction and slows things down. Guardrails are automations that constantly watch your deployments, find deviations from desired baselines and can even automatically remediate issues. Yet, even cloud-native guardrails can become problematic if they aren’t designed properly. Having both built a fair few myself and worked with many organizations as they built out their cloud security programs, here are some of the lessons I’ve learned along the way and some code to show you how it works.

    Building Awesome Guardrails

    My first guardrails were a bit of a hodgepodge in terms of structure and capabilities. And practically speaking, there are a lot of ways to attack the problems. Even today I’ll take different approaches depending on the particular challenge in front of me, but most of my guardrails (including operational ones) tend to follow a consistent pattern:

    • Define the problem: I like to start by defining the business outcome we want, and then move onto the technical problem. I used to start with a narrow technical problem to solve but found that didn’t always lead me to the right outcome. For example, is the problem that we want to stop all internet-facing port 22 access, or is the real outcome we don’t want to expose internet-facing admin servers to the open internet and we want to use a guardrail to restrict to our known corporate IPs instead? This helps prevent a guardrail from becoming a blocker.
    • Set the scope: Multiple cloud accounts (or subscriptions/projects for you Azure/GCP people) are now the standard. The first step is to determine how broad a guardrail you want, which will also define some of the technical specifications. For example, if you want it to apply to multiple accounts in Amazon Web Services, that affects where you run it from and how you manage the required IAM privileges.
    • Build or pick the deployment model: There are a lot of technical specifics regarding where and how to run your guardrail. Today’s example will run in AWS within a single account, but if you manage a larger, enterprise-class deployment you will need to look at how and when to balance centralized and localized guardrails. More often than not, I prefer a serverless approach (FaaS with other supporting PaaS), but I’ve also found longer-running guardrails often need containers as well. Thanks to API gateways, it isn’t bad to build deployments that can support both.
    • Define filters: It’s awesome that you can automatically close all internet-facing security groups with 20 to 40 lines of code—right until you break a production application. Filters allow you to tune your guardrail for a particular environment or project’s needs. Basic filters might work on a simple resource tag, while more complex filters might evaluate a combination of include and exclude rules for greater flexibility. This can, for example, treat development and production environments differently from the same guardrail.
    • Determine triggers: We have a rich range of trigger options on the major cloud platforms today. The main categories are time-based assessments and event-based triggers. I tend to use both, with event-based for more deterministic events (an S3 bucket policy was changed) and time-based sweeps to catch conditions I might miss purely on an event trigger.
    • Perform the analysis: This is the meat of our guardrail. Based on the trigger, collect the information we need and, in combination with our scope and filters, make a decision. For example, if we find port 22 open to the internet, if it is a Dev environment we might decide to update the security group to restrict to known corporate IP addresses. If it is in Prod we might delete the rule completely unless it was implemented through a CloudFormation template that went through an approval process.
    • Send notifications: Even when you have a completely automated guardrail, you likely want a parallel notification. These can go straight to email/text/Slack or drive into a ticketing system. For security guardrails, I also recommend sending them to your SIEM or other security alert/management system. If you’re fancy, you can turn these into the first step of a chatOps process.
    • Take actions: Actions are what separates guardrails from all the annoying alerts of the past. Based on the analysis and filters, actions can be fully automated remediations or request human intervention. Actions also can follow different paths based on the conditions to provide the best options for that situation. The key is that even if a human is called to make a decision, the decision should be a simple click to run the action, saving the person from having to fix things manually. Unfortunately, we still sometimes need human beings to make a decision. Then again, the alternative is bowing down to our robotic overlords. It’s a trade-off, I suppose.
    • Validate and log: Don’t assume the action actually did anything. Rechecking is vital, as is logging and sending the results as a second notification to the ticketing system/SIEM/etc. I fully recommend sending both incident open and close notifications to the destination system, even if the guardrail fixed everything within seconds. This keeps a consistent record and also allows you to do nifty things such as measure your mean resolution time and compare automated versus manual.

    Running a Demonstration Guardrail

    One of my favorite guardrails to demonstrate is automatically reversing security group changes in AWS. Here’s how to set it up for yourself and run my demonstration code to see what it looks like:

    • The first step is to make sure you don’t start running this in production or anyplace else where you might break things before you actually tune the filters. If you do, it isn’t my fault. The desired outcome in this case is ensuring we don’t have network exposures due to unapproved security group changes on designated resources.
    • The scope of this guardrail is a single local account, with a corresponding deployment model. We will use CloudTrail, CloudWatch Rules, Lambda and SNS.
    • There are three prep steps to configure your account before building the guardrail:
      • Log into the AWS console and set up a destination for notifications. If you haven’t done this before, you want to use SNS (Simple Notification Service). Create a topic for security alerts and a subscription to wherever you want to send the alerts. I recommend starting with a simple email subscription.
      • Make sure you have CloudTrail turned on and streaming to CloudWatch. For you newbies, CloudTrail records (almost) all the API calls on your account and we need this for our trigger. Sending the logs to CloudWatch allows us to build near real-time triggers in CloudWatch based on events such as the API call we will look for.
      • Create an IAM Role for Lambda to use. It will need the rights to AuthorizeSecurityGroupIngress, DescribeSecurityGroups and RevokeSecurityGroupIngress. To save logs, it also needs PutLogEvents, CreateLogStream and CreateLogGroup. These are also in a JSON template in the project repository for cut and paste.
    • With that in place, create a new Lambda function (Python 2.7) and load in our code located at https://github.com/disruptops/RevertSecurityGroups.
    • Now go to CloudWatch Rules and create a new rule. For the trigger, use the following:
      • Service Name: CloudTrail
      • Event Type: AWS API Call via CloudTrail
      • Specific Operations: AuthorizeSecurityGroupIngress
    • Then set two targets: your Lambda Function and your SNS topic for security alerts. This will revet the changes and simultaneously send you a notification.
    Screenshot of configuring the trigger in CloudWatch

    This should now revert any new ingress rules added to a security group. If you play with the code you can also use this for egress rules. This is just demo code and there are edge cases it will miss.

    The code also includes a number of filter options, including:

    • Restricting to a region or VPC.
    • Restricting based on a tag or a specific security group ID.
    • Allowing the change if it comes from a designated IAM user, and reversing if it was originated by anyone else.

    It takes the action automatically; if you want to manually review first you would modify the code to send a notification instead. We have some alternative examples posted at https://github.com/disruptops/demos-tools. These includes some samples that protect S3 and also show how to embed notifications in the Lambda itself for more-flexibility.

    Hopefully this gives you some ideas to refine your own guardrails to dramatically improve security while still moving blazingly fast.

    — Rich Mogull

  • IBM Survey: Rise of Multi-Cloud Computing to Force DevOps Issue

    IBM Survey: Rise of Multi-Cloud Computing to Force DevOps Issue

    A survey of 1,106 business and technology executives published this week by the IBM Institute for Business Value finds 85 percent of companies are already operating in multi-cloud environments, with 98 percent forecasting they will be using multiple hybrid clouds within three years. But the survey finds that only 39 percent of the respondents have implemented some level of DevOps processes to help streamline the management of multiple cloud computing platforms.

    Blaine Dolph, vice president of Global Business Services for IBM, said it’s clear that DevOps will play a critical role in enabling IT organizations to centralize the management of multiple clouds. But the IBM survey suggests that the management of multiple clouds within organizations is not nearly as efficient as it should be.

    Only 41 percent of survey respondents said they currently have a multi-cloud management strategy in place, and just 38 percent have the procedures and tools in place to operate a multi-cloud environment. Only 30 percent of enterprises have a multi-cloud orchestrator or other multi-cloud management platform to choreograph workloads, while fewer than 40 percent of organizations have cloud configuration management tools that provide information about resource configuration and relationships between resources, the survey found.

    All told, the IBM survey is only able to identify 20 percent of respondents as “multi-cloud maestros” capable of employing multiple clouds to smoothly and routinely deliver one or more business functions. The survey found almost two-thirds of multi- cloud maestros report they employ cloud management platforms.

    The survey also found that multi-cloud maestros are advanced in terms of adoption of microservices and containers. A full 61 percent of multi-cloud maestros predicted that within three years at least 80 percent of new applications will be developed using containers. Currently, 50 percent of multi-cloud maestros are using Docker containers and 63 percent are using containers as a service (CaaS). By 2021, 65 percent expect to use container orchestration. More than half (56 percent) of the multi-cloud maestros are already microservices and 59 percent plan to use serverless cloud services in their cloud environments, according to the survey results.

    Dolph said IT organizations in general would be well-advised to focus on more than just “lifting and shifting” applications into the cloud. They may need to leverage Docker containers to move an application into the cloud. But once there, an effort should be made to modernize that application by turning large swaths of it into a set of microservices that ultimately are easier to maintain and consume by developers building complementary applications, he said.

    The IBM survey makes it apparent that multiple clouds are the new IT normal. Almost half of respondents said they are establishing a multi-cloud architecture to develop new and enhanced products and services. A total of 62 percent of respondents require multiple clouds to create innovative business models, while 52 percent said they need them to produce new revenue streams. Sixty-six percent need them to enhance margin and 46 percent said they need a multi-cloud environment to support agile business processes.

    From a DevOps perspective, more than half (51 percent) of respondents use multiple clouds to cultivate a flexible, modular infrastructure that quickly absorbs and leverages technological advances.

    Obviously, one of the great paradoxes of multi-cloud computing is that in the pursuit of becoming more agile, many organizations are making their IT environments more complex by employing multiple platforms. The challenge they will face next is maturing their DevOps processes to the point where some order is finally brought to that multi-cloud computing chaos.

    — Mike Vizard