Tag: kubernetes

  • Platform9 Brings Workflows to Serverless Computing

    Platform9 Brings Workflows to Serverless Computing

    Serverless computing frameworks are widely regarded as the next big thing in enterprise IT because they are based on an event-driven architecture that makes servers, storage and networking resources dynamically available as needed using a programming construct known as functions. But DevOps teams are challenged in finding a way to orchestrate all those functions in concert with one another.

    To enable the orchestration of those functions, Platform9 has added support for Fission Workflows to its instance of a serverless computing framework. Platform9 CEO Sirish Raghuram says Fission Wokflows will make it possible for IT organizations to craft complex applications made up of thousands of functions.

    On the one hand, serverless computing frameworks promise to make DevOps easier by eliminating the need to provision infrastructure every time a workload exceeds whatever amount of capacity that has been assigned to it. On the other hand, attempting to manage what could easily become thousands of functions in an IT environment is likely to make DevOps even more challenging than it is today. While not a new concept, Raghuram says Fission Workflows provides IT organizations with a familiar construct for managing functions.

    Benefits of Fission Workflows include an ability to reuse functions as well as being able to upgrade entire related sets of functions all at once. There are also fault-tolerant capabilities that make possible to recover and restart a workflow in the event of an infrastructure failure.

    Platform9 is specifically making the case for a serverless computing framework based on Kubernetes clusters that can be deployed on-premises or in the cloud. Platform9 also provides support for containers on an instance of Kubernetes, and Raguham says the company expects that IT organizations will mix and match containers and serverless computing frameworks as they see fit. For example, a container might invoke a function via an API call to run a stateless process while Docker containers are employed for all the stateful processes that require access to primary storage, he says.

    Raguham notes the trouble with serverless computing frameworks put forward by cloud service providers such as Amazon Web Services (AWS) is that they lock an IT organization into a specific platform instead of taking advantage of a portable cluster technology such as Kubernetes, which is backed by the Cloud Native Computing Foundation (CNCF). Kubernetes also provides the added benefit of a common platform for both containers and serverless computing, he says.

    It’s still early days for the use of serverless computing frameworks in the enterprise. Nevertheless, it’s clear they will need to be managed from a DevOps perspective alongside cloud-native applications running on containers and platform-as-a-service (PaaS) environments and legacy applications running on virtual machines and bare-metal servers. Given the fact that none of these architectures is likely to supersede any of the others anytime soon, IT organizations would be well-advised to start crafting a DevOps strategy now on the assumption that the overall IT environment is about to become even more complex.

    — Mike Vizard

  • Characterizing and Contrasting Container Orchestrators

    Characterizing and Contrasting Container Orchestrators

    Admiral Calcote, also known as Lee Calcote (@lcalcote) or the Ginger Geek to his friends, gave a presentation titled, “Characterizing and Contrasting Container Orchestrators,” at the 2016 All Day DevOps conference.

    Okay, he isn’t really an admiral—nor does anyone call him that—but he used the title “admiral” to describe what container orchestrators do, relating it to an admiral directing a fleet of container ships. You could also say that they are like the conductor of an orchestra, directing the individuals to work together as a group toward a common goal while each musician is still able to play their own instrument.

    lee1.png

    Lee is the head of Technology Strategy at SolarWinds, and for his talk, he discussed four open-source container orchestrators: Nomad, Swarm, Kubernetes and Mesos-Marathon.

    He emphasized the obvious: There is no one perfect solution. Each organization is different, so for each solution, he looked at:

    • Genesis and purpose
    • Support and momentum
    • Host and service discovery
    • Scheduling
    • Modularity and extensibility
    • Updates and maintenance
    • Health monitoring
    • Networking and load balancing
    • Secrets management
    • High availability and scale

    Lee noted that while there are many core capabilities, any orchestrator must have cluster management and scheduling.

    lee2.png

    He then dove deeper into the four solutions. Below are summaries (the full talk is chock full of information and is online here):

    Nomad

    • Designed for both long-lived and short-lived batch processing workloads
    • Cluster manager with declarative job specifications
    • Ensures constraints are satisfied and resource utilization is optimized by efficient task packing
    • Supports all major OSs and workloads
    • Written in Go and with a Unix philosophy
    • Host discovery: Gossip protocol – Serf is used; servers advertise full set of Nomad servers to clients; creating federated clusters is simple
    • Service discovery: Integrates with Consul
    • Scheduling: two distinct phases – feasibility checking and ranking; optimistically concurrent; three scheduler types when creating jobs
    • Uses task drivers to execute a task and provide resource isolation, but it does not support pluggable task drivers
    • Built for managing multiple clusters/cluster federation

    lee3.png

    Docker Swarm 1.12

    • Simple and easy to set up
    • Architecture is not as complex as Kubernetes and Mesos
    • Written in Go – lightweight, modular and extensible
    • Strong community support
    • Host discovery: used in the formation of clusters by the Manager to discover Nodes (hosts); pull model – worker checks in with the Manager
    • Service discovery: Embedded DNS and round-robin load balancing
    • Scheduler is pluggable and is a combination of strategies and filters/constraints
    • Ability to remove “batteries”
    • Rolling updates are supported
    • Managers may be deployed in a highly available configuration, but does not support multiple failure isolation regions or federation

    lee4.png

    Kubernetes

    • An opinionated framework for building distributed systems
    • Written in Go and is lightweight, modular and extensible
    • Led by Google, Red Hat and others
    • Young – about 2 years old
    • Robust documentation and community
    • Scheduling is handled by kube-scheduler
    • Pluggable architecture and an extensible platform
    • Choice of database for service discovery or network driver and container runtime
    • Supports rolling back deployments, automating deployments and rolling updating applications
    • Inherent load balancing
    • Uses Pods, an atomic unit of scheduling. Each pod has its own IP address, no NAT required and intra-pod communication via localhost

    lee5.png

    Mesos-Marathon

    • Mesos is a distributed systems kernel
    • Mesos has been around the longest (since 2009)
    • Mesos is written in C++
    • Marathon is a framework that runs on top of Mesos
    • Mesos is used by Twitter, AirBnB, eBay, Apple, Cisco, and Yodle
    • Marathon is used by Verizon and Samsung
    • Mesos-DNS generates an SRV record for each Mesos task
    • Marathon ensures that all dynamically assigned ports are unique

    lee6.png

    Wrapping up, Lee provided the following overview comparing the different container orchestration solutions.

    lee7.png

    Lee packed a tremendous amount of information into his talk. If you use containers, his talk is worth your time and is available online here. If you missed any of the other 30-minute long presentations from All Day DevOps, they are easy to find and available free of charge here. Finally, be sure to register you and the rest of your team for the 2017 All Day DevOps conference here. This year’s event will offer 96 practitioner-led sessions (no vendor pitches allowed).  It’s all free and online Oct. 24.

    — Derek E. Weeks

  • Sprinting in a DevOps Marathon: Tips for Success

    Sprinting in a DevOps Marathon: Tips for Success

    Remember the good old days of 2007, when Heroku was the shiny toy everyone wanted? Thanks to Moore’s law and the evolution of tech, 2007 seems like a lifetime ago. In fact, 2015 seems like a lifetime ago, now that machine learning has advanced for artificial intelligence to “wake up”—but I digress.

    DevOps and building applications in a microservice architecture appears to be here to stay. Monolithic applications have been around since the advent of the “stack,” but thanks to DevOps, we can build faster and smarter with better quality and lower cost. Consequently, it’s dumbfounding why only .5 percent of engineers operate DevOps (according to Gene Kim). DevOps would unlock trillions of dollars if the other 99.5 percent made this adoption, thereby ushering in a new era of economic prosperity.

    If you’re part of the other 99.5 percent, let’s go through the considerations for DevOps adoption—it’s the wave of the future, so you better start now!

    The Road to DevOps Success

    Assess the competency and maturity of development practices and processes of your organization across the following parameters:

    • Culture (i.e. collaboration and standardization)
    • Design and architecture
    • Data analysis and management
    • Build, continuous integration and continuous deployment
    • Assessment and testing
    • Application deployment across environments (Dev, Test, Production)
    • Continuous delivery and application upgrades/updates

    Assessing the aforementioned practices and processes result in a baseline to compare against industry standards (see Puppet’s “2017 State of DevOps Report“) and identify areas of improvement. This may seem intuitive or obvious, but true agility through DevOps requires frequent evaluations as new tools, practices and methodologies become available (i.e., you’re not still using dial up now are you?).

    Enhance and Improve the Culture with Shared Values and Objectives

    Design, Dev, Operations, QA, Product and other applicable teams historically have operated independently of one another in “silos,” which is the antithesis of DevOps. The goal need not be pushing code, configuring applications or testing; the goal should be to deliver value to the customer. Unfortunately, the culture of most organizations inhibit teams from maximizing value because they’re removed from the customer and seldom listen to the sales team (which I emphatically state as a seasoned sales professional!).

    That said, this is the easiest thing to fix by implementing the following steps:

    • Introduce a feedback loop including customers, sales, marketing and customer support to introduce new ideas to product so value is maximized.
    • Clearly articulate goals to motivate product teams and get them excited about their work.
    • Institute a culture of continuous improvement by listening to developers, operations, design, product, etc.

    Base Decisions on Facts, not Conjecture

    Seldom do organizations correlate sales and customer satisfaction metrics with product investment and product operations. All too often, organizations let sales or product dictate the best direction for the company when they need to be evaluated in conjunction due to their impact on profitability (i.e., building products to drive sales or letting sales dictate what products to build). DevOps expedites the delivery of products, thereby aligning the objectives of product with the goals of the sales team. This enhanced agility makes it’s easier to incorporate feedback from sales to enhance products while enhancing sales by delivering value faster, lowering the sales cycle and fostering happier customers!

    Choose the Right Tools, not the Popular Ones

    Months ago, the Bluemeric team started an engagement with a Fortune 500 team that was beginning its journey into containers. Long story short, this client evaluated the container platforms based on name recognition, not on the relative strengths and weaknesses or how the PaaS will achieve the client’s goals with the least amount of effort. The client reviewed dozens of platforms and entertained 10 for a proof of concept, but only one (and a relatively unknown PaaS) was able to install on its infrastructure (AWS, Azure and Google), spin up a cluster, build the application and deploy the applications. The competitors had tens if not hundreds of millions of dollars in revenue and thousands of customers, but the one that got the job done wasn’t the first, second or even third choice. It was the lightweight PaaS that Bluemeric recommended, which promised simplicity, minimal effort and ease of use that doesn’t drown customers with unnecessary features—and which actually worked. Obviously, this isn’t limited to PaaS; it’s an example on why decisions to adopt tools, methodologies or vendors must have a thoughtful evaluation, free from outside opinions and bias.

    That being said, implementing and adopting DevOps is neither easy nor fast, but it will be a requirement for all businesses in the very near future. As noted earlier, Gene Kim discovered that .5 percent of all software engineers currently operate in a DevOps environment, which is astonishing when comparing profitability against those who aren’t practicing DevOps. Once the trend becomes mainstream in the world of finance, the other 99.5 percent of software engineers better catch up or face the consequences.

    — Gabriel Deupree

  • DevOps Chat: Purpose Built Appliances for Containerized Applications with Diamanti’s Mark Balch

    DevOps Chat: Purpose Built Appliances for Containerized Applications with Diamanti’s Mark Balch

    Back in my StillSecure days, building purpose-built appliances for security was all the rage. Over time, though, security-as-a-service captured much of the market (although security appliances are still a big seller). For containerized applications, you may think cloud is the way to go. But the folks at Diamanti say that purpose-built appliances for containerized applications are a superior option.

    I spoke with Mark Balch, their VP of product and marketing about this, and he gives us Diamanti’s views here and, frankly, makes a good case. It is early in the containerized application market and the winners are yet to be crowned. Mark tells it better than I can. Have a listen and let us know what you think.

    As usual, the streaming audio of our conversation is below and the transcript of our conversation is below that.

    Streaming Audio

    Written Transcript

    Alan Shimel: Hey, everyone. Alan Shimel, devopsy.kinsta.cloud here for another episode of DevOps Chat. Very happy to be joined today on DevOps Chat with Mark Balch, VP of products and marketing at Diamanti. Mark, welcome to DevOps Chat.

    Balch: Hey, Alan. Thanks for having me. It’s great to talk to you.

    Shimel: Thank you, Mark. So, Mark, let us first start off with Diamanti. I think that’s a fairly new company to our audience or at least to us here at devopsy.kinsta.cloud, although the company itself is not new, but, for those of our listeners who may not be familiar, why don’t you give us a little background?

    Balch: Yeah, definitely. We’ve been around since 2013, and Diamanti spent the first couple of years in stealth mode, as you may say, building our product. And we launched the company last year. We just launched our GA product this past week, in February.

    And just a very quick background or why are we here. Well, we saw the container revolution occurring. We believe that containers are the next mainstream application platform for new cloud-native applications, moving forward. And we’ve seen that users are spending a tremendous amount of time and money trying to custom-engineer their own infrastructure and figure out how to make cloud-native applications operate at scale, with reliability and performance. And we said that, “You know, we wanna make sure that users can do this out of the box, very simply, get control of their costs and be able to focus their time, instead of reinventing the wheel and infrastructure, on building the unique business applications that they’re being measured on.” So that’s why we’re here.

    Shimel: Excellent. And, you know, Mark, what was interesting is, at a time when we’re seeing software-defined everything, right, Diamanti’s out here with a purpose-built appliance for containerized applications. And I would imagine there are many people who say, “Oh, no, we don’t wanna sell hardware.” But, really, if you want control over your environment, you don’t have much of a choice.

    Balch: It’s a real tricky situation for a lot of users, right, because just go out there and spend a little bit of time searching on the web and going to meetups and looking at user testimonials about the challenge of being able to get reliable performance at a reasonable cost point in the traditional model. It’s really very difficult. People spend a lot of time custom-engineering, and that’s time that they can’t spend building their unique services. So, for us, really, it’s a very simple value proposition.

    We’re not saying that you have to customize your application or do anything proprietary; what we’re saying is that, “Continue to go with your open-source approach—standard Docker images, standard Kubernetes orchestration, standard Linux—but operate on a platform that is going to give you the automation of getting that entire application deployed properly, that’s gonna give you predictable performance and outcomes and that’s gonna let you run at a reasonable cost point.” I mean, people have massive bills, whether it’s public cloud, whether it’s looking at virtualized traditional infrastructure that was built for a different type of application. People spend a tremendous amount of time and money trying to get cloud-native applications working, and our goal is to make sure that people can get those applications working within minutes, not within weeks or months.

    Shimel: Sure. And that is the advantage of having that sort of purpose-built platform. Mark, I wanted to, first of all—you know what? Shame on me. I didn’t even ask. Can you give our audience a little bit of the background of who you are, though? And I’ve said your title, but who’s Mark Balch and how did you come to Diamanti?

    Balch: Yeah. That’s a good question. So I’ve been in the IT and application industry for 20 years. I started off my career as an engineer. I love doing development work. I love to build things. I love to deliver great products that have an impact. And then, somewhere along the way, I got into product management because I wanted to figure out whose job was it to make sure that we’re building the right products at the right time, that it could solve real needs. And so, for the last more than 10 years, I’ve been focused on product management and product marketing, doing everything from data center automation and application provisioning software to cloud automation software, and, now here at Diamanti, focused on the open-source side of application development and, again, focusing more on the outcome. Whether we’re talking about an appliance, whether we’re talking about software you can deploy, at the end of the day what I care about, as a product manager, is making sure that there’s a value at the end that’s gonna make someone’s life better, that’s going to give them a greater experience in being able to be personally successful in getting their applications into people’s hands. So that’s what motivates me.

    Shimel: Great. Appreciate that. Mark, if I may, I wanted to take a moment and talk a little bit about Kubernetes. There are obviously choices for container orchestration and container management out there, and it seems like Diamanti has clearly put their chips in the Kubernetes market. As I said, there are other choices. Can you give us a little bit of the background on why Kubernetes and—or maybe I’m wrong. Can Diamanti customers choose other container orchestration options as well?

    Balch: Yeah, you know, I think, when it comes to talking about container formats and orchestration, I think we all have to be a little humble, in that the market is still very early and very dynamic. There’ve been so many changes. Kubernetes is doing a great job. Mesos is doing a great job. Docker’s doing a great job. You’ve got now standardization happening in the Cloud Native Computing Foundation for the Open Container Initiative. So I think, honestly, it is too early to call a winner, and I think all of us have to make sure that we’re giving users choice in working with an open ecosystem. It’s not about forcing people down a particular path; it’s about giving people options. Right? So we can work with a variety of different options, but let me tell you why we have been doing a lot of work with Kubernetes. And we’ve been doing this work with Kubernetes for the last couple of years.

    Kubernetes, from a governance perspective, has been a great ecosystem to work with. Obviously, Google contributed that code to The Linux Foundation last year, and you’ve got large and small contributors from all over the ecosystem, including Diamanti, contributing to make Kubernetes bigger and better and more extensible. So what attracted us to work with Kubernetes early on was that governance, where, really, anyone could come to the table. Everyone has been welcome to pitch in and to take leadership roles, frankly. For example, Diamanti contributed something called “FlexVolume” into Kubernetes. It was upstream; it was a year ago. And we were welcomed with open arms, and now there are, literally, more than two dozen members in the storage SIG who we work with. Same thing around scheduling for Kubernetes—we upstream contributions there. Working with CNI, container network interfaces, is another example.

    So we’ve just seen a lot of great governance and a lot of extensibility, where the Kubernetes community seems very focused on this concept of extensibility, where it’s not that my solution is better than yours; it’s that we have APIs that we can use to extend different options and then users can pick and choose what’s gonna work for them, but it’s supported with a common framework. So that’s what we love about Kubernetes.

    Shimel: Got it. Interestingly, Mark, in our several studies, surveys on this with our audience, overwhelmingly—well, not overwhelmingly ’cause, actually, Mesio was pretty close, but Kubernetes clearly was the choice among our audience for container management and orchestration, so probably not a bad move there. Mark, what about people who have invested in public cloud infrastructure for containers? What’s the Diamanti value proposition to them?

    Balch: That’s a really good question, Alan, and people ask us that a lot. And, by the way, I’ll say that Diamanti is gonna be at the Google Cloud Next Conference next week, so we—

    Shimel: Really?

    Balch: Yeah, so we actually do a lot in terms of public cloud, okay? So I think the first thing is that it’s back to that concept of choice and no vendor lock-in. We need to make sure that we’re all offering users capability—we’re not trying to force them down a path with something that is proprietary—so everything we do at Diamanti is standard Docker runtime, and, as that evolves and maybe we have more standardization around runtime, that’s gonna evolve into making sure that everyone participates in the standard format. So, if you develop an application and you’re using Google Cloud or using Amazon or Azure or someone else, one of your goals, probably, should be to do it in a way where you can be portable, where you’re not locked in, where you can bring it to another provider if that provider offers you better economics, better performance, better reliability, data governance, things like that.

    And that’s where we come in. You could think of Diamanti as a provider, but we are an on-premises provider. We allow users, like MEMSQL or NBCUniversal, to be able to build an on-premises private cloud that is fully open source and you can take your existing containers, you can move them onto Diamanti, you can get all the great performance and automation benefits that we talk about. If, for some reason, you wanna go elsewhere, you wanna go to Google, to Amazon, somewhere else, we don’t stop you. You can do that. You can go back and forth, right? So we wanna make sure it’s very easy to go back and forth so that the user is in control of deciding what’s more important to them, in terms of all those things like governance, data sovereignty, performance, and economics.

    Shimel: Sure. Do you envision a time when people sort of have hybrid Diamanti-public-cloud containerized applications?

    Balch: Yeah, that’s a good question and actually brings up a really—another good wrinkle around Kubernetes. Kubernetes has an emerging project called “Ubernetes,” or its formal name, “cross-cluster federated services.” Now one thing I will say—I love the Kubernetes community, but sometimes, from a marketing perspective, we could do a little better. “Cross-cluster federated services” doesn’t really roll off the tongue very quickly, but what it means is, essentially, it can mean hybrid cloud or multiple availability zones.

    And so, just like Diamanti has made upstream contributions to the community around scheduling, we can now extend those scheduling capabilities, using Ubernetes, so we can move that across the availability zones and into hybrid cloud scenario. Right? So I definitely envision a scenario like you’re talking about, where you can have multiple Diamanti clusters in different data centers, you can have availability zones up in Google or Amazon or Azure, and you can, basically, allow the cross-cluster federated services to figure out where to deploy that application based on your performance requirements, on your sovereignty requirements, on your compliance requirements, or you can say, “Hey, I wanna make sure this thing goes into public cloud,” or, “I wanna make sure it goes into my Los Angeles data center with Diamanti.” You’re in control of that, from a policy perspective, and that’s, I think, what’s really interesting about the cross-cluster federated services road map, moving forward.

    Shimel: Absolutely. And you’re right; that does just roll right off the tongue. But, look, they’re engineers; they’re not marketing people. Mark, let me ask another question, and, not to hold your feet to the fire, but, for people who may be interested at home, can you give us sort of a ballpark pricing on Diamanti?

    Balch: Yeah, absolutely. So we’ve announced our pricing. Each Diamanti appliance has a list price of about $40,000, slightly under that, in fact. Obviously, discounts apply. So we are highly competitive with all alternatives, whether you look at commodity, whether you look at converged infrastructure, hyper-converged infrastructure. We are highly competitive from a price perspective. And then, when you look at the consolidation and the time savings, frankly, there is no competition there. So, certainly, love to chat with any of your listeners who are interested, and I pretty much guarantee, from an economic perspective, it’s gonna work out great.

    Shimel: So that brings up—you opened that box, Mark. For any of our listeners who may be interested in finding out more about Diamanti or speaking to someone, is there an e-mail address or a web address where they can check it out or contact someone?

    Balch: Yeah, definitely. So please check out our website, it’s www.diamanti.com, or just send us an e-mail, info@diamanti.com. We can get you set up really quickly.

    Shimel: Okay. Mark, we’re running low on time, but I did want to just mention that Diamanti is sort of riding the wave, if you will, of this tremendous momentum building up behind containers and micro-services and Docker, Kubernetes, etc. Where in the adoption cycle do you think all of this is? Has it really gone mainstream yet or are we still at the early-adopter stage?

    Balch: That’s the multibillion-dollar question, so we are seeing adoption all over in different kinda fits and starts, to be honest with you. So the industries we see the most adoption, Alan—we see media moving quickly. We actually do see financial services moving quickly. We see web infrastructure. We even see service methods moving ahead, wanting to offer containerized services and competing with the large public-cloud vendors. So where we see containers happening is developers are downloading Docker on their laptop; they’re able to accelerate their application lifecycle and development processes. Operations is a little slower. To be honest with you, it is lagging a little bit behind. That’s where we come in—we’re working with ops teams to help them get off the ground—and, just like always, developers ultimately have the influence where they’re defining application architecture, and then they work with their ops teams to figure out how to get those architectures deployed successfully into production.

    So there are lots of different numbers out there—451 Group had some good numbers; I’ve seen numbers from Forrester and from Gartner. The industry’s still early, maybe at only about 10 percent of adoption, and that’s a very anecdotal number, but it’s growing really quickly and we’re seeing a lot more inbound requests now because everyone’s trying to learn about it. The cat’s out of the bag; it’s not a secret anymore. And I think either people are doing it today or they see it coming quickly and they wanna figure out, when it comes down the pipe from their development team, how do they be successful with containers and actually make it work for their business.

    Shimel: Got it. Got it. Sounds great. Mark, we’re actually way over time. I apologize to our listeners and to you, but I thought we had some good information here and I didn’t wanna cut it off, but we need to wrap up. Mark Balch, thanks so much for being our guest today. Continued success with Diamanti. It sounds great. And, obviously, through Container Journal and devopsy.kinsta.cloud, this is a space we watch pretty closely, and I’m sure we’ll be hearing more from Diamanti soon.

    Balch: Thank you so much, Alan. This has been a great conversation. I appreciate it.

    Shimel: You know, one last thing. Mark, I forgot to mention. You mentioned you were going to be at the Google Cloud conference, which is the week of March 9th, something like that, right?

    Balch: Yep.

    Shimel: In March 8th and 9th?

    Balch: Yeah.

    Shimel: And then –

    Balch: March 8th through 9th.

    Shimel: – DockerCon is next month in August – in Austin.

    Balch: April.

    Shimel: In April.

    Balch: Yes. We will _____.

    Shimel: You’ll be there as well.

    Balch: Yep. Definitely gonna be there, yep, and we’ll also see you at OpenStack Summit, so we’ll be at Google Cloud Next in March, DockerCon in April, and Open Stack Summit in May.

    Shimel: And I think that one was in Boston?

    Balch: It is in Boston this year. That’s right.

    Shimel: Yes. Okay. Hey, we actually have some DevOps Connect events going on in Boston, Austin, and Chicago, but for another day. Mark Balch, Diamanti, thanks for being this episode’s guest on DevOps Chat and we’ll speak to you soon.

    Balch: Thanks a lot, Alan.

    Shimel: Thank you. This is Alan Shimel for devopsy.kinsta.cloud and we hope to see you soon on another DevOps Chats.

    — Alan Shimel

  • DevOps Chat: Madhura Maskasky, Platform9

    DevOps Chat: Madhura Maskasky, Platform9

    This DevOps Chat is with Madhura Maskasky, co-founder and VP of product at Platform9. If you are not familiar, Platform9 offers an “open source as a service.” While that sounds pretty cool, think managed hybrid cloud and now managed Kubernetes in an infrastructure-agnostic service.

    Madhura does a great job of explaining this, so have a listen for yourself. As usual, below the streaming audio player is a written transcript of our session.

    Alan Shimel: Hey, everyone, this is Alan Shimel of devopsy.kinsta.cloud here for another DevOps Chat. Our guest on today’s DevOps Chat is Madhura Maskasky. Madhura, welcome and thanks for joining us.

    Maskasky: Thanks for having me. I’m excited to be here.

    Alan Shimel: Madhura, I didn’t mention in introducing you; you are with Platform9, an open source-as-a-service company. Correct?

    Maskasky: Yeah. I’m VP of Product and co-founder at Platform 9 Systems. What Platform9 does is we pioneered a unique model, a SaaS-based model for deployment of complex open-source cloud-native frameworks such as OpenStack and Kubernetes.

    Alan Shimel: Beyond sort of that tagline, Madhura, I believe—was it your co-founder or CEO that I might have interviewed at—I want to say it might have been the DevOps Enterprise Summit.

    Maskasky: Yeah. It must be Sirish Raghuram.  

    Alan Shimel: Yeah. It was kind of my first introduction of Platform9 and I was kind of taken with the business model. I thought it was interesting and unlike anything I had seen to that point. As I mentioned, Madhura, beyond the tagline, can you give our audience really a sense of what it is that Platform9 is doing?

    Maskasky: Absolutely. As we started, our roots come from VMware. That’s where myself and my co-founder spent about seven-plus years. A common theme that we saw working with mid to large-sized enterprise customers at that time was the amount of struggle they were going through trying to realize this vision of a private cloud, which was a concept that had started becoming popular for a about a year at that point.

    The reason for it starting to become popular was because public cloud had set such an exceptionally high bar for how easy it can be to consume infrastructure. But yet, enterprises, customers of VMware or otherwise, were really failing to live up to that high bar and create an equal offering that utilized their internal resources that might be based on their data centers or in a hybrid way, but create a similar and seamless interface for their internal consumers to consume those resources.

    So Platform9 started with the vision that it should be easy, fundamentally as easy as consuming a public cloud, for anyone to consume resources from any infrastructure endpoints, whether they’re private of pubic or hybrid. We wanted to do this using possibly an open source software that’s available, because we also felt strongly that open source is going to be the new norm of how cloud infrastructure is consumed.

    So specifically what Platform9 does, our very first product offering was managed OpenStack, our OpenStack-as-a-service. What that model means is that we deploy the OpenStack control plane in a SaaS-hosted manner. The control plane is deployed outside of our customer’s data center, and then it integrates with their data center or their public cloud endpoint via a mechanism of an agent.

    An agent, if you have a private infrastructure, the agents become the bridge between the private infrastructure and our cloud-hosted control plane. And if you have a public cloud, then we have drivers that interact directly with that pubic cloud endpoint. So, that’s really how the model works, and it lets you get up and running with OpenStack in a matter of minutes.

    Now we’ve repeated the same model with Kubernetes, which is the most powerful container orchestration framework for building your applications using the microservices paradigm, where Kubernetes is also deployed as a SaaS-hosted service on the infrastructure endpoints of the device, so private or public endpoints.

    Alan Shimel: Got it. Of course when I look at it, Madhura, I see, okay, so you had the OpenStack offering, OpenStack-as-a-service, and now you dig into the Kubernetes and containers. I have to assume that—and maybe I shouldn’t assume. Let me ask you: How quickly is that OpenStack market growing?

    Maskasky: We’ve seen a pretty significant growth in our OpenStack customer base just year over year. In 2016 we were at about 50 customers strong. There have been multiple statistics that forecast a pretty strong overall growth in OpenStack revenue for 2017-plus as well. Just yesterday I think I was reading an article that was mentioning a multibmillion-dollar projected revenue growth in the overall OpenStack market, and the fact that Fortune 500s are going to be consuming increasingly OpenStack.

    So even with containers, virtualized infrastructures are not going away any time soon, and the big challenge that enterprise customers face that we see is, on one hand, they are looking to save some money by converting part of their currently virtualized infrastructure into open-source Linux KVM and use OpenStack versus, say, going entirely the VMware route. Or, on the other hand, some customers start by being 100 percent public cloud consumers, but then realize that that’s a very expensive model at scale, and hence want to go to more of a hybrid deployment, where part of their infrastructure is hosted privately or in a service provider data center.

    For both of those cases Platform9 is an extremely good fit because we support both VMware as well as KVM and because of our as-a-service model.

    Alan Shimel: Got it. What about Kubernetes? Maybe it’s because I live in a DevOps bubble, but that seems to be kind of taking off, huh?

    Maskasky: Kubernetes is taking off in a pretty significant way. We felt that Kubernetes was going to start building this level of momentum about a year or so ago, as we were looking at the different container orchestration frameworks and evaluating them for their pros and cons, and the other two optimal ones are Mesos and Docker Swarm.

    Mesos and Docker was in its very early stages with its offering at that time, but the primary reason for Kubernetes being so popular is I would say three important factors associated with it. One is from project maturity. They were kind of the most mature projects—it was built inside Google and vetted inside Google before it became the open source project that it is today. It had a pretty significant community outside of simply Google developers, who are actively contributing to it.

    A new version comes out every three to four months and the pace of iteration is just astounding. In 3 it has a number of enterprise customers publicly vouching for it by showcasing the use cases and examples of how they are leveraging it.

    Alan Shimel: Got it. Madhura, now that we’ve got that straight, let’s talk a little bit about recent news from Platform9. Do you want to share with our audience?

    Maskasky: Absolutely. We shared two pieces of news recently, actually just a week ago. The first one was that the managed Kubernetes offering, which has been in beta for about six months, is now generally available. They’ve had the opportunity to have the offering beta tested with a couple dozen customers since we announced it around June of 2016. So now we are excited to share with the world that it is generally available.

    What it is again, pure Kubernetes deployed as a SaaS-managed service. We think it’s the industry’s first infrastructure agnostic Kubernetes-as-a-service offering, because it lets you consume both your private and public infrastructure, and it comes built-in with a number of enterprise readiness features like multi-tenancies, single sign-on, quota management, as well as being able to deploy highly available Kubernetes clusters.

    Alan Shimel: That’s great. Definitely, again, I just think there’s something—I think the market desperately needs this kind of offering. I remember when OpenStack first came out. Actually, if you remember Cloud.com, Martin Nickens I think was involved in that one and several other companies. It was really at that point in the evolution of migration to the cloud; it was important that people were able to get all the pieces in one box, if you will—cloud-in-a-box, for lack of a better word.

    But in today’s world even that paradigm doesn’t work. People want as-a-service. So if they can get a Kubernetes kind of infrastructure-as-a-service like this and agnostic, it’s fantastic. Also, something called Fission was introduced as well, yes?

    Maskasky: That’s right. That was the second part of our announcement, which is we announced a new open-source framework called Fission. What it is, is it’s designed to be the de facto open-source alternative to AWS Lambda.

    Alan Shimel: I’m sorry. I didn’t mean to cut you off. Go ahead.

    Maskasky: No worries, Alan. So just some context behind it. When it comes to serverless, AWS has really made Lambda extremely popular and introduced this new paradigm of function as a service. But a problem with it that we see is the fact that it locks you into the AWS ecosystem, because Lambda is firmly designed to integrate with all the other Amazon services that are offered in their ecosystem, but it doesn’t necessarily provide built-in integrations for the most popular open-source tools and frameworks that your developers might want to use.

    It kind of goes back to an important piece that we’ve always held, which is the development community loves to embrace the right sort of tools that let them do their job well. They like choice and they do not like being told or being scripted to go to a set of one tools only. So those are some of the motivations behind Fission.

    Now Fission is also built off of Kubernetes, and by having that focus on Kubernetes it has an added advantage that it simplifies consumption of Kubernetes as well.

    Alan Shimel: Got it. In my mind, the company has positioned these Kubernetes offerings to really take a leadership position as we see Kubernetes and containers really kind of explode onto the scene. What’s next for Platform9?

    Maskasky: Kubernetes and containerization is a big part of our road map and our strategy for 2017-plus. We’re just getting started with managed Kubernetes going generally available. We have a ton of work ahead of us as part of our road map, and we’re looking for customer feedback as well, truly making the offering cloud-agnostic, enabling as many current points as our customers would like us to offer with built-in integrations, providing the ability to create federated clusters across these current point. So, in a nutshell, really realizing that true hybrid container-as-a-service vision through the managed Kubernetes offering.

    That’s one. The second big part is, again, Fission. Fission has just gotten started as an open-source project. We’re looking to make it the de facto way of running function as a service within Kubernetes. So we’re looking to work with the CNCF, the Cloud Native Computing Foundation, and we are looking for the Kubernetes community to really start investing in it.

    We’re seeing some grassroots early traction adoption. Someone in the community recently built a dot-net client for Fission and out-of-box support for Node.js and have been building support for Go. We’ve been hearing folks who started using Fission on top of their internal community’s deployment and they’re telling us about their use cases. So these are all some points of feedback, so we want to grow and build on that.

    Alan Shimel: Got it. Madhura, as I promised you before we came on, the time goes so quickly. We’re already past 15 minutes, but it’s okay. I think we got a great—at least a taste of what Platform9 is all about, and perhaps we could find out more on another DevOps Chat or maybe an article here on devopsy.kinsta.cloud or Container Journal, as the Kubernetes stuff certainly lends itself to that.

    Madhura, where can people find out more information?

    Maskasky: The best way to find out more about everything happening on Platform9 is by going to our website, Platform9.com. You can also follow us on Twitter. The handle is @Platform9Sys. Fission has its own website, Fission.io, and you can also follow Fission on Twitter as well. The handle is @Fissionio. So that’s the best way to stay on top of all our news.

    Alan Shimel: Fantastic. Madhura—and I’m still mispronouncing your name probably, I apologize—Maskasky, thank you for being this episode’s guest on DevOps Chat. Continued success of Platform9 and please keep us posted. I love the open-source angle. I love the Kubernetes sort of as a service, the serverless stuff with Fission, very, very exciting stuff you guys are doing over there. So please keep us posted and best of luck.

    Maskasky: Thanks, Alan. Thank you for having me. This was a lot of fun chatting with you and absolutely I will keep you posted.

    Alan Shimel: Absolutely. This is Alan Shimel for devopsy.kinsta.cloud on another DevOps Chat, and we hope to see you on another one, another chat hopefully soon. Have a great day, everyone.

    — Alan Shimel

  • New Kubernetes Fundamentals Course Now Available From The Linux Foundation

    SAN FRANCISCO, January 26, 2017 – The Linux Foundation, the nonprofit advancing professional open source management for mass collaboration, today announced the availability of a new training course, LFS258 – Kubernetes Fundamentals.

    LFS258, developed in conjunction with the Cloud Native Computing Foundation, is geared towards developers and administrators who want to get started with Kubernetes. It teaches students how to use Kubernetes to manage their application infrastructure. Topics covered include Kubernetes architecture, deployment, how to access the cluster, tips and tricks, ConfigMaps and more. This self-paced, online course provides the fundamentals needed to understand Kubernetes and get quickly up-to-speed, to start building distributed applications that will scale, be fault-tolerant and simple to manage.

    Kubernetes Fundamentals will distill key principles, such as pods, deployments, replicasets and services, and give students enough information to start using Kubernetes on their own. The course is designed to work with a wide range of Linux distributions, so students will be able to apply the concepts learned regardless of their distribution of choice.

    Kubernetes is quickly becoming the de-facto standard to operate containerized applications at scale in the data-center. It builds on 15 years of Google’s experience managing containerized applications. With a growing open-source community, Kubernetes is poised to change the way applications are built and managed, as well as change the role of system administrators. The 2016 Open Source Jobs Report from The Linux Foundation and Dice open source professionals reported that containers are the second more important skill, behind only cloud technologies. There is a lack of formal training options available for those wishing to get started with Kubernetes though, which LFS258 aims to address.

    “Kubernetes is rapidly maturing in development tests and trials and within production settings, where its use has nearly tripled in the last eight months,” said Dan Kohn, executive director, the Cloud Native Computing Foundation. “We are thrilled to partner with The Linux Foundation to provide a program that will train, certify and promote Kubernetes Managed Service Providers (KMSP). This program will ensure enterprises receive the support they’re looking for to get up to speed and roll out new applications more quickly and more efficiently.”

    LFS258 also will help prepare those planning to take the Kubernetes certification exam, which will be launching later this year. Updates are planned for the course ahead of the certification exam launch, which will be specifically designed to assist with preparation for the exam.

    The course, which has been available for pre-registration since November, is available to begin immediately. The $199 course fee provides unlimited access to the course for one year to all content and labs. Interested individuals may enroll here.

    About The Linux Foundation

    The Linux Foundation is the organization of choice for the world’s top developers and companies to build ecosystems that accelerate open technology development and commercial adoption. Together with the worldwide open source community, it is solving the hardest technology problems by creating the largest shared technology investment in history. Founded in 2000, The Linux Foundation today provides tools, training and events to scale any open source project, which together deliver an economic impact not achievable by any one company. More information can be found at www.linuxfoundation.org.

     

    The Linux Foundation has registered trademarks and uses trademarks. For a list of trademarks of The Linux Foundation, please see our trademark usage page: https://www.linuxfoundation.org/trademark-usage.

    Linux is a registered trademark of Linus Torvalds.

    — Jules Louis

  • Confessions from Fannie Mae, Liberty Mutual and Capital One

    Confessions from Fannie Mae, Liberty Mutual and Capital One

    Learn from real-world stories from presenters at the All Day DevOps conference …

    True Story

    Over the past few years, Fannie Mae transformed the way in which it delivered software. Deploys increased from 1,200 per month to 15,000 per month.  At the same time, productivity increased by 28 percent while costs were reduced by 30 percent.

    But, how did Fannie Mae do it?

    During the All Day DevOps conference, more than 13,500 practitioners from around the world came together to learn from their peers in the industry. Barry Snyder, senior manager of DevOps at Fannie Mae, was one of 57 practitioners who shared his real-world journey through his enterprise transformation.

    Choke Point or Checkpoint?

    Eddie Webb, director of Software Delivery Platforms at Liberty Mutual, also shared his organization’s journey into DevOps.  Eddie described how Liberty Mutual evolved its organization, tool chain and processes to go from 20 percent of builds being deployed to 60 percent over the past 18 months. He described how the company documented its value delivery chain and identified all of its choke points in the continuous delivery process that would need to be automated to achieve its new velocity and quality targets.

    Screen Shot 2016-12-05 at 11.35.04 AM.png

    Run the Gauntlet

    The All Day DevOps conference recorded more than 57 practitioner-led sessions (30 minutes each) including those from Barry and Eddie. Every session is now posted online and is free to watch. The conference itself had three tracks: CI/CD, Automated Security and Modern Infrastructure, including:

    • Application Secret Management with KMS – Andrey Utis of Capital One
    • Getting out of the Job Jungle with Jenkins – Damien Coraboeuf of Clear2Pay
    • Serverful Computing – Taking One for the Team – Paul Czarkowski of Blue Box Group

    One of the best ways to learn about DevOps is to hear from the experiences of others who have run the gauntlet. Take 30 minutes this morning, at lunch or at home this evening to watch your first online session with All Day DevOps On-Demand.

    — Derek E. Weeks

  • An IT Practitioner’s Guide To DevOps

    An IT Practitioner’s Guide To DevOps

    Two technology innovations of the past decade have fundamentally changed how IT works and given rise to the “DevOps” phenomenon. Before we dive into what DevOps is and how IT organizations should adopt this culture, let’s review what gave rise to it in the first place.

    Pull from Agile Development

    The last decade has seen many organizations abandon the traditional “waterfall” software development process in favor of “agile” development. Agile is all about small, specific user-centric stories, described by a product owner and delivered in short sprints of two to four weeks by small, focused development teams.

    Agile development led to the emergence of more frequent code drops from software engineers, which needed to be deployed more quickly. No longer was it okay to take days or weeks to deploy a new software version; developers working in two-week sprint cycles want deployment to work within minutes with a high degree of predictability.

    Enabled by the Cloud

    Before the cloud came along, access to infrastructure required human interaction and manual operations. Processes typically could not be automated effectively. This meant that operations were slow and error-prone and created a huge point of friction in the software development life cycle.

    Around the same time that developers were adopting agile development, the cloud became real, and infrastructure could be orchestrated effectively using APIs to provision everything from machines and networks to storage, routing policies and firewalls.

    DevOps: Closed Loop Automation Between Development and IT

    Agile development and cloud computing created the perfect storm in which DevOps was born: this new way of doing IT is all about ensuring closed loops, with 100 percent automation.

    picture2

    Figure 1: DevOps is about ensuring a closed-loop, fully automated workflow between software development and deployment.

    In the simplest terms, this meant that each of the steps outlined in Figure 1 need to be fully automated, and integrated with preceding and following steps. Doing this ensures software can be developed faster, deployed quickly, and iterated upon more rapidly.

    Benefits

    Done right, DevOps leads to a virtuous cycle of greater productivity across development and IT, greater agility in bringing new experiences to market and greater impact on the business.

    Adopting DevOps: You Can Phase It In

    Organizations looking to adopt DevOps should do so in phases and define metrics for each phase. Here’s a typical phased approach that you can use as a baseline and customize to your organization’s unique needs.

    Phase 1: Adopt Agile Development and Cloud Computing

    Agile development and cloud computing are profound technology shifts and create the underlying culture in which DevOps can thrive. So, the first step is to ensure organizational buy-in for both of these trends:

    1. Agile development can be adopted in stages. The most important is to agree that software will be built and shipped in short, two-to-four-week sprint cycles, and with a focus on user stories.
    2. Cloud computing can be adopted no matter what your IT infrastructure looks like today. Use hybrid cloud platforms such as OpenStack to provide programmatic access to a VMware or Linux environment, or use a public cloud platform such as Amazon Web Services.

    Phase 2: Commit to a CI/CD Workflow

    Continuous integration (CI) systems such as Jenkins or TeamCity provide a way to pipeline and integrate the various steps in a DevOps workflow. The choice of system doesn’t matter as much as ensuring that all steps of the workflow will be integrated (eventually) into this workflow.

    Phase 3: Repeatable, Automated Software Deployment

    Many organizations tend to have some manual steps in the packaging and deployment of their software, especially for getting it into production. This often reflects underlying assumptions about manual oversight and intervention, and it breaks the closed-loop philosophy of DevOps.

    To succeed with DevOps, you need to fully automate the software packaging and deployment process. In addition, ensure that production builds are deployed in the exact same manner as developer builds, using identical environments.

    Cloud technologies such as orchestration (OpenStack HEAT or AWS CloudFormations) and configuration management tools such as Ansible, Puppet or Chef can be used to implement this phase.

    Phase 4: Automate Software Testing (Continuous Delivery)

    Once software deployment has been automated, the next phase is to automate the validation process for new software releases. If you’re like most organizations, testing involves several rounds of back and forth with QA teams and product managers, and it often takes weeks to resolve the last few bugs in each release. To reach the next level of DevOps automation, this process has to be completely lights-out and automated.

    This requires a fully automated software test suite, across the back-end and front-end, so that product managers have confidence in the quality of the software being released. Once implemented, you finally have a closed loop between the critical elements of the DevOps life cycle. While release to production may still require someone to approve it or “push a button,” this is a very respectable state in the DevOps journey.

    Phase 5: Continuous Deployment

    To get to the most advanced DevOps phase, you need to automate the “go live” process for software releases. This in turn will allow you to take even smaller, more frequent code drops and push them into production.

    The most advanced software teams in the world now operate this way, and the organizational implications can be transformative: Developers come into work feeling that they can implement a feature and have it go live immediately, and product managers can make commitments to the business with the confidence they can deliver, because they have this incredibly efficient software delivery machine backing them. Most importantly for you, IT is no longer seen as a cost center; it’s seen as the fundamental enabler for business productivity and time-to-value.

    Closing Thoughts

    Getting to Phase 5 represents the state-of-the-art in the industry today. However, another significant breakthrough promises to raise the bar even higher.

    In each of the prior phases, the DevOps workflow is integrating software development with IT operations; but there is nothing influencing software architecture to take operations into account.

    In other words, we’re still bolting IT operations automation onto software development.

    This is about to change. We’re on the cusp of a new era in software: instead of bolting operations onto the software development process as an afterthought, a new framework promises to bring development and operations together by design. With declarative, infrastructure-agnostic constructs to describe how applications are composed, interact and managed, Kubernetes promises to increase the operability of modern software systems by an order of magnitude.

    Everyone interested in the DevOps journey should be learning about Kubernetes and evaluating how it will improve your software development and operations workflows.

    About the Author / Sirish Raghuram

    sirishr_platform9Sirish Raghuram is co-founder & CEO of Platform9. As CEO, Sirish quarterbacks the team at Platform9 in realizing the company’s vision, which, on a given day, can range from strategy and road map to shipping code and running tests. Before co-founding Platform9, Sirish was an early engineer at VMware, where he held technical and management leadership roles and helped ship multiple vSphere products. Sirish’s work was reflected in several patents, VMworld talks and internal product innovations within VMware Engineering. Sirish has a B.E. in Computer Science from the University of Pune, India. Connect with him on LinkedIn.

  • NodeSource Announces Containerized N|Solid Deployment for Kubernetes

    Developers can now deploy enterprise-grade Node.js on Kubernetes

    San Francisco, CA – July 26, 2016 – NodeSource, the Enterprise Node Company™, today announced at Node Summit 2016 the beta launch of its containerized N|Solid™ deployment for Kubernetes. With this release, NodeSource is offering the first version of Node.js with production-ready continuous performance and security monitoring within a Kubernetes cluster.

    A recent Node.js Foundation survey found that 45 percent of developers use Node.js with container technologies such as Docker. As companies look to build out their next-generation architectures, Node.js and containerization are emerging as two key components for powering rapid technical innovation.

    Kubernetes makes deploying, scaling and managing containerized services fast and simple. With an N|Solid deployment for Kubernetes, developers get real-time performance monitoring and vulnerability scanning of Node.js applications running in a Kubernetes cluster.

    “Node.js is ideal for containers because it allows developers to quickly build, iterate and scale cloud-native APIs, apps and microservices,” says Aparna Sinha, Product Manager, Kubernetes & Google Container Engine. “It is great to see NodeSource offering a Kubernetes deployment for Node-specific monitoring and security, allowing companies to scale with confidence.”

    N|Solid combined with Kubernetes orchestration provides developer and operations teams with:

    • Efficient scaling of microservice/distributed architectures

    • Node-specific performance insights and analyses

    • Continuous Node.js package security vulnerability scanning

    • Expedited deployment of new Node applications

    • Automated rollouts and rollbacks

    • Service discovery and load balancing

    • Horizontal application scaling

    “We are seeing a popular trend emerging among our enterprise customers with the adoption of containerization for Node.js and Kubernetes for cloud orchestration,” says Joe McCann, CEO and co-founder of NodeSource. “Adding the powerful monitoring and security features of N|Solid, we are creating the trifecta for high-performance Node.js services.”

    N|Solid is available for download as a native package (Linux and Mac), Docker images and now as a pre-configured Kubernetes deployment. For more information, visit <landing page>.

    About NodeSource:

    NodeSource is a technology company dedicated to delivering enterprise-grade solutions in support of a sustainable ecosystem for the open source Node.js project. We aim to drive and expand the Node.js ecosystem by providing best of breed solutions that specifically target the needs of the enterprise. Customers include NASA, HomeAway, Condé Nast, Craftsy,  and other progressive Node.js adopters who are leading the charge in microservices via server-side JavaScript instead of maintaining legacy monolithic applications like Java. We are a Distributed by Design organization – our teammates report to work from all corners of the world: Asia, Australia, Europe, South America, as well as NY, SF and Austin. NodeSource is backed by RRE Ventures, Crosslink Capital and Resolute.vc and our AngelList Open Source Syndicate. For more information visit www.nodesource.com and follow @nodesource.

  • It’s Time to Slay the Universal Installer Unicorn

    It’s Time to Slay the Universal Installer Unicorn

    While many people want a universal “easy button installer,” they also want it to work on their unique snowflake of infrastructures, tools, networks and operating systems. I call them snowflakes because their small differences expand combinatorially. That divergence turns the idea of a prescriptive reference architecture into a destructive myth that limits community and ecosystem growth.

    Because there is so much needful variation and change, it is better to give up on open source projects trying to own an installer and instead focus on making their required components more resilient and portable.

    Universal Installer Fallacy

    So what does an easy button install look like? Let me tell you about one that I built.

    Crowbar Mascot

    It was 2011, the dawn of the OpenStack Era, and our team at Dell was already battle-weary from trying to make repeatable installs around Azure, Eucalyptus, Joyent and Hadoop. The simple reality was there was no success pattern we could follow deploying any one of those platforms—and certainly none cross-platform. Even though all used basically the same compute and network infrastructure, each had custom deployment code and configuration approaches. We jumped on becoming an OpenStack founding partner as a way to try to break the failure pattern.

    What was failing? Each platform had deeply embedded environmental assumptions that were not repeatable or portable.

    For example, one of those platforms required thirty (30!) full racks of gear and tethering back to the master deployer because its management and topology was hardwired into its infrastructure. Others were not much better: Installation often was “ssh apt-get install platform,” then “scp pre-built configuration files.” Even worse, those steps were always proceeded with 30 pages of prescriptive manual system prep steps that were updated daily.

    Incorrectly, we believed that the solution was to create a better installer for OpenStack.

    The Crowbar project, which is still used by SUSE, created a fully integrated install sequence from PXE boot and discovery to testing the final OpenStack installation. We could build a cloud on a complete rack of gear from boxes in minutes. The open source project even used open tools such as Chef and hardware-agnostic options to increase its community appeal. As the first installer (it predated even DevStack), the project was widely used in early OpenStack evaluations.

    Despite that success, as a v1 technology, Crowbar’s major shortcoming was that it tightly coupled components and services together. That made the installation and management process very rigid and non-conforming to disparate operational models and environments.

    Our install scars showed us the real battle lines that prevented portability. Even minor infrastructure variations resulted in broken or site-specific configurations. A major part of the challenge is that variation is not neatly contained; instead, it is spread throughout the environment. We needed to create automation that could both cope with heterogeneity across the community and be change-tolerant after installation to enable upgrades.

    We came to accept that there is no single installer. Instead, scalable installers must assume that they will operate with other parts of the system in a coordinated way, because the diversity of options makes it impossible for them to control everything. In fact, we’ve seen that smaller, composable install steps are much easier to automate consistently. Even more importantly, smaller steps make it possible to manage systemwide upgrades in a controllable way.

    Asking for an “easy button” installer is the wrong question. Instead, let’s ask, “Can we create sharable components that can be automated in multiple ways?”

    There’s no universal installation script to create cloud rainbows; however, there are strategies we can use to make individual projects installable over a wide range of options. These approaches include composability, consistent configuration and use of standard tools and patterns for operation. These items help projects focus on their own concerns and allow operators to apply best practice techniques.

    Basically, the first step is for project developers to limit their scope and delegate operational concerns to operators.

    Even in a platform-as-a-service (PaaS) project that attempts to hide operational complexity from developers, such as Kubernetes, there remains someone who must operate that platform using foundational infrastructure and tooling. We make that job more complex when we get distracted trying to apply the new concepts to the old infrastructure. The addressing of an impedance mismatch pulls effort away from the central abstraction.

    Since we do need consistent installs from the community, projects do have a responsibility to help those efforts.

    How can they help? By ensuring that their component services are well-bounded with clear input configurations, operational requirements, upgrade patterns and output expectations.

    But really, that’s a future post. Please let me know what you think and where we should go from here.