devopsy.kinsta.cloud

Category: Blogs

  • DevOps > Cloud, SaaS, SDDC, and apple pie

    DevOps > Cloud, SaaS, SDDC, and apple pie

    Over the weekend I was chatting with a really successful venture investor who recently published a top trends list.

    That made me think about DevOps vs. Cloud and vs. SaaS and vs. Mobile and other disruptors.  I’m convinced more than ever than DevOps is the right focus for professional investors and for all of us investing our time in developing, adopting and operating IT. Examining and understanding DevOps is more useful than understanding cloud, SaaS, SDDC (software defined data centers) and more – and here’s a few reasons why.

    1.  DevOps is a common denominator

    Keep peeling back the layers at the top operators and, underneath, you find the organizational structure and the set of tools that is DevOps. Some of these operators use containers, but some use bare metal, and some use virtualization – so there is quite a bit of variance there. Yes, there are other common patterns like developing applications that are stateless themselves and able to aggregate underlying resources – however DevOps in organization and in technology is a common denominator in a way that, for example, virtualization alone or technologies X, Y or Z are not.

    2.  DevOps is more open (and hence generalizable) and more powerful than SDDC

    This might be a little ironic coming from me as I did a bit to help popularize the concept of software defined storage and of software defined data centers while helping lead Nexenta. And we tried hard there to be specific in what we meant.

    However, the discourse about SDDC was and is really controlled by EMC and by VMware. And, as such, SDDC as commonly understood is well aligned with VMware’s vision. In that vision, VMware software becomes the orchestration or control system for the data center. Their APIs, such as VASA for storage, pass along information about what applications need to have provisioned and so forth and the rest of the data center infrastructure dances to that tune.

    The SDDC is a powerful vision. However – because it is VMware centric it is by definition one and only one way that an application centric view of the data center could be built. For example the VMware vision of SDDC assumes that virtualization, as opposed to containers or aggregation via Mesos for example, are the fundamental building blocks in compute. And building a SDDC in that way does not mean that users will necessarily achieve the productivity boost of 10-100x that are being achieved by adopters of DevOps.

    As I’ve mentioned in a previous blog post, the current VMware product line up does not fit nicely into a DevOps workflow. So to the extent that SDDC really is a VMware led narrative, it is not quite compatible with DevOps.

    I could make a similar case about DevOps being at the core of – and more important in some ways – than SaaS. SaaS that only scales up and that uses expensive legacy technology is not long for this world thanks to competition from DevOps centric SaaS shops. Because of this, again, as an investor, pulling on the DevOps thread is more valuable than pattern matching for SaaS.

    3.  DevOps is more about people

    Most importantly, DevOps is people plus technology – done right, to move fast, to build and operate code. By including how people work in its very core, DevOps as an approach is more likely to actually deliver its promise. No technology is of any value if people don’t actually use it. And DevOps as a movement, as an idea, as an approach, again, to building and operating code has the importance of people built right into even its name.

    Who knows, maybe in 2015 the discourse will shift away from DevOps. But I don’t think so. I think DevOps is a more important and more valuable concept than SaaS, or SDDC, or “cloud” for that matter. And I think that value will continue to be demonstrated for years to come.

  • How Ansible makes automation easy

    How Ansible makes automation easy

    Continuing our series of videos from the recent Gluecon 2014 conference, here is our next one again courtesy of CloudSoft. In this video Peter Sankauskas talks about using Ansible to make automation easy especially with AWS.  Hope you enjoy it.

  • Environment Variables – The Ultimate Gotcha

    Environment Variables – The Ultimate Gotcha

    We all think that we understand variables very well.  After all, if you are a coder, a variable is something you create and use every day.  But when it comes to DevOps the variable is the thing we seek to eliminate, but often neglect.

    (more…)

  • Enterprise DevOps: organizational change a must

    Enterprise DevOps: organizational change a must

    There has been a lot of discussion in the blogosphere, including here at devopsy.kinsta.cloud, lately about the challenges of adopting DevOps in the Enterprise space. These challenges at the end boil down to the need for organization change. There are organizational structures that appear to hinder DevOps adoption in large enterprises, that are inherent to enterprises and smaller companies do not have. In enterprises different functional teams – Dev, Ops, Security, QA, etc. may be under totally different management chains. It is not uncommon to have to go up thru several layers of management to reach a common senior executive. In other cases, there may be an entire software supply chain in place that is made up of not just internal but also outsourced or external suppliers. These suppliers may or may not ‘play ball’ when it comes to the changes in processes required to adopt DevOps. Especially if they are bound by outsourcing contracts that do not make change possible or profitable for them.

    DevOps as a philosophy has had as its centerpiece the principle that all the application delivery teams (Dev, QA, Ops, Business Analysts, Architects, Security…) need to align better and collaborate towards the common goal of delivering applications, in a efficient and constantly improving manner. This is a people and organizational principle, not just a process centric principle. To me adopting this is more important when adopting DevOps than any process or tool.

    This post discusses key guiding principles on which this DevOps driven alignment of different teams in an enterprise should be based. They are designed to improve collaboration between these teams and to break down the proverbial ‘wall’; to end the water-SCRUM-fall, when it comes to the relationships between the different application delivery stakeholder teams in the enterprise. This is not an easy effort. It depends on the existing culture at the organization, the presence of outsourced suppliers, politics, contracts and above all, the desire to change.

    These guiding principles are:

    1. Shift Left:

    Organizationally the goals of DevOps are to bring all the teams involved in application delivery together closer. Not just at deployment time, as they may do already, but all thru the delivery cycle. It requires Ops to allow Developers to take more responsibility of the operational characteristics of the applications they are building, for example. In turn, it requires Developers to keep Operational considerations in mind while building their applications. It requires analysts and architects to think holistically and work with Dev and Ops teams as they design and enhance the application. This is referred to in the DevOps world as ‘Shift Left’. As in shifting towards the left, Ops responsibilities. Shifting it to earlier in the software delivery lifecycle, towards Dev. It is shifting left the testing of your software to as early in the lifecycle as possible. It is shifting left the point at which collaboration between teams begins. It is shifting left the responsibility to deliver code to production – way left. Can someone in finance be held responsible for code not being delivered on time because they delayed approving a budget line item?

    2. Collaborate across all teams:

    Different functional teams typically use different tools to manage their projects and activities, for change management and (outside of email) different collaboration tools. In order to better collaborate across the organizational boundaries, all teams should start using the same Change Management and Work Item Management tools or in the worst case, use tools that are integrated. This allows seamless visibility across tools and traceability between respective Change Requests. Real-time collaboration using a common tool is obviously the best scenario.

    3. Build ‘Application Aware’ Environments:

    As we move towards Software Defined Environments, we have the ability to build, version and manage complex environments, all as code. All of the benefits of this are moot if the environments are not a perfect fit for the applications and more importantly the changes to the applications being delivered. The goal is hence to build Environments that are ‘Application Aware’ or are fine-tuned for the applications they are designed to run. No more cookie cutter Virtual images for all kinds of applications. More importantly, one needs to ensure that the environments are architected in a manner to allow for the evolution of the applications, both as they are developed; as they are projected to change or as they may evolve in the future. This obviously, would require close collaboration between Dev and Ops. Development Architects and Product Managers need to work with the IT Architects and Operations Managers to architect environments and their projected evolution to align with that of the application. Not only that, but also allow enough resilience in the environments to allow for unexpected change. For example, massive change caused by a super successful App. Think Instagram and how the founders had to keep changing their server environment almost daily as millions joined their service.

    4. Environment Sprints:

    Agile Development Principles prescribe that at the end of every Sprint, developers have a build. An executable build that runs, even if does not do much in terms of functionality. The next evolution of this concept is to extend it to environments. This would mean that at the end of each Sprint, the Dev team would have an executable AND the Ops team would have an environment, a production-like environment ready that the executable can be deployed on. This would allow the build to be tested. That too in a production-like environment. This would also provide immediate feedback to the Ops teams on the behavior of the application in their environment and use that feedback to improve the environment. This requires the Ops and Dev teams to align better. They will both have to use a single Sprint plan for releases – of their Builds and Environments respectively and will provide feedback to both at the end of every Sprint, using which Dev can enhance their Apps to improve functionality and performance and Ops can enhance the environment for the same. Leveraging full stack deployment – deploying at each instance the full stack that includes the environment, middleware and the application being deployed – could very well be called the ‘holy grail’ of DevOps. (Brief commercial break – IBM just announced the new offering UrbanCode Deploy with Patterns that provides full stack deployment leveraging OpenStack HOT patterns).

    The Human factor

    These principles are designed to foster Application Delivery team alignment from a teaming perspective, from a people perspective. These however do not replace the need for good old team building. Whether it is thru face to face get togethers or in todays distributed worlds, thru regular virtual meetings and online collaboration in real time. The best teams are the ones where the people know each other, trust each other, look out for each other and when there are challenges, know who to pick up the phone (or launch Skype) and talk to.

  • Sold out Dockercon highlights Dockers coming of age

    Sold out Dockercon highlights Dockers coming of age

    Docker grows up with release of version 1.0

    There is little doubt that Docker is one of the hottest names in tech.  Anything written about them seems to be a magnet for readers. When Solomon Hykes, CTO and co-founder keynoted the recent Gluecon 2014 conference it was standing room only. It’s pretty much the same story at this week’s initial Dockercon conference in San Francisco.

    I spoke with Scott Johnston, SVP of product at Docker last week in anticipation of this week’s conference. Due to a prior commitment I couldn’t make it out for the conference. It seems I am the only one not there. Scott told me the only thing holding them back from letting more people in is the fire marshal. They have a waiting list of something like 300, while others are scalping passes to the show!

    The big news from the show is the official release of version 1.0 of Docker. Of course the previous release .11 was a preview release of 1.0. It is not that version 1.0 has any new important features that no one has seen before. It is more a case that Docker is now official. In an era where Google kept products in beta forever it seemed, Docker is out there.

    Speaking of Google, one of the presentations at Dockercon today is about how Google is now supporting and using Docker in Google Cloud. Red Hat is also a big proponent/partner of Docker and the fact that IBM is a platinum sponsor should give you a hint as to what Big Blue thinks of Docker.

    In fact in both my conversations with Johnston and with Scott Rich, IBM Distinguished Engineer, DevOps at IBM Innovate last week, they mentioned some interesting IBM/Docker work. I think we may have to wait a bit longer to get more facts on that.

    Besides the 1.0 release, Docker has a few other important announcements this week dealing with some of the new services and products that Docker is rolling out around the open source version. The best way to explain these to you is to frame it the way Scott Johnston did for me.

    The open source Docker project is now called Docker engine. That is the platform. On top of this Docker and others will roll out services and products. They can be rolled out via the Docker Hub. Think of Docker Hub as the beginning of a Docker app store or marketplace. I think that you will see both Docker and 3rd parties with offerings in the Docker Hub very soon.

    From the Docker Hub announcement major features include:

    • An integrated Console  for managing users, teams, containers, repositories, and workflows;
    • The Docker Hub Registry, offering more than 14,000 “Dockerized” applications, available to all users as building blocks for their own applications;
    • Collaboration tools, enabling  users to manage and share their applications through both public and private repositories, and to invite collaborators to participate in any stage of the application development lifecycle;
    • The Automated Build Service, which keeps applications up-to-date by automatically rebuilding and updating an application’s public or private repository whenever the source code is updated on GitHub or Atlassian Bitbucket. Over 25% of the more than 14,000 Dockerized applications in the Docker Hub Registry are now created using Automated Builds, providing both automation and end-user assurance of container origin;
    • The Webhooks service, which enables users to automate repetitive workflows for build pipelines or continuous deployment, Interoperable with any RESTful API,  webhooks enables organizations to take advantage of the web APIs published by any service or software package, like GitHub, AWS, or Jenkins; and
    • The Docker Hub API, which includes a user authentication service, so that third party applications and services can gain authenticated access to applications in a user’s public and private repositories. Third-party services that have already integrated with the Docker Hub API include AWS Elastic BeanStalk, Deis, Drone.io, Google Compute Engine, Orchard, Rackspace, Red Hat, Tutum, and many others.

    In addition to Docker Hub, Docker also announced enterprise support services in conjunction with the release of the 1.0 version. This will allow large enterprises to rest easy knowing they can get expert help and support with SLAs wrapped around using Docker.

    Not that lack of support is slowing down the hype around Docker. Another subject Scott and I spoke about is how Docker can possibly live up to the hype surrounding it?  All too often we have seen companies and technologies crushed under the over-hyped expectations which make it impossible to live up to. Scott and the whole Docker team are keenly aware of this. They are really trying to under promise and over deliver. They are intentionally not fanning the fire with much speculation.

    Every technology goes through the Gartner hype cycle I guess. Overblown expectations lead to the trough of disillusionment. But for leadership at Docker including Scott this is not their first rodeo. They are doing everything they can to balance the hype and promise with the reality of a project/product that is still taking baby steps.

    The Docker team is also hard at work on the next round of Docker Hub offerings, as well as improvements to the Docker engine itself.  Scott assures me the team is committed to open source and the Docker engine will remain open source. The project itself is getting more support and code contribution than ever, so it remains vibrant.

    If you are lucky enough to be a ticket holder to Dockercon, hold on to your stub. It may be worth something someday. You could say you were there at the very first one. In the meantime the Docker team is hard at work making sure that they fulfill the promise that this 1.0 release holds. Congrats and good luck to Ben, Scott, James Turnbull, Julien Barber and the rest of team over at Docker.

  • Releasing your team from Release Thinking

    Releasing your team from Release Thinking

    How much time do you spend optimizing your delivery pipeline?  My guess is none. Many teams have figured out great ways to automate their releases, and some even automate testing and monitoring.  But they don’t often take the next step and learn from those results to refine the delivery processes.

    (more…)

  • SecDevOps: the new black of IT

    SecDevOps: the new black of IT

    Much has been written about the role of Security in a DevOps culture. Almost universally the thought is that DevOps can help improve security.

    Why?

    How?

    Really?

    An old friend of mine from the security world Andrew Storms and I will discuss this and other issues around DevOps and security on June 11th at 1pm eastern time, 10am pacific time on a webinar entitled “SecDevOps: the new black of IT“.

    Andrew is the Sr. Director of DevOps at CloudPassage. Prior to that Andrew headed up the security research team at nCircle. He is oft cited in the security press and highly respected.  Andrew and I ill discuss how just when DevOps is catching on, adding to security to the mix can make it even more powerful. How can security help DevOps to succeed? How can DevOps help us with better security?

    We will discuss what this means in terms of different kinds of security tools and services, the cloud and more.

    If you are interested in security and DevOps register now and  tune in to the webinar next Tuesday

  • Using Docker & Ansible by John Minnihan @Gluecon 2014

    Using Docker & Ansible by John Minnihan @Gluecon 2014

    Continuing our series of videos from the DevOps track at Gluecon this past year, our next video is “Why Docker + Ansible Make Chef and Puppet 100% Unnecessary by John Minnihan, ModernRepo”  Again many thanks to CloudSoft for recording and making these videos available to us to post. Also the picture above is from the Ansible website originally.

    This presentation sounds pretty controversial if you just look at the title. Actually John is not advocating throwing away your Puppet or Chef scripts.  If you already have Chef or Puppet set up and you like the way it is working, there is no reason to change.  However, John believes that Docker with Ansible can negate the need to use either of these tools going forward. John is primarily talking about a greenfield situation here I believe.  Anyway, the video like the previous ones in this series are about 30 minutes. Enjoy!

  • Is Ops the quarterback of the DevOps team?

    Is Ops the quarterback of the DevOps team?

    Without stepping too deeply into the charged debate on whether or not developers are at the pinnacle of the IT food chain, I’ve recently observed that many of the DevOps leaders I’ve been speaking with are from operations. For me it begs the question – are the ops folks quarterbacking the DevOps process at most companies? In other words let me put it bluntly, are the Ops folks leading the way forward in DevOps?

    Many DevOps thought leaders have taken great pains to describe and articulate that development, ops, qa, security and many other groups need to work closely together to make DevOps a success inside of a company.

    Some have focused in on the leadership role that developers have taken (and the associated burden that goes along with that role). For larger companies developers are often not the leads when it comes to DevOps. Developers are no doubt a key part of the equation and integral to practicing DevOps within the organization, but the Ops people seem to be calling the plays and leading the teams. This has led me to the question of who really leads the DevOps process within a company?

    Why Ops?

    As with any key initiative within a company, there needs to be a focal point, a champion or leader to advance it. DevOps is no different. So, then, how do companies organize themselves to become and execute a DevOps methodology?  Recently, we’ve spoken with three senior IT leaders – all in ops – from companies that are beyond the initial startup stage, but aren’t yet behemoth enterprises either. All three ops leaders either had the title and role of VP of Operations or VP of DevOps (without trying to get into the “is DevOps a job title controversy). Their job is to deliver code to production as fast and safely as possible. This meant that they are directing, helping even conniving their counterparts in engineering to deliver smaller batches of code faster and on a regular basis. They then push this code out to production. In essence they were playing the role of the quarterback of the DevOps process.

    Just as the quarterback directs the players, calling the play in the huddle and directing the offense, these Ops folks are quarterbacking the process of DevOps.

    As such, these senior leaders were building the overall product delivery pipeline – from ideation all the way through deployment and feedback. While the ops team may not be involved in each part of the pipeline, these leaders were integral in building out the product delivery pipeline. For most organizations that product delivery pipeline is starting with sales and marketing – what needs are the business trying to solve in this particular iteration of the product or service. From there, the various teams are understanding the implications for their particular piece of the pie. Engineering is understanding how to build the features and functions. QA is understanding how to test the functionality as well as automate the QA process. Security is making sure that the functionality doesn’t create vulnerabilities or security holes. And, ops is thinking through how to push all of these features out to the customer. As in any DevOps organization, there are a lot of moving parts with many pieces needing cross functional coordination. And, therein lies the question – we all may practice DevOps and should all practice it; but, who leads the effort?

    Clearly other senior leaders could take the lead on this activity. There is frankly no reason why the VP of Product or VP of Engineering could take the lead, but it has been an interesting observation that I have been seeing more and more ops leaders taking the reigns of DevOps at mid-sized organizations. We would really appreciate hearing who you have seen taking the leadership role in DevOps – in your organization or elsewhere. Let us know by commenting below!

  • Dev + Ops + ???

    Dev + Ops + ???

    If there’s one thing the DevOps cultural and professional movement taught us, it’s to question the status quo for the betterment of our customers, businesses, and ultimately, our teams. The combination of improved tooling, questioning structural barriers between teams, and a “shared outcome” culture have been powerful forces to date.

    Granted, we’ve seen the most demonstrated benefit in smaller organizations, but let’s face it… a big part of the movement is culture, and a set culture in a large organization is not easy to evolve. As we’ve mentioned before, you can’t sell DevOps. It will take time for large organizations to benefit from the movement.

    While we wait for the cultural evolution of large organizations to DevOps, it’s worthwhile to watch the continued professional and cultural evolutions of smaller organizations. The ones that first embraced DevOps from their start, what’s next for them?

    There are three ways I am currently exploring this question:

    • Dev + Ops + ???: Addition of a third functional pillar
    • DevOps evolution in parallel to Sales & Marketing evolution: Some interesting parallels
    • Org-wide evolution: DevOps cultural tenets deployed throughout the organization

    I would love to hear your observations as well.

    Dev + Ops + ???: As the title suggests, the most common finding thus far is another legacy functional silo having the structural, cultural and tooling barriers to collaboration torn down. These organizations have a solidly executing group of engineering and operator mindsets working closely together, and have actively embraced a third pillar of the organization. The most common I’ve found is the security team and the embrace of the term SecOps, though I’d love to find others. Do you fit this profile? Leave a comment below on your observations.

    DevOps evolution in parallel to Sales & Marketing evolution: In a small handful of organizations I’ve talked to, I’ve seen an interesting parallel evolution in the go-to-market teams to what was evolved in the technical teams.

    If DevOps tore down the fence where code was tossed over, and eliminated the “blame game” when things went wrong of “It’s not my servers, it’s your code” and “It’s not my code, it’s your servers”, there’s a remarkably similar parallel evolution happening in go-to-market teams… of course not about code, but instead about leads. To put it the most bluntly ableit controversially, there has been a long-standing tension between marketing, historically charged with generating leads for new customers, and sales, historically charged with engaging the leads and nurturing them into prospects and ultimately happy customers.

    Cue the blame game: “It’s not my leads, it’s your close” and “It’s not my close, it’s your leads”. Enabled by better tooling and visibility (e.g., customer relationship management software, marketing automation, etc.), progressive organizations are spending less time blaming each other, and more time collaborating on a shared outcome. Personally, I find the parallels between the evolution of technical teams and the evolution of go-to-market teams fascinating, and if you fit this profile, I’d love to hear your observations.

    Org-wide evolution: Our preceding two profiles are fundamentally about change from within; specific cultural, structural and tooling barriers being torn down between functions of an organization. We see examples of technology teams tearing down walls, and examples of go-to-market teams tearing down walls, but what about the many other key functions of an organization?

    Where’s finance? How about talent and human resources? Customer services and support? Product management? I would love to learn of more examples where cultural evolution is occuring not just in functional pockets of a comany, but across the entire organization at scale. If you fit this profile, I’d love to hear your observations.