Tag: containers

  • Puppet, Like DevOps, Continues to Mature

    Puppet, Like DevOps, Continues to Mature

    Over the course of the last several years we have seen the DevOps movement continue to mature and evolve. It has moved beyond the early adopter, visionary stage into the mainstream of the market. Therefore, it should be no surprise that some of the leading solutions in the DevOps space should likewise mature and evolve as well.  Yesterday Puppet (formerly Puppet Labs) made several announcements as part of a broader message that they are ready, willing and able to continue to be a leader in this space for years to come.

    There are several parts of the Puppet announcement yesterday, lets have a look at each of them:

    Growth indicators – Somewhat mandatory, Puppet announced that they continue to enjoy rapid growth. puppet growth

     

     

     

     

     

    Not a lot to add here, the numbers speak for themselves.

    Puppet Labs is now just Puppet. In my conversation with Nathan Rawlins, SVP of marketing, he said that this was a case of recognizing what the market is already doing.  I don’t know how many people you have heard refer to the company as Puppet Labs. Most everyone I know says Puppet. So why push rope up hill? Henceforth the company will be known simply as Puppet.

    puppet

    But there is more than just the name here.  Rawlins told me that along with the new name, they have also gone through a complete rebranding. New logo, new website look, new collateral look. The new brand seeks to give the impression of simplicity, stability and maturity.

    sanjayNew leadership – The company has hired a software/tech industry veteran leader, Sanjay Mirchandani as President, COO. Mirchandani has held senior leadership roles at VMware, EMC and Microsoft. His responsibilities will include go-to-market operations, including sales, marketing, customer success, and global expansion. From what Rawlins told me, Sanjay will be leading much of the day to day operations of the company.

    This is not a case though of a professional leader coming in to replace the founder/CEO.  Puppet’s Luke Kanies remains in the CEO role. Mirchandani will free up Kanies to do what he is best at. Be a visionary and chart the course for Puppet and in turn help shape the DevOps and software delivery market as well.  It takes a confident founder to bring in a real professional to help run the business to allow you to do what you do best. Many are too insecure to share management with a newcomer who “just wouldn’t love their baby the way they do.” Kudos to Luke for recognizing this move for the long term good of the company.

    blueshift by puppetProject Blueshift– Many people are focusing on Blueshift’s capabilities of using Puppet with container technologies like Docker, Kubernetes, Meseosphere, CoreOS, etc.  While Blueshift does focus on this as of the release today, there is actually more to it than that. Blueshift is really a collaboration between the Puppet community and the company itself to ensure that Puppet will always be able to help companies adopt next-generation infrastructure technologies. While today those next-gen technologies are container related, the idea is to grow Blueshift into something capable of working with new technologies as they emerge. It is a longer range project beyond 2-3 years, instead looking at what could be on the horizon 5-10 years from now. It is an ambitious vision. It is always hard to know what is the next big thing. But the fact that Puppet is even attempting to keep its eye on this is commendable.

    Upgrades to Puppet Enterprise and Puppet Orchestrator – Mixed in all of the news is that Puppet has released upgrades to its Enterprise product and the Orchestrator product that was released last year. Without going into a feature by feature recap (you can read the link to the PR if you like), Puppet continues to polish their enterprise products as part of their maturation.

    Integrations with Splunk and HipChat – As DevOps matures, the market will want greater integration between their tools. Puppet has taken a nice step in that direction by releasing a Splunk App that will allow you to automate set up of Splunk servers and give greater visibility into enterprise DevOps environments.

    The HipChat integration allows your Puppet Orchestration management to show up right in the HipChat GUI and allows users to manage it from within HipChat. A nice bonus is the access rights which make sure that those without permission do not have access to the Puppet Orchestration data.

    All in all, this is a major step up by Puppet. To be fair, the folks at Chef have also been taking steps to further mature their product and organization. As DevOps continues to grow, I am sure we will see some of the other DevOps tools startups follow the leaders here in maturing and growing along with the DevOps market.

  • Streamlining Cloud Tools, Platforms Management

    Streamlining Cloud Tools, Platforms Management

    DevOps and microservices have transformed the way organizations develop, deploy and manage applications—as well as the underlying infrastructure required to support it, be it cloud on on-premise. On the one hand, DevOps and microservices enable much greater speed and agility. On the other hand, that speed and agility come with a heaping side of complexity.

    It’s a beautiful thing when all of the elements of DevOps development are working in harmony. The diverse array of tools and platforms utilized, though, often are missing key security features and capabilities IT teams need.

    Apcera has a comprehensive portfolio of open source projects that provides customers with the ability to effectively manage and secure applications across various platforms and cloud infrastructures. The Apcera portfolio includes, among other things, NATS, a cloud-native messaging system; Kurma, a flexible container runtime; and Libretto, a virtual machine provisioning library. The company recently announced that it is extending its platform to also support Kubernetes.

    “Apcera … allows customers to securely move to any cloud, and deploy, orchestrate and govern the widest range of workloads on any infrastructure—including on-premises,” said Derek Collison, Apcera founder and CEO, in a press release statement. “Customers and developers are rapidly aligning behind Kubernetes as their API of choice when deploying container-based workloads. With its fast-growing ecosystem and strong market momentum, this was the obvious direction for us to take.”

    Kubernetes is an open source container orchestration platform, originally created by Google and now managed by the Cloud Native Computing Foundation (CNCF). Kubernetes enables organizations to manage large collections of containers as a single system. Embracing Kubernetes expands Apcera’s sphere of influence to include hybrid cloud management. Apcera also will actively contribute to the Kubernetes open source project.

    “We believe delivering trust to the cloud is critical, and Apcera has deep and proven expertise in this area,” said Kelsey Hightower, developer advocate, Google Cloud Platform. “We welcome their support of Kubernetes and future participation in the project.”

    Containers in particular have been a major security concern. A blog post from Gartner noted, “Security properties of containers are a largely unexplored field and there is a lot of controversial discussion about whether containers do contain or not. At times it seems that the discussion is driven by (hidden) business agendas, partnerships and financial dependencies rather than by plain technology. So, leaving all of this aside, can you make your containers contain or not?”

    A lot of progress has been made in the past year toward more secure container technologies. Hyper-V Containers from Microsoft, Clear Containers from Intel, and recent improvements from CoreOS are all examples of the fact that the container world is aware of—and vigilantly working on addressing—the security problem.

    Organizations need to understand how secure their infrastructure is, and be able to answer questions such as, “How can I effectively move into production quickly and with scalability?” Streamlining management of diverse cloud tools and platforms enables IT organizations to balance innovation and speed with safety and stability. Rolling Kubernetes support into the Apcera portfolio may be the key to effectively managing and protecting applications and infrastructure in a hybrid cloud environment.

  • Trusting Continuous Delivery

    Trusting Continuous Delivery

    DevOps, continuous delivery and the pipeline for faster software development are fundamentally different from traditional software development. It’s such a different approach and a different culture that it can take some time before Dev and Ops teams embrace and trust it—a necessary step for DevOps to make progress in any organization. By mixing containers and their value proposition into DevOps, this new software creation and deployment paradigm can readily foster that trust.

    What Continuous Delivery Brings

    DevOps and continuous delivery offer application development using containers, providing the required level of trust to get over that hump and move things forward. According to David N. Blank-Edelman, a DevOps veteran and currently technical evangelist at Apcera, the following container features lead to trust because they answer Dev and Ops concerns:

    • Containers are resource-friendly replacements for virtual machines, built to encase the coder’s development environment
    • Containers are the complete package format replacement, possessing everything you need to run an app
    • Containers create an abstraction that enables development and its results to sit in a host in any of a number of infrastructures including on-premise and inside any of several public cloud providers
    • Containers are the fastest means to launch short-lived, purpose-built testing sandboxes as part of a continuous integration process
    • Containers are the end result of a build pipeline (artifact-to-workload)
    • Containers form the boundary for loosely coupled components as part of a larger app or service. Such is the case, for example, when each part of a microservice-based app lives in its own container

    According to Blank-Edelman, the aforementioned features imbue trust because they satisfy the following Dev and Ops queries:

    • Is the container allocated to use only the resources it should?
    • What resides in that container / workload?
    • Where did that container come from? Is it OK from a security, coding and business requirements perspective?
    • Where is the container permitted to live and run such as in what cloud, environment, country, and in proximity to other containers technically?
    • Can the container communicate with only the things it should such as other containers and any hypervisor or other management resource? “If I move the container to another location, do the answers to these same questions still apply?” asks Blank-Edelman.

    Though these questions express concerns in direct and simple language, experienced DevOps engineers who have felt the frustration of building development processes and attendant infrastructure understands these are difficult questions to satisfy. Without modern containerization efforts to sustain DevOps, development and production goals grow further out of reach.

    The Importance of Policies

    As containers undergird DevOps and continuous delivery, policies support containers. Developers input changes and innovations in code to the continuous delivery pipeline. Developers then respond to the needs of the growing code in a nurturing fashion, fixing errors and enabling new features. This pipeline works when automation, together with each progressive step in the development process, lays a foundation for the next step in a predictable and reliable manner.

    “Here’s where it gets tricky: First, not only do you have to be able to answer the trust-related questions for each step, but it is entirely possible (and entirely likely in an enterprise environment) that different people/teams will share control of and input to each step on that path,” says Blank-Edelman. With so much collaborative effort in play juggling all these pieces, policies are necessary to configure these steps, address networking and other elements, and control the component parts of the process ahead of time and everywhere along the way.

    Policies that make particularly good container watchdogs include those that set steps in the DevOps/continuous delivery pipeline apart, hemming them in on all sides. “Just like having your papers in order makes for an easier border crossing when you travel abroad, so, too, does having your policies reflect the trust required to move from one step to another,” says Blank-Edelman.

  • Flash Mob Inflection: Rugged DevOps Revolution

    Flash Mob Inflection: Rugged DevOps Revolution

    Truthfully, I was never a huge fan of the HBO series “The Sopranos.” It’s not that it wasn’t entertaining; I just didn’t agree with the “best ever” label that so many espoused during the show’s halcyon days.

    This had something to do with living in Hoboken, N.J., at the time and feeling the show was a heavily glamorized caricature of the real-life Mafioso I still perceived to be in my midst. More specifically, it’s hard for anything mob-related to stand up next to “Goodfellas” and “The Godfather”—among my favorite films, ever.

    Either way, I currently find myself identifying with the series’ defining sentiment (copped directly from The Godfather, but hammered home as The Sopranos main theme): “Just when I thought I was out … they pull me back in.”

    It wasn’t that I assumed that I’d never delve back into the domain of IT security after transitioning into the DevOps market last year; leveraging DevOps methodologies to improve security, policy compliance and related workflows stands as a core element of the overall movement.

    But, in recent months and weeks it has become increasingly obvious that the security facet of the larger DevOps revolution appears poised for explosion as the buzz around emerging best practices, business models and solutions providers grows from a low hum into a steady roar.

    After spending a few days talking DevOps security with Derek Weeks of Sonatype at CA World ’15 (we were booth neighbors and dinner mates) and giving more consideration to his company’s angle on securing the systems development life cycle (SDLC) supply chain, the internal gears really got spinning.

    Regardless of whether you buy into Sonatype’s security-heavy approach, the concept of aggressively and proactively addressing security across the DevOps spectrum makes infinite sense. The ultimate downfall of applications security has always been the challenge of securing all the underlying layers of code (and today, microservices), especially as applications are assembled using heavier doses of shared, open source componentry.

    As further evidence, nearly every major solutions vendor focused on the DevOps space—whether oriented toward the vibrant application performance management (APM), configuration management and continuous testing segments, among many others—seems to have launched some initiative aimed at integrating security into their products.

    The rise of security capabilities adjacent to—and supporting inclusion into—the DevOps world, such as container giant Docker’s Content Trust initiative, along with the advancement of related startups including Twistlock, Scalock and StackRoxs, offers more proof of this growing momentum. And that’s only the container security piece of the puzzle.

    Of course, there are also a number of emerging startups aimed at addressing the DevOps security opportunity, backed by well-known venture capitalists. A shortlist might include providers such as Immunio, Prevoty and UpGuard (formerly known as ScriptRock, which recently rebranded itself under the guise of a “DevOps security” provider), and we’re sure to see many more. And don’t forget the growing emphasis on DevOps security among established security industry stalwarts such as CyberArk, Snort and Tripwire, among others.

    For a comprehensive list of the various types of DevOps security best practices and tools that we should expect to see gain wider adoption, this October 2015 blog by noted security analyst Adrian Lane of Securosis serves as a basic framework. Each of the areas of focus that he cites clearly offers significant opportunity for both new and existing methodologies.

    Finally, one need look no further than this year’s RSA Security Conference, to be held in San Francisco at this end of this month, as tacit proof of the continued growth of what people now varyingly refer to as DevSecOps or Rugged DevOps (I prefer the latter as it just sounds cooler).

    In addition to multiple conference sessions, notably a “DevOps Throwdown” between security industry veterans Caleb Sima, Chris Wysopal and Gary McGraw, there’s also the 2nd Annual DevOps Connect: Rugged DevOps Edition. If last year’s event is any indication (standing room only, venue literally at max capacity), this daylong DevOps security track at RSA will be an even more high-profile element of the entire week.

    Further, if DevOps security—or, at the very least, the growing focus on adjacent segments such as container security—isn’t the unofficial theme of this year’s entire RSA show, I’d be extremely surprised.

    Meanwhile (self-serving), my employer CA Technologies absolutely has a lot going on that complements and contributes to this growing DevOps Security market momentum.

    For all of those reasons, here I find myself again, drawn closer back into the security space that I’ve called home for roughly the last decade. I guess that’s just how it goes, as DevOps and security are inarguably two of the most significant trends in IT these days and inextricably linked at their respective cores.

    There are definitely worse fates out there. But, just when you think you’re out …

  • Will Microservices Make the ‘Ops’ in ‘DevOps’ Obsolete?

    Will Microservices Make the ‘Ops’ in ‘DevOps’ Obsolete?

    DevOps is a portmanteau of “developers” and “operations.” The concept of DevOps is about shifting IT culture in a way that enables developers and operations to work together more seamlessly. As more and more elements of DevOps become automated and leverage microservices such as container platforms, though, the operations side of that equation plays a diminished role.

    Microservices and Automating Ops

    What you’re left with essentially is developers with the processes and tools to automate the infrastructure back end. Technologies such as containers and services such as AWS Lambda simplify greatly the process of programmatically generating, modifying and maintaining the underlying infrastructure.

    AWS Lambda seems to be a step toward removing the Ops from DevOps. It creates a system for use and runs code via APIs. The instance is spun up, the code is run and the instance is spun back down. AWS Lambda can decrease provisioning time and enable developers to be even more efficient than working with traditional virtual machines in partnership with operations.

    Container technologies such as Docker allow applications to be broken down into very granular elements—many of which can be leveraged repeatedly. The ability to reuse containerized functions across different apps and to drive that process through APIs enable substantially greater automation.

    With each step forward in the evolution of DevOps, organizations find new bottlenecks or roadblocks, and then look for or develop solutions to address them. In this API-based, automated future, the APIs themselves become a challenge. Organizations have to find ways to effectively manage and update a rapidly growing library of APIs.

    Ops Will Live On

    If you work in operations, or aspire to someday work in operations, don’t fret: Operations still will exist on some level.

    The role played by operations will simply evolve. “Operations is not going away, but the job is definitely changing,” says Shannon Williams, VP of Sales and Marketing at Rancher Labs. “Automation and microservices are technologies that are enabling operations teams to spend less time ‘firefighting’ to maintain configurations, etc., and more time on bigger-picture issues that will help improve performance and resiliency, [and] help bring down costs for the business.”

    That’s a step in the right direction. In this future scenario—next-gen data center—those in operations will be able to invest their time in more valuable endeavors. They will be able to automate routine, mundane infrastructure tasks and focus on incorporating new technologies that enable the organization to derive greater value from DevOps and IT in general.

    “What is going away is the time spent arguing about what went wrong, not operations itself,” Williams suggests. “Given the additional time that automation will free up for Ops, they can start to find ways to improve their specific business unit. They can now be focusing on optimizing for performance, costs, etc.”

    Evolution of DevOps

    I have pointed out before that every roadblock in DevOps is actually an opportunity. As DevOps principles are applied in new areas, and new tools are developed to automate more tasks, it just accelerates everything to the next bottleneck. Rinse and repeat.

    To some extent, the infrastructure side of IT and DevOps are two of those roadblocks. That’s why tools are emerging to allow many components of infrastructure to be automated, freeing the operations team to focus its efforts on whatever the next roadblock is.

  • Survey on Modern IT – We Need Your Opinion!

    Survey on Modern IT – We Need Your Opinion!

    Change is the only constant in the world of IT. Developers, ITOps and the DevOps community are undergoing significant changes in needs, processes and services. This survey will provide better understanding of how the latest trends are impacting innovation, security and your team.

    This survey is sponsored by Apcera, a highly secure, policy-driven platform that lets enterprises innovate at speed with full confidence and trust. You can learn more by reading an interview with Apcera CEO, Derek Collison, on Container Journal.

    The survey is under 20 questions and takes about 5 minutes to complete. Share your thoughts on what is leading innovation, what stands in your way and how new technologies are are influencing the IT landscape.

    By taking the survey, you’re helping the entire DevOps community better understand where we stand and where we’re headed. As always, the results will be released when the survey period is complete.

    Take the Survey

  • Creating Dev Environments With Docker Compose

    Creating Dev Environments With Docker Compose

    down-the-rabbit-holeThe Slack notification popped up on my phone at 2pm Friday afternoon. “Argh. This is never good” I thought to myself. The message said “do you know anything about Vagrant?”. It was from a client of mine and I knew their environment well enough to know that this question was going to go down a rabbit hole the likes of which would leave Lewis Carroll awestruck. To compound matters, I was in the middle of nowhere, halfway between Las Vegas and Phoenix. I found a place to pull off, overlooking a vast, expansive maze of canyons and riverbeds leading nowhere and everywhere at once. I thought to myself “how fitting” as I started to reply on my phone.
    When launching a startup, there is no rule other than “find out what the
    customer is willing to pay for, then build it”. My client, Roger, had nailed it. His next message confirmed this as I read “I’ve been beating my head against a wall for two days trying to get this Vagrant image running. I’ve got new developers starting next week and need to get them a dev environment setup”. Roger had just discovered startup rule #2: “nothing can prepare you for the panic of success”. craft beerHis current environment was very solid: running in AWS, everything was deployed via OpsWorks to both staging and production environments. It works great… if you’re the only one in staging. It wasn’t going to support multiple developers changing code simultaneously, so he was looking at Vagrant for a solution. Each developer needed mongodb, elasticsearch, redis, and two node.js servers: one for the api and one for the web ui. Vagrant images are great, but I’ve often found them to be most valuable when hand-curated, like craft beer. They are also difficult to share as it requires shuttling large vm images around between developers.
    After reviewing the error logs, I could tell he was facing issues with security permissions between the vagrant image and the host OS. I decided I wanted to try something different. I wanted to try building it out using Docker containers. I had a feeling I could accomplish the same results with Docker, plus the added benefits of the entire environment being stored as config files in a git repo. Small, concise, version controlled, portable… I was liking this idea more and more. I sold him on the idea as well as buying enough time to get back to civilization where I could work on the idea (though the overlooking view was pretty inspirational).
    Scenic Overlook
    Back at the office, I broke open my laptop. I was familiar enough with Docker, but had a suspicion that Compose would help me tie the disparate pieces together. I started with the node.js servers. It was easy enough to kickstart them using a Dockerfile. (NOTE: I could have done this in compose as well, but had a feeling that I would want some extra flexibility later so I created separate Dockerfiles for them.)
    FROM: node:latest
    EXPOSE 3000
    Next, I created the docker-compose.yml file to define the environment. I was able to define all of the containers (minus my previously created node.js servers) using images from Docker Hub.
    web:
      build: ./web
      ports:
        - “3000:3000"
      volumes:
        - ./code/web:/srv/www
      command: node /srv/www/server.js
    
    api:
      build: ./api
      ports:
        - "3000:3000"
      volumes:
        - ./code/api:/srv/www
      command: node /srv/www/server.js
    
    redis:
      image: redis
    
    mongodb:
      image: tutum/mongodb
      ports:
        - "27017:27017"
      environment:
        MONGODB_PASS: password
    
    elasticsearch:
    image: elasticsearch
    ports:
    - "9200:9200"
    - "9300:9300"
    
    Once this was done, I created links in the web container. This creates environment variables and hostfile entries in the web container, allowing the web container to access the elasticsearch container via the hostname “elasticsearch”, and the mongodb container via the hostname “mongodb”, etc…
      links:
        - api
        - redis
        - mongodb
        - elasticsearch
    
    Docker Compose links
    The environment was complete, and I brought it up with the “docker-compose up” command. The added benefit of this environment is that the docker-compose terminal window maintains a steady stream of logs from all of the containers in a central window. This allows the developer to view the logs realtime, without having to hunt for them.
    Docker container logs
    VirtualBox portsThe only real stumbling block was in the use of VirtualBox. I created the environment on OS X. Since OS X can’t run Docker natively, it all runs in a VirtualBox machine on my workstation. This meant that I needed to go into VirtualBox and setup port forwarding for the required ports (3000, 3030, 27017, 9200, 9300,6379) so that I can access the services locally.
    Including time spent learning the basics of Compose, the entire environment was up in 2 hours. By using Docker, Roger’s new developers will be able to clone the config from a repo and be up and running in minutes regardless of whether they are developing on OS X, Windows, or Linux. Future additions to the repo can be shared by all by continuing to update the repo.
    Interested in trying it out for yourself? All the code described can be found in my git repo at https://github.com/rekibnikufesin/docker-compose. Be sure to let me know your thoughts in the comments or via twitter @wfbutton!