devopsy.kinsta.cloud

Tag: containers

8 Signs of Docker Ecosystem Empire-Building

Docker’s meteoric rise on the strength of the continuous delivery movement’s thirst for containerization has been nothing short of remarkable. There’s perhaps no bigger sign of Docker’s uptake than in the build-up of its third-party ecosystem. The past year has seen partnerships galore from Docker and the biggest IT industry players, plus notable support of the Docker ecosystem amongst key service providers. Here’s our take on some of the cross-industry relationships that will make the Docker ecosystem stronger.

1. Google

Google has had a big hand in helping to foster a collaborative Docker community, notably with the open source release of its Kubernetes container cluster manager. The project has gained tons of cross-industry support from IBM, Microsoft, Red Hat and more. Google is further strengthening its ties in collaboration with Mirantis Partners with the reveal last month that Goog would work on a project with the OpenStack firm to simplify the installation and management of Kubernetes applications on OpenStack.

2. VMware

Late last summer, VMware announced that it was working alongside Google, Pivotal and Docker to make it easier for enterprises to scale container-based applications on VMware infrastructure. As part of the deal, VMware agreed to collaborate on open source projects like libswarm, libcontainer and libchan, and to work with Docker to improve interoperability between Docker Hub, vCloud Air, vCenter Server and vCloud Automation Center.

3. RedHat

Just this month, Red Hat brought to fruition its plans to support Docker and other containerization technologies with the launch of its Red Hat Enterprise Linux Atomic Host operating system. Based on work done in the Project Atomic open source project, the OS is meant to offer a reliable platform for Docker, Kubernetes, Flannel and others for the enterprise market.

4. Amazon

For the past year Amazon has been bending over backwards to help customers more effectively leverage Docker on AWS infrastructure. Most notable was its big splash last November with the launch of its EC2 Container service (ECS), designed to simplify the management of Docker containers across a managed cluster of EC2 instances. It continues to add to that service—last month it provided updated enhancements to ECS that makes it able to use images stored in private docker repositories and to mount volumes in containers in ECS.

5. Rackspace

Rackspace’s long-running relationship with Docker starts first as a user, but extended notably beyond that in June last year when it was working to contribute to the Libswarm project and had created a backend plug-in for Libswarm that would enable its users to launch Rackspace Cloud Servers directly from the Docker CLI.

6. IBM

After a year full of big wins for the Docker ecosystem in 2014, it ended with a bang in November when IBM joined forces with the containerization upstart to integrate it into a range of applications for the enterprise market. As a part of the partnership, IBM launched IBM Containers, a Docker-based service delivered as a part of its Bluemix open cloud platform for application development.

7. Microsoft

Microsoft has been incrementally upping the ante with its Docker relationship, starting in June 2014 when it added support for Docker Microsoft added support for Docker in Microsoft Azure on Linux VMs. By October it had publicly committed to support Docker on future Windows Server releases, by adding resources to implement in-house tweaks to Windows Server and to support the open-source development of the Docker Engine for Windows Server. And in January Microsoft showed its commitment to the partnership with the release an Ubuntu image fully integrated with Docker and ready to deploy from the Microsoft Azure marketplace.

8. Puppet Labs

The most recent among Docker’s ecosystem wins, Puppet Labs this week pushed forward a new version of Puppet Enterprise that includes a special module specifically for managing the Docker Platform and launching containers in Puppet-managed infrastructure. The idea is to help users side-step configuration issues with the Docker daemon running containers.

March 30, 2015
The Docker whale swallows another one, Kitematic

Docker announced another acquisition yesterday, the 2^nd one in the last two weeks. This time they have swallowed up Kitematic, a Canadian open source project that makes it setting up Docker on a Mac really easy, Like up and running in 5 minutes easy.

I know some of you may be asking yourself, “I am not going to deploy Docker on my Mac, I only use Mac for my laptop.” Why would Docker spend money or more probably some of their valuable stock, buying an open source tool that lets users run Docker on Mac computers, not servers, not in the cloud. But Kitematic’s popularity is with developers who like the convenience of setting up Docker containers quickly on their laptops so they can develop, test and experiment on the fly. That is Dockers customer base right now.

It is a great little tuck in that gives Docker a good tool that will make their developer supporters happy as they develop apps in Docker containers. I spoke to the Docker folks about this acquisition and they tell me the 3 people behind Kitematic are already on their way to San Francisco to join the team.

The fact that Kitematic uses one of the recently released Docker Orchestration tools is a nice bonus showing what can be accomplished using the Docker tools.

At the end of the day it really is a nice addition to the Docker toolset. It also continues a trend of Docker cherry picking the best of the tools that are popping out of the Docker ecosystem. It makes for a great, quick exit strategy for those folks developing tools for Docker. In the case of Kitematic it was really quick. The 3 founders are barely out of school themselves. But from Docker’s point of view, it makes sense. Buying them early usually means buying them cheap. Usually but not always.

Bigger picture I was more excited about Docker’s recent acquisition of SocketPlane. The company while short on actual code and product had a good roadmap on SDN type of functionality to allow multiple Docker containers to work in concert. There is still a lot of work to be done to make that a reality, but more than a tuck in, Socket Place is a key to Docker success going forward.

There is no doubt that Docker is a darling of the tech world and developers in particular. The issue is when are we going to see Docker move from people tinkering with it to see what is possible, to actual deployments in a big way. Yes there are some real world deployments of Docker. I am not saying there is not. But that is rather small potatoes compared to what the promise holds.

To fulfill that promise enterprises will need to deploy dozens if not hundreds of Docker containers across disparate clouds and infrastructures. These containers will have to communicate and act in concert. That is still probably a little while off, but that is the vision that will see Docker live up to the hype.

The only dark cloud (no pun intended) on the horizon if this type of functionality cannot be engineered in an elegant manner or more likely another container option comes to market with this type of functionality first. Not saying that is going to happen, but Docker buying up these companies at the pace they are inhibits competition from popping up as well.

In any event, we haven’t see the end of Docker acquistions. They will continue building the business by swallowing the best of the ecosystem that is growing around it. There is also a ton of activity on the partner front with IBM working with them on Enterprise Containers, Microsoft working on Azure. In fact I was told in my Docker briefing that Docker themselves are bringing Docker to Windows for Azure in a much faster timeline than even Microsoft.

Stay tuned for more on Docker and containers the rest of this month on devopsy.kinsta.cloud.

March 12, 2015
Red Hat launches platform to support Docker, Kubernetes and enterprise-grade containerization

Red Hat continued its momentum to capture a piece of the containerization market last week with the official launch of its new Red Hat Enterprise Linux Atomic Host operating system. According to Red Hat executives, this is a natural progression in the Red Hat ecosystem as more of the company’s customers embrace Docker and other containerization technologies.

“Today, with the launch of Red Hat Enterprise Linux 7 Atomic Host, we are doing the same for Linux containers, bridging innovative open technology with the stability and security required by the enterprise,” said Jim Totton, vice president and general manager of the platforms business unit at Red Hat. “More than just an addition to the Red Hat Enterprise Linux portfolio, Red Hat Enterprise Linux 7 Atomic Host showcases the future of the enterprise application, a powerful, flexible application greater than the sum of its parts, and entirely fueled by the power of open innovation.”

–THIS STORY HAS BEEN MOVED TO CONTAINER JOURNAL–

March 10, 2015
Are VMs becoming El Caminos?

In 2010, it looked as if virtualization had won. We expected all servers to virtualize workloads and the primary question was which cloud infrastructure manager would dominate. Now in 2015, the picture is not as clear. I’m seeing a trend that threatens the “virtualize all things” battle cry.

Really, it’s two intersecting trends: metal is getting cheaper and easier while container orchestration is advancing on rockets. If metal can truck around the heavy stable workloads while containers zip around like sports cars, that leaves VMs as a strange hybrid in the middle.

What’s the middle? It’s the El Camino, that notorious discontinued half car, half pick-up truck.

The explosion of interest in containerized workloads (I know, they’ve been around for a long time but Docker made them sexy somehow) has been creating secondary wave of container orchestration. Five years ago, I called that Platform as a Service (PaaS) but this new generation looks more like a CI/CD pipeline plus DevOps platform than our original PaaS concepts. These emerging pipelines obfuscate the operational environment differently than virtualized infratructure (let’s call it IaaS). The platforms do not care about servers or application tiers, their symantec is about connecting services together. It’s a different deployment paradigm that’s more about SOA than resource reservation.

On the other side, we’ve been working hard to make physical ops more automated using the same DevOps tool chains. To complicate matters, the physics of silicon has meant that we’ve gone from scale up to scale out. Modern applications are so massive that they are going to exceed any single system so economics drives us to lots and lots of small, inexpensive servers. If you factor in the operational complexity and cost of hypervisors/clouds, an small actual dedicated server is a cost effective substitute for a comprable virtual machine.

I’ll repeat that: a small dedicated server is a cost effective substitute for a comparable virtual machine.

I am not speaking against virtualize servers or clouds. They have a critical role in data center operations; however, I hear from operators who are rethinking the idea that all servers will be virtualized and moving towards a more heterogeneous view of their data center. Once where they have a fleet of trucks, sports cars and El Caminos.

Of course, I’d be disingenuous if I neglected to point out that trucks are used to transport cars too. At some point, everything is metal.

March 6, 2015
Docker tinkers with Docker Hub for performance improvements

Docker’s strengthening its infrastructure components to accommodate the volume of use and wide range of use cases that inevitably crop up from such hockey-stick growth. The latest improvements come by way of refinements made to Docker Hub over the last month, according to an announcement made by the company yesterday.

The Docker Hub team reported that many of the changes were precipitated by performance issues experienced at the end of 2014, following the addition of security enhancements to prevent brute force login attempts. They reported that the enhancements added “overhead to the login process.”

“Under normal load there wasn’t much difference, but as the number of login attempts increased the site performance started to degrade,” they wrote. “The decreased performance affected not just the Docker Engine logins, push, and pulls, but also the Docker Hub website as a whole.”

In the process of identifying immediate issues causing the problem, Docker’s engineers audited the entire system and is in the middle of rearchitecting the authorization, push and pull system to improve performance and isolate systems for a bulkhead effect, ensuring that there aren’t cascading problems.

In the immediate term, some of the fixes already made include bulking out infrastructure capacity, including a tripling of servers in the Docker cluster for four-times the compute power and a production database upgrade for twenty times the RAM. It also improved login infrastructure to include added database indexes and more caching. The team also tuned queries for speed and fixed an issue that was causing database locks. Docker also switch its background worker queue broker from Redis to RabbitMQ and moved background workers to their own server cluster. Additionally, improvements were made to monitoring and metrics gathering, as well as to search functions.

Meanwhile, down the line they expect to make further improvements that include API throttling, breaking up larger services to improve scalability, replacing the brute force prevention library with a better performing library and adding API metrics and monitoring to head off performance issues earlier.

“We have made some good progress, but we are not finished yet, and we still have a lot more planned,” the Docker Hub team wrote. “We will continue to make improvements with every release, so that Docker Hub only gets faster and more reliable as time goes on.”

Such hiccups as the ones in December are understandable given the meteoric growth of the Docker community. Even just six months ago, the total number of downloads of Docker were under 3 million. By December that number had risen to 100 million. In spite of some questions about the security of Docker containers, Docker’s trajectory seems promising. Just in September the company closed on $40 million in series C investment from Sequoia and over the past few months it has been announcing a number of high-powered partnerships, including those with VMware, Microsoft and AWS.

Docker’s most recent partnership was announced last week when it teamed up with Google so that the online giant could offer a private registry service for Docker on its cloud platform. Currently in beta, the Google Container Registry will provide Google cloud customers with secure hosting, sharing and management of private container repositories that will include strong access control and server-side encryption.

January 27, 2015
Containers Are Designed For an Antiquated Application Architecture

In his most recent GigaOm article, “We’re finally headed towards autonomous, self-migrating containers for cloud-application automation“, David Linthicum discusses the value that containers, such as Docker and CoreOS Rocket, are bringing to DevOps and cloud systems management. For the record nothing Dave states in the article is incorrect with regard to the expressed value that containers provide for portability, however, there is an issue with the big picture view discussed for the role of containers in cloud application delivery.

Containers are predicated on the goal of deploying and managing n-tier application designs. By their nature, containers manage n-tier application components, e.g. database servers, application servers, web servers, etc., at the operating system level. Indeed, portability is inherent because all operating system and application configuration dependencies are packaged and delivered inside a container to any other operating system platform. Containers are…

–THIS STORY HAS BEEN MOVED TO CONTAINER JOURNAL–

January 13, 2015
Webinar: Managing CoreOS Container Performance for Production Workloads
As containers grow in importance, Linux distributions focused on container performance and management also are becoming mainstream. One of the most prevalent of these distributions is CoreOS. But how are real businesses using containers and CoreOS, and how are they monitoring container performance. After attending this webinar, you will have a better understanding of: * What is CoreOS and how does it work with containers * The trends in container usage and CoreOS * The key performance metrics you need to keep an eye on * How to manage performance as the container deployments grow This webinar will be led by Kelsey Hightower from CoreOS and Matt Williams from Datadog and they will leave plenty of time for all of your CoreOS, Containers, and Performance questions. By the end of the session, attendees will have a list of key performance metrics for containers on CoreOS they can start monitoring today.

Video

Slides

[seoslides embed_id=”e307162e07b0″ script_src=”https://devopsy.kinsta.cloud/embed-script/coreos-datadog-slides/4095/” overview_src=”https://devopsy.kinsta.cloud/slides/coreos-datadog-slides/4095/” site_src=”https://devopsy.kinsta.cloud” site_title=”devopsy.kinsta.cloud” title=”coreos-datadog-slides” /]

Date: Tuesday, January 27th, 2015

Time: 2pm eastern standard time
```
REGISTRATION LINK:(CLOSED) 
```
About Speakers:

Kelsey Hightower is Developer Advocate at CoreOS. has worn every hat possible throughout his career in tech and enjoys leadership roles focused on making things happen and shipping software. Kelsey is a strong open source advocate focused on building simple tools that make people smile. When he is not slinging Go code you can catch him giving technical workshops covering everything from Programming, System Administration, and his favorite Linux distro (CoreOS).

Matt Williams is the DevOps Evangelist at Datadog. This means he speaks, writes, and codes the technology to get everyone to believe in monitoring events and metrics using Datadog. In the past he has been the Evangelist at other startups including Placester and Yottaa, a Linux guy at Microsoft, and all sorts of other roles at Sun, Oracle, OpenText, and more. You can find Matt on Twitter at @Technovangelist.
December 23, 2014
Native support for Docker apps coming soon to Parallels

When I think of Parallels, the first thing that comes to mind is virtual machines. I have used Parallels for years as a means of running a virtualized instance of a Windows operating system from within Mac OS X. In addition to virtual machines, though, Parallels was also doing containers before containers were cool, and announced last week that it will soon include native support for Docker applications as well.

Parallels was working with container technologies years ago with Virtuozzo and the open source OpenVZ platform. The company claims to be the most widely-deployed containers platform in the world, with over one million instances. With the launch of native Docker support, service providers will be able to provide container-based virtual private servers on the Parallels Cloud Server platform to customers that want to build with Docker applications. Parallels

“A challenge customers face today is running Docker images in a virtualized environment based on a hypervisor,” explained James Bottomley, chief technology officer of virtualization for Parallels. “This takes a core benefit of Docker—that the applications run in containers at higher density and performance—and reduces that benefit considerably.”

Bottomley added, “With Parallels solution, the service provider can offer the customer an environment where Docker is running on top of Parallels Cloud Server containers-based virtualization for optimal performance and at high-density.”

Why does this matter? I spoke with Bottomley myself to get more information. He told me that the primary advantage of containers—particularly in the cloud—is they have different elasticity and density capabilities than a hypervisor. It is faster and easier to scale containers than entire virtual machines.

There is a disadvantage as well. Bottomley stressed that container technology requires that all of the containers share the same kernel. So, it’s not possible, for example, to launch a Windows-based container on a Linux kernel, or a Linux-based container on a Mac OS X kernel.

The inclusion of native Docker support in Parallels Cloud Server will be significant for smaller companies. Large enterprises like Microsoft or Amazon have virtually unlimited resources to invest in scalability. They can simply run container apps in separate hypervisor instances, and just continue to add more hypervisor instances to meet demand, but that is a costly and inefficient way to address scalability.

With Docker apps running on Parallels Cloud Server, smaller cloud service providers will have a secure, high-density virtual server environment with the elasticity and scalability to compete with much larger public cloud providers. Bottomley told me that the Parallels solution with Docker containers reduces the physical server overhead by a factor of one-third.

Philbert Shih, managing director, Structure Research agrees there is significant market opportunity for cloud service providers to deliver a cost-effective platform for developing and deploying Docker applications. “The answer lies in building an optimized infrastructure environment that is dense and scales, while being able to maintain high levels of performance and facilitate portability of workloads.”

The Parallels announcement is just the latest in a string of victories for Docker. Microsoft added support for Docker containers on Linux virtual servers in Azure earlier this year, and recently announced a partnership with Docker to bring native container support to the next iteration of Windows Server.

As the container concept gathers mainstream steam, though, Docker also faces increasing competition from rivals like Rocket. Expanding Docker’s sphere of influence with native support on popular, and widely-used platforms will help to cement it as the de facto leader of container apps.

The update to Parallels Cloud Server to introduce native Docker support is expected in the first quarter of 2015.

December 18, 2014
Microsoft partners with Docker to bring containers to Windows Server

Docker has taken the DevOps world by storm, and that has caught the attention of Microsoft. Now, Microsoft and Docker are teaming up to bring native support for the container technology to the next generation(s) of Windows Server.

Docker is an open platform that enables organizations to run any application as a Docker container. The Docker containers can be implemented across virtually any infrastructure—delivering tremendous agility and flexibility for managing virtual applications. The container concept accelerates software development timeframes, and allows organizations to reduce costs and improve efficiency at the same time. It’s easy to understand why Docker is so popular.

There have been more than 21 million downloads of the Docker platform, and there are over 45,000 “Dockerized” applications available on the Docker Hub Registry, and 13,000-plus Docker-related projects on GitHub. Just last month Docker took in $40 million in another round of investment.

Major names in tech, like Rackspace, Google, and IBM have already pledged their support for Docker. Now Microsoft wants to bring the hot technology to Windows.

Docker Engine for Windows Server will allow customers to create Docker container applications that can run locally on Windows Server, or on virtual Windows Server instances in the cloud on Azure. In a blog post announcing the Docker partnership, Ross Gardler of Microsoft Open Technologies explains, “As part of this announcement, MS Open Tech is contributing code to the Docker client that supports the provisioning of multi-container Docker applications on Azure. This code removes the need for our cross-platform CLI to bootstrap the Docker host. In other words, we have taken a simple process and made it even simpler.

“This development is interesting because Docker has been a really fast moving technology having come out of almost nowhere a year ago to where now it is an incredibly widely supported technology,” stated Al Hilwa, an IDC analyst. “It is a bit of engineering but Microsoft wisely decided that they did not want to leave Windows developers out of the Docker party. This is good for Microsoft because those 45K ‘Dockerized’ applications will be able to run on Windows.”

To be fair, Microsoft isn’t completely new to Docker. Earlier this year it added support for Docker in Azure, but only on Linux. This new partnership launches an open source project to develop a Docker Engine runtime for building, running, and managing Docker containers natively within Windows Server. Docker Engine for Windows Server will be managed as a Docker open source project, but Microsoft will be a major contributor to the community.

The Microsoft blog post describes why Docker’s container technology is so important for the fast-paved development of a DevOps environment. “Containers do not include a full operating system, consequently rapid development and scaling of container-based applications is possible through very quick boot and restart operations. Furthermore, highly efficient creation of modified container images, by virtue of only capturing the differences between the original and new containers, enables improved management and distribution of containerized applications; the resulting images are both small and highly portable across almost any platform.”

Hilwa summed up with some thoughts about the new strategic vision of Microsoft. “What is even more interesting is that the work on the Docker engine in Windows is being done in open source—really showcasing the new Microsoft attitude towards open source (e.g. the open sourcing of Roslyn a few months ago). I really think we are seeing a faster and more open Microsoft, one that is more willing to integrate and collaborate with competing technologies without a lot of hesitation.”

October 16, 2014
Why We Build in Docker Containers

We are using Docker as a virtualization technology for our continuous delivery/deployment pipelines, in Rultor.com. We experimented with other technologies and Docker’s approach proved to be the best fit for our needs. Docker is a command line tool that can run a shell command in a virtual Linux, inside an isolated file system. Every time we build, we want to run build scripts in their own Docker containers, perfectly isolated from other builds in other projects.

There are three main reasons…

–THIS STORY HAS BEEN MOVED TO CONTAINER JOURNAL–

September 30, 2014