devopsy.kinsta.cloud

Tag: docker

Q&A: Sematext’s Thies on Docker Monitoring, Part 1
Stefan Thies is DevOps evangelist Sematext Group. I recently spoke with him about Docker’s use of Sematext Logsene logging, anomaly detecting and alerting software and SPM monitoring software. Read on for part one of our conversation.

David Geer: Technically speaking, for the purposes of this article, what is logging?

Thies: Logging means to store the output of programs (applications, servers) running on a computer. Typically, log messages that software applications produce are stored in log files. However, software running in Docker containers writes its log messages to a virtual terminal (console). The Docker Daemon (the server process managing the containers) has several facilities to forward log messages via log drivers or the Docker API to external applications, which are responsible for storing the log messages. This is not the traditional way of working with log data. This changes the log collection and storage mechanisms and forces many organizations to change their workflows for log management by introducing new tools.

Typically, you would use a service called syslog to receive and store log messages on servers, but this requires special configuration for all Docker containers. While Docker made it easier to deploy applications, the new technology made logging and monitoring much harder as you could not use the traditional tools apart from integration with Docker.

This is where companies like Sematext provide added value, by enabling:
- Integration into the Docker Ecosystem
- Cloud storage for log files
- Indexing for full text search, analytics, visualization and alerting on log data
Geer: What has Sematext Logsene contributed to logging, technically?

Thies: Logsene Log Management includes all required workflows and security procedures to handle log data. Logsene provides role-based access controls for access to log data. Logs are collected and shipped to Logsene servers (running in the cloud or on premises) where the data gets stored and analyzed.

Other important features are:

• Full text search
• Analytics
• Alerts
• Anomaly detection

Geer: In what way is Logsene/Sematext available to users and organizations that seek solutions to capture logging for monitoring their Dockerized distributed applications?

Thies: Logsene is available on premises and as SaaS in the cloud. It fits in hybrid environments as well such as running the Dockerized application in-house on bare metal while shipping logs to Logsene SaaS. In this case, only Sematext Agent for Docker runs on premises.

Sematext Docker Agent automatically detects running Docker containers and automatically collects logs from the containers. The Sematext Docker Agent can transform unstructured log messages into structured data for analytics. We do this with integrated parsers and a pattern library, which detects log formats to structure logs. The final task of Sematext Docker Agent is to ship the structured logs from all containers securely to Logsene, where indexing for full text search and analytics queries takes place. The Logsene user interface provides all tools for search, visualization, anomaly detection and alerts on log data.

Geer: What is SPM for Docker? What role does it play?

Thies: Both Logsene and SPM collect data via the Sematext Docker Agent. SPM is a performance monitoring, anomaly detection and alerting solution for many server applications like Web servers (e.g., Apache, Nginx), databases (e.g., Cassandra, MongoDB, HBase, MySQL, etc.), big data engines (e.g., Spark, Storm, Hadoop, etc.) and full text search engines (e.g., Elasticsearch, Solr). Sematext Agent for Docker collects not only container logs but also performance metrics of all containers at the same time, plus Docker events.

SPM for Docker provides operational insights into application performance on the Docker platform. It is not limited to performance monitoring—it integrates with Logsene for log management in a single user interface. DevOps teams can find log messages in correlation with potential performance bottlenecks. Most organizations use different tools for performance monitoring and logging. Sematext has recognized that metrics and logs must be available in a single platform to provide detailed operational insights without having to switch between different tools and without overhead for things like integration of multiple tools, multiple user interfaces, logins managing access permissions for multiple tools, and billing or licensing from multiple vendors such as NewRelic or Splunk.

Watch this space for part two of my conversation with Stefan Thies of Sematext.
May 3, 2016
Container Usage and Adoption, Here is Your Chance to Tell Us What You Think

We are teaming up once again with ClusterHQ for an update on Container usage and adoption. Our new survey is available and shouldn’t take more than 5-7 minutes of your time. We are looking to see if the container adoption and usage has changed since our last survey last summer. Please take a moment to fill out the survey. If you fill out contact info at the end, you are eligible to win a $50 dollar Amazon gift card. One in 50 wins a card. If you don’t want to give any contact information, you don’t have to but we can’t contact you if you were to win a gift card. Either way we would love your input!

While containers are capturing lots of buzz and attention, many still doubt their impact in real world procurements. However, we are hearing enough subjective feedback to think that this is not the case. We think containers are catching on quickly, perhaps even quicker than cloud adoption did at similar time frames. But that is subjective. We are looking for numbers that prove or disprove it. So even if you are not using containers now, we want to know that too.

The standard line is that containers are being tested in dev and test environments, but are not ready for prime time. Yet Docker, CoreOS, Kubernetes and the entire container ecosystem point to more and more customer references. Can everyone be just playing with containers in their labs? We don’t think so.

On the other hand, there are questions about security, scalability, networking. Should you lock in to one vendors complete stack? How open should the ecosystem be? What about data, databases, OS, etc. Lots of questions and we really don’t have a lot of objective data points. Help us change this by giving us your own thoughts, experiences and ideas.

We will share the full report with analysis with our readers when complete. But don’t miss this chance to be heard and make a difference.

April 26, 2016
State of DevOps Survey – The Ultimate Feedback Loop

If there’s any consistent criticism of the larger DevOps movement, it’s that the talking heads and industry gadflies are driving the conversation, not the people in the trenches.

All the thought leadership panels and “how-to” content (including the written and AV materials on this very site) make for great watercooler discussion, but lack the input of those dev, ops and IT management pros who are actually living the DevOps experience, some critics have said.

Well, fair enough. Talking heads are paid to talk. Thought leaders are chosen to lead the debate at conferences and on webcasts. Bloggers blog. Reporters report. This is simply the way of the world. Certain voices are heard via the channels they frequent. It is, as so many people are fond of saying these days, what it is.

So what about everyone else? What of the people reading this stunning prose and listening to these talks while rolling their eyes based on the connectivity of it all to their real-world experience? How can their voices be heard?

Independent of jumping into the blogsphere or applying to speak at said industry events – which you should, btw – there’s one significant opportunity available to anyone who cares to take the time – participating in industry research.

Over the next few business days, citizens of the DevOps world will have their last chance to contribute to the industry’s most widely read and respected research project, the State of DevOps Report.

If you haven’t done so already, take the time to fill out this year’s survey, and make sure that your voice is heard when the subsequent report is published in June. Sure, it takes a little time, about 15-20 minutes; but that’s because it goes to the trouble of exploring the real-world issues that weigh most heavily on today’s practitioners.

Why is this particular project so important, or even credible? Consider that the minds behind the survey and analysis resulting in the 2016 State of DevOps Report are those authors and talking heads that actually get the most respect.

Backed by DevOps Research and Assessments (DORA), the independent think tank operated by a partnership of Jez Humble, Gene Kim, and Nicole Forsgren – perhaps the most respected voices on the landscape, along with Nigel Kersten, the project is categorically aimed at surfacing the truth about DevOps adoption today – not merely promoting the interests of vendor sponsors.

Like previous iterations of the report, the 2016 edition will focus on topics including IT performance, technical practices, culture, lean management, and organizational performance. This year’s survey also examines several new areas, including the ROI of IT performance, security and compliance, and containers.

To further entice participation, project leaders are also offering free tickets to the DevOps Enterprise Summit, along with forthcoming books from some of the industry’s leading advocates to a lucky few who take the time to get involved.

DevOps success is fundamentally based on the notion of increased collaboration and continuous feedback among the people scattered across the SDLC. Likewise, the legitimacy of this research project is utterly dependent on the input of as many real-world practitioners as possible.

If you’re one of those individuals, and you really care whether or not everyone else involved in the advancement of DevOps understands what this all means to you, do yourself a favor and participate.

Much like an application being built using DevOps practices, this quality of this project will be directly determined by the degree to which anyone with insight is willing to get involved and build toward optimal results.

Take the 2016 State of DevOps Survey… today!

April 15, 2016
DevOps Debates: Monoliths or Microservices
From the time software came into existence, application software has been booming to leverage the benefit of the computing paradigm. During this cycle, it has transformed decades of design and architectural thinking, and has crossed all aspects of application development and management needs, be it configurability, reusability, maintainability, scalability or security (a good list is of these “ilities” is here).

While many have been addressed, one “ility” that has turned a lot of eyes and attention in the last few years is deployability. More than that, it is the ability to do so continuously. This requirement has changed the thinking of building software for two main reasons: (a) Businesses have started asking to release software faster than ever, even in verticals unheard of before; and (b) Cloud and infrastructure as code has made faster deployment a easier possibility. When you look at application design with deployability in mind, given the need to be agile, what you end up getting is a microservice.

Most applications of old style addressed most “ilities” in an architecture but did not consider “continuous deployability” as a core need as well. I call these monolith from a deployment perspective. They are tightly coupled, and for any small change a complete build of the application was necessary.

Now the question is, Should you really look at microservice as a silver bullet to deploy all software? Should you consider that as the de facto standard to deliver any application functionality? Will applications cease to exist the way we know? Here are my thoughts. As usual, feel free to add yours or write to me personally, as some of you do as well.

Applications as monoliths will stay as they are
1. Monoliths have evolved over generations of design and architecture tests and stood the test of time. The focus on modular development is, in fact, a way of creating modules, which are built with separation of concerns in mind.
2. Monoliths were built to run for a long time. It had addressed some of the “ilities” well before they became considerations.
3. Applications will remain easier to debug, as most of the debuggable areas are in developer’s hand, instead of having to work with a team.
4. Applications will perform better, as they can share data in memory; they don’t have to do out-of-process calls to get one task executed.
5. Applications will be more secure, as they will be a black box from an end user perspective or even a hacker perspective, with little coming out from the system.
Microservices will be a monolith-killer all the way
1. Deploy, deploy and deploy to correct your problems away. Fail fast and fix faster is the mantra that you can practice.
2. It helps bundle two-pizza teams owning microservices and hence embrace better ownership and faster response.
3. The overall architecture to deliver an end-to-end use-case may become complex, but each piece of the jigsaw puzzle can be handled faster.
4. In the faster-moving world, business wants to try a concept rather thinking through all the “ilities” ahead of time, and progressively make everything better. Microservices helps abstract all complexities much better.
5. Microservices will help developers think out of box from day one, while helping business deliver smaller chunks, without having to make the system a “big ball of mud.”
At the end of the day, irrespective of the approaches, the key is to ensure the applications are managed well, changes are refactored well, the code that is written is readable and maintainable, and significant test assets are created to regress the entire system quickly. So whether it is a large system or microservices, you are able to make any necessary changes quickly and can deploy fast.

While microservices surely will help this new faster-deployment thinking, I—and I am sure you all—have also seen large monolithic systems that are making delivery at an astounding pace.

Thoughts?
March 18, 2016
Live Blogging Container World

February 17, 2016
DevOps Debates: Image-Driven or Build-Driven Delivery
In the last 12 months, Docker and the containerization paradigm has challenged the foundation of some of the software delivery principles. People are resorting to things that were unheard of 18 months back and are now using them to successfully deliver software faster and cheaper. One such thing is the debate around build-driven delivery vs image-driven delivery.

Over the last few decades, software has been packaged and built by a build system and tagged with a number; testers aligned their testing to the tag as the entity being tested and promoted them until it reached production. The build number was one of the most important tag for anyone to communicate about the status of the software in the delivery pipeline.

For some time now—primarily due to Docker—several teams are moved to a methodology in which the image defines the version of a software. Every change in application program and deployment program will together create a new image, which is tagged and used as the version of the software to be launched for downstream use and finally into production.

The debate then is, which option is better?

Let me take a shot at this, and will invite your thoughts on this as well.

For builds-driven delivery:
1. Builds helps software to be tagged irrespective of the target hardware and other deployment options. It bundles all dependencies to ensure the software works when deployed to the expectation. This provides the development team a good hand-off point to move on to the next set of work, or sprint.
2. Builds are much smaller in size, as the build bundles only necessary software needed by the application; hence, keeping them aligned with every change will not be a costly storage proposition.
3. Versioning of artifacts and bundling them at run time has given tools such as maven to build and link dynamically, collecting all dependencies from the cloud. This makes it easier to keep a thin build and further reduce the redundant data to be kept in each build.
4. Build repositories have matured over the years to provide a very easy way to version and mark dependencies so deployment tools can find new versions and automatically upgrade environments as needed.
For images-driven delivery:
1. Images takes the concept of build to the next level and ensures the variability in target hardware and deployment options are removed completely. It bundles the application as given in the build or built as part of the image creation and pushes the image as a build.
2. Docker registries have started to compare and contrast themselves into a build repository by providing similar options for end users.
3. The images-driven option provides a much better benefit of standardizing all your environments, removing the question,”It works in my environment; does it in yours?” If it works in the container in a developer’s laptop, it should work in production as well—everything is same, except for the size and number of resources.
4. Images are much larger than builds, given that it has to now carry the system software and the application base software within the image. This option was unheard of earlier, but with container images coming down to megabytes, this is not such a big issue today. With layering sizes can be reduced much further as you start peeling the onion of dependencies. Take a look at this nice article by Brian on this topic.
I guess the choice depends on where your application is in its life cycle, the questions teams should really ask are:

a) Do we need to change too much to pick an option for an incremental benefit?

b) Do I have the issue of environmental differences?

c) Can I merge these options to get a hybrid solution going?

But for me the larger question is, Which option would you choose, if you were starting from scratch tomorrow? I would love to hear your thoughts.

Finally, irrespective of which way you lean, image-driven solutions is one of the best way to bring DevOps culture to a team: If the image works, the team has won; it doesn’t matter whether it was ops code or dev code!
February 8, 2016
JFrog, Puppet, Datical & Sendachi – Money Continues to Pour Into DevOps

The stock markets may be bearish, the price of oil is down the drain, but money continues to pour into the DevOps tools and solutions market. Just in the last few days 4 companies have announced significant new funding. Whether this is a last gasp before the doors slam shut, the tech industry thumbing their nose at the rest of the business world or proof that the recent market downtrend is a red herring I don’t know. But as someone who has lived through more than a few bubble bursts in the tech world it bears watching.

For now though, the good times continue to roll! Lets review the new funding news:

JFrog raises an astounding $50m! That amount of money is nothing to sneeze at! I spoke with Shlomi Ben Haim, CEO at JFrog. They have over 120 people working there in offices in Israel, HQ in Santa Clara and France. Their JFrog Artifactory and JFrog Bintray are hosting software for organizations the world over, big and small. Ben Haim and his team, as well as his blue chip investors believe the time is now as DevOps goes mainstream. They want to make sure they have the money to capitalize on the opportunity. Congrats to the JFrog team and we are excited to see what this money translates to for them.

Puppet Labs raises $22m in debt financing. While Puppet continues its phenomenal growth it took the opportunity to grab $22m in debt financing from Silicon Valley Bank. I spoke to Puppet CEO Luke Kanies about the raise and he told me that he is not even sure the company will use this money, as they still have a sizable portion of their last raise in the bank and are getting close to profitability. But at today’s low interest rates who can blame him for taking advantage of available cash? There was a time when debt financing was frowned upon. Why take on debt when you could give some equity for cash. But in today’s low interest rate/ high valuations world, why give valuable equity when you can get money for near free? In addition to the money, Puppet is also expanding their offices by about 37,000 square feet and have appointed financial veteran, Lou Lavigne to their board. This continues their preparation towards and IPO down the road. Of course with the market where it is, they may be better off staying private.

Datical raises $8m Series B round. Databases as part of the DevOps movement are going to have their day and Datical wants to be the company to make it happen. To do that they have raised a Series B led by S3 Ventures. They money will be used to continue development of their agile Database automation tools, as well as sales and marketing. I had a chance to speak with Ben Geller and Robert Reeves of Datical. One of the things they want to use the funds for is to expand beyond the North American market. That is a wise move given the economic environment and the reality of a global marketplace. There was a time when Internet companies could concentrate on the US and thrive, but the demographics just don’t support it anymore.

Finally, today marks the debut of Sendachi. Sendachi is a consulting firm dedicated to helping companies with their DevOps transformation. It was (initially anyway) formed by the merger of Seattle-based Clutch and London based Contino. Along with $30m dollars from Columbia Capital. The money will be used to expand the firms marketing and business teams. Some portion of it though will also probably be used to fund further acquisition. I had a chance to speak with former Clutch and now Sendachi CEO,Steven Anderson and former Contino founder and my friend Ben Wooten. They were both very excited by the potential of this Atlantic spanning venture. Anderson said that the $30m is just the start and if necessary more capital could be available for strategic purposes. My feeling is that we may be seeing the birth of a roll up in DevOps consulting practices in Sendachi.

In other M&A news on this topic, Docker announced it has acquired Unikernel Systems as well.

Congratulations to all of the companies raising capital and buying and selling. Anyone who has had to raise money or have bought and sold a company knows that it is not an easy thing to accomplish. Who will be the next DevOps company raising money? We probably won’t have to wait long to find out.

January 21, 2016
Agile Infrastructure: Because your Infrastructure Deserves Tests, Too
Editor’s Note: This post was updated Aug. 4, 2018, to reflect an updated slide presentation at the end of the article.

Infrastructure as Code is a truly powerful concept whose significance is best described with the words of Adam Jacob, CTO at Chef, in Web Operations: Keeping the Data On Time: “Enable the reconstruction of the business from nothing but a source code repository, an application data backup, and bare metal resources.” And so it’s not for nothing that treating infrastructure as code is regarded a key principle for reducing cycle time and maximizing flow in Continuous Delivery and DevOps, respectively. Even if your plans are less bold as you read this article, wouldn’t you agree that consequent validation of what you automate on the infrastructure level — long before it goes into production — would be extremely powerful?

So, bear with me for a while and please don’t rush head over heels into automation by simply picking one of the established infrastructure automation tools such as Chef or Puppet, getting everyone trained in Ruby, and blindly start to “automate all the things”. If that’s how you do it, then you are most probably doing it wrong. In You’re Doing DevOps Wrong. Automation in the Enterprise., author Alan Sharp-Paul suggests that, before doing any automation, you first have to understand precisely the requirements and how to validate them to gauge success and maintain quality of your automations over time:
- Are the configuration files in place? Do they contain the right settings?
- Is this port open? Is that port closed?
Why Agile Infrastructure?

Honestly, you and I wouldn’t expect anyone to come up with reliable software merely by choosing an arbitrary programming language and a nifty IDE without having some sort of engineering process in place, right? Likewise, the (automated) provisioning of a solid production environment demands a no less methodical approach. Luckily, Infrastructure as Code enables us to treat the provisioning of infrastructure as well as automating deployments and configuration as an Agile engineering discipline. Although the concepts of Agile Infrastructure are not entirely new, testing against live infrastructure has only recently been facilitated with the consolidation of tools such as Ansible, Chef, Puppet, Docker, Vagrant and Serverspec under the hood of Test Kitchen (see more on that below). Here are some of the key implications of applying Agile software development practices to the infrastructure level:

Alignment of Devs and Ops

Agile software development teams are driven by customer demand and aim to produce outcome in short, regular intervals. Enabling IT Operations and Infrastructure departments to work in small iterations and react to changes quickly allows them to align with and effectively become driven by software development. The outcome of these joint efforts will be both working and deployable software at the end of each iteration.

Fail Fast: Feedback Loops

Executing infrastructure tests as part of your Continuous Delivery build pipeline (before an application is deployed and long-running tests are executed) establishes another important feedback loop that enables you to “fail fast” and get the right people involved. The underlying principle is simple: by expressing your intentions twice — once in the code and once in a test — you’ll know you’ve caught a bug whenever these assumptions don’t match.

Regression Safety

Believe it or not: whether you are using Ansible, Chef, Puppet or any other — you’re running on code! Their respective issue lists are full of unresolved issues of various severities with new issues coming in regularly. Having a sufficient safety net of tests for your infrastructure in place allows you to quickly identify bugs as they creep in (from either side) and protects you against unwanted regressions. After all, a bug in your infrastructure can be very hard to track down and can be much more severe than a bug in your application.

Refactoring

Code refactoring becomes a breeze when you have tests in place that immediately alert you when you broke the desired state of your infrastructure (and all you wanted was to make your code simpler and more maintainable). This is invaluable when you want to recompose your existing infrastructure from 3rd-party components offered on public hubs such as Ansible Galaxy, Chef Supermarket or Puppet Forge, and especially when you are migrating away from your existing automation stack.

The Red, Green, Refactor Cycle of Test-Driven Development

Test-Driven Development (TDD) is a popular software development process that involves writing code in short, repeating development cycles with the goal to enforce good design and enhance confidence. While it is certainly no silver bullet and has been controversially discussed in the past, the Red, Green, Refactor Cycle, which is at the heart of TDD and whose principles I want to quickly summarize here, hopefully gives you some inspiration on how to incrementally build up your infrastructure:
1. Red: Understand the requirements and the state you want to implement. Then think about how to capture this state in a test. Implement the test, then run your entire test suite and see the added test fail — and if it doesn’t, something is really wrong.
2. Green: Implement just enough functionality to make the test pass. Don’t care much about readability, simplicity and design for now — just make it work. Run your tests again and watch all tests pass. If they don’t, either fix your test or your implementation, or take a step back (revert to a point in your code that’s known to work) and start over with a smaller increment.
3. Refactor: Now that your tests pass, improve your implementation and make sure, by running your tests again, that you didn’t break anything. Once you are done refactoring, start the cycle over again.
Keep in mind that each repetition of the cycle refers to a small increment. In terms of infrastructure, such an increment could be the installation of a user, a package or the placement of a configuration setting. When you arrive in a spot where you broke something, you can easily revert to a point in your code that’s known to work. Leveraging such practices really makes the difference between Infrastructure Automation and Infrastructure as Code, as described in Infrastructure as Code – Automation Is Not Enough by Keith Morris.

Test Kitchen, Terraform, AWS and awspec

Now, how to get there? Test Kitchen is an easily extensible test harness that allows you to test your code written in Ansible, Chef, Puppet (and others) on various cloud providers, virtualization providers and operating systems with tests written in a variety of test frameworks.

In the following presentation, I give hands-on examples for you to quickly dive into the topic of using Test Kitchen with awspec as a framework to test Terraform deployments on AWS:
November 13, 2015
Containers

Containers

— ROELBOB

October 21, 2015
Creating Dev Environments With Docker Compose
The Slack notification popped up on my phone at 2pm Friday afternoon. “Argh. This is never good” I thought to myself. The message said “do you know anything about Vagrant?”. It was from a client of mine and I knew their environment well enough to know that this question was going to go down a rabbit hole the likes of which would leave Lewis Carroll awestruck. To compound matters, I was in the middle of nowhere, halfway between Las Vegas and Phoenix. I found a place to pull off, overlooking a vast, expansive maze of canyons and riverbeds leading nowhere and everywhere at once. I thought to myself “how fitting” as I started to reply on my phone.

When launching a startup, there is no rule other than “find out what the
customer is willing to pay for, then build it”. My client, Roger, had nailed it. His next message confirmed this as I read “I’ve been beating my head against a wall for two days trying to get this Vagrant image running. I’ve got new developers starting next week and need to get them a dev environment setup”. Roger had just discovered startup rule #2: “nothing can prepare you for the panic of success”. His current environment was very solid: running in AWS, everything was deployed via OpsWorks to both staging and production environments. It works great… if you’re the only one in staging. It wasn’t going to support multiple developers changing code simultaneously, so he was looking at Vagrant for a solution. Each developer needed mongodb, elasticsearch, redis, and two node.js servers: one for the api and one for the web ui. Vagrant images are great, but I’ve often found them to be most valuable when hand-curated, like craft beer. They are also difficult to share as it requires shuttling large vm images around between developers.

After reviewing the error logs, I could tell he was facing issues with security permissions between the vagrant image and the host OS. I decided I wanted to try something different. I wanted to try building it out using Docker containers. I had a feeling I could accomplish the same results with Docker, plus the added benefits of the entire environment being stored as config files in a git repo. Small, concise, version controlled, portable… I was liking this idea more and more. I sold him on the idea as well as buying enough time to get back to civilization where I could work on the idea (though the overlooking view was pretty inspirational).
Back at the office, I broke open my laptop. I was familiar enough with Docker, but had a suspicion that Compose would help me tie the disparate pieces together. I started with the node.js servers. It was easy enough to kickstart them using a Dockerfile. (NOTE: I could have done this in compose as well, but had a feeling that I would want some extra flexibility later so I created separate Dockerfiles for them.)
FROM: node:latest EXPOSE 3000
Next, I created the docker-compose.yml file to define the environment. I was able to define all of the containers (minus my previously created node.js servers) using images from Docker Hub.
```
web:
  build: ./web
  ports:
    - “3000:3000"
  volumes:
    - ./code/web:/srv/www
  command: node /srv/www/server.js

api:
  build: ./api
  ports:
    - "3000:3000"
  volumes:
    - ./code/api:/srv/www
  command: node /srv/www/server.js

redis:
  image: redis

mongodb:
  image: tutum/mongodb
  ports:
    - "27017:27017"
  environment:
    MONGODB_PASS: password

elasticsearch:
image: elasticsearch
ports:
- "9200:9200"
- "9300:9300"
```
Once this was done, I created links in the web container. This creates environment variables and hostfile entries in the web container, allowing the web container to access the elasticsearch container via the hostname “elasticsearch”, and the mongodb container via the hostname “mongodb”, etc…
```
  links:
    - api
    - redis
    - mongodb
    - elasticsearch
```
The environment was complete, and I brought it up with the “docker-compose up” command. The added benefit of this environment is that the docker-compose terminal window maintains a steady stream of logs from all of the containers in a central window. This allows the developer to view the logs realtime, without having to hunt for them.

The only real stumbling block was in the use of VirtualBox. I created the environment on OS X. Since OS X can’t run Docker natively, it all runs in a VirtualBox machine on my workstation. This meant that I needed to go into VirtualBox and setup port forwarding for the required ports (3000, 3030, 27017, 9200, 9300,6379) so that I can access the services locally.

Including time spent learning the basics of Compose, the entire environment was up in 2 hours. By using Docker, Roger’s new developers will be able to clone the config from a repo and be up and running in minutes regardless of whether they are developing on OS X, Windows, or Linux. Future additions to the repo can be shared by all by continuing to update the repo.

Interested in trying it out for yourself? All the code described can be found in my git repo at https://github.com/rekibnikufesin/docker-compose. Be sure to let me know your thoughts in the comments or via twitter @wfbutton!
October 13, 2015