devopsy.kinsta.cloud

Author: Alberto Arias Maestro

What’s hot with DevOps

DevOps as a movement to transform people, technology, processes for higher business value is here to stay. It’s disrupting organizational processes replacing the old with the new. Command and control IT, a historical approach to regulating tech top-down, is trending toward decentralization thanks to DevOps.

Software source control is another such case. We saw Git take on the source control bigwigs and redefine how we collaborate and build software. Now, which other areas is DevOps disrupting?

What’s hot with DevOps?

Application services

We have a famous saying in my home country Spain, The King is dead, long live The King! I say, “The application is dead, long live the application.” I’m referring to the practice of implementing applications as monoliths. Monolithic applications are a dying concept. Why? Because it’s better to design them as a collection of services that link together. Such a flexible structure allows service reuse and control at a granular level. And you manage their lifecycles independently. Attempting to control these services centrally though is the equivalent of trying to balance the federal budget.

It’s challenging to manage the lifecycle of services because of the very nature of agile business decision-making. Since services are much more granular, managing their lifecycle is also such. That’s why current application lifecycle management tools do not scale for service-oriented applications. Next generation application lifecycle management requires a focus on the service not on the application.

Microservices and Docker

Microservices is to infrastructure what service-oriented architecture (SOA) is to applications. Next generation applications require agile infrastructure to sustain the cadence of faster releases. Microservices align infrastructure management to the much more granular world of application services.

Over the last year, microservices gained traction fueled by the container revolution. The unit of infrastructure shrank from a VM to a container allowing applications to run as a composition of smaller self-contained services making application releases that much more agile. So my second bet is on containerizing application services.

Hybrid and multi-cloud

Third, cloud infrastructure management tools must scale to the next generation of the container and service-ready applications. In the world where infrastructure is a service, application architecture can no longer rely on infrastructure implementing a particular way. A development or QA environment does not have the same SLA needs or constraints of a production environment as an example. So applications must be able to run across a diverse infrastructure.

The first generation cloud management tools focused on managing servers in public clouds. That focus made every infrastructure provider a special case. Special cases don’t scale though. The cloud management tools must scale by removing dependencies between applications and infrastructure. Thereby, infrastructure becomes a choice, not a dependency. Application services must be able to run in private, public, hybrid, or multiple cloud environments. Such elasticity and independence are what applications demand from infrastructure.

Having been in this space for more than a decade now, I live and experience these DevOps trends daily. What are your thoughts, are there any trends you’re following closely?

May 26, 2015
Turn IT into a Service Catalog

Unless you live under a rock in the world of cloud computing, you’re probably aware of a growing generational gap between public clouds and enterprise IT. (more…)

January 19, 2015
Bird’s Eye View of IT as a Service

In working closely with large enterprises for several years, we consistently hear businesses talk of IT needs to achieve critical and strategic goals. Commonly these large enterprises span many verticals like technology, online media, finance, banking, and more.

At the heart of their business goals, lie technology hurdles they need to overcome to scale as enterprises of the next decade.

Visualizing IT as a service

So we captured in this mind map high-level business goals that require a broad set of IT services ranging from infrastructure, applications, management, integration, service provider, and architecture.

If you look to the left, you see needs centered around applications. These detail all the services required to provision, deploy, and manage applications including application architecture models to design automated deployments and serve all the right infrastructure resources.

On the right, you see needs that revolve around compliance, reporting, and integrating with various service providers to supply virtual infrastructure and a slew of related services to keep the infrastructure running smoothly and reliably.

We find that this visual captures the entire IT-as-a-service industry as a whole and this exercise helps to understand where the customers’ needs are and where in that spectrum we assist them.

See if you can pinpoint the areas where DevOps and IT Ops come into play.

Enjoy the mind map and click to see a larger PDF view!

January 8, 2015
Bringing Collaboration, Orchestration and Lifecycle Management to Docker Containers

As you most likely know, Docker is a Linux container technology that stands to revolutionize or standardize the delivery mechanism of applications and services in the cloud.

Docker, which just turned 1.0, has created an enormous amount of buzz, a vibrant community, and an extremely interesting ecosystem upon which developers and operations teams can draw to ultimately build better applications and better utilize cloud resources.

We, at ElasticBox, wanted to see what we could do by combining these two great technologies. With developer and IT operations in mind, we’ve created the ability for you to consume Docker as a service with collaboration, orchestration, and lifecycle management capabilities. You can now create Boxes and use Docker as a service, giving you and your team the ability to distribute Docker files as Boxes across your organization, collaborate on the continued development and maintenance of these assets, provide an orchestration tool for the deployment of application or infrastructure components, and finally, manage their lifecycle across different environments – development, test or production.

Tutorial – Deploy the Ghost blogging platform in a Docker Container on any cloud using ElasticBox

To demonstrate this integration, let’s launch an instance of Ghost, a simple but powerful publishing platform, in a Docker Container on the Cloud Provider of your choice. By the end, you will have built a Ghost / Docker Box in ElasticBox that can be deployed, versioned and shared. For step by step instructions, click through the Slideshare below.

Deploy ghost in a docker container through elastic box in 6 easy steps

For more information on how to use Docker in ElasticBox, please visit our documentation.

July 30, 2014
ElasticBox Jenkins Plugin – Easy Continuous Integration Setup

Introduction to Jenkins and ElasticBox

ElasticBox has recently released a Jenkins plugin which makes it possible to configure a continuous integration or continuous deployment scenario without writing any code or scripts! How have we done this you ask? Simple, it was magic. In all seriousness, magic is the end result if you integrate ElasticBox and Jenkins with the source control software of choice.

For those who rely on Jenkins for daily work, performing integration builds, orchestrating complex, large scale scenarios for testing or managing continuous delivery of applications with many configurations across multiple environments, is not uncommon. These types of scenarios require a lot of time setting up slaves and performing manual steps to configure the provisioning of cloud providers. Keeping consistency and across the environments becomes exponentially complex when you factor multiple clouds or hybrid cloud environments into the picture.

You can reduce the amount of manual configuration and scripting, while abstracting the provisioning of infrastructure, by using ElasticBox with the ElasticBox Jenkins Plugin. This means that you can use ElasticBox to solve complex continuous integration or deployment use cases in the quickest, most consistent and reliable way possible.

How Does It Work:

Here at ElasticBox we trigger tests, builds, and deployments on pull requests and merges, which is a very common use case. These actions start workflows that result in Jenkins triggering tests, builds and the deploying of instances. For more information on our internal continuous deployment scenario, take a look at this slideshare deck we presented earlier this year.

The first thing to understand is that we manage Jenkins as an instance that we deploy into our private datacenter through ElasticBox. We have a single Box that installs Jenkins Server on the provider of your choice. This makes it convenient to scale when multiple builds are slowing things down or to simply redeploy if something breaks all together.

The second piece of the puzzle are the required plugins, like the recently releasedElasticBox Jenkins Plugin, which, along with the Github Jenkins Plugin, makes it entirely possible to set up a continuous integration or deployment server without writing any code.

Setting Up Jenkins and ElasticBox:

First you have to set up ElasticBox as a cloud in the configuration page of Jenkins. You simply enter the ElasticBox url (http://elasticbox.com), a maximum number of instances, retention time, in minutes, and your ElasticBox username and password. Once all this information is entered correctly, you can go ahead and then test the connection.

ElasticBox Jenkins Example:

In this basic example, we’ll show you how we test changes to our website. When a branch is submitted as a pull request, we deploy a new staging instance of our Website Box. We validate the changes on an instance that is a replica of our production environment before merging the pull request into our master branch. When the code is merged, we then trigger a Jenkins job that reinstalls our production instances with the latest code changes (not shown in this demo). Now, our workflow for website changes is almost fully automated, minus the pull request and merge!

1) The first step is to create a new Jenkins item where we select “Build a free-style software project”

2) Once we’ve created a new item, we add the Github project URL and check the ‘this build is parameterized’ box. While this step isn’t required, it allows you to trigger a build manually and execute code from a specific branch.

3) In the source code management section, we specify the source code management tool, the ‘Repository URL’, provide credentials in the form of Username / Password, and for the Branch Specifier, use ‘${sha1}’ which represents the commit url of the branch submitted in the pull request. The Jenkins Git plugin is extremely complex and we highly recommend reading the Wiki on the plugin.

In the Build Triggers section, you’ll want to select “Github Pull Request Builder” Which is what will be triggering the Jenkins build and instance deployment in ElasticBox.

4) The final step is configuring the Build steps. With the ElasticBox plugin, you’ll have access to all your workspaces, boxes and deployment profiles. For more information on each of these features, please check out the links to our documentation. Specify the Workspace, the Box and the Deployment profile you would like to use. In our case, we specify our Production Workspace, our Website Box and our Staging Deployment Profile.

Conclusion:

Once all these configurations are set, you can test changes on new staging instances just by submitting pull requests. Even though this is a basic example, by continuous integration standards, you can see how the ElasticBox Jenkins plugin can seriously reduce the amount of configurations required to setup a continuous integration server.

With ElasticBox abstracting the infrastructure provisioning and allowing you to reuse existing application or infrastructure components, it’s much easier to scale the process in a repeatable and dependable way. We’ll be following up with blog posts that provide more technical examples on how to use Jenkins and ElasticBox for integrated builds, testing and continuous delivery models.

July 15, 2014
Deploying a MongoDB Cluster with ElasticBox

Our founders, back in the day, were part of the .NET framework team at Microsoft. So it’s no surprise that their first version of ElasticBox was written on C#, .NET framework and SQL Server.

Fast forward to 2012. As the ElasticBox vision crystallized, we wanted a robust, scalable architecture to match our vision. So we reevaluated all the existing technologies.

Among other things, we decided to move from a relational database model to a document based approach. The main reason was because we wanted to store hierarchical data and be able to version it.

Today, to accommodate our data needs, ElasticBox relies on MongoDB clusters, which we run on three separate providers–two public, one private. Thanks to ElasticBox, we can now deploy our MongoDB cluster in a matter of minutes.

The Basics

What is MongoDB? MongoDB is an open-source document store database.

Why would you want to cluster MongoDB? To provide redundancy and high-availability for production deployments.

How We Use MongoDB

In our case, MongoDB is using a replica set model. A replica set is a group of MongoDB instances that host the same data set. One MongoDB, the primary, receives all write operations. All other instances, secondaries, apply operations from the primary so that they have the same data set.

The primary accepts all write operations from clients. Replica sets can have only one primary. Because only one member can accept write operations, replica sets provide strict consistency.

Deploying MongoDB through ElasticBox

ElasticBox provides a default MongoDB box that’s available to the public. It deploys a standalone instance of the MongoDB server. Our MongoDB production box very closely resembles the default box, minus a few configuration settings required for our production scenario.

While the default MongoDB box configuration is suited for development purposes, it needs to be configured further to use in production. To configure the MongoDB default box for production use, follow the steps below:

Step 1. Create a new box called MongoDB Replica. Add a box variable for the MongoDB Server (this is to use the default box settings).

Step 2. Add a text variable REPLICA_SET to hold the replica set name. This value must be shared between all the members of the cluster.

Step 3. Expand the MongoDB Server box. Edit its file variable COMMON_YAML and paste the following:

mongodb::replica_set: {{ REPLICA_SET }}

mongodb::key_file: /etc/mongod.key

This file will be used by hiera to pass arguments to puppet automatically without modifying the existing puppet default.pp file.

Step 4. Add a binding named primary of type MongoDB Replica (the box you just created).

Step 5. Add an install script. Get the script here.

Step 6. Add a start script. Get the script here.

When you deploy the box without a binding, it creates a cluster for you with an initial configuration. On the other hand, if you bind to an existing instance, it adds the instance to the existing cluster!

There, you have it, a MongoDB cluster in a few easy steps!

July 11, 2014
Flexible and Scalable Multi-tier Application Deployments

This is a guest post from Alberto Arias Maestro, CTO and co-founder of ElasticBox

One of the biggest problems facing software engineers since the dawn of the multi-tier application is, well, how to make it multi-tier.

It’s more than just having several supporting applications – it is about connecting the layers correctly and allowing them to communicate with each other. All this is to create a scalable and responsive deployment that can be easily updated and adapted to changing business needs.

It is about ensuring that your infrastructure can co-ordinate the order in which your application tiers are spun up, even when the apps themselves have not been designed to perform these critical dependency checks.

What Are You Looking For?

Intelligently handling your solution’s dependencies is an inherent problem in multi-tier deployments – at whatever scale you are operating. For instance,

As a developer working on a project:

“I want to ensure that my database is up and running before my web-server is deployed.”

As the CTO of a rapidly growing startup:

“I want to bootstrap on basic AWS services (such as managed cache, load balancing, and managed databases), but as the product evolves, I want to give myself the freedom to evolve the services I connect to and consume – experiencing as little downtime as possible.”

As the Head of IT Operations of a large-scale enterprise:

“I need to reign in a heterogenous environment where I have hundreds of highly inter-dependent applications that range from just-released to legacy and everything in between.”

At ElasticBox, we’ve found a way to solve this problem using the concept of Boxes, and now, Bindings, a feature we are launching today.

‘Bindings’

ElasticBox lets you build and deploy complex, multi-tier applications using boxes as basic building blocks. Specifying a ‘binding’ from one box to another is a way to declare that there exists a dependency between the boxes.

It is a construct that allows the different tiers of a multi-tier applications to communicate with each other and exchange data.

Bindings enable users to introduce not just structure, but also flexibility to their multi-tier deployments. For example,

Our developer’s Apache Web-server Box can use a required binding to specify, that to launch correctly it needs a connection to a running MySQL DB instance.

Our startup CTO, who has been trying to make a choice between AWS SQS (Simple Queue Service) and RabbitMQ, can simply switch bindings, with his business application experiencing little to no downtime.

Our Head of IT Operations, can chain his applications so that they come up in the right order. He has the flexibility to modify this chain at anytime – to accommodate new applications or possibly remove old ones, without having to bring the entire system down. If you are thinking about deadlocks – don’t worry, bindings are designed to handle these.

On-Prem? Cloud?

For enterprises, bindings can be a very powerful way to evaluate different cloud strategies for business solution with minimal disruption to the existing deployment. Bindings allow switching between an AWS RDS instance and an on-prem database all with a single click.

As you can see from some of the use-cases above, ‘bindings’ is a powerful concept that solves a very prevalent problem in advanced deployments and provides users a simple way to evaluate different strategies. To learn more about how to incorporate bindings into your deployment, please visit our documentation. For a free consultation about how bindings can benefit your particular use-case, please email us at support@elasticbox.com.

July 1, 2014