Tag: containers

IBM Brings Cloud-native Environment to Private Clouds
New IBM Cloud Private software – based on containers, microservices and APIs –bridges Public Cloud and traditional IT environments

ARMONK, N.Y. – 01 Nov 2017: IBM (NYSE: IBM) today announced a new IBM Cloud Private software platform to help companies unlock billions of dollars in technology investment in core data and applications and extend cloud-native tools across public and private clouds.

IBM Cloud Private software is designed to enable companies to create on-premises cloud capabilities similar to public clouds to accelerate app development. The new platform is built on the open source Kubernetes-based container architecture and supports both Docker containers and Cloud Foundry. This facilitates integration and portability of workloads as they evolve to any cloud environment, including the public IBM Cloud.IBM also announced new container-optimized versions of core enterprise software, such as IBM WebSphere Liberty, Db2 and MQ – widely used to run and help secure the world’s most business-critical applications and data. This makes it easier to share data and evolve applications as needed across IBM Cloud private and public clouds and other cloud environments with a consistent experience.
An airline, for example, could use IBM Cloud Private to bring a core application that tracks frequent flyer miles into a private cloud environment and connect it to a new mobile app in the public cloud. A financial services firm could use it to keep customer data in-house as it works to meet its security and regulatory requirements while taking advantage of new analytic tools and machine learning in the public cloud to quickly identify investment trends and opportunities.

Enterprises to Spend More Than $50 Billion a Year on Private Clouds

While public cloud adoption continues to grow at a rapid pace, organizations, especially in regulated industries such as finance and health care, are continuing to leverage private clouds as part of their journey to public cloud environments to quickly launch and update applications. IBM estimates companies will spend more than $50 billion globally starting in 2017 to create and evolve private clouds with growth rates of 15 to 20 percent a year through 2020, according to IBM market projections.

“Innovation and adoption of public cloud services has been constrained by the challenge of transitioning complex enterprise systems and applications into a true cloud-native environment,” said Arvind Krishna, Senior Vice President for IBM Hybrid Cloud and Director of IBM Research. “IBM Cloud Private brings rapid application development and modernization to existing IT infrastructure and positions it to be combined with the services and experience of a public cloud platform.”

Hertz, Ilmarinen Transforming IT with Private Cloud

The Hertz Corporation, one of the top car rental companies in the world, believes both public and private cloud is critical to achieving its technology objectives.

“Private cloud is a must for many enterprises such as ours working to reduce or eliminate their dependence on internal data centers,” said Tyler Best, Hertz Chief Information Officer. “A strategy consisting of public, private and hybrid cloud are essential for large enterprises to effectively make the transition from legacy systems to cloud. Hertz is an early adopter of both public and private IBM cloud and we could not accomplish our technology goals without private cloud as part of our overall cloud portfolio.”

Ilmarinen, a Finland firm responsible for the pensions covering approximately 900,000 Finns, has started a pilot project with IBM Cloud Private and plans to be in production in early 2018. “Ilmarinen is excited about the capabilities in IBM Cloud Private as a flexible, compliant cloud-native platform that it can deploy and manage behind our firewall for creating new applications and modernizing traditional workloads,” said Jani Itkonen, Production Manager, Ilmarinen.

IBM Cloud Private Provides Flexible Platform with Support for Enterprise Developers

Innovations used in IBM Cloud Private are designed to simplify cloud management, accelerate app develop and maintain security across cloud environments. Key features include:

Cloud Management Automation – Streamlined management across cloud environments to help launch, monitor and manage services and help ensure consistent security protocols.

Security & Data Encryption – Security Vulnerability Advisor to scan containers across the cloud to surface potentially serious issues and weaknesses; Ability to encrypt all data in flight and provide for strict access control by users within a cluster.

Core Cloud Platform – Includes a container engine, Kubernetes orchestration, Cloud Foundry and essential management tools surrounding developer runtimes, which are fully integrated and automated.

Infrastructure Choice – Compatible with systems from leading manufacturers including Cisco, Dell EMC, Intel, Lenovo and NetApp, as well as IBM Systems, including IBM Power Systems, IBM Z and IBM Hyperconverged Systems powered by Nutanix software.

Data and Analytics – Integrations with database services, such as IBM Db2, PostgreSQL and MongoDB, to leverage data and gain business insights.

Application Support & DevOps Tools – Containerized versions of software and development frameworks, including IBM WebSphere Liberty, Open Liberty, MQ and Microservice Builder, as part of software bundles; Optimized management and DevOps tools, including APM, Netcool and UrbanCode, which can be added for a separate fee.

IBM also offers cloud deployment, management, brokerage and storage services to help clients manage multiple clouds and can help clients with migration of applications to IBM Cloud Private. More details on Cloud Private can be found in the product overview.

IBM Cloud Private Reinforces IBM Commitment to Open Source

IBM Cloud Private is another significant step in IBM’s support for organizations to create, deploy and manage apps in the cloud environment of their choice. IBM and Google recently announced an effort to create and open source the Grafeas project to help developers with security of containers. Last month, IBM released Open Liberty, a new lightweight, open-source version of its WebSphere Liberty Java application server, to make it easier for developers to get apps up and running in any cloud environment.

IBM also is a founding member of the Cloud Native Computing Foundation, which hosts Kubernetes and Cloud Private is in compliance with its interoperability conformance tests. To get started with IBM Developer Journeys as they relate to Kubernetes, please visit: https://developer.ibm.com/code/journey/

For more about IBM Cloud Private, visit here.

For more about IBM Cloud, visit here.
Contact(s) information

Joe Guy Collier
IBM Media Relations
1 (248) 990-4707
jgcollie@us.ibm.com
November 10, 2017
The Need for Speed Demands New-Era APM
Applications have become the primary conduit by which customers experience a brand, making the application experience critical to business success. As such, continuous delivery and highly available systems that deliver optimal end user experiences at lightning speed have become top priorities for organizations. Modern IT is all about speed, efficiency and customer experience. Simply put, companies that develop new applications and services the fastest will win.

That’s where containers and microservices come in. Containerized microservice applications enable organizations of all sizes to approach the ultimate goal of continuous integration and continuous delivery. This accelerates everything within the application life cycle, from initial development to component updates and new functionality.

To match the development speed that microservices and containers enable, Ops teams need tools that can also operate hyper-fast, providing an immediate understanding of the impact of changes made, real-time analysis of vast amounts of data across applications and technologies, visibility across a range of languages and infrastructures, and knowledge of how the application is performing for end users.

In fact, we are at a critical inflection point in application performance management (APM), where a generational shift is needed to keep pace with the new hyper-dynamic application architectures consisting of microservices, containers and hybrid cloud environments. To fully understand the impact of this shift, we can examine the genealogy of APM.

APM Evolution

The first generation of APM, established at the end of the last century, collected statistics and metrics out of system and application components, providing a first generation of user-defined transaction profiling by using instrumentation technology. APM 1.0 required a lot of effort to implement and maintain, but helped to monitor and analyze the performance of newly developed 3-tier applications.

The second generation of APM was driven by more distributed, service-oriented architectures (SOA) and the rise of more elastic, virtualized or cloud-based infrastructure. APM 2.0 added application topology discovery and could analyze distributed applications. User experience monitoring of web and mobile apps quickly became part of APM 2.0 as more businesses depended on the service quality of their applications. With a primary focus on application monitoring, not management, APM 2.0 required an even broader set of subject matter experts to configure and interpret monitoring data from dashboards and reports. Similar to APM 1.0, the second generation still required way too much human analysis and setup to fully understand application problems and derive actions.

The continuous change in today’s modern dynamic applications makes it difficult, if not impossible, for those older tools to keep up. Throw in additional architectural challenges such as hybrid clouds, containers and orchestration (such as Kubernetes and Mesosphere) and the picture gets murkier, all in the name of high-speed delivery. Meanwhile, the microservice architecture and development strategy spawns wide technology diversity—each engineer can choose the programing language, middleware and databases best suited to code their individual microservice. Scale, complexity and constant change is the new reality for today’s dynamic applications.

Wanted: A New Breed of APM

In this environment, traditional IT monitoring tools won’t work. These new applications use a vastly larger set of technologies, must scale dynamically and are in a constant state of flux.

In this effort to engineer for speed and efficiency, complexity and scalability have increased exponentially, requiring continuous monitoring and more visibility than ever before. As organizations embrace agile methodologies for development and containerized microservices for deployment, developers have shifted their thinking in technology choices. Developers now continually select the most optimal technology for their individual goals regardless of DevOps’s ability to configure, deploy and manage the specific technology.

With several teams across the enterprise constantly publishing new software releases, understanding the impact of these new releases and changes to the overall environment has become a significant challenge. Today there is no single source of truth assisting communication between various organizations, resulting in what seems a wild west–like environment for IT. With so many groups and technologies in play, everybody seems to be speaking a different language, especially when it comes to delivering high-quality applications. In this environment, a new breed of APM is needed that can provide a “Truth Machine”—complete with deep visibility across all application components in the technology stack.

This new breed of APM must be automatic and dynamic to deal with the ever-changing world of applications running in containerized microservices. To deliver these critical capabilities, the tool must provide real-time visibility into dynamic, containerized, microservice applications, regardless of language or infrastructure. New thinking is required to meet the challenges of these complex, large and dynamic application environments; a new monitoring and management approach that can handle dynamism by:
- Delivering real-time analysis of vast amounts of data across all applications and technologies.
- Conducting continuous discovery and automatically mapping every component, host and container, (hosted on-prem, private cloud, public cloud), to the exact architecture of complete technology stack.
- Visualizing the application architecture and delivering actionable suggestions that assure the operational health of systems and applications.
- Providing real-time accuracy to visualize, structure and identify problems immediately.
- Monitoring the complete set of diverse technologies that run in microservice environments.
- Operating in a heterogeneous environment and automating deployment across nearly all programming languages.
Artificial Intelligence for Next-Generation APM

DevOps and IT organizations are struggling to match the speed of agile development, and this is creating exponentially more challenges around new application architectures. The only constant in these modern application environments is change. Whether change is due to microservice architectures, container orchestration or simply more agile delivery execution to effectively manage microservice applications, DevOps needs up-to-date visibility, a precise picture of the application’s structure and an immediate understanding of its health. To accomplish that, DevOps must be able to apply artificial intelligence to the act of monitoring modern dynamic applications.

Rather than spending time configuring and reconfiguring monitoring tools, organizations must automate management steps with as little human setup and configuration as possible. To have complete control and visibility of their environments, regardless of language and infrastructure, organizations must use a modern application management solution that is automatic and AI-powered to deal with the ever-changing world of dynamic containerized microservice applications.

About the Author / Pete Abrams

Pete Abrams is a co-founder and the chief operating officer at Instana, the leader in application performance management for containerized microservices applications. Formerly vice president of customer centric innovation at AppDynamics, Pete has spent more than two decades in the technology industry focused on introducing new products to market and ensuring customer feedback and desired capabilities are incorporated into future product innovation. Connect with him on LinkedIn.
October 27, 2017
Finishing the Pipeline: Monitoring, Tools, Automation and …

It is true that DevOps has grown to be the beast in the room. I’ll co-opt the “cloudwashing” term to call it “DevOps-washing” because it seems every tool you’ve ever used to develop and deploy applications now has a DevOps angle to it. For example, the Jenkins update center is kind of overwhelming. While I can’t be certain, there’s a chance that our new coffee maker has a plugin for Jenkins.

But it’s also true that DevOps really is coming of age. The success of Jenkins is largely based on CI/CD, and the tools built both up and downstream from it are folding in more and more of the DevOps process, making life easier—if not exactly easy—for IT.

Even with platforms, the choices are growing and the automation is, too. Whether one of the major cloud vendors, where deployment can be fully automated or hyperconverged infrastructure and where deployment can be fully automated, we’re moving toward a future of not caring about physical infrastructure but rather, “Can I get a connection to it?” Of course, it’s not that simple, but running Docker or Kubernetes on hyperconverged infrastructure means we have on-demand deployment, and the cloud does much the same.

Tooling for monitoring is also getting better, whether you are monitoring user experience or a particular API’s CPU usage. For the most part, we are not yet to the point where automation can deal with the majority of alerts from monitoring, but that boundary is being pushed daily. We certainly can tool our systems—both the infrastructure and the app sides—to offer us more data about application health than we’ve been able to get in the past. From physical hardware all the way up through containers we have a wealth of options for monitoring, and application monitoring is getting even more attention than infrastructure monitoring.

Heading Down the Trail: Monitoring and More

Which leaves one wondering where this is bound. Let’s assume that the areas that are a bit behind—like over-arching network configuration from ARA tools—catch up. They will, and that’s pretty obvious, so I’m trying to remove the short-term roadblocks to talk about what we might be planning for and facing moving forward.

If we can assume a place to deploy that has reasonably unlimited capacity—such as the cloud, where cost is the limit, or hyper-converged, where capacity is a limit but expandable in chunks—and we can assume that ARA is running the Dev/Test side, then pairing it to a preferred server configuration (be that VM or container, depending on app, group, company, etc), and pushing out the deploy, then we’re left with the monitoring piece, and automating the responses to monitoring issues. That is growing, too.

Right now, the standard in the DevOps world for a server that is malfunctioning in some way is to kill it and start a new instance. This often masks real-world problems, and if it appears to “fix” the problem, reduces the priority of resolving what went wrong. This is, in a nutshell, the real error for Cloudbleed. They were seeing servers spew errors, when one appeared to core, they simply started another. It allowed a simple programming error to persist over months and become a security issue.

So the next great leap will be to find a quick way for monitoring to determine exactly where problems are occurring, and force a developer to fix it. There are tools that can auto-generate bug reports, but they’re rudimentary at this point in time, and we need them to be more advanced. There will also always be the trade-off of how much in resources you use for monitoring when the point of the app has nothing to do with monitoring. And like so many things, the hidden cost of keeping that monitoring up to date will be a technical debt—one of many that the DevOps community seems slow to acknowledge.

There Always Seems to be More

On the “time we waste” side, the fight over internal/external needs to end. The fact is that internal cloud isn’t going anywhere anytime soon, and if you’re running in containers, you don’t have to care. My company is currently looking at a tool that can run hundreds of containers in a very (physically) small space. We can deploy to the cloud or deploy there, as long as we have container templates. It doesn’t matter what the business need forces us to use; containers really will run pretty much everywhere. It does no one any good, harping about internal/external. A deployed app is a deployed app.

Now, talking about the differences in networking between internal/cloud, external/cloud, and container manager (regardless of where it sits): That is a discussion worth having.

— Don Macvittie

October 16, 2017
With Distelli Technology, Puppet Expands DevOps Portfolio

At its PuppetConf 2017 conference, Puppet made it clear its ambitions now extend well beyond its namesake IT automation framework for IT infrastructure. On the heels of acquiring Distelli in September, the company announced it is adding several products to its portfolio based on technologies developed by Distelli.

The new offerings include Puppet Pipelines for Apps, to automate commits; Puppet Pipelines for Containers, to automate the building of Build Docker Images from a source repository and then deploying them on a Kubernetes cluster; and Puppet Container Registry, to host Docker images and unify views of them regardless of whether the repositories are local or remote.

At the same time, the Blueshift platform developed to automate deployments of container platforms such as Kubernetes, Docker Swam and Mesos is also being enhanced. The Kubernetes module now installs the container cluster in a more secure way that is also highly available. The Docker module adds support for Puppet Code Manager, Docker Swarm mode and Docker Secrets. A Kream tool makes it easier to deploy Kubernetes and Helm, a package manager for Kubernetes, at the same time. Helm also now can manage applications deployed on Kubernetes.

In addition, the company announced Puppet Discovery, a tool that makes it simpler to identify infrastructure distributed across a hybrid computing environment, and Puppet Tasks, which employs new open-source agentless software dubbed Puppet Bolt to automate entry-level IT tasks. Bolt is capable of running existing scripts written in any framework and executing them on any platform that Puppet supports.

The company is also adding a Task Manager to Puppet Enterprise that makes it easier to manage ad hoc tasks and executing them across hundreds of Puppet nodes. The latest release of Puppet Enterprise includes tools for inspecting packages deployed on and a tool to simplify configurations of those nodes.

Puppet has also expanded its alliance with Splunk to tighten integration between the two IT operations’ platforms, and formed new alliances with Baracuda Networks, CloudPassage, Conjur/CyberArk, Cumulus Networks, Electric Cloud, F5, Intelliment Security, Onyx Point, Praecipio Consulting and Sensu. In addition, the Puppet module for the Microsoft Azure cloud has been updated to provide access to more disk storage, expanded networking support and support for Azure tags.

Tim Zonca, vice president of marketing and business development, says with these products Puppet should no longer be considered a one-product company. It is now committed to developing a full range of DevOps management capabilities. That’s significant, notes Zonca, because today there is too much tension between IT operations teams and developers caused by silos of automation across various DevOps processes. As part of that effort, the company is also committing to start to synchronize the release cycles of the various elements of its portfolio.

As DevOps continues to mature, the line between how infrastructure, applications and security are managed continues to blur. Many IT organizations prefer to acquire a set of DevOps tools from one vendor because it simplifies purchasing and support. Over time, those vendors combine those tools into an integrated suite that aims to unify the management of infrastructures, applications and security. Most vendors today are a long way from unifying those functions, but a wave of consolidation across the DevOps sector suggests the race is on to provide a unified DevOp stack capable of being employed across multiple platforms.

The percentage of IT organizations that make extensive use of IT automation frameworks such as Puppet is relatively small. Much of the resistance to IT automation frameworks can be attributed now to simple inertia. But as the size and scope of IT environments exceed the capabilities of an internal IT team to manage manually, it’s now only a matter of time before reliance on IT automation increases.

Puppet and other vendors also have infused their platforms with machine learning algorithms to expand the scope of what can be automated within an IT environment. In general, Zonca says reliance on those capabilities tends to increase in time once the DevOps teams gains confidence in the algorithms’ abilities to make the right decision. Until such time, Zonca says most IT professionals employ those algorithms to either present them with a series of suggested options based on the most recent events that have occurred. Given the nearly infinite number of possibilities that can be inferred from those events, most DevOps teams already are going to need all the machine learning algorithm help they can get.

— Mike Vizard

October 12, 2017
Cloud Foundry Study Confirms Accelerated App Development

When it comes to accelerating the development of applications, having a standard set of runtimes can make all the difference. A new study of 753 IT professionals conducted by ClearPath Strategies on behalf of the Cloud Foundry Foundation (CFF) finds a marked decrease in the time it takes organizations using the Cloud Foundry platform-as-a-service (PaaS) environment to develop code—just 18 percent need three months or longer, compared with 51 percent of respondents that don’t use the Cloud Foundry Application Runtime. And, according to the study, the 46 percent of responding companies that use the Cloud Foundry Application Runtime see application development times of less than a week, compared with only 16 percent of non-users.

Abby Kearns, executive director for CFF, says use of Cloud Foundry Application Runtime has expanded considerably in the last year. Nearly half (45 percent) of the respondent have been using the PaaS environment in the last year. The survey also finds that more than half (51 percent) now have more than 10 developers employing Cloud Foundry Application Runtime, while 28 percent now have more than 50 developers.

Kearns says those results are noteworthy because it shows that CF is now being used for a lot more than simple proof-of-concept projects. In fact, Kearns notes use of CF to develop a broader range of cloud-native application is increasing. The study finds half of Cloud Foundry users currently use containers, with another 35 percent evaluating or deploying containers.

Kearns says there’s a high correlation between implementations of CF and adoption of DevOps processes. He notes that often organizations decide to implement CF to force the DevOps issue—a PaaS environment the size and scale of CF requires every process and function within an IT environment to participate in what has become a large-scale continuous application/continuous integration (CI/CD) platform operating at scale. Confronted with that requirement, Kearns says it’s not too long before the IT culture adjusts to that new reality by implementing a new organizational structure.

CF is not the only PaaS game in town. But via CF distributions from Pivotal, IBM, SAP and others, use of CF has steadily grown, especially among larger enterprises that have the skills required to implement CF at scale. The survey finds that enterprise IT organizations with more than $1 billion in revenue account about half (49 percent) of all CF users. As deployments of the various elements of a PaaS environment becomes more automated, adoption of CF in small-to-medium enterprises should increase.

In the meantime, IT organizations need think through the amount of time and money developers are wasting using alternative approaches. Not every IT organization appreciates the opinionated workflows that are baked into a PaaS environment. But given the length of time it takes to build applications using other platforms—as well the total cost of maintaining those environments—building and deploying applications at a higher level of PaaS abstraction is difficult to ignore. In fact, managing IT at a higher level of abstraction may eventually prove to be the difference between thriving and merely surviving.

— Mike Vizard

October 11, 2017
Platform9 Brings Workflows to Serverless Computing

Serverless computing frameworks are widely regarded as the next big thing in enterprise IT because they are based on an event-driven architecture that makes servers, storage and networking resources dynamically available as needed using a programming construct known as functions. But DevOps teams are challenged in finding a way to orchestrate all those functions in concert with one another.

To enable the orchestration of those functions, Platform9 has added support for Fission Workflows to its instance of a serverless computing framework. Platform9 CEO Sirish Raghuram says Fission Wokflows will make it possible for IT organizations to craft complex applications made up of thousands of functions.

On the one hand, serverless computing frameworks promise to make DevOps easier by eliminating the need to provision infrastructure every time a workload exceeds whatever amount of capacity that has been assigned to it. On the other hand, attempting to manage what could easily become thousands of functions in an IT environment is likely to make DevOps even more challenging than it is today. While not a new concept, Raghuram says Fission Workflows provides IT organizations with a familiar construct for managing functions.

Benefits of Fission Workflows include an ability to reuse functions as well as being able to upgrade entire related sets of functions all at once. There are also fault-tolerant capabilities that make possible to recover and restart a workflow in the event of an infrastructure failure.

Platform9 is specifically making the case for a serverless computing framework based on Kubernetes clusters that can be deployed on-premises or in the cloud. Platform9 also provides support for containers on an instance of Kubernetes, and Raguham says the company expects that IT organizations will mix and match containers and serverless computing frameworks as they see fit. For example, a container might invoke a function via an API call to run a stateless process while Docker containers are employed for all the stateful processes that require access to primary storage, he says.

Raguham notes the trouble with serverless computing frameworks put forward by cloud service providers such as Amazon Web Services (AWS) is that they lock an IT organization into a specific platform instead of taking advantage of a portable cluster technology such as Kubernetes, which is backed by the Cloud Native Computing Foundation (CNCF). Kubernetes also provides the added benefit of a common platform for both containers and serverless computing, he says.

It’s still early days for the use of serverless computing frameworks in the enterprise. Nevertheless, it’s clear they will need to be managed from a DevOps perspective alongside cloud-native applications running on containers and platform-as-a-service (PaaS) environments and legacy applications running on virtual machines and bare-metal servers. Given the fact that none of these architectures is likely to supersede any of the others anytime soon, IT organizations would be well-advised to start crafting a DevOps strategy now on the assumption that the overall IT environment is about to become even more complex.

— Mike Vizard

October 9, 2017

Continuous Deployment and Monitoring of Microservices

Continuous deployment and continuous monitoring have grown up in DevOps, and microservices have grown up as a delivery platform. While all are relatively stable concepts at this point, there are still a fair number of practitioners that are struggling with the merging of these concepts to create a holistic environment that allows for deep understanding and easy deployments.

So I thought we could take a look at what’s out there, how it’s going, and even mention a resource or two for you.

Continuous Deployment

While continuous delivery (CD) and continuous integration (CI) are well in hand, continuous deployment has lagged a bit behind. There are reasons—both good and bad—why this is true, but true it is. Continuous deployment is the act of actually deploying the results of the CI/CD cycle. The whole point of continuous deployment is to speed the release of new features to market by releasing them as soon as they pass the CI/CD muster, or soon after, or for some industries, on demand. Continuous delivery is a little different from CI/CD in that the rest of the organization—sales, support, marketing, accounting, etc.—have to be ready for the release, so in the slowest case, the point is that IT is not the hold-up to releasing new features under the continuous deployment model.

Microservices

Simply put, microservices are decomposition of an application into smaller parts that can be run independently. There are a lot of different ideas for what “smaller parts” means, with some shops breaking their application down to the functional level similar to serverless functions, so an API named calculateSalesTax would do that and only that, given location and dollar information, while others prefer a more broad definition that includes entire groupings of APIs. In this scenario, one company split all database work into a single microservice that was scalable. Either way, you will have more instances (normally containers, but sometimes VMs or even physical servers) running to build the application than historically would be the case.

Why Containers

As mentioned above, normally a microservices architecture is built on containers. The portability of a container definition is one of the major reasons why companies lean this direction. The ability to say, “This container can run on our hardware, on our private cloud, on a public cloud, in a public cloud VPC or on a VM,” is a way to ensure that you can target the system to the environment most suited to the organization’s needs.

Containers are also far lighter than VMs, meaning faster spin-up times, and an easier time expanding the microservices architecture to meet changing user needs.

Container Management

Speaking of containers, container management systems are used to make certain that containers are alive and responding. In advanced cases, they can be made to check that a container is doing what it is supposed to do also. While I am a fan of Apache Mesos, the world seems to have gone for Kubernetes, so we’ll focus on it, without further reference to the others out there.

On top of making certain the correct number of a given container are running, and taking steps to spin up new ones if one disappears, Kubernetes does things such as manage secrets for containers, perform load balancing between instances in a group (these groups are generally called a service in container management speak), scaling up/down of instances as directed, and provide rolling updates to services so the service is not down while being updated.

This makes a container management tool an important part of any continuous delivery and continuous monitoring effort.

Taken together

That gives us tools including Jenkins for CI and CD, and often for release management, though other tools specialize in the release portion, too. Combine Jenkins with an infrastructure-as-code solution such as Teraform or any of the others we’ve discussed in the past. Add in a tool such as Prometheus for gathering monitoring data, and a tool such as Grafana for visualizing the data, and you’re on the road to continuous delivery and continuous monitoring.

Of course, it isn’t that simple, I’m overviewing here. While researching this blog, the people at Infostretch shared some information with me, and they too would see this as the 50,000-foot view. But that’s where we start. For a more complete picture, here is a simple diagram of continuous deployment that Infostretch shared with me:

And as long as we’re discussing the information that Infostretch shared with me, let’s also look at a sample of what the company monitors. You’ll see it isn’t “everything under the sun,” but focused points that give them actionable intelligence. A few application-specific monitoring points from within the application, and this turns into an application performance gold mine.

Classification	Metrics
Cluster Metrics	Overall Cluster Memory Utilization
	Overall Cluster CPU Utilization
	Overall Cluster FS Utilization
	Multi-series graph for individual service CPU Utilization
	Multi-series graph for individual service Memory Utilization
	Multi-series graph for individual service FS Utilization
Application Metrics	Service Request rate
	Service Error rate
	Service latency

There is so much more

There really simply isn’t enough room in a single blog to cover all that is required and possible in continuous delivery and continuous monitoring. I’ll continue to try continuous blogging to help you out, and the other authors here on devopsy.kinsta.cloud have good things to share on the topics also.

— Don Macvittie

September 27, 2017

DevOps Predictions for the Future
Among the few powerful trends we had experienced in the recent times, one is undoubtedly the adoption of DevOps practices. As we are move closer to 2018, it is becoming more obvious that DevOps is becoming mainstream, with many large organizations getting on board with DevOps.

DevOps Within Enterprises

DevOps is becoming the standard way of working for businesses. Adoption of DevOps within the organization is rising on a broader scale, and businesses are trending toward it.

Enterprises of today are well-aware of the trending technology services in modernizing environments. DevOps is growing its recognition due to changes in complexities of today’s software that lies more in the diverse set of operation systems and platforms. It is less about doing things a particular way and more about moving the business forward and turning technology into a strategic advantage.

DevOps builds upon best practices to help drive enterprise performance in the modern digital economy. It offers organizations a new way to deliver value to their customers quickly. An increasing number of businesses are recognizing the power that DevOps can bring for operational efficiency and overall performance.

DevOps Key Trends

As businesses are trending toward DevOps adoption, let’s look at the changing landscape and key trends in DevOps, along with its trajectory.

Some of the DevOps trends include:
- Many organizations will challenge agile and recognize DevOps as the new ALM methodology.
- Many DevOps automation tools are going to roll up.
- Automation and testing would gain their prominence.
- DevOps Engineers are highest paid IT practitioners today.
- Testing and deployment are done more frequently as Ops people are getting their DevOps knowledge in production for testing and automation operations.
DevOps Predictions

Based on the market so far, here are my predictions for the future of DevOps.

Adoption of Microservices and Containers

Several software enterprises are pushing toward microservices infrastructure. Microservices architect an enterprise solution, independently, over a set of services.

Microservices are best-suited for DevOps due to its features including:
- Isolated nature
- Improvised cycle times
- Well-defined interfaces
- Aligned architecture
Containerization also is growing in popularity. Containers are designed to run one process at a time with minimal deployment, serving as an ideal deployment model. According to one study, 50 percent of enterprises are expected to run containers in production by the end of 2018.

If an enterprise has decided to leverage microservices and containers, it is highly essential to incorporate solutions that support microservices and containers along with traditional application model.

Culture, the Top Prerequisite

Culture is the key differentiator for DevOps. Culture and process are the principal paths for the successful DevOps journey. Culture breaks down the organizational silos and encourages integration across all the groups that need to work together to achieve a great business outcome. Implementing DevOps culture makes an organization achieve success easier. It even enables continuous learning environment that will see bigger benefits.

Notion of Shifting Left

DevOps tools and solutions always prefer to shift left. DevOps orchestration tools automate from end to end, making traceability easier. Activities that are performed traditionally, at the later stages of development and release process, are making a shift to the left of the pipeline. The ultimate goal of it is to recognize issues earlier, along with performance and disaster recovery testing.

Downtime

Adopting DevOps holds a lot of benefits. For one, it enables enterprises to ensure uptime and enhance cadence and quality. High-performing IT organizations employing DevOps practices deploy 200 times more frequently than the low-performing enterprises. Similarly, they also enjoy faster recovery from failure and minimize downtime.

Collaboration of Big Data and DevOps

Big Data and DevOps are together taking a step forward toward the design of predictive analytics. DevOps has now become an expert in automating the process and automating configuration.

Consolidation is Critical

Large and complex organizations are focusing on consolidation and standardization of DevOps processes so as to scale DevOps within the enterprise. A shared and self-service DevOps automation enables end-to-end pipelines. Eventually, consolidation allows reusability, visibility, auditability and security.

Supporting Hybrid

“Hybrid-everything” makes DevOps solutions future-ready to support any tool, application and process in the hyper-hybrid state. “Hybrid” state of existence is achieved for infrastructure, architecture, release pipelines and more.

DevOps Monitoring

Until now, the focus has been on pre-production provisioning. However, more attention is being given to the post-deploy feedback loop. With the application of new architectures including microservices, new monitoring technologies and pricing models are effectively introduced.

Expansion of DevOps Across IT

DevOps is maturing in other areas such as database management, security and testing, all with the goal of application release.

Business-centric Agile

The development methodology approach is becoming more critical to the continuous integration and continuous delivery process.

DevOps Skills

People with DevOps skills are in demand in the industry and expected to proliferate as businesses are witnessing great results from DevOps. Organizations incorporating DevOps training practices are extremely high-functioning. If you aim to build your career in DevOps, below are certain notable skills.

Most common DevOps positions include:
- Site reliability engineer
- DevOps engineer
The core skills of DevOps are:
- Coding skills
- Knowledge of automation tools
- Communication skills
Roles of a DevOps Engineer

Many DevOps developers find interest in deployment and network operations. A core set of DevOps skills includes:
- Ability to code and script using various technologies and tools.
- Close relation with Systems and IT operations.
- Strong base of automation tools and its skills.
- Incremental code testing and deployment.
- Strong focus on data management skills and business outcomes.
Final Words…

DevOps skills will only continue to grow and DevOps is here to stay. Dev and Ops teams are increasingly moving to rapid development and reliable, high-performance services delivery.

DevOps is being adopted by more and more companies, particularly smaller enterprises. Many industries today are keen on incorporating DevOps and its key points into their respective departments. It will continue to evolve for the foreseeable future, by incorporating executives and other business units into it. Companies that utilize DevOps effectively will experience higher innovation and quality and easier adoption to other changing conditions.

Adoption of DevOps enables applications and technology solutions. It power businesses with faster and better quality. Investing in DevOps helps organizations better develop and release software.

About the Author / Sirisha Paladhi

Sirisha Paladhi is a content writer at TekSlate. During her career, she has written many articles on technology innovations.

LinkedIn URL: https://www.linkedin.com/in/paladhi-sirisha-74987881/
September 25, 2017
5 C’s That Changed the Future – Webinar Series
CI, CD, Containers, Cloud & Culture

Join us for case studies of innovation and success with continuous integration (CI), continuous delivery (CD), containers, cloud (infrastructure & migration), and culture (the DevOps kind).

If you’re on the same mission – to create sustainable systems that empower your business to deliver innovation faster and predictably – then you’ll gain incredible insights from this series. We’ll share the journeys of some very cool companies, joined by a few of the experts who are making it happen.

Why 5 C’s?

The consistent theme in these successes is DevOps – culture, process and technology. And when we looked at the results that had the most notable impact on the business, they related to implementations of some or all of the 5 C’s. Results such as:

• 10x faster new product delivery.
• Internal & client confidence in iterative development, driven by innovation in quality & feedback.
• Modernized traditional apps by adding self-healing, auto-scaling, and rolling deployment capabilities.
• Blameless culture, enabled by feedback and consistency across dev, test & prod environments.
• Security integrated across the stack and not simply a checkbox.

The series:

• Tuesday, Oct. 3 – CI/CD, Containers & Culture
• Tuesday, Dec. 5 – Cloud Infrastructure & Culture
• Tuesday, May. 15 – Cloud Migration & Containers

View On-Demand Webinar (Oct. 3):
Faster Innovation with Confidence – How CI/CD & Containers Help Arrayent Go Faster

View On-Demand Webinar (Dec. 5):
Scaling Cloud Infrastructure for Millions of Devices

View On-Demand Webinar (May 15)
Cloud Migration: Solving for Growth with Containers & DevOps on AWS

September 20, 2017
Characterizing and Contrasting Container Orchestrators
Admiral Calcote, also known as Lee Calcote (@lcalcote) or the Ginger Geek to his friends, gave a presentation titled, “Characterizing and Contrasting Container Orchestrators,” at the 2016 All Day DevOps conference.

Okay, he isn’t really an admiral—nor does anyone call him that—but he used the title “admiral” to describe what container orchestrators do, relating it to an admiral directing a fleet of container ships. You could also say that they are like the conductor of an orchestra, directing the individuals to work together as a group toward a common goal while each musician is still able to play their own instrument.

Lee is the head of Technology Strategy at SolarWinds, and for his talk, he discussed four open-source container orchestrators: Nomad, Swarm, Kubernetes and Mesos-Marathon.

He emphasized the obvious: There is no one perfect solution. Each organization is different, so for each solution, he looked at:
- Genesis and purpose
- Support and momentum
- Host and service discovery
- Scheduling
- Modularity and extensibility
- Updates and maintenance
- Health monitoring
- Networking and load balancing
- Secrets management
- High availability and scale
Lee noted that while there are many core capabilities, any orchestrator must have cluster management and scheduling.

He then dove deeper into the four solutions. Below are summaries (the full talk is chock full of information and is online here):

Nomad
- Designed for both long-lived and short-lived batch processing workloads
- Cluster manager with declarative job specifications
- Ensures constraints are satisfied and resource utilization is optimized by efficient task packing
- Supports all major OSs and workloads
- Written in Go and with a Unix philosophy
- Host discovery: Gossip protocol – Serf is used; servers advertise full set of Nomad servers to clients; creating federated clusters is simple
- Service discovery: Integrates with Consul
- Scheduling: two distinct phases – feasibility checking and ranking; optimistically concurrent; three scheduler types when creating jobs
- Uses task drivers to execute a task and provide resource isolation, but it does not support pluggable task drivers
- Built for managing multiple clusters/cluster federation
Docker Swarm 1.12
- Simple and easy to set up
- Architecture is not as complex as Kubernetes and Mesos
- Written in Go – lightweight, modular and extensible
- Strong community support
- Host discovery: used in the formation of clusters by the Manager to discover Nodes (hosts); pull model – worker checks in with the Manager
- Service discovery: Embedded DNS and round-robin load balancing
- Scheduler is pluggable and is a combination of strategies and filters/constraints
- Ability to remove “batteries”
- Rolling updates are supported
- Managers may be deployed in a highly available configuration, but does not support multiple failure isolation regions or federation
Kubernetes
- An opinionated framework for building distributed systems
- Written in Go and is lightweight, modular and extensible
- Led by Google, Red Hat and others
- Young – about 2 years old
- Robust documentation and community
- Scheduling is handled by kube-scheduler
- Pluggable architecture and an extensible platform
- Choice of database for service discovery or network driver and container runtime
- Supports rolling back deployments, automating deployments and rolling updating applications
- Inherent load balancing
- Uses Pods, an atomic unit of scheduling. Each pod has its own IP address, no NAT required and intra-pod communication via localhost
Mesos-Marathon
- Mesos is a distributed systems kernel
- Mesos has been around the longest (since 2009)
- Mesos is written in C++
- Marathon is a framework that runs on top of Mesos
- Mesos is used by Twitter, AirBnB, eBay, Apple, Cisco, and Yodle
- Marathon is used by Verizon and Samsung
- Mesos-DNS generates an SRV record for each Mesos task
- Marathon ensures that all dynamically assigned ports are unique
Wrapping up, Lee provided the following overview comparing the different container orchestration solutions.

Lee packed a tremendous amount of information into his talk. If you use containers, his talk is worth your time and is available online here. If you missed any of the other 30-minute long presentations from All Day DevOps, they are easy to find and available free of charge here. Finally, be sure to register you and the rest of your team for the 2017 All Day DevOps conference here. This year’s event will offer 96 practitioner-led sessions (no vendor pitches allowed). It’s all free and online Oct. 24.

— Derek E. Weeks
September 20, 2017