devopsy.kinsta.cloud

Tag: automation

Shift Left Without Fear: The Role of Security in Enabling DevOps
It’s the age of DevOps, and we all want to shift left, which refers to the idea of performing processes earlier in the CI/CD cycle. That includes security checks and audits. By starting security operations earlier in the delivery pipeline, it becomes easier to find problems, and teams have more time to address them before pushing code into production.

The challenge, of course, is building a shift-left security operation that allows you to perform security operations early without slowing down delivery speed. Let’s take a look at how to square that circle.

Defining Shift-Left Security

A popular term in DevOps culture is “shift left.” But what does shift left mean? Shifting left refers to the effort by a DevOps team to implement measures to ensure application quality at an early point in the software development life cycle. In the case of application security, this means implementing measures to ensure security concerns are taken into consideration while the application is being developed, rather than at the end of the process.

The overall goal of shift-left security is to identify and resolve potential vulnerabilities earlier in the development life cycle, when these issues are less expensive to fix. Doing so will help to mitigate the risk potential security concerns pose to both the delivery schedule and your end -users.

DevSecOps: Integrating Security Into the Development Process

Let’s be honest: Faster and more frequent development cycles aren’t going away. Slowing delivery speed to analyze and bulletproof the application at the end of the development process is no longer a viable format for implementing security. DevOps and CI/CD need to be embraced and enabled by the security staff.

For security teams to enable the DevOps process, collaboration is key. Through this collaboration, the process of DevOps transforms into something known as DevSecOps. In DevSecOps, security personnel work closely with the rest of the DevOps organization to help those who create and deliver the application to gain a better understanding of what is needed to address security concerns as early in the development life cycle as possible.

The following tips serve as a jumping-off point for implementing this collaboration:
- Make security the responsibility of the entire DevOps team: Impress upon the development and operations staff that security is now the responsibility of the entire organization. Simply addressing potential vulnerabilities post-development will eat up too much time and increase the risk that something will be missed. Buying into this process is an essential part of implementing successful collaboration between the security team and the rest of the organization. An example of this would be impressing upon developers to code with security in mind.
- Facilitate regular discussions about application security throughout the development process: It is the responsibility of the security folks to keep security considerations at the forefront of a developer’s mind while they develop new features. Use the agile process to your advantage. Backlog meetings can involve security folks to ensure the creation of user stories includes consideration for security concerns. Your daily standup meetings are another place to include security personnel. This will provide security staff with an opportunity to speak up when the work described by the developer requires—or might require—considerations related to security that have yet to be addressed. Doing so will help to ensure the ball is not dropped, allowing for potential issues to be rectified before they even exist.
- Use test automation and continuous integration to your advantage: When it comes to addressing security concerns throughout the development process, continuous testing is your best friend. And, in cases where it makes sense, the DevOps team should integrate automated test scripts for the detection of security and into their continuous integration process. Such tests can include developer-written scripts to test for vulnerabilities in the code base and tools for static code analysis. Another part of your continuous testing strategy could involve the use of tools for infrastructure monitoring.
The data produced by these strategies can then be analyzed by security folks. And from this analysis, recommendations can be made to the DevOps team regarding the path forward to close potential gaps in security before they ever reach production and can be exploited.

Understanding the Role of Security Personnel in the Development Process

Security folks are often not coders, and development personnel are often not security experts. And while it may be frustrating at times, the solution to managing this disconnect is in the collaboration described above. Situations will arise where developers need to develop with various technologies to build a quality product. It’s possible that these technologies will lend themselves to security concerns that require security expertise. The job of security personnel shouldn’t always be to fight back against the use of these technologies. The security team should act as facilitators—take what development and operations staff intend to do and foresee the potential impacts that innovation will have on security—then take these potential impacts and provide insight into what needs to be done to reduce the risks.

Conclusion

In the past, security often was considered as an afterthought in the development of an application and its infrastructure. The movement toward faster development cycles and more frequent application releases has forced development organizations to think about security differently. Nowadays, so much can change in an application in a short time. This has forced DevOps organizations to integrate security into the development pipeline to make security a continuous consideration in the development life cycle. With this integration, the responsibilities of security personnel have shifted. Security teams now need to be enablers of the DevOps culture, empowering DevOps processes with security in mind.

To see how Symantec CWA can help you shift left without fear, sign up for a free trial to see how it can help secure your cloud.

— Scott Fitzpatrick
June 11, 2019
Textbook DevOps vs. Real-World DevOps

DevOps is one of those things—like Marxism or baking macarons—that is easy to understand in theory, but much messier to implement in practice. Real-world DevOps rarely aligns with textbook DevOps or the DevOps practices that organizations should follow in theory.

That is true for a number of reasons. Let’s walk through some of the biggest below.

Your Company Is Not Organized in the Way DevOps Theory Assumes

According to DevOps theory, every company has an IT department and a development department. Doing DevOps boils down to making those departments work well together.

The reality, of course, is that not all companies organize their staff or technical operations in this way. Some don’t silo developers and IT engineers by default. Others (especially smaller companies) don’t have developers or IT staff on hand at all; they outsource that work.

These companies can still benefit from DevOps principles such as collaboration and automation. But they should avoid fixating on DevOps teachings regarding the organization of technical work.

Real DevOps Skills are Hard to Find (And Expensive to Obtain)

“DevOps engineer” is now a trending job title. But few technical training programs have caught up with the demand for DevOps.

Most universities still silo majors and courses in software development away from those that focus on IT. Other types of technical training programs function similarly.

What this means is that it is more difficult to hire a DevOps engineer, or other technical staff with a broad set of development and IT skills, than it may first appear. DevOps engineers exist, but they are usually experienced staff who learned their real-world DevOps skills on the job. They’re more expensive to hire and difficult to poach from their current gigs.

Full Automation Is a Pipe Dream

The DevOps mantra tells us that we should automate everything, from software testing and builds to deployment, production-environment monitoring and incident response.

Automation is great in theory. But the reality is that you can never automate everything. Human engineers will still need to respond to problems occasionally, or to unanticipated issues that arise during testing, deployment and production management.

An important point to emphasize here is that even if you successfully automate virtually all of your processes, the mere fact that human engineers must be available to respond when something does go wrong undercuts the idea that you can rely on automation alone. Humans still have to watch the automated processes, even if they rarely need to intervene. The simple act of watching and being available to respond when necessary is a significant burden.

Agility Conflicts With Simplicity and Reproducibility

In some respects, DevOps is a contradiction. On the one hand, it prioritizes agility and scalability. On the other, it emphasizes simplicity and reproducibility.

What this means in practice is that DevOps teams adopt microservices and containers in the interest of maximizing agility. But microservices and containers are much more complex to manage than monoliths and virtual machines.

I suspect the reality is that most DevOps organizations end up erring toward more complex technologies and architectures. They ultimately find greater value in them than they do in keeping things simple.

Still, the idea that you can be agile and be simple at the same time doesn’t apply well in the real world. (And sure, automation, infrastructure as code and the like can help in simplifying the management of complex agile technologies, but they only go so far.)

Most Organizations Still Aren’t Doing DevOps

If you spend your time surrounded by other people who have adopted DevOps and implemented it successfully, it’s easy to assume that the rest of the world is like you.

The reality, however, is that we’re now a decade into the age of DevOps, and yet a majority of organizations have made only halting steps toward implementing basic DevOps processes. If DevOps were half as easy to implement as it appears to be when you read the latest book about how to do DevOps, many more companies would be doing CI/CD by now.

Conclusion

DevOps delivers real benefits, and it’s important to continue preaching the DevOps gospel to organizations that can benefit from DevOps philosophy and practices.

However, it’s important to balance the conversation about DevOps with the reality: Real-world DevOps is much messier than it often seems in theory. Not many organizations have the configuration or resources to do textbook DevOps.

— Chris Tozzi

May 30, 2019
Using Automation Tools to Drive Collaboration, Innovation

How automation can help business move past the walls of confusion between dev and ops

In the current state of things, businesses are leaning more than ever on the collective intelligence of their tech teams to drive innovation within their respective organizations. These teams are always comprised of developers and operations professionals. With the foray into automation and the desire for companies to do more with less, it has never been more critical that all individuals, regardless of team, come together to do amazing things. Of all of the services available, Chef is a great example of a suite of tools that want to empower teams to do just that: come together and innovate. It is all about bridging gaps, which can range from simply redefining how teams think about each other to more complex things such as knowledge gaps.

The first gap that almost always seems to come up when bringing teams together and driving business value is the duties gap. For too long we have sat on opposite sides of the coin. In operations in particular, too often we see ourselves as the ones that have to deal with the aftermath of unfortunate problems within code. But the fact is, we should all care about our systems, we should all care about our code and we should all be ensuring the longevity of both.

Once you’ve moved past the first hurdle, then comes the gap in knowledge. Scripting has become pretty common as companies press for automation, but in some instances, it becomes necessary to build up existing team members. Using enterprise automation tools such as Chef, entire teams and entire companies can begin caring about a shared thing: the configuration, health and state of their infrastructure and applications. In doing so, those same teams can begin speaking a shared language. Once this knowledge and shared language are in place, developers no longer view the ops team as an “other,” but rather as collaborators. They can each use their various areas of expertise and speak the same language without having actually having to speak the same language.

The third common gap is the process gap. Process can represent all of those past processes that a company is reluctant to let go of, which can hinder the ability of the developers and operations humans when they’re attempting to collaborate. The process gap can become particularly difficult to move past without leadership buy-in; as such, it is critical to involve business decision-makers in this transformation. While nearly the entire industry is beginning to move toward more collaborative environments, the entire leadership team must be onboard with the requirements to make changes happen. With the use of automation tools in particular, though, processes can begin to shift more toward collaboration-friendly policies and you can then make the case for further expansion with proof that it can aid in your transformation.

Empathy is the final gap that must be overcome. The great part about the other gaps is that they work hand in hand to solve each other’s problems. By promoting collaboration, resolving these gaps are also facilitating collaboration. The empathy gap is largely caused by silos of the past that have led to the commonly talked-about “wall of confusion.” Closing the knowledge and duties gap, developers and operations humans can now communicate and understand the same language and processes each is following. This allows the wall to be broken down and operations humans and developers alike can care about both code and stability.

It’s easy to start getting dev and ops teams working together and on a more even playing field once these gaps have been eradicated. This becomes a big driver for innovation and is, in many cases, a true recipe for success within the company. During my career, I’ve both watched and been involved in these transformations. Working with solutions such as Chef has allowed me to cross the bridge myself, and likewise, be able to share with others how that can be scaled out to entire organizations.

— Brittany Woods

May 17, 2019
Technology: Isolation in the Age of Automation

Dependence on technology makes us more independent overall, but it also makes us more isolated. So, what happens if things come to the point where we depend on machines more than we depend on other human beings?

Earlier this month I attended a presentation about machine learning on mobile phones at Google IO 2019. Right now Google is spending a lot of time and money developing machine learning models and capabilities that operate solely within the mobile device. No connection to the internet is required. It’s a big deal.

During the presentation, the company showed a video that featured an illiterate woman in India who uses her phone’s text recognition features to read to her. All she needs to do is point the phone’s camera at a recipe or newspaper and the phone converts the text to speech.

I found the story inspiring as a technologist and as a human being. But it’s not all peaches and cream. At the end of the video, the woman said something that gave me pause. When talking about using the technology to book a train ticket, she said, “I can book it myself. I don’t need anyone’s help.”

There’s no doubt the phone liberated her, very much in the same way that the light bulb liberated millions of other people in times past, bringing human activity forward beyond the setting sun. Dependence on a particular technology can create greater independence overall. However, I do wonder what happens when things evolve to the point where we depend on machines more than we depend on others, when we no longer need “anyone’s help.”

To my thinking, human interdependence is the thread that binds the fabric of a society together. The thousands of little interdependencies that make up the human experience have beneficial side effects. To get along, we learn the basic rules of civility. We learn to say “please” and “thank you.” We learn how to wait in line at the DMV. We pick up after our dog. We open the door for others. It’s all part of the implicit contract that makes living with other people possible.

But, these days it seems we depend more on machines than the kindness of strangers. The fact is, you don’t have to be nice to a machine, even for a self-serving purpose. You can yell at your cell phone. You stick your tongue out at the ATM as it accepts your deposit. Whether you’re nice to it or not doesn’t matter. Your deposit is not going to be processed any faster or slower. Your cell phone isn’t going to disconnect you because you were rude to it.

Take away these simple rules of civility and strange things happen. It’s interesting that road rage is common, yet sidewalk rage is rare. Maybe wrapping ourselves up in a piece of silicon-powered transportation hardware predisposes us to a hostile attitude when interacting with others in the same situation.

As I stated earlier, depending on technology makes us more independent in the big picture. But, it also makes us more isolated. Whereas in the past, talking about the weather with a stranger might be a prelude to another conversation that’s more meaningful, today the stranger next to me usually has his ears plugged up listening to his own interests. And, if I really want to know about the weather, I can ask Alexa.

Modern technology gives me help on demand. I no longer ask a stranger for directions to the train station, I use my phone. It seems as if I need the help of others less during those small, random moments of vulnerability. A machine is but a voice prompt away. And, thus, the thread that binds me to the world at large gets a little bit looser.

Now don’t get me wrong—isolation is nothing new. Five hundred years ago most people never traveled far from where they were born except on very special occasions. Village life is a case study in group isolation. But, while villages were isolated from the outside world, inside the village it was a different story. The social fabric was dense; the threads were tight. Being shunned by the group was a matter of life and death.

Today we’re all over the place, literally. But we move in our own domains in which we end up wishing happy birthdays to people we’ve never met but who somehow have meaning to us by way of LinkedIn, Facebook and the plethora of other social platforms brought to life and sustained by something other than human labor.

Is this such a bad thing? Dunno. I like it that my phone is now my personal stenographer. It helps me think better. But, I’m not sure that Google Assistant will ever be a good best friend. What’s even stranger is that a time might come when I don’t even need one. It will be just me and my mobile device. Then I won’t need anyone’s help, ever.

— Bob Reselman

May 17, 2019
The Role of Bots in DevOps
DevOps has evolved from its infancy into a mainstream focus area for a majority of CIOs. DevOps has become a main focus and has been shaping the world of software and infrastructure engineering and operations for the last few years.

Some of the key DevOps trends include:
- DevOps assembly lines automation.
- Smart, actionable alerts from monitoring tools.
- Monitor and orchestration infrastructure.
Improved Collaboration

With the increase of DevOps adoption, organizations today face many challenges in the end-to-end service delivery life cycle.
- For applications with complex environment configurations, creating, setting up and deploying a new environment is costly, time-consuming and prone to errors.
- Due to manual interventions, moving/promoting code across environments involves risks and may give rise to outages.
- Development teams are looking to maximize change by writing new code or enhancing existing code, while operations teams are looking to minimize change to keep up with key performance indicators (KPIs) and service level agreements (SLAs). These goals are opposing in nature and create a culture of blame between dev and ops teams.
- Organizations are unable to keep dev, test and production in synchronization due to process and tool gaps. A manual process cannot bridge this gap in consistency; thus, outages in production are common.
Various collaboration mechanisms let teams work and learn together to produce better results. The figure below depicts the end-to-end life cycle of DevOps. Collaboration across the phases is essential to the success of your enterprise DevOps transformation.

What are ChatOps and Bots?

ChatOps is a term coined by GitHub to describe “putting tools in the middle of a conversation.” In a typical CI/CD pipeline, there are multiple tools that you interact with that have varied functionalities, such as source code management (SCM), defect management, continuous integration, continuous deployment and others. These are all discrete tools, but there is nothing that bridges them together. This is where ChatOps comes into play.

A bot is a bridge between the collaboration tool and the DevOps tools. The bot receives a request from the user in the form of a chat command through a collaboration tool, analyzes the request and executes a set of commands on the target DevOps tool.

The main components of ChatOps and bots are:

Chat app/collaboration tool: This is the front-end chat system that connects the various stakeholders and enables them to interact between themselves and the systems around them.

The bot: This is the core of ChatOps. A bot is the glue between your collaboration tool and your systems. It facilitates a two-way communication channel where you can fetch relevant information from the systems and act on the information received. A bot fits in between your collaboration tool and the DevOps tools, e.g. Hubot, Lita, ErrBot or AWS Lex.

Target platform/infrastructure: Integration with the target platforms/DevOps tools is a key ingredient to a successful implementation of bots. These are the target tools and platforms on which the bot will execute the required commands. The tools integration allows your bot to communicate with tools and execute actions such as build, deploy, test, provision and scale infrastructure directly from the chat window.

Where Bots Fit in the DevOps Pipeline

Bots can prove to be an accelerator in various stages of the CI/CD pipeline. Below are the details of the integration with various DevOps tools in each phase of the life cycle:
- Proactive planning – JIRA
- Requirements management – JIRA
- Continuous integration – Jenkins
- Continuous deployment – Ansible, Chef
- Continuous monitoring – Nagios, Grafana, Splunk
- Continuous feedback – JIRA
A chatbot plays a major role in bringing the different teams together into a single, persistent chat room to triage and resolve the issue at hand. In addition to just troubleshooting and triaging issues, bots can be used for various other business-as-usual jobs including:
- Cleaning up servers.
- Kicking off build and deployment jobs.
- Rotating server logs.
- On-boarding application on the monitoring tools.
- Collecting metrics from tools such as Nagios or AppDynamics.
Bot Technical Blueprint

To achieve the aforesaid targets, we need to have a supportive bot platform in place. How closely the tools integrate with the bot ensures the quality of the implementation and opens new opportunities of automation.

As the diagram depicts, the chatbot is hosted on a chat tool such as Slack. This can be on the cloud platform or an existing chat/communicator application. The bot is integrated with a key-value pair reference database, which is fed with the responses depending upon the activities or capabilities that have been fed to it. The key could be the name of the application to retrieve data from or any unique ID that would reduce the number of reads. A lookup of the defined key corresponding to the secondary key is performed on the response database and a response is sent back to the chatbot. Depending upon the number of outputs, the queries can be defined and stored in the response database.

DevOps Bots: Use Cases

Below are some examples of use cases for bots. This includes the complete life cycle of DevOps:

Proactive Planning
Create new user stories in sprint planning tools and assign sprints to teams. Update sprints and product backlogs.

Build and Continuous Integration
Execute build jobs and report back statistics of the build job. Perform daily/nightly builds and report on the status of the build to the relevant teams.

Continuous Deployment
Run deployment jobs and perform deployments across environments. Roll back deployments in case of failures, status checks, perform post-deployment actions such as JVM restarts.

Infrastructure Provisioning and Configuration Management
Kick off Chef jobs to provision infrastructure or provision application environments. Enable monitoring and restarting services and servers.

Continuous Monitoring
Provide application statistics or performing daily data lookups and grabbing metrics.

Continuous Feedback
Monitor application health and perform analytics.

Key Measurement Metrics

To achieve optimization and enhance performance of the continuous delivery process, metric measurement is the key. These indicators will help in taking certain corrective and preventive actions based on the results. Below are some of the key metrics indicators that can be measured.

Effort Reduction
Indicates the reduction of human/manual efforts for tasks such as triggering a build or deployment or collecting metrics from monitoring tools.

Human Error Reduction Rate
Reduction in the effort required due to rework caused by human errors.

Reduce Issue Triaging time
Indicates the time taken to resolve issues where multiple parties are involved and triaging of problems is required.

Conclusion

DevOps bots can significantly improve team efficiency, as well as a team’s ability to respond to emergencies. They save time and money, are fun to build and use and can have a positive impact on team morale, as well as increases the overall productivity of teams. This applies to both the engineering and operations teams. Enterprises embracing digitization are looking for partners with an engineering mindset who know how to bring in collaborative agile teams with the very latest knowledge of open source platforms and DevOps toolchains.

Contact the Authors

For more details please contact the authors on LinkedIn @
Shriniwas Sathe
Saswati Mohanty

— Shriniwas Sathe
May 16, 2019
The Terrain of DevOps is Strewn with Pitfalls

As Gene Kim, one of the leading thinkers in DevOps, once said, “DevOps is not about what you do, but what your outcomes are.” What he’s telling us is that DevOps is about enabling a rapid flow of work that transitions from development to testing to operations, but the bottom line is that it’s about achieving results. Unfortunately, this too often gets lost in modern-day DevOps practices.

In this article, I will discuss five common pitfalls that occur in DevOps if you don’t pay attention to the endgame and how your organization can avoid falling victim to them.

DevOps and Automation: Measuring the Wrong Outcomes

Some people believe that, at its core, DevOps is about trying to automate everything. But ask yourself this: Once you automate, are you really delivering a better outcome?

The answer is not always yes. While metrics such as deployment frequency or mean time to repair (MTTR) are interesting DevOps indicators, they are no more interesting to determining if your team is delivering actual value than measuring the number of lines of code submitted or counting the number of keystrokes by the development team. You measure outcomes by determining a baseline from where you start and compare that with where you end.

We need to understand that DevOps is about much more than automation; it’s about the entire pipeline and how your team moves from ideation to delivery to production. Even large companies with world-class DevOps teams don’t always clearly see the problem and can struggle managing the value stream and putting together multiple pipelines. An anonymous but real-life example of this is an organization I know that had been doing DevOps for 18 months, yet hadn’t delivered anything into production in that span. Are they really doing DevOps? DevOps is a process, but if it produces zero value, it’s not delivering on its promise.

The key is focusing on delivering value and measuring value delivery into production. However, before you can begin measuring value, you have to define what it means to each individual deliverable. Common metrics include the number of story points, actual revenue from product comparison and customer satisfaction. With your clearly defined metrics in place, you can measure value as it gets delivered to the customer.

DevOps and Fragile Environments

One Achilles’ heel in the DevOps process is the pre-production environments where code gets deployed and tested. Automating a delivery pipeline relies completely on code being built and deployed to various environments. Fragile environments mean that the developers and testers are wasting time dealing with problems originating from the environment and not the code itself.

While we wish all environments could be on-demand; the fact is, most are not. Another DevOps expert, Randy Bias, has an analogy about pets versus cattle in describing servers, and the same analogy can be used for environments. If environments are not standardized and repeatable, you have to be willing to move on from it like a farmer moves on from livestock; you can’t hold onto sentimental attachments like we have with our pets. What’s most important is that in no case can an environment be singular in nature.

One element of the solution is to create code and automate, but the key is building up environments that are complete. Every environment needs proper builds, proper data and proper tests validating the environment. Additionally, the automation must have the stability to perform successfully every time. If you have piece of automation that fails even just 5% of the time, that’s unacceptable.

The only way automation works in an environment is if you can trust the environment. When the environment is unstable, it’s like playing Russian roulette: No one knows exactly where the danger lies. It could be in the environment, the code, the data, the middleware … a number of places. When a problem arises, developers are often the last line of defense, but dealing with an issue such as this pulls them away from getting real work done.

Disconnected Teams

On this point, I’d like to share a warning: Disconnected toolchains lead to disconnected teams. The planning, development and testing teams can lack cohesion, and the release and operations teams often struggle to be on the same page. Each of these groups have their own tools and management, but they need to be able to communicate with each other effectively to make things work smoothly. In an ideal world, planning initiatives would flow into features for the development and testing team, which would flow into release pipelines and operational changes. Every team still can speak its own language, but the tools need to be interconnected and unified across the teams.

Mismanaging Hybrid

The term “hybrid” has many different uses in the software development space. Hybrid methodology refers to combining waterfall, agile and DevOps practices. Hybrid environments means your environment is housed in multiple locations such as the cloud, on-premises, virtual or code. There’s also hybrid geography, hybrid architecture, hybrid style—the list goes on.

The key to transitioning to DevOps is properly managing the different types of hybrid. If you can’t manage hybrid delivery, you can’t manage your entire delivery and you can’t improve your delivery holistically. Organizations must learn to manage in a world of hybrid delivery and be able to handle cloud and on-prem, as well as teams all over the planet.

Fixing Non-Constraints

The Theory of Constraints is about pinpointing the bottlenecks in your process and fixing them. Any work that’s not focused on eliminating the constraint will not result in improvement and could be considered a waste of time.

It’s important to remember that if you haven’t fixed the bottleneck, you haven’t improved anything. At the same time, if you’re automating the non-constraints, you’re wasting efforts by trying to improve what doesn’t need improving.

As we’ve pointed out, DevOps does not equal automation. If an organization has messy processes and tries to automate them, they’ll be left with an automated mess. Therefore, organizations will see the best results from automating their constraints and just as important, not automating the non-constraints.

— Jeffrey Keyes

May 13, 2019
With RHEL 8, Red Hat Adds More Automation to IT Ops

At the Red Hat Summit 2019 conference, Red Hat today announced the general availability of Red Hat Enterprise Linux (RHEL) 8, which includes among other things a Smart Management add-on designed to make it easier to manage RHEL 8 across hybrid cloud computing environments.

In addition, RHEL 8 includes Application Streams, a set of utilities for updating languages, frameworks and developer tools without impacting the core resources, as well as updating RHEL 7 to RHEL 8 in place.

RHEL 8 also adds Red Hat Enterprise Linux System Roles, which provides modules to automate complex tasks using the Ansible automation framework along with support for the Red Hat container toolkit and the general availability of the Red Hat Universal Base Image, a userspace image derived from RHEL that makes it easier to build Linux containers certified to run on RHEL.

Red Hat also revealed that Red Hat Insights, a set of predictive analytics tools, will be included with the RHEL 8 subscription and that RHEL 8 supports both the OpenSSL 1.1.1 and TLS 1.3 cryptographic standards.

Gunnar Hellekson, director for product management for Red Hat, said Red Hat is focusing on automating as many tasks as possible as part of an effort to make the life of IT operations teams simpler. As part of that effort, Hellekson said Red Hat is moving to lower the barrier to becoming a RHEL administrator by abstracting away many of the complexities historically associated with trying to manage Linux environments using a command line interface (CLI). Much of that focus will manifest themselves via ongoing enhancements to the Compose and Cockpit tools Red Hat provides for deploying and managing instances of RHEL, said Hellekson.

Red Hat is making it clear it wants to make RHEL much more accessible to a larger number of IT administrators, especially those that historically have favored a more graphical approach to managing operating system environments. At the same time, however, Red Hat is infusing more automation into RHEL 8 as part of an effort to make it easier to manage deployments at scale. That latter capability is crucial for DevOps teams that, for example, need to be able to update a development environment without waiting for an IT operations team to reconfigure and reboot an operating system. Via add-ons to the core RHEL platform, Red Hat is now starting to expand those automation capabilities across both on-premises and public cloud computing environments.

It’s unclear going forward how many instances of RHEL any one IT administrator might be able to manage on their own. Trying to determine that number might not even be worth the effort because advances in automation are now coming so rapidly that trying to determine that right number of IT administrators per environment is not possible to determine consistently. Instead, IT organizations should just assume that, thanks to automation enabled by machine learning algorithms and predictive analytics, the operating system environment is now flexible enough to scale up and down as needed.

— Mike Vizard

May 7, 2019
Autonomous Networks: The Future We Need Now

Automation has become a necessity for managing systems in today’s age of dynamic and agile cloud-based environments. Operators are looking to optimize with efficiency and minimalism—to do more with less—as the goal. This same line of thinking should be applied to managing the network or system infrastructure.

While network operators face the same challenges as what is seen in compute environments, they appear to have the advantage of fewer pieces to manage. However, as the tendency of building more densely utilized CLOS fabrics, each network node has 10 to 20 times the number of servers under it. Even though there is a lower ratio of network devices to servers to manage, this multiplier means each network node has a higher potential impact—which means we can’t accept the notion that fewer devices result in a lower interest in automation, due to the high risk of outages. Automating these systems should be prioritized, spearheading the movement toward being completely autonomous, lowering the risks of day-to-day operations.

What’s the Holdup?

In the past decade, the transition to mobile has required the shift of applications to be flexible enough to move resources between sites and locations on demand. Led by hyperscale operators, the strategy causes application monoliths to separate into microservices, driving virtualization and containerization to allow for maximized portability. East-west bandwidth requirements expand as applications become more dispensed. This makes the network increase throughput and reliable—inciting the easy choice to adopt CLOS networks. These operators use off-the-shelf components, utilizing reliable (and cheaper) merchant silicon at the core. However, this pressures legacy network management methods by effectively increasing the number of devices from two to four distribution/core devices to 48-plus CLOS nodes. Hyperscalers addressed this issue with homegrown network operating systems (NOS), constructed with network automation to make it easier to manage at scale.

But when small to midsized organizations try to imitate the hyperscale operators, they become restrained by legacy NOS architectures. We’ve been hindered by an establishment of manually driven networks, writing scripts on top of the command line interface (CLI). In the best-case scenario, we have tools such as Ansible or Puppet, which can bring automation and some orchestration to the network. In the worst case, network managers resort to just hiring more engineers. As a result, there remains a substantial gap between network and compute infrastructure.

To display this gap between the industry’s current state and where we should be, let’s look at this graphic which shows the levels of service automation maturity:

Most modern networks operate between Level 0 and Level 2, meaning operators are barely scraping by with a combination of zero-touch provisioning and screen-scraping scripts (and the increase of engineers). They are reacting to changes that occur and aren’t actually able to verify that automation worked, outside of logging in and manually reviewing the result—which absolutely does not scale, due to frequent errors and eventual, and immense, inconsistencies between design and actual implementation.

Should Operators Look for the Solution?

Cloud-native solutions provide significantly better experience, both in maturity and agility. They have sufficiently progressed at using software to manage other software, operating at Level 3 or higher on the above scale. We can leverage this for the network, but it has to start discarding obsolete architecture and modernizing the starting point—the network OS.

Microservices, containerization and orchestration (with Kubernetes) are the foundation of cloud-native architectures and are directly addressing the challenge of automation—such as how network and application services automate with the same level of maturity by looking at service fabric mesh in the public cloud. By utilizing Kubernetes and employing modern containerized microservices architectures into the NOS, we unite NetOps and DevOps. The network is essentially just a complex distributed application and can be automated like other similar applications. We need to widely adopt autonomous networking and accept that a modern NOS architecture is needed to implement a method of building networks that match the same level of automation maturity we see with cloud-native applications.

— Adam Casella

May 7, 2019
Dell Technologies Drives Automation Deeper into Platforms

Dell Technologies this week launched a series of initiatives intended to provide IT organizations with multiple deployment options for Dell EMC appliances and servers, including forthcoming managed service offering through which Dell will manage private cloud platforms based on VMware software on behalf of customers.

Announced at the Dell Technologies World 2019 conference, Dell EMC also unveiled a new appliance series and revealed an alliance with Microsoft under which its VMware arm will make available cloud software on the Microsoft Azure cloud, in addition to existing support for Amazon Web Services (AWS), IBM Cloud and servers deployed on-premises.

Jon Siegal, vice president for product marketing for converged platforms and solutions at Dell EMC, said Dell EMC plans to make it easier for IT organizations to consume IT infrastructure in an on-premises IT environment as a service or continue to use their capital budgets to acquire servers and appliances.

To advance that latter approach, Dell EMC is launching VxFlex, a set of appliances that include an operating system that enable compute, storage and networking separately. In contrast, the existing VxRail series of appliances are optimized for VMware and assume compute, storage and networking will be scaled in tandem, said Siegal. Dell EMC is also now unifying a set of hyperconverged appliances and converged infrastructure servers under a common VxFlex brand, and Siegal said in the future those offerings will be extended to add support for Kubernetes alongside platforms from vendors other than VMware.

At the same time, Dell EMC is previewing a VxRail Analytical Consulting Engine (ACE) that will be embedded in VxRail HCI system software that Dell EMC provides in its appliances optimized for VMware. That analytics engine employs machine learning algorithms to maximize application performance, while also providing self-optimization recommendations and capacity forecasts.

The Dell Technologies Data Center-as-a-Service offering, meanwhile, is scheduled to be generally available in the second half. It marks the first time Dell has launched a managed service under its own brand. VMware, however, has been providing managed services for the past year. The Dell Technologies Data Center-as-a-Service offering extends that concept to include Dell EMC VxRail appliances.

Automation is clearly at the heart of the evolving Dell Technologies strategy. The challenge organizations that have embraced DevOps will face is aligning their processes with the automation frameworks Dell Technologies is embedding in both VMware and the underlying infrastructure on which VMware gets deployed. To address that issue, Dell Technologies over time will bundle frameworks such as the open source distribution of the Pivotal platform-as-a-service (PaaS) environment from Pivotal Software, an arm of Dell Technologies, on its platforms, Siegal said.

These moves signal Dell Technologies is trying to fulfill a promise to make it as easy to manage IT infrastructure deployed in an on-premises IT environment as it is to manage on a public cloud. There’s obviously still a gap. But as Dell Technologies starts to make IT infrastructure available as a service, it may one day become challenging for DevOps teams to tell where any one cloud happens to be running at any given moment.

— Mike Vizard

April 30, 2019
Sauce Labs Acquires Screener Visual Testing Tool

At the SauceCon 2019 conference, Sauce Labs today revealed it has acquired Screener to add a visual testing tool to its portfolio of automated testing tools.

Lubos Parobek, vice president of product for Sauce Labs, said the addition of Screener will make it possible for DevOps teams to not only test functions and performance using existing Sauce Labs tools, but also now confirm things such as the colors of a visual element of an application are being rendered correctly, for example.

In addition to building their own visual tests, DevOps teams can employ Screener to automate the running on those tests and store baselines in the cloud to create a visual testing life cycle.

At the same time, Sauce Labs announced the general availability of Sauce Headless, which leverages container technologies to make it possible to leverage a cloud service through which developers can now run tests via a Chrome or Firefox browser every time they commit code. Parobek said the goal is to make it easier for organizations to embrace continuous testing as an extension of the pipelines constructed using a continuous integration/continuous deployment (CI/CD) platform.

The acquisition of Screener, he said, is part of an ongoing Sauce Labs effort to extend the reach of its testing portfolio beyond functional testing. Sauce Labs earlier this month launched Sauce Performance, its first set of application performance testing tools.

The rise of DevOps is challenging traditional approaches to application testing as organizations move to combine various types of testing into a single continuous process. To meet that requirement, Sauce Labs is expanding its portfolio of automated testing tools at a time when the emergence of microservices-based applications is significantly increasing the level of dependency between all the components that make up an application. That increased level of dependency will require DevOps teams to rely more on automated testing tools. In fact, Parobek said the day is not far off when DevOps teams will include digital assistants that leverage machine learning algorithms to enable DevOps teams to continuously run those tests at the right moment in an extended DevOps pipeline. Those digital assistants will not replace the need for human testers anytime soon, but they will augment their capabilities to make is feasible to implement continuous testing at scale, he noted.

In the meantime, as application testing continues to shift further left in the DevOps process, more developers will be building and running their own tests during earlier stages of the application development process. At some point, either another developer or dedicated tester still will need to evaluate that code to provide an unbiased opinion. But the number of common mistakes making it into the latter stages of the application development process should be substantially reduced. Ultimately, that shift to the left should result not only in more development projects meeting deadlines, but just as significantly higher levels of overall application quality. The challenge now, of course, is figuring out how to implement a best DevOps practice that pushes application testing as far to the left as possible.

— Mike Vizard

April 24, 2019