Category: DevOps Practice

  • How DevOps Orchestration and Feedback Loops Can Enhance Your DevOps Pipeline

    How DevOps Orchestration and Feedback Loops Can Enhance Your DevOps Pipeline

    DevOps orchestration and feedback loops can significantly impact and improve your organization’s delivery pipeline. In this post, we’ll define and illuminate the importance of DevOps orchestration and feedback loops, and how together they can make a difference in your pipeline.

    DevOps Orchestration

    Although people widely use the terms “automation” and “orchestration” interchangeably, their scopes vary with significant process differences between them. DevOps automation automates a single function or a task in a DevOps pipeline that is routine and monotonous. Orchestration describes a string of such automated tasks consolidated into a streamlined workflow. In environments where automation would not work because of barriers among the groups, orchestration can be used to help organizations solve issues despite such barriers. Hence, DevOps orchestration simply brings together already existing automation networks into a cohesive, consolidated platform.

    The Need for DevOps Orchestration

    DevOps teams deal with constant navigation between development and operations. It would be highly beneficial if a single platform could pull together the tools teams typically use as various automation pieces from multiple DevOps toolsets. This is where DevOps orchestration services come into the picture, as orchestration enables teams to achieve that.

    The primary goal of DevOps orchestration is to ensure seamless and quick delivery of new builds into production. It also aims to achieve streamlined and error-free software releases. Different organizations use various coding standards, guidelines and policies. DevOps orchestration plays a major role in efficiently managing and ensuring adherence to such policies. DevOps orchestration also is pivotal in properly defining and maintaining third-party software and their multiple versions across different environments.

    DevOps Monitoring

    Effective DevOps monitoring is vital. It is the key to understanding and evaluating various performance metrics in the production environment. When done right, DevOps monitoring can provide visibility into different critical aspects such as deviations in performance, application functionality issues and service availability problems. In any DevOps environment, the ability to identify and pinpoint such issues proactively can be highly beneficial, and DevOps monitoring enables executives to do so.

    Feedback Loop

    Feedback loops are the essence of any process improvement and DevOps is no exception. Typically, such loops initially function with an organization’s policies and progress all the way through implementation, policy adjustments (if any) and usually ending with reimplementation. They are not limited to specific process cycles in a DevOps environment—for example, it’s applicable to CI/CD practices, infrastructure monitoring and application development

    Types of Feedback Loops

    Broadly, there are two categories in feedback loops: reinforcing feedback loop and balancing feedback loop. People refer to the former as “a loop of accelerating change.” In a reinforcing feedback loop, increased value in the first entity results in an increase in the second entity, which in turn further increases the value of the first entity. Conversely, in a balancing feedback loop, value increase in one entity results in a decreased value in the second entity, which consequently decreases the value of the first entity.

    Significance of Feedback Loops

    One of the major capabilities that differentiates and carves a unique niche for DevOps from the rest of its predecessor methodologies is its feasibility to incorporate feedback from customers as well as other systems in the pipeline, including the monitoring system. The feedback obtained helps to bridge the gap between the software’s function and customer expectations. It also provides valuable insights about what you can do to improve the build of the software and ultimately enhance its functionality and reliability.

    Feedback loops are particularly crucial when it comes to continuous testing. To carry out continuous testing effectively, building automated tests is only a job half done; the rest depends on the visibility of the test results and how you can utilize them to enhance the existing process. To do that, you need comprehensive feedback of your software’s performance across various stages of the software development life cycle (SDLC), starting right from development and continuing all the way to post-production. Implementing the right feedback loops will help you gather such detailed feedback.

    Thus, feedback loops serve as the difference between merely speculating how an end-user might consume a product and methodically improving the existing workflow to better suit the end-user requirements through feedback.

    — Kavin Elango

  • DevOps: Assessing Behavior Flows Through an Organization

    DevOps: Assessing Behavior Flows Through an Organization

    I’ll admit it, this article is a little strange. It is an experiment. But DevOps is all about experimentation, right? This article explores how information and behaviors in fact “flow” through an organization. It is based upon my observations with literally hundreds of DevOps teams. I have used the Three Ways of DevOps to show how and why many organizations struggle with this flow of information and behaviors, and what you can do to identify and prevent behavioral “defects” in your organization.

    Simple, Really: Mindsets and Behaviors

    IT organizations embarking upon Agile or DevOps ways of working are basically attempting to realize behavior change—simple. Consistent, new “desirable behaviors” that deliver value and reduce risks, wasted effort and costs—simple. This sustainable behavior change demands a change in mindsets and beliefs; it also means unlearning old ways of behaving—not so simple. This requires feedback and correction.

    Start with the ‘Why’

    Changing mindsets relies on the right messaging. Understanding the “why” and the “purpose” is seen as a critical factor in changing mindsets and ultimately behaviors. The right messaging is not just in terms of words, but also in terms of behaviors, such as people “walking the talk” and “leading by example.”

    Desirable behaviors are about people doing the right things. Doing the right thing implies making the right decisions; for example, prioritization of work. To be able to make the right decisions requires having the right information, such as mentioned above—the why. Knowing and understanding the why helps underpin decision-making and prioritization. However, we see many teams that do not fully understand the why behind new ways of working. Somehow the message did not arrive. Somewhere in the flow of information the “why as a justification” ended up as the “why as a question.”

    This made me realize that information and behavior flows through an organization. I decided to look at many of the challenges organizations face when adopting new ways of working, by using the Three Ways of DevOps.

    The First Way: Flow

    The First Way of DevOps is all about the flow from left to right, understanding the flow of work, seeking to increase the flow and never passing a defect downstream.

    Understanding the Flow

    The messaging I mentioned above flows through the organization traditionally top-down and bottom-up; with the shift to Agile ways of working, this flow is through the end-to-end value stream. This flow is upstream and downstream: Downstream is the flow and hand-offs; upstream is feedback.

    I mentioned above that desirable behavior is related to making decisions. If we consider the traditional decision-making authorities in an organization, we have from left to right: strategic, tactical and operational decision-making. This can be seen, for example, as the CIO and business management team (strategic decisions), e.g., product portfolio and key goals; product owners and IT delivery managers (tactical decisions), e.g., product backlog, IT changes; teams and employees (operational decisions), e.g., sprint planning, incident prioritization.

    The importance of decision-making was also confirmed in a recent Forbes article, “The Netflix Decision Making Model Is Why They Are So Successful.” Netflix’s desirable behaviors are driven by a set of corporate values. The first value is judgment, which “outlines the company’s decision-making ways and accompanying expected behaviors.  Notably, Netflix encourages its employees to ‘make wise decisions despite ambiguity’.”

    Every so often, it is decided to embark upon an Agile transformation at a strategic level. This decision is communicated throughout the decision-making levels as an intent—usually the why is vague and left open to interpretation. A set of values or principles are also communicated, such as customer-centric, end-to-end responsibilities and continuous improvement. However, these usually are not translated into desired behaviors. An implied set of behaviors are linked to these values and principles, and can end up being inconsistent with the original intention. This miscommunication of the why and the desired behaviors downstream is a form of a defect—often causing assumptions, driving poor decisions and resulting in undesired behaviors.

    For example, as a frustrated senior level manager complains that teams won’t take ownership and frustrated teams protest about lack of empowerment, desired behaviors were not defined and agreed. As a result, they were not displayed, waste occurred, frustration, delays, lack of return-on-value from the effort and behaviors spent.

    Another concept of flow is constraints—constraints stopping or slowing down the flow. In terms of behaviors, a constraint could be attitudes and beliefs (not understanding or believing in the new behavior), lack of skills or ability to perform the behavior and allocation of time to learn and experiment or improve behaviors. So, not only must we ensure that we do not pass defects downstream, we also must ensure feedback can identify constraints, impediments and barriers that are stopping the right behaviors from flowing smoothly through the end-to-end stream.

    Other types of constraints could be ignorance, indifference, confusion, fear, uncertainty, doubt—or, as Mike Orzen said in a discussion, focus, attention and trust.

    So, on one side we have:

    • Intent, implied, assumptions and inconsistent behaviors flowing from left to right (cause), meeting….
    • Indifference, confusion, fear, uncertainty, doubt, lack of trust etc. (effect)—you get the picture.
    • A management solution to this is then to tell people to collaborate, or to tell people they are now empowered—again, without defining the associated behaviors.

    “… Application delivery is a set of very human-oriented tasks, a key factor that determines the speed to value is trust,” wrote Carmen DeArdo in his latest book, “A Leader’s guide to Digital Transformation,” co-authored by Jack Maher. “As teams lose trust in the validity of the work as it flows through the lifecycle, a large amount of rework and waste is introduced.” he wrote. Trust is probably one of the biggest enablers or barriers to the smooth flow of behaviors through the system.

    The Second Way: Feedback

    The Second Way of DevOps talks about shortening and amplifying feedback right to left so that corrections can be made. One form of this is giving feedback when poor information is passed downstream or undesirable behaviors are displayed, enabling us to correct the messaging and the behaviors. Inconsistent messaging, in the form of managers giving conflicting signals or failing to “walk the talk,” are forms of defects. A commonly chosen ABC card (Attitude, Behavior, Culture) in global workshops is, “Don’t do as I do … do as I say.” However, if there is a culture of blame or a lack of safety associated with giving feedback to correct defects in behavior, then feedback isn’t given. Undesirable behaviors and inconsistent messaging goes unchecked.

    Failing to stimulate and act on feedback is another form of defect. This means undesirable behaviors continue to occur. All these behavioral defects create cultural and behavioral debt. As this debt accumulates, the ability to deliver value diminishes. In response to this, managers often revert to old management and control styles of behavior (cause), creating more resistance and reducing the willingness to give feedback or experiment (effect). Again, examples of passing defects downstream and creating more barriers and impediments to the flow of desired behaviors.

    As suggested in DeArdo’s book, “Feedback should be treated as though it were the most precious of our resources, because it is.”

    The Third Way: Experimentation and Learning

    The Third Way of DevOps is about continuous experimentation and learning—recognizing that repetition and practice are the prerequisites to mastery.

    Agile ways of working are still relatively new. DevOps is still evolving. Old management styles and models are no longer fit for purpose in this new world. Leadership in this new world is also characterized by experimentation. As one CIO stated, “Leaders are expected to walk the talk, but we don’t know what that looks like in this new model.” Leaders often fall back to an S1 leadership style—which is in conflict with the culture associated with Agile ways of working. Poor leadership styles and behaviors, as mentioned above, are also a form of defect—more behavioral debt.

    Feedback becomes even more important to validate experimental approaches and to learn and improve. However, the Third Way also stresses the need to allocate time for the improvement of daily work. We sometimes see impediments on team backlogs, but we don’t see behavioral or cultural debt as a backlog item. We don’t see this on the backlog of management teams’ behavioral debt, either. Teams often are frustrated with a lack of time reserved for technical debt, impediments or improvement work—let alone behavioral debt. As Orzen stated, “Even more important than daily work is the improvement of daily work.” Continual learning and improvement should be core skills and behaviors.

    As stated in the Third Way, repetition and practice are prerequisites to mastery. There is nothing new about this. It isn’t that the DevOps movement came up with this “new” philosophy: Aristotle, around 300 B.C., stated, “We are what we repeatedly do. Excellence, then, is not an act but a habit.”

    We need to practice and repeat behaviors until they become habits, or culture—like “the way we do things here.” Practice also requires feedback to validate improvement gains as a result of practice.

    Top athletes practice with the guidance and feedback from coaches. Scrum and Agile coaches are often assigned to Agile and DevOps teams as they practice and grow. These coaches are skilled in training on Agile and DevOps tools, such as standups, retrospectives, value stream mapping, Kanban and Kaizan. But what we see globally is that not all coaches have behavioral management skills.

    Another form of feedback from practice is measurement. In terms of DevOps, teams measure metrics such as number of releases—successful releases. We don’t often measure the behavioral change. We can measure the frequency and decisions from standups, but can we measure giving feedback, confronting undesirable behavior? One small example I use is measuring the amount of “yehbuts”—yes, but!—in a dialogue, or worse still, “yehbuts” before somebody has even finished talking. This is an example of a communication defect. Another example is assuming that we understand what somebody has told us without confirming if what we believe we heard was the actual intention.

    Communication defects are a major barrier to successful flow of information. Another measurement needs to be related to the impact of behavior change, such as improved customer and employee satisfaction and less rework because of assumptions.  Although measuring the actual impact is difficult, having the dialogue itself is valuable: How is what we are doing adding or reducing value to the team and to the organization?

    Conclusion

    If we want desirable, consistent behaviors focused on creating high-performing teams, we need to look at how messaging and behaviors flow through the system and recognize defects we are passing downstream. Our vague intentions need to be quantified in terms of desirable behaviors. We need to build in feedback mechanisms to signal defects and undesirable behaviors. We need to identify the constraints and impediments that may be causing undesirable behaviors such as fear, uncertainty, doubt—behaviors that may be slowing down or stopping our transformation to new ways of working. We need to recognize that experimentation and practice is needed—knowing that we will struggle and fail, but ensuring we learn and improve as an end-to-end capability. Leadership skills, styles and behaviors need to be aligned to facilitating the three ways of behavior flow. Finally, remember there is really nothing new about this.

    Case

    In a Phoenix Project simulation, after recognizing the above defects, a team posted the strategic portfolio and goals on the visual management board. The CFO explained these goals and why each backlog item was prioritized against value creation (benefits and value to be realized)—an example of clarity of the why. At the same time, the CFO displayed the right behaviors—being at the standup to explain to the team and ask for direct feedback and understanding.

    At the same standup, the CISO and IT team explained the potential “value leakage” backlog items in terms of debts, risks and defects, so that together—business and IT—could make the right decisions on prioritization.

    “This made me realize,” said the product owner, “I was now able to make a better-informed decision, because of the team’s input on value leakage—giving feedback from the usage of the applications and the negative impact. It also gave me more trust in the team as they demonstrated an understanding of the ‘why’ and had a clear understanding of value. If, as a team, we did not fully know or understand the value, this created a great dialogue to explore it and ensure we didn’t pass a ‘defect’ (wrong decision on prioritization downstream).”

    The team in the simulation learned that as well as visualization of impediments on their backlog, they also needed to visualize the behavioral defects that needed work on and allocated time.

    — Paul Wilkinson

  • SRE: How Do You Get to Blameless?

    SRE: How Do You Get to Blameless?

    Site reliability engineering (SRE) is the companion to DevOps. If we take my normal divisions of DEVops versus devOPS, traditional DevOps is mostly the developer side, and SRE is mostly the Ops side. This isn’t a perfect comparison, but it is a good starting point, and for this blog, I’ll run with it.

    While DevOps focuses on getting releases out consistently, quickly and with minimum disruption, SRE focuses on putting those releases into a robust infrastructure. Tracking application issues, regardless of whether you’re using DevOps, always requires information about the software and hardware infrastructure. SRE takes the proactive steps to try and make sure the environment is stable and designed to host the applications being thrown onto it. When that process fails, SRE attempts to identify the code/architectural problem and resolve it without blaming staff members—hence, the idea of “blameless.” It’s about the environment and improving it, not about the individual and punishing them.

    There are a lot of interesting parts of SRE that most IT groups could look at and see as a good place to start taking advantage of. Closely tracking errors in a cross-section of IT interest areas—by application, by infrastructure, by toolset—means problem areas can be definitively identified and fixed. But there is more behind SRE. In this example, once tracking of where errors occur more frequently is in place, a selective reduction in deployments using that piece of the IT puzzle can go into effect until the issues are resolved. This is not unlike what we are doing today, but it is formalized and has measurements around it that generally are not in place.

    Which brings us to Blameless, a company that just came out of stealth to champion SRE. While some large organizations with large application footprints are using SRE already, Blameless is offering a platform to make SRE more accessible to companies without large staff and application footprint.

    We took a bit of time to talk with Blameless last week, and their vision is a strong one: Address directly what DevOps addresses more indirectly. Build and environment that can measure and improve the entire application ecosystem.

    If nothing else, the idea of error budgets is intriguing. Counterintuitive at first glance, the idea of saying, “Piece X gets Y amount of issues before we start backing off of it or raise the priority to fix the underlying issues,” is a solid one.

    If your organization is looking for a mechanism to improve reliability and stability while reducing time to resolve issues, SRE in general and Blameless specifically are worth a closer look.

    — Don Macvittie

  • Blameless Launches DevOps Platform Based on SRE Principles

    Blameless Launches DevOps Platform Based on SRE Principles

    Blameless has emerged from stealth to launch a platform for managing DevOps based on the site reliability engineering (SRE) best practices defined by Google.

    Fresh off raising an additional $20 million in funding, the Blameless platform provides across-the-board visibility into the complex network of application programming interfaces (APIs) and applications distributed across an organization, said CEO Ashar Rizqi. Specific modules within the Blameless platform address incident resolution, reliability insights dashboards, postmortems, service level objectives (SLOs) and automatic error budget calculation.

    Blameless takes its name from an SRE principle that no one should be held at fault when complex systems fail. Rather, the assumption should be that inherent complexity of the IT environment is at fault. While DevOps is more of a philosophy surrounding the management of DevOps, SRE is emerging a more prescriptive instance of implementing DevOps that Google is now promoting. The goal is to make sure that whatever the issue, the right people to address it are brought together at the right time, said Rizqi.

    The Blameless platform was created by Lyon Wong, who now serves as the COO of Blameless. Previously, Wong was a partner at Lightspeed Venture Partners, founder of S28 Capital and a product manager for Microsoft Windows. Rizqi is a former head of core platform at MuleSoft and SRE manager at Box.

    Rizqi noted that while most organizations are trying to embrace DevOps principles, they often lack the tools needed to build the workflow processes required. All too often, organizations rely on little more than a Wiki document to manage their response to an incident. The Blameless platform is designed to provide access to an integrated set of tools for managing everything from the incident itself to the impact that event had on the business, he said.

    It’s still too early to say with any certainty how widely SRE practices will be adopted. This far, most organizations that have adopted DevOps have tended to prefer to define a set of processes that are customized to their environment. But there are just as many, if not more, organizations that have found it daunting to get started. Platforms such as Blameless represent an opportunity for those organizations to embrace DevOps using a more prescriptive approach.

    Of course, a big part of the reason that DevOps has been embraced unevenly by various departments within the enterprise is the lack of a consistent approach to implementing DevOps at scale. Teams that have embraced DevOps with enterprises have tended to create their own individual processes using a wide variety of tools. Many of those organizations have also found it challenging to get IT operations teams to embrace those processes because they require an approach that can be implemented consistently across multiple projects at scale. SRE provides IT operations teams with a framework that Google has already proven works at scale.

    Regardless of the path chosen, the one thing that is for certain is more agile approaches to managing IT operations will be required to keep pace with the rate at which applications are now being developed and deployed.

    — Mike Vizard

  • SolarWinds Adds Ability to Profile Live Code to APM Platform

    SolarWinds Adds Ability to Profile Live Code to APM Platform

    SolarWinds is advancing its campaign to become a leading provider of DevOps tools by adding the ability to profile live code to SolarWinds AppOptics application performance monitoring (APM) platform.

    Keith Kuchler, vice president of engineering for SolarWinds, said live code profiling provides a breakdown of the most frequently called functions and methods in a transaction, including class, method, filename and line number. That capability makes it possible for DevOps teams to identify what line of code might be causing a performance issue without requiring any changes to the code itself, he said.

    According to Kuchler, the ability to profile live code extends a distributed tracing capability that SolarWinds previously incorporated into SolarWinds AppOptics. Collectively, the two offerings show how committed SolarWinds is to providing DevOps teams with access to advanced capabilities at a fraction of the cost of rival APM platforms delivered as a software-as-a-service (SaaS) application, said Kuchler. Pricing for SolarWinds AppOptics starts at $7.50 per month. SolarWinds AppOptics is part of what SolarWinds describes as a DevOps Toolkit that also includes Pingdom, a website performance monitoring tool, Papertrail, a log management service, and Loggly, a log analytics service.

    The ability to profile live code supports applications written in Java and PHP, with support for additional programming languages forthcoming, said Kuchler. The goal is to reduce the amount of time DevOps teams need to spend guessing where the source of any issue might be after an application is deployed or updated, he added.

    As the rate at which applications continue to be deployed and updated, DevOps teams need to be able to pinpoint issues at the code level to enable developers to address that issue as soon as possible. Sharing a bunch of generic alerts typically doesn’t provide a developer with enough actionable intelligence to solve the problem. Not surprisingly, a recent survey of over 300 developers conducted by SolarWinds finds that most of them spend a lot more time troubleshooting existing applications versus writing new code. All that time spent troubleshooting code winds up having an adverse impact on the ability to make the transition to DevOps.

    In general, SolarWinds is trying to expand the total APM market by making its tools available as a service that organizations of any size can invoke. Historically, reliance on APM tools has been limited to mission-critical applications mainly due to cost. But over time, the number of applications deemed to be mission-critical has increased. In addition, because of the all interdependencies between applications, the potential impact a single line of code can have across the enterprise IT environment can be substantial. As a result, the need for an affordable approach to APM has become more acute.

    Naturally, it will be interesting to see how much pressure there is on APM pricing over the course of the coming year. Competition between providers of APM platforms is already fierce. The challenge will be finding a provider of an APM platform that not only provides all the capabilities required, but also has deep enough pockets to be around for the long haul.

    — Mike Vizard

  • Why Unikernels Are Great for DevOps

    Why Unikernels Are Great for DevOps

    DevOps tooling seems to be in a never-ending state of self-transformation. The continuing evolution of infrastructure and the tools that go with it are forever creating new ecosystems. On the infrastructure side, we saw VMWare, Citrix and other companies bring to market massive virtualization in the late ’90s and early ’00s. This, of course, prompted a little bookstore in Seattle to unleash something known as “the cloud,” forever changing the way developers push code to production.

    The cloud, of course, is just an API around virtualization. A lot of the infrastructure changes prompted configuration management software, including Chef and Puppet, to emerge. Puppet is now 15 years old and Chef is around 10; even Ansible is now 7 years old.

    In the past years, we’ve seen tools such as Terraform emerge and other higher-layer infrastructure, such as Docker, which led into Kubernetes. There’s a lot of tooling out there to make applications run on servers. Just as the containers/Kubernetes ecosystems have tried to implement change on top of virtualization infrastructure, we are starting to see developers dig back down into the operating system layer to improve infrastructure there. This leads us to our topic today: unikernels.

    Unikernels are single-purpose virtual machines (VM). They only run one application—which is interesting when you think about it, because that’s precisely how a lot of DevOps practitioners provision their applications today. There is way too much software to provision nowadays—pools of app servers, clusters of databases, etc. Unikernels are fast to boot, fast to run, small in size, way more secure than Linux or containers and, if you have access to your own bare metal—either in your data center or through a provider—you can run thousands of them per box.

    Some stats? Some can boot in less than five milliseconds—that’s slightly slower than calling fork (three milliseconds) but orders of magnitude faster than Linux. Some can run databases including Cassandra and MySQL, 20 percent faster. Some people (yours truly included) have provisioned thousands of them on a single box with no performance degradation. Some are reporting them running as a VM faster than a comparable bare-metal Linux installation of that application. Clearly, this is worth investigating.

    There are many different methods that a given organization can take to provision software, but it’s pretty clear that popping into a server over SSH and sudo apt-get install something is very much of a downtrend. Phrases such as “immutable infrastructure” and “continuous deployment” fly right in the face of this sort of practice. You are either rebuilding your containers, rebuilding your instance or both. Unikernels really embrace this model as there is no apt-get-like functionality in them and you can’t log in to them remotely. They don’t even have the concept of users, mainly because at the end of the day, why do you even need them?

    At scale, it’s getting more difficult by the day to make the argument for this construct. If the app is misbehaving, shoot it in the head and take it down. Debug the reason locally. It’s extremely rare for an end user to reason that there is something wrong with the host operating system.

    In a unikernel, the only reason you’ll see a reference to a user is that most apps will demand to be run as a particular user or do things such as check for root privileges. In our case, if you look at underlying calls such as getuid, they are just stubs because we don’t need or use them. Keep in mind that this isn’t the mid-’90s anymore. We don’t have boxes laying around, where there are half-dozen users all running their code on the same box anymore. That model has long been gone. Unikernels don’t have shells and don’t have the notion of running other programs besides the one that is running.

    If it’s easier to think about, consider that unikernels are much more like processes than they are like operating systems, although they share characteristics of both. While many link, the application in others will load. Unikernels are a concept that are largely misunderstood because everyone assumes it’s somehow like Linux. Just as unikernels are not just “containers 2.0,” unikernels are very different from Linux as well. Linux is a general-purpose operating system, while unikernels only run one application. Linux was made to run on bare metal, while unikernels are made to run as VMs. Keep in mind Linux predates VMWare by a solid eight or nine years and the cloud by many more. Operating system researchers wanted to go this route in the ’90s, but we just weren’t ready yet.

    Another way to think about them is that they implement a sandbox for server-side applications. Much in the same way that the Chrome browser isolates each tab from each other to prevent one evil site from messing with another browser tab, unikernels prevent one application from messing with another application. Although under the hood, the technical implementation is quite different. More importantly, the unikernel single-process model eliminates the capability of running other ad-hoc programs inside the same VM. This is a huge blow to attackers who wish to run other programs on the same machine. It doesn’t prevent things such as remote file inclusion (RFI) attacks for various scripting languages, but for compiled and VM runtimes, it’s a striking blow because it deals with the fundamental problem of attackers using exploits only as a key to enter the server. Once they are on the server, they want to run their programs such as mysqldump or a cryptominer. They could care less about your software. Those organizations practicing shift-left security, or zero-trust infrastructure, should be keeping the development of unikernels on their radar.

    However, unikernels are not without their problems. These are computers we are talking about here. One of the larger problems for unikernel adoption was that to build them, you had to be a lower-level systems engineer doing things such as twiddling CFLAGS in makefiles or patching foreign codebases you’d never patch yourself. This was problematic and something we wanted to fix with a new open source tool. So, we opened up OPS—which can take any existing Linux application and run it as a unikernel, making their use a lot easier.

    If you’ve made it this far, I’m going to reward you: Let’s build and run a unikernel right now. Then, at least, the next time someone asks you if you’ve used them before, you can say yes.

    First thing is, you’ll want to grab OPS. You can paste this into a Linux or Mac shell, and it will download and install itself:


    curl https://ops.city/get.sh -sSfL | sh

    If you’d like to examine the source and build it yourself go to https://github.com/nanovms/ops.

    Next, let’s create a project directory:


    mkdir nodejs && cd nodejs

    Put this into a hi.js file:


    var http = require('http'); http.createServer(function (req, res) { res.writeHead(200, {'Content-Type': 'text/plain'}); res.end('Hello Worldn'); }).listen(8083, "0.0.0.0"); console.log('Server running at http://127.0.0.1:8083/');

    This is simply a small little web server that listens on port 8083.

    At this point you might be asking yourself, Do I need to install node on my system? The answer is no—not unless you plan on using external packages.

    Now we can run it:

    If it’s the first time you run ops, it will first download a node “package,” which is the interpreter and anything else it needs to run. For some languages, such as Python and Ruby, this can be substantial.

    Then it builds a small disk-bootable disk image that contains your application code and, as you can see, boot straight into it. There is no init manager like Linux and there aren’t 30 different services that get fired off. Keep in mind this is a single process running.

    Now you should be able to hit it up with curl and see it run:

    You’ve just built and run your first unikernel. How cool is that?

    As certain trends such as “serverless,” “functions as a service,” and “no ops” gain steam, these concepts go hand in hand with unikernels. DevOps practitioners at organizations that use microservices and containers will also want to take notice as the amount of work to deploy artifacts such as these decrease considerably. Why is that? One of the many problems that containers had to struggle with was that all the existing underlying infrastructure layers that live at the virtualization layer, such as file systems and networking, had to be shifted up a few layers. Not only did this introduce performance and security challenges, but it also threw out the 20 years of virtualization software and ecosystem development that already existed. What’s interesting about unikernels is that they are always deployed as VMs, so they get to re-use a lot of the underlying ecosystem that has been existent, with very little or no change required.

    From an adoption standpoint, unikernels are flying at ultra-sonic speeds under the radar. A lot of people are not aware of how many projects or organizations are working on unikernels today. Unikernels are starting to attract major attention from many various groups and institutions. Even famous, old stalwart Linux contributors such as Ulrich Drepper, the maintainer of libc, are working on unikernel projects today. Now there are even foundations and consortiums for unikernels. One recently announced was the UNICORE project. IBM even listed its No. 1 cloud patent recently as a hypervisor built for unikernels.

    If you’re are a self-described DevOps person, it’d behoove you to learn more about unikernels and start playing around with them to understand them better. They aren’t going to kill Linux and they’re not going to kill containers, but do expect to see massive adoption in the coming years—not just because of their security and performance benefits, but because of the new forms of compute they unlock.

    — Ian Eyberg

  • G2G3 DevOps Simulation: Lessons from the Facilitator POV

    G2G3 DevOps Simulation: Lessons from the Facilitator POV

    The G2G3 DevOps Simulation is a three-round game that gives a complete immersive experience to the participants. It has software that’s controlled by the facilitator and used to see the performance and improvement in each round.

    As a facilitator, I help on the retrospective at the end of each round, as the participants share about what they did and didn’t do well. At the planning stage for each round, the participants decide what they want to change and get to see the result.

    Below is a list of what I focus on as a facilitator.

    Looking at the Same Objective

    Participants in the G2G3 DevOps Simulation learn that, regardless of role, their only objective is generating maximum revenue. This is the common goal. They can see this objective on the screen from the system. The participants begin to understand the goal on round two and by round three the objective becomes clear. This helps them to work in collaboration, working in a “squad” and helping each other to achieve that one common goal.

    Collaboration and Sharing

    The G2G3 DevOps Simulation gives the participants the chance to experience how collaboration and sharing helps in delivering stable products faster. Each team starts helping each other to ensure that their business makes the maximum revenue. They understand the need for delivering the outcome.

    The simulation also shows the challenges of communication between “run the business” and “change the business.” They experience how the communication drops between the two and how to take care of it with better knowledge management.

    Project-to-Product Approach

    The participants learn how to work as a squad—one single team responsible for both development and operations. They learn how the value flows from the time change is requested, until it is deployed and then the deliverable is maintained.

    Business is Most Important

    The G2G3 DevOps Simulation shows the participants that business is the most important—and IT is business. Everyone in the value chain must work toward the same objective.

    Automation

    The simulation also gives the participants the experience of how automation helps in achieving the objectives faster and better. This is clearly shown in the performance report at the end of each round and the comparison can be seen between rounds with more automation in place.

    DevOps: 3 Ways

    The G2G3 DevOps Simulation explains the basics of DevOps, bringing all participants to the same page on various DevOps terminologies, the principles (Three Ways) and DevOps values.

    Conclusion

    The G2G3 DevOps Simulation is a good practice for anyone in any organization who is on the journey of DevOps or is planning to start the journey. The best can be achieved if members from the various departments of the organization participate, which is required to deliver any change.

    — Niladri Choudhuri

  • Atlassian Adds Business Planning App by Acquiring AgileCraft

    Atlassian Adds Business Planning App by Acquiring AgileCraft

    Atlassian announced it is acquiring AgileCraft to give business leaders visibility into strategic projects in a way that is closely mapped to IT projects being driven by DevOps processes.

    The value of the deal, expected to close next month, is estimated to be $166 million.

    Many DevOps teams already make extensive use of the Jira project management software developed by Atlassian. AgileCraft emerged in 2013 as a provider of a collaboration application aimed at business users to keep track of the various strategic IT initiatives that are largely dependent on DevOps processes.

    Atlassian and AgileCraft previously established a technology alliance to facilitate that goal. Atlassian president Jay Simons said the acquisition represents an opportunity for Atlassian to extend its reach into the realm of business users who increasingly are trying to keep track of complex digital business transformation initiatives that are dependent on multiple IT projects. Many of those digital business initiatives tend to quickly fall behind schedule because the dependencies between various IT initiatives are not clear to all the key business stakeholders, he said.

    In addition, Simons said many businesses now are trying to apply the agile methodologies pioneered by DevOps teams more broadly within the IT organization. To achieve that goal, business users need a project management application that has been optimized to enable them to keep track of multiple inter-related DevOps projects, he noted.

    Most organizations are not yet anywhere near the point where the IT organization is completing IT projects faster than the business can absorb. But as agile methodologies and best DevOps practices become more widely employed, there may come a day when IT organizations will complete IT initiatives faster than most business will be able to adapt. In fact, it’s not uncommon for organizations that have embraced DevOps to deliver multiple updates to applications a week. AgileCraft planning software will make it easier for business users to appreciate all the business implications associated with each of those updates. The irony is, the constant debate over the inability to align IT with the business usually comes down to the inability of IT organizations to keep pace with the demands of the business. It soon may turn out the business is no longer capable of absorbing the rate at which new applications are being rolled out.

    In the meantime, applications such as AgileCraft should make it easier for IT leaders to have better-informed conversations with business leaders concerning where limited resources need to be applied and when. Business leaders will be able to see for themselves now what impact any given set of missed deadlines is likely to have on any strategic initiative. In fact, there may come a day when business leaders proactively make more resources available to achieve a specific goal without requiring IT leaders to attend a meeting to explain in excruciating detail how every delay adversely impacts the grand scheme of the business. Instead, business leaders will be able to see for themselves exactly where additional resources are needed to achieve a specific business goal.

    — Mike Vizard

  • 5 Steps for Successful Continuous Testing Implementation

    Continuous integration (CI) calls for code to be always working. Continuous delivery (CD) and continuous deployment mean that we must be able to roll out a working configuration to production at any time. As DevOps becomes the norm, bugs in production are now a developer’s personal problem. Developers (and their managers) want to know immediately if code changes they are working on can break something.

    What is Continuous Testing?

    The only guaranteed way to prove code and configurations are working is to test them. That’s why we need continuous testing. In organizations making this transition, performance engineers and test automation experts shift from minding a backlog of test-building work, to facilitating a democratization of testing. Everybody builds their code and everybody tests their code, continuously.

    Continuous testing is becoming the “new normal.” In this modern and agile testing approach, developers are equipped with low-friction testing capabilities and tools and are expected to unit test any new or updated code. With testing shifted all the way left to developers in pre-production, and all the way right to post production monitoring, organizations are just seconds away from proving code is ready and working to acceptable service levels.

    Although continuous testing is becoming a more standard practice, it certainly presents unique challenges for execution.

    Here are five steps to help any company successfully implement a continuous testing policy:

    Define Tests Early

    Lack of clear requirements can result in wasted time and delays. Behavior-driven development (BDD), acceptance test-driven development (ATDD) and model-based testing using tools should be used so that all requirements are clearly documented and communicated.

    Test cases need to be clearly defined and test scripts created ahead of time, to enable continuous testing of code at all stages of production.

    Optimize Testing Processes and Test Coverage

    Save time, money and resources by testing only what you need to test. Visual models allow various paths to be explored and optimized so that only your tests provide maximum coverage, without unnecessary testing.

    It is also recommended to import test cases, remove duplicates and distribute and allocate optimized test cases in existing tools.

    Shift-Left Your Testing (and Shift It Right as Well)

    What do we mean by shift-left testing? Tests should be run earlier in the development cycle. Developers test as they go, with test automation including functional testing, performance testing, monitoring and security testing. This requires developer-friendly tools for easy adoption and use.

    By shift-right testing, we continue to test through production and post-production, including continuous monitoring. Gartner recently released a report on the shift-right testing, calling it a “must” for adopting DevOps practices.

    Provide Complete Test Environments

    The ability to provide virtualized test environments is critical to achieving continuous testing. Eliminate blocks and reduce wait times by providing complete test environments on demand, with dev-friendly tools (as code, CI/CD integrations, supported open source).  These environments should include test data on demand to ensure that teams can perform comprehensive tests with production-like data. These environments should be ephemeral; ready on demand, decommissioned after use.

    Get the Right Test Data

    The inability to get robust test data causes significant delays in many application release cycles. To accurately test new functionality, test data should be as close as possible to what the application will encounter in production. If the test data lacks certain real-world characteristics (i.e. actual fields, data specifications, negative scenarios, etc.), the tests are unlikely to find many potential issues or break the application where there are weak points. Understanding data models in order to extract the right data is a special skill in and of itself.

    Production data is clearly the best data to use for testing, but data privacy regulations and team silos often restrict its availability. The answer to this dilemma is a robust test data management tool that lets you copy production data and mask it to protect sensitive information, while maintaining the characteristics that make production data desirable for testing.

    Remember, continuous delivery and continuous testing are processes that can take time to implement successfully, but teamwork, continuous feedback based on data and the right tool set will help you on your journey. You can read more about implementing a continuous testing policy here.

    — Sushil Kumar

  • Integrating the Impactful Immersive Experience Team

    Integrating the Impactful Immersive Experience Team

    Immersive: providing, involving or characterized by deep absorption or immersion in something (such as an activity or a real or artificial environment) — Merriam Webster Dictionary

    We immerse a teabag into hot water to brew tea, but when we speak of the “immersive experience” we’re not planning to boil anyone. We may, however, make them feel as if they’d been boiled, or frozen, or hurtling through space, enjoying the beach or any of many other experiences. What we are really talking about are digital-based immersive experiences.

    The First Immersive Experience

    From July 1930 when Orson Welles first asked, “Who knows what evil lurks in the hearts of men? The Shadow knows …” or January 1933 when actor Brace Beemer first intoned, “Return with us now to those thrilling days of yesteryear, when from out of the past come the thundering hoofbeats of the great horse Silver! The Lone Ranger rides again!” radio was the first analog immersive experience.

    It was immersive because the audience employed their imaginations to add imagery to the spoken word and the sound effects. While listeners knew these were fictional tales about fictional characters, they became immersed in the stories because they exhibited what poet and aesthetic philosopher Samuel Taylor Coleridge first described in 1817 as “the willing suspension of disbelief.”

    Coleridge posited that if a writer could infuse a “human interest and a semblance of truth” into a fantastic tale, the reader would suspend judgment concerning the implausibility of the narrative.

    Television and movies leveraged that same willingness to achieve even more immersive experiences.

    While these are all examples of entertaining immersive experiences, it wasn’t long before these turned up in practical applications. As an example, astronauts learning to rocket to the heavens first experienced those travels in a flight simulator that combined film imagery and audio to create the impression that the user was actually in the rocket.

    Digital Immersive Experiences

    The actual word “immersive” wasn’t used much until the mid- to late-1980s, a time when Atari, Nintendo, Sega, Apple, IBM and others were busily introducing new personal computers and game consoles.

    Computer-based gaming was perhaps the precursor, or even the first example, of digital immersive experiences. Players often entered a catharsis in which they perceived themselves to be existing in a fantasy world. The image quality created by the software continued to become more and more lifelike and real to enhance that perception of participation.

    Now retailers are beginning to leverage immersive experience technology to help customers shop by showing them what garments might look like—on themselves. Varying colors and accessories instantly by clicking at a kiosk or computer screen.

    Many training courses and support activities are leveraging immersive technologies to augment the student or practitioner experience. A technician in the field can don a headset and see the support expert’s hands showing them exactly what to do and where. NASA has astronauts walking around on a virtual Mars. Students sitting at home feel like they’re in a classroom with the instructor and other students. Peloton customers mount their home spin-cycle in their family room and participate in a class now occurring in a New York studio.

    Many producers of software describe the importance of the user experience to the success of their products. Citrix, for example, designed its entire product suite around the idea that the mobile user experience should closely approximate the on-premises user experience, even though available bandwidth differed widely between the two. Today’s virtual desktop experience, which manifests in many ways, follows that design ethic. It’s worth noting that Citrix continues, to this day, to find ways to improve video and audio transmissions between the data center and the user in a constant effort to enhance the user experience.

    Extending the DevOps Model

    Beyond the “continuous improvement” goals of the DevOps model of computer and software management is the idea that the developers who write the code and the operators who make sure it runs well on the network infrastructure must work in concert to rapidly identify, quantify and resolve all issues.

    Immersive experiences take this concept further, extending the integrated team to include not only the coders and operators, but also the artists and writers who create the written, audio and visual content and the subject matter experts who assure the veracity and integrity.

    Like game developers, this extended team must work together in concert to assure that the content is appropriate, the code is efficient and the delivery is continuous, flawless and totally secure. Even the slightest latency may damage the user’s willingness to believe.

    Channel Choosing

    This team must also determine best practices for developing and delivering experiences through appropriate, effective and impactful channels. In the 1930s, the only vehicle was radio. In the 1940s, image was added to sound with the introduction of television. Opportunities exploded starting in 1969, with the introduction of what would become the global internet.

    Today we can categorize virtual reality, augmented reality, mixed reality and more into immersive experience options. It’s one thing to have a 360-degree camera; it’s another to put it into the hands of a talented 360-degree photographer or videographer. Developers are starting to design impactfully for Oculus Rift, Microsoft HoloLens and other immersive devices.

    Even the tools people use for everyday communications are becoming more immersive. Where once you could text someone, but then had to change to other software and hardware to call them, today’s unified communication solutions create a far more immersive environment. Now you can start by texting to each other, then switch to voice and/or video with a click, share applications and edit content together with a click and even add other users by click.

    How to Increase Immersion

    It stands to reason that the more human senses you stimulate, the more immersive the experience becomes. Adding image to sound as we moved from radio to television certainly resulted in greater audience immersion. Theaters are experimenting with combining 3-D projection with seats that move and shake in accordance with the on-screen action. Only the olfactory sense has not been treated yet, but how long can it be before the introduction of “smell-o-vision”?

    Effective creation of immersive experiences requires software developers, network designers and implementers, writers, photographers, graphic layout experts, subject matter experts, social engineers and behaviorists to work together effectively. The combined creativity in a team like that will, itself, be thoroughly immersive.

    — Brad Mallard