Category: Blogs

Software Development: A Better Way to Measure Success

Here’s a metaphor for software development you probably haven’t heard before: It’s like flying a plane. You have a starting point and a destination in mind, there’s a good chance you’ll change course midflight, and … sometimes you get a little nauseated?

Okay, it’s not the best analogy. But there is one aspect of piloting a plane that offers valuable insight into how to measure your team’s performance more consistently. It’s called the performance-control technique, and it may be the best method you’ve never heard of for keeping your engineering teams aligned behind a common goal.

Imagine for a second sitting in the cockpit of a plane, surrounded by clouds. Forget using the ground to orient yourself; you have nothing but your instruments and wits to guide you. It’s a situation I’m plenty familiar with, having taken flying lessons out here in rainy Seattle. In those cases, you use your altitude and power settings to set up a particular control scenario that results in your desired output, whether it’s ascending, descending, straight and level or turning. And then you monitor your performance—in this case via altitude, vertical speed and airspeed indicators—to verify that you’re achieving your desired output.

Now unless your software development pipeline is irredeemably broken, you are probably not flying completely blind as you work to build and ship new features. But the technique still applies: Choose your control metrics (time to release, for example), and then set up a series of performance metrics to keep it on track.

Cloudy Skies Ahead

The greatest strength of the performance-control technique is its acknowledgement that any control can result in unintended consequences. Take reducing the time from check-in to release. It’s a fantastic goal: The quicker you get code into production, the quicker you deliver updates to your customers. The quicker you do that, the quicker you get feedback and respond to their needs. The quicker you do that, the quicker you can iterate on features. And the cycle continues.

But it’s also a goal that can result in unintended consequences: When you single it out for attention, it’s only natural team members will want to do everything they can—at the expense of almost everything else—to achieve it. That’s not to say they want to do wrong; rather, they’re just trying to be good citizens. (And, okay, they don’t want their name to show up in a report about why the team didn’t meet its quarterly goal.)

So people make compromises. Maybe they drop full regression passes and replace them with incremental feature and integration testing. It’s not that they don’t care about quality. But in their quest to help the organization meet its single-minded goal of reducing the time from check-in to release, they opt for the easiest change that could get them there, indirectly prioritizing speed over quality in the process. And that happens because there aren’t checks in place to keep it from happening.

Taking Control

How do you put this into practice at your own organization? Start by choosing your goals and then selecting the control metrics. For many, it could be as simple as increasing speed; it’s a natural goal to chase, for the reasons already discussed. But it may be something different; only you and your teams can decide by being intentional in how you evaluate your production environment.

Then it’s time to choose your performance metrics for each of your goals. You can do that yourself, but it may be more illuminating—and, honestly, more fun—to get your software development team involved. That could range from encouraging them to brainstorm the negative consequences of implementing that goal to straight-up asking them how they’d game the system.

The following chart offers examples of control goals, their resultant unintended consequences, and the performance metrics necessary to keep them in check.

Desired Output	Control Goal	Performance Goal	Unintended Consequences
Climb to 4500’	Increase power, nose up	Airspeed, Altitude	Stalling
Reduce bounce rate	Reduce site latency	Revenue, Conversions, Customer Engagement	Smaller pages, poor experience, feature attrition
Generate more conversions	Increase availability	Releases, Backlog Ticket aging.	Lower frequency of releases, cherry picked work items, no risk taking.
Reduce MTTR	Speed up ticket closure	Re-open rate, new ticket rates.	Premature closure, delay in opening tickets for issues.

It’s important to note that how closely you monitor each of these performance metrics will vary, and in some cases significantly. Rather than attempt to apply a one-size-fits-all approach, instead think of each in terms of how long it takes to detect that the metric is falling out of its control envelope and how long it would take to recover if it did. If it takes one day to detect that you’re outside of your envelope and three weeks to recover, then by all means, check in daily. But if it takes three months to fall out of the control envelope and just a day to recover, you may only need to check it once every couple of weeks.

A Soft Landing

As we’ve supported software development teams in their efforts to modernize their software delivery methods, we’ve heard time and time again, “What’s the best metric for measuring success?” And we struggle to come up with an answer because there is no golden metric that works for everyone. But more important, even the good ones can lead well-meaning engineers and team leaders to optimize for one thing while letting others suffer.

So rather than search for a magic bullet, the secret is to find goals that work for your needs—and then build in metrics that keep you and your software development team accountable.

— Rob Duffy

June 15, 2018

DevSecOps: Embedding a Security Practice into your DevOps Approach

It’s a no-brainer that the element of security cannot be compromised even to the smallest of extents in today’s competitive, fast-paced, modern technology-driven IT infrastructure. However, to keep up with the rapid developments of other processes in this agile world, security is often given relatively less importance and in some cases, even left behind. As the term suggests, DevSecOps is primarily concerned with the incorporation of security in the DevOps pipeline.

The intended primary function of DevSecOps is to help overcome the aforementioned barrier by extending the conventional framework of DevOps and inculcating security testing by means of various security tools. This article attempts to explore the significance, characteristics, benefits and challenges involved in DevSecOps implementation and practice.

Why DevSecOps Matters

Since the rate of cybercrime over the past few years has been increasing at a significantly high rate, the need for adoption and implementation of DevSecOps is also intensifying. An analytical study from Cybersecurity Ventures predicts that the damages incurred from cybercrime will be as high as $6 trillion annually by the year 2021, double the $3 trillion in 2015. As the benefits reaped from implementation are linked directly to the reduction in cyberattacks, DevSecOps is becoming the center of attention of IT decision-makers.

Breaking Down DevSecOps

The following factors constitute the core of the DevSecOps approach and are the key to a successful implementation:

Automation

While the concept of DevOps revolves around automating the build, test and deployment sections, DevSecOps additionally focuses on automating security. Automation is crucial as the security—besides being able to be scrupulous and comprehensive—also has to catch up with the much faster release cycles driven by DevOps. The target of DevSecOps is automating all of the security controls, thus eliminating the need for manual interference.

People, Process and Technology

The trio of people, process and technology is the pillar and it directly influences the extent of success of any DevSecOps approach and practice. The People—considered as the weakest link of the three—are the security specialists and integration of the security team with the development team. Appointing “security champions” who form a cross-functional team to work on the application security, is a key element of the People practice. The Process involves standardizing the workflow, documentation and execution of the same to make sure that the security is transparent with other processes in the workflow. The Technology refers to the various facets deployed in DevSecOps such as the automated vulnerability management, automated compliance scan, etc., whose applications are directly involved in the implementation.

Different Tools for Different Functions

There are a number of security tools that specialize in various aspects of the DevSecOps approach, including testing, secrets management, attack modeling and red team. Selecting the right tool for the right function is paramount and it is not always easy since many of them are still in the emerging phase.

With DevSecOps, You Get …

Enhanced overall security: The overall security of the infrastructure is strengthened by identifying and reducing vulnerabilities as and when they occur. In case some minor breaches happen, the rate at which the recovery happens is also escalated.

Total Cost Reduction: Unlike the conventional approach, in DevSecOps the security issues are identified and dealt with in the development phase itself. Consequently, this leads to reduced overall costs incurred in the development and security of applications.

Accelerated delivery speed: The implementation of DevSecOps consistently strives to detect and eliminate the security bottlenecks at various stages of development. This, in turn, increases the speed at which the product is delivered.

Besides the above-mentioned benefits, DevSecOps also nurtures an environment of transparency and helps in increasing the customer value.

But, Implemention is not a Piece of Cake

Comprehensibly, there are many challenges that lie ahead before DevSecOps implementation. The availability of sufficiently skilled cybersecurity professionals is one of them. This is because the extent of expertise needed in cybersecurity is underrated when compared to other business units. Unlike the traditional environment, a DevSecOps methodology means that the various internal teams of the business unit such as the development and security teams need to work in unison, which might not always go smoothly. The fact that building secure code is time-consuming can frustrate developers, as it hampers their speed of delivery. Many mid- and low-level organizations are skeptical about security as, economically, it is viewed as more of a liability rather than an asset.

In my next article, I will be applying a templated approach to shortlist tools that would be suitable for your organizational needs. For illustration, a series of webinars that cover the various aspects of implementation of DevOps can be found here.

According to the WhiteHat Security Application Statistics Report, “The average customer takes 174 days to fix a vulnerability found when using dynamic analysis in production. However, those who have implemented DevSecOps do it in just 92 days. If we look at vulnerabilities found in development using static analysis, an average company takes 113 days, while the DevSecOps companies take just 51 days.” It is evident that, in the long run, the benefits reaped from DevSecOps outweigh the challenges. A successful approach drastically reduces the chances of succumbing to cybercrimes while being agile and revolving around the mindset, “Everyone is responsible for security.”

— Kavin Elango

June 14, 2018
Puppet Report Sees Long DevOps Journey Ahead

A recent report published by Puppet suggests the level of DevOps maturity within most organizations is still relatively slight. In fact, most of the respondents reported high levels of manual work still occurring across configuration management, deployment, testing and change approval processes.

Alanna Brown, director of product marketing for Puppet, said the “State of DevOps: Market Segmentation Report” makes it clear that DevOps is being embraced to solve specific pain points with specific IT departments. As a result, silos of DevOps processes are springing up across the enterprise. That next challenge IT organizations will face is trying to unify all the DevOps processes, she said.

The survey finds deployment automation, version control, continuous integration and infrastructure automation were the most common starting points for DevOps. Less-common starting points were lean practices, visualization of key quality and productivity metrics and setting limits on work in progress.

Brown said the one surprising thing that study found is there is little correlation between the size of the companies surveyed and their relative DevOps maturity. Reliance on DevOps process may have more to do with needs arising from satisfying customer demand than anything else, noted Brown. The survey finds that media and entertainment and retail industry have the lowest percentage of low IT performers, while financial services, insurance and manufacturing companies had the highest proportion of low performers.

Not surprisingly, given the relative amount of DevOps immaturity, the survey also finds that reliance on IT automation frameworks is not yet especially widespread.

The Markets Segmentation report is based on the 2017 “State of DevOps” survey of 3,200 IT professionals that Puppet previously published. While it’s possible many companies may have advanced further down the DevOps journey since then, the report makes it clear that for both technical and cultural reasons fully embracing DevOps is an extended process. In fact, Brown said the report shows DevOps is still mainly being driven by the bottom of the organization up rather than by senior IT leaders. It’s only when DevOps success becomes visibly demonstrated that organizations attempt to replicate that success elsewhere in the organization. At the same time, widespread usage of DevOps processes suggests that DevOps itself is now much more than a grassroots movement, she said.

Change obviously comes hard to enterprise IT organizations. Many DevOps concepts were initially pioneered by web-scale companies that have the luxury of not having to support legacy applications and processes. Traditional enterprise IT organizations now typically are trying to support DevOps processes alongside process based on Waterfall and ITIL methodologies. Not every application being supported requires that number of release updates that many modern applications require.

Nevertheless, it’s also clear that as the number of applications being built, tested, deployed and supported continues to grow, the average IT organization is increasingly short-handed. Finding IT professionals with DevOps skill sets is a major challenge. But as more companies come to realize they are really a software company that happens to be selling a specific thing, the higher priority DevOps becomes.

— Mike Vizard

June 11, 2018
There’s No Such Thing as DevSecOps
Should DevSecOps be categorized separate from DevOps?

After years of debate, few doubts remain about the viability of the public cloud for enterprise workloads. Enterprises are shifting to cloud-first and cloud-only strategies, looking to take advantage of all the good the public cloud has to offer. For applications built and deployed in the public cloud, it is all about automation. The cloud enables you to be elastic, reduce costs and deliver products faster. To achieve these goals, you need to automate your processes—building, testing and deployment.

Security Challenges of DevOps

This need for faster innovation led to the rise of DevOps, a set of practices and tools that result in tighter collaboration between development and operations. In the past, controlling the deployment processes was the responsibility of dedicated teams that wrote scripts. However, with DevOps, development and deployment merge. Software engineers must do more than write their infrastructure and business logic code; they also need to write the automation that surrounds it and take responsible for deploying it through the different stages until production.

However, even as development and operations come closer together, security has remained a function apart. While DevOps workflows are heavily automated, security checks for application and infrastructure code often are manual. DevOps and security teams are measured differently—delivery versus protection—and often are at odds with each other. The siloed approaches to security hardening that worked in the past are incompatible with the holistic, iterative software development and deployment model of DevOps.

For a developer, adding the responsibilities associated with DevOps is not complicated, but securing the environment is a different level of challenge that requires mindset shifts in multiple areas:
1. Knowledge: Software engineers are great at writing business logic. They write efficient code and develop highly available environments that can handle millions of concurrent users. However, the vast majority of developers don’t know about the latest CVEs or the proper configurations of SSL/TLS in the load balancer.
2. Pace: Successful development organizations move fast, frequently adopting new development trends and frameworks. Public cloud vendors release new features every week, and it can be difficult to follow and keep up, as engineering is usually measured by the ability to deliver and the quality of output. DevOps enables rapid release cycles, and the security must not hold them back.
3. Scale: Modern applications make the transition to work with microservices. When each development team owns its cloud environment (and sometimes multiple environments), the security teams suddenly find themselves dealing with huge distributed deployments. They can’t manually configure security when new features are released a few times a day in different environments.
4. Heterogeneity: Multi-cloud deployments are another emerging trend. New product teams are often given a free hand choosing their tools and infrastructure, and they may pick a new cloud vendor. While security teams have a voice in this decision, they do not have the last word. It may have taken the security team years to learn to deal with the existing cloud solution, and suddenly the organizational policy needs to be applied across different infrastructures.
5. Consistency: Enforcing organizational policy across different development teams is challenging. Each team has its own style, environment and development frameworks, meaning it’s never easy to ensure the security is applied uniformly.
However you choose to address all these challenges, one thing must be clear: Your DevOps cycle is not complete without handling the security.

Choosing a Security Solution

Clearly, you can’t deploy services without properly securing them—you don’t want anyone to hack into your systems and steal your (and your customers’) information. Since cloud environments are dynamic and deployments tend to span across dozens of accounts, manual steps are not an option. Automation is necessary. Whether you acquire an enterprise security solution, adopt an open source security project or write your own tools, you need to make sure that they are:
1. CI/CD ready: The solution can easily integrate into the continuous integration/continuous compliance pipeline. User-friendly APIs are required.
2. Scalable: Assessing hundreds of cloud environments requires the solution to be scalable and robust. When covering more cloud vendor capabilities, performance becomes critical and increasingly challenging.
3. Flexible: Each organization is different, and security needs can vary with each deployment. The baseline configuration of your security solution will never meet your requirements by 100 percent, so it’s important that you be able to adapt it to your needs.
4. Continuously updated: Cloud vendors add new features at a rapid pace, which also means new potential misconfigurations are generated all the time. Whatever solution you choose, it needs to match the cloud vendors’ speed and agility. Security solutions that are not being updated soon become useless.
5. Conveniently Reportable: Aggregated reporting is critical. Consuming the security findings in a convenient manner can assure that issues are properly mitigated.
6. Automatic remediation-capable: Many of the findings are easy to remediate automatically. Applying automatic remediation would enable the security team to focus on complicated tasks and reduce alert fatigue.
DevOps and DevSecOps Are (or Should Be) the Same Thing

Security is clearly an integral part of DevOps. DevSecOps is emerging to bring security and compliance management practices in line with the rest of DevOps. DevSecOps serves to call out the temporary chasm that exists between security and DevOps.

As security checks and controls get automated and integrated, DevOps and DevSecOps are converging to the same thing. Just like adjusting the mirrors and putting on the seat belt are part of standard procedure before pulling your car out of the driveway, assessing your security posture before releasing a new version of your product must become habit. To remain relevant, your security team must integrate into the CI/CD and make sure the newly created environments meet organizational policies.

— Offir Zigelman
June 7, 2018
Living in a DevOps World (Part 1)
The concept of DevOps has grown and evolved into a conceptual and working model for more effective and efficient software development and implementation. That said, there are some differences of opinion on the real-world value of any DevOps approach to date, and on the best way to create and implement a real-world DevOps environment. This two-part article will focus on what an agile DevOps approach is meant to address, and what it is not meant to address.

DevOps sits at the nexus of three essential business technology functions: software development, quality assurance and operations. A short and concise definition of DevOps proposed in 2015 seems as appropriate as any:

DevOps is a set of practices intended to reduce the time between committing a change to a system and the change being placed into regular production while ensuring high quality.

The definition was suggested in the book, “DevOps: A Software Architect’s Perspective,” and the authors have hit upon the essence of the practice. The key, of course, is how to put that concept into practice.

Perhaps the first step on the journey to effective DevOps is the recognition that the concept is the result of the rise of the Lean and Agile software development methodologies. Those methodologies, among other things, emphasize the following:
- A focus on customer value.
- The elimination of waste.
- Reduced cycle time (accomplishing work faster, releasing faster).
- Shared learning.
- Avoiding batching (don’t do things until required).
- Theory of constraints (break things up, focus on individual issues).
- Continuous integration, testing and delivery.
- Faster time to market.
DevOps in Practice

Adherence to the principles above meant that something had to be invented to accomplish them and that something was DevOps. Over time, an effective DevOps practice should address any number of business technology pain points. The following short list of those pain points and the DevOps response should prove instructive.

System Downtime

The developers’ curse since systems were developed, system outages are inevitable as long as systems are designed, tested and implemented—even with increased automation—by imperfect beings. DevOps acknowledges that by changing the focus from trying to create applications that never fail to designing systems that can recover quickly, thus decreasing aggregate systems outage time over the life cycle of any application or system.

Stagnation

This was a staple of traditional systems development and is most closely associated with the waterfall methodology for systems development. After requirements were created, the development team would be locked away for weeks, months or, in some cases, years before emerging with “fully” working software that inevitably no longer satisfied rapidly evolving business requirements. DevOps is designed to fit hand-in-glove with the Agile practice of short windows of incremental changes instead of long release cycles, putting working software in the hands of customers as quickly as possible.

Team Conflict

Having been borne from the cultural combination of Agile and Lean, DevOps has taken on the problem of functional silos that are often erected between development, operations and the business customers. It follows the methodological approaches of collaboration and teamwork first to understand what others know and to leverage the best of it to solve business problems more rapidly. There is also a cultural bent toward experimentation, continual learning and constant improvement. This leads to blameless post-mortems, where instead of finger pointing when something goes wrong there is collaborative discussion and learning to correct and prevent the problem from occurring again.

Knowledge Silos

Functional silos have led to compartmentalized knowledge. If the old game was that knowledge is power, the new game in the DevOps world is that knowledge is freely exchanged as an enabler to solving business problems. DevOps addresses the problem of information being lost in translation between the development and operations functions by eliminating the functional barricades and making knowledge sharing the highest form of collaboration.

Inefficiency

Waiting for things to happen used to be a standard operating procedure in the pre-DevOps world. Project plans were created and managed to account for the time it might take for new code to be moved into a testing, quality or even production environment. This was a momentum killer for projects and at times a morale killer for developers waiting to see what changes they might need to make to their code set. The combined Agile and DevOps approach has rewritten the traditional approach to code migration, smoothing and eliminating wait times so that projects flow more seamlessly from start to finish. It also has the benefit of keeping business resources—testers, approvers, etc.—more engaged as a result of a constant flow of new functions and features to test and use. And lest this be viewed as simply a way to keep the technical stuff moving along, it’s important to remember that there is a financial aspect to this as well. Reducing speed to market with new functionality, reducing or eliminating idle hands—be they technical or business—and delighting customers with a steady stream of enhancements and features all go directly to an organization’s top and bottom lines.

That, after all, is in many ways what the DevOps approach is all about. All of these critical areas become the means to accomplish it. Part two of this article will focus on some more of the benefits of a DevOps approach, and how to achieve them.

— Dave Farinelli
June 5, 2018
Case Study: Time Series Helps PipelineFX Customers Save Time, Money

Digital rendering billing and licensing issues have been reduced by a time series solution

PipelineFX helps organizations of all sizes better manage the cost of rendering for digital media applications and programs. For more than 16 years, organizations of all sizes in the film, visual effects, post production, broadcast, design and education industries have been using PipelineFX’s Qube! to manage their render pipeline and make critical business decisions, faster and smarter than the competition.

Time Series Solves Customer Problems

PipelineFX wanted to help its customers meet their digital media deadlines without having to spend the capital to build and maintain their own render farms. The steps a graphics system needs to perform to render a 3D scene to a 2D screen is a very “bursty process,” so one customer may need 10,000 cores one night and only a couple of hundred the next night. In addition, customers’ deadlines are not variable, yet the time needed to render each project is very different, based on the level of computer graphics, layers per shot, resolution of the rendered project and the number of frames per second. To meet a deadline, customers may need to scale from 500 to 1,500 machines for three days. But after those three days, customers don’t want to pay for rendering licenses that are not being used. Their job is finished, so they should not need to pay!

Real-time view of charges displayed in a monitoring dashboard

What makes the PipelineFX offering so compelling is the ability to measure utilization every minute and bill at a similar time interval. This high level of precision ensures that PipelineFX customers get a great return on their investment. With this data, PipelineFX is able to provide a very flexible pricing model that can accommodate this high-frequency charging for any license utilization over a certain per-paid minimum. In addition, the system needs to provide reports and insights into what average utilization rates are and whether a customer should consider increasing its minimums.

In addition to solving the problem of rendering licenses during bursty processes, PipelineFX wanted to enhance its SaaS infrastructure to optimize customer demand and server costs. PipelineFX customers need to be able to reserve the right number of servers and nodes to hit their peak demands, but they don’t want (or need) to purchase perpetual licenses for these short-lived servers.

A comparison of installed vs. consumed licenses displayed in a monitoring dashboard

Since PipelineFX wanted to measure and bill at a high-frequency time interval, as well as determine real-time peak demands, its traditional relational database was no longer proving effective.

To gather this high-frequency data, PipelineFx deployed a time series solution. Customers’ rendering data today is collected each minute and transmitted to the PipelineFX service every 15 minutes. If customers lose network connectivity, the application accumulates the batch and, once connectivity is restored, the whole batch is transmitted. Often, eight to 12 hours of data is uploaded—some late in arriving, but the system takes care of this as well. PipelineFX uses this data store to generate customer dashboards and provide reporting of real-time usage.

PipelineFX’s extensive experience with monitoring tools and software development made it an easy decision to use open source projects in its product. The company found early success using open source time series projects to collect and analyze the data needed for its real-time billing offering. Although it was easy to use, PipelineFX was spending valuable development cycles maintaining the solution and decided to review the cost of hosting the solution themselves versus purchasing a hosted version. In its review, PipelineFX found it made better financial sense to go with a hosted version and rely on vendor expertise to keep the software up to date, keep the clusters fully optimized and rely on a 24×7 uptime.

Results

PipelineFX has been able to help its customers achieve maximum efficiency and optimize their existing and future infrastructure. The combination of solutions has allowed for metering and billing at minute-level precision. In addition, customers now have a real-time view of their current usage and billing charges, which enables them to maximize their investment in Qube! without having to commit in perpetual licenses that may go unused.

— Chris Churilo

May 31, 2018
Using CALMS to Assess an Organization’s DevOps

Of the tried-and-tested frameworks that allow enterprises to assess DevOps in their organization and how it can be improved, the CALMS model remains particularly useful.

CALMS, which stands for Collaboration, Automation, Lean, Measurement and Sharing, is particularly helpful for analyzing an organization’s DevOps structure, and ultimately, its utility in any organization. The CALMS framework covers all stakeholders within DevOps, including business, IT operations, QA, InfoSec and development teams, and how they collectively deliver, deploy and integrate automated processes that make business sense.

“DevOps as a technique is slowly but steadily maturing, so it’s good to see general assessment methods like the CALMS model,” Holger Mueller, an analyst for Constellation Research, said. “The CALMS model gives a good frame of reference to compare the maturity of a DevOps team, and as such, is invaluable for assessing the state of teams for the transformational change that goes with it.”

Applying CALMS

Here are a few ways CALMS can be applied to DevOps for enterprises either thinking about shifting to a DevOps structure or seek to make improvements to an existing deployment.

Culture

It is long-established that technology adoption must be to serve a business need rather than investing in technology for its own sake. This shift in mindset applies to how the “culture” component of CALMS supports how the forecasted ROI associated with automating a process must be championed across the DevOps team. This mindset is especially important for getting the non-technical business teams onboard, Mueller said.

“DevOps must have the backing from the business executives. That’s a crucial factor in the culture part,” he said. “One thing software teams can do is use successful software deployments to help convince the business executives in DevOps to make the investments. Those are the kinds of tribal events that every DevOps organization needs.“

Automation

Netflix is well-known as an exemplary case study for DevOps automation. Among other things, it can deploy code and software updates in a matter of minutes across its entire massive infrastructure, many times throughout the day. But Netflix’s example of automation does not take into account what can go wrong at firms that want to follow Netflix’s example when automating their software deployments and updates, Mueller said.

“The risk is always deploying too fast and you end up deploying bad code,” Mueller said. “Automation, of course, needs to take this into account and compensate. But not everybody can do that.”

Lean

Once completed, it is relatively easy to document cost savings by eliminating resources. Within the CALMS framework, the corresponding “lean” component in DevOps is thus relatively easy to assess, Mueller said.

The good thing about DevOps, Mueller said, is companies can come to a consensus ahead of time regarding what they consider to be a lean operation.

However, he added, “There is always the risk of being so lean you begin to err on the QA side.”

Measurement

The success of software deployments is relatively easy to measure once completed. However, attempting to measure future deployments as part of a forecast model is, of course, much more challenging.

“It can be surprisingly tricky to measure how good DevOps is ahead of time,” Mueller said. “Measurements should thus allow errors to be pinpointed quickly during the post-deployment stage to fix the code if needed very quickly.”

Sharing

Sharing, in many ways, overlaps with culture. This is where assessing whether all of the different teams within DevOps are working together and communicating comes into play.

“Thanks to DevOps, the days of the lone developer are long gone. But sharing is not always easy,” Mueller said. “Even within separate teams, it is easy to fall into the trap of wanting to get your own work done without necessarily thinking about the team as much as you should.”

The End Result

While applying CALMS to assess DevOps never hurts, the framework is still not as important as one of the main objectives of DevOps—namely, using technology to make organizations’ operations more agile and reactive to customer needs, Mueller said.

“At the end of the day, assessment methods such as CALMS are great, but they are never as important as focusing on implementing next-generation applications,” he said.

— B. Cameron Gain

May 25, 2018
Microservices Momentum Accelerates

A global survey of 353 application development professionals conducted by Dimensional Research on behalf of LightStep, a provider of application performance management (APM) tools, finds a full 86 percent of respondents expect microservices to be the default architecture within five years.

A total of 60 percent of respondents said they already have microservices in production or deployed as part of a pilot, and 92 percent said they expect to grow their use in the coming year.

Top reasons cited for employing microservices are agility (82 percent) and scalability (78 percent).

The downside of increased reliance of microservices is the need to manage them all. The survey finds 99 percent report challenges in using them. More than half (56 percent) reported that each additional microservice increases operational challenges, and 87 percent of those that have deployed microservices in a production environment reported they are generating more application data.

To make matters more challenging, a full 98 percent of respondents said they have trouble identifying the root cause of performance issues associated with microservices. More than three-quarters (76 percent) said it also takes longer to resolve issues. In fact, 73 percent said it is more difficult to troubleshoot application performance problems compared to a traditional monolithic application.

Not surprisingly, just less than three-quarters (74 percent) said they plan to increase investments in microservice performance management tools in the next year.

LightStep CEO Ben Sigelman said that while microservices come in many forms, advances in how software is developed is clearly coming at a cost. Not only is software becoming more challenging to manage, APM tools are now a prerequisite. Historically, IT organizations only monitored their most mission-critical applications. But the level of dependency each microservice has on others requires a more holistic approach to APM, Sigelman said, noting that IT teams now need to be able to see well beyond the border of any one microservice.

Challenges associated with managing microservices are not likely to deter organizations from eventually adopting them. The level of agility and resilience inherently built into in a microservices-based application are too high to ignore. But the rate at which they are employed might be impacted. There’s usually a direct relationship between the availability of tooling and expertise and the rate at which any emerging technology gets deployed in a production environment. The good news is advances in machine learning algorithms and other forms of artificial intelligence (AI) should make it easier to manage microservices in the years ahead.

In the meantime, IT organizations can take some comfort in the fact that developers are being held more accountable for the microservices they build. That level of accountability tends to vary widely by organization. But the end result is generally higher levels of software quality because developers increasingly realize they can’t throw code over the proverbial IT operations wall without it coming back to them. The challenge now is to bridge the historic divide between developers and IT operations teams in a way that works best for everyone concerned.

— Mike Vizard

May 10, 2018
How DevOps Can Create Smarter, More Agile 5G Telecom Networks
There is no doubt DevOps revolutionized software development with its introduction of lean and agile techniques. By applying continuous deployment and continuous integration methodologies into software development, enterprises can build, test and deploy applications and release new services more rapidly. In this article, we will see how the DevOps approach is being utilized in 5G telecom networks with the rise of software-defined systems.

Emergence of Software in Legacy Telecom Networks

In last few decades software has changed our lives in myriad ways—mostly for the better. Now, software is starting to disrupt the telecommunication industry with the eagerly anticipated 5^th generation of telecom network, also known as 5G. It is expected to provide low latency and high bandwidth network connections to serve use cases including artificial intelligence, smart city, 3D video, industry automation and self-driving cars, and to provide end-to-end real-time experiences such as virtual reality and augmented reality. A variety of these use cases have different network requirements and dependencies, and 5G is expected to provide such flexibility along with agility.

Software-defined networking (SDN) and network functions virtualization (NFV) form the backbone of the upcoming 5G network, enabling service providers to deliver lucrative services to their customers and realize a competitive advantage. SDN makes the network programmable and centrally controlled, while NFV replaces dedicated network appliances with software packaged in virtual machines on top of high-performance servers. SDN and NFV enables:
- Better control over overall network.
- Reduction of manual network configuration processes.
- Lower CAPEX.
- Network resiliency.
- Orchestrate network dynamically.
- Faster time to market, i.e. faster introduction of new services and features to customers.
Application of SDN and NFV technologies in 5G telecom networks indicates the use of software in the legacy telecom network, wherein many network functions including firewall, content delivery network (CDN), network address translation (NAT), routers, deep packet inspection (DPI) and broadband remote access server (BRAS) can be virtualized and serve as virtual network functions (VNFs) in NFV architecture. This stands as the core feature of upcoming 5G networks, as most of hardware-dependent operations will be performed by software, providing higher performance with tons of benefits to end users.

Cloud-Native Enables DevOps

Virtual network functions (VNFs) are the software representation of network function appliances in NFV environment and a core element in 5G networks. Services can be provided by chaining different VNFs developed by different vendors. There are complexities associated with having multivendor VNFs, as each VNF has its own performance and operational issues including manual integration in NFV infrastructure, utilization of NFV hardware resources, lack of interoperability and multitenancy. For a 5G network to deploy successfully, each VNF must be configured and managed in automated way.

As software becomes part of telecom network, the term “cloud native” becomes part of the vernacular for VNFs.

By harnessing a cloud-native approach—including containerization, microservices-based architecture and dynamic orchestration—VNFs with their own life cycle enable:
- Decomposition of sub processes into microservices—resources required will be automatically scaled and communicated with other services using APIs.
- Management and simplified orchestration of VNFs.
- Deployment within containers and connected by APIs.
- Orchestration by container scheduler—manages provision into service and lifecycle management.
- Portability to different NFV environments.
More importantly, a cloud-native approach allows VNF vendors and service providers to impose DevOps (continuous deployment and integration) methods to automate process of building, validating and deploying VNFs into NFV environments.

A DevOps approach in 5G networks brings benefits for overall network operations, enabling deployment automation and management of service chains and 5G network slices. Complete automation is required, especially for 5G network operations, which may increase operational efficiency and decrease human error, saving OPEX and decreasing expensive downtime.

For 5G networks, requirement of DevOps methods in VNF can be for:
- Automated onboarding and deployment of VNFs in NFV environment.
- Automated performance testing and benchmarking for VNFs.
- Enabling self-scaling of VNFs—automated resource provisioning for VNFs.
- Rnabling self-healing after an error in VNFs residing in VMs or containers.
- Real-time monitoring of VNF activities for analysis.
More ‘Intelligent’ Network

Cloud-native capabilities and a DevOps approach opens the possibility of incorporating artificial intelligence in 5G telecom network operations analysis and real-time monitoring. Cisco Systems has already introduced its solution based on machine learning, which it calls Intent Based Networking. We can imagine a network operation having “zero touch” feature by just providing an intent to complete network for any configuration or specific network-based instruction. The collected analysis data can be used in the management and organization of NFV for corrections of network errors, adjustments in VNF and NFV elements configuration and self-healing/scaling of network functions.

Summary

Development of 5G network is still in process. By taking into account a revolution in SDN, NFV, cloud native and DevOps, it is expected that we will have 5G networks by some of leading players Verizon, AT&T, Telefonica and Vodafone reasonably soon.

— Sagar Nangare
May 8, 2018
451 Research Report Highlights Value of DevSecOps for the CI/CD Pipeline

One of the compelling benefits of DevOps for many organizations is speed. Between breaking down silos, streamlining processes and automating routine tasks, the process of developing and deploying software can be significantly faster. A new report from 451 Research and Synopsys, however, points out the value of incorporating DevSecOps in the equation, especially with continuous integration/continuous delivery (CI/CD), and the pitfalls of prioritizing speed without also considering security.

For the “DevSecOps Realities and Opportunities” report, 451 Research surveyed 350 enterprise IT decision-makers across the United States and Europe, focusing on organizations that have implemented CI/CD. The goal was to learn more about how organizations are implementing CI/CD and the role security plays in that equation.

For most organizations, embracing DevOps and adopting CI/CD is about speed, not security, the survey found. Shocker! As per usual, security is typically an afterthought—something that might be nice to do if time and resources permit.

Survey respondents indicated speed as a leading driver for and expectation from CI/CD. “The most organizations (36 percent) sought a 4x factor of improvement in time to deploy from their CI/CD workflows. A significant number (15 percent) sought a 5x factor of improvement. Another 7 percent sought more than a 5x factor of improvement, and 5 percent were seeking a 10x or greater factor,” according to the report.

The irony is that organizations seem to prioritize speed over quality and/or security at their own peril. Maximizing for speed in the CI/CD pipeline without consideration for security typically leads to more bugs in the code, more downtime and increased time and effort to maintain and remediate applications.

451 Research points out that as DevOps implementations mature, the focus will shift to what’s truly important. “We believe that as organizations evolve these processes, the focus will naturally move from ‘how fast can development teams iterate’ to ‘how reliably can we deploy high-quality, appropriately secure code each and every time,’” the researchers wrote. “That critical change in focus—that more mature view of risk management—will realign everyone’s expectations on what ‘4x’ really means.”

Of those organizations that participated in the survey, only about half actually include some sort of application security testing as a function of the CI/CD workflow. Identifying open source software components with known vulnerabilities—also known as software composition analysis (SCA)—is ranked as the most crucial security component for CI/CD by survey respondents, yet nearly 40 percent of the participants admitted that they don’t do any SCA or insist they simply aren’t using any open source code. That seems highly unlikely, though, given that a recent report from Black Duck Software, “Open Source Security and Risk Analysis,” found that 95 percent of applications contain open source elements.

“While some DevOps teams are starting to incorporate application security into their CI/CD workflows, driven by factors such as improved software quality, compliance and risk avoidance, there is ample room for improvement,” said Jay Lyman, principal analyst at 451 Research. “In many cases, security testing is not being integrated often or early enough in the process for organizations to fully benefit from reduced risk and rework headaches.”

With security, it always seems a bit like the Bill Murray movie, “Groundhog’s Day”: No matter how many years security professionals have pushed, and fought, and screamed from the mountaintops that security needs to be woven into the foundation of software and integrated into the development life cycle rather than being tacked on with duct tape and chewing gum after the fact, organizations continue to insist on doing the things the harder—and more expensive—way. Hopefully, reports such as this will open the eyes of some organizations and they will break the cycle and start prioritizing security, recognizing that it is actually the best way to develop reliable code as quickly as possible.

— Tony Bradley

April 27, 2018