devopsy.kinsta.cloud

Tag: devops

Executive Buy-in May Be Preventing DevOps Success
A common misconception in the executive suite is that an organization can reap all the benefits of DevOps without fully embracing the change necessary. That’s understandable, given that a DevOps transformation is a lengthy, costly disruption for companies of all sizes. DevOps requires businesses to transform, from team organization to culture to development processes. This shift often sparks reallocation of power and influence, which sometimes can make high-level executives uncomfortable. While most companies have begun to recognize the overall business benefits of DevOps, few are fully committing to such a large undertaking. As a result, a hesitant executive suite holds an organization back from realizing the full potential of DevOps.

The key to successful DevOps implementation has four facets. Executing against one element just isn’t enough—an organization must combine all of them to realize the benefits of DevOps:
- Training: Once executives have educated themselves on the importance of DevOps and accepted the disruption that comes with it, they must make sure the individuals are properly trained. Everyone involved needs to be open to learning new skills and new ways of functioning. A big part of DevOps is breaking down team silos, so individual contributors need to be willing to work across functional boundaries.
- Tools: DevOps adoption can be viewed as a cultural and methodological change, and therefore teams also need the right tools that help them work together to create scalable and secure applications efficiently. The executive suite should invest in platforms that support automation and collaboration.
- Support and determination: To provide support and guidance throughout the shift to DevOps, organizations should consider hiring a DevOps expert or bringing in a third-party consultant. This person can help the executive suite map out a DevOps strategy and help train the teams involved. Lack of expertise is one of the reasons businesses often fail to complete a DevOps transition.
- Cross functional and accountable teams: A DevOps team should have all of the resources necessary to implement and deliver functionality to the customer. They should also be accountable for operating and supporting it.
Another important consideration is security. The CISO and security team are responsible for security in most organizations, but developers need to understand the critical role they play in creating secure applications. They need to be accountable for the security of the software they write. It’s up to the security team to train and nurture them until they are self-sufficient. The development organization as a whole must be willing to make security part of the annual goal-setting process and to report on progress transparently at each level of management.

Every DevOps implementation begins with education, from management to the individual contributor. Educational resources increasingly are available as DevOps continues to gain momentum in the industry. If faced with a hesitant executive suite, teams should leverage proof points from other companies that have successfully implemented DevOps. Leadership should be ready to measure, experiment, iterate and, most importantly, stick with it. When armed with the right training, tools and support, organizations can gain a competitive edge with DevOps.

— Pete Chestna
September 28, 2017
DevSecOps and DevNetOps: New Heroes in the DevOps Saga

The evolution of DevOps is by no means done, but it’s safe to say that there is enough agreement and acceptance to declare it a hero. DevOps has helped glorify IT to the point where it’s no longer preventing business, nor a provider nor a partner of the business.

Often IT is the business, or its vanguard for competitive disruption and differentiation.

Splintering the success of this portmanteau hero, we now hear more and more of two trusty sidekicks: DevSecOps and DevNetOps. Lesser known in their adolescence, these tots are still frequently misunderstood, are still forming their identities, and still need a lot of development if they’re to enter the IT hall of fame like their forerunner.

Just as the terms look, DevSecOps and DevNetOps are often assumed to be about wrapping DevOps principles around security and networking: operators hope to assuage technical debt and drudgery by automating in proficiency and resiliency. For networking, I’ve covered how there is a lot more to that than coding, but to be sure, these sidekicks certainly espouse operators learning how to do develop while DevOps was equally, if not more, about developers learning to operate.

The Shift Left: SecDevOps and NetDevOps

As if it wasn’t hard enough to tell what DevSecOps and DevNetOps want to be when they grow up, we’ve gone and given them alter egos: SecDevOps (aka “rugged” DevOps) and NetDevOps. Think about them exactly as the words look—it’s about the shift to the left. Left of what?

Traditional DevOps practices focus on business-specific applications development. The development timeline is known as a concept to cash, and with all the superpowers of DevOps we try to reduce our enemy—the lead time and repeatable processes between code and cash.

Security and building infrastructure, like networks, were considered supporting tasks, not revenue-generating nor competitive advantages. Thus, security and networking were far to the right on the timeline with concerns that deal with operational scale, performance and protection.

Today’s shift left propels security and infrastructure considerations earlier down the timeline, into coding, architecture and pre-production systems. It’s a palpable penny-drop amid daily news of security breaches and infrastructure outages causing technology-defined establishments to bleed money and brand equity.

Fill the bucket with cash, but don’t forget to forestall the leaks!

DevOps and Infrastructure: Challenge and Opportunity

Automation sparks have flown over the proverbial wall into the camp of I&O pros. Operators trading physical for virtual, macro for micro, converged for composed, and configuration for code is proof that the fire has caught security and networking. Controlling the burn now, is key, so that healthier skills and structures arise in place of the I&O dogma and duff. Fortunately, this is precisely the destiny for our newfound heroes, DevSecOps and DevNetOps.

However, doing DevSecOps and DevNetOps, embracing security and networks as code, we mustn’t be so credulous as to forget the formidable DevOps practices and patterns that need transforming along the ultimate automation journey. Testability, immutability, upgradability, traceability, auditability, reliability and other you-name-it-abilities are not straightforward to achieve.

Discounting “aaS” technology consumed as a service, a fundamental challenge to innovating SecOps and NetOps, compared to application ops, is that applications are crafted and built; security and networking solutions are mostly still bought and assembled.

Security and network infrastructure as code is something that needs to be co-created with the vendors. Other than in the cloud, it will take a while before security and networking systems are driven API-first, and redesigned to offer simulation, finer disaggregation into microservices, and orchestration with scale and resilience.

While this will land first in software-defined infrastructure, there is still a way to go to manage most software-defined security and networking systems with continuous practices of artifact integration, testing and deployment. Hardware and embedded software will be even more challenging.

Finding Strength in Challenge

So on one hand, DevOps is evolving with security and networking shifting left. On the other hand, traditional security and networking ops are transforming with DevOps principles.

Is the ultimate innovation to squeeze out those traditional operations altogether? Does NetDevOps + DevNetOps = DevOps?

There is a parallel train of thought and debate, with success on both sides. Purist teams cut out operations with the “you build it, you run it” attitude. Other companies such as Google have dedicated operations specialist teams of SREs. While the SRE reporting structure is isolated, SRE jobs are very integrated with that of development teams. It’s easy to imagine the purist approach, subsuming security and networking into DevOps practices, but only if we assume the presence of cloud infrastructure and services as a platform. Even then, there is still substantiation for the SRE.

Layers below, however, somebody still needs to build the foundations of the cloud IaaS and data center hardware. As they say, “Even serverless computing is not actually serverless.”

Underpinning the clouds are data centers. And then there’s transport, IoT, mobile or other secure networks to and between clouds. In these areas, it’s obvious there is a niche for our two trusty sidekicks, DevSecOps and DevNetOps, to shake up ops culture and principles. These two heroes can rescue software-defined and physical infrastructure from the clutches of so many anti-pattern evils, like maintenance windows and change controls (ahem, it’s called a “commit”).

We may not require rapid experimentation in our infrastructure, but we would warmly welcome automated deployments and updates, failure and attack testing drills, and intent-driven continuous response. They will boost resiliency and optimization for the business and peace of mind for the builders.

Teams operating security, networks and especially clouds, need to honor and elevate DevSecOps and DevNetOps, so that on the journey now afoot, our teams and our new heroes may realize their potential.

About the Author / James Kelly

James Kelly is a lead expert architect on cloud and software-defined networking at Juniper Networks. He also serves as a private yoga instructor, and previously, he founded a capital management firm in Ottawa, Canada. Connect with him on LinkedIn and Twitter.

September 26, 2017
OutSystems: Low-Code Apps Require Sound DevOps Processes

As more IT organizations embrace low-code application development platforms, many of them are discovering that accelerating the rate at which they develop applications can be too much of a good thing.

António Barros, product manager for OutSystems, says most of the organizations that have adopted the company’s namesake low-code platform are just now coming up to speed on automated testing. Barros estimates that only 20 percent of the organizations using the OutSystems platform are also employing automated testing tools. The majority of OutSystems customers continue to rely on manual testing even though it’s apparent low-code platforms will produce applications much faster than manual testing can keep pace with. OutSystems has identified five stage of DevOps maturity, he notes, but none of its customers has achieved that level of DevOps maturity.

To help them get a little further down the DevOps path, OutSystems, at its recent NextStep 2017 conference, announced an extension of its partnership with Indigo to provide tighter integration with the OutSystems platform. In addition, OutSystems plans to make available digital transformation playbooks, including one for DevOps, beginning in 2018. Outsystems has already integrated its platform with the Jenkins automated server to provide continuous integration and continuous development (CI/CD) capabilities.

Barros says it’s not only critical to engage in continuous testing, but the application needs to be designed to make it easier to isolate user interface, application programming interface (API) and unit testing. Otherwise, automating the testing process becomes more difficult than it should be, says Barros.

Developers for years have resisted low-code platforms, which automatically generate much of the code that needs to be written for an application. Some fear the platforms can’t build applications that scale. Others have simply decided they prefer to write code themselves. But these days, organizations are under more pressure than ever to deliver applications faster, which has led many organizations to rely more on low-code platforms such as OutSystems.

It’s not clear just how popular low-code platforms will become. OutSystems revealed at its conference it has signed up 122 new customers in the last quarter alone. On average, it now takes organizations eight weeks to build and deploy an application on the OutSystems platform, with new release cycle updates occurring every 48 hours, says OutSystems CEO Paulo Rosado.

Michael Facemire, an industry analyst with Forrester Research, told conference attendees that adoption of low-code platforms is now a matter of economics. The starting salary for a developer in the San Francisco Bay area is $145,000, says Facemire. As the salary of a developer with a few years of experience now reaches more than $200,000 in the Bay area, companies in that geography will have no other option but to employ low-code platforms, he says.

Obviously, developers might be less expensive in other regions. But Facemire notes that no matter what the region, developers are in short supply. That need for speed, he adds, is also why organizations are rapidly adopting containers and serverless computing framework. The challenge IT organizations will face is finding a way to put quality control processes in place to don’t compromise either the speed of the development process or the quality of the end user experience, he says.

By the same token, however, no one really appreciates code that arrives late, no matter how great it might be.

— Mike Vizard

September 26, 2017
DevOps Predictions for the Future
Among the few powerful trends we had experienced in the recent times, one is undoubtedly the adoption of DevOps practices. As we are move closer to 2018, it is becoming more obvious that DevOps is becoming mainstream, with many large organizations getting on board with DevOps.

DevOps Within Enterprises

DevOps is becoming the standard way of working for businesses. Adoption of DevOps within the organization is rising on a broader scale, and businesses are trending toward it.

Enterprises of today are well-aware of the trending technology services in modernizing environments. DevOps is growing its recognition due to changes in complexities of today’s software that lies more in the diverse set of operation systems and platforms. It is less about doing things a particular way and more about moving the business forward and turning technology into a strategic advantage.

DevOps builds upon best practices to help drive enterprise performance in the modern digital economy. It offers organizations a new way to deliver value to their customers quickly. An increasing number of businesses are recognizing the power that DevOps can bring for operational efficiency and overall performance.

DevOps Key Trends

As businesses are trending toward DevOps adoption, let’s look at the changing landscape and key trends in DevOps, along with its trajectory.

Some of the DevOps trends include:
- Many organizations will challenge agile and recognize DevOps as the new ALM methodology.
- Many DevOps automation tools are going to roll up.
- Automation and testing would gain their prominence.
- DevOps Engineers are highest paid IT practitioners today.
- Testing and deployment are done more frequently as Ops people are getting their DevOps knowledge in production for testing and automation operations.
DevOps Predictions

Based on the market so far, here are my predictions for the future of DevOps.

Adoption of Microservices and Containers

Several software enterprises are pushing toward microservices infrastructure. Microservices architect an enterprise solution, independently, over a set of services.

Microservices are best-suited for DevOps due to its features including:
- Isolated nature
- Improvised cycle times
- Well-defined interfaces
- Aligned architecture
Containerization also is growing in popularity. Containers are designed to run one process at a time with minimal deployment, serving as an ideal deployment model. According to one study, 50 percent of enterprises are expected to run containers in production by the end of 2018.

If an enterprise has decided to leverage microservices and containers, it is highly essential to incorporate solutions that support microservices and containers along with traditional application model.

Culture, the Top Prerequisite

Culture is the key differentiator for DevOps. Culture and process are the principal paths for the successful DevOps journey. Culture breaks down the organizational silos and encourages integration across all the groups that need to work together to achieve a great business outcome. Implementing DevOps culture makes an organization achieve success easier. It even enables continuous learning environment that will see bigger benefits.

Notion of Shifting Left

DevOps tools and solutions always prefer to shift left. DevOps orchestration tools automate from end to end, making traceability easier. Activities that are performed traditionally, at the later stages of development and release process, are making a shift to the left of the pipeline. The ultimate goal of it is to recognize issues earlier, along with performance and disaster recovery testing.

Downtime

Adopting DevOps holds a lot of benefits. For one, it enables enterprises to ensure uptime and enhance cadence and quality. High-performing IT organizations employing DevOps practices deploy 200 times more frequently than the low-performing enterprises. Similarly, they also enjoy faster recovery from failure and minimize downtime.

Collaboration of Big Data and DevOps

Big Data and DevOps are together taking a step forward toward the design of predictive analytics. DevOps has now become an expert in automating the process and automating configuration.

Consolidation is Critical

Large and complex organizations are focusing on consolidation and standardization of DevOps processes so as to scale DevOps within the enterprise. A shared and self-service DevOps automation enables end-to-end pipelines. Eventually, consolidation allows reusability, visibility, auditability and security.

Supporting Hybrid

“Hybrid-everything” makes DevOps solutions future-ready to support any tool, application and process in the hyper-hybrid state. “Hybrid” state of existence is achieved for infrastructure, architecture, release pipelines and more.

DevOps Monitoring

Until now, the focus has been on pre-production provisioning. However, more attention is being given to the post-deploy feedback loop. With the application of new architectures including microservices, new monitoring technologies and pricing models are effectively introduced.

Expansion of DevOps Across IT

DevOps is maturing in other areas such as database management, security and testing, all with the goal of application release.

Business-centric Agile

The development methodology approach is becoming more critical to the continuous integration and continuous delivery process.

DevOps Skills

People with DevOps skills are in demand in the industry and expected to proliferate as businesses are witnessing great results from DevOps. Organizations incorporating DevOps training practices are extremely high-functioning. If you aim to build your career in DevOps, below are certain notable skills.

Most common DevOps positions include:
- Site reliability engineer
- DevOps engineer
The core skills of DevOps are:
- Coding skills
- Knowledge of automation tools
- Communication skills
Roles of a DevOps Engineer

Many DevOps developers find interest in deployment and network operations. A core set of DevOps skills includes:
- Ability to code and script using various technologies and tools.
- Close relation with Systems and IT operations.
- Strong base of automation tools and its skills.
- Incremental code testing and deployment.
- Strong focus on data management skills and business outcomes.
Final Words…

DevOps skills will only continue to grow and DevOps is here to stay. Dev and Ops teams are increasingly moving to rapid development and reliable, high-performance services delivery.

DevOps is being adopted by more and more companies, particularly smaller enterprises. Many industries today are keen on incorporating DevOps and its key points into their respective departments. It will continue to evolve for the foreseeable future, by incorporating executives and other business units into it. Companies that utilize DevOps effectively will experience higher innovation and quality and easier adoption to other changing conditions.

Adoption of DevOps enables applications and technology solutions. It power businesses with faster and better quality. Investing in DevOps helps organizations better develop and release software.

About the Author / Sirisha Paladhi

Sirisha Paladhi is a content writer at TekSlate. During her career, she has written many articles on technology innovations.

LinkedIn URL: https://www.linkedin.com/in/paladhi-sirisha-74987881/
September 25, 2017
DevOps: Time for Developers to Dig In
Speaking to developers and project managers at the Gartner Catalyst Show, Jenkins World and Atlassian Summit revealed some insights about how most IT specialist perceive DevOps. Yes, I had a busy end of summer schedule attending events and talking to a wide range of developers and project managers about DevOps.

two dogs digging together in one hole in the ground

What I learned is that most people have not yet figured out what is meant by DevOps. What most people told me is that they are doing DevOps explaining they have a CI/CD pipeline. When questioned further about how they perceived DevOps, they believed DevOps was about creating a CI workflow that developers and sometime-testing teams could execute to automate builds, deploys and testing. One of the most ironic of these conversations was project managers making the claim that their CI server performed builds and deploys.

Well folks, DevOps is not about a CI/CD pipeline. And contrary to popular beliefs, your CI server does not perform builds and deploys any more than it performs automated testing. Your CI/CD pipeline orchestrates build and deploy processes in the same way that it executes an external tool for test automation. The ugly truth is that your CI/CD pipeline is executing one-off build and deploy scripts, the same scripts used in your waterfall practice, but just triggered by your CI/CD process. These scripts cannot easily adapt across the continuous delivery pipeline. Testing and production environments are configured differently. The build and deploy scripts must be tweaked, maintained and debugged in the same way they were during waterfall practices. Once ready, they are then called by your CI/CD workflows. DevOps is different.

DevOps demands that your developers and operation teams share and collaborate from the earliest phases of a project. In essence, this requires that more of the operational duties be shifted left and directly onto the shoulders of developers. This is precisely were the responsibility and control should be. Operation teams need knowledge and transparency so they can communicate infrastructure requirements early and often to the developers. Eventually what we will see is a decline in the number of operational resources. A “release engineer” who managed production deployments said, “If we define our DevOps process correctly, my job will not exist in the future. Instead, automation will replace the manual Ops scripts, allowing Dev teams to do 70 percent of my work.” And that, my friend, is real DevOps.

Developers, Dig In

So it is time for developers to really dig in to DevOps. What I mean is developers must begin walking away from automating most of the processes by a one-off script and begin incorporating tools that perform DevOps tasks from continuous deployment, security scanning and performance monitoring. And different platforms need different tools. Project teams will need the power to select the tools that work for them. The days of “legacy” purchases of a single tool for the enterprise is long gone. High-dollar operational tools are high-risk and incur technical debt when developers choose to use a different solution. DevOps means giving development teams more control and purchasing authority for automation tools that can be managed at a project level, not an enterprise level. These tools must support a process where the developers work on the declarations followed by testing and production consuming the process with a good amount of transparency. Scripts cannot achieve this, even when triggered by CI.

Our future is becoming more and more complex. Most companies I spoke to still managed over 70 percent of monolithic applications. The writing on the wall is in the 30 percent. With microservices and containers, operational tasks are going to become more complex, not less. The sooner developers can become the core owners of operational tasks, the easier it will be to adopt a microservices architecture. Managing microservices will not be the job for the operations team. It will need to be done by the developers who have the intimate knowledge of their application.

To mature from CI/CD to DevOps is going to require a close look at the engine under your CI/CD hood. When developers and project managers begin to really dig in and see how the engine is running, it will be clear that achieving DevOps is a bit down the road. And there is urgency in this process. Leaner and better ways of writing and releasing code is upon us. Most of us are just not ready to toss out some of the old habits from waterfall practices to get there.

Some departing thoughts:
- Acknowledge that CI/CD is a good place to be, it is just not DevOps.
- Look carefully at your underlying processes that are called by CI/CD. How well do they support your production environment?
- Can your CI/CD workflow at Dev support a production deploy? If not take the time to understand why.
- Are your developers working late to get a code update to production?
- Do you have a clear continuous feedback loop that shows where a code update is running, and on what exact endpoints?
- Do your developers know what infrastructure updates are going on in production today?
- How do your developers communicate infrastructure updates to production teams? Can they do it alone?
The time is now for self-examination and a clear honest appraisal of just how automated your core operational tasks are. If they are built upon one-off scripts, you should assume you have some work ahead to really begin moving to a DevOps practice that can be declared by developers, consumed by all runtime environments and is transparent to all.

And remember, microservices are chasing you.

About the Author / Tracy Ragan

Tracy Ragan is COO and Co-Founder, DeployHub. She has extensive experience in the development and implementation of business applications. It was during her consulting experiences that Ms. Ragan recognized the lack of build and release management procedures for the distributed platform that had long been considered standard on the mainframe and UNIX. In the four years leading to the creation of OpenMake Software she worked with development teams in implementing a team-centric standardized build to release process. Connect with her on LinkedIn.
September 25, 2017
AIOps: Managing the Second Law of IT Ops

Or, why IT systems keep breaking

The tension between the two halves of DevOps is long-standing, going back long before the term was even invented. What underlies the split between Development and Operations is a fundamental difference, like an actual physical law. Let’s explore that analogy together.

IT Ops is fundamentally a constant fight against entropy, meaning disorder. Sysadmins expend significant energy trying to keep everything up and running, in a process which can sometimes feel like trying to shore up a sandcastle while the tide is coming in. Systems are constantly breaking down; users need changes made that introduce risk; and, as if that wasn’t enough, the whole shebang is constantly under attack from malicious outsiders. It’s no wonder that sysadmins have a reputation for being a bit grumpy!

It’s not just a good idea, it’s the law.

In contrast, Dev actually increases the level of entropy. Every new release, be it as small as a one-line fix or finding that errant semicolon, is a change that has to be made, reducing the order that Ops fights so hard to maintain. Dev is entropogenic, embracing change as a positive force.

The interaction between Dev and Ops is analogous to the second law of thermodynamics: Entropy can only increase over time, unless energy is introduced to the system from outside. In other words, unmanaged change will eventually cause something to break, unless people put in continuous effort to prevent those breaks.

Thermodynamics: Lots of Heat, not Much Light

These conflicting goals, taken to extremes, can get unhealthy, with Ops actively fighting change or Dev going around Ops to increase their velocity of change. Left unchecked, this tension can turn into a vicious cycle, as frustration with the “Department of No” drives developers into unmanaged “shadow IT.”

The good news is that the analogy with hard laws of physics is not perfect. The process of increasing entropy is not irreversible, and does not require constant inputs of energy to prevent. New agile approaches to IT Ops are emerging that are light and flexible, taking their structure from that tension. In the physical world, this is called “tensegrity,” and was originally formulated by Buckminster Fuller, one of those annoying polymaths who seem to make breakthroughs in every field they touch.

Tensegrity structure

IT Ops needs its own equivalent to tensegrity, a flexible, holistic structure that is able to embrace change while maintaining its shape. Structures that try to resist change through rigidity are brittle and ultimately fragile.

Today, this flexible strength is enabled by AIOps, a term coined by Gartner to describe a new way of operating IT, based on augmenting human skills with automated analysis enabled by modern algorithmic and machine-learning approaches.

Complexity Breeds Complexity

Modern IT is too complex for any single person or even team to keep track of, instead requiring multiple specializations. In turn, each of these functions requires its own specific support tools that enable that particular role. Beyond a certain point, it becomes very difficult to combine these partial views to gain a holistic understanding of what is actually happening in the environment.

Because of the scale of modern IT environments and the sheer rate of change that businesses require, old-style rules-based “Managers of Managers” can no longer keep up, requiring more effort to build and maintain their worth. Especially as infrastructure becomes more and more self-modifying in response to changing conditions and demand, IT Operations needs to accelerate just to keep up.

AIOps brings developing issues to the attention of sysadmins before they become visible to customers and end users, and gives them the tools to collaborate effectively between different teams and specialist roles to avoid outages or restore service quickly and efficiently. In this world, automation is used to augment the capabilities of the human operators to make them more effective, letting people concentrate on adding value, not just fighting fires.

This is how Ops can get out of the business of fighting Dev, and instead become a supportive partner. AIOps can catch issues that are affecting services and flag them to the right people early on, including the DevOps engineers that are responsible for production support.

For developers, one of the fears around DevOps is that in “picking up the pager,” they are opening themselves up to a constant stream of irrelevant alerts. Old Ops hands nod wryly at this, or maybe try to share their personal approach to filtering the noise. Instead, AIOps ensures that only valid, actionable alerts get as far as being shown to a human being and taking up their valuable time.

Eyes on the Prize

It’s not all roses. As the AIOps market begins to mature, there is a certain amount of “algorithm-washing” going on, with more and more vendors trying to claim that either their legacy tool was AIOps all along, or that their one-trick pony is actually a whole three-ring circus. What gets lost in the excitement is what all of those algorithms are actually supposed to achieve.

Me too!

In considering a shift toward AIOps, the goal should not be simply to build a better mouse trap, meaning a smarter filter for monitoring events. The success of the project can be measured not so much in the mathematical results of the algorithms, but in the smoother exchange between the human users. This easier collaboration is especially important across team boundaries, whether these be different specialist units within the Ops function or developers and software engineers who need to work with Ops to understand the performance of their code in production.

The benefit of transforming operations this way—moving from sequential, waterfall Ops to abile, collaborative Ops—is to turn DevOps-enabled teams into a sort of perpetual-motion machine, able to continue operating without requiring constant massive inputs of energy from outside the system. Modern applications and the complex infrastructures that support them cannot be managed with the sorts of rigid, cobbled-together, industrial approaches that were appropriate in the mainframe days. AIOps is the agile, flexible approach that can not only accomodate that complexity, but thrive on it.

— Dominic Wellington

September 22, 2017
Total DevOps is Needed, Plugins are Not Enough

Trace3 was proud to be a Gold Sponsor, exhibitor, trainer of the DevOps Institute leadership certification course and newest partner of CloudBees at the CloudBees Jenkins World 2017 event in San Francisco last month. Jenkins World 2017 attracted 54 sponsors/exhibitors, an 80 percent increase over 2016.

The growth of interest and sponsors for this event demonstrates that the number of available tool choices for creating the continuous tool chains to realize continuous delivery and DevOps is growing. One of our top takeaways from Jenkins World 2017 is: Plugins are not enough!

Indeed XebiaLabs’ “Periodic Table of Tools” is a valiant attempt to illustrate categories of DevOps tools and attributes of specific tools. Version 2 of the Periodic Table is now released, and I heard that version 3 is already on the way. While the table shows an impressive 15 categories of tools, I estimate there are actually 29 categories, if you consider tools for cloud management, test management, test creation, code analysis, code collaboration, ALM systems, dashboards, security and others.

XebiaLabs’ table shows five attributes for each tool: open source, free, freemium, paid and enterprise. In my opinion, there are at least two additional categories: ecosystem and cloud readiness. While the table indicates 120 tools, my own estimate counts 300+ DevOps tools, including CloudBees, Electric Cloud, Perforce, Scalyr, Service Now, Tricentis. SmartBear, SonarQube, Parasoft, Coverity, Fugue, QA Symphony and many others.

While the exact number of tool categories, tools and attributes is arguable, there are many tools to choose for a continuous delivery toolchain. The state of practice for integrating tools into a seamless toolchain is the plugin or do-it-yourself scripting. Anyone who has implemented a toolchain knows that the existence of tool plugins, while useful, is not enough. The quality and completeness of each plugin varies significantly. Some off-the-shelf plugins are so basic they are little more than marketecture. Even the best plugins are limited to a pairwise matchup between tools of specific version pairs, and not a universal tool or version. For example, a tool with a plugin for Jenkins version X may not function with Jenkins version Y or CloudBees Enterprise of other tools at any version.

The DevOps Express collaborative initiative announced by 14 participating vendors at Jenkins World 2016 was an attempt to resolve the plugin problem. One year later, the DevOps Express website has shown a growth to 123 tools across 18 categories. However, just like the XebiaLabs Periodic Table of Tools, it is remarkable that the number of tool categories and tools that are not comprehensive (and lacking plugin compatibility) are not guaranteed for any combination of tools that may be needed for a toolchain for a specific enterprise.

Each enterprise has unique toolchain requirements and preferences. Selecting and an integrating DevOps tools is not like buying a box of “standard” Lego bricks that are guaranteed by the manufacturer to fit together. Each tool and plugin is different.

The choice of tools, the combination of tools and the integration of the chosen tools all must be carefully chosen to fit the specific customer requirement. More importantly, the integration must be architected to ensure the chosen combination works well together in a seamless toolchain.

Until there is an absolute standard plugin architecture that all tool plugins are guaranteed to be compliant with, expert consulting services will be essential to make tools choices and make sure the tool chain works.

With such a rapidly growing list of tools and vendors, only the most capable consulting teams with deep insight and partnerships with the widest array of tool vendors will be able to address enterprise DevOps solutions properly.

— Marc Hornbeek

September 21, 2017
Software Security is an Engineering Problem
We live in a world where those who sell products are generally held accountable for defects. Automobile manufacturers, pharmaceutical companies and toy makers are among the businesses that have been handed fines and tarnished reputations for distributing items that were later deemed ineffective or dangerous.

One notable exception to this “consumer first” ethos is the traditional shrink-wrapped software world, where responsibility for product problems is hazy. The vendors that develop and sell software and the customers who purchase and install that software are in a constant battle to shift security liability. The vendors strive as best they can to reduce and remove product vulnerabilities and offer secure configurations while customers deploy, configure and operate the software in very different environments, with very different configurations, which can produce very different security outcomes. Because no customer can (or should) assume receipt of 100 percent vulnerability-free software, they must always take on the primary burden of protecting the environments where that software lives to offset the risks—which typically falls to the customer’s IT and security teams.

Although there are excellent arguments for both sides, the current situation leaves both parties wanting—and needing—a higher level of assurance in both the development and runtime settings.

Software Delivery Has Changed (Along with Security Responsibility)

The good news for software customers is that software delivery is rapidly moving away from an on-premises install approach to a cloud-based subscription model where applications are delivered via the internet. Upfront costs for hardware and software, as well as the expense of personnel to install and manage that software, are replaced with business-friendly, predictable, subscription-based operational costs. Software is easy to launch, upgrades are uncomplicated and scaling to meet changing business needs is simple.

Customer expectations are high for software-as-a-service (SaaS) solutions. Purchasers demand an elevated level of innovation and want new features more often. They assume built-in support of global information security and privacy regulations such as the European Union’s General Data Protection Regulation (GDPR) and various U.S. regulations, including the Health Insurance Portability and Accountability Act of 1996 (HIPAA) and PCI Digital Security Standard (PCI DSS). SaaS customers also expect the vendor to take on larger responsibility for any potential software vulnerabilities that might negatively impact the customer’s data and operation.

The new reality is that deploying software in the cloud significantly changes the paradigm of how software security should be treated at the vendor level. This means companies now have to think critically about security as an engineering problem as the balance of the liability burden shifts focus to the cloud provider’s engineering practices to ensure constant delivery of a secure and trusted cloud service, rather than just secure software.

Now is the Time to Rethink Software Security

Cloud-based application delivery brings together under a single company umbrella the ones that make the software and the ones who run the software. Informing software development of runtime behavior and vice versa can have significant advantages in enhancing the overall security of the software by offering a closed-loop system. In such a perspective, the rapid pace of software releases work in favor of enhancing security.

Shifting security to an all-around engineering responsibility requires the elimination of traditional work silos. Geared-for-speed developers must receive precise information early, allowing them to correct security problems during the initial coding process. DevOps team members also must assume security responsibilities. And security engineers who understand the attacker mindset, security risks and compliance requirements must provide guidance at all points in the software development process.

Change is never easy in any organization, so transforming disconnected teams into one in which security is an across-the-board responsibility is difficult. Managers must be careful to not alienate team members who already have full plates. And although we’re starting to see some shifts in the educational model to one in which professionals are taught early on how to seamlessly integrate coding and security, we have a long way to go.

While we wait, the most effective way to overcome the chronic problem of heavily siloed teams is to provide the entire software engineering team with a means to quickly identify issues between software development, its deployment and runtime across all the components. These tools must give users the ability to:
- Scan and provide real-time analysis that integrates source code analysis and its runtime behavior. After all, they are two sides of the same coin;
- Obtain accurate and full visibility into the security impact of changes from build-to-build;
- Ease the audit analysis process;
- Reduce the number of false positives; and
- Allow developers, DevOps, and Security to collaborate to enhance the security of the environment.
The goal? An ability to fix software mistakes, weaknesses and vulnerabilities at the beginning of the development process instead of post-deployment, when liability, revenue and reputational risks are the greatest.

Consolidating today’s siloed security practices into an integrated engineering effort is the only way to ensure the appropriate level of security at the appropriate time in today’s fast-moving cloud-based software industry. This sort of shift in thinking and processes will give customers less security worries and better products. Development teams will become more effective and efficient. DevOps will have real-time visibility into how each release impacts the infrastructure. And Security will know how each release impacts both security and compliance requirements. SaaS companies will reap the financial benefits that come with strengthened industry reputations for rapid innovation capabilities and high levels of software security. It’s a win-win-win for everyone.

About the Authors

Manish Gupta is founder and CEO of ShiftLeft Inc. He is a 20-year veteran of the security industry and has led product development, management and strategy at FireEye, Cisco Systems, Intel, McAfee and Redback Networks.

Craig Rosen is vice president and CISO at AppDynamics. He has held leadership roles for more than two decades in the security industry, including at FireEye, Pacific Gas & Electric, TDI and CGI.
September 21, 2017
Next-Generation DevOps: Self-Learning Systems
The massive enterprise movement toward all things “digital” has accelerated the adoption of DevOps in all kinds of projects, but particularly in digital transformation initiatives. Apart from the productivity benefits gained by automating software development and rollout, the increased agility has helped move DevOps adoption to gain critical mass in a relatively short time.

However, challenges remain, particularly in ensuring the alignment of the three dimensions of business systems enabled by DevOps—people, process and technology—are coordinated enough to achieve desired outcomes. The “churn” in each of these three dimensions with every software release—particularly given constantly changing technology, constantly evolving operating procedures to support business requirements and end users and the impact the churn has on the “people” dimension—cannot be underestimated.

To mitigate this churn, it is essential to implement a DevOps “composite system” that manages it methodically and systematically to bring some order to the seemingly chaotic mess of managing multiple moving systems at the same time. The composite system should essentially be able to deliver the following capabilities to address people, process and technology:

A Visibility Dashboard for all Stakeholders in the DevOps Value Chain

Visibility and relevance of their work is of prime importance for people to feel connected to a mission. In a typical software development life cycle (SDLC) in an Agile methodology executed through DevOps, there are many stakeholders involved, including business stakeholders, developers, testers, and support services personnel. In an integrated life cycle of constant and continuous software development, testing and rollout, the cycle can seem complex and never-ending. For a successful and motivated team committed to do their best, systems should provide a visibility dashboard customized to each individual based on their role in a project. The dashboard could involve gamification and relative performance visibility of trends at an individual and group/project level.

The purpose of such a dashboard is not just to improve visibility and individual performance but also bring in alignment and connect each member of a DevOps team with a broader mission of the project/company.

Event-driven Technology Automation Spurred by API Integration

The sheer complexity and number of technologies and platforms involved in a DevOps life cycle can be unnerving and intimidating at the same time. Ultimately, the aim of deploying technology to a process is to simplify and create efficient systems. To this end, DevOps technology architecture must be governed by two critical design principles:
1. An event-driven architecture that enables asynchronous integration of data generated by different tools and technologies involved.
2. An API-enabled interface at each level of DevOps process flow so that a change in underlying tooling does not (or should not) have an impact on the business.
Data Driven Self-Learning Systems to Constantly Streamline and Improve Processes

In systems heavily laden with interconnected tools and technology, a typical DevOps process is heavily automated. However, this is just a “first generation” maturity of DevOps evolution. Just as with the people dimension, where team members are constantly measured, benchmarked and improved, systems and processes need to evolve continuously.

Because of automation, integrated DevOps systems generate lots of data (systemic/process-related data including build cycle times, testing cycle times and operational data such as issue resolution) that can be harnessed and plowed back in. Next-gen maturity in DevOps is achieved through a data-enabled feedback engine that constantly improves the quality of DevOps processes and their performance. Advanced data analytics and machine learning algorithms can be implemented to monitor, track and improve DevOps performance metrics.

In summary, to implement next-generation DevOps, organizations need to consider a holistic approach to manage changes at people, technology and process levels. Each of these dimensions require unique needs and a keen understanding of what it takes to make a DevOps system work for all stakeholders involved.

About the Author / Madhavan Krishnan

Madhavan Krishnan serves as vice president of Cloud & DevOps solutions at Virtusa. In this role, Madhavan is responsible for competency development, client solutions, and practice team development from Asia and P&L. With more than 17 years of global IT industry experience across multiple technology and business roles, Madhavan’s experience cuts across different dimensions of Virtusa’s IT Services business including global delivery, enterprise architecture, incubating and scaling technology practices with P&L responsibility, and presales and business development. Madhavan is an active participant in various industry and client forums including CII, MAIT and RFID World Asia, and was a member of RFID Integration Consortium in 2005.
September 20, 2017
Keys to Improving Enterprise DevOps Practices
As enterprises move to adopt DevOps practices, there is a desire to measure where teams are on their journey. This sometimes leads to the development of maturity models used to assess teams’ levels and progress. While I am not going to pass judgment on what might be useful for certain organizations, I believe that the real key is to drive positive DevOps practices, which include continuous improvement through data such as lead time, frequency of deployments and mean time to resolution (MTTR) (i.e. the key metrics from the State of DevOps Reports).

Any static maturity model will become outdated. What’s more important is that teams are taught concepts that can help them continuously evaluate what is stopping them from delivering more quickly so that they identify areas for improvement. Key to this is the lean concept of value stream analysis. If you walk up to a team and ask them why they can’t go faster, they will typically say they are waiting for something. They could be waiting for more work to flow into their backlog. They could be waiting for infrastructure needed for development or testing. Or, they could be waiting for another team to develop a service they need to consume.

Many teams don’t spend time in retrospectives talking about how to overcome these types of blockers or wait states and developing countermeasures as continuous improvement initiatives. One reason for this is the lack of objective data that can be analyzed to provide insights on this. This is where the lead time metric comes into play. Historically, lead time has been defined as (last) code commit to deploy into production. While this is important, it only tells part of story because accelerated delivery and feedback really starts with a customer concept and measuring how long this takes until it is delivered into production so feedback can be provided.

One challenge to providing this kind of end-to-end metric, which spans the entire delivery value stream, is the lack of an integrated delivery pipeline from which data can be automatically collected. A customer concept starts in the portfolio space, which might result in data being entered in some type of PPM (portfolio management) system. Once approved, this results in features being created which eventually should flow into a product backlog. Stories are then created that can be pulled by the agile line supporting that product into an iteration. This information is typically in some agile management system (e.g. Jira or IBM Rational Team Concert). Then the team develops and builds the stories, tests them and deploys them into production via a deployment tool like UrbanCode. So, this flow of work goes through various tooling and generally is not visible or easily traceable across the value stream.

Our initial goal in measuring lead time was to shift left and measure the time from story card backlog until the story was released into production. This would then give us four discrete measures that would provide more insight into the time spent in different parts of the value stream.
1. Time spent in the backlog
2. Time spent developing the story
3. Time spent testing the story
4. Time spent once the story was completed (marked “done”) until it was deployed
We mapped the first and last categories into wait times and the second and third into process times to provide a baseline of this metric. While we have just started providing this information to teams, we are finding that scrum masters believe this type of data can be used in retrospectives to better understand why some lead times are high. They also believe that this will help the team better understand when blockers are encountered during development and testing and what contributed to them. We also plan to make that visible as part of our metrics (time spent in a blocked state). Tooling like TaskTop Integration Hub can help integrate your pipeline tooling to provide visibility and insight.

Once teams gain more insight into what is slowing them down, they can identify countermeasures that can be applied. Categories of wait-states include:
- Waiting for a dependent area to provide functionality necessary to complete a feature
- SLAs associated with a service that is provided by a centralized organization (e.g. performance testing or security testing)
- SLAs associated with infrastructure needed for an environment
- Contention for a shared test environment
Countermeasures associated with these types of wait-states are:
- Architecture decoupling
- Dark Launching / Feature Toggling
- Expanding cross-functional teams to enable them to do self-service performance and security testing (at least some portion of it)
- Automated Provisioning Capability for environment to avoid wait-states and contention
The “State of DevOps Report 2017” contains information on the importance of architecture and how decoupling dependencies (using APIs and microservices) can mitigate certain types of wait states. DevOps Case Studies provides more information on applying DevOps practices such as dark launching.

But again, at the heart of this is getting teams to think more about what their value stream actually looks like, how long it takes to deliver business value that can generate feedback and what is getting in their way of going faster. This is not a technology play. This is not something that somebody can sell or educate you on. This is something the team itself needs to take on.

A good starting place for a team is to educate themselves on some of these concepts for DevOps practices using a book club. A great place to start is to study the “Phoenix Project.” This novel approach combines aspects of both DevOps and Lean to drive home some key points in the journey to discover types of work, manage work in progress and ultimately make visible and gain control over the work being done for the business. Once a team has an understanding of their value stream and blockers, they can then study the “DevOps Handbook” that provides practical guidance for improving their delivery capability and speed.

About the Author / Carmen DeArdo

Carmen DeArdo is the director of Build Technologies at Nationwide Insurance. He is responsible for helping drive accelerated delivery utilizing Lean and DevOps techniques across the end-to-end delivery value stream. One current focus area is the creation of an integrated delivery pipeline capability.
September 19, 2017