devopsy.kinsta.cloud

Tag: enterprise

Liberty Mutual: Taking an Organic Approach to DevOps
Some people are directors, managers, and administrators. Others are disrupters.

Eddie Webb (@edwardawebb) is an IT disrupter for Software Development Platforms at Liberty Mutual and was a presenter at the 2016 All Day DevOps conference. His talk, “Organically DevOps: Building Quality and Security into the Software Supply Chain at Liberty Mutual,” looked at Liberty Mutual’s transformation to continuous integration, continuous delivery and DevOps. For a large, heavily regulated industry, this task can be not only daunting, but also viewed by many as impossible.

Eddie’s talk looks at the origins of DevOps at Liberty Mutual, the challenges of centralizing DevOps, and how to make it easy for developers and operations to do the right thing. He noted that DevOps is not Liberty Mutual’s goal; continuous delivery is the goal and DevOps is the enabler.

It all started at Liberty Mutual in 2004 with a vision: “Black-box reusable services that benefit the business.” The company also had a concept that “developers own the quality of the code; we have no QA team.” Employees were empowered with this “concrete vision but malleable strategy,” as Eddie said. The company gave them time and space to explore solutions, because ownership increases adoption.

The team started with fixing what frustrated them, including CI/CD pipelines, artifact and code repositories, elastic cloud-native runtime and Agile planning and delivery.

But what about centralizing DevOps?

The reality is that tools can be centralized, but culture cannot.

Eddie laid out an example of a large organization’s IT structure, saying that this is how things work at Liberty Mutual. Eddie’s team is the Central IT, and their customers are the Market IT team, not Liberty Mutual’s end customers. As Eddie pointed out, the more layers you create, the more friction that enters the system. Often, organizations try to reduce the friction through microfixes, but Eddie’s team asked how to change the culture to reduce the friction.

This required distributing innovation and centralizing the tools that work. As an example, if you build a manufacturing plant, you wouldn’t build your own power plant. If you are in IT, why build your own cloud?

This is the Netflix way. At Netflix, teams are independent, but responsible. Engineers can choose the best tools for the job. No one single team is responsible for innovation. However, the Engineering Tools team helps direct other teams’ experimentation toward new products without getting in the way of innovation.

Eddie quoted the book, “Drive,” when looking at how to motivate innovation in complex work environments. The author says if you want to incentivize or motivate cognitive tasks, you need three things: autonomy, mastery and purpose.

When you centralize non-value added platforms, it enables teams to:
- Master their specific domains of business;
- See clear purpose in everything they build; and
- Maintain autonomy of choice of frameworks, patterns, and approach.
When you decentralize, it raises the question: What about security? Developers are humans, and humans inherently try to avoid tasks they see as unnecessary or overly burdensome, and this becomes easier when the compliance person isn’t there (think of how we handle speed limits). Traditional security measures erode profit and market share, and, from a stat Eddie quoted from Gartner, 77 percent of IT security professionals agree that security slows down IT.

Whether you are centralized or decentralized, security (and other compliance-oriented tasks) can increase adoption rates by making it easy to do the right thing. How do you do this? Eddie proposes governing with tools (think of speed governors in fleet cars).

You also need to have immutable environments, because, “We can’t trust humans to manipulate production.” So, again, you have to make the path you want them to take, and make this the easiest path. As Eddie notes, “You can’t put too much friction into the pipeline, because developers are good at bypassing friction.” You have to make your CI/CD pipelines the easiest way to get ideas to production by. You also have to abstract complexity and automate policies. You have to have an “opinionated platform, and observe your customer so you can address their pain.”

Eddie concluded with the following final points:
- Don’t mandate DevOps. Give employees the chance to master their discipline with examples to set and follow.
- Favor deep end-to-end accomplishments over broad but incremental steps forward. Focus on taking the right teams far before encouraging broad adoption.
- Centralize the platforms and tools that your teams shouldn’t be thinking about. Provide foundational services/commodities and let teams stay on purpose.
- Incorporate contributions from everyone; don’t stifle autonomy. Stay open to new ways of working.
- Challenge security policies, but respect intentions. Find new ways to enforce concerns without abandoning precaution.
You can listen to Eddie’s entire talk online here. You might learn an unexpected lesson.

If you missed any of the other 30-minute long presentations from 2016 All Day DevOps, they are easy to find and available free of charge here. Finally, be sure to register you and the rest of your team for the 2017 All Day DevOps conference here. This year’s event will offer 100 practitioner-led sessions (no vendor pitches allowed). It’s all free and online Oct. 24.

— Derek E. Weeks
September 12, 2017
9 Essential Ingredients for Modern Enterprise Applications

Digital transformation. It’s the latest in a series of buzz phrases intended to describe how enterprises are embarking on top-to-bottom modernization of their IT environments. It’s easy to see why it’s a hot topic. Emerging technologies including cloud, containers, microservices and distributed applications hold the promise of greater flexibility, lower cost and faster release cycles. The benefits to companies that make this journey can be enormous.

Unfortunately, making this journey can take years with today’s manual processes and antiquated tools. Not to mention that most IT budgets are at least 80 percent focused on keeping the lights on. Application modernization projects can be extremely complicated since legacy, monolithic enterprise applications are intrinsically dependent on underlying infrastructure and data stores. Cloning complex application environments, including critical production data, for upgrades or testing often can take days or weeks, slowing down cycles and contributing to lower quality releases. Even new applications are too often designed with infrastructure dependencies, trapping organizations onto specific cloud, server or container vendors. This vendor lock-in reduces choice and leaves IT organizations open to unexpected price hikes down the road.

The truth is that the way we have designed, built, deployed and managed enterprise applications needs to change. Business-critical software is too often built in monolithic blocks that are nearly impossible to move to new infrastructure or storage options without lots of effort and time. The applications are hard to upgrade since updating even one small portion means rebuilding and retesting the entire thing. Managing the various versions of the discrete application modules and the resulting testing permutations can be quite challenging. It’s not unusual to hear from CIOs that they feel trapped in a world where they can only reasonably expect to update critical enterprise applications just twice a year at most.

Business needs to move faster. There’s no reason why release cycles, even for enterprise applications, couldn’t be every few days instead of months. Application quality should actually be better in this world. Continuous delivery models such as DevOps should be an integral part of the process. And security should not be an afterthought; it should be built-in. To accomplish this, the future of enterprise applications must look very different from today.

What are the essential ingredients for modern enterprise application platforms and tools in this new world? Here are a few suggestions.

Complete Portability, Without Code Change

Enterprise applications must be abstracted and separated from any dependency on underlying infrastructure, including sources of data. By abstracting the functions of an application into components that can run anywhere, it’s possible to then move the application to different infrastructure combinations, without changing a single line of code. It’s only through complete portability between cloud environments, container tools and servers that IT organizations will break vendor lock-in and gain the flexibility required to move their applications to vendors that offer the best combination of price, performance, reliability and features.

On-Demand

All enterprise applications should be on-demand, meaning the provisioning of new instances should be hands-free and fully automated. This implies tight integration with orchestration tools that most companies are already using to provision infrastructure. The key difference here is that the provisioning of storage options also needs to be fully automated. This is made possible by the fact that modern applications won’t have any dependencies on the specific data stores they rely on. This on-demand nature is not restricted to technical teams but includes business users that can easily perform user acceptance testing by simply provisioning full-fledged application environments.

Self-Healing

Modern enterprise applications should be self-healing, which means applications not only should have measurable data points to define their health but can then react based on these metrics. They must be able to scale up (or down) based on performance characteristics up to a certain cost threshold, restart on a healthy node in case of infrastructure bottlenecks and cache or deploy closer to the user based on latent use performance.

Composable (DevOps-friendly)

All enterprise applications should consist of a number of building blocks: APIs, web servers with content, databases seeded with data, security policies, the operating system, network configurations, deployment configurations and application binaries. By breaking apart monolithic enterprise applications into smaller components that can then act as independent services, IT organizations gain the flexibility to upgrade and automatically deploy individual components through standard DevOps processes and orchestration tools. The services can run anywhere desired, giving flexibility of choice in infrastructure or sources of data. And developers can use a point-and-click UI to pull pre-defined components together to build larger functions of an application.

Model, Snapshot and Clone

Once an application is separated into its essential components, a catalog can be maintained that makes it easy to create new versions of the application. Developers, test engineers and DevOps teams can then pick and choose whatever combinations of components they need to create or clone complex application environments for testing or deployment. The state of the application environments can be snapshotted, cloned and replicated. This cloning process should be efficient and quick. This dramatically speeds up development and testing processes, which to-date have been largely manual.

Scalable

Scaling application and infrastructure environments automatically, on-demand, and in near real-time is one of the essential capabilities that makes modern infrastructure so valuable. Any solution that is used to develop, test, deploy and manage enterprise applications should offer these same capabilities, giving the ultimate in cost flexibility, performance and scalability. As containers are moving quickly from the lab into production environments, look for a container-based solution that complements this strategy.

Security is Included

Application security should not be tacked on after deployment. This slows down continuous delivery processes like DevOps and creates friction between development/ops and the security team. Instead, security should be considered an essential component of the overall application environment and treated the same as any other component and baked into the application from the start.

Push-Button Upgrades

Upgrading enterprise applications should be made much easier by taking a point-and-click, push-button approach through a modern UI and integration with continuous deployment tools. Once an application is blueprinted into its various components, using a modern UI to choose versions and assemble the final application becomes entirely possible, saving time and reducing manual errors.

Single-Pane-of-Glass Application Management

By taking an end-to-end view of the enterprise application development, testing, deployment and management process, it’s now possible to create a “single pane of glass” console that gives development and deployment teams a powerful way to manage what’s been deployed to various infrastructures in the organization. With this approach, it’s also now possible to collect invaluable utilization and cost metrics which can help an IT organization continuously improve their portfolio.

Enterprise applications of the future are going to look very different from today’s monolithic behemoths. By leveraging new technologies and processes, IT organizations will be able to dramatically speed up their modernization efforts and release cycles, improving competitive position. The inherent cost savings of this approach will be transformational as well, as IT budgets shift toward net-new innovation, not just keeping the lights on. The future looks bright indeed.

About the Author / Rahul Ravulur

Rahul Ravulur is the cofounder and CEO of appOrbit, which was founded in 2014. Most recently, he was the Director of Product Management at VMware managing the Availability suite of products. Before that, Rahul managed the vCenter Server and Client set of products with a special focus on scalability and extensibility to chart the evolution of vCenter to a broad virtualization management platform. Prior to VMware, Rahul held engineering leadership roles at BMC Software and Marimba.

August 10, 2017
From ‘Water-Scrum-Fall’ to DevSecOps
As organizations abandon the waterfall method of software development for Agile, many are stuck in what Hasan Yasar terms Water-Scrum-Fall. That is, the organization has not effectively embraced Agile and DevOps principles and remains in silos with no links to business goals. Enter DevOps, an extension of Agile thinking. While Agile embraces constant change and embeds the customer into the process, DevOps embraces constant testing and delivery and embeds operations into the team to internalize expertise on deployment and maintenance.

This is how Hasan started his talk, “Multi Security Checkpoints on DevOps Platform,” at the All Day DevOps conference.

In his talk, Hasan lays out a plan to get organizations to DevSecOps. Really, DevOps is a risk mitigation strategy, built on situational awareness, automation, and repetition. But, security is where a lot of DevOps implementations fall down. The goals for each organization should be:
- Protecting private user data
- Restricting access to data/systems
- Protecting company data/intellectual property
- Standards compliance
- Safeguarding disposition/transition
But, how do organizations get there?

First, integration and communication. Every point of the product development life cycle should be integrated and communicating, including among the tools. Once this is achieved, you can automate many, if not most, of the tasks. The automated steps are the ones that require less human actions/input to the software development process. This allows everyone to focus on innovation and better code and less on tasks that can be automated by autonomous systems. Also, tasks that can be automated are less susceptible to errors.

Of course, it is the team that ultimately designs, develops and delivers the software. Your team consists of development, IT operations, quality assurance and security. Each has its own skill set and focus, and the overlap is Secure DevOps.

The team is in place, processes are automated and development has started. Development in this day and age has evolved tremendously from even just a few years ago. Previously, software was limited to size, function and audience, and the supply chain was practically non-existent. Your team built each component. Now, development has grown beyond the ability of an organization to develop outside of its core competencies. The supply chain now involves many sources for the code. It is more like plug and play, and this creates lots of vulnerabilities.

Hasan notes the software supply chain risk factors:
- Supplier capability – Does the supplier follows practices that reduce supply chain risks?
- Product security – Is the delivered or updated product acceptably secure?
- Product distribution – Does the method of transmitting the product to the purchaser guard against tampering?
- Operational product control – Is the product used in a secure manner?
To reduce your supply chain risk, Hasan recommends:
- Ensure supplier security commitment;
- Evaluate a product’s threat resistance;
- Create a centralized private repository of vetted third-party components for all developers;
- Establish good product distribution practices; and
- Minimize variation of components to make things easier.
Finally, as you transition to DevSecOps, remember that security must be addressed without breaking the rapid delivery, continuous feedback model.

You can watch Hasan’s entire talk online here for more details and other tips.

If you missed any of the other 30-minute long presentations from All Day DevOps, they are easy to find and available free-of-charge here. Finally, be sure to register you and the rest of your team for the 2017 All Day DevOps conference here. This year’s event will offer 96 practitioner-led sessions (no vendor pitches allowed). It’s all free and online Oct. 24.

— Derek E. Weeks
July 27, 2017
Atlassian Builds a DevOps Stack for the Enterprise

One of the major challenges associated with DevOps is that it’s not something an enterprise IT organization can buy. Because of that issue, it’s often challenging for IT organizations to forecast what types of DevOps tools they are going to need. To make it a lot easier to cope with that issue, Atlassian has announced Atlassian Stack, a suite of software that makes Atlassian DevOps tools available in a single package.

Sean Regan, head of product marketing for Atlassian, says this approach will make it easier for enterprise IT organizations to embark on a DevOps journey without having to worry about what type of tool they may need now versus in the future. For example, an IT organization may opt to get started using the Atlassian Jira software to track software development projects and then work its way up to deploying Atlassian Bamboo continuous integration/continuous deployment (CI/CD) software. From there, Regan says, a DevOps team may opt to embrace collaboration software to streamline communications across their various teams.

The move to launch Atlassian Stack comes after the company updated a slew of its DevOps offerings, including the release of Bitbucket Server and Data Center 5.0 and Bamboo 6.0. In addition to tightening the integration between both DevOps tools, the company added capabilities for verifying commits, smart mirroring and the ability to define a Bamboo configuration as code.

Rather than making DevOps teams acquire each tool individually, he says, many senior IT leaders now request a means through which the entire Atlassian portfolio can be made more readily available. That shift in purchasing behavior suggests that the level of DevOps maturity inside many organizations is advancing considerably, he notes.

Of course, there are many who would contend that level of DevOps maturity has been a long time in coming. Web-scale companies have been employing DevOps tools for years to accelerate the rate at which new applications are released and updated. In contrast, traditional enterprise IT organizations have been slow to implement DevOps processes. In fact, it’s taken the pressure arising from digital business transformation initiatives that have accelerated application deployment timetables to get enterprise IT organizations to fundamentally rethink IT operations. Business leaders are making it painfully obvious that their patience with extended rollouts for new applications is at an end.

Naturally, the rate at which those transformations are occurring within organizations will still vary considerably. However, the silos that exist within enterprise IT organizations need to be consolidated. The next issue is determining to what degree altering those processes needs to come in the form of a DevOps edict from above versus letting teams come together of their own accord to meet a specific application deadline. Whatever the approach, the way enterprise IT has been managed historically is no longer up to the digital business transformation tasks at hand.

— Mike Vizard

June 15, 2017
Embedding Ownership: A DevOps Best Practice
From where I sit in the DevOps community, there is often more focus on dev than on ops. Damon Edwards (@damonedwards) of SimplifyOps sought to change that with his talk, “Ops Happens: DevOps Beyond Deployment,” at the All Day DevOps conference.

Damon dove right into the primary, systemic force behind most DevOps problems: silos. The product development process goes like this: Planning—>Dev—>Release—>Operate. The problem lies in the tendency of many enterprises to place similar functions together. Everyone ends up in a silo. Then walls build up between the silos. Eventually, people only know life in their silo, making handoffs even harder.

We often find application knowledge and business intent are heavily emphasized on development side but light on the operations side. Likewise, operational knowledge is heavy on the ops side, but light on the development side. Furthermore, development has ownership but limited accountability, while operations has accountability but no ownership.

While many enterprises are striving toward building cross-functional teams, the reality is that the transformations often stop short of truly integrating operations. The result? Silos remain.

So, one has to ask, why is this so hard?

The reality is that enterprise operations are under tremendous pressure. One side is telling them to go faster and open it up, and the other side is telling them to be more secure and be more reliable. These are often seen as competing priorities.

To solve this, enterprises need to “shift left” in the product development cycle operations activity as much as possible. They need to do as much as possible during development. For the deploy function, enterprises should be:
1. Writing/running automated tests
2. Writing/exercising deploy automation
3. Running security scanning tools
For the operate function, enterprises should be:
1. Writing/exercising automated runbooks
2. Writing/exercising monitoring/metrics
3. Operational control (safely!)
However, shifting operations to the left is much more difficult. How do you do it? Embed ownership.

Let me repeat: embed ownership.

First, those who build something define the procedures to fix it, and those who build something fix it when it breaks.

That sounds simple, but raise questions:
1. How do you safely and securely give out access?
2. How do you enable the experts to contribute remediations?
3. How do you give the experts visibility into operations?
4. How do you do postmortems days/weeks/months later?
Damon recommended four steps.

Step 1: Establish a secure ops portal

Step 2: Establish a software development life cycle (SDLC) for ops procedures

Step 3: Connect with enterprise management systems

Step 4: Make Compliance really happy

Ticketmaster is a real-life of example of this working at a large, prominent scale. Ticketmaster calls its system, “Support at the Edge” and it involves:
- Automated Ops procedures written/vetted by the delivery teams
- Ops remained in full control of what can run and security policy
- Empowered support teams with self-service ops tasks
- Empowered developers with limited self-service operations
- Combined with new incident response model
Ticketmaster has seen transformative results. Before Support at the Edge, the average mean time to respond was 47 minutes. Now, support at the Edge has reduced that to just 3.8 minutes in addition to decreasing escalations 50 percent and overall support costs 55 percent. Ticketmaster has seen real results.

Damon has more details in his talk, which you can watch online here. If you missed any of the other 30-minute long presentations from All Day DevOps, they are easy to find and available free of charge here.

Finally, be sure to register you and the rest of your team for the 2017 All Day DevOps conference here. This year’s event will offer 96 practitioner-led sessions (no vendor pitches allowed). It’s all free, online on Oct. 24.

— Derek E. Weeks
June 12, 2017
Dev & Ops: These Are Reconcilable Differences

The new world of IT requires a different approach from what worked in the past. As usual, the technology is the easy part, but without people and processes being aligned, it won’t deliver value. When evaluating how data science and machine learning can help manage highly dynamic IT environments, old reflexes may be actively harmful.

By their nature and due to their experiences, Ops people tend to be a pretty (small-C) conservative bunch. Rule No. 1 of Ops is “If it ain’t broke, DON’T TOUCH IT!” and if we’re honest with ourselves, most of the rest of the rulebook is just variations and commentaries on that theme.

One of the ways in which Ops people try to control their world is through automation. DevOps mostly looks to the moment of release and deployment, as that is where the handover to Ops typically occurs—and this is a big focus of the drive to automate. However, there is another part of Ops where automation is key, something that Dev is not always nearly as involved in; namely, day-to-day operations. Which alerts should even be sent, whom they should be routed to, what are the valid responses and so on—all these have been automated over the years, to a greater or lesser extent.

Ops people have laboriously documented their architecture and spent a long time in meetings planning which information is relevant to share and who should respond to alarms. Much of this is implemented in various pieces of software, whether commercial, open-source, home-grown or that special grey area, where something that started out as a standard component has been so customized that it is effectively bespoke. Rules, filters and thresholds determine what action to take in which case.

These approaches have been in place for years or even decades, but now they are starting to fail. Simply put, IT’s increased complexity and the ever-accelerating rate of change are outpacing administrators’ ability to keep up and reconfigure their management systems.

The Coming ‘New Normal’

New approaches are emerging to deal with the new normal, but as ever, the technology is the easy part. For new technology to be a success, people and processes need to be in sync.

Right now, the emerging approach is to use data science and machine learning to process events, instead of the old deterministic rulesets and databases. The results speak for themselves, especially at scale and in highly dynamic environments. People building container-based or SDN architectures no longer assume that there will be a database of configurations that will always be up to date; quite the opposite.

The new way is to throw out all the filters and rules, feed all the alerts into a single place, and then use data science and machine learning techniques to sift them and identify relationships with them. The result will be a shift in focus from individual alerts to the business problems those alerts are the symptoms of.

Algorithms are great at these sorts of repetitive, high-volume tasks. They can look at massive volumes of events and figure out which are even relevant and worth taking a second look at, and then identify how those events relate to each other and what they really mean. This is where the humans come in, with their strengths in low-volume and unpredictable situations. Because they are not constantly drowning in irrelevant noise, they can work together effectively to understand the actual problem and get it fixed quickly.

Where’s the Catch?

The difficulty is that for many Ops people, there is a big cultural change required to adopt this model. The old deterministic systems may have required laborious maintenance, but at least it was possible to model them and understand why a specific result was achieved. New algorithmic techniques take a radically different approach, and while the results are impressive, they come from a black box. This does not inspire confidence for people who are very comfortable with the old approaches.

Often, the response of Ops people coming from legacy tools is to get stuck into the minutiae of what is going on inside the black box and to look for traditional failure modes. The problem is that these approaches are no longer valid in the new world, as I discussed in my last post about getting to SRE.

For instance, while it is indeed important to make sure that no issues are missed due to over-aggressive filtering, this does not mean that we need to see every single alert. As long as we know about the actual business problem, we don’t actually care about the single alerts, except insofar as they help us debug the underlying issue and resolve it.

The other direction is even more insidious, where people spend a lot of time chasing down “extraneous” events that they think they should not be seeing. The risk is, of course, that the Ops team ends right back up in the same place that they were trying to escape, where they are only able to identify expected problems, but are getting blindsided on a regular basis by the “unknown unknowns,” the conditions that simply should not happen.

Ultimately, Ops needs to let go of reflexes that were good in the old world, and focus on the goal: ensuring the uptime and stability of business-supporting systems. The how is less relevant than the what and the why; the algorithmic black box might as well be full of magic leprechauns, as long as they are telling Ops what they need to know, when they need to know it. Call it the “Chinese Room” theory of Ops: For as long as the results are useful to Ops, it does not matter what is inside the room.

The benefit for Ops is precisely that they no longer need to focus on the minutiae of individual alerts, but can concentrate on solving actual business problems. That is how IT can make itself a value generator and avoid being viewed as another undifferentiated cost center.

Pay no attention to the algorithm behind the curtain.

— Dominic Wellington

June 9, 2017
Q&A: Michael Beckley, CTO/CCO, Appian

Recently I had the chance to speak with Michael Beckley, chief technical officer and chief customer officer at Appian, about his company’s move to go public and the DevOps space in general. For those of you who don’t know Michael, he is a founder of Appian and leads the company’s technology vision and oversees customer initiatives worldwide. He also serves on the Industrial Advisory Board for the University of Virginia’s Computer Science Department and is a member of the University of Maryland Advisory Board in Computing. He has an interesting take on the IT space and DevOps in particular. Read on for our discussion.

Alan Shimel: Michael, what do you think has been the “special sauce” that has led to Appian’s success to date?

Michael Beckley: With Appian, companies build unique business applications up to 20x faster and mobile-enable and modernize existing systems. For example, a company like Ryder was able to build a web and native Android tablet application that cut the time it took to rent a truck in half.

But the “special sauce”? That’s Appian’s culture of innovation. Our engineers think long-term and invest in breakthrough tech like our patented SAIL UI technology for zero code native mobile apps. Thanks to SAIL, the web apps you configure in Appian automatically deploy to iOS and Android as native, full-featured apps or specialized micro apps we call “Sites.” As long as we get the culture right, we expect to continue to deliver enduring value for our clients.

Shimel: With becoming a publicly traded entity, some companies have found “managing to the street” does not match their own long-term business plans and their customers needs, how does Appian avoid this?

Beckley: Companies, like people, are an aggregation of their habits. Appian’s habit has always been to hold to our long-term vision of enabling every company to be a software company. We are not your typical Silicon Valley tech company. Our roots are in DC, we largely bootstrapped this business, and learned to be good stewards of capital while helping to support the core missions of federal agencies and some of the world’s largest brands. The proceeds following this IPO will be invested back into our business to benefit our customers.

Shimel: What do you see post-IPO being the biggest areas for growth as it relates to new products and services?

Beckley: We plan to invest in innovating and enhancing our platform. While our platform is industry-agnostic, we have recently made, and plan to continue to make, investments to enhance the expertise of our sales and marketing organization within our key industry verticals of financial services, health care and government. We have made, and will continue to make, investments in research and development to strengthen our platform and expand the number of features available to our customers. As we continue to increase the functionality of our platform and further reduce the amount of developer skill that is required to build robust applications on our platform, we believe that we have the potential to expand the use of our platform.

Shimel: Digital transformation is a term that is being used a lot these days. How far along the adoption curve are we on digital transformation?

Beckley: While many speculate that digital transformation is just a tech industry buzz word, in reality, digital transformation is a living, breathing effort. According to a recent survey, digital transformation is a C-level priority and is considered a top focus for almost 50 percent of all companies this year. From Appian’s perspective, we believe that the utilization of low-code software to drive digital transformation is still relatively new and our customers will be able to succeed in their digital transformation efforts by incorporating low-code development.

Shimel: Where does the DevOps market intersect with the Appian business model?

Beckley: Organizations are turning to DevOps to accelerate development and bring an application’s capabilities to a user quicker. The Appian low-code development platform fits into the DevOps model and supports the seven major DevOps categories including code, build, test, package, release, configure and monitor.

Overall, our business model supports the DevOps market; we’re living in a world where citizen developers are playing such an important role in the enterprise and our platform empowers the citizen developer while alleviating developer resources to focus on the more challenging, business-critical developer work.

Shimel: What about containerization and microservices? How do these technologies affect Appian?

Beckley: Appian already uses containers and microservices. Appian’s enterprise customers often require a mix of cloud, hybrid cloud, and on-premise installations. We view containers and microservices as enabling technologies to support the diverse workloads and deployment models required by our enterprise clients.

Shimel: What about security? What challenges does cybersec pose to Appian’s development of their platform?

Beckley: Appian works with some of the nation’s most important and highly regulated industries, therefore we have built a comprehensive security and compliance structure that meets several industry standards. We frequently undergo third-party audits to ensure that we are always operating effectively to protect our customer data.

— Alan Shimel

June 7, 2017
DevOps: Trust, Obligations and Promises
Mark Burgess (@markburgess_osl) is a theoretical physicist, but in his keynote at the 2016 All Day DevOps conference, he talked more about economics and human interactives than physics. What does either have to do as the keynote for a conference on DevOps?

Well, for a little more background, Mark Burgess is also the founder and former CTO of CFEngine, a configuration management and automation framework, and is the author of “Promise Theory.” While at CFEngine, Mark worked to apply a theory of how the autonomous agents in software interact with each other. Promise Theory was born.

Exploring Promise Theory

Promise Theory sits counter to obligation theories. Obligation theories view human interactions and behavior under the assumption that people (or agents, in the case of software) choose behavior based on their obligation to follow the rules. Agents are obligated to take a certain action. Promise theory contends the desire to follow the rules is voluntary and that agents can only be responsible for their own behavior, not the behavior of other agents.

In his presentation, Mark started out looking at human behavior and trends in our society. For instance, with technological advances, our need for one another is decreasing, even as our interdependence through communication is increasing. For example, our smartphones enable us to do almost anything without the direct interaction of someone else.

Looking at interactions in society, trust is the basis for our exchanges, working communities, and function as a society. He contends, “The promise is the source of intent by which all promises and observers may calibrate their expectations.” Money is an example of this. Money does not have intrinsic value, but we use it because we trust the backer will stand behind its value and it is more convenient than bartering goods. He explains that in the old monetary economy, banks and governments were the provider of trust; in the new service/IT economy, the service provider is the basis of trust.

How Agents Make Promises

How do you model this trust? Enter Promise Theory, which describes how agents make promises with each other and how this leads to trust between them. Mark outlines the “ingredients” of Promise Theory:
1. Agents and super agents make promises in a scaling hierarchy.
2. An agent can never make a promise about another agent’s behavior (only its own).
3. An agent’s promise need not be accepted or used by its intended or unintended recipient!
4. An agent makes its own valuations: what you have is only worth what another is willing to give you for it.
5. Dependency invalidates promises.
At the core of understanding this, you have to understand that collaboration is made up of semantics, what words mean and dynamics: all how things work. And that our ever-increasing service economy is driving us to an explosion of semantics, because words matter so much more than the mechanics. As an example, Mark sketched the hierarchy of promise dependencies in the old and new economies. Looking at the old economy, where we have more dynamic money, the hierarchy is:

Retail→ Money→ Banks→ Central Banks→ Government.

In the new economy, where we have more semantic services, the hierarchy is:

Client→ Network Availability→ Provider→ Internet Authorities→ Company/Service Organization.

The Need for Cooperation

None of these systems could work without cooperation at the bottom of it all. So, production is a cognitive process between the producer and the consumer, but also between the agents inside those organizations that have to interact. Often we create pipelines to describe these interactions, but the interdependencies within the systems are often much more complex than the pipeline can model.

This is all just the tip of the iceberg. Mark has three books that dive into it in detail: “Thinking in Promises,” “In Search of Certainty,” and, “Promise Theory.”

In the end, Mark made the point that technology does an excellent job of both binding us together and pulling us apart. We need to be careful that we don’t take ourselves out of the systems we create, or we could scale ourselves out of existence.

devopsy.kinsta.cloud Supports the All Day DevOps Conference

If this piqued your interest, you can watch Mark’s full All Day DevOps conference session (just 40 minutes). When you do, you can view the other 56 presentations from the 2016 conference online and free of charge. Finally, be sure to register you and the rest of your team for the 2017 All Day DevOps conference here. This year’s event will offer 96 practitioner-led sessions (no vendor pitches allowed). It’s free and online on Oct. 24. devopsy.kinsta.cloud is a media sponsor for All Day DevOps 2017.

— Derek E. Weeks
May 19, 2017
Avoid Crashes: 5 Best Practices for Load Testing

Demanding customer expectations and the rapid distribution of information across social media have made website and applications crashes an off-limits path for today’s companies and organizations.

Crashes resulting from traffic spikes cause immediate and long-term loss of sales, as well as tarnishing the brand name. When servers or websites go down during highly anticipated events such as Black Friday, the Big Game or before a major concert, the repercussions can be permanent.

Yet, avoiding the “website temporarily unavailable” screen is also easier than ever before. By performance testing your website continuously and proactively you can easily detect errors, bottlenecks and bugs, and then fix them to ensure your website handles any amount of traffic.

Here are five best practices for load testing your website or app:

Define Your Load Goals

Before starting your tests, you need to figure out what it is that you need to test.

This includes the number of concurrent users, traffic patterns and areas that users can visit on your site. For example, if your site has 10,000 users concurrent at its peak, but 3,000 arrive in the morning and that total slowly climbs up to 10,000 in the afternoon—and these visitors mostly just browse your homepage—you’ll need to run a different test than if you have 10,000 concurrent users on your website all day long, constantly browsing multiple pages.

To determine your load goals, you need to engage directly with your business colleagues. For instance, get your product manager to set the system’s expected abilities, ask the marketing team to inform you about any special campaigns that might increase website traffic, and then look at server logs and tools such as Google Analytics to determine previous customer patterns.

Choose Your Load Testing Tools

There are multiple open-source load-testing tools that you can use to run your tests.

JMeter is the most popular tool, but you can also use Gatling, Locust, Tsung, The Grinder or Selenium. See here to learn more about the differences between these toolsets.

Does this seem complicated? Taurus is an open-source automation test framework that simplifies running these tools, by using a simple YAML script. If you need to run your tests from the cloud, utilizing rich analytics or requiring user collaboration, you might consider using BlazeMeter.

Take Your System to the Limit

To make sure you stand up to your load goals, you need a buffer; specifically, you need to make sure that your system can handle heavier loads.

This ensures that the CPU stays low and you don’t experience memory leaks. However, we also recommend you take your system to the limit and see when and how it fails. This ensures that you know exactly what your system’s weaknesses are, giving you a better ability to fix any errors that might occur.

This is especially important if you come across an unexpected traffic spike—say, if your competitor’s website crashes and all their customers come running to you. Wouldn’t you rather be prepared for that?

Monitor Test Results

Set aside resources for looking at test results.

Examine KPIs including Response Time and Hits/s, as well as the correlations between them, to learn what your system’s abilities and bottlenecks are. This information will help you see what needs fixing.

As you continue testing over time, you will be able to identify KPI trends, which will help you monitor system health. We also recommend you monitor backend KPIs like DB Queries, with APM tools.

Test Again

So, you’ve tested, you’ve analyzed, and you’ve fixed.

Now you have one last thing to do: Run your tests again to make sure everything is working smoothly after the committed changes. That ensures nothing gets overlooked. It would be a shame if all your hard work went down the drain due to a minor oversight.

Now, let’s get testing!

To get more load testing tips, check out this free whitepaper, “How to Ensure Your Website or App Won’t Fail at Peak Times.” To learn more about BlazeMeter, request a demo.

— Noga Cohen

May 16, 2017
The Industrial Internet of Things Still More Promise Than Reality
New survey finds while the majority of executives expect big things from the Industrial Internet of Things they’re not quite yet prepared.

The Industrial Internet of Things holds enormous promise to transform business in how they manage their manufacturing processes, streamline their supply chains and directly service their customers. But, to date, that’s all it has largely been: a promise. According to a survey-based report released today by the BPI Network, there is broad consensus that the Industrial Internet of Things will provide significant value in the years ahead, but few organizations are ready to capitalize on the Industrial Internet of Things.

The report, “The Impact of Connectedness on Competiveness,” conducted by the BPI Network in conjunction with the CMO Council, Penton’s IoT Institute and The Nerdery. Based on an international survey of roughly 350 executives, the report includes insights from companies such as Airbus, Balfour Beatty, Embraer, Philips Lighting, GE, Whirlpool, LafargeHolcim, TVH, Hitachi and more.

Some statistics and predictions cited in the report are staggering: The business consultancy Accenture predicts that the Industrial Internet of Things will be incremental to the global economy to the tune of $14.2 trillion by the year 2030. GE, meanwhile, estimates the Industrial Internet of Things will improve its manufacturing productivity by 1 percent. And according to Juniper Research, there will be more than 46 billion connected Internet of Things by 2021—a 200 percent increase from last year.

But at the same time, executives are skittish. “We really don’t know what our long-term strategy should be,” said Dave Murray, head of thought leadership for the BPI Network, in an interview with devopsy.kinsta.cloud. “We haven’t thought through how this is going to change our business models. We don’t have the talent, in terms of you know, technology integration, and data integration, and analytics, to get the job done.

“I think there is a certain amount of concern of falling behind,” he added. “I think a lot of companies are just starting to incrementally experiment and are starting primarily with their own products. For instance, equipment manufacturers are integrating sensor technology into their products to improve uptime, and preventative and predictive maintenance.”

Top findings in the report include:
- Fifty-two percent of executives at large enterprises and 41 percent of executives at all companies expect the Industrial Internet of Things to have a significant or major impact on their industry within three years.
- Some 55 percent of all executives say the Industrial Internet of Things is gaining adoption within their industries, including both pilots and larger-scale adoption.
- However, just 1.5 percent of executives at large companies say they have a “clear vision with implementation well underway,” while another 57 percent are either beginning implementation, have pilots underway or are committed and in the planning stages.
- New products and services lead as the area most companies say they will focus their Industrial Internet of Things investments (35 percent), followed by customer touch points (29 percent), and manufacturing (23 percent).
- More cost-efficient operations (47 percent), product and service differentiation (36 percent) and improved customer engagement and satisfaction (34 percent) are seen as the top benefits of the Industrial Internet of Things.
- Security and data privacy are seen as top concerns by executives, followed by the cost and complexity of Industrial Internet of Things adoption and the need for new management and workforce skills and training.
While most of the executives interviewed, 62 percent, see the Industrial Internet of Things as a rising competitive necessity and are very interested in embracing the Industrial Internet of Things for their business—yet only 9 percent of respondents said industry peers are aggressively pursuing adoption.

Finally, one of the most surprising, if not startling, aspects of this survey is the finding that less than 2 percent of those surveyed said that they have a clear strategic vision with implementation underway. “I think what we’re going to see is that adoption is going to be incremental,” said Murray. “It can’t happen all at once. They don’t have it figured out. They don’t have the skills and technologies in place to make it all happen, but they are starting to implement and I think that [implementation] will just grow over time,” he said.

— George V. Hulme
April 25, 2017