devopsy.kinsta.cloud

Category: Features

  • Acquisitions to Fuel Programmable Analytics

    Acquisitions to Fuel Programmable Analytics

    Now that Tableau Software has agreed to be acquired by Salesforce and Looker is set to become a part of Google, the rate at which analytics will be embedded within almost every application is about to accelerate.

    Tableau today is relied on by more than 86,000 organizations to surface business intelligence via data visualizations created by professional analysts and line of business executives. That user base accounts for why Salesforce is willing to spend $15.7 billion to acquire Tableau.

    Competition within the analytics software space, however, has always been fierce. The rise of the cloud has resulted in several analytics startups challenging incumbents for a piece of one of the fastest growing sectors of the enterprise software market. One of those startups is Looker, which created a cloud-based analytics platform that Google is now set to acquire for $2.6 billion.

    Google has already signaled its intent to make Looker a part of its expanding Google Cloud Platform (GCP). As such, Google Cloud CEO Thomas Kurian last week noted one of the primary reasons for the Looker acquisition is to add a software-as-a-service (SaaS) offering that will enable developers to embed analytics within their applications by invoking a set of application programming interfaces (APIs) exposed via a cloud service. Today that analytics service runs on GCP alongside the Google BiqQuery data warehouse. Google, however, has also signaled its intent to make Looker available on multiple clouds alongside the rest of its software portfolio.

    Tableau, which will continue to operate as an independent arm of Salesforce, is playing catch up in the cloud. Tableau is deployed widely in both on-premises and public cloud computing platforms, including as a managed service provided by Tableau through which the company manages a dedicated instance of its software on behalf of customers. The merger with Salesforce will create an opportunity for Tableau to add a full software-as-a-service (SaaS) application instance to its portfolio that leverages the same infrastructure that Salesforce employs to deliver customer relationship management (CRM), marketing and customer service applications. Salesforce CEO Mark Benioff this week said going forward, Salesforce fully expects analytics generated by Tableau to drive a raft of digital business transformation initiatives.

    Most of the analytics that will be embedded within applications will manifest themselves in three forms. The first is embedded software that will be updated continuously as new data is entered in an application. The second is embedded predictive analytics modules that will invoke big data repositories and machine learning algorithms to surface recommendations on how to optimize a business process or customer experience. The third—and more ambitious—goal is an embedded prescriptive analytics capability that automatically optimizes business processes and customer experiences in real time based on events occurring, for example, within an internet of things (IoT) application.

    Every provider of analytics capabilities, from traditional Tableau rivals such as Qlik Software and Microsoft to TIBCO Software, IBM, MicroStrategy, SAP, Oracle and host of others, is pursuing all these emerging use cases for programmatic analytics. Most recently, Information Builders revealed it has embraced Docker containers and Kafka messaging platform as a means for achieving that analytics goal.

    The arrival of programmable analytics platforms will, of course, make implementing best DevOps practices all the more challenging. It’s hard to think of a business process that will not benefit to some degree from embedded analytics. The challenge DevOps teams will face is embedding and then continuously updating all the analytics modules embedded within the applications they deploy. The good news is that as organizations move to embed analytics everywhere increases, so too does the need to embrace DevOps to manage it all.

    — Mike Vizard

  • CloudBees Acquires Rollout for Feature Flagging

    CloudBees Acquires Rollout for Feature Flagging

    CloudBees today announced it would extend the reach of the Jenkins continuous integration/continuous development (CI/CD) platform by acquiring Rollout, a provider of feature flag tool that gets employed to simplify testing of new modules within an application.

    Company CEO Sacha Labourey said the acquisition of Rollout would result in a feature flag capability that is more seamlessly integrated with the gates that make up a CI/CD pipeline process. The goal is to make it easier for DevOps teams to apply A/B testing of new application features against a subset of users before deciding to update an entire application environment, he said.

    Feature flagging, also known as feature toggling, enables DevOps teams to hide features from certain users at runtime. In effect, this creates multiple branches from the same source code that DevOps teams need to manage. Rather than having to employ a separate tool based on an entirely different user interface, CloudBees is moving to make feature flagging a core capability of its extended toolchain.

    In theory, feature flagging should be more widely employed than it is today. Many organizations, however, have found it challenging to incorporate separate feature flagging tools within their DevOps processes. CloudBees is now using some of the money it has raised of late to address that issue.

    Feature flagging today is largely confined to testing. However, there may come a day when more organizations will employ feature flagging to make different classes of user experience available to different end users. That approach, in theory, would make it possible to charge different fees for disparate application experiences using the same core application.

    In the meantime, the goal is to garner more feedback concerning new capabilities that might be added to an application before the features are made generally available. Developers often spend months developing new features only to discover end users are not employing those capabilities because of some trivial glitch that could have been discovered easily using A/B testing.

    There currently is no shortage of options when it comes to tools for implementing feature flagging. Labourey said CloudBees elected to acquire Rollout because the team behind it was deemed to have the best cultural fit with CloudBees. It’s too early to say whether other providers of feature flagging tools will be rolled up into other CI/CD platforms—providers of those platforms can either develop their own feature flagging toolsets or move to absorb a third-party provider either by acquiring them or forming a technology alliance. The depth of the toolchain being provided these days often has as much to do with what CI/CD platform ultimately gets adopted as the core integration and delivery capabilities.

    Regardless of the path chosen, it’s apparent that thanks to feature flagging, continuous testing processes will become more sophisticated. That may require some adjustments to existing best DevOps practices, but the days when development teams blindly experiment on end users in the hopes they are building the next killer feature mercifully are coming to an end.

    — Mike Vizard

  • Which Desktop OS Is Best for DevOps?

    Which Desktop OS Is Best for DevOps?

    Which operating system (OS) is best for DevOps engineers to use on their workstations?

    That’s a weighty question. Talk about which OS is “best” for any purpose is always controversial.

    I’m not going to stick my neck out by saying that one OS is definitively better than others when it comes to doing DevOps work. But I would like to consider the strengths and weaknesses of each of the major desktop operating systems—Windows, Linux and macOS—for performing DevOps work.

    Windows

    Statistically speaking, Windows is the OS that most people use. By extension, it’s the OS that most DevOps engineers probably use. (I can’t confirm that fact, because I’ve yet to see a study of which OS DevOps engineers actually use.)

    As I see it, Windows has two main advantages from a DevOps perspective, including:

    • Popularity. This is a strength because DevOps is all about having freedom to pick and choose which tools to use. Given that Windows is so popular, almost everything is compatible with it. OS compatibility problems, therefore, don’t get in your way of doing DevOps as you see fit.
    • The Windows subsystem for Linux, which lets you run some Linux CLI tools on Windows. This feature is handy if you need to do DevOps work for both Linux and Windows server environments.

    The main drawback of relying on Windows is that, well, it’s Windows. Compared to Linux, Windows offers very little opportunity to customize, and that restraint can be a drawback from a DevOps perspective. When it comes to core tasks, such as working from the command line, you’re stuck with the tools Microsoft offers you. And you can’t do very much to harden your Windows system for security purposes.

    Linux

    For my money, Linux is the best OS for DevOps. But I say that only because I’m a die-hard Linux lover and I tend to think Linux is the best for everything.

    But even if you are not a Linux fanboi, you have to admit that it has certain advantages for doing DevOps work:

    • Linux is ultra-customizable. You can choose to set up your system in whichever way works best for your workflow. You can use a wide array of tools. You can make lots of changes to add extra security. Given that DevOps values choice, this is a big benefit.
    • In many cases, DevOps engineers are supporting software delivery pipelines that deploy to Linux server environments. Being able to work from a native Linux environment can make it easier to do things such as test and stage those applications on your local device when it becomes necessary to do so. You could, for example, spin up a Linux container on your local machine, which you couldn’t do with a Windows or macOS workstation.
    • On most mainstream Linux distributions, installing all of the DevOps tools you need is extremely fast and simple using repositories and package managers. It’s not exactly difficult to install most DevOps tools on other systems, either, but I don’t think any OS can beat the package management systems used by Linux distributions.

    Linux’s major limitation is that learning it takes some time, particularly for DevOps engineers weaned on Windows. You also run the risk of needing a tool that doesn’t support Linux, although I’m having a difficult time thinking of any DevOps tools that doesn’t work on Linux systems.

    MacOS

    Last but not least is macOS. Overall, I don’t think macOS is ideal for doing DevOps work. The only strength I can think of is:

    • It gives you a native Unix-like environment. You can’t run your Linux containers directly on a Mac, but you can at least use some native Unix tools.

    Drawbacks for macOS are many. It’s not very customizable. It lacks compatibility with some applications and tools. It comes with its own learning curve if you have never used a Mac before.

    Does OS Choice Even Matter?

    Now that I’ve weighed the pros and cons of Windows, Linux and macOS as operating systems for DevOps engineers, let me state the obvious: It’s 2019, and in many respects it no longer really matters which OS you run on your local machine. Lots of applications—including key DevOps tools, such as IDEs—now run in the cloud as often as not. Plus, there are a plethora of hardware virtualization tools out there that will let you run virtually any OS in a virtual machine within another type of OS. You can run Linux and Windows side-by-side as virtual machines on your Mac, if you want.

    Still, not everything runs in the cloud, and having to switch between virtual machines is not always practical. It’s therefore worth thinking hard about which desktop OS will best serve your DevOps needs and making that the one that controls your bare metal.

    — Chris Tozzi

  • Textbook DevOps vs. Real-World DevOps

    Textbook DevOps vs. Real-World DevOps

    DevOps is one of those things—like Marxism or baking macarons—that is easy to understand in theory, but much messier to implement in practice. Real-world DevOps rarely aligns with textbook DevOps or the DevOps practices that organizations should follow in theory.

    That is true for a number of reasons. Let’s walk through some of the biggest below.

    Your Company Is Not Organized in the Way DevOps Theory Assumes

    According to DevOps theory, every company has an IT department and a development department. Doing DevOps boils down to making those departments work well together.

    The reality, of course, is that not all companies organize their staff or technical operations in this way. Some don’t silo developers and IT engineers by default. Others (especially smaller companies) don’t have developers or IT staff on hand at all; they outsource that work.

    These companies can still benefit from DevOps principles such as collaboration and automation. But they should avoid fixating on DevOps teachings regarding the organization of technical work.

    Real DevOps Skills are Hard to Find (And Expensive to Obtain)

    “DevOps engineer” is now a trending job title. But few technical training programs have caught up with the demand for DevOps.

    Most universities still silo majors and courses in software development away from those that focus on IT. Other types of technical training programs function similarly.

    What this means is that it is more difficult to hire a DevOps engineer, or other technical staff with a broad set of development and IT skills, than it may first appear. DevOps engineers exist, but they are usually experienced staff who learned their real-world DevOps skills on the job. They’re more expensive to hire and difficult to poach from their current gigs.

    Full Automation Is a Pipe Dream

    The DevOps mantra tells us that we should automate everything, from software testing and builds to deployment, production-environment monitoring and incident response.

    Automation is great in theory. But the reality is that you can never automate everything. Human engineers will still need to respond to problems occasionally, or to unanticipated issues that arise during testing, deployment and production management.

    An important point to emphasize here is that even if you successfully automate virtually all of your processes, the mere fact that human engineers must be available to respond when something does go wrong undercuts the idea that you can rely on automation alone. Humans still have to watch the automated processes, even if they rarely need to intervene. The simple act of watching and being available to respond when necessary is a significant burden.

    Agility Conflicts With Simplicity and Reproducibility

    In some respects, DevOps is a contradiction. On the one hand, it prioritizes agility and scalability. On the other, it emphasizes simplicity and reproducibility.

    What this means in practice is that DevOps teams adopt microservices and containers in the interest of maximizing agility. But microservices and containers are much more complex to manage than monoliths and virtual machines.

    I suspect the reality is that most DevOps organizations end up erring toward more complex technologies and architectures. They ultimately find greater value in them than they do in keeping things simple.

    Still, the idea that you can be agile and be simple at the same time doesn’t apply well in the real world. (And sure, automation, infrastructure as code and the like can help in simplifying the management of complex agile technologies, but they only go so far.)

    Most Organizations Still Aren’t Doing DevOps

    If you spend your time surrounded by other people who have adopted DevOps and implemented it successfully, it’s easy to assume that the rest of the world is like you.

    The reality, however, is that we’re now a decade into the age of DevOps, and yet a majority of organizations have made only halting steps toward implementing basic DevOps processes. If DevOps were half as easy to implement as it appears to be when you read the latest book about how to do DevOps, many more companies would be doing CI/CD by now.

    Conclusion

    DevOps delivers real benefits, and it’s important to continue preaching the DevOps gospel to organizations that can benefit from DevOps philosophy and practices.

    However, it’s important to balance the conversation about DevOps with the reality: Real-world DevOps is much messier than it often seems in theory. Not many organizations have the configuration or resources to do textbook DevOps.

    — Chris Tozzi

  • Open Innovation Labs: How Lockheed Brought Better Security to Fighter Jets

    Open Innovation Labs: How Lockheed Brought Better Security to Fighter Jets

    How Red Hat’s Open Innovation Labs have helped accelerate Lockheed’s software development for the F-22 Raptor

    As the key government contractor charged with helping the U.S. Air Force keep its F-22 Raptor jets in fighting trim, Lockheed Martin Aeronautics increasingly values software delivery speed as much as engine speed. So much of the continued evolution of these fighter jets depends upon continual upgrades of the software code powering all of the physical systems built into them, from tip to tail.

    Until a year ago, these jets, which first rolled out of the manufacturing plant in the 1990s, were supported by the waterfall development methods of that same era. Lockheed executives and Air Force brass recognized that the F-22 needed a DevSecOps transformation, and fast.

    “When you have a world-renowned platform like the F-22 Raptor, adversaries are constantly looking for ways to counter it,” said Michael Cawood, vice president of F-16/F-22 product development at Lockheed Martin. “This means we must constantly add capabilities and improve the F-22. And we must do it faster than ever before. We needed to transform our own organization and how we did things.”

    To accelerate that process, Lockheed last year sent a small contingent of engineers through a residency program at the Red Hat Open Innovation Labs to fast-track the process of implementing an open source architecture onboard the F-22 while adopting agile development methodologies and DevSecOps practices. Since bringing back their learning to the organization, these engineers helped to catalyze a faster development process that has helped Lockheed introduce software features for the F-22 years ahead of schedule.

    How Open Innovation Labs Works

    Now in their third year of operations, the Open Innovation Labs were developed by Red Hat as immersive training grounds to help its enterprise customers figure out how to deliver better software, faster. A standard engagement has companies such as Lockheed pay for residency for anywhere between three to six developers and operations staff to get paired one to one with Red Hat employees—engineers, consultants and others—with deep knowledge in DevOps and agile development. They work with them in a time-boxed four- to 12-week engagement to build out application prototypes using DevSecOps practices along the way. The residents are also teamed with a designer and an engagement lead who doubles as an agile coach, typically working with the customer’s product owner for the prototype.

    Obviously, Red Hat has a vested interest in training these people in technology such as its OpenShift Container Platform, getting them up to speed on how to orchestrate and manage containers and microservices. But the labs approach is in a different league than the standard vendor training program. It more closely follows the DevOps dojo model that enterprises like Target and Capital One are running internally to scale up cultural and process transformations across their IT organizations. Like those dojos, the working model is to build real software through a guided learning process that helps participants get hands-on experience in how to change the way they deliver software.

    Red Hat runs the labs either out of a handful of dedicated spaces it runs around the world or within pop-up labs of the customer’s choosing. Besides Lockheed, other notable alumni include the likes of Volkswagen, UNICEF and Heritage Bank. According to a recent analysis by Forrester Consulting, participants in the Open Innovation Labs program are accelerating application development by more than 60%.

    Lockheed’s Accelerated DevSecOps Transformation

    In Lockheed’s engagement with the Open Innovation Labs, the organization sent out a cross-functional team of five developers, two operators and a product owner. They were tasked with building out a new application for the F-22 on OpenShift, using that process as a model for future development workflows. As a part of the process, Lockheed also worked with the labs to help it build out an internal dojo that could facilitate the scale-out of the knowledge gained by its small team across the entire F-22 development team. Six months after the initial residency, Lockheed managed to scale its OpenShift deployment and DevSecOps transformation across its entire 100-person team.

    “By working with the Red Hat Open Innovation Labs team, we changed everything—our toolchain, our process and, most importantly, our culture,” explained Cawood.

    As a result, the F-22 development organization has been able improve its ability to forecast for future sprints by 40%. More importantly, it has cut what was once an onerous five- to seven-year process of rolling out new software capabilities for the aircraft. Lockheed is on track to deliver new communications capabilities for the F-22 this summer a full three years ahead of schedule.

    — Ericka Chickowski

  • Digital Transformation Vital for Business Survival

    Digital Transformation Vital for Business Survival

    Businesses are in a fight for their survival, and they know it. Their direct competitors, large and small, are digitizing their businesses. And those businesses that effectively digitize first have much to gain competitively. However, as we’ve reported in “Digital Transformation: Coming from Behind” and other stories, many organizations are experiencing significant challenges when it comes to executing on their digital transformation strategies.

    To get a sense of where companies are in their efforts, managed SD-WAN and cloud services provider Masergy contracted research firm Webtorials to conduct its 2019 Digital Transformation Market Trends Report. The report is based on a survey of large and small companies in 35 countries, and it found that the early digital transformation wins tend to be among customer-facing investments, such as customer service and customer experience.

    Not surprisingly, the report found 78% of respondents believe that their company’s primary industry is rapidly evolving and that digital transformation is necessary to survive the change underway. The benefits respondents cited (ranked in order) were enhanced customer experience and customer relationship management, the ability to maintain competitiveness, enable new ways of doing business, increased productivity, cost savings, increased revenue generation and enhanced product development.

    With the majority of digital transformation efforts failing to meet their established objectives, businesses are clearly facing no shortage of obstacles. According to the Masergy survey, the challenges respondents cited as barriers to a successful digital transformation effort, in order of difficulty, are budget commitment, executive support, in-house skill sets, organizational buy-in and lack of adequate IT infrastructure.

    These findings paralleled findings from other related and recent surveys, including the “State of Digital Transformation” report from research and advisory firm Altimeter. The report found that CIOs (28%) or even CEOs (23%) typically own digital transformation at an organization. That report also found that enterprisewide support is a leading barrier, with many organizations citing the belief that digital transformation is a cost center (28%), elusive ROI (29%) and resistance to change (26%) also are barriers.

    The survey also found that only one-third of respondents believe they can meet their digital transformation goals without help from a solutions provider. This finding parallels other reports. Earlier this year, IDC found that business services related to digital transformation will grow at a healthy 29% annual rate over the next five years.

    Interestingly, the “2019 Digital Transformation Market Trends Report” from IDC found those looking for outside help prefer smaller service providers (58%) rather than larger providers (43%).

    What technologies most support digital transformation? According to this survey, those that help in migrating to cloud or in improving security, as well as data analytics and AI, mobility, collaboration and emerging technology such as virtual reality and IoT.

    These reports show that while businesses are aware of what is at stake for their enterprises, the bad news is they are having a hard time making headway. The good news is that it doesn’t appear too late, just yet, for businesses to get their digital houses in order.

    — George V. Hulme

  • Technology: Isolation in the Age of Automation

    Technology: Isolation in the Age of Automation

    Dependence on technology makes us more independent overall, but it also makes us more isolated. So, what happens if things come to the point where we depend on machines more than we depend on other human beings?

    Earlier this month I attended a presentation about machine learning on mobile phones at Google IO 2019. Right now Google is spending a lot of time and money developing machine learning models and capabilities that operate solely within the mobile device. No connection to the internet is required. It’s a big deal.

    During the presentation, the company showed a video that featured an illiterate woman in India who uses her phone’s text recognition features to read to her. All she needs to do is point the phone’s camera at a recipe or newspaper and the phone converts the text to speech.

    I found the story inspiring as a technologist and as a human being. But it’s not all peaches and cream. At the end of the video, the woman said something that gave me pause. When talking about using the technology to book a train ticket, she said, “I can book it myself. I don’t need anyone’s help.”

    There’s no doubt the phone liberated her, very much in the same way that the light bulb liberated millions of other people in times past, bringing human activity forward beyond the setting sun. Dependence on a particular technology can create greater independence overall. However, I do wonder what happens when things evolve to the point where we depend on machines more than we depend on others, when we no longer need “anyone’s help.”

    To my thinking, human interdependence is the thread that binds the fabric of a society together. The thousands of little interdependencies that make up the human experience have beneficial side effects. To get along, we learn the basic rules of civility. We learn to say “please” and “thank you.” We learn how to wait in line at the DMV. We pick up after our dog. We open the door for others. It’s all part of the implicit contract that makes living with other people possible.

    But, these days it seems we depend more on machines than the kindness of strangers. The fact is, you don’t have to be nice to a machine, even for a self-serving purpose. You can yell at your cell phone. You stick your tongue out at the ATM as it accepts your deposit. Whether you’re nice to it or not doesn’t matter. Your deposit is not going to be processed any faster or slower. Your cell phone isn’t going to disconnect you because you were rude to it.

    Take away these simple rules of civility and strange things happen. It’s interesting that road rage is common, yet sidewalk rage is rare. Maybe wrapping ourselves up in a piece of silicon-powered transportation hardware predisposes us to a hostile attitude when interacting with others in the same situation.

    As I stated earlier, depending on technology makes us more independent in the big picture. But, it also makes us more isolated. Whereas in the past, talking about the weather with a stranger might be a prelude to another conversation that’s more meaningful, today the stranger next to me usually has his ears plugged up listening to his own interests. And, if I really want to know about the weather, I can ask Alexa.

    Modern technology gives me help on demand. I no longer ask a stranger for directions to the train station, I use my phone. It seems as if I need the help of others less during those small, random moments of vulnerability. A machine is but a voice prompt away. And, thus, the thread that binds me to the world at large gets a little bit looser.

    Now don’t get me wrong—isolation is nothing new. Five hundred years ago most people never traveled far from where they were born except on very special occasions. Village life is a case study in group isolation. But, while villages were isolated from the outside world, inside the village it was a different story. The social fabric was dense; the threads were tight. Being shunned by the group was a matter of life and death.

    Today we’re all over the place, literally. But we move in our own domains in which we end up wishing happy birthdays to people we’ve never met but who somehow have meaning to us by way of LinkedIn, Facebook and the plethora of other social platforms brought to life and sustained by something other than human labor.

    Is this such a bad thing? Dunno. I like it that my phone is now my personal stenographer. It helps me think better. But, I’m not sure that Google Assistant will ever be a good best friend. What’s even stranger is that a time might come when I don’t even need one. It will be just me and my mobile device. Then I won’t need anyone’s help, ever.

    — Bob Reselman

  • Microsoft Gains DevOps Momentum

    Microsoft Gains DevOps Momentum

    Microsoft, at its recent Microsoft Build 2019 conference, launched a series of efforts intended to make it easier for organizations to embrace best DevOps practices, including making it possible to address both continuous integration and continuous delivery within the same YAML file using Azure Pipelines.

    Microsoft also announced it will extend Azure Pipelines to provide tighter integration between GitHub, which Microsoft acquired last year, and Kubernetes. Via that integration, Azure Pipelines can be used to automate everything from creating container images and packaging them as Helm charts to generating Kubernetes deployment artifacts and then deploying the workload.

    Continuing that Kubernetes theme, Microsoft also is launching the Kubernetes-based Event Driven Autoscaling (KEDA) project, which makes it possible to run Azure functions on top of Kubernetes clusters. Developed in collaboration with Red Hat, KEDA is available in developer preview for the Red Hat OpenShift platform based on Kubernetes.

    In addition, Microsoft has extended the Azure IoT Edge runtime to Kubernetes to enable internet of things (IoT) applications to be deployed on Kubernetes and Azure SQL DB Edge, which is packaged as a container and makes it possible to deploy a database on ARM or x86 systems deployed at the network edge.

    Also during the conference, Microsoft launched MLOps, which applies DevOps principles to deploying artificial intelligence (AI) models using the GitOps workflow; Azure SQL Database Serverless, which allows organizations to consume a database service on a per-second billing basis on the Azure cloud; and ML.net, a framework for building AI models.

    Microsoft also announced it is now making available a full Linux subsystem within Windows 10 and the ability to link IoT devices to the Azure cloud without writing code.

    Scott Guthrie, executive vice president for AI and cloud at Microsoft, told conference attendees Microsoft is squarely focused on making it easier to build applications that can be deployed in hybrid cloud computing environments anchored around the Microsoft Azure cloud.

    Microsoft historically has focused on making it much easier to consume emerging technologies at the point in time they start to appeal to the broadest number of customers. As DevOps best practices continue to find mainstream adoption, that same philosophy is being applied to cloud services. Microsoft may not have invented the concept of CI/CD, but it is working to make it easier for organizations that have struggled to implement continuous delivery, for example, to master that challenge. The same concept applies to Kubernetes, which some argue is still too complex for the average IT organization to master and deploy.

    It remains to be seen how much Microsoft’s efforts, most of which are only available as previews, will result in more applications being deployed on Azure. Because an application is built on a public cloud doesn’t always mean it will be deployed on that cloud. It does, however, increase the chances of that occurring. In fact, providing a better DevOps experience across a hybrid cloud computing environment is clearly key to Microsoft’s strategy as it seeks to gain ground on Amazon Web Services (AWS).

    — Mike Vizard

  • DevOps Chat: OPA and the CNCF, With Torin Sandall

    DevOps Chat: OPA and the CNCF, With Torin Sandall

    The Open Policy Agent (OPA) was accepted for incubation by the Cloud Native Computing Foundation. In this DevOps Chat we spoke with one of the founders of the project, Torin Sandall, about OPA and why the CNCF is the right place for OPA. We also discuss the company he works for, Styra, which works with OPA.

    As usual, the streaming audio is immediately below, followed by the transcript of our conversation. Enjoy!


    Transcript

    Alan Shimel: Hey, everyone. This is Alan Shimel, MediaOps, the people behind DevOps.com, Container Journal, Security Boulevard and DevOps Chat, and you’re listening to another DevOps Chat.

    I’m really happy to have as a guest for this DevOps Chat, Torin Sandall, and Torin is actually one of the co-founders of the Open Policy Agent (OPA) open source project, which recently came under the guidance management of Cloud Native Computing.

    So, let’s welcome Torin. Torin, you there?

    Torin Sandall: Hey, Alan. Yeah, great to talk to you.

    Shimel: Great to have you here, and a pleasure to have you joining us on this podcast. Torin, let’s first start off with, maybe there are people out here who don’t know what OPA is—what Open Policy Agent is, right? There’s some great, if you go to—actually, if you go to CNCF.io, there’s a blog article all about it and I’ll put it in our show notes. But for those who are following along or listening to this in their car or what have you, talk to us—what’s Open Policy Agent?

    Sandall: Yeah, that’s a great question, and it’s a good way to get started.

    So, Open Policy Agent is a domain agnostic policy engine, okay? So, it’s general purpose, meaning that you can kind of take it and use it to enforce all kinds of policies and governance regimes across a wide range of technology.

    So, for example, if you want to say that, you know, all of your containers have to be sourced from an internal image registry, you can express that kind of a policy with OPA, or the Open Policy Agent. If you want to do something like say that certain users can only see certain pieces of data served by a micro service, you can also do that with the Open Policy Agent. And if you want to restrict who can SSH into a server or run a pseudo command on a server, you can also do that with the Open Policy Agent.

    So, the idea is that we give people this engine that allows them to unify policy control across various pieces of their stack.

    Shimel: Excellent. And you know what’s interesting? I was talking to someone the other day and we were talking about, you know, how does Kubernetes fit into this whole Cloud Native thing? Because Cloud Native and the CNCF is bigger than just Kubernetes. And they gave me a really good analogy. They said, “Think of it as like, you know, a fleet of warships or, you know, a U.S. Navy fleet and maybe Kubernetes is the aircraft carrier. But there’s more to the Navy than the aircraft carrier. There’s more to a fleet than an aircraft carrier. There’s battleships and cruisers and destroyers and subs and frigates and amphibious landing ships,” and you know, I’m not a Navy expert. [Laughter] But there’s all these different elements that make up the fleet—and it’s a huge fleet.

    And so, you know, when I look at OPA, you know, Open Policy Agent—and we’re gonna talk about joining Cloud Native Computing Foundation—it’s a necessary ship in the fleet, right? That really makes it easier and more practical to use a lot of the tools that are not only in the fleet, but a lot of the commercial tools that kind of exist on the periphery as well. Fair?

    Sandall: Absolutely, yeah. The CNCF is a collection of projects that provide a lot of value within the overall ecosystem, and there are a few things that kind of bind them together, and those are kind of high level principles. So, you know, they’re all open source, they are all, you know, they’re oriented around containers. They’re oriented around very dynamic systems that you need to manage at scale.

    And so, one of the interesting things about a lot of the projects within the CNCF is that they are kind of building blocks. You can think of them as libraries or building blocks for the overall system, right? And so, you know, they integrate together, you can plug them into other systems and so on. And so, that’s how we kind of look at OPA. That’s how we think about OPA, is as a building block or as a library for enabling rich, fine grain control across a variety of parts of your environment.

    Shimel: Absolutely.

    Sandall: And, yeah, and so, policy is particularly important when you start to talk about adoption of technology within large organizations, right? As Kubernetes has matured and enterprises have really started to run with it and deploy real production workloads on top of it, they’ve recognized the need for not just simple best practice kind of security measures, but much more sophisticated and rich and fine grained rules and constraints that need to be enforced in a system in order to keep everybody safe. And so, that’s where OPA fits into Kubernetes.

    Shimel: Yeah, absolutely. You know what? I realized, and I apologize, I really didn’t do a good job of letting you introduce yourself to our audience on top of [Cross talk].

    Sandall: [Laughter] That’s okay.

    Shimel: So, Torin, I apologize. In addition to you being one of the co-founders of the project, why don’t you share a little bit about what you do in your day job and the rest of your life?

    Sandall: For sure, yeah. So, I think what’s relevant here is that I’m the co-founder of the Open Policy Agent project. I’m also a software engineer at a startup called Styra. We’re the makers of the Open Policy Agent. We founded the project about three years ago—a little over three years ago, actually, and yeah, so we’re kind of like the stewards of the project today. That said, we’re starting to see a lot more people from the community kind of come in and start contributing to Open Policy Agent and related kind of projects in the ecosystem.

    So, that’s sort of what I do. My focus is kind of on Open Policy Agent, but I’m super interested in the overall Cloud Native ecosystem and security and policy and all that kinda stuff.

    Shimel: So, you know, and I should also mention—I did a terrible job laying this one out, Torin, I apologize. We should also mention that the big news coming out—I guess it was this week, or maybe late last week—was that OPA, Open Policy Agent, has now become or come under the auspices of Cloud Native Computing Foundation (CNCF).

    Sandall: That’s right, yeah. So, we actually, we joined the CNCF as a sandbox to your project last year, in 2018. OPA was added into the CNCF sandbox. So, the CNCF has multiple tiers, and right now there’s sandbox, incubation and graduated.

    And so, this year, basically, they do a review of projects, you know, every year, and they check in on how the projects are doing and how they’re growing and how their communities are evolving. And that’s just useful, right? To kind of maintain, kind of, you know, good hygiene within the ecosystem. But this time around, we felt that—everybody kinda felt that OPA was growing quite a bit and it kind of deserved to move up to the next level incubation within the CNCF.

    And so, the move from sandbox to incubation is really a reflection of the progress that we’ve made over the last year in terms of both production users as well as proof of concepts and integrations, and new external contributors joining the project.

    Shimel: Got it. And, you know, I will just mention that, you know, CNCF has really done an amazing job of shepherding these projects, from sandbox into incubation, and then from incubation into graduation. I forgot the number, if it was five or six projects they’ve graduated now.

    Sandall: Yeah, I think there’s about five graduated projects today. You know, Kubernetes is one of them, it’s kind of one of the flagship projects, and there are others. And so, yeah, so, they’re really doing a good job of curating a very important set of building blocks, like I said, that are gonna help organizations get to cloud native.

    Shimel: So, let me ask you a question—and I think this is something our audience would really enjoy. How did you, as a co-founder of the OPA project, right, what was your sort of motivation for, you know, bringing it under the CNCF?

    Sandall: So, yeah, that’s a great question. What we found was that after, you know, the project had been around for over a year, or rather over almost two years, I guess. And what we found with talking to users and potential users was that they wanted to see OPA as part of a kind of vendor neutral organization, right? So, Styra, the company that I work for, had kind of built OPA, and it’s been open source since day one, but people really like to see these kinds of projects, these kinds of core infrastructure components a little bit decoupled from, like say, any particular organization.

    Shimel: Mm-hmm.

    Sandall: And so—

    Shimel: Well, not decoupled from any particular organization. Decoupled from any one particular vendor.

    Sandall: One vendor—yeah, sorry. One vendor.

    Shimel: There’s not a single vendor.

    Sandall: Yes. And so, you know, we looked around and we looked at the CNCF and we really liked what they were doing and we figured that that was the right place for OPA to be.

    So, you know, the CNCF basically provides us with this kind of vendor-neutral home for the project, and it allows us to kinda create a rallying point around policy in the cloud native ecosystem. So, that was the main, high level motivation for joining CNCF.

    Shimel: Got it. And it makes perfect sense. I mean, I’ve gotta tell you, you know, I don’t know if you’re familiar, but recently, the CNCF launched—not the CNCF, excuse me—the Linux Foundation, right, which is kinda the parent of the CNCF, right?—

    Sandall: Right, yes. Yes.

    Shimel: —launched another foundation called the CD Foundation, Continuous Delivery.

    Sandall: Yes, yes.

    Shimel: And CloudBees, the folks at CloudBees and the Jenkins project was given under to the CD Foundation. Jenkins had Jenkins X, Netflix put in Spinnaker and Google put in—I forget its name now, with a T.

    Sandall: Tekton?

    Shimel: Tekton. They put, you know, put Tekton into the thing. And I had a good chance to interview some of the CloudBees folks with this, and I asked them the same kinda question I just asked you. You know, and it was a very similar answer, that not only do end users perhaps shy away from a single vendor supported project, but it’s very hard for a project to break out of that mold and get other vendors or talent at other vendors to contribute, because they think they’re doing it for the benefit of one of their competitors.

    So, by putting it under a neutral party like this, you know, not only is it more palatable to your end users and to the community as a whole, but it’s especially more attractive to that segment of the community that are made up of vendors. And whether you hate vendors and you think they’re all commercial slime who sold out to the man or not, the fact of the matter is, those are the people who have the resources to put into these projects and to move the needle and to keep that ball moving forward and push the rope up the hill and all of that. So, can’t live with ‘em, can’t live without ‘em, and this is a great kinda middle ground.

    Sandall: That’s a good point, and I think that, you know, what we’re starting to see are more and more security vendors getting interested in OPA or Open Policy Agent. I like to call it OPA. It’s hard to call it O-P-A when you’ve got that OPA word that you can use, but anyway—

    Shimel: Yeah, yeah, yeah. [Laughter]

    Sandall: [Laughter] But yeah, so, we’re seeing more and more security vendors get interested in OPA and, you know, that is gonna be instrumental in the long run to achieving the goal of unifying policy control across a number of different systems, right? There’s never gonna be one vendor that rules everything and there’s always gonna be an amalgamation of different technologies. And so, you know, the best that we can do, I think, is provide this building block that can be embedded all over the place and that has, like, the fundamental kind of groundwork that enables unification. So, very exciting times.

    Shimel: Absolutely. So, I’m gonna ask you now to put on your entrepreneur’s hat, right? As a co-founder of your company as well, and let’s look at it Torin from that point of view. How is this helping—and I’m not familiar with your company, so I’m gonna apologize in advance. Say the name for us, again?

    Sandall: Sorry, the name of the company is Styra.

    Shimel: How do you spell that?

    Sandall: S-T-Y-R-A.

    Shimel: Got it. So, how does this help Styra?

    Sandall: That’s a great question. So, yeah, when the founders—I’m not a founder of Styra, I’m an employee at Styra.

    Shimel: Oh, I’m sorry. Okay.

    Sandall: But anyway, when the founders of Styra started the company, one of the things that they observed was that, in order to unify enforcement of policies across a wide range of technology, there would need to be kind of like a language based approach applied to the problem domain.

    And so, that’s one of the things that the Open Policy Agent provides—it gives people this high level declarative language that they can use to codify policies. And so, that was kind of recognized as something that would be needed, and what you quickly kind of—the conclusion you quickly arrive to is that, this was in 2016, 2015, and even more so today, it’s true, but what you quickly arrive at is the realization that that kind of a technology needs to be open source, right? It’s unlikely that organizations are going to want to adopt a proprietary, closed source language. It’s just not going to happen.

    And so, right from the very beginning of the company, it was recognized that an important part of the technology was going to be open source. And so, that’s how the kind of project sort of got started.

    Shimel: Got it. Got it, got it, got it. You know, I don’t wanna put you on the spot, but a little Styra background—Styra’s venture backed or bootstrapped, or?

    Sandall: Yeah, we’re venture-backed, and we recently did a bit of a launch at the RSA Conference a few weeks ago.

    Shimel: Really?

    Sandall: And so, that was very exciting, yeah. And so, you know, you can think of Styra as a control plane for policies. So, OPA kind of acts as the data plane, right? It’s this thing that you can embed into applications, you can embed it into Kubernetes, you can put it all over the place. But, what you need is also the ability to kind of manage those OPAs that are running all over the place.

    And this is what we see within the OPA community—the people that are using OPA building it into their systems. Companies like Netflix that have built an internal security platform around OPA, what they’ve ended up doing is creating a centralized management plane for policy and for security.

    And so, that’s what we’re doing at Styra is, we’re helping people kind of think about distributed enforcement of policy and centralized management of policy.

    Shimel: Got it.

    Sandall: So, we’re focusing today on Kubernetes and you can check out the website, we’ve got a bunch of information on there, but yeah, that’s the high level.

    Shimel: Excellent stuff. As a matter of fact, just looking at the website, I didn’t realize this but Styra is Bill Mann’s company.

    Sandall: That’s correct, yeah, he’s the CEO.

    Shimel: So—just to show you how old I am. I’ve known Bill Mann a really, really long time, like, I remember the day he joined CA.

    Sandall: Bill knows a lot of folks, yeah. [Laughter]

    Shimel: Yeah, he gets around. Bill’s been around, and so have I.

    Sandall: Yeah.

    Shimel: I don’t know if that’s a good or a bad thing, you know? But hey, we’re still kickin’. So, tell Bill I said hello, he’ll get a chuckle out of it.

    Sandall: Yeah. I will do that, yeah. And we’ve got a great founding team. They are some of the folks that did Nicira, which is one of the very, you know, basically, the first SDN kind of company, right? They [Cross talk]—

    Shimel: Yep, I remember that one as well.

    Sandall: Yeah. So, we’ve got a deep background within the security and policy space, and we’re kinda taking that and really bringing it into the cloud native ecosystem.

    Shimel: Very cool. You know, I don’t wanna make this about Styra because we really wanted to concentrate on OPA and CNCF, but maybe we’ll still, maybe we’ll schedule something, a second one, and we’ll talk—we’ll dive more into Styra.

    Sandall: Yeah, that would be great.

    Shimel: You guys are gonna be at KubeCon in Madrid—or I mean, in Barcelona?

    Sandall: We are gonna be in KubeCon, yes, in Barcelona in May. There are going to be a bunch of talks about Open Policy Agent. There’s gonna be, I think there’s at least three or four sessions that are happening. There’s also a little thing called Cloud Native Rejects the weekend before. [Laughter] These are the talks—like, the BSides Talks, basically for—

    Shimel: Yes. Well, that’s how Security BSides started.

    Sandall: Yeah, exactly. And so, there’s gonna be a session there with OPA content in it. So, yeah, it’s gonna be a great event.

    Shimel: Excellent.

    Sandall: And, of course, we’re gonna have a Styra booth at the event, so.

    Shimel: So, we will have a, you know, Digital Anarchist is our video platform, and we’ll have a Digital Anarchist set there where we’ll be doing videos. So, be sure—we’ll talk offline, but we’ll get you guys in for a video. We’ll follow up more on that.

    Sandall: Perfect! Yeah, looking forward to it.

    Shimel: So, is there a timeline for graduation from incubation at CNCF, or that’s kinda open?

    Sandall: It’s open. So, you know, they do a yearly review of the project, and it’s kind of up to, you know, the community and CNCF and everybody to kind of decide when that’s right. And so—yeah, so right now, one of the main criteria for graduation is around external contribution. And so, you know, over the next year, we’ll see how external contribution around OPA evolves and if it’s in the right place, then we’ll go ahead with that.

    But we’re really happy right now just with the move to incubation because it brings more visibility to the project, it lends more credibility to the project—you know, in addition to all the people that are using it in production, having it more prominently represented within the CNCF is great. So, we’re really happy with where we are today.

    Shimel: Fantastic, man. Hey, Torin, I had promised you we’ll do this in 15 minutes, and that was 25 minutes ago.

    Sandall: [Laughter]

    Shimel: So, I apologize for running over, but I thought we had a good conversation here.

    Sandall: Yeah, yeah. Thanks a lot, Alan. Yeah, I appreciate the time to talk about OPA and Styra and I look forward to—

    Shimel: Oh, it was really my pleasure. I’d like to—we’ll continue this at maybe KubeCon in May. But before we sign off—so, Styra is Styra.com, S-T-Y-R-A.com. The OPA project, right, is—

    Sandall: Yep.

    Shimel: —OpenPolicyAgent.org?

    Sandall: That’s correct, yep.

    Shimel: And, of course, you can find more information on it at the CNCF.io site, right?—

    Sandall: Absolutely, yeah, and it’s on GitHub and we also have a blog if you go to blog.OpenPolicyAgent.org, you can find out more there. So, yeah, check it out, and—yeah.

    Shimel: [Cross talk] It’s all good. Hey, Torin Sandall, co-founder of the OPA, Open Policy Agent project, which is now being incubated by Cloud Native Computing Foundation, as well as an employee at Styra—for my friend Bill Mann.

    Sandall: [Laughter]

    Shimel: So, good to have you on there. Say hello to Bill for me. Best of luck with OPA.

    Sandall: Will do.

    Shimel: And look forward to seeing you in May at KubeCon.

    Sandall: Looking forward to it. Talk to you later. Bye.

    Shimel: Alright Torin, thanks. Hey, this is Alan Shimel for DevOps.com, Container Journal and Security Boulevard. You’ve just listened to another DevOps Chat.

    — Alan Shimel

  • 7 Big Digital Transformation Trends

    7 Big Digital Transformation Trends

    As enterprises continue their rush to digitally transform their organizations, they’re investing big in retooling their software development pipelines, new artificial intelligence (AI) and robotic process automation (RPA) technologies and low-code development platforms, as well as putting considerable money into customer engagement software. Yet, the extent of success following these investments remains unclear and varies significantly from industry to industry.

    To shine a light on the digital transformation efforts currently underway, as well as the degree of their success, we’ve collected a number of fascinating trends in the most essential areas of digital transformation.

    Investment in AI, Machine Learning and RPA Technologies Hitting an Inflection Point

    According to the consultancy KMPG, enterprise investments in AI, machine learning, and RPA will rise to $232 billion by 2025 from $12.4 billion in 2018. “Nearly two-thirds of respondents indicate plans to fully implement RPA within three years. Another 18% plan selective use. As for cognitive automation, nearly half noted intentions to use these approaches at scale within 3 years, while 29% indicated selective reliance on cognitive automation capabilities. Some 10% said they would launch pilots and proof of concept projects,” according to the report.

    DevOps is Just Getting Started

    While we’ve been writing about the enterprise move to DevOps for years now, the investment in DevOps is just setting itself up to truly soar. According to the market research firm Market and Markets, the DevOps market will grow 24.7% annually, to $10.3 billion by 2023 from $2.9 billion in 2017. “The demand for DevOps solutions and services among enterprises is expected to gain huge traction, due to the increasing need for fast application delivery with high quality,” the firm said.

    Digital Transformation Spending to Top $1 Trillion This Year

    In research firm IDC’s “Worldwide Semiannual Digital Transformation Spending Guide,” global digital transformation spending will reach $1.18 trillion this year, rising nearing 18% from 2018. “Worldwide DX technology investments are expected to total more than $6 trillion over the next four years,” said Eileen Smith, program vice president with IDC’s Customer Insights and Analysis group.

    For 2019, the two industries expected to invest the most into digital transformation are discrete manufacturing and process manufacturing, which will invest roughly $222 billion and $124 billion, respectively. According to IDC, the top digital transformation spending will be in smart manufacturing, which will be supported by autonomic operations, manufacturing operations and quality. The retail, transportation and professional services industries will follow closely behind.

    IT and Business Leadership Chasm Widen

    According to research firm consultancy Capgemini, there appears to be a growing disconnect between the CIO and senior business executives. Capgemini’s research found that while in 2012, 65% of organizations believed that the CIO and senior business executives possessed a shared understanding of the role of IT in their organization. In their most recent survey, that figure dropped to 37%. Additionally, Capgemini found, 59% of enterprise respondents in 2012 believed that the CIO and senior business executives had a shared understanding of how IT could be used to increase organizational productivity. That figure also declined precipitously, hitting 35% in 2018.

    Low-Code Development Continues to Gain Higher Ground

    The demand for enterprise software seems to have no bottom. And no matter how hard they try, developers can’t code software quickly enough. This is why more enterprise software needs are being met by low-code development platforms.

    According to research from ResearchandMarkets.com, the global low-code development platform market will grow to about $27 billion by 2022, up from $4 billion in 2017 and at an annual compound growth rate of 44%.

    Digital Transformation, Customer Engagement Software Grow

    Digital transformations don’t mean much if they don’t reap increased revenue, land new customers and improve relationships with existing customers. And how enterprises engage with their customers is one of the most rapidly evolving aspects of enterprise technology today, from AI and machine learning to IoT devices and ubiquitous adoption of mobile devices. Customer engagement software ranges from staid CRM systems to customer self-service software. This class of software was a nearly $13 billion market in 2017, but will likely reach $22 billion by 2023, an annual growth rate just shy of 10%, according to market research firm Mordor Intelligence.

    Digital Transformation Success Grows Harder to Come By

    The success rate for digital transformation efforts is persistently, and dismally, low. According to research from McKinsey and Company, over time fewer than 30% of digital transformations succeed.

    And that failure rate appears to be growing worse, not better. McKinsey’s 2018 survey results show only 16% of enterprises believe their digital transformations will improve organizational performance over the long term. “Even digitally savvy industries, such as high tech, media and telecom, are struggling. Among these industries, the success rate does not exceed 26%. But in more traditional industries, such as oil and gas, automotive, infrastructure and pharmaceuticals, digital transformations are even more challenging: success rates fall between 4% and 11%,” McKinsey found.

    While enterprises are clearly investing into their digital transformation efforts and the desire to succeed is high, the high digital transformation failure rate says enterprises either aren’t taking the right approach to these efforts or they are not executing properly.

    Whatever the reasons for the high rate of failure, with the vast amount of business capital on the line and the increasing risks upstart businesses pose to those established in their markets, more enterprises better figure out soon how to get their transformation efforts right.

    — George V. Hulme