devopsy.kinsta.cloud

Category: Features

DOES18 Las Vegas: Video Interviews, Day 1

Despite what you may have heard to the contrary, not everything that happens in Vegas should stay there. Take the DevOps Enterprise Summit 2018 (DOES18), for example: The three-day event attracted the best and brightest in the DevOps world—and me—to discuss all things DevOps. Some of those great conversations were captured in a series of video interviews, which are now available on DevOps TV.

During the two days of interviews in our glass-walled video booth, a variety of folks stopped by to chat about topics ranging from audits and compliance to NextGen Ops and the future of DevOps. Below is a recap of the Day 1 video interviews from DOES18 (Day 2’s recap will post next week):

The Future of DevOps

Wesley Pullen of Electric Cloud, Torsten Volk of EMA and Chris Nowak of FlowStates stopped by to discuss the latest DevOps trends, where DevOps is headed, and understand key emerging practices and technologies that are driving success.

Venture Capital and DevOps

Sid Sijbrandij of GitLab, CJ Napolitano of Electric Cloud and Ubaid Dhiyan of Union Square Advisors chatted about the current state of VC investment into DevOps, how this could evolve in the future and the impact this investment ecosystem has on the global economy.

ITSM/DevOps Crossover

IT Skeptic Rob England and his partner Dr. Cherry Vu discussed what holds ITIL back from more successful transformation of IT work, and how we can help it succeed, as DevOps and ITSM finally converge on each other in the common ground of service operations.

‘Dear Auditor’ — Compliance and DevOps

Sam Guckenheimer of Microsoft, James Wickett of Signal Sciences and Topo Pal of Capital One talked about the current state of DevOps and audit/compliance. They are part of a team that wrote a love letter to auditors from DevOps, in which a promise was made to make life better. They also compiled a list of audit concerns and documented them in a DevOps Risk Control Matrix with lot of details around the controls, practices and evidences that are collected to support the control.

NextGen Ops

Damon Edwards of Rundeck, Cornelia Davis of Pivotal Software and Tom Limoncelli of Stack Overflow chatted about the next generation of IT Ops and what emerging patterns and practices teams should consider for the future of IT.

Workplace Burnout

John Willis (Botchagalupe) is passionate about so many topics around DevOps, cloud and technology. One of the areas he is really passionate about, though, is the topic of burnout and fatigue in the IT field. He and Dr. Christina Maslach discussed the concept of burnout and examined the causes and effects of this problem, and the most promising strategies for dealing effectively with it.

Project to Product

Dr. Mik Kersten, CEO of Tasktop, reviewed the transition from project-based work to product-based workflows—the most optimized and scalable DevOps environment will be a highly prized gem for any business only if the product management team is prioritizing the right capabilities into the DevOps process.

Tech for Tech Leaders

Anders Wallgren of Electric Cloud and Erica Morrison of CSG discussed the technology landscape and what things tech leaders need to not only understand from a tech and architecture perspective, but also what they need to know to help build technology and tech platforms for the future of software delivery.

Banking on DevOps

Mark Howell of Lloyds Banking Group, Shaun Norris of Standard Chartered Bank and John Rzeszotarski of KeyBank reviewed a few of the top DevOps journeys in banking and what other financial institutions should consider along their DevOps journeys.

Well, there ya have it: An entire day’s worth of videos compiled for your viewing pleasure. Be sure to check back next week for a recap of our Day 2 video interviews from DOES18 Las Vegas.

— Charlene O’Hanlon

November 15, 2018
Too Many Projects: Breaking Initiative Overload
Initiative overload. Those words send a chill down the spine of IT leaders everywhere. Not only are overloaded teams juggling too many projects, morale is usually low, with long hours every week and no end in sight. Yet, leaders can’t arbitrarily kill or stall projects without suffering blowback from somewhere. If you’ve been tapped to lead a DevOps team and you see too many projects in play, breaking initiative overload is a top priority. Here’s where to start.

It’s very common that highly productive DevOps teams suddenly end up with a list of 20 or 30 “priorities” and rapidly developing gridlock. As the team’s newly anointed coach, you get one shot to fix it. What’s your strategy? Addressing these objectives is critical in breaking initiative overload:
- Prioritize the right initiatives.
- Shorten the priority list, ideally to five or fewer projects.
- Allocate resources so priority projects start and finish promptly.
- Communicate and defend prioritization with a transparent metric.
- Identify and plan the “next project up” as priority projects finish.
- Empower the team with “not a priority right now” if approached.
- Get buy-in and hold all team members accountable.
Gather the List

Sounds complicated, right? Keeping it simple is usually good advice. When you’ve got a long list of projects, first recognize they are not all created equal. Some are truly initiatives, such as that mandate from the CEO for tracking widget sales in real-time. Functional departments have their important projects, providing needed cost reduction or efficiency improvement. You’ll probably see a few wish list projects from various sponsors with weaker justification.

Next, understand that every project carries a couple of hidden costs. One is the opportunity cost of assigning precious resources to it and not to something else. Another is the switching cost teams incur when they are multitasking too heavily. Neither of these costs can be eliminated; however, they can be optimized within the context of what resources are needed.

So, make a list of every development project on your radar. List the untouchable initiatives with C-level sponsors. Add the functional projects and those pet project requests. Don’t filter the list at all—anything consuming resources or awaiting start needs visibility. Now, you should see what all the initiative overload chatter in the hallways is about. Take a deep breath, call your team leads together and order your preferred brainstorming chow for a meeting.

Create the Plot

Seated around you should be a mix of your best technologists and line-of-business experts. Look at the list together. Sponsors may have justified a project request without a real assessment of merit. If you need someone to speak to a project that team leads haven’t seen, bring in a departmental expert. If they can’t explain it or it’s an outdated request they want to revise, put it in a parking lot. When you’re sure your team leads all understand every project on the list, assign a letter designation for each project.

On your whiteboard or shared screen, plot each project against two dimensions: implementation ease and business impact. Use a relative scale of low, medium or high for each axis. Implementation ease can incorporate many variables including cost of tools, technical difficulty, in-house expertise and more. Business impact can address revenue increase, cost reduction, competitive advantage or regulatory compliance. Don’t worry about absolutes; you’re looking to see how a project compares to all other opportunities. Adjust the pins if necessary after the first few projects are plotted. You’ll get the hang of it in just an hour or two, even if it’s a long list.

Line Up Priorities

Few projects are priority no-brainers with high business impact and high implementation ease. More likely, you have several projects in the “priority box,” scoring better than medium/medium. If you have more than five projects there, take another look and jostle pins a bit to get a manageable list. Make sure your available resources match projects identified in the priority box. Allow one exception: That C-level initiative may score as high business impact and low/medium implementation ease. Killing it just because it’s difficult isn’t a good move, but prioritization within context can help address resourcing.

That’s the priority list you take back to your team. If it’s on that list, work it and finish it. If something isn’t on that list, your team kicks any inquiries on those projects back to you. Push projects with low business impact and low/medium implementation ease back to their sponsors for justification or removal. When one of the priority projects finishes, then and only then start the next best ranking project. Rescore the entire project list after the first group of projects is complete.

Too easy? Here’s a recent HBR article, “Too Many Projects,” describing a more elaborate approach for bigger initiatives. You may need such in-depth analysis for a C-level initiative requiring more resources. For most projects on your list, more analysis takes substantially more time and likely produces the exact same prioritization as this faster method.

The key to breaking initiative overload is starting and finishing the right projects. Creating a vetted priority list quickly and getting the team rolling again is well worth the price of an afternoon and some food. Morale will improve rapidly as your team sees progress and the business sees value faster.

I’d enjoy your feedback on how this process works for you.

— Don Dingee
November 15, 2018
Using Netflix’s HubCommander to Automate GitHub Organizations
Netflix owes a great deal of its exponential growth to its phenomenal tech stack. Throttling its content through a single internal API, the company was able to deliver content agnostic of device type and quickly disrupt its contemporary competitors in the consumer entertainment industry.

We’re intrigued, then, when Netflix exposes some of its internal architecture for others to use. HubCommander is just one recent addition to Netflix’s growing collection of open source tools. It’s a ChatOps tool for GitHub management.

We’ve previously covered an open source approach to ChatOps, tracking how to bring free operational tools into the conversation for increased transparency and efficiency. In this article, we’ll resume this research by learning how Netflix has utilized HubCommander and delve into its open source iteration to see how others might leverage it to automate its GitHub organization.

Problems

Managing many GitHub repositories while working in a large organization can be frustrating; especially difficult is the task of managing permission levels between various user subsets. Not only are various user permission levels difficult to trace, but there is an intrinsic security problem to address as well, especially when working with external teams.

“One of the biggest challenges with using GitHub organizations is user management.” — Netflix

Project managers need to administer different organizations, and GitHub requires admin privileges to manage such repository settings; if onboarding thousands of employees, editing this manually could be an extremely tedious job.

Benefits!

At Netflix, HubCommander standardizes its user management approach, enabling automation for GitHub repo permissions across development teams.

“Management of many users on GitHub can be a challenge without tooling. We needed to provide enhanced security capabilities while maintaining developer agility. As such, we created HubCommander to provide these capabilities in a method optimized for Netflix.”

HubCommander thereby provides this missing link, granting admins the ability to quickly grant privileges. Easily to embeddable into chat, the HubCommander is made with ChatOps in mind. Therefore, privileged GitHub organization management tasks can be performed from the Slack channel, “without granting administrative or owner privileges to your GitHub organization members.”

Using HubCommander

To use HubCommander, first, you must have Python 3.5+, Slack, a GitHub organization, and a GitHub bot user with ownership level privileges.

Next, Slack channels with HubCommander installed can make the call !help to return a list of bot commands:

Source: Netflix

To create a repo, for example, you can simply use the command !CreateRepo:

Source: Netflix

Utilizing HubCommander also has security benefits as it reduces the number of GitHub API permissions granted per user—reducing attack vectors inherently increases security. Another point on security is HubCommander’s integration with Duo for additional authentication:

Source: Netflix

The total list of HubCommander features include:
- Repository creation
- Repository deletion
- Repository description and website modification
- Granting outside collaborators specific permissions to repositories
- Repository default branch modification
- Repository PR listing
- Repository deploy Key listing/creation/deletion
- Repository topics creation/deletion
- Repository branch protection enabling/disabling
- Enable Travis CI on a GitHub repo
- Safeguard commands with 2FA via Duo
Automating GitHub With ChatOps

If your code resides in multiple Git repositories, like Netflix, having a method to configure permissions across all entities in a convenient way becomes a necessity. With a bot to perform operational tasks on GitHub repositories, Netflix is benefiting from automatic manipulation and a wider team access, adding to its agile goals:

“The reduction in administrative overhead has significantly simplified our open source efforts.”

Netflix operates with three development channels (a core Netflix OSS, Spinnmaker and a skunkworks project for new ideas). Thus, having a consistent permissions model across organizations can decrease friction. Also, since Netflix collaborates with third parties, such as contractors or other open source contributors, HubCommander also acts as an effective way to maintain security.

Final Thoughts

Netflix recognizes a growing popularity in the ChatOps approach and cites other benefits such as team transparency, a timestamp for each event and a self-service nature—all reasons for adopting a ChatOps strategy into its developer tooling.

HubCommander appears like a good ChatOps tool for GitHub repository management. However, GitHub Actions (still in beta at the time of writing) is a new feature that will aid development workflows, offering automation for the GitHub operational duties themselves. This may spark alternative bots to be developed, especially in terms of workflows that respond to events.

For now, with open source utilities such as HubCommander, certain GitHub duties can be quickened, stimulating a culture of more efficient management to decrease redundant tasks. While shorthand for GitHub repository management might not seem at first glance as a priority, it could scale down operational headaches tremendously.

— Bill Doerrfeld
November 14, 2018
Boxes vs. Services: The Infrastructure Equation

Infrastructure is changing as more services replace the need for hardware-based architectures

There has been a lot of change in the last few years, and that change shows no sign of slowing down. I’m one of those people who likes to know they could work offline if needed, yet in this day and age if you want me to install a fat client on my laptop, I go looking at your competition.

Granted, for me the selection of these apps is small—I like having Office on my desktop to be able to use it to write or create presentations at will, regardless of whether I’m connected—but for most people, it is growing at a faster pace. The idea of buying a monolithic system that you have to install and maintain is becoming less and less appealing.

That same syndrome is occurring in the data center, too, but with a twist. It is largely infrastructure that is feeling this pinch. Just prior to cloud/container mania, the idea was that each non-compute box in your infrastructure was a platform. It offered you a ton of functionality, all designed to simultaneously lock you in and help you get your job done. All in all, it wasn’t a bad trade-off, and we can see the same idea in the large public cloud vendors.

The difference is, with an infrastructure box, you were making a serious investment upfront that offered you other functionality to sweeten the deal. Once that investment was made, it was natural and realistic to want to take advantage of the bundled platform functionality rather than search for yet another box to drop in your network.

Containers and what survived of software-defined networking (SDN) have changed all of that. The ability to quickly spin up something in software that fits your needs perfectly and can be automated with your DevOps toolchain makes the idea of a platform box—with its high cost of entry and bells and whistles you will never use but might have to lock down—seem quaint.

In the agile/DevOps world, boxes for low-level infrastructure are still necessary. Don’t get me wrong on that. If you are a large org, you are using one of a few vendors to do core switching and routing. But top-level (layer 4-7 and 8) networking is not something you would use those static boxes for.

It used to be that all networking functionality was the same—it needed to be rock-solid and slow to change to guarantee that apps were functional. Over the last few years we have seen a move from “all networking is networking” to “this is low-level stuff that must be rock-solid, that is closer to application layer stuff that we can change rapidly, and thus need not be in the ‘rock solid’ category.” Load balancing is a great example of this (and yeah, I used to work for F5, which is the king of load-balancing): Load balancing was a network function that you went to a hardware vendor for if you were beyond a certain size. The resulting physical box might serve one or 100 apps and required specialized training or experience to manage.

Today, load balancers are a systems function that you go to a software vendor for and work into your network with as many copies of the software load balancing as you needed. Or you turn on load balancing within your container management system and manage it there. Either way, it fits in with your agile/DevOps environment better, and is easier to get budget for your project.

I expect to see this change intensifying. There is power in physical boxes for some things, but there is also power in portability and packaged deployment scripts. And a physical box is not portable to a public cloud, nor is it readily dropped into container management tools (though container management integration is getting better in most network security/performance appliances). Portability and ease of use will win over technical edge, in my opinion. Not because people are cheap and lazy, but because portability is key to agility and they are too busy.

There are still things that just aren’t that portable, and I have yet to see convincing tools to make them that portable. Data is my go-to example because it is easy. If you have 2TB of data, your portability options are limited. For those with 2PB of data, there is no real option; the app is going where ever the data is, and that data is unlikely to move. Oh, if you were motivated, there are ways to move that much data, but then you’ve just shifted the problem to the new platform, because the data is still largely non-portable, and apps will have to go where the data is. Again.

But for most functionality? Anything that can be made into highly portable software will be. And that software will run wherever it is most needed or most beneficial to the user.

All in all, this is a good result. Multi-core, multi-network, multi-terabytes of memory PCs can rival many appliances for speed and functionality, and portability plus ease of integration makes software a winner. Just stay aware of where/when a physical box still makes sense, because the world doesn’t change as fast as high-tech marketing wants you to believe. A working appliance that does what is needed quickly should not be ripped out just because a software solution is available.

But you knew that. Because you all rock. Keep kicking it, and use whatever solution serves the business best—which is a different equation for every business.

— Don Macvittie

November 13, 2018
CA Technologies Report Highlights App Modernization Paradox

The rush to modernize IT environments using microservices, application programming interfaces (APIs) and containers may very well turn out to represent something of a double-edged sword.

A survey of 1,087 IT and business professionals conducted by Frost & Sullivan on behalf of CA Technologies, a unit of Brocade, finds that organizations that have modernized their application environment are eight times more likely to achieve key performance indicators (KPIs) for the business. Four out of five respondents also said agile practices are helping them build what customers really need to improve overall customer satisfaction.

On the downside, however, the study also finds more than half of respondents (56 percent) claimed that software issues can negatively affect the achievement of KPIs. A full 69 percent of all respondents say the pressure to release new applications or updates negatively affects quality and security, and 67 percent admit that their organization sometimes trades off security for application development and deployment speed.

Nine out of 10 respondents who claimed to widely utilize modern application architectures also concede it’s challenging to manage and monitor APIs, microservices and containers, and that APIs and microservices add new risks for their organizations. A total of 80 percent of respondents also say they view continuous application security testing and DevSecOps as critical components of their strategy to help resolve those issues.

Ashok Reddy, general manager for DevOps at CA Technologies, said that while organizations are clearly seeing a business benefit after embracing DevOps to modernize their application environments, the process hasn’t always been easy. The challenge they now face is that the rate at which digital business transformation projects are being launched has substantially increased. Most organizations are not going to achieve those goals in a timely fashion unless they modernize their application development and deployment processes, said Reddy, noting that only then can IT become flexible enough to meet the needs of the business versus continuing to try and force the business to bend to the way their applications work.

The paradox many organizations find themselves wresting with is that making the business more agile requires a more complex approach to IT. In theory, each microservice deployed is easier to manage than a monolithic application. But as thousands of microservices get deployed, the number of dependencies between services becomes more than the average IT organization can track. In fact, all those dependencies is one reason more organizations are starting to investigate applying machine and deep learning algorithms within the context of an artificial intelligence (AI) model to automate IT management at scale. Clearly, most IT organizations are now on a journey that starts with a simple set of microservices and ends up fundamentally transforming enterprise IT, regardless of whether anyone likes it.

It may take a while longer before the future of IT becomes a completely settled question. But for now, it’s apparent microservices in some regards are too much of good thing. The challenge is that from a digital business perspective, those microservices have become something most organizations can’t do without.

— Mike Vizard

November 12, 2018
2018 DevOps Dozen Finalists Announced
We are proud to announce the 2018 DevOps Dozen finalists. When we first launched four years ago, I hoped the DevOps Dozen would grow into an award that really did recognize the best of the best in the maturing DevOps space.

This year, there is no doubt in my mind that the finalists really are unsurpassed in DevOps. In fact, the whole process of nominations was open to the public for selections of finalists; the judges and the criteria really do represent our best efforts.

Congratulations to each and every one of the finalists in the 12 categories. It is quite an accomplishment to make it this far into the process. Whether you’re chosen as a winner or not, you should be proud to be part of such an impressive list.

Since the nomination process was opened, our team has gone through thousands of votes to pick out the finalists in each category. It was not an easy task, as we couldn’t automate (DevOps) it—we had to do it by hand. Many thanks to the everyone here at DevOps.com for doing this.

The next step in the process is for the finalists in each category to submit background info, which will be passed onto our panel of judges. The, the judges will pick the winners of this year’s DevOps Dozen. The 12 (it is the DevOps DOZEN, after all) winners will be announced right after the new year.

Here are your finalists:

Category 1 – Best DevOps Open Source Project
1. GitLab
2. Jenkins X
3. Jenkins
4. Artifactory
5. Taurus
6. Zowe
Category 2 – Best DevOps Commercial Solution
Category 3 – Best DevOps Solution Provider Exec
Category 4 – Best DevOps Solution Provider
1. Accenture
2. CloudBees
3. Compuware
4. Electric Cloud
5. IBM
6. JFrog
7. Perfecto
8. XebiaLabs
Category 5 – Most Innovative DevOps Solution of the Year
Category 6 – Top DevOps Evangelist
Category 7 – Best DevOps Transformation
1. Autodesk
2. CafePress
3. Capital One
4. Dutch Railways
5. Standard Bank
Category 8 – DevOps Transformation Leader
Category 9 – Best New DevOps Solution Company
Category 10 – Best DevOps Book/ebook
Category 11 – Best DevOps Presentation
Category 12 – Best DevOps.com Article
Thanks to everyone who voted and nominated candidates. Good luck to all of the finalists!

Stay tuned and, as always, thanks for supporting DevOps.com and the rest of the MediaOps family of communities. If you have questions, please write us DD@devops.com. Also if you want to be included in next years DevOps Dozen awards you need to subscribe to DevOps.com and look for announcements next September.

— Alan Shimel
November 9, 2018
DevOps Chat: Instrumentation with Influx Data’s Evan Kaplan

Evan Kaplan, CEO of Influx Data, is someone who knows his way around the technology world. In a lifetime of experience that has seen him found and manage several successful companies from startups to public entities, Evan has had a great track record of success.

Influx Data is yet another success for Kaplan. Building on the open source InfluxDB, the company boasts more than 200,000 users of Influx today.

In this DevOps Chat, Evan and I discuss systems instrumentation today and what big data and velocity means for developers and everyone up the stack. It is a very different world from just 15 or 20 years ago.

As usual, the streaming audio is immediately below, followed by the transcript of our conversation.

Transcript

Alan Shimel: Hey, everyone, it’s Alan Shimel, DevOps.com, and you’re listening to another DevOps Chat. Happy to be joined in this episode of DevOps Chat by, actually, someone who grew up in my neighborhood. We knew each other – well, we didn’t know each other way back when, but we’ve come from a very small area of concentrated – lot of people, you know, we know from that area, but, anyway, Evan Kaplan, CEO of InfluxData and of North Woodmere. Evan, welcome.

Evan Kaplan: [Laughs] Thanks, Alan. Thanks for having me. Look forward to _____ –

Shimel: My pleasure. So, Evan, it’s been a long, strange trip from North Woodmere. Why don’t you give our audience a little bit of your background and your journey? And then we’ll jump into –

Kaplan: Yeah. Sure. Sure. Very long for me. If those of you don’t know, that’s a suburb of New York City, right near Kennedy Airport, that we both grew up. My strange trip took me to Seattle, where I finished up college and worked as a climbing guide for years. And then worked in aerospace and started a company, the first company to develop SSL-based VPNs, called “Aventail,” back in the mid-90s. _____ through the bubble. Ran a public company called “iPass,” and, about two-and-a-half years ago, met Paul Dix, the founder of Influx. And Paul and I really connected about his vision and what he was doing, and we partnered together to start building Influx from 25 employees back then to north of 100 now. So growing fast.

Shimel: Good stuff. Good stuff. And, Evan, I’m gonna assume most of our audience has at least heard of InfluxData, but, for those who have or even those who haven’t, just to set the level-set, tell us about what InfluxData does.

Kaplan: Yeah, so InfluxData is an open-source time-series-based data platform. It’s the leading time-series platform out there. Time-series is the fastest growing market in the database, data category. And so Paul, our founder, spun up Influx back in 20 – I think it was late 2013. It received a lot of accolades on GitHub. It was a very popular project and it has grown dramatically since then, from roughly – I don’t know – 1,000 sites running it to, today, we’re approaching 200,000 sites running it.

Shimel: Wow.

Kaplan: And, you know, most of the large Global 2000 is running it somewhere, so.

Shimel: That’s really impressive, man. That’s something to be proud of. So, Evan, we could probably spend our whole 20 minutes talking generally about Influx DB and what’s happening, but I’m wanted to kinda dive in a little bit and talk specifically today around instrumentation in modern software installs, in the modern software factory. This whole term “modern” has become sort of a catchall buzzword, dog whistle, if you will, for when we talk about things like Agile and DevOps and serverless and microservices and all of these ways that we’re seeing software being written, managed, and deployed today. So talk to us a little bit about it, if you will, Evan. Where’s your take on this?

Kaplan: Yeah, so I think we’re – we grew up in the industry, going back a couple of decades ago, the common myth for monitoring or instrumentation of systems was you would buy a package from HP OpenView or CA or a mainframe management package, and you’d buy a monitoring-for-a-service package. They were expensive and they were largely horizontal across the stack, and you’d implement those. And what’s happened over – certainly over the last ten years, during the DevOps revolution, is that stuff has fractured and you’ve seen companies, like Splunk and Elastic and a variety of other folks, arise to handle the modern workloads in an open-source sort of fashion. And now, if you take a look a look at DevOps world, most of solutions are dominated by open-source kinds of models.

If you look at state of the art now, that people are really paying attention to, the revolution that brought you to logs and log views of stuff and the consolidation of logs has moved much closer to metrics and events. And what I mean by “metrics and events,” the instrumentation of real time of these complex systems. And they’re not just software systems that are DevOps things, so – those are obvious, but they’re also hardware systems; they’re IoT things. And so what’s happening is people wanna see these complex systems and they wanna interact with them in real time because the only way to characterize these systems is to watch them behave in real time. And “observability,” as you know, Alan, has become sort of a keyword here. And observability generally means real time and it means metrics and events; it doesn’t mean verbose logs.

And so what we’re trying to do is help developers, specifically and, up the chain, architects and CTOs, build instrumentation or telemetry natively into these complex systems, so they can view these things in real time. As you can guess, the data loads with this are absurdly high. You’re talking about hundreds of millions of points a second in some of these systems and you’re talking about to visualize those in near real time, you know, control loops in near real time. And you’re talking about this sort of next generation of, lack of a better way to say it, system instrumentation and management.

Shimel: Sure. Evan, if I could stop you there for a sec, you know what I find interesting? We’re talking about building this instrumentation and to give people insight and who are we giving insight? Well, you mentioned developers and managers and architects and CTOs, et cetera, but you didn’t mention ops. And is that –

Kaplan: Yeah. Yeah.

Shimel: Isn’t that a little bit of, you know – I don’t know if it’s Freudian or if it’s just insight into what today’s world looks like for these things.

Kaplan: You know, that discrete ops, right? That discrete ops role is changing, right, into what you obviously cover, which is this DevOps. And so our view of how our – uptake of our platform into large corporations, companies like IBM and Cisco and SAP and Tesla and PayPal, has been from developer or DevOps people up. So we tend to think of “Make it super easy for developers, right? Get their time to value down to really, really minimal and then it will rise up and be taken as a complete operational system.” So we tend to think of it from developers up. Not to say the operations guys aren’t relevant – they’re super relevant – it’s just, when we enter into a large opportunity or business or that sort of stuff, it’s usually coming from a developer who figures out he can instrument his code, his service, his offering relatively quickly in Influx.

Shimel: Got it. And I’ll tell you something I – and it goes back to what you mentioned earlier – the sheer volume and the acceleration of data, right, the – so, no, it’s how much data and the speed that it comes at us. With automation and everything that that brings, it easily outstrips the ability of a human, a lot of times, to stay on top of it, so you almost need to build automation, machine learning, AI into these types of instrumentations, that you bring what the human really needs to see and touch or make a decision on to his attention ’cause so much of it is just whizzing by right now.

In the last week or two, Evan, I’ve probably had a couple of discussions – trying to think of a friend of mine who – CEO at ScienceLogic and another company called “Ravel Systems”; some friends of mine in security just launched something called “DisruptOps.” And they’re all talking about this term, “AIOps.”

Kaplan: Yeah. Yeah.

Shimel: And I’m wondering is this something relevant to what you’re talking about, with instrumentation? Where does that fit in? How do we account for just the big data nature of this?

Kaplan: Yeah, I think, first of all, absolutely. So, if you think about – so the journey. And let’s call it the end state of all of this is true autonomy, which is very rare and, in fact, almost never the case, right? But, if end goal state is true autonomy, what’s that journey look like? It looks like, first and always and most importantly – and, by the way, the stuff that we skipped a lot, when we were in the early stages of our career – it is, in fact, instrumentation, is telemetry. Telemetry always was an afterthought before whether shipping a product or a service or that sort of stuff. Now the ability to drive that cost of instrumentation down so low that the ability to see what’s going on and create the data – not just see as in “human” see but create the data – has changed dramatically.

So, if you follow that journey, you go from instrumentation; you go to observability; you go to automation in the form of “I’ve observed these sorts of things, whether it’s through AI capability or not. I built control loops because I’ve observed them, so I’ve built control loops to keep it in a relative stasis. Right? To autonomy.” All of that – the last two steps, right, even the last three steps – involves some form of machine processing of learning, whether it’s machine learning or AI or call it, but all of that gets increasingly sophisticated. But it means taking in, as you sort of articulated, you know, millions, hundreds of millions – in some case, billions – of points per second, right?

Shimel: Yep.

Kaplan: Think about it. Now that’s just completely – you know, what a different world, than doing server management with your CA Unicenter product 20 years ago.

Shimel: [Laughs] To say the least.

Kaplan: Right?

Shimel: Yeah, I hear ya. I mean, it really is. If we transported – you know, if we could do that – if we could transport someone from not that long ago, Evan, an hour or _____ _____ –

Kaplan: No. No.

Shimel: From 1997, ’98, right? I was helping an early ASP, Interliant. We were building – opening data centers, managing Lotus Notes and PeopleSoft and stuff like that.

Kaplan: Right.

Shimel: If I could take a NOC guy from there and show him today’s velocity and the fact that we actually do have telemetry and we actually are building tools that allow us to manage at that level, it would be like taking someone from colonial times and showing them highways with cars. I mean, the supports –

Kaplan: But, Alan, these solutions are still sitting out there. They’re a huge _____ _____ _____ –

Shimel: No – well, that’s the scary thing. Yeah, I mentioned my friend, ScienceLogic, my friend Dave Link. He was saying that’s his – you know, and they’ve done very, very well, and his thing was just replacing the Tivolis and the Unicenters and that generation, if you will, of network management, of systems management.

Kaplan: If I could tell you what, back in the late ’90s, is I was building an ASP also, based on the Aventail, the SSL VPNs, what we were standing on Netcool, back in the day, and things like that, _____ _____ –

Shimel: And Netcool was a new program, right?

Kaplan: Yeah.

Shimel: Before IBM bought it, right? It was one of the cool, new kids at the time.

Kaplan: Exactly. If I could tell you what somebody could take with Influx today and do, relative to what Netcool did back in the day, and the difference, right – first of all, they could do it for nothing. Forget paid licenses. And the amount of scale, the amount of data collection, and the ease of sort of visualization has dramatically – and all those _____ _____ –

Shimel: It’s phenomenal. Phenomenal, man. It’s just crazy. So, Evan, let’s bring this back to Influx. How does all of this now manifest itself with your business today?

Kaplan: Yeah. So that’s right. So we look at our businesses in two segments, right? We look at it as the instrumentation of and the collection of data around software, being the platform of record for this next generation of cloud software. And I don’t have to describe to you is is that, if, in the old days, it was an X Windows terminal talking to a Unix server app, today, there is no “there” there. There’s a virtual network; there’s VMs, containers, and microservices that are growing at exponential rate. The whole infrastructure is ephemeral, so the monitoring and the instrumentation workload has dramatically changed. So one side of our business is really focused on that, that opportunity, and we’d call that the “DevOps chain” of the business, right? Because it comes up into organizations through DevOps folks who bring in Influx and instrument.

The other side is IoT, which is completely different. So, while DevOps is largely metrics-driven, IoT is largely event _____ driven. And that’s the instrumentation of the physical world, right? So our watches, our clothing, our health care, our cars, our homes, are all being instrumented with sensors. The amount of data pouring in through that is driving whole other systems into business. And, while most of our time was spent on the management of software systems back, now it’s this deep, deep integration of software and hardware. Deep. And so our business breaks out into those two categories, largely.

Shimel: Got it. Got it. Evan, anything – any new news coming out of Influx you wanna share with our audience or –

Kaplan: You know, where – we have – you know, we’re almost constantly releasing stuff into the community on a regular basis, so no news that I’d share at this moment. We can touch base in three or four weeks. We have our big InfluxDays event. We’ll make some announcements there, but you can look for us to build increasingly sophisticated solutions that allow people to do their machine-learning workloads, their application tracing, their log and metrics, their asset management, on top of Influx. What we want is people to use us as the platform of record for all of these metrics and events pouring in.

Shimel: You know, we live in interesting times and I can’t wait to see where this all goes as well, so we’re about out of time, though, here. As I mentioned to you, the 15, 20 minutes goes so quickly.

Kaplan: Yeah, it really does.

Shimel: And you barely ever – you know, it is what it is. But, Evan, thanks for joining with us. Evan Kaplan, CEO, InfluxData. You know, so, in three, four weeks, you have your event coming up; maybe we’ll have you back. We can talk a little more about that. We’re doing a big – I didn’t even announce it yet, but I’m gonna announce it right here on the show with you – we’re gonna be doing a big virtual event, December 18th, called “Predict 2019,” and we’re gonna –

Kaplan: [Laughs]

Shimel: Yeah. We’re gonna be opening up the crystal ball and asking a lot of pundits and experts and so-called experts what their predictions are for what 2019 has in store. Maybe we can have you back then and we can do some crystal-ball gazing.

Kaplan: That’d be great, Alan. Always a pleasure. Always a pleasure.

Shimel: All right, my friend.

Kaplan: Have a great day. All right.

Shimel: Well, Evan Kaplan of North Woodmere and InfluxData, thanks for being our guest on this episode of DevOps Chat. This is Alan Shimel for DevOps.com, and you’ve just listened to another DevOps Chat. Have a great day.

Kaplan: Take care.

— Alan Shimel

November 9, 2018
DataOps: DevOps Plus Big Data

In traditional DevOps, there are the complimentary forms of development operations (that I call DEVops), and development operations (that I like to call devOPS). Between them they automate the toolchain and bring the people working on getting an application out to users onto the same team—not necessarily the same organizational team, but using their strengths in tandem to meet the needs of the business.

DataOps is an interesting extension of DevOps. You still need all of the coding side to get data into the system and queries consistently maintained. You still need all of the operations side to get your database/noSQL/whatever up and running. In fact, while the development side in a DataOps environment might be (but isn’t necessarily) lighter, the operations side is almost always more complex. No matter the big data engine in use, it is a complex system in addition to other supported systems in a normal environment. My first installation of a big data environment (Cloudera, as it happens) was a weeks-long learning voyage. Only after I’d completed it did I use an automation tool (which is no longer available) to make it easy. My second round was hours … But it assumed the knowledge I had gained in the first attempt.

The other bit of DevOps—ongoing monitoring and management—is also more complex in a big data environment, but we’ll come back to that in a moment.

Once the normal DevOps systems are in place, ETL/data import tools will need to be supported also. The volume that these tools crunch through in a day makes their inclusion in DevOps in a data-heavy environment critical. If the data uptake is slow or the data itself inaccurate, there is an impact on the organization. This step also requires inclusion of data scientists, people who traditionally have not been pulled into the DevOps model. But they are the ones that can gauge the accuracy of data, and normally are responsible for data acquisition anyway.

Which brings us to the monitoring and management stage of DataOps. Normally when we talk monitoring and management in DevOps, we are talking about tools that ultimately help with things like availability, responsiveness, auto-scaling and recovery. DataOps needs all of these things, like any other application does. And we’ve already mentioned that it has a complex environment embedded in your complex environment. But it also (some would argue more importantly) needs data monitoring. A knowledge of the state of the data is imperative. Whether from a piece of code gone wrong or data poisoning, monitoring the health of big data archives is essential to keep results from getting out of whack. This monitoring will again involve the people who know the data—data scientists by whatever title. This iterative process will need to run relatively constantly, checking on whatever makes the data valuable, normally using cross-field checks to verify that data hasn’t somehow been corrupted, or comparing trending averages to current average to monitor for sudden swings in results.

Of course, for a system (and in research there are many) that spins up, finds results and spins down, the monitoring part is less essential, but for anything that keeps accessing and adding to the data pool, it is the key to a healthy application or application portfolio based upon the dataset.

If all that sounds a bit intimidating, you’re not alone. The phrase “DataOps” has been around for a few years, and while data scientists use it, it hasn’t yet gained serious traction in the enterprise. Here’s how I would approach it as CIO: Given that DevOps is running on other projects, that is the low-hanging fruit. Given the rate of change for servers (as queries—particularly long-running ones—increase and decrease, systems needs change is one example), traditional DevOps is a greater benefit to big data than many other corporate systems. So roll out DevOps first without serious concern for DataOps beyond how DevOps impacts ETL—not ETL data quality, but ETL data availability/performance. Only when the system and the team are in place and the software/hardware/code part of the overall system is fully in a DevOps environment, start looking at ways to move data into it. Data monitoring will be the big one, but ETL under the initial round of DevOps implementation will have shown places it could be improved and ETL data integrity can be added, so improving those areas also. A measured, step-by-step approach that builds off of organizational experience is key here. Data monitoring and management will require different tools than normal DevOps monitoring and management, so having the data scientists look into available toolsets and how they mesh with existing tools early on will help with eventual implementation.

Make no mistake, it is a big job, moving large data driven environments to DevOps. But the benefits are increased awareness of data quality and data quality issues, along with more responsive systems and teams that DevOps offers every business application. The amount of data available to any give organization is currently on a massive increase as IoT, open datasets, partnerships and ML drive greater pools. It is worth getting the automation and cooperation of DataOps into the mix as early as possible.

— Don Macvittie

November 6, 2018
How DevOps Can Nurture the IoT Brownfield For Results

Your latest IoT project sailed through proof-of-concept but started running into problems during rollout. Fortunately, your DevOps superhumans got things running. Now, perhaps a year or more later, you’re seeing more and more issues crop up as the project grows. If this scenario sounds vaguely familiar, or you’re worried you may be headed there, you’re not alone. Welcome to the IoT brownfield.

“Nah, can’t be. We selected and evaluated all-new hardware and software during the PoC. This has been a greenfield project from the start.” Be careful of this trap. Many IoT projects indeed start off as greenfield efforts, free of the constraints of integrating legacy hardware and software. Your team developed requirements with collaboration between IT and line-of-business folks. You introduced these new tools, and got results on a small scale. According to Cisco’s 2017 IoT survey of 1,845 IT and line-of-business decision makers, 60 percent of IoT projects stall at PoC, so you’re ahead of the game.

Or so it would seem. Now comes the escalating commitment. Management assumes that if it worked on a small scale, it should just work when rolled out across the enterprise. Data and insight will just come pouring in, and everyone will have visibility to what’s happening everywhere in real-time. They write a big check for the rollout, and away you go.

First Signs of Trouble

At the first rollout project review, folks start splitting hairs over the definition of “complete success.” You rolled out X locations, Y of them are working, X-Y of them are in various states of triage. How can that be? You took the “same” configuration and scaled it. As the team investigates the non-working population, you’re finding little things that aren’t the same, calling for tweaks here and there.

Tossing out the trivial, over scale and time there is no such thing as an IoT greenfield project. Every IoT project with an extended life cycle becomes a brownfield, with a mix of new and old hardware and software. Rigorously controlling a configuration, à la defense or medical applications, is prohibitively expensive and unnecessary for most IoT applications. Your vendors won’t sell you the exact same hardware or software two years from now, five years from now, 10 years from now. Unless, that is, you pay them for things such as lifetime buys of electronic components or freezing open source code releases.

Ideally, we should send DevOps teams into IoT projects. With a big tool stack acquiring data, the line-of-business types figure out more things to do as the picture develops. The whole point is gaining insight and adjusting for optimized results. Cycles of learning and security enhancements are part of the game.

Coping with an IoT Brownfield

Here’s the paradox: the longer your IoT application survives, the more complex this situation gets. How can you improve your chance of success by applying IoT brownfield thinking? Technical requirements vary, but some core philosophical tenets can help.

Roll with the changes. Your team is already skilled at quickly rolling application software releases. Get more aggressive with ongoing changes at the firmware and network level. Incorporate over-the-air (OTA) technology for updating code on any wireless devices. Deploy multi-protocol gateways that can deal with revised protocol stacks (or entirely new ones) as IoT specifications continue evolving. Update security vulnerabilities relentlessly.

Abstract everything. You’ll read tons about IoT interoperability, and what people typically mean is wire-level interoperability of protocols. Those multi-protocol gateways have one job—getting real-time data onto an IP network and into the cloud. What keeps me up at night is data-level interoperability. Use edge or fog computing for pre-processing streams so the cloud isn’t stuck sorting out various formats of the same class of data. Abstract your data using JSON or other lightweight techniques so that if a next-generation sensor outputs data in a slightly different format, pre-processing can be revised close to the source. Consider a publish-subscribe protocol for easier routing between nodes.

Optimize the cloud. Scale can bite hard in the cloud tier. Heterogenous, distributed architectures usually win. Deploy a hybrid cloud solution where there is real-time data. Let the public cloud do what it’s good at: presentation, business intelligence and archiving. In a private cloud instance for heavier analytics lifting, use a framework such as Apache Spark to minimize data movement, with faster nodes sprinkled in for critical algorithms. Try dedicating a small processing core to each data stream for predictable deadlines, or using FPGAs in workflow optimized nodes for boosting compute performance.

After All, It’s Just Business

Ultimately, the evaluation of complete success of an IoT application will be up to the line-of-business decision-makers who wrote that big rollout check. A DevOps model is ideal for nurturing the IoT brownfield over scale and time, featuring ongoing strong collaboration between technologists and line-of-business types. Early in the project, reach out for some external IoT expertise until you have a few cycles of learning under your belt.

As that Cisco 2017 IoT survey alluded to, 61 percent of practitioners believe we’re barely scratching the surface of IoT transformation for businesses. Avoiding the greenfield trap and incorporating these IoT brownfield coping strategies sets you up for the best possible chance of success in the long run.

— Don Dingee

November 5, 2018
How IBM’s $34B Bid for Red Hat Will Spur DevOps Adoption

IBM’s proposed acquisition of Red Hat for $34 billion will have an immediate impact in raising awareness for a programmatic approach to managing hybrid clouds based on best DevOps practices.

A recent survey published by International Data Corp. (IDC) finds most organizations (64 percent) have already embraced multiple clouds to varying degrees. Only 7 percent of respondents said they have built a truly hybrid cloud. Another 24 percent, however, said they have achieved a high degree of interoperability across multiple clouds. IBM made it clear this week that its primary motivation for acquiring Red Hat was to tap into a trillion-dollar opportunity that will manifest itself as hybrid cloud computing continues to evolve. In fact, IBM is betting that Kubernetes clusters will play a key role in advancing the state of hybrid cloud computing.

But for hybrid cloud computing to take hold, it’s also clear the level of mastery over DevOps within enterprise IT organizations will need to improve substantially. In fact, hybrid cloud computing might not even be the ultimate destination, said CloudBees CEO Sacha Labourey. Enterprise IT is increasingly being defined by programmatic services rather than platforms, he noted, and those services need to be weaved together to drive modern applications. But where each of those services resides is becoming less relevant as organizations become more DevOps proficient, Labourey said, noting the rise of hybrid cloud computing will drive more organizations to embrace DevOps to bring order to the services being consumed.

“Within a hybrid cloud DevOps will be required,” said Labourey.

While platforms are still relevant, they increasingly are being hidden behind services that expose an application programming interface (API), added Chip Childers, CTO of the Cloud Foundry Foundation, which oversees the development of the open source Cloud Foundry platform-as-a-service (PaaS) environment. A developer consuming a service might have little to no idea what platform that service runs on.

“IT teams are building on the abstractions that have gone before,” Childers said.

The ability of one organization to differentiate itself will come down to the maturity of the DevOps processes they rely on to most efficient consume a service, he added.

Demand for that level of DevOps expertise is already starting to accelerate, said Tim Beerman, CTO of Ensono, a provider of managed IT services. As IT continues to transform into microservices enabled by containers and serverless computing frameworks, the complexity of the overall IT environment requires a more sophisticated approach to managing the overall environment, he said.

But, he added, “A lot of organizations simply don’t have the talent.”

Of course, IBM is betting that once it owns Red Hat, more organizations will look to IBM Services for that expertise. But as always, competition across the IT services space remains fierce. IBM is also betting that it will provide the management plane through which hybrid cloud services will be unified. But even there, competition from rivals small and large will be significant. Vendors such as Rackware are already providing frameworks for managing hybrid clouds.

While hybrid cloud computing is still relatively immature, organizations that embrace DevOps will be much further down that path than others, said Todd Matters, chief architect at Rackware.

“DevOps will absolutely be a critical part of hybrid clouds,” he said.

IT has always been about striking a balance between people, process and products. It’s too early to say whether IBM will recoup its $34 billion investment to acquire Red Hat. But the deal inevitably will lead to a greater awareness for a more comprehensive approach to DevOps capable of spanning a hybrid cloud computing environment that includes multiple platforms.

— Mike Vizard

November 2, 2018