devopsy.kinsta.cloud

Category: Features

IoT Future-Proofing for the Long Haul

Think about this: The IoT application your DevOps team is developing right now, if successful at scale, will likely be the longest-living application in your shop. It may even outlive the tenure of many of your team members with your company. Is IoT future-proofing possible over a life cycle of 10 years or more? Yes, with some insight for the long haul borrowed from industrial and defense computing experience.

That may sound counterintuitive for DevOps because those industries aren’t exactly known for agile processes. They’re designed around minimizing change and containing risk in mission-critical situations. IoT technology is evolving rapidly and it’s difficult to say what things may look like in a year or two, much less 10. What IoT applications have in common with industrial and defense systems is that they are often complex. Your operational technology (OT) people probably appreciate how complex their existing production environments have been. IoT applications add elements of wireless connectivity, cloud architecture, real-time performance, end-to-end security and algorithmic sophistication.

At-scale IoT applications will require mastery of complexity, and lessons learned can help. The objective is not to avoid change but rather to “build the airplane while flying.” The core DevOps concept of continuous application release and testing needs to extend beyond just the application code. Teams will need to be detail oriented, looking at device configurations, interoperability, data integrity and how analytic insights evolve as data accumulates.

Device Configurations

Buying an enterprise server usually means selecting a commodity: Pick a processor model and frequency, memory size, number of Ethernet and USB ports, storage interfaces and packaging. In the cloud, selections in elastic instances are even easier. As long as the box fits and runs the desired operating system, the details of what’s inside aren’t that important. If you have to buy another one in two or three years, there are likely higher performance choices available.

In long-lifecycle computing for IoT applications, boxes aren’t commodities. Real-time performance drives differences in processor and board architecture. If you have to buy another one in two or three years, that could be a problem. Vendors often offer industrial and defense customers revision control services, ranging from notification of changes to extended availability of specific configurations. Assuring supply sometimes means a lifetime purchase of components, perhaps as bonded inventory for a customer.

Interoperability

Even simple IoT devices, such as wireless sensors, can present a problem on a 10-year scale. Connectivity specifications evolve, with both major changes and minor revisions. Cellular standards reach sunset as mobile subscribers leave them behind. Vendor support for older gear is often given reluctantly, frequently expensive and sometimes non-existent unless an upgrade is executed.

Choosing interoperable frameworks with robust ecosystems will be critical for success in the long haul. Teams should be prepared with a “plan B” if a vendor is acquired and stops offering a specific configuration, or if they cease operations altogether. Devices with over-the-air (OTA) programming help keep pace with specification changes and alleviate security concerns as they arise.

Data Integrity

Teams won’t be able to just stop the IoT plane and work on it. Data collected on day 1 will be just as valuable as data collected on day 3650, if it’s handled correctly through IoT future-proofing tactics. Often, IoT devices connect to networks easily. The real test of an IoT application comes when data starts flowing, perhaps from thousands of devices. Is the data from an authorized device? Is all the data in the correct format? Is it time-stamped properly? Is it checked for errors and missing frames?

Presumably, when an IoT application succeeds, the temptation to scale quickly sets in. Suppose a new group of sensors is added with the same class of information in a slightly different format. The system should abstract formats and sanitize data at the edge, so all data reaching the cloud is ready for algorithmic processing. For example, JSON is one tool that can help with abstraction. Edge computing may also use generative models to reduce data transmitted to the cloud, forwarding only out-of-bounds conditions for further analysis.

Analytic Insights

At the onset of an IoT project, teams have an idea of what they have set out to accomplish. Inevitably, as projects mature new insights come out of analytics. The architecture may not be optimized to deliver those insights in real-time, and changes may be necessary to speed up delivery of algorithmic results. Workflow optimization is an ongoing process for IoT teams. This is the ultimate instance of rebuilding the plane while in flight, extracting more value from an in-place system.

The Ethos of IoT Future-Proofing

IoT future-proofing is possible by embracing both the longevity and the complexity of the application. Shifting your team ethos from maintaining a static configuration to developing a living application architecture, helps overcome events certain to happen over a 10-year life cycle and beyond.

— Don Dingee

March 4, 2019
SoulCycle Flexes Its Muscles with DevOps and CI/CD
When SoulCycle wanted to improve application development and deployment, including improving services for customers across its 80-plus indoor cycling centers in the United States and Canada, the company turned to Mark Sost, a veteran developer and engineer who worked at several media companies, including Hearst.

Over the last two-plus years, Sost, who now holds the title of senior director of engineering at SoulCycle, has built-up the company’s engineering and DevOps teams, increasing the headcount from eight or nine developers to about 40 employees now.

The first major project was revamping a legacy, LAMP-based stack used to support applications to focus more on a modern platform that could take advantage of DevOps methodologies, as well as continuous integration and continuous delivery practices (CI/CD).

Sost and his team dubbed this new platform Podium. However, Podium is more than a development platform, Sost explained.

“Podium became the ethos and the methodology, as well as the architecture and application stack for re-platforming Soul’s technology stack in total,” Sost said in a telephone interview. “We moved from that monolithic to more of a service-oriented architecture. As such, we started to break out functionality on multiple applications.”

The re-engineering also meant investing in new infrastructure, as well as other cloud-based technologies. This includes a multi-cloud approach.

While most of the legacy applications have now been moved to Amazon Web Services, SoulCycle also uses Google Cloud Platform for Podium and all new developments. Sost also takes advantage of Kubernetes and Docker for container orchestrations, as well as investments in other managed services.

Other tools in the DevOps arsenal include Jenkins to run unit tests; DataDog and StackDriver for monitoring and analytics for workflows; combing Travis CI, Cloud Builder and GCR for CI; and technology from startup Harness, which provides CD-as-a-service. Sost called Harness the “last-mile” tool needed for application deployment.

“For Harness, that’s CD for us,” Sost said. “That’s continuous delivery to our various Podium environments. We’re leveraging it for delivery to all our Podium environments. Everywhere from UAC (user account control) during the code review cycle to QA after merge, to production, load test and everywhere in-between.”

This revamped approach to application development now means that the SoulCycle engineering team has created approximately 30 different apps over the last 24 months, this includes internal apps used within the company environment, and ultimately to customers using the facilities and equipment.

Additionally, Podium helps create a more unified architecture to help ensure that the entire team—whether it’s DevOps, QA or the production team—can paint a 360-degree view of the problem that needs to be solved and what the implications are for the business once the solution is put into place.

This is what Sost and his team call “Podium principles.”

“When we say that we’re building things with Podium principles in mind, we’re talking about the fact that everything is version-controlled,” Sost said. “We have full test coverage. We have everything in terms of Kubernetes manifests to infrastructure automation written down and version control checks it as part of source code. That applications are conforming to more of that 12-factor methodology and that we’re really focused really heavily on developer empowerment.

“I think for us, that really is a shift that Podium has introduced; a move from writing the code as an application developer and then throwing it over a wall to an operations person—a QA person—to test it and deliver it somewhere,” Sost added. “Instead, we provide a developer the tools and the patterns of principles to go about controlling more of their application development life cycle all the way through to delivery.”

The results have helped improve the overall SoulCycle experience for customers, which includes:
- Reducing deployment time by approximately 80 percent.
- Instead of one hour, deployments now take one developer 10 to 15 minutes.
- QA verification has been reduced by approximately 80 percent.
- It now takes three QA engineers one hour to verify, compared to three to six hours previously.
At the same time, Sost believes that the Podium approach gives developers more control over their projects and lets them focus on writing good code that creates apps people want to use.

“It’s more, ‘Now, I know what’s involved with getting my application to people,’” Sost said. “Because at the end of the day, engineers are just concerned with writing things that people want to use. … This is how we’re going to give you more control of the code and applications that you’re writing, so that you can see meaningful results quicker, without having to wait for someone.”

What’s next for Sost and his team at SoulCycle? The answer: Machine learning and a bit of automation to the DevOps process.

Here, SoulCycle is using Harness tools to log more data and infrastructure monitoring into the platform, which allows the algorithm to learn more about the various development projects. This can make the code better and eliminate errors that come with constant development and deployment.

“We’re letting it get smarter over time around how it can detect anomalies and understand whether our code was successful because there are going to be times where things slip through unit testing or slip through integration tests and there are even going to be times where they slip through load tests, where things like 99 percent are just not going to cut it,” Sost said.

— Scott Ferguson
March 1, 2019
How Mainframes Live On

TL/DR: Highly controlled change is sometimes the best answer.

For much of my career, I’ve worked in mainframe shops. I have never worked on a mainframe team, but I have worked closely with them. I’m in Northeast Wisconsin, and at one point we had the highest per-capita number of mainframes in the world, so pretty much anyone from this area can say the same. They were the “stable core” of the system for which I was the “new shiny”—and at times, that was painful.

If you’re an established enterprise, even after years of trying to eliminate the mainframe, you probably still have one. They don’t go away so easily because, despite frustrations with rate of change, they do host core systems in established organizations and are reliable. I have seen similar in client server systems that are core to the business.

We are reaching a point where there are some great products out there to help modernize the mainframe and/or split up client/server applications. Some of them have been around a while (Linux) and are coming into wider adoption, some of them are building on that bit of more familiar interface and many are simply making the UI more familiar to developers like me that only ever developed in a mainframe terminal for college, if at all.

One of the main reasons why mainframes and future iteration’s applications are “sticky” is because they, and the systems they host, are generally reliable. No matter what you think of the timeline for updates, the data coming out of the mainframe and more controlled distributed applications is considered accurate in most organizations, over most other sources.

To be clear, there are groups like this from each successive wave of computing. The client/server app that replaced your mainframe customer information system has learned over time that a fault in their system impacts the core of the business, for example.

The goal is to increase the rate of change, without undermining the reliability. You hear a lot about throw-away instances and “fail fast,” but dropping your billing system isn’t really an option, so these need to be tempered with the knowledge of what failure implies in core systems.

Make the change, do the modernization, but listen to lessons learned on core systems. Take into account that your informational website being down is a serious problem for marketing, but the billing system being down is a disaster for the company. There is a degree of difference, and the approach should be less about continuous deployment and more about stable deployment. Yes, it can be a scale, and you can start the motion from one to the other slowly to make sure you’re covering your bases.

In short, assess the risk levels of these systems before deciding how to proceed. This is pretty much common-sense advice—but better to read it and not need it, than to find out when corporate revenues are impacted.

I worked at one shop where a certain core application was using technology that was a couple of generations old and no longer supported. The pool of available developers was shrinking, but it was an important application. This type of thing is where modernization is critical; having someone say, “We don’t touch that,” is no longer acceptable. That story ends in disaster when the system fails, and no one knows how to troubleshoot—let alone fix—it.

As with so many things, the balancing act is up to you; determine your organization’s level of risk acceptance, then determine the impacts of this application being unavailable and proceed from there.

Each step of mainframe development is a little different from what developers coming out of college expect, but there are new tools and mechanisms available of which these developers can take advantage.

— Don Macvittie

March 1, 2019
DevOps Chat: Repos and Nexus Firewall Access, with Sonatype

There are really only two repositories of any scale for software components today: the Nexus repo managed by Sonatype and the Artifactory artifact repo managed by JFrog. Up until now they were separate and apart, and working with one was independent of another.

In a big move toward keeping DevOps open and secure, the Sonatype people have released a plugin that will allow their Nexus Firewall to work with Artifactory as well as Nexus.

This means that users of both repos can now use the Nexus firewall to make sure that components or artifacts they download are in compliance with the policies they have set up. They can make sure there are no known vulnerabilities to the version users are downloading to use. This is really a big deal and kudos to Sonatype.

In this DevOps Chat we speak with Wayne Jackson, CEO of Sonatype, and Brian Fox, CTO and co-founder of Sonatype, about what this means for the DevOps community.

Transcript

Alan Shimel: Hey, everyone. It’s Alan Shimel, DevOps.com, and you’re listening to another DevOps Chat. Today’s DevOps Chat is breaking some big news from our friends at Sonatype. I’m really honored and happy to be joined by Sonatype Wayne Jackson and Sonatype CF – CTO, Brian Fox. I hope I didn’t mess that up, guys. Wayne, Brian, welcome.

Brian Fox: Thanks.

Jackson: Thank you, Alan. Great to be with you and great to be speaking again.

Shimel: Yeah. Well, Wayne, you and I go back just a couple years.

Jackson: [Laughs]

Shimel: Let’s not go and do a history thing. Let’s get right to the breaking news today, and that is kind of a major addition to the Sonatype – I know it’s a firewall. Do you still call it Nexus Firewall or is there a different name for it now?

Jackson: Yes, we – yeah, that’s correct.

Shimel: It is the Nexus Firewall. So before we talk about what the major addition is, let’s lay a little groundwork here. What is the Nexus Firewall? Brian, you’re the CTO, let’s give that one to you.

Fox: Alright, sure. Nexus Firewall is a set of capabilities that allow us to do analysis of components as you pull them into your supply chain through your repository and be able to match that against your policies to enforce things like architecture, quality, license, et cetera, and so specifically it’s intended to be able to stop components from coming into your supply chain when they’re known to be against your policy right up front. So you can stop developers from pulling in a component that has a known vulnerability or pulling in a component that has a incompatible license, those types of things. So that’s the role Firewall plays in the overall portfolio.

Shimel: Perfect. And, you know, let’s be clear; that – you know, a Nexus Firewall, if it’s in place the Equifax breach never happens for all intents and purposes.

Fox: Well, that might – what it doesn’t do – you know, it’s not gonna go back and take out the things you’re already input.

Shimel: You’ve already put in, you’re right.

Fox: Right. Certainly. If – you know, we still see lots of cases – I think we’ve done some of the studies recently that show organizations are still downloading the same thing that got Equifax attacked, right?

Shimel: Absolutely.

Fox: It’s the same version of Strut. So, Firewall would definitely stop that behavior, no doubt, and be able to prevent it from further propagating through your supply chain.

Shimel: And just a little shout out, Equifax is now a Nexus – a Sonatype Nexus customer, correct?

Fox: Yeah, that’s correct.

Jackson: Yes, that’s correct.

Shimel: So hope – well, at least it’ll prevent the next one, hopefully. So – but guys, so we understand what Nexus Firewall is now, and again, to put in context the importance of today’s announcement, Nexus Firewall worked when you were pulling components out of the Nexus repository up until now. Am I right or am I off base here, guys?

Jackson: No, that’s correct.

Shimel: And so the big news today is you’re expanding beyond the Nexus repository to offer protection from artifacts and code and so forth from other repositories.

Jackson: That’s right. It’s – you know, it’s an acknowledgement, and it may be the first one in this category, that just acknowledges that the DevOps world is heterogenous. You know, there are bunches of vendors and, you know, people within organizations make different decisions and we just acknowledge the need to provide universal protection.

Shimel: Absolutely. So Wayne, work with me here. It’s the Nexus repository, and with this new release out today you guys are also announcing support and protection for the Artifactory repository, correct?

Jackson: That’s correct.

Shimel: And –

Jackson: And between those two, you know, I’m not sure what the coverage is but it’s gotta be in the 90^th percentile of implemented repositories.

Shimel: I would agree with you. I think between Artifactory and Nexus it’s gotta be I would say high 90’s even, right? We’re not talking 91 or 92 percent. I think it’s probably way up there, and it’s certainly in the billions when you look at the amount of whether it be code or components or artifacts that are contained between the two of them. And Wayne, you hit on it before about the heterogenous nature of today’s DevOps world. It really is; it’s become Artifactory and Nexus as the two big ones.

But there’s another aspect to the heterogenous nature of today’s DevOps world and that is this whole notion of the software development life cycle, managing the software pipeline, the modern software factory; different companies are calling it different things but it all refers to kind of the way that software is being both written and assembled in – by, you know, modern IT departments today. Why don’t you guys expand on that a little bit.

Jackson: Yeah, I – it’s been fascinating to watch. As you know, I came from the network security world before Sonatype and had no grasp of just how much software was being written. As I’m sure you know, you know, one of the assets that we manage in the ecosystem is this thing called Maven Central, and last year we served 142 billion download requests, and so – and that’s just in the Java ecosystem and, you know, other ecosystems like NPM are just as big.

Shimel: Yeah.

Jackson: And so as we started looking at the patterns in these download requests we started to see this weird disconnect. You know, the disclosure of a vulnerability had almost no impact on usage patterns, and that led us to thinking about, you know, why that might be and how we might think about solving these problems more systematically, and it dawned on us that we had a massive supply chain problem. You know, where the producers of things didn’t have a very effective mechanism for communicating to the consumers of those things because there were just millions and millions of projects that had to be monitored. And so, you know, we started looking at the works of Edwards Deming, who was the guy that transformed Toyota from being a textile manufacturer to the world’s most efficient automobile producer, and it really wasn’t all that complicated. You know, you have fewer, better suppliers, you use the highest quality parts, maintain transparency throughout the system, and don’t pass defects downstream.

If you can do those four things, lo and behold, [laughs] producing cars becomes a lot easier. And, you know, we concluded that those exact same principles applied perfectly to software where, you know, to your point, you know, now all of a sudden these software development functions, especially in the more modern organizations, look like a software factory, and so let’s use fewer, better projects, let’s use the versions of those projects that don’t have security defects, that have the kind of license that matches our distribution model and, you know, let’s build a model where we know [laughs] what’s in the system. And if along the software factory assembly line we discover that there’s an issue, stop it there. You know, don’t pass it downstream.

So anyway, so it’s a model – you know, I’m a little bit of a car guy so it [laughs] certainly works for me, but it feels like a model that’s right.

Shimel: I agree. It absolutely does. And even before you mentioned Deming, and of course it’s obvious, I was thinking third-party suppliers just like we do see in the automobile industry. But what’s interesting is as you referenced, the sheer magnitude of the numbers here – you know, 142 billion – think about what that means. I mean it’s just an incredible, incredible amount of downloads of usage and everything else and, you know, it doesn’t take many bad apples to spoil the party for everyone, obviously. You know what’s interesting though, guys? Again, getting back to the DevOps community, the DevOps market today, Sonatype is very much – you know, who you are and what – it seems to me anyway – you know who you are, what you are, where you play and how you fit into this.

And I think that’s in contrast to – you know, we’re in a little bit of a time of upheaval in the DevOps spaces as the market matures and vendors are jostling and trying to – I think some of them might, you know, even be going through an awkward teenage phase, you know, where they’re trying to figure out who they are and what they’re gonna be when they grow up. You know, what – Wayne, you’re the CEO there – what’s the vision for Sonatype? Are you kind of comfortable in your own skin now and what is that role?

Jackson: Yeah. I think we are comfortable in our skin. I think this embrace of software supply chain management is who we are, it’s who we’re going to be, it’s sort of a unique position to take in the DevOps category, and it allows us to play very nicely with the vendors that are all a part of the assembly line, you know, whether it’s Artifactory or CloudBees or any of the other vendors. And so being able to produce information that gets translated into knowledge that, you know, automates decision-making with regard to supply chain integrity, I think is a good spot for us.

Shimel: I agree. I mean and not only is a good spot I think it’s a pivotal role to play at this – you know, as we see the continued maturation of the market and, you know, it’s all wrapped around the digital transformation and everything going on. Brian, you know, your background, you were an early chair of the Maven Project. You’ve been in this opensource, you know, world for some time, and Wayne did too coming from the Sourcefire Snort folks, but what’s your take as we see this, you know, maturation? I mean at some level it’s gotta be – I would imagine it’s gotta be encouraging to see cross – like, you know, being able to protect across different repositories, whether it be Maven and Java or Artifacts or what have you, right? It’s gotta feel like we’re making progress. What do you think?

Fox: Yeah, I think so. I mean our approach since we kind of went down this path, what, 2012 or so is to take an approach that sits above all of the tools, because we recognize from the early days when we were doing a lot of Maven training and consulting and we had the privilege I guess to go to companies and see lots of horrifying processes and all the ways in which attempts to make this better inadvertently made things worse, and we recognize that it requires a sort of across-the-life cycle approach, which means you can’t solve the problem just by adding features to, you know, the repository or the CI. And frankly, given that we were building Nexus Repo Pro back then – I mean we still are, but that was our main product back then – adding it as features on top of the Repo would’ve been the most natural thing for us to do, but we understood through all of that learning that that wasn’t going to be sufficient.

It required integration to the Repo and to the CI and to the build and to the release automation tools, et cetera, et cetera, right, to be able to most effectively nuance the process of the components that flow through that, because you may choose to respond to a policy violation differently when something is about to actually be pushed into production versus something that’s just happening in your – you know, on every commit CI build. And so for that reason, being able to integrate into all of these tools, including all of the different repos, is just natural. It’s a natural evolution of the approach we’ve been taking the whole time.

Shimel: Got it. So, I guess the next question is now that we have – you know, between the two repositories we’ve got high 90 percent of the market – how do we get 90 percent coverage with this Firewall product, right? I think we’d all sleep better. You know, what is the key, guys, to getting more people to realize this is kind of a no-brainer that they need to get done here, right? I mean what do you think is preventing us from really seeing this?

Fox: Culture. Culture and history. I think there’s a history of application security tools specifically that, you know, are prone to false-positives just by the way they worked and led – and a history of bad approaches, like I was talking about before, that tried to force bad tools onto development that ultimately led to rebellion or disuse and gear grinding, and so in some ways the security tribe I think has become a little bit afraid to engage development in a lot of organizations, right? So they very much try to work at the end of the process, try to produce lists. We know that those things tend to get ignored, that they don’t usually have a profound effect on what’s happening, and so the organizations that are really successful are the ones that, you know, can set all of that history aside and still work together in a DevSecOps type of way and, you know, look at the problem holistically and understand that, yeah, actually it makes sense. If we can stop the bad things from coming into the process to start with we don’t have to chase them out later.

The security guys don’t have to track it in the list and the Dev guys don’t have to go rework it, right? And so, you know, that’s kind of the approach that we’ve been trying to take, that all of these different constituents have an important job to play in the success of a business and it works best when [laughs] they’re all working together, not working against each other and throwing grenades over the walls.

Shimel: Absolutely.

Fox: And so really, you know, the biggest hurdle to adoption in most places just simply is that culture and the history of disdain and things like that.

Shimel: Yeah, I agree, Brian. And guys, I’m gonna tell you something. Here’s a private theory that I’ve been working on and I’m gonna be speaking about it in the months ahead. You know, we’ve worked really hard – Sonatype more than most any other vendor I can think of – to advance the cause of DevSecOps. You know, Derek and Mark on your team and myself have been putting on DevSecOps events at RSA for the last five years now, and we do it in Singapore at RSA and we go all over the world with this DevSecOps thing. And recently, in the last couple months, I’m starting to see sort of a bifurcation in how security is dealing with DevSecOps, right?

I’m seeing some companies that are embracing the Dev sides of things, and let’s call that DevSec. They’re working with developers to make the code more secure and make sure that pre deployment and everything is tested, everything is secure, and we release really tight, good quality, right? Security is synonymous with quality, and let’s call that DevSec. Then there’s another group of vendors, Brian, that are dead into what you’re talking about, right? They’re almost afraid, intimidated, or just fed up with dealing with developers, and they’re really focusing on the post deployment aspect of things here, right, and let’s call that SecOps, because they want to work more with the Ops folks, post deployment, you know, monitoring and that kind of thing.

And I’m not saying that there’s not a reason to do that or it’s not right to do that – it absolutely is – but when we start looking at it as DevSec and SecOps, we’re committing the same mistakes that we did to begin with and we’re losing that it is DevSecOps for a reason. It’s Dev and Ops and Sec all working together. We can’t say, “Oh, we’re gonna work with the Ops guys but not the Dev guys or vice versa.” It’s gotta be everyone’s in it to win it. And I –

Fox: Completely agree.

Shimel: But I’m – yeah – but I’m seeing security vendors now kind of, you know, taking sides here on this. I’ve recently done a few podcasts – and some of them are new companies that are launching or coming out of stealth and RSA, and they’re growing this distinction that they’re SecOps, not DevSecOps, or they’re DevSec, not Dev – and I think it’s a huge mistake, a huge mistake.

Fox: I think that’s largely led by the constituent. If you go and talk to the security guys, that’s what they’re gonna tell you that they want, right? It’s part of that history.

Shimel: Yeah.

Fox: You know, so if you allow yourselves to be pulled directly in the direction of what people want based on their perceived understanding of just how things work and that’s the way it’s always gonna be, that’s I think where you end up, so you have to buck that trend a little bit and say, “Guys, it doesn’t have to be this way. It can be better.” But we have to work together to do it, right, and we have to think about things a little bit differently. You know, and I have many conversations – you know, it’s happening more and more as AppSec and OPSEC is becoming more generally aware of the opensource component, you know, risk. They’re starting to come at the problem, but unfortunately they’re just trying to add more data to their existing process, and I have to remind those guys all the time that developers are picking new components all the time, they’re picking new versions of those components all the time.

They’re not only doing it because you handed them a list of vulnerable components or a list of CVEs and said, “Fix it or else,” right? So wouldn’t it be nice if you had a process and a set of tools that could support them in that behavior so they don’t come on your list later? [Laughs] Right? And a lot of times that is like a shocking realization to that tribe, right, and that’s that culture that I was talking about. It’s a big challenge.

Shimel: Agreed. And look, our work is not done, right? That’s what that means to me. We still have more work to do evangelizing DevSecOps and making sure people under – take a whole holistic aspect to it, and it goes back, Wayne, to what you were talking about in terms of the pipeline and the software, you know, automation and pipeline kind of thing, is it’s not bifurcated at the pre deployment versus post deployment juncture or junction. It – DevSecOps is about the whole thing from end to end.

Jackson: Yeah, the organizations that can think about this holistically are just bound to be more effective and more efficient and, you know, to your point about evangelism, I mean one of the things that we’re going to see, and I think we are seeing, is that the most highly-performing organizations are the ones that are getting it.

Shimel: Yeah.

Jackson: They’re the ones that are gonna be leveraging software to more effectively differentiate their business. [Laughs] And at some point, unless you want to lose, you’re gonna have to start thinking about the ways that these, you know, forward-thinking, advanced organizations are doing what they do to make themselves better every day.

Shimel: Absolutely. And you’ll have winners and losers, and look, that’s capitalism and that’s the market dynamics at play, right? You can adapt or die or not, right? I mean it’s up to you. Guys, we’ve spent a lot of time talking, you know, about this at a macro level. Brian, if you don’t mind, I’d like to get a little geeky on you and ask if you can explain to our audience at a nuts and bolts level really what’s going on with the Nexus Firewall.

It’s software, right? It’s not an appliance, per se. Where does it sit, how does it intersect, how do you set policies in it? You know what I’m saying? Let’s give our audience a little bit of a…

Fox: So our Nexus platform is kind of composed of maybe three parts. At the core is a piece of software that you run on prem or in your cloud that we call the IQ Server, and this is the thing were you define your policies, it’s the database of all of the components and which applications they’re used in, the dashboarding, the reporting; you know, all of that type of stuff occurs within the IQ Server. The IQ Server works with services that we have in our cloud to provide the real-time information around the components and the security and the licensing and all that type of stuff, right? So that’s a real-time feed, it’s not a data dump that you have to download periodically.

And then there’s all of the plugins, the plugins that go into all of the places I mentioned before; your build, your IDE, your CI, your repository, your automation tools, et cetera, right? And so what we’re talking about here is we’ve added a new plugin that plugs into JFrog Artifactory. So when a component is being proxied from an external repo, such as Maven Central, the plugin will do the analysis of the thing being requested, it will understand what it is, work with the IQ Server to understand if it’s in compliance with the policy or not, and the depending on how the policy is configured it can either allow it, it can quarantine it, or it can just outright block it, right? So that’s the same capability that we’ve had for a couple of years now inside of the Nexus repository, so it replicates that functionality for our JFrog users.

Shimel: Fantastic. Wayne, how are we going to market with this?

Jackson: I get that question asked to me a fair bit, and we have a very multifaceted go-to-market model. You know, we still have folks that are very senior out in key regions that are evangelizing, because this is still an immature market, but increasingly as the market’s maturing are starting to lead with product and product capabilities in our marketing function. So, a mixed model, but one that we think aligns with where the market is in terms of its maturity, you know, that will change over time.

Shimel: Understood. Guys, we’re almost out of time. If it’s okay I wanted to return back to the DevOps market in general. Wayne, you know, you and I kind of saw the security industry mushroom, right, from let’s say early 2000’s through even 2008, ’09, ’10, you know? I – and we now have both – and have both seen the DevOps market mushroom, you know, from 2013, ’14 to today. It’s only been four or five years. And, you know, we touched on it earlier with companies trying to kind of jostle for where they are, what they do and how they fit into this brave new world. You know, put on your visionary hat, if you will. Where’s this going? Where is it – you know, is there a logical sort of conjunction of security in the DevOps? And if so, what does that mean for the broader market? What do you think?

Jackson: I disagree with you a little bit in that – with regard to DevOps having mushroomed. I think we in the industry kind of feel like it has because now all of the sudden people are starting to use our words back at us, but I think in terms of, you know, DevSecOps being a mainstream thing that’s just a part of how people work is still a bit away, and I think, you know, the work that you’re doing, I think the work that we’re doing and other vendors in this space is still enormously critical because even organizations that aspire to DevSecOps still need help. It’s not an easy thing to embrace and implement, as you know, and so there’s still some work to do, which I think is cool.

Shimel: Yeah. Well, it keeps me going every day and gets me up. And you’re right, you know, we tend to forget that we live in a bubble. When you’re a hammer, everything looks like a nail, right?

Jackson: [Laughs] Right.

Shimel: When you’re talking DevSecOps all the time people tend not to look at you and say, “What are you talking about?” So, you know, we kind of breathe our own exhaust, if you will. But there is, there’s a lot of work to be done, even – forget DevSecOps – even in the basic DevOps level, right? When you – a lot of companies have dabbled in DevOps but how many are truly end-to-end DevOps, right, and that number I’m gonna guess is well – is below 20 percent, probably closer to 10 percent.

Jackson: I think you’re right, I think you’re right.

Shimel: Yep. So our work is still in front of us. But hey, you know what? It wouldn’t be fun if it wasn’t, right? I mean that’s what keeps us –

Jackson: [Laughs]

Shimel: Anyway, guys, when is this new plugin for Nexus Firewall available?

Jackson: More or less now.

Shimel: Okay.

Jackson: I think the official release date is March 1, but yeah, happy to show it to anyone who’s curious to see and very proud of the technology.

Shimel: Absolutely. And they can get more information at Sonatype.com?

Jackson: That’s right.

Shimel: Fantastic. Wayne Jackson, Brian Fox, CEO and CTO, cofounder of Sonatype, respectively. Thanks for being our guests on this episode of DevOps Chat.

Wayne Jackson: Thank you. It’s been our pleasure.

Fox: Thanks for having us.

Shimel: Always a pleasure. Always a great time, guys. Okay, this is Alan Shimel for DevOps.com. You’ve just listened to another DevOps Chat. Have a great day, everyone.

[End of Audio]

— Alan Shimel

February 28, 2019
Smashing the (DevOps) Stack for Fun and Profit

Most organizations that I deal with have a similar story for the promulgation of DevOps in their data center. The story is similar for bringing in most new technology and techniques: A team that usually does a new project without any attachments—relatively small compared to projects at the center of the business—adopted DevOps from the start. A smaller (by far) number shoehorned DevOps into an existing project as a first taste. Some decided to go “all-in” and “convert” IT to DevOps. This is a painful method; I can’t say I recommend it. I suspect my friends at those organizations wouldn’t recommend it, either.

When I say “DevOps” in this sense, I am talking more about process and procedure changes, even cultural changes, rather than tools. My experience is that most companies that have been around longer than the last spring shower started adopting the tools of DevOps before they started looking at ways to streamline the process. And that isn’t a terrible thing at all. It doesn’t matter what marketing spin is used to sell the product; if it makes your life better or allows you to react faster, implementing it is worth consideration. I just wanted to be clear what we’re talking about here.

Those companies that started small generally grew small, also. It is a big deal to move that 20-year-old codebase with a ton of processes that touches everything into DevOps when compared to saying, “New projects will use DevOps.” This, too, is in line with what we’ve always done. Iterative replacement has always been the case, where the large, important systems are concerned. In a couple of industries I am familiar with, I’d even argue that those large, important systems weren’t completely upgraded to Customer Information Control System (CICS), so don’t hold your breath on seeing them completely upgraded to DevOps. But that is a different blog.

Once a critical mass has been hit, these shops start referring to themselves as a “DevOps shop” or “DevOps first shop.” This is the point I want to talk about today (though it took me a bit to get here).

You’ve got a bunch of apps or a couple of application portfolios that are now running under DevOps. In most organizations, the tool sets are not a perfect match. Indeed, in some organizations the DevOps toolchains look like they were assembled on opposite day. Teams tend to be more standardized, but even team composition varies from project to project, as do communications.

This is the next point of inflection in DevOps growth. You have it in your grasp to jump again, from amazing to truly amazing.

Time to smash the DevOps stack.

Get the teams together, build a list of each system used at each step in the process for each app, each manual intervention, each communications channel—essentially each difference between DevOps teams—and start to look for synergy. If one team is using ChatOps, check out what benefits they perceive from doing so and if ChatOps fits all teams. Does one team have a standard process to roll from continuous delivery to continuous deployment? Check it out. If you have several disparate DevOps teams, it is almost guaranteed that one team will handle DevSecOps better than the others. Take a look at what/how they’re doing and promulgate it to less security-intensive projects as needed.

Basically, build the corporate process/toolchain from what is successful. That sounds like simple advice, but it isn’t. Teams often love their tools and processes, and are certain their choices are better than others. It takes work to get to a common understanding and there are always exceptions in a complex environment: “Well, yes that works, but for our app, we have this thing over here that makes it not possible/feasible/affordable.” It requires extra work, but that’s always true. Because the environment is complex and has been for a long time, these issues have always existed; you’re just dealing with them with new processes.

Keep kicking it! Keeping the virtual lights on is as important as keeping the physical lights on in today’s environment. Unless your company is listed in “haveibeenpwned” emails, or your website has been suffering catastrophic outages, you are kicking it.

— Don Macvittie

February 22, 2019
DevOps Chat: JFrog Acquires Shippable

The big news in DevOps today was the acquisition of Shippable by JFrog. This adds a CI/CD solution to the JFrog Enterprise+ platform and further expands JFrog’s end-to-end DevOps offering beyond Artifactory and more.

In this DevOps Chat, we speak with Shippable CEO Avi Cavale, JFrog CEO Shlomi Ben Haim and JFrog VP of Marketing Ayally Goldschmidt about the thinking behind this acquisition and JFrog’s plans going forward.

As usual, the streaming audio is immediately below, followed by the transcript of our conversation.

Transcript

Alan Shimel: Hey, everyone, it’s Alan Shimel, DevOps.com, you’re listening to another DevOps Chat. Today’s DevOps Chat has some breaking news. It was just announced this morning—JFrog has acquired or merged with Shippable in a rather, you know, this is big news within the DevOps world, and we’re very, very happy to be joined by the CEOs of both Shippable and JFrog. From Shippable, we have Avi Cavale, and from JFrog, of course, our friend, Shlomi Ben Haim. And joining us from JFrog is Ayalla Goldschmidt, who really runs marketing at JFrog. I’m not sure of your exact title, Yes, Ayalla?

Ayalla Goldschmidt: VP of Marketing, thank you, Alan.

Shimel: VP of Marketing—thank you for joining us. So, first of all, to all three of you, congratulations, Mazel Tov as they say in Seattle. [Laughter]

Goldschmidt: Thank you, thank you.

Shlomi Ben Haim: [Laughter] Thank you, Alan.

Shimel: Thank you. And so, you know, Avi, let’s turn it over to you first. I think a lot of our audience is familiar with JFrog, and most of our audience have heard of Shippable, but let’s just lay that right out there. What’s Shippable about?

Avi Cavale: Shippable is about a modern CI and CD platform. So, our focus is to streamline the delivery—packaging and delivery of software, and that’s kind of where we are focused on.

Shimel: Fantastic. So, Shlomi, I think I already know the answer, but I’m gonna ask you—why Shippable for JFrog?

Ben Haim: That’s a great opening question. Actually, Alan, you know JFrog very well. You know that everything starts with the culture, and when we met Avi and the team, it looked like we’d found the brother Frogs in a different company. So, for us, it was just to make sure that the technology and the strategy are aligned. And when we looked at the technology and the next generation of CI/CD that was built by the Shippable team with Avi’s leadership, it was a no-brainer for us.

Shimel: Absolutely. But the other thing I think we should mention is, you know, JFrog, many people know JFrog from Artifactory, maybe Xray, you know, point solutions that we can call up. Artifactory is a whole platform in and of itself.

But, really, the big move for JFrog—and if I’m wrong, correct me; Ayalla, you’re the Marketing person—the big move for JFrog was, you know, this JFrog enterprise platform. And, to me, if you’re gonna have an enterprise platform in the DevOps space, you need that CI/CD pipeline kinda delivery capability.

So, to me, you know, this really kinda helps fulfill that vision of the JFrog platform. Yeah, no, wrong?

Goldschmidt: Yeah, that is exactly right, Alan. That’s the vision that we have for it. Obviously, we’ll be doing a lot of work here in the coming weeks to clarify the roadmap and the timeline, but you hit the nail on the head. That’s the value that we’re seeing for our customers and what they’re looking to, what they’re asking us for as far as, you know, we love JFrog, we have very happy customers, they love what we do, and they’re looking to take it to the next level.

Shimel: Excellent. So, let me ask my next question to Avi and Shlomi, and that is, you know, I guess it’s part of the maturation of the DevOps market where all—you know, there are now, everybody wants to offer the whole enchilada. Do you know what I mean?

Ben Haim: [Laughter]

Shimel: People aren’t content any more just offering a piece of it, we wanna offer it all, that whole SDLC, we’re gonna help you throughout that life cycle. It seems everyone is pointing towards that, and the money—look, JFrog, it’s no secret, raised $165 million. Some of your competitors have raised also eight figures. That’s where the smart money’s going, towards companies that are building out this end-to-end DevOps framework.

Avi, is that what you saw at Shippable, and is that what attracted you guys to want to do this deal with JFrog?

Cavale: Yeah, I think what we are seeing is that there is one thing which is very critical for customers. It’s like—yes, they want the whole enchilada as a single platform, but at the same point in time, they also want a platform that is open, and they want to make sure that they can plug in the tools that they like in certain places.

So, of course, they want to see the native capability end to end, but extensibility is also super important. And so, what we kind of saw with what JFrog was doing and what we were doing also was that we were being open to everything. I mean, yes, we want to have a standard of how we think about an end to end platform, but we want to be open and make sure we embrace all the practices that developers want to do and extend the platform to DevOps capabilities.

Shimel: Absolutely. So, Shlomi, Avi makes a very good point here, and that also goes to the heart of JFrog. One of the things about JFrog is that they play nice with everyone, basically, right? You could pick whatever CI/CD platform you wanna work with or whatever—you know, whether it’s Puppet or Chef or the Red Hat software, you know, JFrog works with everyone, so to speak.

But now, you have an in-house CI/CD solution, so how do you balance that in your go to market?

Ben Haim: That’s a great question, Alan. I think that what we’ve started to see in the market for more than 12 months now is that DevOps is maturing and the organization and enterprises are looking to get an end to end solution, yet the driving force of DevOps production is still in the hands of the developers, which, obviously, would love to see the freedom of choice and the best of breed options kind of open to them.

So, you are right. Our universal approach is something that always led us. We always provided our users with the power of choosing their tools pack, and it will still be the case.

However, on the JFrog platform, what we call the enterprise class, we want to make sure that we take artifacts seriously, from the moment you build it from code all the way to production. And, together with Shippable automation power, together with the integration and the visibility that technology like Shippable can provide to the platform, I think we will be able to combine the two, to allow a better universal approach. So, you can still use your Jenkins, your CircleCI, your other DevOps tool sets. But you will also have a JFrog end to end solution that takes you all the way from code to production.

Shimel: Excellent. So, Avi, you know, how do you—I’ll reverse, mirror that question back to you, right? How do you see the Shippable team, the Shippable product fitting into that JFrog Enterprise Plus, you know, the Artifactory platform, all of the different tools that JFrog and services that JFrog is putting together—how do you make sure you fit in there?

Cavale: So, as Shlomi mentioned, right, what is happening is—so, the artifact is something that is immutable, and that’s something that you build and you want to kind of take it through your entire value chain of how you deliver software to your end customers or end devices or whatever they might be.

The issue is that, if you don’t have the pipes to kind of control that in a high fidelity fashion, you lose a lot of information as you go from stage to stage. Because you’re leaving it up to the customer to make sure all the data is being persisted and you’re keeping all the hooks so that eventually what you’re really trying to do is, you want to be able to go to something that’s running on the end device and be able to track, if there are problems, what are all the pieces that went into actually building that piece of software that is running there?

So, losing that ability to connect that entire life cycle, all the way from code to the point where it’s running on a device is a very critical piece of DevOps. And what we see is, that kind of, now, our marriage to the whole Artifactory plus Xray and plus distribution, all of those pieces now can be connected through a common pipeline and we’re making sure that we have high fidelity information exchange as things move through this work flow end to end. And I think that’s kind of the critical thing that we are adding to the product.

Shimel: Excellent. Ayalla, a question for you—you look at, all of a sudden, you’re embarrassed by the riches of JFrog in terms of their product, right, in terms of the product that you have to work with, the products that you can bring to market. But at the same time, at some level, it’s a double-edged sword from a marketing perspective, right? You have so much now, too, that you need to market, that you need—how do you make sort of a holistic message for market that includes all of these pieces and how they fit together and how, if you don’t want to use all the pieces, you don’t have to, either?

Goldschmidt: You know what, I feel like the luckiest marketing person on the planet. And, you know, Shlomi promised me when I joined this company a little over a year ago that there will be no shortage of news, and he has delivered.

Shimel: [Laughter]

Goldschmidt: So, since I’ve joined, the acquisitions, our Series D announcement, and of course, all of the new technologies that we’ve supported along the way, and the most recent exciting edition of support for Go and Go Center, we love all of these audiences and we do actually put in a big effort in tailoring our message to make sure that we’re communicating with a developer first mindset, and that seems to be working for us.

So, you know, until today at JFrog, everything we’ve built has been through an inbound marketing funnel with an inside sales team. We haven’t had field people knocking on doors, and that seems to be working great for us, and now we’re ready to take it to the next level. But we’ll always remain developer first.

Shimel: Excellent. Let me—Avi, let me ask you some sort of typical M&A kind of questions. So, is your whole team staying on as part of this acquisition?

Cavale: Absolutely, and we’re all joining the JFrog family, and so there is really no change to the business and how we are running it. And I usually tell my team that I have a new boss, that’s all—nothing else is changing, and we’re gonna keep marching towards the vision, and all we have is an opportunity to broaden that vision that JFrog brings to the table and we are part of a much bigger family than what we were before.

Shimel: Excellent. And so, Avi, what do you see particularly as your role going forward, here?

Cavale: Sorry, was that for me?

Shimel: Yes, specifically you—what’s your role going forward? Is it just heading up Shippable? I mean, I’ve been in your role, to tell you the truth. I’ve had my company acquired more than once. And, while you still want to maintain your team and run the team, you also need to integrate now, right? You’re part of an executive team at JFrog that has—I will tell you, I think one of the best cultures in the DevOps world or in the tech world, period.

You know, how do you see yourself? How do you see this changing your role? It’s more than just having a new boss.

Cavale: I think the way—I’m actually in Israel right now, and we have already started collaborating. So, one of the things that I actually see is that the team is like, it’s all about brainstorming, quick integration. So, my role is to make sure that I’m delivering the products that JFrog’s customers love as well as most importantly the trust that it actually runs the way it needs to be running. And we’ve focused on the CI/CD aspect of it; nothing is changing from that perspective.

But what I have now is more brains who are kind of helping me think through what we need to actually do so that we are focused on the enterprise and driving towards that, so that’s kind of how I see myself, as part of the team.

Shimel: Excellent. So, Shlomi, let me ask you, now—what’s next? What worlds are left to conquer? What pieces of the puzzle are missing? What’s next?

Ben Haim: You know, Alan, I’m sitting here and listening to Ayalla and Avi and this is great. And from people from outside JFrog, they might think that JFrog is trying to complete the pipeline with all kinds of pieces.

But the one great feedback that I’m getting from the community is that there is no question mark. People understand what are we doing, it all makes sense. Every tier, every brick that we put on the DevOps wall sounds like the right, next logical step.

But we are the binaries people, and we will stay the binaries people. We will just make sure that this pipe is completed, and when we speak about the liquid software vision, when we speak about continuous update and solving the pain for the DevOps world, we are taking it seriously, and, as you see, we are super determined to complete our mission.

What next? We have to stay focused, we have to be a great solution to our customers and users as a community. We are very active in the community, providing them with more tools and more capabilities. It is now our responsibility to make sure that Shippable is fully integrated and the user experience is smooth and fun to use, and as Ayalla mentioned, thinking of the developer first and not the full platform story, but the user experience and the fun and simple-to-use solution.

Shimel: Excellent, excellent. Alright. Guys, we’re about out of time, I think. We’re probably over time. But again, Avi, congratulations. Welcome to the JFrog family.

Cavale: Thank you.

Shimel: It’s an amazing culture, and I’m sure you probably already know that, but you’ll continue to find out. Shlomi, my friend, congratulations to you. Keep up the great work. I can’t wait to see you.

Ben Haim: Alan, thank you.

Shimel: Thank you.

Ben Haim: Thank you so much, partner. Thank you so much.

Shimel: Alright. And Ayalla, always a pleasure.

Goldschmidt: [Cross talk]

Shimel: You, my friend, have work ahead of you now. [Laughter]

Goldschmidt: [Laughter] Yes, I do. The work is just beginning.

Shimel: Good for you.

Goldschmidt: Thanks for having us on.

Shimel: Alright. So, big news today—JFrog acquires Shippable, and another brick in the wall, as Shlomi mentioned. I tend to think of it as more of a Pink Floyd song than any political thing, but it certainly is continuing this DevOps maturation.

Guys, congratulations. We’ll see you all soon. And for now, that’s a wrap on this DevOps Chat. This is Alan Shimel for DevOps.com. Have a great day, everyone.

— Alan Shimel

February 21, 2019
DevOps Chat: Mainframe DevOps Update With Compuware’s Chris O’Malley

Christopher O’Malley is one of the leading voices in the DevOps for mainframe community. He is relentless in advocating how the mainframe platform can not only exist but also thrive in today’s Agile/DevOps environments. His evangelism keeps his company, Compuware, front and center in the mainframe DevOps discussion and it really lifts the entire market.

In this DevOps Chat, we catch up with Chris to talk about the latest release of Topaz, trends in the mainframe market and what we can expect in the near term.

Chris is always a great interview with lots to say and this interview is right on point. As usual, the streaming audio is immediately below, followed by the transcript of our conversation.

Transcript

Alan Shimel: Hey, everyone, it’s Alan Shimel, DevOps.com, and you’re listening to another DevOps Chat. This DevOps Chat features a return of our friend, Chris O’Malley, CEO of Compuware. Chris, happy 2019 and welcome back to DevOps Chat.

Chris O’Malley: Thank you and glad to be here, Alan.

Shimel: Absolutely. You know, Chris, it is. It’s 2019; DevOps continues to mature. And I think one of the trends that we’ve seen, certainly over the last year and I expect it to even accelerate this year, is changes are coming to the DevOps and mainframe market. Right? Mainframes aren’t going anywhere; in fact, shops that are using mainframes, I think their footprints are increasing, rather than decreasing.

And, from what we see in our readers and in talking to them, is that they are eagerly anticipating new developments, new functionality, that their mainframe software will offer, that will allow them to leverage DevOps types of foundations, DevOps types of functionality. And, I mean, DevOps and Agile and, really, code word: continuous delivery, all of these new ways that we’re defining the modern software factory, if you will. You’re in the driver’s seat for it, right, Chris? I mean, Compuware is kinda leading this charge. What do you see – you know, put on your analyst glasses here. What do you see in the market?

O’Malley: Well, first, just confirming what you said about the mainframe, is that the z14 has been awakening, I think, for customers and analysts, in terms of its success, so the z14 is the most successful mainframe model in over a decade. It’s had five consecutive quarters of year-over-year growth, which probably hasn’t happened in maybe forever. And, as you said, the reason that’s happening is the mainframe is still unmatched in terms of its reliability, its performance, its security, its transactional efficiency. And all of those things, all those virtues, are incredibly important in a digital age, where you try to basically create a customer experience through software. So the mainframe, as you say, it’s not just doing well; it’s actually doing better than historically because of the digital age.

Shimel: Mm-hmm.

O’Malley: So, just giving my disposition on the mainframe and the mainframe’s role within DevOps, I think, if you kinda zoom out a little bit to make sense of what’s happening to the mainframe, you gotta start first with the fact that all of these companies that use mainframe are of a mindset that “We’ve gotta be customer-centric. We’ve gotta serve our customers, whether they’re individuals, if they’re federal agencies, whether they’re citizens or it’s business-to-business.” But, for 100 years and for 100 years from today, companies that are the most customer-centric, obsessed with customers, do the best.

And, when you look at kinda this modern age, to do well from a customer-centric perspective, you’ve gotta create awesome digital experiences. And that’s done by writing software. So companies – you know, and Amazon has taken the lead in a lot of these kinda more newer-age companies. They’re incredibly good at taking ideas that matter to customers – and, again, it’s usually through digital means, increasingly – rapidly turning those into deliverables that make a difference. And, whether it’s centered on creating these awesome experiences and when they kinda look at the measures of what they’ve gotta do better to turn those ideas into deliverables, they’ve gotta be really good at velocity, how fast they do those things. They can’t do it in ways that compromise quality, so quality has gotta be something that’s constantly getting better and they’ve gotta become more efficient.

I mean, in the manufacturing space, they talk a lot about getting rid of non-value-added efforts and thinking about retooling things that are a cause for technical debt, that encumber your ability to increase velocity and improve quality, so you’ve gotta fixate on the efficiency side of it. So these three metrics are emerging as foundational to improving things like customer sat and improving financial performance.

So, when you start now getting closer to large enterprise IT, you’ve got a mainframe, these ideas that matter to customers don’t neatly fall either on the mainframe, in terms of the coding work that’s gotta be done or the cloud or some other platform, almost always, these ideas span these platforms. So there’s a necessity to improve velocity, quality, and efficiency across the entire enterprise.

So, you know, the mainframe, historically, has been this kind of silo-ed organization that had its own culture, its own set of processes, its own set of tooling. And, as you said, the mainframe’s not going away for all the right reasons. In terms of creating these awesome experiences, reliability matters. Performance matters. Security matters. How efficiently you do that, from a cost standpoint, matters. But you gotta draw the mainframe to be a like system and make it different only in syntax from these other platforms.

So there’s a strong move within these large enterprise ITs to break down this silo, get rid of a house divided against itself, bring the mainframe into a culture that’s centered on innovation, not on maintenance, bring it from waterfall to Agile, to bring it to a tooling set that’s familiar to non-mainframe platforms, the products like Jenkins and SonarSource and SonarLint or Paris. All these type tools that are used aggressively to improve velocity, quality, efficiencies on these other platforms, bring these to bear on the mainframe.

So that’s what’s causing, I think, the mainframe to be in this kind of disrupted state of trying to get the culture to move, getting people to think and behave differently, to embrace this kind of new order of how work is done, Agile techniques specifically, and to get away from – you know, the mainframe has always been selectively automated in incredible ways. I mean, the transaction volumes it can support are unmatched within the marketplace, but it’s selective automation, not intrinsic automation, which is a big part of what DevOps tries to push. And getting rid of non-value-added work on the mainframe and getting to the point that the value-added work can be done, again, faster, greater-quality, greater-efficiency.

So that dynamic is now gone from kind of innovators and early adopters within the mainframe market, you’re now getting into more of the early majority, so you’re seeing it more prod-based, in terms of people looking at the mainframe as that “We’ve gotta make this like any other platform.” And I think the success of the z14 makes sense, why it’s doing so well. Not just better than the z13, but better than previous releases over in the last 10, 20 years, so it’s exciting times in the mainframe. And I think its success is gonna be something you see emerging over the next ten years.

You’ll see fits and starts and, obviously, problems, but one of the things that’s making me so excited – you’re seeing successes in the market that are outrageous. We’ve got customers at Compuware that are increasing their development productivity by a factor of 4, I mean, a 400-percent increase in productivity, so they’re getting 4 times the number of story points completed within a release cycle. They’re taking testing time from weeks to minutes and with better quality. And, when that is understood by customers, you know, if you can go from weeks to minutes, that’s a lot of non-value-added work that you’re doing, right? That’s like burning money in the parking lot because a customer’s not gonna give you more money to do the same or less quality in what you can do in minutes, as opposed to somebody that’s a laggard doing it in two weeks.

Shimel: Absolutely.

O’Malley: So you’re seeing these amazing success stories. Again, it’s not just our tools; it’s _____ –

Shimel: No, no, it’s important – Chris, it’s that 10X, right? Everyone’s looking for that 10X. And we’re starting – you know, 4X is great; 5X is better. We’re getting those – we’re hearing success stories, the 10X success stories, with mainframe. And it goes back to something else you said, that, instead of people saying, “Oh, I’m winding down my mainframe or I’m migrating mainframe,” no, I’m looking at a ten-year roadmap on mainframes. It’s not – a couple years ago, there was this – I don’t know – fake media, fake news, about “Yeah, well, it’s here now, but we gotta kinda wind down.” What you’re saying about the z14 is exactly right; it’s bigger than anything that’s come before it, in essence, right?

Here’s another thing, though, Chris, and I’m interested in your take: kudos to the mainframe software market and vendors, who are recognizing – not all of ’em, not all of ’em, but the ones who are, the enlightened ones, who are recognizing this and are capitalizing on it by bringing the tools the market wants to bear. It’s more than just the hardware, right? The hardware’s phenomenal – I’m not gonna deny that; IBM has done amazing with the z14 – but the software vendors, including Compuware, right – take a bow – have really delivered. You guys have a new release coming out or just came out, is that right?

O’Malley: Right. It’s the 17th consecutive quarter that we’ve come out with new capabilities, upgrades to our classic offerings and integration with awesome and preferred DevOps tools, so, yeah, we’re very excited about this newest release.

Shimel: So talk to me. Give me the three biggest things that get you excited on this release, Chris.

O’Malley: So, each quarter, we’re turning the crank in terms of trying to nail the jobs that are changing, right? As I explained earlier, when you’re going from waterfall to Agile and from selectively automated to intrinsically automated, you’re trying to go from maintenance to innovation, the jobs that people have to do change radically. And they change radically in ways that are difficult to achieve. I mean, when you’re working on the mainframe, you’re working with large code bases, large in terms of the quantity of those code bases and also the size of any one of them.

So these techniques, at a high level, just seem obvious, but the practical things that need to be done from an innovation standpoint to help people to nail these things, it takes a lot of innovation. I mean, we’ve had to do a lot of work to get to the point where you’re seeing orders-of-magnitude improvement in the way people develop, the way people test, so this latest release is just another chapter in that story.

So we came out with Topaz team profiles, so that’s a good example of a capability that allows those that are working on systems to basically be able to store their configurations, the inputs and outputs of these applications, the test artifacts that are used, and build those artifacts and configurations in ways that can be shared broadly within teams or across the organization. That’s important for a couple things – one is the work to do that, you wanna do it once and then do it once for every one ’cause, in the DevOps mindset of continuous improvement, you wanna get rid of things that aren’t value-added, things that customers won’t pay for.

So, in creating this capability, we allow now, as teams get spun up, right, you get a team that’s gonna work on a two-week sprint, you want them, out of the gate, to be productive. So being able to share profiles day one, with people that may have never worked on these systems before, is crucial so that you’re productive in the first minute of the first day.

You also have situations where you’ve got, instead of technical debt issues, maybe a team, a large team, that you wanna go after certain work. It’s gonna help to improve the future throughput of the incremental efforts you have, from a development perspective, in the future. And you’re gonna bring large groups of people to work on something and that may be an outsourcer that you’re engaging; it may be new college recruits that you’re engaging. But you, again, want those people to be as productive as possible, as quickly, so, if you’ve got an experienced person on your staff that can create all those configurations and then share that broadly, again, you’re gonna lessen the ramp-up time for them to do their effective work.

So, again this is another example of us thinking about the mainframe going from waterfall to Agile, at scale, and having a scenario where it’s not just people that have been on your staff for 30 years, working on the same subsystems of your applications, every day for their entire career. Now, ’cause ideas are paramount and where people work has more to do not with what they know but what the company wants to do in better servicing their customers, so there’s more of a randomness to the work and a changing nature to the work, and what you work on is fluid. So we’re really thinking deeply about the implications of that and, for profiles within Topaz, that’s a major step in the direction of getting a different way of working within IT.

The second thing that we’ve released is another Jenkins plug-in. Jenkins is a central part of CI/CD pipelines, in terms of the automation side of it, and we’re constantly thinking about “What are the things or plug-ins that need to be created to make the mainframe equal to any other platform in all ways, other than syntax?” So JCL, Job Control Language, is a very important part of how programs are run within the mainframe. Creating a plug-in to help bring it into the CI/CD pipeline efforts and the automations efforts was a crucial thing that needed to be done, so it’s a contribution, from an artifacts standpoint, that, again, takes another step for our customers.

And then, lastly, is in the area of testing. Compuware is the leader, if not exclusive, provider of unit and functional testing as a combined offering. It’s a necessary part of the job that people have to do within Agile, with this concept of shifting left. The developers aren’t just coding; they’re coding and testing, both in terms of bugs and performance, and becoming much more extended in terms of their role. And so their accountability and responsibility is bigger.

So they need tools, like unit and functional testing tools, to participate within a two-week sprint process so that their deliverable, at the end of that cycle, is one that’s close to bug-free as possible and not putting future work into the system where it comes back as some quality issue 6 months later, 12 months later, when it runs into production, or, you know, as usability concern. So these three things are another turn of the crank, if you will, and Compuware’s offered to help customers nail these jobs, these new jobs that are created within kind of Agile work environments and DevOps work environments.

Shimel: Excellent. Anything else on the new release before I go off on another direction?

O’Malley: No, it’s just we’re very excited ’cause we released a technology called “zAdviser,” six months or so ago, which is a method of collecting what business KPIs for customers and then looking at, actually, stats relative to our products, so we can see kind of the behavior of developers. And we’re working with companies collaboratively to treat their people as they should, like high-performance athletes rather than slaves in a Roman ship galley. You can say, “Work harder. Work faster.” Well, you’ve gotta be constructive in helping to share best practices and methods that have a positive effect in things like velocity, quality, and efficiency.

So, with every release, we’re seeing a significant increase in this KPI performance, so, as we’re talking about earlier, the mainframe is certainly on a march to become a first-class citizen within the digital age, within these enterprise customers, but the more _____ –

[crosstalk]

Shimel: I’m sorry; go ahead, Chris. Go ahead.

O’Malley: Go ahead.

Shimel: Well, no, I was gonna say, you know what I find interesting is, yeah, we’re talking about the mainframe machines, but so much of your focus is on the people, on the developers, on the team, because, at the end of the day, that’s the gold. Right? That’s the almost irreplaceable asset, are the people and the developers and the IT teams that are working on these mainframes. And it’s about empowering them. Right? So I think that it’s the right place to put the focus.

O’Malley: Yeah, people –

Shimel: Yeah, I –

O’Malley: Just to restate the obvious here, but people matter most. When you think about the mainframe in the context of Agile and DevOps, asking people to change their role, though they’ve been around for 30 years, 40 years, dramatically – I mean, the day in the life in Agile and DevOps is vastly different than the day in the life of status quo. And it’s important that, as leaders, you’re promoting the appropriate culture and getting people to understand why they need to change, important that you’re giving ’em the necessary education, teaching them how to be effective in Agile sprints. I mean, that’s a different way of working. I mean, it’s not that you’re working 24 hours a day, but you’re definitely working in a more intense way. And then products matter then. The tools have gotta be enablers, not things that fight against your efforts to adopt this new culture and Agile nature.

And, you know, I really try to express to IT leaders, “You gotta think about these things ’cause – ” you know, we use this example, the Gallup survey. Gallup survey have been doing this thing for 20, 30 years of employee engagement in large enterprises, and what’s most interesting about it is, over 20, 30 years, the employee engagement statistics haven’t really changed. And it’s disturbing when you actually hear about them. Two-thirds of employees in most large enterprises are not engaged, half of them are kinda, “It’s a job; I don’t care that much,” and 16, 17 percent are actively disengaged – they try to sabotage the efforts of the companies.

So one of the biggest problems is engagement and employee engagement, and so I try to get people to understand – people matter hugely. And you’ve gotta give ’em the culture necessary to thrive in this more competitive age, you’ve gotta give ’em direction in terms of the process, the way that work is done, but you gotta give ’em the tools. You know, asking ’em to use mainframe tools out of a NASA movie from 1965 and thinking that something’s gonna happen is incredibly naive, so –

Shimel: Absolutely. You’re not landing people on the dark side of the moon with that. Chris, we’ve got a few minutes left. I wanted to touch on another subject – we didn’t really kind of pre-talk about it, but I’m sure you’ve got some great views on it – I was out in Seattle last month for KubeCon, you know, Kubernetes, cloud-native. It’s a funny question to ask the mainframe guy the cloud-native question, but, certainly, containers are no strangers to mainframes.

O’Malley: I mean, they’ve been on the mainframe for –

Shimel: 20 years, maybe more, right? 25. So anything specific to Kubernetes in the Topaz or Compuware kinda roadmap that you can discuss right now? Or what effect do you see on your world?

O’Malley: Yeah, so, when you think about kind of the flow of work in large-enterprise IT and what developers are fixated on, I mean, part of it is innovation and functionality that they’re trying to build, which is most important. I mean, that’s the value-added efforts of developers that make a difference in the eyes of the customer. And you wanna do that as quickly as you can, with the greatest degree of quality.

But the other things you gotta think about are things like quality itself. I mean, if you get anything that you release that’s got bugs in the system, how rapidly you can respond to it. That’s always part of the work that’s being done and I don’t know of any organization that doesn’t have some degree of bugs. And bugs may be software written improperly – and, again, our tools are helping to fix that – or usability issues, integration into other systems.

There’s also the work of compliance and security that you gotta do, but this last thing is this thing, technical debt, right? That any code base that’s had any life to it, has meandered through the efforts to serve customers, it’s something that grows over time. And it has nothing to do with mainframe; it happens everywhere. You know, the cloud – you know, Amazon’s got debt. And you gotta deal with that technical debt, so, as you’re dealing with technical debt, the smart approach is to start using more of these modern techniques. SOA was kind of a philosophical approach to breaking down these larger systems into smaller subsystems. When I think about API enabling, REST Y enabling these kind of smaller chunks, but you also wanna think about containerizing, using these as a form of containers, Docker being the method of creating the container, Kubernetes of being the way of managing these.

So we do see this playing an increasing role _____, as people start really looking at affecting technical debt ’cause that gives you the biggest bang for the buck. I mean, you certainly wanna do it with any new functionality you’re doing, but you’ve got some huge numbers of lines of code that you’ve developed over four years that works awesome. I mean, it’s really good working code, but making it more manageable and getting rid of that technical debt. Part of that equation is RESTful APIs; it’s containerizing it and managing it more in a cloud-like fashion. So, yeah, you’ll see us enabling those tools.

Our approach has always been to go to the edge of the mainframe and then we stop. We take care of what we know best and getting rid of these esoteric differences, but then, when we go off the edge of the mainframe, we work with all these other vendors. We allow SonarLint and SonarQube to do its work on COBOL, as akin to what it does with Java or C. Parasoft is a testing framework, allowing our products to work within that orchestration.

So there’s all these tools that work in kind of the DevOps world that are preferred and we’re enabling them, so you should think of us doing things in support of Docker and Kubernetes over the long term. But, yeah, it’s an important part of making mainframe IP one that the throughput can constantly be improved and those techniques obviously help enormously.

Shimel: Absolutely. Chris, I’m sorry to get you going ’cause it doesn’t take much when we start talking mainframes and DevOps, but we’re way over time and I need to wrap up. Hey, we’ll have you on, certainly, next quarter for the – what will it be? – the 45th time?

O’Malley: [Laughs] We’ll be doing this and it’ll be our 1,000th. You and I will be in wheelchairs, talking about it.

Shimel: By then, you should be down in Florida with me, but, anyway, Chris, it’s a pleasure having you on. Continued success with Topaz and Compuware. And we’ll catch up with you soon?

O’Malley: Always a pleasure and anytime I can help.

Shimel: All right.

O’Malley: And thank you for the hard work that you do.

Shimel: Thank you.

O’Malley: _____ important _____ –

Shimel: All righty. Chris O’Malley, CEO, visionary, leader at Compuware, on DevOps Chat with me today. This is Alan Shimel and you’ve just listened to another chat.

— Alan Shimel

February 21, 2019
Will ITIL V4 Bring Order to a DevOps World?

It is difficult to dispute the importance that the ITIL (Information Technology Infrastructure Library) approach has had on the world of ITSM (IT service management). After all, the best practices established by ITIL V3 have come to be the accepted way ITSM providers conduct business—so much so that many other IT processes have come to look to the ITIL framework to bring order and control to complex processes.

Yet, ideologies such as DevOps have struggled to adapt to the somewhat rigid doctrine of ITIL V3, instead favoring Agile methodologies to meet the rapid deployment of updates and the iterative cycle of software development. Perhaps, the Agile-versus-ITIL differences can be attributed to how life cycles should be managed and how the management of those life cycles impact both development and operations in the IT space. Nevertheless, change is coming to the world of ITIL, which may help to make the framework more suitable for DevOps practices, and perhaps make ITIL and Agile symbiotic in the field.

The ITIL V4 framework is scheduled to be released in Q1 2019, some two years after it was announced by AXELOS at the Fusion 2017 conference. “ITIL is the most firmly entrenched ITSM framework in the infrastructure of large, complex organizations, with many—if not most—Fortune 500 companies using it,” said Cathy Kirch, president of itSMF USA, the organizer of FUSION 2017 and the leading independent professional forum for IT Service Management professionals in the United States. “Currently, ITIL and complementary practices, such as DevOps, Agile and Lean, are used successfully in conjunction with each other; however the next evolution of ITIL will promote and underpin dynamic collaboration.”

The More Things Change, the More They Stay the Same

ITIL V4 doesn’t propose to completely rewrite the ITIL V3 narrative; in fact, the core elements are expected to remain the same. ITIL V4 will simply include new material on integration with additional best practices aimed at mitigating some of the biggest criticisms around ITIL V3, the less-than-ideal relationship with the ITSM community and failure to keep up with recent trends in software development and IT operations.

In essence, ITIL V4 will address those criticisms by shifting V4 to a community-driven initiative and integrating other practices such as Agile, DevOps and Lean into traditional ITIL best practices. That means ITIL may become complimentary to those other frameworks and begin to embrace other services management best practices in the context of new technologies, such as serverless, containers, microservices and multi-cloud.

With the DevOps community picking up steam, the Agile Manifesto and Lean development displacing ITIL V3 in some practices, it will be interesting to see what impact ITIL V4 will have on the IT industry as a whole. While it is far from ITIL’s last gasp, ITIL V4 will face significant challenges in a DevOps world that is constantly changing. However, with more than a million ITIL V3 certificate holders worldwide, ITIL V4 will definitely garner a great deal of interest when it hits the streets on Feb. 28.

— Frank Ohlhorst

February 20, 2019
DevOps Chat: Serverless and Abstraction with Nimbella’s Anshu Agarwal

Anshu Agarwal has a tremendous record of working in startups that had successful exits. Now for the first time, along with her co-founders, she is actually founding the startup herself. Nimbella is a serverless platform that is cloud-agnostic and makes developing and deploying applications easy. Whether in a public cloud, private data center or both, Nimbella can help.

In this DevOps Chat, we speak with Anshu about her own personal journey and where Nimbella is and what the plans are for going forward.

As usual, the streaming audio is immediately below, followed by the transcript of our conversation.

Transcript

Alan Shimel: Hello, everyone, it’s Alan Shimel of DevOps.com and you’re listening to another DevOps Chat. Today’s chat actually features a company that’s just starting out here, but it’s a guest who’s had a rather long career in the tech space. As long as mine, even, at some point, but I’m sure I’m much older. Anyway, I’d like to introduce you to Anshu Agarwal. And, Anshu, if I messed up your last name, I apologize. Was that good?

Anshu Agarwal: Thank you, Alan. You said it perfectly right. Thank you for having me today.

Shimel: Thank you. So, Anshu, let’s start off – I referenced that you’d been in tech a while. Let’s give people an idea of your personal journey that brings you here today, even going back to Speedera and beyond, if you’d like, but if you wanna start with Speedera, that’s fine. But let’s give folks an idea of who you are, what your background is.

Agarwal: Sure. Thank you. So I am an engineer by background and I moved into product management and marketing after I finished my MBA. And, after that, I joined a startup and then the startup bug never left me. So I joined Speedera, which got acquired by Akamai. Then I joined another startup, Ankeena, which got acquired by Juniper. Another startup, ConteXtream, got acquired by HP. And my most recent startup, Cedexis got acquired by Citrix, so I have been acquired four times. And, in all of those companies, I kind of delivered the success for other people’s ideas because they were all successful startups, got acquired by very, very large companies, so they were indeed – from value standards, they’re considered successful, but, for myself, I had never really done a company of my own.

So, when I got the opportunity of just starting and being a founder of a cloud infrastructure, because that’s my background – I’ve been doing cloud infrastructure for last 15 to 20 years – there was no question that I wouldn’t do it. And we got started on this company – the company name is “Nimbella.” And it is in the serverless framework or serverless platform space. And we are kind of disrupting what serverless is, so – or we are providing what serverless promises to the developers. So that’s what we are working on now.

Shimel: Excellent. You know, you’re way too modest and humble because, for many people listening, whether they’re in the valley or Boulder or Austin or New York or Boston or London or Bangalore or anywhere else in this world, having four startups that were acquired by companies like Akamai, HP, Citrix, and I forgot the fourth one – I apologize – a lot of people would call that a career. Right? I mean, whether they founded the companies or not, just being associated with four companies that had successful exits to large companies like that is something to be proud of and it speaks volumes as to either you’re really, really lucky or you’re doing something right. Right? So congratulations to you for that.

So you’ve mentioned Nimbella and I wanna kind of share with our audience up front that Nimbella is – you have raised some funds for the company and there’s a great startup – there’s a great founders group involved. You mentioned, I think, one of your cohorts from Speedera; I think there’s two other founders in the group. But the company’s sorta still – I don’t wanna say stealth, but pre-product, let’s call it. Pre-public-product. So I don’t wanna put you on the spot. I’d rather talk, I think, if it’s okay, Anshu, about some of the problems you see and potential solutions that we see.

You know, we live in a great time for developers and engineers. Internet time, right? We’ve gone from servers to hypervisors to containers. Now serverless. And serverless seems to be all the rage, but, like any new technology, it’s not birthed mature, right? It doesn’t come out mature; it doesn’t start mature. And so there’s gonna be growing pains. Can you share with our audience a little bit about what you see as some of the growing pains around serverless or what you’re hearing from people around that?

Agarwal: Absolutely. Well, thank you for – this is a great question because it is definitely the best thing that’s happening for developers. The reason I say that is because what developers really want is to develop their business logic or develop the code and not worry about the infrastructure. And serverless promise is all about that, that you don’t have to worry about the infrastructure. You don’t ever have to worry about how it scales, how it is secured. Anything that requires the knowledge of infrastructure should not be a developer’s worry.

But, in today’s serverless frameworks – and there are a few serverless frameworks and some are very popular ones – there is so much complexity still for the developers because they have to combine many services. And, for instance, they have to think about storage, they have to think about identity management, and they have to learn all these services and integrate. Not only that, the management and debugging is still an issue. And these are growing pains, as you mentioned. The serverless computing is still in a nascent stage and, therefore, it’s lacking all the abstractions that are needed for the adoption to accelerate into new markets and application domains.

Not only that, serverless today, if you look at it, is used only for stateless applications, so you’ll see it in IoT applications, you’ll see it in reactive kind of applications only. It is not suited for long-running or stateful workloads. The reason being is because serverless today is serverless compute; it is not serverless storage. So that’s another problem. We may consider it as a problem or we may consider it as the limited adoption of the technology, but that’s something we have to address if we want this to be adopted in general, across all workloads. Then the –

Shimel: I’m sorry. We may also consider it as an opportunity, right? I mean –

Agarwal: Oh, absolutely.

Shimel: Yeah.

Agarwal: Absolutely. And that’s what we’re doing. And I’m gonna talk about Nimbella and how Nimbella is addressing it.

Shimel: Mm-hmm.

Agarwal: Another important aspect of serverless today is, like any new technology environment, there is serious vendor lock-in because, if you are using AWS’s Lambda serverless framework, you are programming to the nuts and bolts of Lambda. If you are using Google, you are programming to GCP, and so it is with Microsoft Azure. So, in a sense, you are programming to proprietary frameworks and, therefore, your code is not cloud-portable.

So, tomorrow, if your company decides that, “Now we are going to move from AWS to Google,” you gotta rewrite the whole thing again. So that is actually, again, the serverless promise, and, therefore, vendor lock-in is very real, especially for the companies, actually, that choose not to go to public cloud because they’re part of regulated industry or whatever _____. They want something to be running in their own private cloud; these serverless frameworks are not an option for them. And then, say, you were able to write code for all different frameworks, but then the complexity of operations is there because you can’t monitor and observe at each level since these are proprietary frameworks across these clouds.

So there are serious issues for widespread adoption and that’s where we come in. That’s the opportunity we looked at when we decided to create this company, is we want to create a platform, a framework, that is providing all the necessary abstractions so the developer is not to worry about how the infrastructure works, how to scale the code, how to secure the code. The code is just nebulous logic that they write. Then it is also suited for long-running, stateful, streaming workloads, such as AI/ML workloads. I mean, we have a greater adoption of AI happening in every industry. If we are not going to be able to handle AI workloads, we are missing a big market.

And those are real, real problems and we are addressing it with Nimbella, you know, the technical problems. Then there are operational issues where you want cloud portability, so we are also addressing that part because we are providing a layer of abstraction and it’s open-core-based, so a developer can program the way they program the business logic and not worry about which cloud they’re writing.

Shimel: Got it. Got it, got it, got it. So, again, I don’t wanna be uncomfortable or make you uncomfortable with what we’re gonna talk about, but, you know, I was at KubeCon out in Seattle last month – seems like a year ago, but it was only last month – and, surprisingly or not surprisingly, “serverless” seemed to be on everyone’s lips. Right? And maybe some people listening to this are saying, “Wait a second, Kubernetes is for containers and containers can run on bare metal, yeah, but they run on VMware or hypervisor as well.” What do you think the role of Kubernetes is in the serverless future? Fair?

Agarwal: Kubernetes is a great technology. I really think it has great future. And serverless frameworks use Kubernetes or something else like that. And we were also at KubeCon. It was actually our first conference as an Nimbella person attending – one of my cofounders, who actually is the creator of OpenWhisk Apache, a serverless framework, he attended it. And he mentioned the same thing that you mentioned, that serverless was on everybody’s slide deck. Okay? It is something everybody is looking at; it is something everybody is kind of they have it on their roadmap, that they wanna work on it. They wanna look at it and see if it works for their environment.

And Kubernetes is perfect, actually. It provides the platform needed because it is widely accepted these days and it provides the necessary environment, but, again, it should not be the burden of a developer to know how to run, work with Kubernetes. It should be the infrastructure serverless framework that provides that to the developer and the developer’s still only programming the logic. It’s not for the developer to worry on what environment, what container environment they are working on and what VMs are being used and what bare metal is being used. That’s all abstracted away for the developer.

Shimel: Got it. Got it, got it, got it. So, you know, I’m sure there are people listening to us today who are thinking to themselves, “This woman has been in four startups and has had tremendous success.” And now I’m sure you’re a CEO, right, of Nimbella and you’re cofounder –

Agarwal: That is correct.

Shimel: Right? And I’m hoping there’s a whole bunch of women engineers out here or women entrepreneurs who are listening to you, saying, “Hey, if she can do it, I can do it.” And we don’t have a lot of time left, but I wanna address those folks there, that are near and dear to me for a lot of reasons. What’s your advice to them?

Agarwal: Absolutely. If I can do it, they can do it. That is a very true statement. And, I mean, you said some great words about me, but I consider myself average successful person and, yes, I have been lucky that the four companies I’ve worked for had great exits. But being a founder and CEO is still very different from what I did for the other companies before. So, basically, when you are part of the company, other companies, you’re not bothered by operational aspects of the company. You’re not bothered by hiring for every possible function in the company. So the skill sets that I’m developing now, there is no other experience that could have taught me that, so, even though – I say, “Everybody can do it and anybody can do it,” but they still have to learn the art of running a company on the job. It just – there is no course. There is no education that can be given because you have to experience it.

So, to all the women engineers that are listening, I would say, “If I can do it, you can do it. You just have to just get into it. You just have to dive into it and it will come to you. You will develop your advisors. You will develop the people who help you on the way because you won’t have all the skill sets, but you will know who to go to for what skill set, as you take that role.”

Shimel: Absolutely. Absolutely. And, you know, women or not women, entrepreneurs – I’ve also founded and helped start my fair share of companies – I don’t know if it’s courage or just pure moxie, if you will, right? You just put one foot in front of the other, right, and you just keep moving forward. And you can’t boil the ocean, right, but you can – these things are incremental, right, and they’re evolutionary. They’re not _____ –

Agarwal: Absolutely. You’re absolutely right. These are such an incremental thing and you may be not finding the best solution, but you are optimizing your solution, at every step, because you’re trying to handle so many things and, if you wait for the best solution to arrive, it may be too late. So you’re kind of optimizing the solution for whatever issue or whatever problem you’re handling, and you move on with that. And it’s just only time will tell how successful I am in the role, but this is my experience six months into this company, six or seven months into this company, and I’m still learning the art.

Shimel: Absolutely, it is. So what do you – short-term future, where can we find out more on Nimbella or when will we find out more, do you think?

Agarwal: Oh, absolutely. So there is some information on the website – very little information, I would say – but it gives you a flavor of what we are doing. Then, in Q1, this quarter, we are going to be releasing our developer playground, which is actually – because the buyer of serverless technology is the developer and developer experience is the most important thing, a developer has to feel one with the programming experience. And that’s the true serverless promise and we wanna deliver on that promise, so, in Q1, we are going to release that.

And, in later quarter, in Q2, since I have mentioned private cloud and hybrid cloud, there will be offering for private cloud in Q2. So, as this quarter progresses, we are going to populate the website more with the content and we’re going to host more meetings like this and make it available to the developer community in general, our solution, and then, later on, for enterprise sites.

Shimel: Great. Great, great, great. Anshu, when I started, I said, “These are only about 15 minutes. It’s gonna go so quick, you won’t believe it.” If I told you we’re closer to 20 minutes already, you’d probably say, “Oh, my,” but we are and we’re gonna need to call an end to this DevOps Chat. You know what? It sounds like there’s a lot going on here and, if we can have you back on, you know –

Agarwal: Oh, that would be great, Alan. I really hope we can –

Shimel: Yeah?

Agarwal: I really hope we can get back when we have our developer playground because I’d love to talk more about it.

Shimel: I promise you we will.

Agarwal: Thank you.

Shimel: Promise you we will. Thank you. Anshu Agarwal, founder/CEO of Nimbella, our guest here on this DevOps Chat. This is Alan Shimel and you’ve just listened to another chat.

— Alan Shimel

February 18, 2019
Automation: We Have Not Yet Achieved Balance

I’m going to start this week’s blog out with a very clear statement that regular readers will already be aware of: I fall on the automation side of the DevOps equation. I see DevOps as automation first, process second. The problem is getting to that “second” part. The act of automating the entire pipeline should, by example, show the benefits of integrating these disparate tools and disparate teams. But it seems it often does not.

Simple Graphic for Discussion

So I’m going to take a second to tell you why it should. If you have integrated tools and teams, kudos! Hopefully you get absolutely nothing out of this blog, and you are enjoying a drink on a beach somewhere while your entire operation is taking care of itself. Everyone else, please read on.

Look at the generic graphic. Notice how each step on the top is a different shape? That’s because they are individually implemented with limited integration of both people and processes. So you have a tool to submit for testing … Does testing (or monitoring at the end) feed back into your bug tracking system? Does it create tickets to get stuff done? When a container fails, is it simply dropped and a new one spun up to replace it because “they’re just cattle”? If so, does the system take steps to keep the failed container—or at least information about it—alive so troubleshooting can occur? Ignoring failed instances that were replaced and their impact on users was low is exactly how CloudBleed happened a couple years ago.

Automate first, get the benefits of automation and drive faster delivery. Then take advantage of all that automation and make it into a streamlined process. It’s a massively complex system, even if you deliver everything on a single “platform,” so it will never be perfect. But issues that occur with regularity can be automated also to provide integration between systems. Isolating an instance and sending a notice to the defect tracking system that it is alive but not communicating (because it has been replaced) is something that is relatively easy to script/integrate and offers the ability to find what actually went wrong. Contrasted with just dropping the instance and starting a new one, this process is an order of magnitude better, and with integrated teams, Dev and Ops can work together to resolve whatever caused the instance to go down.

Strong integration and a smooth pipeline is what technologies such as application release automation are designed to deal with, but most shops see these tools as “unnecessary” overhead. I disagree. Find one you like—there are a million of them—and use it to connect things. It’s just one step, but it is a step to transforming your operational delivery systems from unique snowflakes to part of a collection that works together.

That’s it. Automate first because the gains are phenomenal, but don’t stop there. Really—don’t. You are doing yourself a disservice if you do. The whole “We’re going to change your world!” crowd is annoying, because changing your world is your decision, but listen to them anyway, and see which of their ideas fit your organization. You’ll be surprised once automation is solidly in place how much of what the process side of DevOps offers makes sense.

— Don Macvittie

February 18, 2019