Category: Features

  • DevOps Chat: JFrog Acquires Shippable

    DevOps Chat: JFrog Acquires Shippable

    The big news in DevOps today was the acquisition of Shippable by JFrog. This adds a CI/CD solution to the JFrog Enterprise+ platform and further expands JFrog’s end-to-end DevOps offering beyond Artifactory and more.

    In this DevOps Chat, we speak with Shippable CEO Avi Cavale, JFrog CEO Shlomi Ben Haim and JFrog VP of Marketing Ayally Goldschmidt about the thinking behind this acquisition and JFrog’s plans going forward.

    As usual, the streaming audio is immediately below, followed by the transcript of our conversation.

    Transcript

    Alan Shimel: Hey, everyone, it’s Alan Shimel, DevOps.com, you’re listening to another DevOps Chat. Today’s DevOps Chat has some breaking news. It was just announced this morning—JFrog has acquired or merged with Shippable in a rather, you know, this is big news within the DevOps world, and we’re very, very happy to be joined by the CEOs of both Shippable and JFrog. From Shippable, we have Avi Cavale, and from JFrog, of course, our friend, Shlomi Ben Haim. And joining us from JFrog is Ayalla Goldschmidt, who really runs marketing at JFrog. I’m not sure of your exact title, Yes, Ayalla?

    Ayalla Goldschmidt: VP of Marketing, thank you, Alan.

    Shimel: VP of Marketing—thank you for joining us. So, first of all, to all three of you, congratulations, Mazel Tov as they say in Seattle. [Laughter]

    Goldschmidt: Thank you, thank you.

    Shlomi Ben Haim: [Laughter] Thank you, Alan.

    Shimel: Thank you. And so, you know, Avi, let’s turn it over to you first. I think a lot of our audience is familiar with JFrog, and most of our audience have heard of Shippable, but let’s just lay that right out there. What’s Shippable about?

    Avi Cavale: Shippable is about a modern CI and CD platform. So, our focus is to streamline the delivery—packaging and delivery of software, and that’s kind of where we are focused on.

    Shimel: Fantastic. So, Shlomi, I think I already know the answer, but I’m gonna ask you—why Shippable for JFrog?

    Ben Haim: That’s a great opening question. Actually, Alan, you know JFrog very well. You know that everything starts with the culture, and when we met Avi and the team, it looked like we’d found the brother Frogs in a different company. So, for us, it was just to make sure that the technology and the strategy are aligned. And when we looked at the technology and the next generation of CI/CD that was built by the Shippable team with Avi’s leadership, it was a no-brainer for us.

    Shimel: Absolutely. But the other thing I think we should mention is, you know, JFrog, many people know JFrog from Artifactory, maybe Xray, you know, point solutions that we can call up. Artifactory is a whole platform in and of itself.

    But, really, the big move for JFrog—and if I’m wrong, correct me; Ayalla, you’re the Marketing person—the big move for JFrog was, you know, this JFrog enterprise platform. And, to me, if you’re gonna have an enterprise platform in the DevOps space, you need that CI/CD pipeline kinda delivery capability.

    So, to me, you know, this really kinda helps fulfill that vision of the JFrog platform. Yeah, no, wrong?

    Goldschmidt: Yeah, that is exactly right, Alan. That’s the vision that we have for it. Obviously, we’ll be doing a lot of work here in the coming weeks to clarify the roadmap and the timeline, but you hit the nail on the head. That’s the value that we’re seeing for our customers and what they’re looking to, what they’re asking us for as far as, you know, we love JFrog, we have very happy customers, they love what we do, and they’re looking to take it to the next level.

    Shimel: Excellent. So, let me ask my next question to Avi and Shlomi, and that is, you know, I guess it’s part of the maturation of the DevOps market where all—you know, there are now, everybody wants to offer the whole enchilada. Do you know what I mean?

    Ben Haim: [Laughter]

    Shimel: People aren’t content any more just offering a piece of it, we wanna offer it all, that whole SDLC, we’re gonna help you throughout that life cycle. It seems everyone is pointing towards that, and the money—look, JFrog, it’s no secret, raised $165 million. Some of your competitors have raised also eight figures. That’s where the smart money’s going, towards companies that are building out this end-to-end DevOps framework.

    Avi, is that what you saw at Shippable, and is that what attracted you guys to want to do this deal with JFrog?

    Cavale: Yeah, I think what we are seeing is that there is one thing which is very critical for customers. It’s like—yes, they want the whole enchilada as a single platform, but at the same point in time, they also want a platform that is open, and they want to make sure that they can plug in the tools that they like in certain places.

    So, of course, they want to see the native capability end to end, but extensibility is also super important. And so, what we kind of saw with what JFrog was doing and what we were doing also was that we were being open to everything. I mean, yes, we want to have a standard of how we think about an end to end platform, but we want to be open and make sure we embrace all the practices that developers want to do and extend the platform to DevOps capabilities.

    Shimel: Absolutely. So, Shlomi, Avi makes a very good point here, and that also goes to the heart of JFrog. One of the things about JFrog is that they play nice with everyone, basically, right? You could pick whatever CI/CD platform you wanna work with or whatever—you know, whether it’s Puppet or Chef or the Red Hat software, you know, JFrog works with everyone, so to speak.

    But now, you have an in-house CI/CD solution, so how do you balance that in your go to market?

    Ben Haim: That’s a great question, Alan. I think that what we’ve started to see in the market for more than 12 months now is that DevOps is maturing and the organization and enterprises are looking to get an end to end solution, yet the driving force of DevOps production is still in the hands of the developers, which, obviously, would love to see the freedom of choice and the best of breed options kind of open to them.

    So, you are right. Our universal approach is something that always led us. We always provided our users with the power of choosing their tools pack, and it will still be the case.

    However, on the JFrog platform, what we call the enterprise class, we want to make sure that we take artifacts seriously, from the moment you build it from code all the way to production. And, together with Shippable automation power, together with the integration and the visibility that technology like Shippable can provide to the platform, I think we will be able to combine the two, to allow a better universal approach. So, you can still use your Jenkins, your CircleCI, your other DevOps tool sets. But you will also have a JFrog end to end solution that takes you all the way from code to production.

    Shimel: Excellent. So, Avi, you know, how do you—I’ll reverse, mirror that question back to you, right? How do you see the Shippable team, the Shippable product fitting into that JFrog Enterprise Plus, you know, the Artifactory platform, all of the different tools that JFrog and services that JFrog is putting together—how do you make sure you fit in there?

    Cavale: So, as Shlomi mentioned, right, what is happening is—so, the artifact is something that is immutable, and that’s something that you build and you want to kind of take it through your entire value chain of how you deliver software to your end customers or end devices or whatever they might be.

    The issue is that, if you don’t have the pipes to kind of control that in a high fidelity fashion, you lose a lot of information as you go from stage to stage. Because you’re leaving it up to the customer to make sure all the data is being persisted and you’re keeping all the hooks so that eventually what you’re really trying to do is, you want to be able to go to something that’s running on the end device and be able to track, if there are problems, what are all the pieces that went into actually building that piece of software that is running there?

    So, losing that ability to connect that entire life cycle, all the way from code to the point where it’s running on a device is a very critical piece of DevOps. And what we see is, that kind of, now, our marriage to the whole Artifactory plus Xray and plus distribution, all of those pieces now can be connected through a common pipeline and we’re making sure that we have high fidelity information exchange as things move through this work flow end to end. And I think that’s kind of the critical thing that we are adding to the product.

    Shimel: Excellent. Ayalla, a question for you—you look at, all of a sudden, you’re embarrassed by the riches of JFrog in terms of their product, right, in terms of the product that you have to work with, the products that you can bring to market. But at the same time, at some level, it’s a double-edged sword from a marketing perspective, right? You have so much now, too, that you need to market, that you need—how do you make sort of a holistic message for market that includes all of these pieces and how they fit together and how, if you don’t want to use all the pieces, you don’t have to, either?

    Goldschmidt: You know what, I feel like the luckiest marketing person on the planet. And, you know, Shlomi promised me when I joined this company a little over a year ago that there will be no shortage of news, and he has delivered.

    Shimel: [Laughter]

    Goldschmidt: So, since I’ve joined, the acquisitions, our Series D announcement, and of course, all of the new technologies that we’ve supported along the way, and the most recent exciting edition of support for Go and Go Center, we love all of these audiences and we do actually put in a big effort in tailoring our message to make sure that we’re communicating with a developer first mindset, and that seems to be working for us.

    So, you know, until today at JFrog, everything we’ve built has been through an inbound marketing funnel with an inside sales team. We haven’t had field people knocking on doors, and that seems to be working great for us, and now we’re ready to take it to the next level. But we’ll always remain developer first.

    Shimel: Excellent. Let me—Avi, let me ask you some sort of typical M&A kind of questions. So, is your whole team staying on as part of this acquisition?

    Cavale: Absolutely, and we’re all joining the JFrog family, and so there is really no change to the business and how we are running it. And I usually tell my team that I have a new boss, that’s all—nothing else is changing, and we’re gonna keep marching towards the vision, and all we have is an opportunity to broaden that vision that JFrog brings to the table and we are part of a much bigger family than what we were before.

    Shimel: Excellent. And so, Avi, what do you see particularly as your role going forward, here?

    Cavale: Sorry, was that for me?

    Shimel: Yes, specifically you—what’s your role going forward? Is it just heading up Shippable? I mean, I’ve been in your role, to tell you the truth. I’ve had my company acquired more than once. And, while you still want to maintain your team and run the team, you also need to integrate now, right? You’re part of an executive team at JFrog that has—I will tell you, I think one of the best cultures in the DevOps world or in the tech world, period.

    You know, how do you see yourself? How do you see this changing your role? It’s more than just having a new boss.

    Cavale: I think the way—I’m actually in Israel right now, and we have already started collaborating. So, one of the things that I actually see is that the team is like, it’s all about brainstorming, quick integration. So, my role is to make sure that I’m delivering the products that JFrog’s customers love as well as most importantly the trust that it actually runs the way it needs to be running. And we’ve focused on the CI/CD aspect of it; nothing is changing from that perspective.

    But what I have now is more brains who are kind of helping me think through what we need to actually do so that we are focused on the enterprise and driving towards that, so that’s kind of how I see myself, as part of the team.

    Shimel: Excellent. So, Shlomi, let me ask you, now—what’s next? What worlds are left to conquer? What pieces of the puzzle are missing? What’s next?

    Ben Haim: You know, Alan, I’m sitting here and listening to Ayalla and Avi and this is great. And from people from outside JFrog, they might think that JFrog is trying to complete the pipeline with all kinds of pieces.

    But the one great feedback that I’m getting from the community is that there is no question mark. People understand what are we doing, it all makes sense. Every tier, every brick that we put on the DevOps wall sounds like the right, next logical step.

    But we are the binaries people, and we will stay the binaries people. We will just make sure that this pipe is completed, and when we speak about the liquid software vision, when we speak about continuous update and solving the pain for the DevOps world, we are taking it seriously, and, as you see, we are super determined to complete our mission.

    What next? We have to stay focused, we have to be a great solution to our customers and users as a community. We are very active in the community, providing them with more tools and more capabilities. It is now our responsibility to make sure that Shippable is fully integrated and the user experience is smooth and fun to use, and as Ayalla mentioned, thinking of the developer first and not the full platform story, but the user experience and the fun and simple-to-use solution.

    Shimel: Excellent, excellent. Alright. Guys, we’re about out of time, I think. We’re probably over time. But again, Avi, congratulations. Welcome to the JFrog family.

    Cavale: Thank you.

    Shimel: It’s an amazing culture, and I’m sure you probably already know that, but you’ll continue to find out. Shlomi, my friend, congratulations to you. Keep up the great work. I can’t wait to see you.

    Ben Haim: Alan, thank you.

    Shimel: Thank you.

    Ben Haim: Thank you so much, partner. Thank you so much.

    Shimel: Alright. And Ayalla, always a pleasure.

    Goldschmidt: [Cross talk]

    Shimel: You, my friend, have work ahead of you now. [Laughter]

    Goldschmidt: [Laughter] Yes, I do. The work is just beginning.

    Shimel: Good for you.

    Goldschmidt: Thanks for having us on.

    Shimel: Alright. So, big news today—JFrog acquires Shippable, and another brick in the wall, as Shlomi mentioned. I tend to think of it as more of a Pink Floyd song than any political thing, but it certainly is continuing this DevOps maturation.

    Guys, congratulations. We’ll see you all soon. And for now, that’s a wrap on this DevOps Chat. This is Alan Shimel for DevOps.com. Have a great day, everyone.

    — Alan Shimel

  • DevOps Chat: Mainframe DevOps Update With Compuware’s Chris O’Malley

    DevOps Chat: Mainframe DevOps Update With Compuware’s Chris O’Malley

    Christopher O’Malley is one of the leading voices in the DevOps for mainframe community. He is relentless in advocating how the mainframe platform can not only exist but also thrive in today’s Agile/DevOps environments. His evangelism keeps his company, Compuware, front and center in the mainframe DevOps discussion and it really lifts the entire market.

    In this DevOps Chat, we catch up with Chris to talk about the latest release of Topaz, trends in the mainframe market and what we can expect in the near term.

    Chris is always a great interview with lots to say and this interview is right on point. As usual, the streaming audio is immediately below, followed by the transcript of our conversation.

    Transcript

    Alan Shimel: Hey, everyone, it’s Alan Shimel, DevOps.com, and you’re listening to another DevOps Chat. This DevOps Chat features a return of our friend, Chris O’Malley, CEO of Compuware. Chris, happy 2019 and welcome back to DevOps Chat.

    Chris O’Malley: Thank you and glad to be here, Alan.

    Shimel: Absolutely. You know, Chris, it is. It’s 2019; DevOps continues to mature. And I think one of the trends that we’ve seen, certainly over the last year and I expect it to even accelerate this year, is changes are coming to the DevOps and mainframe market. Right? Mainframes aren’t going anywhere; in fact, shops that are using mainframes, I think their footprints are increasing, rather than decreasing.

    And, from what we see in our readers and in talking to them, is that they are eagerly anticipating new developments, new functionality, that their mainframe software will offer, that will allow them to leverage DevOps types of foundations, DevOps types of functionality. And, I mean, DevOps and Agile and, really, code word: continuous delivery, all of these new ways that we’re defining the modern software factory, if you will. You’re in the driver’s seat for it, right, Chris? I mean, Compuware is kinda leading this charge. What do you see – you know, put on your analyst glasses here. What do you see in the market?

    O’Malley: Well, first, just confirming what you said about the mainframe, is that the z14 has been awakening, I think, for customers and analysts, in terms of its success, so the z14 is the most successful mainframe model in over a decade. It’s had five consecutive quarters of year-over-year growth, which probably hasn’t happened in maybe forever. And, as you said, the reason that’s happening is the mainframe is still unmatched in terms of its reliability, its performance, its security, its transactional efficiency. And all of those things, all those virtues, are incredibly important in a digital age, where you try to basically create a customer experience through software. So the mainframe, as you say, it’s not just doing well; it’s actually doing better than historically because of the digital age.

    Shimel: Mm-hmm.

    O’Malley: So, just giving my disposition on the mainframe and the mainframe’s role within DevOps, I think, if you kinda zoom out a little bit to make sense of what’s happening to the mainframe, you gotta start first with the fact that all of these companies that use mainframe are of a mindset that “We’ve gotta be customer-centric. We’ve gotta serve our customers, whether they’re individuals, if they’re federal agencies, whether they’re citizens or it’s business-to-business.” But, for 100 years and for 100 years from today, companies that are the most customer-centric, obsessed with customers, do the best.

    And, when you look at kinda this modern age, to do well from a customer-centric perspective, you’ve gotta create awesome digital experiences. And that’s done by writing software. So companies – you know, and Amazon has taken the lead in a lot of these kinda more newer-age companies. They’re incredibly good at taking ideas that matter to customers – and, again, it’s usually through digital means, increasingly – rapidly turning those into deliverables that make a difference. And, whether it’s centered on creating these awesome experiences and when they kinda look at the measures of what they’ve gotta do better to turn those ideas into deliverables, they’ve gotta be really good at velocity, how fast they do those things. They can’t do it in ways that compromise quality, so quality has gotta be something that’s constantly getting better and they’ve gotta become more efficient.

    I mean, in the manufacturing space, they talk a lot about getting rid of non-value-added efforts and thinking about retooling things that are a cause for technical debt, that encumber your ability to increase velocity and improve quality, so you’ve gotta fixate on the efficiency side of it. So these three metrics are emerging as foundational to improving things like customer sat and improving financial performance.

    So, when you start now getting closer to large enterprise IT, you’ve got a mainframe, these ideas that matter to customers don’t neatly fall either on the mainframe, in terms of the coding work that’s gotta be done or the cloud or some other platform, almost always, these ideas span these platforms. So there’s a necessity to improve velocity, quality, and efficiency across the entire enterprise.

    So, you know, the mainframe, historically, has been this kind of silo-ed organization that had its own culture, its own set of processes, its own set of tooling. And, as you said, the mainframe’s not going away for all the right reasons. In terms of creating these awesome experiences, reliability matters. Performance matters. Security matters. How efficiently you do that, from a cost standpoint, matters. But you gotta draw the mainframe to be a like system and make it different only in syntax from these other platforms.

    So there’s a strong move within these large enterprise ITs to break down this silo, get rid of a house divided against itself, bring the mainframe into a culture that’s centered on innovation, not on maintenance, bring it from waterfall to Agile, to bring it to a tooling set that’s familiar to non-mainframe platforms, the products like Jenkins and SonarSource and SonarLint or Paris. All these type tools that are used aggressively to improve velocity, quality, efficiencies on these other platforms, bring these to bear on the mainframe.

    So that’s what’s causing, I think, the mainframe to be in this kind of disrupted state of trying to get the culture to move, getting people to think and behave differently, to embrace this kind of new order of how work is done, Agile techniques specifically, and to get away from – you know, the mainframe has always been selectively automated in incredible ways. I mean, the transaction volumes it can support are unmatched within the marketplace, but it’s selective automation, not intrinsic automation, which is a big part of what DevOps tries to push. And getting rid of non-value-added work on the mainframe and getting to the point that the value-added work can be done, again, faster, greater-quality, greater-efficiency.

    So that dynamic is now gone from kind of innovators and early adopters within the mainframe market, you’re now getting into more of the early majority, so you’re seeing it more prod-based, in terms of people looking at the mainframe as that “We’ve gotta make this like any other platform.” And I think the success of the z14 makes sense, why it’s doing so well. Not just better than the z13, but better than previous releases over in the last 10, 20 years, so it’s exciting times in the mainframe. And I think its success is gonna be something you see emerging over the next ten years.

    You’ll see fits and starts and, obviously, problems, but one of the things that’s making me so excited – you’re seeing successes in the market that are outrageous. We’ve got customers at Compuware that are increasing their development productivity by a factor of 4, I mean, a 400-percent increase in productivity, so they’re getting 4 times the number of story points completed within a release cycle. They’re taking testing time from weeks to minutes and with better quality. And, when that is understood by customers, you know, if you can go from weeks to minutes, that’s a lot of non-value-added work that you’re doing, right? That’s like burning money in the parking lot because a customer’s not gonna give you more money to do the same or less quality in what you can do in minutes, as opposed to somebody that’s a laggard doing it in two weeks.

    Shimel: Absolutely.

    O’Malley: So you’re seeing these amazing success stories. Again, it’s not just our tools; it’s _____ –

    Shimel: No, no, it’s important – Chris, it’s that 10X, right? Everyone’s looking for that 10X. And we’re starting – you know, 4X is great; 5X is better. We’re getting those – we’re hearing success stories, the 10X success stories, with mainframe. And it goes back to something else you said, that, instead of people saying, “Oh, I’m winding down my mainframe or I’m migrating mainframe,” no, I’m looking at a ten-year roadmap on mainframes. It’s not – a couple years ago, there was this – I don’t know – fake media, fake news, about “Yeah, well, it’s here now, but we gotta kinda wind down.” What you’re saying about the z14 is exactly right; it’s bigger than anything that’s come before it, in essence, right?

    Here’s another thing, though, Chris, and I’m interested in your take: kudos to the mainframe software market and vendors, who are recognizing – not all of ’em, not all of ’em, but the ones who are, the enlightened ones, who are recognizing this and are capitalizing on it by bringing the tools the market wants to bear. It’s more than just the hardware, right? The hardware’s phenomenal – I’m not gonna deny that; IBM has done amazing with the z14 – but the software vendors, including Compuware, right – take a bow – have really delivered. You guys have a new release coming out or just came out, is that right?

    O’Malley: Right. It’s the 17th consecutive quarter that we’ve come out with new capabilities, upgrades to our classic offerings and integration with awesome and preferred DevOps tools, so, yeah, we’re very excited about this newest release.

    Shimel: So talk to me. Give me the three biggest things that get you excited on this release, Chris.

    O’Malley: So, each quarter, we’re turning the crank in terms of trying to nail the jobs that are changing, right? As I explained earlier, when you’re going from waterfall to Agile and from selectively automated to intrinsically automated, you’re trying to go from maintenance to innovation, the jobs that people have to do change radically. And they change radically in ways that are difficult to achieve. I mean, when you’re working on the mainframe, you’re working with large code bases, large in terms of the quantity of those code bases and also the size of any one of them.

    So these techniques, at a high level, just seem obvious, but the practical things that need to be done from an innovation standpoint to help people to nail these things, it takes a lot of innovation. I mean, we’ve had to do a lot of work to get to the point where you’re seeing orders-of-magnitude improvement in the way people develop, the way people test, so this latest release is just another chapter in that story.

    So we came out with Topaz team profiles, so that’s a good example of a capability that allows those that are working on systems to basically be able to store their configurations, the inputs and outputs of these applications, the test artifacts that are used, and build those artifacts and configurations in ways that can be shared broadly within teams or across the organization. That’s important for a couple things – one is the work to do that, you wanna do it once and then do it once for every one ’cause, in the DevOps mindset of continuous improvement, you wanna get rid of things that aren’t value-added, things that customers won’t pay for.

    So, in creating this capability, we allow now, as teams get spun up, right, you get a team that’s gonna work on a two-week sprint, you want them, out of the gate, to be productive. So being able to share profiles day one, with people that may have never worked on these systems before, is crucial so that you’re productive in the first minute of the first day.

    You also have situations where you’ve got, instead of technical debt issues, maybe a team, a large team, that you wanna go after certain work. It’s gonna help to improve the future throughput of the incremental efforts you have, from a development perspective, in the future. And you’re gonna bring large groups of people to work on something and that may be an outsourcer that you’re engaging; it may be new college recruits that you’re engaging. But you, again, want those people to be as productive as possible, as quickly, so, if you’ve got an experienced person on your staff that can create all those configurations and then share that broadly, again, you’re gonna lessen the ramp-up time for them to do their effective work.

    So, again this is another example of us thinking about the mainframe going from waterfall to Agile, at scale, and having a scenario where it’s not just people that have been on your staff for 30 years, working on the same subsystems of your applications, every day for their entire career. Now, ’cause ideas are paramount and where people work has more to do not with what they know but what the company wants to do in better servicing their customers, so there’s more of a randomness to the work and a changing nature to the work, and what you work on is fluid. So we’re really thinking deeply about the implications of that and, for profiles within Topaz, that’s a major step in the direction of getting a different way of working within IT.

    The second thing that we’ve released is another Jenkins plug-in. Jenkins is a central part of CI/CD pipelines, in terms of the automation side of it, and we’re constantly thinking about “What are the things or plug-ins that need to be created to make the mainframe equal to any other platform in all ways, other than syntax?” So JCL, Job Control Language, is a very important part of how programs are run within the mainframe. Creating a plug-in to help bring it into the CI/CD pipeline efforts and the automations efforts was a crucial thing that needed to be done, so it’s a contribution, from an artifacts standpoint, that, again, takes another step for our customers.

    And then, lastly, is in the area of testing. Compuware is the leader, if not exclusive, provider of unit and functional testing as a combined offering. It’s a necessary part of the job that people have to do within Agile, with this concept of shifting left. The developers aren’t just coding; they’re coding and testing, both in terms of bugs and performance, and becoming much more extended in terms of their role. And so their accountability and responsibility is bigger.

    So they need tools, like unit and functional testing tools, to participate within a two-week sprint process so that their deliverable, at the end of that cycle, is one that’s close to bug-free as possible and not putting future work into the system where it comes back as some quality issue 6 months later, 12 months later, when it runs into production, or, you know, as usability concern. So these three things are another turn of the crank, if you will, and Compuware’s offered to help customers nail these jobs, these new jobs that are created within kind of Agile work environments and DevOps work environments.

    Shimel: Excellent. Anything else on the new release before I go off on another direction?

    O’Malley: No, it’s just we’re very excited ’cause we released a technology called “zAdviser,” six months or so ago, which is a method of collecting what business KPIs for customers and then looking at, actually, stats relative to our products, so we can see kind of the behavior of developers. And we’re working with companies collaboratively to treat their people as they should, like high-performance athletes rather than slaves in a Roman ship galley. You can say, “Work harder. Work faster.” Well, you’ve gotta be constructive in helping to share best practices and methods that have a positive effect in things like velocity, quality, and efficiency.

    So, with every release, we’re seeing a significant increase in this KPI performance, so, as we’re talking about earlier, the mainframe is certainly on a march to become a first-class citizen within the digital age, within these enterprise customers, but the more _____ –

    [crosstalk]

    Shimel: I’m sorry; go ahead, Chris. Go ahead.

    O’Malley: Go ahead.

    Shimel: Well, no, I was gonna say, you know what I find interesting is, yeah, we’re talking about the mainframe machines, but so much of your focus is on the people, on the developers, on the team, because, at the end of the day, that’s the gold. Right? That’s the almost irreplaceable asset, are the people and the developers and the IT teams that are working on these mainframes. And it’s about empowering them. Right? So I think that it’s the right place to put the focus.

    O’Malley: Yeah, people –

    Shimel: Yeah, I –

    O’Malley: Just to restate the obvious here, but people matter most. When you think about the mainframe in the context of Agile and DevOps, asking people to change their role, though they’ve been around for 30 years, 40 years, dramatically – I mean, the day in the life in Agile and DevOps is vastly different than the day in the life of status quo. And it’s important that, as leaders, you’re promoting the appropriate culture and getting people to understand why they need to change, important that you’re giving ’em the necessary education, teaching them how to be effective in Agile sprints. I mean, that’s a different way of working. I mean, it’s not that you’re working 24 hours a day, but you’re definitely working in a more intense way. And then products matter then. The tools have gotta be enablers, not things that fight against your efforts to adopt this new culture and Agile nature.

    And, you know, I really try to express to IT leaders, “You gotta think about these things ’cause – ” you know, we use this example, the Gallup survey. Gallup survey have been doing this thing for 20, 30 years of employee engagement in large enterprises, and what’s most interesting about it is, over 20, 30 years, the employee engagement statistics haven’t really changed. And it’s disturbing when you actually hear about them. Two-thirds of employees in most large enterprises are not engaged, half of them are kinda, “It’s a job; I don’t care that much,” and 16, 17 percent are actively disengaged – they try to sabotage the efforts of the companies.

    So one of the biggest problems is engagement and employee engagement, and so I try to get people to understand – people matter hugely. And you’ve gotta give ’em the culture necessary to thrive in this more competitive age, you’ve gotta give ’em direction in terms of the process, the way that work is done, but you gotta give ’em the tools. You know, asking ’em to use mainframe tools out of a NASA movie from 1965 and thinking that something’s gonna happen is incredibly naive, so –

    Shimel: Absolutely. You’re not landing people on the dark side of the moon with that. Chris, we’ve got a few minutes left. I wanted to touch on another subject – we didn’t really kind of pre-talk about it, but I’m sure you’ve got some great views on it – I was out in Seattle last month for KubeCon, you know, Kubernetes, cloud-native. It’s a funny question to ask the mainframe guy the cloud-native question, but, certainly, containers are no strangers to mainframes.

    O’Malley: I mean, they’ve been on the mainframe for –

    Shimel: 20 years, maybe more, right? 25. So anything specific to Kubernetes in the Topaz or Compuware kinda roadmap that you can discuss right now? Or what effect do you see on your world?

    O’Malley: Yeah, so, when you think about kind of the flow of work in large-enterprise IT and what developers are fixated on, I mean, part of it is innovation and functionality that they’re trying to build, which is most important. I mean, that’s the value-added efforts of developers that make a difference in the eyes of the customer. And you wanna do that as quickly as you can, with the greatest degree of quality.

    But the other things you gotta think about are things like  quality itself. I mean, if you get anything that you release that’s got bugs in the system, how rapidly you can respond to it. That’s always part of the work that’s being done and I don’t know of any organization that doesn’t have some degree of bugs. And bugs may be software written improperly – and, again, our tools are helping to fix that – or usability issues, integration into other systems.

    There’s also the work of compliance and security that you gotta do, but this last thing is this thing, technical debt, right? That any code base that’s had any life to it, has meandered through the efforts to serve customers, it’s something that grows over time. And it has nothing to do with mainframe; it happens everywhere. You know, the cloud – you know, Amazon’s got debt. And you gotta deal with that technical debt, so, as you’re dealing with technical debt, the smart approach is to start using more of these modern techniques. SOA was kind of a philosophical approach to breaking down these larger systems into smaller subsystems. When I think about API enabling, REST Y enabling  these kind of smaller chunks, but you also wanna think about containerizing, using these as a form of containers, Docker being the method of creating the container, Kubernetes of being the way of managing these.

    So we do see this playing an increasing role _____, as people start really looking at affecting technical debt ’cause that gives you the biggest bang for the buck. I mean, you certainly wanna do it with any new functionality you’re doing, but you’ve got some huge numbers of lines of code that you’ve developed over four years that works awesome. I mean, it’s really good working code, but making it more manageable and getting rid of that technical debt. Part of that equation is RESTful APIs; it’s containerizing it and managing it more in a cloud-like fashion. So, yeah, you’ll see us enabling those tools.

    Our approach has always been to go to the edge of the mainframe and then we stop. We take care of what we know best and getting rid of these esoteric differences, but then, when we go off the edge of the mainframe, we work with all these other vendors. We allow SonarLint and SonarQube to do its work on COBOL, as akin to what it does with Java or C. Parasoft is a testing framework, allowing our products to work within that orchestration.

    So there’s all these tools that work in kind of the DevOps world that are preferred and we’re enabling them, so you should think of us doing things in support of Docker and Kubernetes over the long term. But, yeah, it’s an important part of making mainframe IP one that the throughput can constantly be improved and those techniques obviously help enormously.

    Shimel: Absolutely. Chris, I’m sorry to get you going ’cause it doesn’t take much when we start talking mainframes and DevOps, but we’re way over time and I need to wrap up. Hey, we’ll have you on, certainly, next quarter for the – what will it be? – the 45th time?

    O’Malley: [Laughs] We’ll be doing this and it’ll be our 1,000th. You and I will be in wheelchairs, talking about it.

    Shimel: By then, you should be down in Florida with me, but, anyway, Chris, it’s a pleasure having you on. Continued success with Topaz and Compuware. And we’ll catch up with you soon?

    O’Malley: Always a pleasure and anytime I can help.

    Shimel: All right.

    O’Malley: And thank you for the hard work that you do.

    Shimel: Thank you.

    O’Malley: _____ important _____ –

    Shimel: All righty. Chris O’Malley, CEO, visionary, leader at Compuware, on DevOps Chat with me today. This is Alan Shimel and you’ve just listened to another chat.

    — Alan Shimel

  • Will ITIL V4 Bring Order to a DevOps World?

    Will ITIL V4 Bring Order to a DevOps World?

    It is difficult to dispute the importance that the ITIL (Information Technology Infrastructure Library) approach has had on the world of ITSM (IT service management). After all, the best practices established by ITIL V3 have come to be the accepted way ITSM providers conduct business—so much so that many other IT processes have come to look to the ITIL framework to bring order and control to complex processes.

    Yet, ideologies such as DevOps have struggled to adapt to the somewhat rigid doctrine of ITIL V3, instead favoring Agile methodologies to meet the rapid deployment of updates and the iterative cycle of software development. Perhaps, the Agile-versus-ITIL differences can be attributed to how life cycles should be managed and how the management of those life cycles impact both development and operations in the IT space. Nevertheless, change is coming to the world of ITIL, which may help to make the framework more suitable for DevOps practices, and perhaps make ITIL and Agile symbiotic in the field.

    The ITIL V4 framework is scheduled to be released in Q1 2019, some two years after it was announced by AXELOS at the Fusion 2017 conference. “ITIL is the most firmly entrenched ITSM framework in the infrastructure of large, complex organizations, with many—if not most—Fortune 500 companies using it,” said Cathy Kirch, president of itSMF USA, the organizer of FUSION 2017 and the leading independent professional forum for IT Service Management professionals in the United States. “Currently, ITIL and complementary practices, such as DevOps, Agile and Lean, are used successfully in conjunction with each other; however the next evolution of ITIL will promote and underpin dynamic collaboration.”

    The More Things Change, the More They Stay the Same

    ITIL V4 doesn’t propose to completely rewrite the ITIL V3 narrative; in fact, the core elements are expected to remain the same. ITIL V4 will simply include new material on integration with additional best practices aimed at mitigating some of the biggest criticisms around ITIL V3, the less-than-ideal relationship with the ITSM community and failure to keep up with recent trends in software development and IT operations.

    In essence, ITIL V4 will address those criticisms by shifting V4 to a community-driven initiative and integrating other practices such as Agile, DevOps and Lean into traditional ITIL best practices. That means ITIL may become complimentary to those other frameworks and begin to embrace other services management best practices in the context of new technologies, such as serverless, containers, microservices and multi-cloud.

    With the DevOps community picking up steam, the Agile Manifesto and Lean development displacing ITIL V3 in some practices, it will be interesting to see what impact ITIL V4 will have on the IT industry as a whole. While it is far from ITIL’s last gasp, ITIL V4 will face significant challenges in a DevOps world that is constantly changing. However, with more than a million ITIL V3 certificate holders worldwide, ITIL V4 will definitely garner a great deal of interest when it hits the streets on Feb. 28.

    — Frank Ohlhorst

  • DevOps Chat: Serverless and Abstraction with Nimbella’s Anshu Agarwal

    DevOps Chat: Serverless and Abstraction with Nimbella’s Anshu Agarwal

    Anshu Agarwal has a tremendous record of working in startups that had successful exits. Now for the first time, along with her co-founders, she is actually founding the startup herself. Nimbella is a serverless platform that is cloud-agnostic and makes developing and deploying applications easy. Whether in a public cloud, private data center or both, Nimbella can help.

    In this DevOps Chat, we speak with Anshu about her own personal journey and where Nimbella is and what the plans are for going forward.

    As usual, the streaming audio is immediately below, followed by the transcript of our conversation.

    Transcript

    Alan Shimel: Hello, everyone, it’s Alan Shimel of DevOps.com and you’re listening to another DevOps Chat. Today’s chat actually features a company that’s just starting out here, but it’s a guest who’s had a rather long career in the tech space. As long as mine, even, at some point, but I’m sure I’m much older. Anyway, I’d like to introduce you to Anshu Agarwal. And, Anshu, if I messed up your last name, I apologize. Was that good?

    Anshu Agarwal: Thank you, Alan. You said it perfectly right. Thank you for having me today.

    Shimel: Thank you. So, Anshu, let’s start off – I referenced that you’d been in tech a while. Let’s give people an idea of your personal journey that brings you here today, even going back to Speedera and beyond, if you’d like, but if you wanna start with Speedera, that’s fine. But let’s give folks an idea of who you are, what your background is.

    Agarwal: Sure. Thank you. So I am an engineer by background and I moved into product management and marketing after I finished my MBA. And, after that, I joined a startup and then the startup bug never left me. So I joined Speedera, which got acquired by Akamai. Then I joined another startup, Ankeena, which got acquired by Juniper. Another startup, ConteXtream, got acquired by HP. And my most recent startup, Cedexis got acquired by Citrix, so I have been acquired four times. And, in all of those companies, I kind of delivered the success for other people’s ideas because they were all successful startups, got acquired by very, very large companies, so they were indeed – from value standards, they’re considered successful, but, for myself, I had never really done a company of my own.

    So, when I got the opportunity of just starting and being a founder of a cloud infrastructure, because that’s my background – I’ve been doing cloud infrastructure for last 15 to 20 years – there was no question that I wouldn’t do it. And we got started on this company – the company name is “Nimbella.” And it is in the serverless framework or serverless platform space. And we are kind of disrupting what serverless is, so – or we are providing what serverless promises to the developers. So that’s what we are working on now.

    Shimel: Excellent. You know, you’re way too modest and humble because, for many people listening, whether they’re in the valley or Boulder or Austin or New York or Boston or London or Bangalore or anywhere else in this world, having four startups that were acquired by companies like Akamai, HP, Citrix, and I forgot the fourth one – I apologize – a lot of people would call that a career. Right? I mean, whether they founded the companies or not, just being associated with four companies that had successful exits to large companies like that is something to be proud of and it speaks volumes as to either you’re really, really lucky or you’re doing something right. Right? So congratulations to you for that.

    So you’ve mentioned Nimbella and I wanna kind of share with our audience up front that Nimbella is – you have raised some funds for the company and there’s a great startup – there’s a great founders group involved. You mentioned, I think, one of your cohorts from Speedera; I think there’s two other founders in the group. But the company’s sorta still – I don’t wanna say stealth, but pre-product, let’s call it. Pre-public-product. So I don’t wanna put you on the spot. I’d rather talk, I think, if it’s okay, Anshu, about some of the problems you see and potential solutions that we see.

    You know, we live in a great time for developers and engineers. Internet time, right? We’ve gone from servers to hypervisors to containers. Now serverless. And serverless seems to be all the rage, but, like any new technology, it’s not birthed mature, right? It doesn’t come out mature; it doesn’t start mature. And so there’s gonna be growing pains. Can you share with our audience a little bit about what you see as some of the growing pains around serverless or what you’re hearing from people around that?

    Agarwal: Absolutely. Well, thank you for – this is a great question because it is definitely the best thing that’s happening for developers. The reason I say that is because what developers really want is to develop their business logic or develop the code and not worry about the infrastructure. And serverless promise is all about that, that you don’t have to worry about the infrastructure. You don’t ever have to worry about how it scales, how it is secured. Anything that requires the knowledge of infrastructure should not be a developer’s worry.

    But, in today’s serverless frameworks – and there are a few serverless frameworks and some are very popular ones – there is so much complexity still for the developers because they have to combine many services. And, for instance, they have to think about storage, they have to think about identity management, and they have to learn all these services and integrate. Not only that, the management and debugging is still an issue. And these are growing pains, as you mentioned. The serverless computing is still in a nascent stage and, therefore, it’s lacking all the abstractions that are needed for the adoption to accelerate into new markets and application domains.

    Not only that, serverless today, if you look at it, is used only for stateless applications, so you’ll see it in IoT applications, you’ll see it in reactive kind of applications only. It is not suited for long-running or stateful workloads. The reason being is because serverless today is serverless compute; it is not serverless storage. So that’s another problem. We may consider it as a problem or we may consider it as the limited adoption of the technology, but that’s something we have to address if we want this to be adopted in general, across all workloads. Then the –

    Shimel: I’m sorry. We may also consider it as an opportunity, right? I mean –

    Agarwal: Oh, absolutely.

    Shimel: Yeah.

    Agarwal: Absolutely. And that’s what we’re doing. And I’m gonna talk about Nimbella and how Nimbella is addressing it.

    Shimel: Mm-hmm.

    Agarwal: Another important aspect of serverless today is, like any new technology environment, there is serious vendor lock-in because, if you are using AWS’s Lambda serverless framework, you are programming to the nuts and bolts of Lambda. If you are using Google, you are programming to GCP, and so it is with Microsoft Azure. So, in a sense, you are programming to proprietary frameworks and, therefore, your code is not cloud-portable.

    So, tomorrow, if your company decides that, “Now we are going to move from AWS to Google,” you gotta rewrite the whole thing again. So that is actually, again, the serverless promise, and, therefore, vendor lock-in is very real, especially for the companies, actually, that choose not to go to public cloud because they’re part of regulated industry or whatever _____. They want something to be running in their own private cloud; these serverless frameworks are not an option for them. And then, say, you were able to write code for all different frameworks, but then the complexity of operations is there because you can’t monitor and observe at each level since these are proprietary frameworks across these clouds.

    So there are serious issues for widespread adoption and that’s where we come in. That’s the opportunity we looked at when we decided to create this company, is we want to create a platform, a framework, that is providing all the necessary abstractions so the developer is not to worry about how the infrastructure works, how to scale the code, how to secure the code. The code is just nebulous logic that they write. Then it is also suited for long-running, stateful, streaming workloads, such as AI/ML workloads. I mean, we have a greater adoption of AI happening in every industry. If we are not going to be able to handle AI workloads, we are missing a big market.

    And those are real, real problems and we are addressing it with Nimbella, you know, the technical problems. Then there are operational issues where you want cloud portability, so we are also addressing that part because we are providing a layer of abstraction and it’s open-core-based, so a developer can program the way they program the business logic and not worry about which cloud they’re writing.

    Shimel: Got it. Got it, got it, got it. So, again, I don’t wanna be uncomfortable or make you uncomfortable with what we’re gonna talk about, but, you know, I was at KubeCon out in Seattle last month – seems like a year ago, but it was only last month – and, surprisingly or not surprisingly, “serverless” seemed to be on everyone’s lips. Right? And maybe some people listening to this are saying, “Wait a second, Kubernetes is for containers and containers can run on bare metal, yeah, but they run on VMware or hypervisor as well.” What do you think the role of Kubernetes is in the serverless future? Fair?

    Agarwal: Kubernetes is a great technology. I really think it has great future. And serverless frameworks use Kubernetes or something else like that. And we were also at KubeCon. It was actually our first conference as an Nimbella person attending – one of my cofounders, who actually is the creator of OpenWhisk Apache, a serverless framework, he attended it. And he mentioned the same thing that you mentioned, that serverless was on everybody’s slide deck. Okay? It is something everybody is looking at; it is something everybody is kind of they have it on their roadmap, that they wanna work on it. They wanna look at it and see if it works for their environment.

    And Kubernetes is perfect, actually. It provides the platform needed because it is widely accepted these days and it provides the necessary environment, but, again, it should not be the burden of a developer to know how to run, work with Kubernetes. It should be the infrastructure serverless framework that provides that to the developer and the developer’s still only programming the logic. It’s not for the developer to worry on what environment, what container environment they are working on and what VMs are being used and what bare metal is being used. That’s all abstracted away for the developer.

    Shimel: Got it. Got it, got it, got it. So, you know, I’m sure there are people listening to us today who are thinking to themselves, “This woman has been in four startups and has had tremendous success.” And now I’m sure you’re a CEO, right, of Nimbella and you’re cofounder –

    Agarwal: That is correct.

    Shimel: Right? And I’m hoping there’s a whole bunch of women engineers out here or women entrepreneurs who are listening to you, saying, “Hey, if she can do it, I can do it.” And we don’t have a lot of time left, but I wanna address those folks there, that are near and dear to me for a lot of reasons. What’s your advice to them?

    Agarwal: Absolutely. If I can do it, they can do it. That is a very true statement. And, I mean, you said some great words about me, but I consider myself average successful person and, yes, I have been lucky that the four companies I’ve worked for had great exits. But being a founder and CEO is still very different from what I did for the other companies before. So, basically, when you are part of the company, other companies, you’re not bothered by operational aspects of the company. You’re not bothered by hiring for every possible function in the company. So the skill sets that I’m developing now, there is no other experience that could have taught me that, so, even though – I say, “Everybody can do it and anybody can do it,” but they still have to learn the art of running a company on the job. It just – there is no course. There is no education that can be given because you have to experience it.

    So, to all the women engineers that are listening, I would say, “If I can do it, you can do it. You just have to just get into it. You just have to dive into it and it will come to you. You will develop your advisors. You will develop the people who help you on the way because you won’t have all the skill sets, but you will know who to go to for what skill set, as you take that role.”

    Shimel: Absolutely. Absolutely. And, you know, women or not women, entrepreneurs – I’ve also founded and helped start my fair share of companies – I don’t know if it’s courage or just pure moxie, if you will, right? You just put one foot in front of the other, right, and you just keep moving forward. And you can’t boil the ocean, right, but you can – these things are incremental, right, and they’re evolutionary. They’re not _____ –

    Agarwal: Absolutely. You’re absolutely right. These are such an incremental thing and you may be not finding the best solution, but you are optimizing your solution, at every step, because you’re trying to handle so many things and, if you wait for the best solution to arrive, it may be too late. So you’re kind of optimizing the solution for whatever issue or whatever problem you’re handling, and you move on with that. And it’s just only time will tell how successful I am in the role, but this is my experience six months into this company, six or seven months into this company, and I’m still learning the art.

    Shimel: Absolutely, it is. So what do you – short-term future, where can we find out more on Nimbella or when will we find out more, do you think?

    Agarwal: Oh, absolutely. So there is some information on the website – very little information, I would say – but it gives you a flavor of what we are doing. Then, in Q1, this quarter, we are going to be releasing our developer playground, which is actually – because the buyer of serverless technology is the developer and developer experience is the most important thing, a developer has to feel one with the programming experience. And that’s the true serverless promise and we wanna deliver on that promise, so, in Q1, we are going to release that.

    And, in later quarter, in Q2, since I have mentioned private cloud and hybrid cloud, there will be offering for private cloud in Q2. So, as this quarter progresses, we are going to populate the website more with the content and we’re going to host more meetings like this and make it available to the developer community in general, our solution, and then, later on, for enterprise sites.

    Shimel: Great. Great, great, great. Anshu, when I started, I said, “These are only about 15 minutes. It’s gonna go so quick, you won’t believe it.” If I told you we’re closer to 20 minutes already, you’d probably say, “Oh, my,” but we are and we’re gonna need to call an end to this DevOps Chat. You know what? It sounds like there’s a lot going on here and, if we can have you back on, you know –

    Agarwal: Oh, that would be great, Alan. I really hope we can –

    Shimel: Yeah?

    Agarwal: I really hope we can get back when we have our developer playground because I’d love to talk more about it.

    Shimel: I promise you we will.

    Agarwal: Thank you.

    Shimel: Promise you we will. Thank you. Anshu Agarwal, founder/CEO of Nimbella, our guest here on this DevOps Chat. This is Alan Shimel and you’ve just listened to another chat.

    — Alan Shimel

  • Automation: We Have Not Yet Achieved Balance

    Automation: We Have Not Yet Achieved Balance

    I’m going to start this week’s blog out with a very clear statement that regular readers will already be aware of: I fall on the automation side of the DevOps equation. I see DevOps as automation first, process second. The problem is getting to that “second” part. The act of automating the entire pipeline should, by example, show the benefits of integrating these disparate tools and disparate teams. But it seems it often does not.

    Simple Graphic for Discussion

    So I’m going to take a second to tell you why it should. If you have integrated tools and teams, kudos! Hopefully you get absolutely nothing out of this blog, and you are enjoying a drink on a beach somewhere while your entire operation is taking care of itself. Everyone else, please read on.

    Look at the generic graphic. Notice how each step on the top is a different shape? That’s because they are individually implemented with limited integration of both people and processes. So you have a tool to submit for testing … Does testing (or monitoring at the end) feed back into your bug tracking system? Does it create tickets to get stuff done? When a container fails, is it simply dropped and a new one spun up to replace it because “they’re just cattle”? If so, does the system take steps to keep the failed container—or at least information about it—alive so troubleshooting can occur? Ignoring failed instances that were replaced and their impact on users was low is exactly how CloudBleed happened a couple years ago.

    Automate first, get the benefits of automation and drive faster delivery. Then take advantage of all that automation and make it into a streamlined process. It’s a massively complex system, even if you deliver everything on a single “platform,” so it will never be perfect. But issues that occur with regularity can be automated also to provide integration between systems. Isolating an instance and sending a notice to the defect tracking system that it is alive but not communicating (because it has been replaced) is something that is relatively easy to script/integrate and offers the ability to find what actually went wrong. Contrasted with just dropping the instance and starting a new one, this process is an order of magnitude better, and with integrated teams, Dev and Ops can work together to resolve whatever caused the instance to go down.

    Strong integration and a smooth pipeline is what technologies such as application release automation are designed to deal with, but most shops see these tools as “unnecessary” overhead. I disagree. Find one you like—there are a million of them—and use it to connect things. It’s just one step, but it is a step to transforming your operational delivery systems from unique snowflakes to part of a collection that works together.

    That’s it. Automate first because the gains are phenomenal, but don’t stop there. Really—don’t. You are doing yourself a disservice if you do. The whole “We’re going to change your world!” crowd is annoying, because changing your world is your decision, but listen to them anyway, and see which of their ideas fit your organization. You’ll be surprised once automation is solidly in place how much of what the process side of DevOps offers makes sense.

    — Don Macvittie

  • In AI Initiatives and Hybrid Clouds We Trust

    In AI Initiatives and Hybrid Clouds We Trust

    The 2019 State of the Union Address, delivered Feb. 5, was light on anything tech-related. But just a week later, Feb. 11, technology was the focus of an executive order that launched The American Artificial Intelligence (AI) Initiative. Why do we need a national AI initiative, and what does it have to do with DevOps? (Hint: You may need hardware and a hybrid cloud strategy for your AI initiative.)

    Investments in AI technology are exploding worldwide. Most analysts believe that while the total number of startups and equity deals to date still favors the United States, China is spawning startups rapidly, thanks to their recent AI initiative. According to just-released data from CB Insights:

    “While China accounted for only 10% of global AI deals in 2017, Chinese AI startups took 48% of all AI funding dollars that year, surpassing the US in AI funding for the first time.”

    The long tail doesn’t lie. A big global competition here is a good thing, though, and the scoreboard has spurred movement. The American AI Initiative lays out five principles, paraphrased in brief:

    1. We need to drive technological breakthroughs in AI.
    2. We need technical standards for development and testing.
    3. We need workforce training in skills and applications.
    4. We need to be able to trust AI solutions for privacy.
    5. We need both open markets and protection for “critical” technologies.

    Go, Team America! What, you expected more than motherhood and apple pie out of the White House? I’m not going there. I don’t think any administration could prepare the level of technical insight needed for something of this magnitude. It can, however, lay a course and task agencies for technology plans and funding proposals—and about two-thirds of this executive order is devoted to timelines for those.

    Let’s assume this AI initiative comes together with real dollars, and everyone including your competition is at work on AI. You may be thinking, “This is easy. I’m looking at a prominent cloud vendor’s machine learning webpage right now and it says, ‘No machine learning skills required.’ Its solution is based on its libraries of algorithms plus cloud-based workload acceleration hardware.”

    If that approach solves your specific problem, that’s great. Many use cases will fit those offerings. However, I’m thinking that a lot of you out there won’t turn over your data and your privacy control to a single vendor. Based on what I’ve seen, while the public cloud may be fast enough for some use cases, it’s woefully non-deterministic for others. You may be able to afford all kinds of time and elastic cloud resources for training, where time may be measured in weeks or months on bigger machine learning data sets.

    Real-time inference is a different game. There. Will. Be. Slowness. One missed result can be very costly—ask any autonomous vehicle developer, then look at all the hardware sitting in the trunk. Public cloud solutions are woefully non-deterministic, especially as everyone and their dog starts using them. The CEO of another big cloud vendor just said in an event keynote that most AI solutions will split 40/60 private/public cloud. She went on to say if you’re regulated, it’ll be more like 60/40.

    Most current startup investment is headed for AI chips. Machine learning is the ultimate workload optimization problem. By adding compute nodes with clusters of GPUs, FPGAs or dedicated system-on-chip (SoC) designs, inference speed-up is achievable. Whatever approach you chose, you’re probably going to want workload optimization safely in your private cloud instance within a hybrid cloud AI solution.

    You’re also going to want control over differentiation of your solution. Open source AI software solutions are great because they handle a fundamental base of value. However, AI-enabled applications are very likely to run into problems at scale. Open source will get you to prototyping quickly. Your innovation will be tested as real-world, real-time issues develop. If you can get a truly differentiated hybrid cloud AI solution that proves to be fast, reliable and secure, you’ll have a huge competitive advantage.

    Again, this looks like a perfect job for a DevOps team. Here’s where to start on your AI initiative, in increasing order of difficulty:

    • Have someone get familiar with machine learning and AI public cloud solution offerings—those vendors are easily found.
    • Also get familiar with open source frameworks such as TensorFlow, Caffe, Cognitive Toolkit, PyTorch, MXnet, DeepLearning4J and others.
    • Look at your preferred distributed computing framework for recent innovations in AI workload optimization; for instance, a lot of work is happening around Hadoop and Spark.
    • Inventory the problems you think AI might solve in your organization and see how they line up with those potential solutions.
    • Experiment and discover where the bottleneck in your preferred algorithms might be; those will be candidates for a hybrid cloud workflow optimized solution.
    • If there’s a profound bottleneck, you’ll need to evaluate the best hardware solution, meaning someone on your team (or a consultant) should understand GPU code or FPGA design.
    • For the most severe AI challenges, more companies will be considering either off-the-shelf AI chips or custom SoC designs—even companies that have never designed chips.

    My point here wasn’t to provide a primer on AI technology; that’s a huge topic. The American AI Initiative is a call that teams need to move on AI now or risk being left behind in the competitive prop wash. You also shouldn’t wait for someone to generate the exact solution you need out in the marketplace. With the feds driving a sea change in AI, investments and technology will be shifting even more rapidly, and it’s best to get in front of it.

    For the curious, here’s the entire executive order on the American AI Initiative.

    If you’ve started already, how’s your AI initiative going? Are there other areas in AI or machine learning you’re curious about that would be a good topic for future posts? Reach out if you’d like to discuss this.

    — Don Dingee

  • Python Benefiting From Increase in DevOps Use

    Python Benefiting From Increase in DevOps Use

    DevOps and Python have developed a mutually beneficial relationship.

    In a recent survey released by the Python Software Foundation and software development company JetBrains, researchers found that more and more developers are using the program language for an array of versatile projects, including DevOps and machine learning.

    In fact, Python developers told the researchers that DevOps is increasingly part of their professional development work.

    “In 2018 we had significantly more respondents specifying they’re involved in DevOps (an increase of 8 percent compared to 2017). In terms of Python users using Python as their secondary language, DevOps has overtaken web development,” according to the “2018 Python Developers study,” which is based on responses from 18,000 developers in 150 countries.

    Over the last two years, Python has grown more important to developers thanks to its flexibility, especially when it comes to web development. It’s no surprise that as other projects have gained popularity in the enterprise—DevOps and machine learning—developers have started adopting the language to other disciplines, especially as businesses invest more in application development.

    In TIOBE’s February 2019 index, Python now ranks third behind favorites such as Java and C, edging out C++ for its current position.

    Digging into the numbers, the survey finds that 43 percent of developers are using Python for DevOps projects, as well as system administration and writing automation scripts. That’s an increase from the 35 percent who reported the same types of uses for the programming language in 2017.

    The report notes that these numbers reflect a combination of Python used as either the primary or secondary language for various projects. As the report notes: “DevOps/system administration/writing automation scripts has moved into first place among Python users using it as a secondary language.”

    Machine learning is another up-and-coming category, with 38 percent of developers using Python for these projects compared to 31 percent last year.

    Overall, data analysis remains the No. 1 use case for Python developers, with 58 percent reporting that it’s their main focus when it comes to development.

    In second place is web development, according to the report.

    In terms of the top cloud platforms for Python developers, Amazon Web Service is the top choice, with 36 percent of respondents reporting that they use AWS. From there, it’s Google Cloud Platform (29 percent) Heroku (26 percent), DigitalOcean (23 percent) and Microsoft Azure (16 percent).

    About a third of those surveyed responded that they don’t use any cloud platforms.

    When asked about running code in a cloud production environment, 47 percent responded that they use virtual machines, although 40 percent of Python developers reported that they prefer containers. Another 28 percent use platform as a service (PaaS) and 21 percent are taking the leap to serverless computing.

    Additionally, when developing for the cloud, about 35 percent of those in the survey report that they use Docker containers.

    This increase use of Python within DevOps, as well as other connected areas such as serverless computing, does not surprise Anshu Agarwal, the CEO and co-founder of Nimbella, a cloud-agnostic, serverless cloud platform for developers. She noted the language’s use within a wide range of projects, specifically machine learning and AI, are driving this change.

    “Python developers are driving serverless requirements in new and unique ways, primarily because of the interest in data analysis, ML and AI,” Agarwal wrote in an email. “These data analysis and ML/AI-centric serverless use cases are different than request/response style serverless functions which run for much shorter durations. Python functions in serverless may have much longer durations (10s of minutes or hours), and consume a lot more memory. For such functions a different kind of serverless framework needs to be adopted that can handle long running workloads.”

    When it comes to continuous integration systems, about a third of respondents don’t use any. However, when they do, Jenkins is the most popular at 25 percent, followed by Gitlab CI and Travis CI, with 18 percent each. For configuration management, 20 percent reported using Ansible, while 9 percent used custom tools. Puppet, Salt and Chef also appeared in the list.

    Finally, about two-thirds of Python developers (69 percent) use Linux as their main operating system of choice for development. Microsoft Windows comes in a distant second at 47 percent and Apple’s macOS in third at 32 percent, according to the study.

    — Scott Ferguson

  • Break-Even Analysis: Understanding the Impact of Automation

    Break-Even Analysis: Understanding the Impact of Automation

    Determining whether automation is helpful beyond the CI/CD pipeline can be easier through a break-even analysis

    The pursuit of automating away all the operational tasks in your organization doesn’t stop after you finish building a solid CI/CD pipeline infrastructure using the myriad tools available and making it stable.

    There are many more one-off tasks not part of the CI/CD pipeline, but which are very important for managing the infrastructure and operating the applications running in production, that are not yet automated. These tasks could be very specific to your organization, but some common categories of these tasks are:

    • Provisioning and maintenance of infrastructure, including patching, updating certificates and licenses.
    • Maintenance of monitoring systems.
    • Generation of reports and dashboards.

    To Automate or Not to Automate?

    A CI/CD pipeline is an end-to-end automated system that handles all aspects, from builds triggered by code change through deploying the built artifacts into production, with very little or no human intervention in between. The non-CI/CD tasks described above could be automated or be carried out manually.

    An inexperienced DevOps team can easily get carried away by the notion of  automating everything and end up spending time on low-priority tasks—sometimes, simply because using a new tool or the coding work involved in automating a task can be very attractive to the team members. While such endeavors would look great in a sprint demo and on a resumé, they won’t help the business achieve operational velocity. When there are multiple automation projects competing for the team’s time, it would make sense to do a break-even analysis to obtain better insight on the returns of automating an operational task.

    The Agile methods to select and quantify a task such as backlog prioritization and estimating time for a story are not adequate to assess how the organization would benefit from that project overall. The break-even analysis proposed here would help to determine whether an automation project is worth doing by looking at the overall impact of rolling out that automation in the organization. Agile is limited by its narrow focus on the team that rolls out the automation—the DevOps team—and it cannot provide any metrics on the overall impact of an automated process on the organization.

    The Break-Even Analysis

    Let’s define various metrics involved first and then do the simple math of overheads and breaking even with the benefits.

    Development time (d) – This is the time needed to develop and roll out an automation task. This would be same as the time estimate used for a related story in an Agile sprint. In hours.

    Maintenance time (m) – As any experienced DevOps engineer can attest, developing a script or tool suite is not a one-time deal. The changes will be needed due to many factors:

    • Infrastructure change.
    • Fixing bugs in the tool.
    • Enhancement requests.
    • Training users of the tools, such as SREs and fellow team members.

    Budget the hours needed annually for this task, depending on the complexity and possibility of changes down the road.

    Manual working time ™ – The time needed to complete the task without automation. Doing things manually can cause human errors which adds to the total time. Factor that in the estimate  if this task has been causing issues. In hours.

    Automated working time (ta) – The time needed to complete the task using automation. In hours.

    Jobs count (c) – The number of times this task is run. Make an annual estimation so even infrequent operational tasks can have a score.

    Though arbitrary, my suggestion is to try out an automation for a year for its benefits. Check if the project would bring in impressive returns during that time frame:

    Project overhead = d + m

    Savings from automation = (tm – ta) x c

    Net benefit to organization = Savings from automation – Project overhead

    To embark on an automation project, the organization should benefit greatly from it to do things faster and better. While an Agile way of prioritizing an automation task would narrow the focus only on the DevOps team, this method also gauges and factors in the users’ time.

    DevOps Benefiting Customers

    The tools developed might not be used internally always, which is a major assumption in the above break-even analysis. It is quite possible that an automation process could benefit end users as well. I think it is very important to factor in the time saved by our customers also.

    Happy customers always bring in goodwill to a company.

    — Thomas Theakanath

  • DevOps: Don’t Let Detection Be a Bottleneck for Security

    DevOps: Don’t Let Detection Be a Bottleneck for Security

    AI and automation can help ensure DevOps security doesn’t impact performance

    The world revolves around software and internet-based applications—and speed is crucial. Users have very little patience for delays and will quickly jump to something else if performance lags. Organizations have embraced DevOps and containers in an effort to accelerate development and deployment of software and maintain a competitive edge over rivals. One of the biggest challenges, though, is how to identify vulnerabilities and detect security issues without significantly degrading performance.

    There will always be some impact on performance. You are adding an extra layer of functionality—a layer of security intended to detect and resolve security issues. No matter how fast that detection is, it will still take more time than not having any detection at all. The goal is to implement detection with as little impact on performance as possible. You don’t want detection to be a bottleneck for your cybersecurity efforts, or a drag on performance in general.

    Bridging the Gap Between Speed and Security

    One of the defining characteristics of a cloud-based, containerized, DevOps environment is its dynamic and continuous nature. In an effort to streamline and accelerate business, organizations embrace continuous development, continuous integration and continuous deployment combined with continuous monitoring, continuous testing and continuous improvement—all of which require continuous security.

    Real-time monitoring for suspicious or malicious activity is crucial. The internet doesn’t sleep, and attackers are constantly scanning and probing for vulnerabilities and weak links to exploit in your network and applications.

    Security is important, but not at the expense of productivity. The challenge is how to bridge the gap between speed and security—and ensure you can detect attacks effectively without introducing undue friction. You need balance—security that provides necessary protection without defying the whole point of trying to streamline and accelerate business. In other words, security needs to become a guardrail rather than a roadblock for DevOps.

    Improve Detection with Automation and AI

    Real-time protection is necessary—but not all real-time security solutions are the same. How attacks are detected is as important—or more important—than just detecting them.

    Why? Simple. Attack detection based on signatures or generic patterns such as a core rule set or regular expressions are not customized to recognize threats in the context of the application. For example, with a RASP (runtime application self-protection) solutions, even if the control points are good, the end result will be poor without the right context and detection techniques. This approach is also reactionary and leaves you exposed for some period of time. It is generally ineffective until the signature is developed, and the signature can’t be created until the threat is detected and identified.

    Automation and artificial intelligence (AI) can accelerate the process and provide more effective protection with less friction. An AI-powered platform can learn from stateless traffic and deconstruct application logic to create application-specific rules—and update the rules dynamically without the need for human intervention. Using dynamic, application-specific rules rather than signatures and manual rules can result in efficient detection and an exceptionally low false-positive rate. An automated solution that leverages artificial intelligence also can protect without waiting for signature updates.

    Modern Security for Modern Networks

    The days of periodic scans are long gone. The network architectures and threat landscape today demand constant, real-time protection. Legacy detection techniques are cumbersome and can’t keep pace, but an AI-powered attack detection platform that is automated will be more effective for protecting a dynamic, cloud-based, containerized, DevOps environment.

    — Tony Bradley

  • Minor Crimes and Misdemeanors in the Age of Automation

    Minor Crimes and Misdemeanors in the Age of Automation

    Giving automation the power to detect crime and enforce punishment has ramifications, even for minor infractions

    In November, I broke the law. I crossed over a solid white line to make a right turn at a traffic intersection. At the time I was unaware of my violation. I was on my way to a shopping mall in an unfamiliar part of town to buy my wife a gift for her birthday. My only defense is that I was following the instructions emitted from the map app on my cellphone. It told me to make a right turn. So I did. Little did I know I was being watched.

    Automation enhances law enforcement capabilities

    A few weeks later, I received a citation in the mail. The citation had photos of me, my car’s license plate and the car crossing the solid white line. Turns out, there were cameras at the intersection watching all traffic activity. I was busted. I was also a bit shocked: My experience getting a traffic ticket 10 years earlier involved a real police officer writing me up in real time. Now it was as if Big Brother was indeed watching.

    In addition to the eerie feeling that goes with being charged with a crime anonymously well after the fact, I also felt baffled. The traffic citation did not list the amount of the fine I had to pay. The only information it provided was that I needed to go to the traffic court website to learn the details. So I did.

    I logged into the website and entered my citation number. The site had no record of the ticket, but noted that it can take up to 30 days for an automated ticket to make it into the system. I felt powerless; I was charged with a crime, liable for a fine and yet didn’t know how much my transgression was going to cost me.

    I decided to call the telephone number provided on the citation for more information. I dialed the number using the same cellphone that had gotten me into trouble in the first place. I was connected to an automated answering system. After navigating through various Yes/No and “enter the proper number” prompts, I learned that the system had no record of my citation. My only option was to wait 30 days for the systems to get their collective act together.

    A month later, a formal citation did indeed arrive in the mail, noting the amount of my fine ($480) and options for payment. Also, the citation provided instructions to go online and pay the ~$500 required to attend traffic school, should I so desire. (Traffic school is an option in California in which you attend a one-day class about traffic safety and your violation is kept private from insurance companies. The result is, your automobile insurance rates don’t go up. Here in Los Angeles, a lot of professional actors teach traffic school as a side job. As you can imagine, traffic school can be quite entertaining. But, I digress.)

    At this point, I was completely bewildered. I’d been busted for a crime I didn’t know I committed, by machines I didn’t even know were there. The only way I could get the details about my transgression was by interacting with other machines on the internet or on the phone. And, should I decide to plead guilty and settle matters, my first, best option is to do so via automation.

    Still, the technology was failing me. I needed to talk to a person, any person. So I went down to the courthouse in Santa Monica. There I found a special window on the exterior of the courthouse building for processing traffic inquiries. (One of the nice things about living in sunny Southern California is that providing service outdoors throughout the year is a viable option.)

    I waited in line. Finally, I got called forward to the window—which, by the way, was a pane of blackened glass. I had no way of knowing who or what I was talking to; all I saw was my reflection in the black mirror. A voice from the speaker at the bottom of the window asked me to state my business. I did. The conversation continued. The responses from the speaker seemed brusque, if not a bit preformulated. I started to wonder, Am I talking to a machine or a person? So I asked a random question, “How much is two plus two.” The voice in the speaker responded, “Sorry I am not programmed to respond to that information.”

    Now I was really thrown. Was I confronting yet another piece of automation? After a minute or two, the voice in the speaker said, “I’m just messing with you. I get this all the time. They all think I’m a machine.”

    It turns out that the voice did indeed belong to a human, and the human’s name is Dante Pipkins. Dante shared an interesting fact with me: Even though my violation was caught on camera, the actual OK to create the citation was issued by a human police officer behind the scenes. Go figure.

    Thus, having been enlighted about the details of enforcing traffic law via camera, I finished my business by having Dante give me a date to appear in traffic court, before a human judge, to plead my case. I knew there was no way to proclaim my innocence. I had been caught red-handed. My intention was to throw myself on the mercy of the court to get a reduced fine.

    The Power of Automation

    Automation is a great thing. It empowers us. It saves us time and money. Without automation, there’s no AWS, Google, Azure, Digital Ocean, Netflix, Etsy, Shopify, Facebook, Instagram, Slack, Airbnb, New York Stock Exchange … the list goes on. Automation is more than a technological feature. It’s now a way of life. And, for those of us who work in DevOps, automation is fundamental—none of us want to go back to the days of manual deployment and guesswork system monitoring, to say the least.

    Yet, we do walk a tightrope. As my recent travails with a very minor aspect of the legal system demonstrate, automation is becoming a key factor in everyday law enforcement. On face value, this might seem like no big deal, but consider this: Today, charging a citizen with a crime is still subject to some sort of human intervention. There are a lot of details to consider that go beyond the current capabilities of AI and machine automation. Human judgment is still the most reliable form of judgment. Yet, it’s time-consuming and expensive. Eliminating time-consuming, expensive tasks with cheap, fast, accurate machine-driven processes is a key motivator for using automation.

    As automation becomes more familiar with the activities it is processing, it becomes an expert. And, as automation achieves expert status, it replaces human activity. Just think about detecting credit card fraud; these days, practically all of it is done using automation. While we still have a long way to go in terms of totally automated processes, automation is creeping its way into law enforcement. My recent experience is proof.

    Now, imagine a world in which crimes such as tax evasion, financial fraud, embezzlement, regulatory non-compliance and yes, moving traffic violations are detected, reported and, most importantly, prosecuted using automation. Let’s go further: Let’s imagine that automation becomes the default method of prosecution. Imagine that the justice system becomes so automated that the defendant is presented with an offer to plead guilty to a sentence already predetermined according to an AI algorithm. Or, the defendant can choose to opt into a human interaction but will need to bear the burden of the increased court costs, very much in the same way that an automated audio transcription service charges extra for a human to do the work.

    Is such a scenario possible? Well, think about this: My interactions around the traffic violation were conducted without me ever having once had to interact with a human being. I was guilty as charged and automation proved this beyond a reasonable doubt. Not only did the technology detect my transgression, but it also connected me to my car, thus making citation possible. Hence, there’s a good argument that, when it comes to automation-enabled law enforcement, the distance from catching misbehaving drivers in the act to actually going after real crooks is just not that far a leap.

    So, what does this have to do with DevOps? When you scrape away all of the technological marvels, at some point there’s going to be someone in DevOps either creating, deploying or fixing the systems that make this all possible. It’s a big deal to give automation the power and technical wherewithal to enforce “the law.” As the feasibility of putting complex, comprehensive law enforcement systems into play increases, are we in DevOps going to consider such deployments as just another day in the CI/CD pipeline, or are we going to think through the game-changing impact of what we’re about to make?

    For now, I’ll leave it up to you to decide.

    — Bob Reselman