devopsy.kinsta.cloud

Tag: devops chat

DevOps Chat: Mainframe DevOps Update With Compuware’s Chris O’Malley

Christopher O’Malley is one of the leading voices in the DevOps for mainframe community. He is relentless in advocating how the mainframe platform can not only exist but also thrive in today’s Agile/DevOps environments. His evangelism keeps his company, Compuware, front and center in the mainframe DevOps discussion and it really lifts the entire market.

In this DevOps Chat, we catch up with Chris to talk about the latest release of Topaz, trends in the mainframe market and what we can expect in the near term.

Chris is always a great interview with lots to say and this interview is right on point. As usual, the streaming audio is immediately below, followed by the transcript of our conversation.

Transcript

Alan Shimel: Hey, everyone, it’s Alan Shimel, devopsy.kinsta.cloud, and you’re listening to another DevOps Chat. This DevOps Chat features a return of our friend, Chris O’Malley, CEO of Compuware. Chris, happy 2019 and welcome back to DevOps Chat.

Chris O’Malley: Thank you and glad to be here, Alan.

Shimel: Absolutely. You know, Chris, it is. It’s 2019; DevOps continues to mature. And I think one of the trends that we’ve seen, certainly over the last year and I expect it to even accelerate this year, is changes are coming to the DevOps and mainframe market. Right? Mainframes aren’t going anywhere; in fact, shops that are using mainframes, I think their footprints are increasing, rather than decreasing.

And, from what we see in our readers and in talking to them, is that they are eagerly anticipating new developments, new functionality, that their mainframe software will offer, that will allow them to leverage DevOps types of foundations, DevOps types of functionality. And, I mean, DevOps and Agile and, really, code word: continuous delivery, all of these new ways that we’re defining the modern software factory, if you will. You’re in the driver’s seat for it, right, Chris? I mean, Compuware is kinda leading this charge. What do you see – you know, put on your analyst glasses here. What do you see in the market?

O’Malley: Well, first, just confirming what you said about the mainframe, is that the z14 has been awakening, I think, for customers and analysts, in terms of its success, so the z14 is the most successful mainframe model in over a decade. It’s had five consecutive quarters of year-over-year growth, which probably hasn’t happened in maybe forever. And, as you said, the reason that’s happening is the mainframe is still unmatched in terms of its reliability, its performance, its security, its transactional efficiency. And all of those things, all those virtues, are incredibly important in a digital age, where you try to basically create a customer experience through software. So the mainframe, as you say, it’s not just doing well; it’s actually doing better than historically because of the digital age.

Shimel: Mm-hmm.

O’Malley: So, just giving my disposition on the mainframe and the mainframe’s role within DevOps, I think, if you kinda zoom out a little bit to make sense of what’s happening to the mainframe, you gotta start first with the fact that all of these companies that use mainframe are of a mindset that “We’ve gotta be customer-centric. We’ve gotta serve our customers, whether they’re individuals, if they’re federal agencies, whether they’re citizens or it’s business-to-business.” But, for 100 years and for 100 years from today, companies that are the most customer-centric, obsessed with customers, do the best.

And, when you look at kinda this modern age, to do well from a customer-centric perspective, you’ve gotta create awesome digital experiences. And that’s done by writing software. So companies – you know, and Amazon has taken the lead in a lot of these kinda more newer-age companies. They’re incredibly good at taking ideas that matter to customers – and, again, it’s usually through digital means, increasingly – rapidly turning those into deliverables that make a difference. And, whether it’s centered on creating these awesome experiences and when they kinda look at the measures of what they’ve gotta do better to turn those ideas into deliverables, they’ve gotta be really good at velocity, how fast they do those things. They can’t do it in ways that compromise quality, so quality has gotta be something that’s constantly getting better and they’ve gotta become more efficient.

I mean, in the manufacturing space, they talk a lot about getting rid of non-value-added efforts and thinking about retooling things that are a cause for technical debt, that encumber your ability to increase velocity and improve quality, so you’ve gotta fixate on the efficiency side of it. So these three metrics are emerging as foundational to improving things like customer sat and improving financial performance.

So, when you start now getting closer to large enterprise IT, you’ve got a mainframe, these ideas that matter to customers don’t neatly fall either on the mainframe, in terms of the coding work that’s gotta be done or the cloud or some other platform, almost always, these ideas span these platforms. So there’s a necessity to improve velocity, quality, and efficiency across the entire enterprise.

So, you know, the mainframe, historically, has been this kind of silo-ed organization that had its own culture, its own set of processes, its own set of tooling. And, as you said, the mainframe’s not going away for all the right reasons. In terms of creating these awesome experiences, reliability matters. Performance matters. Security matters. How efficiently you do that, from a cost standpoint, matters. But you gotta draw the mainframe to be a like system and make it different only in syntax from these other platforms.

So there’s a strong move within these large enterprise ITs to break down this silo, get rid of a house divided against itself, bring the mainframe into a culture that’s centered on innovation, not on maintenance, bring it from waterfall to Agile, to bring it to a tooling set that’s familiar to non-mainframe platforms, the products like Jenkins and SonarSource and SonarLint or Paris. All these type tools that are used aggressively to improve velocity, quality, efficiencies on these other platforms, bring these to bear on the mainframe.

So that’s what’s causing, I think, the mainframe to be in this kind of disrupted state of trying to get the culture to move, getting people to think and behave differently, to embrace this kind of new order of how work is done, Agile techniques specifically, and to get away from – you know, the mainframe has always been selectively automated in incredible ways. I mean, the transaction volumes it can support are unmatched within the marketplace, but it’s selective automation, not intrinsic automation, which is a big part of what DevOps tries to push. And getting rid of non-value-added work on the mainframe and getting to the point that the value-added work can be done, again, faster, greater-quality, greater-efficiency.

So that dynamic is now gone from kind of innovators and early adopters within the mainframe market, you’re now getting into more of the early majority, so you’re seeing it more prod-based, in terms of people looking at the mainframe as that “We’ve gotta make this like any other platform.” And I think the success of the z14 makes sense, why it’s doing so well. Not just better than the z13, but better than previous releases over in the last 10, 20 years, so it’s exciting times in the mainframe. And I think its success is gonna be something you see emerging over the next ten years.

You’ll see fits and starts and, obviously, problems, but one of the things that’s making me so excited – you’re seeing successes in the market that are outrageous. We’ve got customers at Compuware that are increasing their development productivity by a factor of 4, I mean, a 400-percent increase in productivity, so they’re getting 4 times the number of story points completed within a release cycle. They’re taking testing time from weeks to minutes and with better quality. And, when that is understood by customers, you know, if you can go from weeks to minutes, that’s a lot of non-value-added work that you’re doing, right? That’s like burning money in the parking lot because a customer’s not gonna give you more money to do the same or less quality in what you can do in minutes, as opposed to somebody that’s a laggard doing it in two weeks.

Shimel: Absolutely.

O’Malley: So you’re seeing these amazing success stories. Again, it’s not just our tools; it’s _____ –

Shimel: No, no, it’s important – Chris, it’s that 10X, right? Everyone’s looking for that 10X. And we’re starting – you know, 4X is great; 5X is better. We’re getting those – we’re hearing success stories, the 10X success stories, with mainframe. And it goes back to something else you said, that, instead of people saying, “Oh, I’m winding down my mainframe or I’m migrating mainframe,” no, I’m looking at a ten-year roadmap on mainframes. It’s not – a couple years ago, there was this – I don’t know – fake media, fake news, about “Yeah, well, it’s here now, but we gotta kinda wind down.” What you’re saying about the z14 is exactly right; it’s bigger than anything that’s come before it, in essence, right?

Here’s another thing, though, Chris, and I’m interested in your take: kudos to the mainframe software market and vendors, who are recognizing – not all of ’em, not all of ’em, but the ones who are, the enlightened ones, who are recognizing this and are capitalizing on it by bringing the tools the market wants to bear. It’s more than just the hardware, right? The hardware’s phenomenal – I’m not gonna deny that; IBM has done amazing with the z14 – but the software vendors, including Compuware, right – take a bow – have really delivered. You guys have a new release coming out or just came out, is that right?

O’Malley: Right. It’s the 17th consecutive quarter that we’ve come out with new capabilities, upgrades to our classic offerings and integration with awesome and preferred DevOps tools, so, yeah, we’re very excited about this newest release.

Shimel: So talk to me. Give me the three biggest things that get you excited on this release, Chris.

O’Malley: So, each quarter, we’re turning the crank in terms of trying to nail the jobs that are changing, right? As I explained earlier, when you’re going from waterfall to Agile and from selectively automated to intrinsically automated, you’re trying to go from maintenance to innovation, the jobs that people have to do change radically. And they change radically in ways that are difficult to achieve. I mean, when you’re working on the mainframe, you’re working with large code bases, large in terms of the quantity of those code bases and also the size of any one of them.

So these techniques, at a high level, just seem obvious, but the practical things that need to be done from an innovation standpoint to help people to nail these things, it takes a lot of innovation. I mean, we’ve had to do a lot of work to get to the point where you’re seeing orders-of-magnitude improvement in the way people develop, the way people test, so this latest release is just another chapter in that story.

So we came out with Topaz team profiles, so that’s a good example of a capability that allows those that are working on systems to basically be able to store their configurations, the inputs and outputs of these applications, the test artifacts that are used, and build those artifacts and configurations in ways that can be shared broadly within teams or across the organization. That’s important for a couple things – one is the work to do that, you wanna do it once and then do it once for every one ’cause, in the DevOps mindset of continuous improvement, you wanna get rid of things that aren’t value-added, things that customers won’t pay for.

So, in creating this capability, we allow now, as teams get spun up, right, you get a team that’s gonna work on a two-week sprint, you want them, out of the gate, to be productive. So being able to share profiles day one, with people that may have never worked on these systems before, is crucial so that you’re productive in the first minute of the first day.

You also have situations where you’ve got, instead of technical debt issues, maybe a team, a large team, that you wanna go after certain work. It’s gonna help to improve the future throughput of the incremental efforts you have, from a development perspective, in the future. And you’re gonna bring large groups of people to work on something and that may be an outsourcer that you’re engaging; it may be new college recruits that you’re engaging. But you, again, want those people to be as productive as possible, as quickly, so, if you’ve got an experienced person on your staff that can create all those configurations and then share that broadly, again, you’re gonna lessen the ramp-up time for them to do their effective work.

So, again this is another example of us thinking about the mainframe going from waterfall to Agile, at scale, and having a scenario where it’s not just people that have been on your staff for 30 years, working on the same subsystems of your applications, every day for their entire career. Now, ’cause ideas are paramount and where people work has more to do not with what they know but what the company wants to do in better servicing their customers, so there’s more of a randomness to the work and a changing nature to the work, and what you work on is fluid. So we’re really thinking deeply about the implications of that and, for profiles within Topaz, that’s a major step in the direction of getting a different way of working within IT.

The second thing that we’ve released is another Jenkins plug-in. Jenkins is a central part of CI/CD pipelines, in terms of the automation side of it, and we’re constantly thinking about “What are the things or plug-ins that need to be created to make the mainframe equal to any other platform in all ways, other than syntax?” So JCL, Job Control Language, is a very important part of how programs are run within the mainframe. Creating a plug-in to help bring it into the CI/CD pipeline efforts and the automations efforts was a crucial thing that needed to be done, so it’s a contribution, from an artifacts standpoint, that, again, takes another step for our customers.

And then, lastly, is in the area of testing. Compuware is the leader, if not exclusive, provider of unit and functional testing as a combined offering. It’s a necessary part of the job that people have to do within Agile, with this concept of shifting left. The developers aren’t just coding; they’re coding and testing, both in terms of bugs and performance, and becoming much more extended in terms of their role. And so their accountability and responsibility is bigger.

So they need tools, like unit and functional testing tools, to participate within a two-week sprint process so that their deliverable, at the end of that cycle, is one that’s close to bug-free as possible and not putting future work into the system where it comes back as some quality issue 6 months later, 12 months later, when it runs into production, or, you know, as usability concern. So these three things are another turn of the crank, if you will, and Compuware’s offered to help customers nail these jobs, these new jobs that are created within kind of Agile work environments and DevOps work environments.

Shimel: Excellent. Anything else on the new release before I go off on another direction?

O’Malley: No, it’s just we’re very excited ’cause we released a technology called “zAdviser,” six months or so ago, which is a method of collecting what business KPIs for customers and then looking at, actually, stats relative to our products, so we can see kind of the behavior of developers. And we’re working with companies collaboratively to treat their people as they should, like high-performance athletes rather than slaves in a Roman ship galley. You can say, “Work harder. Work faster.” Well, you’ve gotta be constructive in helping to share best practices and methods that have a positive effect in things like velocity, quality, and efficiency.

So, with every release, we’re seeing a significant increase in this KPI performance, so, as we’re talking about earlier, the mainframe is certainly on a march to become a first-class citizen within the digital age, within these enterprise customers, but the more _____ –

[crosstalk]

Shimel: I’m sorry; go ahead, Chris. Go ahead.

O’Malley: Go ahead.

Shimel: Well, no, I was gonna say, you know what I find interesting is, yeah, we’re talking about the mainframe machines, but so much of your focus is on the people, on the developers, on the team, because, at the end of the day, that’s the gold. Right? That’s the almost irreplaceable asset, are the people and the developers and the IT teams that are working on these mainframes. And it’s about empowering them. Right? So I think that it’s the right place to put the focus.

O’Malley: Yeah, people –

Shimel: Yeah, I –

O’Malley: Just to restate the obvious here, but people matter most. When you think about the mainframe in the context of Agile and DevOps, asking people to change their role, though they’ve been around for 30 years, 40 years, dramatically – I mean, the day in the life in Agile and DevOps is vastly different than the day in the life of status quo. And it’s important that, as leaders, you’re promoting the appropriate culture and getting people to understand why they need to change, important that you’re giving ’em the necessary education, teaching them how to be effective in Agile sprints. I mean, that’s a different way of working. I mean, it’s not that you’re working 24 hours a day, but you’re definitely working in a more intense way. And then products matter then. The tools have gotta be enablers, not things that fight against your efforts to adopt this new culture and Agile nature.

And, you know, I really try to express to IT leaders, “You gotta think about these things ’cause – ” you know, we use this example, the Gallup survey. Gallup survey have been doing this thing for 20, 30 years of employee engagement in large enterprises, and what’s most interesting about it is, over 20, 30 years, the employee engagement statistics haven’t really changed. And it’s disturbing when you actually hear about them. Two-thirds of employees in most large enterprises are not engaged, half of them are kinda, “It’s a job; I don’t care that much,” and 16, 17 percent are actively disengaged – they try to sabotage the efforts of the companies.

So one of the biggest problems is engagement and employee engagement, and so I try to get people to understand – people matter hugely. And you’ve gotta give ’em the culture necessary to thrive in this more competitive age, you’ve gotta give ’em direction in terms of the process, the way that work is done, but you gotta give ’em the tools. You know, asking ’em to use mainframe tools out of a NASA movie from 1965 and thinking that something’s gonna happen is incredibly naive, so –

Shimel: Absolutely. You’re not landing people on the dark side of the moon with that. Chris, we’ve got a few minutes left. I wanted to touch on another subject – we didn’t really kind of pre-talk about it, but I’m sure you’ve got some great views on it – I was out in Seattle last month for KubeCon, you know, Kubernetes, cloud-native. It’s a funny question to ask the mainframe guy the cloud-native question, but, certainly, containers are no strangers to mainframes.

O’Malley: I mean, they’ve been on the mainframe for –

Shimel: 20 years, maybe more, right? 25. So anything specific to Kubernetes in the Topaz or Compuware kinda roadmap that you can discuss right now? Or what effect do you see on your world?

O’Malley: Yeah, so, when you think about kind of the flow of work in large-enterprise IT and what developers are fixated on, I mean, part of it is innovation and functionality that they’re trying to build, which is most important. I mean, that’s the value-added efforts of developers that make a difference in the eyes of the customer. And you wanna do that as quickly as you can, with the greatest degree of quality.

But the other things you gotta think about are things like quality itself. I mean, if you get anything that you release that’s got bugs in the system, how rapidly you can respond to it. That’s always part of the work that’s being done and I don’t know of any organization that doesn’t have some degree of bugs. And bugs may be software written improperly – and, again, our tools are helping to fix that – or usability issues, integration into other systems.

There’s also the work of compliance and security that you gotta do, but this last thing is this thing, technical debt, right? That any code base that’s had any life to it, has meandered through the efforts to serve customers, it’s something that grows over time. And it has nothing to do with mainframe; it happens everywhere. You know, the cloud – you know, Amazon’s got debt. And you gotta deal with that technical debt, so, as you’re dealing with technical debt, the smart approach is to start using more of these modern techniques. SOA was kind of a philosophical approach to breaking down these larger systems into smaller subsystems. When I think about API enabling, REST Y enabling these kind of smaller chunks, but you also wanna think about containerizing, using these as a form of containers, Docker being the method of creating the container, Kubernetes of being the way of managing these.

So we do see this playing an increasing role _____, as people start really looking at affecting technical debt ’cause that gives you the biggest bang for the buck. I mean, you certainly wanna do it with any new functionality you’re doing, but you’ve got some huge numbers of lines of code that you’ve developed over four years that works awesome. I mean, it’s really good working code, but making it more manageable and getting rid of that technical debt. Part of that equation is RESTful APIs; it’s containerizing it and managing it more in a cloud-like fashion. So, yeah, you’ll see us enabling those tools.

Our approach has always been to go to the edge of the mainframe and then we stop. We take care of what we know best and getting rid of these esoteric differences, but then, when we go off the edge of the mainframe, we work with all these other vendors. We allow SonarLint and SonarQube to do its work on COBOL, as akin to what it does with Java or C. Parasoft is a testing framework, allowing our products to work within that orchestration.

So there’s all these tools that work in kind of the DevOps world that are preferred and we’re enabling them, so you should think of us doing things in support of Docker and Kubernetes over the long term. But, yeah, it’s an important part of making mainframe IP one that the throughput can constantly be improved and those techniques obviously help enormously.

Shimel: Absolutely. Chris, I’m sorry to get you going ’cause it doesn’t take much when we start talking mainframes and DevOps, but we’re way over time and I need to wrap up. Hey, we’ll have you on, certainly, next quarter for the – what will it be? – the 45th time?

O’Malley: [Laughs] We’ll be doing this and it’ll be our 1,000th. You and I will be in wheelchairs, talking about it.

Shimel: By then, you should be down in Florida with me, but, anyway, Chris, it’s a pleasure having you on. Continued success with Topaz and Compuware. And we’ll catch up with you soon?

O’Malley: Always a pleasure and anytime I can help.

Shimel: All right.

O’Malley: And thank you for the hard work that you do.

Shimel: Thank you.

O’Malley: _____ important _____ –

Shimel: All righty. Chris O’Malley, CEO, visionary, leader at Compuware, on DevOps Chat with me today. This is Alan Shimel and you’ve just listened to another chat.

— Alan Shimel

February 21, 2019
DevOps Chat: AIOps and APM with Unravel’s Kunal Agarwal

Unravel is making a name in the AIOps, APM and big data markets. In this DevOps Chat, we speak with Unravel CEO Kunal Agarwal about what makes Unravel a must-have for its customers. We also take a sneak peek ahead to what the near and not so near future holds for big data, AIOps and the tech scene in general.

As usual, the streaming audio is immediately below, followed by the transcript of our conversation.

Transcript

Alan Shimel: Hey, everyone, this is Alan Shimel, devopsy.kinsta.cloud, and you’re listening to another DevOps Chat. Today’s DevOps Chat features Kunal Agarwal, CEO of Unravel or Unravel Data. Kunal, welcome to DevOps Chat.

Kunal Agarwal: Thank you, Alan. Thank you so much for having me here.

Shimel: Excellent. It’s my pleasure to have you here. So, Kunal, let’s get right to it. “Unravel” is the name of the company, correct? Not “Unravel Data.”

Agarwal: “Unravel,” yes.

Shimel: But yet Unravel is about data to a certain degree. I don’t know if everyone in our audience is familiar with Unravel. Why don’t we start there?

Agarwal: Sure. We are unraveling the mysteries of big data, Alan. That’s why the name of the company.

Shimel: Excellent.

Agarwal: Everybody’s jumping on to big data apps, and what I mean by that is you may be doing something with machine learning, artificial intelligence, IoT, even big-scale data warehousing. And there’s just so many problems that can happen, running these applications at scale, that people just have not seen before. And this is a problem because of all these new systems and technologies that are coming about, such as Spark, Kafka, Hadoop, Cassandra, HBase, many, many more. And the complexity’s all about “Hey, how do I figure out why’s my application not performing well or if it’s failing?” And it’s usually a multi-tenant environment that these big data applications run on, which means there may be hundreds of users, thousands of applications, that you are creating every day and running them.

And figuring it all out and making sure that they run well is a black art, so there’s a lot of trial-and-error that goes on into fixing these problems, finding the root cause of these problems. And that’s what we are automating, meaning Unravel, after it’s installed in your big data cluster, will be constantly monitoring and managing everything. And if there is an issue, Unravel will be proactive about it, tell you why that issue happened, what caused it, and then even take a step further where it would resolve those issues for you so that you can spend time creating all these amazing big data applications, rather than focusing on fixing all these systems- and application-level errors.

Shimel: Mm-hmm. So, you know, I mean, there’s a lot there that we’re gonna dive into, but before we do, Kunal, just give our audience – ’cause I always find this fascinating. People wanna know “How do I become a CEO? How does one get to be the CEO?” Give us a little insight into your own personal journey on how you’ve come to be CEO at Unravel.

Agarwal: CEO is “chief everything officer,” when you first start off. You gotta sweep the floors and you gotta raise the funds and you’ve gotta grow a team. There’s so many things a CEO’s role requires to do, especially in a young company, as you’re growing it. Today, now that we’ve grown to about 75, 80 people in total, across 3 different countries – we actually have offices here in the United States, one in London, and one in India – my job is becoming more about recruiting, more about getting the team together.

And I think that’s, personally, for me, if that is taken care of, the entire company just hums and does well. Everything else is secondary for me at this time, but I do spend a lot of time doing marketing and sales and finance and product. So it really depends on the day, Alan, of what my job requires me to do.

Shimel: Amen. You know, I’ve founded or cofounded three or four companies myself and even just running MediaOps, which is devopsy.kinsta.cloud and Security Boulevard, you are chief cook and bottle washer when you’re – you’re chief everything officer, as you said.

Agarwal: Right.

Shimel: You know, looking in your background, you’re a double-major graduate – electrical engineering, computer science master’s degree from Duke. I mean, you’re well positioned to be a CEO, people would say, looking at the background, but it’s – you know, you never know what’s coming out next as the CEO. If I had to ask you, Kunal, what’s kind of been the biggest surprise or biggest kind of curveball that you’ve seen in your time at Unravel that you think, “Geez, I didn’t think I’d be doing this or I didn’t think this was part of the job,” what would you say?

Agarwal: Alan, actually, going to school and getting all those double-degrees does not help you at all becoming CEO.

Shimel: You get it. I mean, you’re talking to someone who went to law school. I know all about it.

Agarwal: Right?

Shimel: Yeah.

Agarwal: I wish there was a school for CEO, but you know what? The only school for being CEO and leading a company is actually leading the company because there’s so many things that you learn along the way. One thing, because we deal in enterprise software, there’s so many complexities to making a deal happen with a big company.

Shimel: Mm-hmm.

Agarwal: And I think that was one of the things that took me a while to grasp and make sure that we’re doing it right. And I mean all the boring stuff, like the contractual paperwork, all the legalities, going through info-sec requirements. All the operational stuff around making a deal happen. You know, when you’re in college and you’re starting to think about a startup, you think about the cool products you’ll make. You’ll think about the impact your product will have in the market and how so easily everybody will flock and come to you and get your product. [Chuckles] But it’s not that simple.

Shimel: I wish it was.

Agarwal: and it takes 9 months to 12 months to close some of these deals, so I think, more than the “surprise” factor, preparing yourself and making sure that you can drive efficiencies in all these different processes. I think that’s been one of the key challenges for us.

Shimel: Absolutely. Absolutely. So, Kunal, if I’m a Silicon Valley VC, I’m looking at a company like Unravel, man, you’re playing in all the right places. Big data is certainly such a huge – I think, when we talk about things like machine learning and AIOps, really, they’re almost the children of big data – you couldn’t have those things without the big data, first of all – so you’re in the right place, obviously. The right time. This is a very happening kind of space. What is Unravel doing that is so – what’s the special sauce? What’s unique about it?

Agarwal: Yeah, Alan. So, when we created this company – and by “we,” I mean myself and my cofounder, Dr. Shivnath Babu, who’s actually a professor of computer science at Duke University – we both met at Duke – and the problem we were trying to resolve is one of the most fundamental but also probably the most complex and challenging problem to solve, which is “How does one ensure that all of these big data technologies actually work and deliver on their promise?” And, you know, our approach to the problem was always “Hey, there will be a shortage of experts in the market. Even if there are experts, these problems will be requiring a machine to solve them on time and efficiently.”

So we took the long path of actually creating not just a monitoring tool but a AIOps tool, if you may, which can decipher the problem, connect the dots, and resolve it for you. So it actually took us about three-and-a-half years to build this product or at least the first version of this product. And we moved west to start working with some companies – I lived in New York; he was in North Carolina – and we gave up our cushy jobs, moved to Silicon Valley, because you wanted to work with companies who were using big data tech at scale.

And some of these companies, we still have advisors in, from Zynga, Rocket Fuel, Twitter, LinkedIn, and they really helped us understand the day-to-day issues that come up when you’re running these technologies at scale. And we took a lot of those learnings into our first product. So we were always about solving a core need that teams in big data tech would definitely need. And, you know, they always say, “Hindsight is 20/20,” right?

Shimel: Yeah.

Agarwal: But we always put a big bet that, “Hey, these technologies – Hadoop, Spark, Kafka, et cetera – will become mainstream and companies will start using this a lot more.” So what LinkedIn was doing five years ago, the biggest banks would start doing it now, and that’s where we are in the journey today.

Shimel: Absolutely. And, you know, it’s funny, full circle, talking about that journey and what you’re doing. That’s how you become a CEO. It’s about being willing to take the risk, giving up those cushy jobs. Right? To do something that you really believe in. And I think that’s what separates CEOs from people who kinda look and say, “Boy, I’d like to do that,” but they don’t actually do it.

Let’s talk about the market. We’re coming to the end of the year. You’re participating in our Predict 2019 panel, which will be December 18th event. You’re gonna be on the AIOps panel, looking ahead. Why don’t you give us a sneak peek? What’s kind of your take on what 2019 holds for AIOps and big data and so forth, Kunal?

Agarwal: Well, yeah, Alan. So AIOps is an evolution of IT Ops or APM or DevOps, where you’re using machine learning to, again, understand all of the data – and this could be monitoring data, instrumentation data – that all these systems and applications are generating to draw insights from.

Now, funnily enough, this is what our customers do with their data set, right? So with some of the biggest banks, some of the biggest insurance companies, some of the biggest health care companies, all Fortune 500. And our methodology of going about monitoring for big data stack was always the same, which is “Why aren’t we using big data technology on our data set, which is machine-generated data, to glean those insights and drive some prediction and actually make things simpler and more accurate?”

So AIOps is also one of the use cases of artificial intelligence and machine learning. Now, just as I’ve said the industry’s evolving, what we predict to see is the evolution of AIOps, so Unravel, for example, is already on one of the end-of-the-spectrum places, where we actually make decision on behalf of the operations and the development teams, whether it is re-evaluating an application, whether it is making sure that your SLAs for your applications are being met, by moving low-priority jobs to a sub-optimized queue and high-priority jobs to a priority queue or killing bad applications in the cluster, all on your behalf, to make it more proactive.

But what I predict is there’s gonna be an evolution in just the incumbent APM companies as well, where it’s not just about giving people a monitoring screen or even a native monitoring screen and saying, “Hey, check out all the information we know about an application,” but then I leave it up to you to decide and understand what’s amiss, what’s wrong, and how do you fix it. But I feel we’re already now in the correlation and the insights phase, which is now I can start to connect the dots, with these new software technologies that are out there. But, in 2019, we’ll start to also see some of these companies take the leap to be able to do automated root cause analysis, as well as guided remedies or automatic solutions, to get out of these problems as well.

This industry’s evolving at a very fast pace, so I’m super excited. And it’s all about making sure that some of the biggest applications in the world, which millions of people actually rely on, all work like a well-oiled machine. And it’s all about guaranteeing reliability, which, at the end of the day, improves confidence of the customers using those products, which further fuels more innovation and more growth. So it’s an amazing cycle and AIOps is gonna be a very, very key part of it.

Shimel: No doubt about it. No doubt about it. You know, in many ways, I look back on 30 years in IT and it really is a golden time to be alive, if you’re a developer or ops person or just in IT in general. Right? There’s the possibilities and the tools, and what’s in front of you is so exhilarating, right? I mean, I think back, if I was 20 years old now, 22 years old, coming into the –

Agarwal: So you mean like 10 years ago, Alan?

Shimel: Yeah, something like that. About 10.

Agarwal: Okay.

Shimel: About 10. And, I mean, just unlimited opportunity. I mean, that’s kinda almost the way I look at it. And, of course, it’s companies like Unravel, right, that are kinda letting these opportunities take shape and packaging them for people to run with. So great stuff. Specifically, Unravel – we only have a couple moments here left, but – so, Kunal, anything on the horizon with Unravel you wanna share with our audience?

Agarwal: Yeah, Alan, absolutely. So, even in big data technologies, there’s so many to cover. We started our life off supporting Spark, Kafka, the entire Hadoop stack, some NoSQL systems, like HBase and Cassandra, and some newer NPP platforms. In 2019, we’re gonna further expand some of those technologies and systems that we cover, especially as we start to see more and more big data workloads move to the cloud.

We’re also seeing a lot of our companies demanding a hybrid real estate management from the cliched “one pane of glass,” that “Can I see everything that’s in my big data environment, whether that’s on premise and on the cloud, together?” So Unravel is launching that support as well.

And, last but not least, we are also creating a solution, from our product, which is directed towards helping companies move from their on-premise environments to cloud environment. So actually using Unravel as a migration project management tool, to help you assess, migrate, and then validate your migration. And, on the assess side, figuring out “What system should I get? Which instance type is best for my workloads? How can I keep under cost? What is the cost of running my big data workloads on the cloud?” to then validating and making sure that your SLAs are met, that you’re under cost and under time, and everything is flowing like a well-oiled machine.

The way we look at it, Alan, again, the way to look at our road map and anything on the horizon is we are here to help our customers in the entire journey of big data tech, so, whatever they’re deploying today and they wanna use tomorrow, Unravel will be supporting that. And this is a natural evolution where companies are going from trying something out to getting something in production, first starting on premise and then going to the cloud, and those are all the shifts and transition points that we wanna hold the hands of the customers that we work with.

Shimel: Excellent. Excellent. Well, Kunal, thanks for spending a few moments with us. As I told you, the time goes quickly.

Agarwal: Yeah.

Shimel: We’re way over time, but it’s okay. Hopefully, people have stayed on till now to listen. I wanna wish you lots of continued success with Unravel. We’ll be checking in on it. Again, a reminder: Kunal will be on our December 18th Predict 2019 virtual event and Kunal will be participating in the CEO panel, taking a look ahead at AIOps in 2019. So join us. You can register for that at predict2019.com.

Agarwal: Thanks, Alan.

Shimel: Thank you. Hello to all of our friends at Unravel. And that’s gonna make a wrap on this episode of DevOps Chat, everyone. This is Alan Shimel for DevOps Chat, devopsy.kinsta.cloud. Have a great day.

— Alan Shimel

January 4, 2019
DevOps Chat: A New Take on Application Security with Portshift

Portshift brings a new identity-based application security model from code to runtime. Cloud or on-prem, Portshift works. In this DevOps Chat we speak with CEO Ran Ilany and VP Business Ops Eran Grabiner about what is unique to the Portshift solution and why you should consider it for your own application security, especially for your DevOps teams.

As usual, the streaming audio is immediately below, followed by the transcript of our conversation.

Transcript

Alan Shimel: Hey, everyone, it’s Alan Shimel, devopsy.kinsta.cloud, Security Boulevard, and you’re listening to another DevOps Chat. In today’s DevOps Chat, we actually have a new company to introduce to our audience anyway and that company is Portshift. And I’m joined by two of the key folks at Portshift. We have Eran Grabiner, who’s VP of Business Ops, and Ran Ilany or Ilani – I mispronounce your name, I apologize – Ran Ilany, CEO and –

Ran Ilany: Ilany.

Shimel: Ilany, great. CEO and cofounder. Gentlemen, welcome.

Ilany: Thank you.

Eran Grabiner: Thank you, Alan. Thank you for having us.

Shimel: Oh, my pleasure. So, Ran, you’re the CEO. I’m gonna let you kick off. A lot of people in our audience may not be familiar with Portshift. Fundamentally, what’s Portshift about?

Ilany: So, fundamentally, Portshift is trying to solve – solving a problem which is a hard problem that wasn’t solved in the last couple of decades and was very much introduced as a solution, during early ’90s, with the firewalls technology. And now we’re moving to the cloud infrastructure, we see that the same firewalls that were used 20 years ago are not longer, I would say, valid, in order to run a application. And, essentially, there are, I would say, two to three reason that to that and I will let Eran elaborate about the reason of why.

Grabiner: Yeah. Sure, so, you know, Alan, when speaking about the cloud, there are two things that fundamentally changed everything for IT folks and DevOps engineers. The first thing: you don’t own the infrastructure, so the level of network and infrastructure, you simply don’t control it.

And, secondly, a lot of change through time in the world of compute. If – I don’t know – ten years ago, we had the monolithic application sit down in our data centers in the office, right now, when we’re in the cloud, our application is broken into many different pieces, many different _____ services. Everything is much more automated. Software is being updated on an hourly basis. And developers and engineers, they have much more power to do things with their software. So, because those two reasons that the old tools that we had, in order to orchestrate security in our network, simply doesn’t work.

Ilany: Yeah, and –

Shimel: Agreed. Ran?

Ilany: And just to add to those points, there are two layers that essentially impose the challenge. The first one is the fact that, if you’d really like to, I mean, to use firewalls within the cloud environment, you really need to understand the network infrastructure. To understand the network infrastructure, you need to understand what is the application is doing out there in the cloud, and, in order to do this, you need to configure those IPs and those ports, which actually create a very, very close binding to the actual network infrastructure, for one hand.

And, for the other hand, you obviously need to have this type of discussion between the security folks to the actual operation folks, which actually creates a lot of bureaucracy that was possibly valid, you know, 15 years ago in the data center, but, now we’re moving to the cloud and everything is automated with CI/CD, you need something else. Okay? You need to break the paradigm, at least from being tied to the actual network, from one hand, and, from the other hand, you need to make things to be much more fast and much more intertwined with the development life cycle. Which is essentially the primary objective that Portshift’s have put in as a target and as a solution.

Shimel: Absolutely. So, guys – and, look, I’ve been in this a while myself. You know, when we first started with the next-gen firewall, right, and that was sort of the premise, is that we were gonna get the developers involved and application-aware firewalls and all that, but it didn’t translate to the cloud. You’re right. But, Ran, I’m interested, as the CEO/cofounder, so this is obviously a problem, right? How do we do this in a cloud environment, at cloud scale, at cloud speed and all of that?

Ilany: Yeah.

Shimel: How did this affect you personally or in your kind of journey to where you’ve got here today, where you recognized and said, “Hey, we gotta do something about this, that this is a problem looking for a solution here”?

Ilany: Yeah. So, essentially, we understand that there is a problem that requires a solution, you know, while talking with customers. I mean, those are usually the No. 1 –

Shimel: That’s the way –

Ilany: Yeah, and –

Shimel: – the best way to do it.

Ilany: And we saw that there are – and we call it “the impossible tradeoff,” okay? They either need to be very, very fast – they want to run very, very fast in the cloud, from one hand, but then they have a security constraint. They really want to control, I mean, the communication between the application and where the application should run. And there, there is the where to be in the curve of being Agile and being secure every organization has to find itself, in this type of curve, and to find what is the level of security that it wants to enforce or impose.

And this is, I would say, what led us to think of changing the paradigm or breaking the paradigm or making a solution that the customer will not have to make this type of tradeoff or make this type of decision and target to be both fast and Agile but also secure from the other way around _____ _____.

Grabiner: And we basically looked at what’s going on in the cloud from the DevOps perspective and we saw that many things changed to be better, with automation, with CI/CD tools, and we ask ourselves why, with security, this is not happening? Why the workstation or the tools that the DevOps engineer is now using to deploy software, why security tools can’t look the same? Why they can’t be part of the same idea and the same work cycles? And this is what got us to think that we need to, in a sense, push security to the left, shift left, and take it to be something that is part of the DevOps life cycle and part of the CI/CD infrastructure of the organization.

Shimel: Sure. So, guys, obviously – so we call this “DevSecOps,” right? Is part of it.

Grabiner: Yeah, right. Yeah.

Shimel: It’s something that I – you know, between DevSecOps Days and events we put on at RSA and so forth, it’s something we’re very involved in. My question for you then is, so it sounds like Portshift is really aimed at helping DevOps engineers, DevOps teams, developers, get more involved in the security or in the cloud firewall kind of paradigm for security. Is Portshift aimed then at the security team, the developer team, or both?

Ilany: So I think – well, obviously, the focus of Portshift is not to try to educate Dev and DevOps to be security folks – this is obviously not the target – but think the good news is the fact that Dev and DevOps have the knowledge, okay? And the knowledge is where, essentially, they know their intention. They know where, essentially, they want to deploy the application and, essentially, who should talk to who, in terms of application containers or processors, depending, obviously, on your infrastructure. And this is, I would say, the basic assumption that the whole concept of Portshift is based on, okay? We actually take this type of knowledge, okay? We create from this knowledge or this attributes, we create the actual identity of the application, which is essentially a signed infrastructure or crypto-signature of those attributes. And we use those intentions that are described by DevOps, in their own natural language – I mean, a recipe in Cookbooks with, you know, _____

Shimel: A Chef or a Puppet.

Ilany: Yeah, of course. Or Ansible or Kubernetes, obviously, infrastructure. We use this type of information while creating the crypto-identity, in order to deploy those application or signed application in the cloud and enforce the actual security that we describe formally. And the whole point is, essentially, that those folks don’t have to change the way that they work, okay? The security is more intertwined with their own system, the CI/CD system.

We leverage this type of information in order to create the security and the actual security folks will actually define guardrails. Okay? They will define operational environment in high level – the production environment, the testing environment, or the preproduction environment – where the actual application should run within. And the security will be sort of enforced and be visible automatically, without the pingpong, okay, or without the never-ending discussion between the security and the DevOps folks, which we all know it’s part of the problem, okay? It’s part of the challenge of securing the application in the cloud prams.

Shimel: Absolutely. So I guess this – well, let me back up a sec. I just wanna make sure. Portshift, I know you work with AWS, obviously.

Ilany: Mm-hmm.

Shimel: Microsoft Azure, Google Cloud as well?

Grabiner: Yeah.

Ilany: Mm-hmm.

Grabiner: We’re totally agnostic to the infrastructure.

Shimel: Okay. And what – I’m curious ’cause this is something that we going back and forth on, back and forth on. The attitude of the developers, the DevOps teams. I mean, you never meet a developer who says, “Oh, I’d wanna make an insecure application or I want my stuff to be hacked,” right?

Grabiner: No.

Ilany: Mm-hmm.

Shimel: And you’re right – they’re not security people, but they’re security-aware and they need to be security-responsible.

Ilany: Mm-hmm.

Shimel: Right? On the other hand, look, we all know security people who’ve said, “The developers will do what I tell them,” right? “And they’ll be secure when I tell them they’re secure.” But that doesn’t work either in this new world. Right? It’s gotta be together. So you talk about integrating with the CI/CD process. Give us an idea of how you work within the CI/CD process.

Grabiner: Got it. Yeah, so, basically, what we’re doing, we are asking the developers, the DevOps engineers, to use a very simple set of APIs, whenever they are using their infrastructure as a code or whatever the tools that they’re using, to send Portshift the relevant information that we need, in order to identify the piece of application that they’re deploying. This information can be technical information about the build, the version, the namespace of the application, and other attributes, that will allow us to rip up an identity that is unique, for each and every one of the pieces of the application. So this is the first part, where we learn your application logic, where we understand who should speak with who, and we know your business.

Now, when we take this identity and we move into runtime, we see all the different pieces and, first of all, we see them for sure. We know because we find them during CI/CD. If something didn’t go through CI/CD, if a developer deployed a container without asking, if, god forbid, an attacker is moving in the network, we would immediately see that because we signed the components during CI/CD. And we can see them. We can see them communicate. And now security folks and also DevOps engineers, they can easily define their agenda and their rules, based on their application logic.

A simple example: if they want their billing app to speak with their financial database, something that can take days just to config because your application is broken into different pieces and you need to config security groups and make sure you did it right and so on, with Portshift, you would simply write this sentence. You would just say, in English, “I want my billing application to speak with financial database,” and that’s it. Everything will work from then on.

Shimel: So it sounds magical, right?

Grabiner: Yeah.

Shimel: You know, I’m too old to believe in magic. What’s going on behind the scenes, though? When I write that, what’s actually happening behind the scenes?

Ilany: Yeah. So, essentially, what happens behind the scenes is we have two phases for this identity bit generation. As eloquently mentioned by Eran, the first phase is done by offline, okay? We take this type of attributes, which are sort of the intention, okay, in the CI/CD. We grab those as attributes of the identity and this is essentially the description of the application – wherever the application should run, who should it talk to within the actual environment that we’re targeting to.

And then we have the correlation with the runtime properties. Every process, every containers have runtime characteristics – for instance, the process ID, the UID, memory footprint, and stuff like that – so we combine all together the offline attribute that was created with the ICD, with runtime attribute. We sign them all together.

And this essentially creates one unique entity that is running out there and this is what we’re using in order to enforce. We are transferring this type of identity in the traffic stream so that a conversation between two different application is, first and foremost, authenticated, okay, and only then authorized. Okay? This is what we call the classical “zero-trust” approach, okay? We’re not trusting the IP; we’re trusting the identity on the counterpart.

And this brings another where we talked, I believe, a little bit about the operational benefits and the fact that it’s very Agile and useful for the Dev and DevOps and the security collaboration. But there is another very important advantage and it’s the fact that, now that we are not talking about an attack surface which are segments, network segments, we are talking about type of access. Okay? So, if an attacker, for instance, get into one of my workloads or one of my application, we are not losing the whole segment, okay? We are losing a single part.

And now we have, I would say, less – we’re much less concerned from the extent that we are exposed, okay, because we are talking about only point-to-point access permitted. They’re not network segments, which are open and, from the attacker perspective, it’s like a dream come true. If I find a flat network and I get into your flat network in some miraculous way, then I can get into every resource or application that is out there.

Shimel: A good target.

Ilany: Yeah. And this is, essentially, the basic concept of the security perspective of identity-based access control between different applications.

Shimel: Sure. Guys, it sounds fascinating. As I mentioned to you when we started, the time goes very quickly. We’re kind of at the end of our time.

Grabiner: Yeah.

Shimel: I apologize, but maybe we can continue the conversation. For people who are interested, Portshift, you can get more information at portshift.io. Correct?

Ilany: Right.

Grabiner: That is correct.

Shimel: And you guys will be at AWS Reinvent, though –

Grabiner: Correct.

Shimel: – _____ _____ after. Will you be at RSA conference?

Grabiner: Yes. We will also be presenting there, at a startup venue.

Shimel: Oh, the Launchpad or the Sandbox?

Grabiner: The – I think it’s called – I think it’s the Launchpad. Yeah, it’s where all the early-stage companies are. Yeah.

Shimel: Yep. Sandbox – that might be the Innovation Sandbox. Launchpad is more for ideas.

Grabiner: Oh, yeah.

Shimel: I think you fit more in – yeah, we’re a big part of RSA, so, yeah, I just did a podcast with them about both of those, so little bit I know. Anyway – I’m sorry. Thank you very much for joining us and we will – if we don’t see you at AWS, we’ll see you then at RSA. Well, we’d like to probably hear more because this kinda fits right into our sweet spot. Ran, Eran, thank you for joining us on DevOps Chat today. Good luck with Portshift. This is Alan Shimel for devopsy.kinsta.cloud. Have a great day, everyone.

— Alan Shimel

December 17, 2018
DevOps Chat: Container Security and Aqua 3.5 with Rani Osnat and Andy Feit

In just a little more than three years Aqua Security has set its mark in the container security space. With its major new release of Aqua 3.5, the company has again raised the bar with serverless and container encryption upgrades and feature sets.

I sat down with the Aqua Enforcer himself, Rani Osnat, and “Boston” Andy Feit to discuss the details of this major release. Rani and Andy give us an inside peek.

As usual, the streaming audio is immediately below, followed by the transcript of our conversation.

Transcript

Alan Shimel: Hey everyone, it’s Alan Shimel, devopsy.kinsta.cloud, and you’re listening to another DevOps Chat. Today’s chat is a little bit about cybersecurity, container security, Kubernetes and a major new release coming out of our friends at Aqua, Aqua Security. I’m happy to be joined with the dynamic duo of marketing at Aqua, Andy Feit and Rani Osnat. Andy, Rani, welcome.

Rani Osnat: Hi Alan, thank you.

Andy Feit: Hi Alan, good to talk to you.

Shimel: And just so people know, Andy, you’re joining us from Boston today. And Rani, you’re out in Israel. Is that correct?

Feit: Yes. Yes. Today I am in _____ _____ [Crosstalk].

Osnat: Right.

Shimel: Modern technology. We have a worldwide panel. But guys, the big news is, Aqua just announced version 3.5 of their platform suite of tools. And you know, Aqua’s not a company that every single new time there’s a new release, and the DevOps mantra, you can’t get too excited about any one release. Right? Because there’s always a next one and a next one and a next one. But this is one to get excited about, huh?

Feit: Absolutely.

Osnat: We certainly think so.

Shimel: So why should we be excited?

Feit: Rani, you want to take that one?

Osnat: Yeah, I’ll take this one. So, with every release we make, especially, and in the beginning of course everything’s new. But we’ve been in this space now for three years. Which is not a long time, but in this space it’s a very long time. And so now we’re at the point where we have a lot of large enterprise customers using our product. And we have a market that’s looking for innovation.

And with every new release we try to balance these factors of, you know, offering something new that the market wants. But at the same time ensuring that our enterprise customers can make use of our platform, as they themselves grow their cloud native container implementations. So there is a maturity factor here as well as an innovation factor.

And so on the innovation side, we’re introducing a few significant innovations. First and foremost, risk assessments for serverless functions. Which is a, you know, a kind of a sideways expansion for us into the serverless technology space, in addition to containers. Thinking that you know, what we see as, it’s basically the same teams and the same benefits that are gotten from containers people expect to get from serverless, it’s just another means to get the same end.

And so we want to provide our customers with all the controls they need to address any challenges they have around securing those technologies. And it doesn’t matter if they use containers or serverless or both, or any sort of mixed environment. We also added something that’s innovative in the space, which is container encryption, and we can talk about that.

And then on the side of enterprise scalability and ease of use, we’ve added quite a significant I would say rearchitecting of how we manage both administrative controls on our platforms and what users can do in terms of access. As well as the policy engine to make it a lot more scalable for multicloud, multiteam, multiapplication use.

Shimel: Got it. Got it. Andy, did Rani leave anything out you want to add?

Feit: No, I mean, those are the big pieces that are in the release. As he said. I mean, it’s very much being driven by our customer base and where they’re headed. And you know, in some aspects, it’s the technology elements. Like adding serverless. And in other aspects, it’s really about living with the solution. You know, as our customers, we now have customers that are we think the largest container deployments in the enterprise.

We have some very large users and span of different industries. As they look to roll out, they’re finding they have multiple teams working on these projects. And they need to implement different levels of security. And in many cases, they’re implementing that on different technology stacks. And some are using containers, some are using serverless environments.

And they may be using even different underlying providers of some of the infrastructure. Whether that’s tools for development or the cloud provider itself. And so we’re really, we’re becoming very heterogeneous in terms of what we need to support. And for our customers who, on the security side, are trying to look at that whole context and manage that whole context and have consistent policies, all our consistent reporting across all of them, and not be monitoring 17 different dashboards. It’s important that it be easy to do, to be able to see that whole network of activity. And that’s really what a lot of this release is about.

Shimel: So guys, one of the things that you mentioned, both of you actually mentioned, was the serverless piece. And this is something that we are, we’re hearing a lot about from our readers and from people we speak to. You know, how quickly, and we live in crazy times, right? So the whole container revolution, if you backed up on hypervisors. And now, how quickly serverless is you know, gaining a foothold and people are building around that type of infrastructure.

Osnat: Right.

Shimel: Let’s, just baseline, what are some of the security challenges that you guys are seeing around this?

Osnat: Right. So I’ll preamble that, you know, just add one more thing. There’s, you know serverless has been around almost as long as containers have, in terms of the, you know, its current incarnation of the use in the cloud. But, the use cases is quite different. And while it is gaining traction, the use cases are much more limited than containers. There’s a whole kind of religious war, you know, between the proponents of containers and the proponents of serverless. Personally I don’t believe it’s a zero sum game. I think that both are going to end up being used, and both are going to end up being used in hybrid architectures.

Shimel: Yeah, I don’t think it’s either or.

Osnat: Yeah, I know, but some people would like you to think it is. I don’t think it is. I think it’s both. And so we, but there are some fundamental differences between containers and serverless when it comes to security. First of all, most of the serverless workloads that happen today are cloud based and specific to a cloud provider, right? Mostly Amazon, because Amazon is the larger cloud provider in general, as well as in serverless specifically with Lambda.

But basically you run those functions and it’s quite cloud specific. That’s one area of difference. The other is of course that you know, these are very small single-function entities that can run for a fraction of a second. So when we talk about run time security for serverless, for example, there is a lot less to do there than you have with containers.

Containers are, there are applications that can run, they could run for a minute but they often run for a lot longer than that. With something that runs for a split of a second, there is really only so much you can do when it’s already running. So a lot of the stuff you need to do is before it ever runs. You want to make sure that what you’re running is actually something that is secure, that passed your policy, and that you know about, right?

So when you look at the challenges, one of those is actually discovery. What functions do you have in your inventory, and what are you using them for? The second is ensuring that what’s in them is secure and not vulnerable, configured properly. And that also includes permissions. So, when you define for example a Lambda function to run on AWS, there is a set of permissions you can give it.

And depending on how you configure it, this can be very liberal or not. And of course you want it to be less, the least liberal you can. So as not to allow it to do anything outside the scope of what it’s supposed to be doing. So right now we’re primarily concerned, and that’s also where our customers are, with preventive controls. Basically ensuring that you are, that when you run something you actually know what it is. You know that you’ve approved it. And you can run it safely. That’s the first step in what we think is the right approach to securing serverless functions.

Shimel: Fair enough, fair enough. And back to what you said kind of right at the get go with the either or thing. It was the same arguments about, would container replace hypervisor type of thing. And I think what we see is the majority of people want containers on hypervisor.

Osnat: That’s true. It’s –

Feit: Yeah, and I would also add that the data centers with lots of VMware and hyper-v haven’t gone away yet either. And they won’t. They probably will never entirely go away. New development today is cloud native development. Most of your mobile customer facing development.

It makes sense to do that in the cloud and as long as you’re doing new development, why not do it on the latest and greatest. But, and the flexibility that you get from doing something on containers or serverless or a hybrid approach of those, makes it a lot smoother, easier, scalable. You get all these wonderful benefits. But we don’t see people going in wholesale and saying, I must clean out this data center now. So I think that –

Shimel: _____ [Crosstalk] throw out the baby with the bathwater. And people –

Feit: Yeah.

Osnat: Yeah. So I think that’s one thing that we did. The other, I think very interesting and innovative feature that we’re announcing is container encryption. And I want to explain the use case. So we work with, you know, we work with many organizations. For example, we work, some of our customers are ISVs, who create software for customers to use.

And that software is proprietary IP that they’ve developed and invested in. And they often want to make sure for those reasons as well as for reasons of governance, to make sure only the latest stuff is being used, et cetera, that they have some control over where it’s running and who’s using it. And so one of the things we provide here is an ability to encrypt, you basically encrypt the container image when it’s built.

And then in order to run it, you need to decrypt all its contents, so there’s basically a key that the customer will have, the end user will have. And that will decrypt the image as it’s running. And if you don’t have that key, the image will not run. So the container itself once it’s running is not encrypted. But up till that point, it’s encrypted and you have to decrypt it in order to run it. And then of course this prevents the sort of, someone reusing or forwarding your software and you can imagine various use cases except the ISV one where companies would want this sort of deep security control over their containers.

Shimel: Sure. Sure. And I think it’s just the, you know it’s funny, Rani. I think you said right in the beginning that over the last three years, you know, the progress that Aqua has made, and the progress the whole industry has made in securing their containers. Like everything that’s come before it, right, with each new release, with each light bulb, _____ security we see like more polish being applied.

So, to me this idea of container encryption is you know, maybe first we have to concentrate on just getting the containers to work in a somewhat secure manner. But now we can start thinking about next level, next, you know, how do we turn it up a notch? Things like container _____. What’s the word I’m thinking. Like the standard, the, you know, the container encryption becomes ubiquitous over time?

Osnat: I don’t know if it will become ubiquitous. I mean, certainly today you have a lot of encryption happening with TLS and all the stuff, all the pipes that handle containers. But as for the code itself, you know, even in highly regulated environments, I don’t necessarily see everyone encrypting everything. But certainly there are some crown jewels that you will want to encrypt.

And it’s especially those areas where you’re not talking about reusable components, right? A lot of the containers that our customers use are reusable based on open source components, that’s fine. You don’t need to encrypt your NGINX webserver container, right? That’s, it’s identical to the same, other NGINX containers people use. But if you have proprietary stuff, that’s where you want to, or something that’s especially sensitive, whether it’s for any kind of security, confidentiality reasons, or for IP reasons. That’s when you would want to use something like that.

But what you said actually, you know, brought me to the other topic I wanted to talk about, which is how the use of these technologies has matured with some of the more leading or pioneering companies that we work with, which is why we’ve invested in these scalability, manageability features. So I want to kind of explain what it looks like in a customer environment.

So when we started out, and we started out kind of together with the market, the companies we were talking to, even larger companies, would typically have one or two teams that actually use containers, right? And so, you were talking about isolated projects. And that’s one level of doing things. But as we move forward, you know, now we have customers that have hundreds of development teams working with containers. They make thousands of images on a daily basis, because it’s all in a CI pipeline.

And it all gets, it all goes into this large blender where it’s being spit out at the end, through Kubernetes or mesosphere, basically through an orchestrator to a variety of delivery platforms, whether it’s OnPrem or cloud or a mix of both. And if you look at it from an enterprise perspective, what they want is a way for them to apply security policies but maintain segregation of duties. Maintain compartmentalization and multitendency models that they have. So what we’ve done is really change both our administrative access model as well as the policy engine to enable that.

So basically, you’re talking about people who use the Aqua platform. They have one installation of the Aqua platform. But they can basically distinguish between what different teams can see and do, what different roles can see and do, and how those policies apply.

I’ll give an example. Let’s say that you have an application that is PCI DSS related. Handling credit card transactions. And so it has to comply with certain criteria that are requirements of the PCI DSS standard. You want, and you’re, but at the basis you’re using some basic elements like, I don’t know, MongoDB or MySQL. That same MySQL image that you apply a policy to within the PCI contexts will have a different policy than in non-PCI contexts. So having contextual policy is important.

So that’s one area. The other area is who’s got access to what. So of course the people who handle the PCI application, the developers, the DevOps team, will need to handle certain aspects of this. But they don’t necessarily need to control the policy. That’s expected to be done by the security and compliance team. And the same token, they don’t need to access, or other teams don’t need to access that PCI application.

So we’re talking about a matrix that really separates who’s got access to what and what they can do with that access, according to the specific application or geographic location, or what cloud it’s on or what registry it uses. All of these parameters. So you know, we’ve done this and we have, now both of these models combine to basically give you this matrix that is, that has you know, dozens of parameters on each facet to basically determine who’s got access to what, and who can do, whether it’s view access or edit access or only access to the logs, for example, for auditors and so forth. And so, it’s really a step up in terms of enabling these large deployments to work in an enterprise environments.

Shimel: Absolutely. Good stuff. Guys, I want to _____ two other topics or areas I want you to hit on. One specific to 3.5, one specific to the market. And we’re coming up on the end of our time. So you know, we hear about immutable infrastructure, and you know one of the advantages of a container environment.

So how do you deploy, you know, now you’ve got a bunch of customers. They want to upgrade to Aqua 3.5. Talk a little bit about how do you actually upgrade or run out this upgrade deployment. Is it a question of throw out what you have and just stand up new? Because Aqua itself is containerized. Or is, what, give us a little insight, nuts and bolts if people want to upgrade. Do you know what I’m saying.

Osnat: Yeah. So our upgrade process is you know, it is simpler because we use containers, right? So there’s nothing to install. You simply run the new container. We do have backward compatibility with our previous versions, so when there’s an upgrade path for anyone using older versions of Aqua to this version.

We actually, we’re the last, we don’t, I mean, we release software more often than what some of our customers would like to follow, in terms of their upgrade schedule, right? And we don’t force them to do that. But in the last version, the last major release, 3.2, we actually had a lot of customers move to that. So we’re in good shape from that respect.

And they can of course upgrade to 3.5 when it becomes available at the end of the month. So there’s really no, there’s nothing in particular they need to do except of course, we always, always recommend doing it in a test environment first. And then once you see everything works, you move that to production.

Shimel: Excellent. Excellent. So let me add –

Feit: There’s one more thing now I was going to jump in with, which is, we’re sort of talking about as our organizations of scale and especially if they have multiple teams that are working on different platforms and policies that need to be implemented. In some of these larger enterprises, it’s become very hard to really even see everything and know about it and track, you know, who’s running what and where, and who are these owners. And so one of the things that’s new in the product is a pretty slick interface.

Probably that’s the prettiest part of our product right now. The dashboard’s always looked great. But you can now go in and do what we call workload explorer. And visually view all of your running workloads, whether they’re on Kubernetes or Docker or OpenShift, and you can see everything that’s running across a large distributed runtime environment. And drill down into them, and you can start doing things like highlighting those that are vulnerable or high-risk, and those that are PCI compliant can be coded to jump out, and you’ll see that much more easily.

You can then do tabular mechanisms. Begin to filter. Show them only those running on this platform. Let’s say that there’s a vulnerability that’s exposed in a certain component. You can show me just those, and start drilling down and looking at them, seeing where are they running? Is this one on-prem? Is this one on the cloud? Which cloud provider is it running on?

So you know, as companies have become multiteam, multicloud, multistack, it’s the matrix of what you’re looking at has become pretty complex. And as workload explorers, it’s a great way to just sort of move around in a large enterprise. See what you have and understand what you need to work on right now.

Shimel: You know, and let’s not trivialize it. Let’s not belittle. I’m having a tough time with the word. Let’s not belittle it. You’re a hundred percent right, Andy. Today’s organizations are multi everything.

Osnat: I agree.

Shimel: _____ _____ distributed on so many different levels, that really, having a 360 degree view of, just of the lay of the land, where all this, how it all plays together. That in and of itself, someone could make a product out of. I mean, it’s crazy.

Feit: It’s eye opening for customers. Just in the last, we’ve been working with this product for about a month or six weeks, with some of our early adopter customers. And some of our sales engineers have been implementing it in proof of concept environments. And when the customers see this map, and they, you start drilling down, they say wait a sec – where’s that running? Show me that one.

They’re finding things that they didn’t realize they had because you can now visualize it and see it. You know, when it was coming in as yet another alert or a text field or you know, something coming in to your SIM system, that was one thing. You had to look at the logs and know the query to type. But when you see it visually and you jump in and start looking around, and says wait, what’s that?

And people were, it was eye opening. Okay, I need to go talk to that person. Literally, people were discovering things they didn’t know they had in their environments. And that’s exactly how, in a big environment, this tool can be very helpful in organizing things and showing you what’s out there.

Shimel: Absolutely, absolutely. Guys, that last question for you is not really 3.5 related but more mark related. You know, we seem to be undergoing a, well, it’s constant. But recently we’ve seen just more and more consolidation in the tech vector general within containers, around Kubernetes, you know, IBM’s Red Hat –

Feit: Cloud, right?

Shimel: IBM’s Red Hat acquisition, $34 billion. I mean, is the king of it. But we saw for instance, CA or Broadcom CA, whatever they call it now, spin on _____ for near a billion dollars. To Tom or Bravo. We saw, who, someone just bought, was it _____ just bought a –

Osnat: Later insight, yep.

Shimel: Container something.

Osnat: Right.

Shimel: I mean, at some point, consolidation to what Aqua’s been out here preaching for three plus years. How do you think that affects, I mean certainly now, 3.5, you’ve raised the bar again. Let everyone else try to catch up. But in this atmosphere, talk to us a little bit. What does that mean around Aqua? Or do you just keep doing your thing?

Osnat: Yeah, that’s a good question, Alan. I mean really, I think you keep doing your thing. Because your thing is understanding where customers are headed and building solutions that address what they really need. We’ve tried to keep our eye on that ball. And we’ve been big pushers of the concept of supporting multiple platforms, from the early days.

We were Docker first, but today we support all kinds of things including things, less broadly used technologies. But pivotal, and Microsoft containers. Supporting all the cloud. So you keep your eye on that ball with the goal of building a customer base that is satisfied and loves your product, is excited about working with you.

And one of the things that’s important to us is our customers are going to continue to renew with us and retain. You know, we’ve only been around three years. But we look at, our largest customers are very happy and using us, and expanding their use of us. They’ve put us in a pilot two years ago, and now they’re in production and they’ve put us in production with five applications. And now they’re on 50 applications. So this is all good. We’re not very focused on hey, we must be acquired by some time or we must be public by some time.

It’s, let’s build the right products, continue to see adoption. We’re growing very rapidly, both within our base and customers deploying more, and new names, new logos, that are using our solution. And that’s what’s important to us. And the market, you know, the market will be the market. If somebody says wait, I can’t let these guys get that much bigger. I’m going to try to snap them up. That may happen. But it’s not something that we’re really focused on. We’re focusing on building the business organically in the right way.

I have to say, we’re all old enough to have been through the dot com boom. And you know, the craze that went on back then. And I think it’s important to keep level headed. We’ve been around for three years now. We’ve just crossed the 100-employee mark. Celebrated that. You know, we work with our customers, we’re happy about that. We keep our heads, you know, leveled and we keep doing the work. You know, someone comes along and offers $35 billion to buy us, like Red Hat, I’m sure the board will approve.

Shimel: They might think about it as –

Feit: They’ll think about it a couple of days, you know. But in the meantime, we’re trying to keep things sane.

Shimel: I hear you. Guys, just one last reminder while I have you both on. I’ll see you both out at the TubeCon Native, cloud native, Linux foundation event, along with Red Hat, right? Sponsoring a Monday conference on container security. I think we spoke about it last time. And I just, I don’t want to mess up the date, but I thought it was December 9^th, or 10^th.

Feit: Nope, December 10^th. So it’s the Monday right before KubeCon starts, and it’s collocated with the event in Seattle. And you can register on the CNCF site when you register for KubeCon, or if you already did you can go back there and add on one of these collocated events. Ours is very focused on enterprise security, so it’s called KubeSec Enterprise Summit.

So for larger organizations, facing you know compliance requirements and deploying containers in production, it’s a great way to learn what others are doing. We have a great lineup of speakers. We have, JPNC is speaking. We have Tinder speaking. We have Starbucks talking about their journey towards secure containers. We have Liz Rice and Michael Hassenboss, who wrote the recent book on Kubernetes security. Which is not even published in paper yet, it’s still an eBook, but it will be first delivered in paper form at KubeCon. We’ll have them in our booth, and they’ll be signing books there.

So it’s kind of a great event that people can get exposed to all of these technologies, and hear from a lot of different people. We ran an open call for presentations, so we have everything from technology to compliance to best practices. It’s going to be a great event. We’ve got a couple hundred people already signed up. We only have room for 300. So, if any of your readers do want to attend, they should do that soon.

Shimel: Absolutely. Well, I’ll be there as well. We’ll be reporting for it. Andy, Rani, we’re way over time as usual. I ____ _____, we’ll need to call a wrap here. Congratulations on Aqua 3.5. I’m sure it’s going to be a great benefit, and the customer base will really appreciate it, and we’ll look forward to seeing you in Seattle.

Feit: Look forward to seeing you, Alan. It’s always great to talk to you. And share what’s going on with your listeners.

Shimel: Thank you. Andy Feit, Rani Osnat, Aqua Security. This is Alan Shimel for devopsy.kinsta.cloud, and you’ve just listened to another DevOps Chat.

— Alan Shimel

December 14, 2018
DevOps Chat: Security in the Cloud with Dome9’s Zolar Alon

Dome9 is in the business of cloud security, so it’s no wonder some of the world’s top companies rely in it to secure their sensitive data and applications in Amazon Web Services, Microsoft Azure and Google Cloud environments. And with last week’s announcement of Check Point’s aquisition of Dome9, an even greater number of users will be able to take advantage of its SaaS solution.

In a recent DevOps Chat, Dome9 co-founder and CEO Zohar Alon spoke with me about what he sees in the cloud security space as many organizations embark on their second generation of public cloud expansion.

As usual, the streaming audio is immediately below, followed by the transcript of our conversation.

Transcript

Alan Shimel: Hello, everyone. It’s Alan Shimel, devopsy.kinsta.cloud, Security Boulevard, and we’re here for another DevOps Chat. Today’s DevOps Chat features and old friend of mine from the security world, Zohar Alon, CEO and co-founder of Dome9. Zohar, welcome to DevOps Chat.

Zohar Alon: Alan, great to be here.

Shimel: My pleasure to have you here, my friend. Zohar, let’s get this out of the way first. Some of our listeners are not necessarily from the security world. They’re from the DevOps world. They may not be familiar with Dome9. So let’s first let them know a little bit. You know, you live and breathe it, but tell them who Dome9 is.

Alon: Sure. So Dome9 is a SaaS solution that helps security organizations and some DevOps and development organizations secure their public cloud environments as they grow and prosper from the single cloud to the multi-cloud focusing on AWS, Azure, and Google Cloud platform. A multitude of solutions.

We’ve been in the market from the very beginning covering elements like network security in the cloud which is a bit different, governance and compliance, continuance compliance, and how our customers can tie continuance compliance to their development processes and their build processes and things like identity and access management protection for public cloud that prevent attackers from targeting the developers as the weak link as a vector of attack for compromising the credentials in real time by hacking into their laptops and pigging-backing on their authenticated and trusted sessions to create mayhem and havoc in the public cloud environment of the customer. Just celebrated joining our hundred employee and going strong in both our headquarters in Tel Aviv and Mountain View, California.

Shimel: Absolutely. Very good. So Zohar, and I should mention, look, I know you and Dome9 pretty much from the year you started if not the day you started, right? I think it was an RSA event?

Alon: Yes, yes, 2012, very early on.

Shimel: I was going to say, you know, in a previous – you have a long history in the InfoSec world. Look, the topic for today is really have we seen, you know, the classic, you know, crossing the chasm model of early adopters, early mainstream, late mainstream, laggards, where are we with this in cloud adoption as it relates to cloud security? And at this point are we confident in saying that, look, it’s basically just the laggards at this point who haven’t already at least started their cloud migrations?

Alon: I can say this for certain, that the level – at Dome9 we focus on the top segment of the enterprise customers, let’s say the global 5000 organizations and their cloud deployments; and there’s not a single industry, there’s not a single geography that we are not seeing a race to the cloud, in many cases, multi-cloud deployments very early on. What I’m sensing is that we have a lot of organizations that already completed their kind of version one of public cloud experience that was comprised of some lift and shift and some building of new systems in the cloud. They are now busy architecting their next generation, their second generation of public cloud expansion where elements of making not just the default but actively working to decommission the old infrastructure.

And remember the motivation is you always have a software or a hardware refresh cycle happening somewhere, whether it’s for your switching fabric or for your compute fabric, in your various data centers or you need to upgrade those Windows 2008 and it’s all involves some interesting costs with it. So the drivers are already there. Once you already get the momentum and the velocity and the confidence, it’s something that is inevitable and it actually happens faster and is encouraged by kind of the known technical people, whether it’s the procurement or the finance people that realize that their support to this activity could save the organization actually a lot of money.

We saw that AWS revenue run rate, just AWS is around $40 billion now, and we just saw that the big Cisco revenue run rate is $48 billion. So the reality that we have crossed the chasm and even kind of gotten to the late point. The late majority is already feeling it quite seriously. I can say this for certain with a lot of certainty.

From our perspective at Dome9, we are also seeing organizations already architecting their second generation of what is to deliver continuous security and compliance and governance to their new architectures in the cloud. This is kind of it goes hand in hand. This is where we also see this kind of the maturity in the level of the people that we talk to and their requirements and their understanding that there’s not just a single or few parameters you need to take care of, but if you count them there are about 10 different areas of attention that you need to be very well covered with in order to provide to continue that trend and complete the migrations and sell those data centers as Intuit just did. I think they announced about two months ago that they sold their last data center. I believe we’re going to see a lot more of those announcements. We don’t need to worry. This real estate is not going away. Amazon buys it through Equinox and its partners on the other hand. So it’s not as if we need less compute. We just need a better, more resilient, more focused compute environment.

Shimel: Understood. Couple of things you mentioned in there that I want to bring to up or I’d like to kind of dive a little bit. First of all, yes, I do agree with you, even the late majority has moved to the cloud; but I think one of the things we’ve discovered about cloud migration is no one up and picks their entire – or very few people but some do – up and pick the entire infrastructure en masse over to the cloud, right? It’s a phased sort of migration, if you will. Some faster than others. And I’m talking now for large enterprises. I’m not talking startups. Larger enterprises. How long a migration do you think that usually is?

Alon: Look, so the key is the data. You know, when Amazon announced their Snowball device to lift and shift a couple of petabytes, I think it was three, four years ago at Korean event, people were in shock and like about the thinking of getting the data to AWS is, you know, FedEx is faster than the largest wipe you can find. And then they surprise last year with the full-blown semitrailer that has essentially storage on it that can take essentially any amount of data from your premises to the public cloud. So this is one thing.

Once you get the sense of trust with some data, then trusting the public cloud with your entire dataset is not something that is inconceivable, right? Trust is very binary. If you have it and if it’s good for the backend of my mobile application with the replication of a lot of my banking stuff, for example, then it’s good for everything. Then it’s just the technicalities on how we get there.

Now, I think one example I want to give is that other than data that can there are no barriers, sometimes the database can be a barrier and that’s why you’d see solutions like AWS will sell you migration services to move from your own – they’ll practically give it to you to move from your Oracle to their cloud-based RDS system whether it’s Aurora or whether it’s any of the other SQL versions that they offer as a service. You can say that probably what’s left is the big old mainframes.

I was fortunate enough to see that in real time about a year ago where Vanguard – so essentially after we figured out how to migrate data and databases, the only thing you might say may prohibit us from flying at speeds to the cloud may be an old system of record that sits tucked in in a mainframe computers in some data center at an unknown location. We have been able to see, to experience, and to help also several financial institutions in the process of securing a lot of replication process where they showed how they create using serverless computing and the AWS DynamoDB, a replica of their system of record of their big old mainframe, in the public cloud to provide essentially zero latency backend to their entire application framework. And I’m certain that one of the examples is Vanguard. There’s a very famous video of their heard of architecture describing how they used I think seven steps to create that replica of their mainframe.

It’s an amazing—and this was announced, this was shown a year ago, so you’d probably think that they’ve been working on it for two years before, and this is the reality, that when it started making sense, you know, the technological barriers to live in tandem until you are forced to make a decision about those old systems or those legacy systems is something that any organization should strive for.

And, you know, you don’t need to work hard, you know? Just, your public cloud rep, whether you go with Google Cloud or Azure or AWS, they will help you with providing the perfect architecture and how you’re baking security into it and how you optimize it for cost—they will pull in the partners that will help you on those things to kind of tie them across the board. And the reality is that—and that’s why I can say that, unless there is real slowness and some legislative or some other reason that could prevent an organization from starting its exploration, most of them are already picking it.

And sometimes it’s not advertised. I’m surprised to see, you know, industrial companies that come to us and, you know, I ask them how much is their public cloud spend, and they say, “We just crossed the $10 million-a-year mark.” And that’s a serious business. That means that the current way that things are going, they’re gonna get to ($50 million) or to ($100 million) in the next two years. And this means that you’re not investing in your on-prem systems any more. That’s what I’m seeing in the market.

Shimel: You know, I’m not gonna argue with what you’re seeing. You’re out there talking to people doing it. Zohar, unfortunately, we’re way past our time, but you know what, I’d love to have you back on. I know we had a little bit of a technical glitch; hopefully, it’ll be ironed out where people are listening to.

But let’s continue this discussion. I think it’s fascinating stuff, and I think people are interested to know. And we didn’t even—I really wanted to touch on what does this mean for cloud security? What does it mean for DevSecOps? Let’s do that in part two, if it’s okay with you.

Alon: Absolutely. Looking forward.

Shimel: Okay. Zohar Alon, Founder and CEO of Dome9—thanks for being our guest on DevOps Chat. This is Alan Shimel. You’ve just listened to another DevOps Chat.

— Alan Shimel

October 29, 2018
DevOps Chat: The State of Software Security with Veracode’s Chris Eng

Every year CA Veracode publishes its “State of Software Security” report, chock full of research about how well software is being secured during development and beyond. It’s eye-opening stuff and definitely worth a read for anyone even remotely interested in helping make software more secure.

In this DevOps Chat, Chris Eng, VP of research at Veracode, recently discussed with me this year’s report findings and some takeaways that are worthy of noting.

As usual, the streaming audio is immediately below, followed by the transcript of our conversation.

Transcript

Alan Shimel: Hey, everyone, it’s Alan Shimel, devopsy.kinsta.cloud, and you’re listening to another DevOps Chat. Joining us on this episode of DevOps Chat is Chris Eng of Veracode. Now I’m proud to say, actually, that I know Chris, actually, before there even was a Veracode. But – so it goes back a while. Chris, I don’t remember your exact title, but don’t you head up research at Veracode?

Chris Eng: That’s right, yeah. VP of research at Veracode.

Shimel: Yep. Welcome and good to have you back on, Chris.

Eng: Good to be here.

Shimel: Yep. So, Chris, you know, I’m gonna assume, if people wanna know who you are, it’s Chris Eng – E-N-G. You could look him up. He’s a bonafide cybersecurity warrior, with a long track record of accolades and just being a great guy in the community. On top of that, though, I’m gonna assume people know that Veracode, for the last couple years, has been putting out a report called “The State of Software Security.” In fact, this is volume nine, so which assumes there was eight before then, right, Chris?

Eng: That is correct. [Chuckles] We do this every year and it gets bigger and better every year.

Shimel: Yep. So volume nine, you know, it’s CA Technologies Veracode now, of course. Some changes. And, Chris, it’s kind of your baby, right? You shepherd this report every year, no?

Eng: I’ve been involved in it since the first year we did it. I mean, certainly, going back to the early years, there were a lot fewer of us sharing the burden and doing all of the writing ourselves. And now, thankfully, we can spread that out among more people and get more eyes on it. And that’s actually one of the things that we did this year that was kinda different, is, in addition to ourselves, we worked with some data scientists, over at Cyentia Institute, to actually help us dig into some of the things we really wanted to probe, around remediation work and looking at different organization behaviors. And I’ll get into that in a little bit.

Shimel: Absolutely. Chris, before we dive into maybe the specifics of this year, let’s just take a minute and talk about, look, nine years running, my god, how the world has changed in nine years. Right? I was talking to the CIO of Box, Paul Chapman, earlier today, and we were just talking about how the world’s changed, let’s say, in the last 20 years but even in the last 9 years. I mean, what we can – you know, security’s still security, and, unfortunately, we still have incidents; we still have vulnerable software; we still have buggy and have trouble closing up vulnerabilities. But the game has changed, though, too, has it, Chris? When you look back, what do you think that some of the biggest changes are?

Eng: Yeah, it’s definitely undergone a huge evolution. When we started Veracode in 2006, application testing primarily was companies doing penetration tests, maybe once a year, against their most critical applications. And, if anything changed between those, you know, in that one year, you kinda only had a point-in-time assessment. You had a lot of applications, a lot of pieces of software, that were just never being tested at all. And, since then, companies have realized that, number one, they need to test more frequently. That’s the only way that they’re able to release software that is going to be secure and also be able to drive the speed that they want to release new features and new products. Right?

The other, I think, realization is that you can’t just focus on the most critical applications to the business because attackers don’t really care how they get in, right? If they find a weaker way to – an easier way to get in, through some weaker application that’s not maybe core to the business but is still connected to the same infrastructure, well, they’re gonna go after that. So there’s been a realization that it’s important to understand everything that you have and to actually build software testing and security testing into all stages of the life cycle, right? Fix early and it’ll be a lot cheaper than fixing later on.

And, you know, we were looking at some of our stats the other day – this isn’t in the report, but to do the first million application scan for our customers took 10 years; the second million scans took one year.

Shimel: Wow.

Eng: Right? So that huge rate of increase is just showing that, across the board, across all industries, company sizes, organizations are realizing that they have to scale their software security programs to cover all the bases. And they have to do it continuously, in order to not rack up security debt. So I think that would be the biggest thing that’s changed in how we approach the problem.

Shimel: I agree. I agree. But, Chris, really, our time’s limited. Let’s jump into this year’s report. I’ve had a quick look through it, the executive summary and some of the other stuff. Boil it down for our listeners. What do you think are the three biggest kinda metrics or facts that jump out here, that they should be aware of?

Eng: Yeah. Let me give you just a quick –

Shimel: Sure.

Eng: – review on – so, like I said, we do this every year. And we take about 12 months of data, so this is across all of the assessments that we do for all of our customers, and it’s anonymized down, of course, and then we look at trends and try and see what’s going on out there, in terms of application security. This data set is about 700,000 application assessments during a 12-month period, and the one big thing that we’ve been able to do this time that we weren’t able to do in the past is actually look at how long flaws are sticking around once they’re discovered. Right? How long is it taking organizations to fix these flaws?

And so, when we detect something, we’re able to kinda track it across multiple assessments, so let’s say we scan an application today, we find a bunch of flaws, we fix some of them and not others, and then we scan it again in a week. We’re able to identify that certain of those flaws, the ones that we detect the second time, are the same as they were the first time. And so that allows us to actually track the life cycle of each individual flaw until it’s fixed, right? And so we’re able to look at “What does that flaw persistence look like?” Kind of a survivability analysis.

And, if you look overall, just across all the flaws that we look at, the numbers are not great. It takes 21 days on average to close 25 percent of the findings, 121 days to get to the halfway point, and 472 days to get to the 75 percent point. So that’s a long time, right? That’s over a year, so it’s not a very rosy picture overall. And so we broke this down and, you know, once the report is out, people can kind of look at “What does that look like in terms of different severities, different industries? What kind of patterns do we see there?” But I think the most interesting one, and the one that’ll be most interesting to you, is around the effect of DevOps or DevSecOps on fix velocity.

Now we don’t have an easy way of saying, for any given application, “Is this a DevOps shop or not?” Right? We can’t – we don’t have that information, so we have to figure out whether or not we think they are, and the proxy that we’ve been using for that is scan frequency, right? If you’re doing CI/CD, you’ve got your scanning baked into your build process or you’ve got it, some way, baked into your automation. And so, if you’re doing a nightly build, for example, you’re doing a test every night. And so we looked at scan frequency to identify that – you know, we think that probably these applications that are being scanned 300-plus times a year, those are probably DevOps shops, right? And the ones that are scanning one to three times a year, probably not.

And so we’ve got this broken down into buckets. If you have this in front of you, you wanna jump over to page 39 – it’s Figure 43 – and this is what I really want to – this is sort of a – our hypothesis was like, “Yeah, I bet DevOps shops fix faster because they’re scanning more frequently and they don’t wanna rack up as much security debt.” And, in fact, it does show that. Right? We see this one line on the report that says, “Applications that are scanned 300 or more times a year are fixing – they’re getting to the halfway point of fixing in five days and they’re closing 75 percent of flaws within seven days.” Right? So that’s a huge amount better than those averages that I just described to you.

And we have this broken down by bucket, right? So you have the 300-plus scans per year, you have 51 to 299, all the way down to one to three scans per year, where it takes close to four years for them to get to the 75 percent close mark. So I think that’s actually – this is what we hoped to see. This is what we expected to see. But it’s nice to see the data prove out that, in fact, yes, when you scan often, when you scan early and you stay on top of the findings, this is actually a very doable thing, right? To fix flaws in a reasonable time frame.

Shimel: Absolutely. Two things there, Chris. No. 1, I would think hand in hand with the DevOps is automation.

Eng: Yeah.

Shimel: Right? I’m wondering if part of it is that, in DevOps-enabled cultures and organizations, organizational cultures, whatever – companies doing DevOps – are they automating the scan process and that’s why we’re seeing them doing 300-plus scans? Right? And then they’re deploying more and, if their MO, if their standard process is “Nothing gets deployed without first being scanned,” right, that’s where you start racking up 300-plus scans. Right? I mean, when you –

Eng: Oh, yeah. Yeah, we’re –

Shimel: – build that into your pipeline, right, you build that into your CD process, those scans get done.

Eng: Yeah, we’re assuming that these – you know, we’ve gotten – I think, at the maximum point, there’s one app that was scanned over 1,000 times and, certainly, that’s – I mean, I really hope that’s not a human uploading something or pressing –

[Crosstalk]

Shimel: Right, _____ – hitting the “Scan” button –

Eng: – a button every time, right?

Shimel: – every time. You would hope not, right?

Eng: Right. Right. I’m assuming that even the ones that are scanned 50 times a year, right, which I assume to be every week, I assume that those are also –

[Crosstalk]

Shimel: Are still better than once _____ –

Eng: – _____ _____ through a build pipeline. Right? And, any time you can – I mean, and that’s been a big focus for us, just in general, is “How do we remove any human effort required to do the testing? How do you make it easier and more transparent and just remove any barriers to doing that?” And so –

Shimel: No, heck, how do you – you gotta build it into your CI/CD pipeline. I mean, it has to be part of that. The same way you breathe air, right? If you stop breathing, you die; you don’t deploy software that’s not been tested and scanned. And –

Eng: Right. Right.

Shimel: You know, it sounds so simple, us talking about it, Chris, but here’s the fact, right? And not everyone listening to this in our audience has been in security as long as you have or even I have. And a lot of people see the reports, Chris, and they say, “Oh, my God, this is year nine,” and I forgot how many years Verizon is doing their report, and, every year, it’s hard to show progress in some things, right? We do show “Hey, if you scan more, you’re gonna have less vulnerabilities go out the door that aren’t fixed and that were at least – they don’t stay – you know, they get fixed faster.”

Eng: Yeah.

Shimel: But the fact is we’re still seeing – you look at the executive summary here, Chris, and it’s not a rosy picture, as you – I think those were your own words. And we still read about breaches every day and 30 million records here and 20 million here. You know, the non-security person says, “What are we gonna do? When does this fundamentally change?”

Eng: Yeah, I mean, things don’t get solved overnight, right? I mean, if companies have only realized a few years ago that they need to kinda get a handle on all the things that they have and start understanding what their risk posture is, and they’ve got 10, 20, 25 years’ worth of software, thousands of applications, it doesn’t happen overnight; it’s baby steps. Certainly, starting any new greenfield projects with a proper SDLC that’s identifying and fixing issues early is gonna prevent them from building up even more security debt.

But, yeah, a lot of these – the way I explain some of these things – for example, pass rates on OWASP Top 10 not getting a lot better or the prevalence of certain flaw categories, like cross-site scripting and SQL injection, staying relatively constant – is that some of the more mature apps that have been scanned for a while now, those are actually decreasing. If we sliced those separately, we would see the rate going down. But, when we look at it in aggregate, out of those 700,000 applications that are part of this data set, a lot of those are things that are just being scanned for the very first time ever. And so they’re gonna have a higher prevalence of some of these issues, that the more mature applications will have gotten rid of already. Right? And so they kind of balance each other out.

And we see this every time. In fact, we should probably start to report on some of those things separately – it’s kind of a difficult thing to do – but I think what’s happening is some of the newer applications are bringing those numbers up and that’s why we see the lines staying relatively flat. There’s a long way to go, but we do see – you know, and things like the DevOps remediation case I just described. I mean, we see little points of optimism, where you can see a clear correlation between a certain behavior and a certain outcome. And so we like to focus on those things, and, yeah, it’s gonna take a while for most companies to really start to chip away at the legacy applications and getting those locked down.

Shimel: Absolutely. Absolutely. Chris, we’re just about out of time. I should – we should put this out there: for people who are interested in really kinda digging in on the report, where can they get it from? Go to Veracode.com?

Eng: It’ll be on Veracode.com, starting on October 24th, and, yeah, they can download it there. There’s a huge amount in here. We break it down into different flaw categories. We look at different industries against each other. There’s even a little geographic breakdown and then, of course, like I mentioned, a huge amount of data around flaw persistence and how long it takes organizations to fix flaws, which, I mean, ultimately, is what’s important, right? Doesn’t matter how often you scan if you don’t fix stuff. Right? You don’t reduce your risk unless you fix stuff. So there’s a lot of great data on that, that we’ve never been able to report out before, so we’re really happy to get those out there.

Shimel: Absolutely. Well, Chris, you know what? I know I got a little hard on you about the – and it’s not your fault that the security industry – and it’s not the security industry ’cause, God knows, we wanna fix the issues and the problems that cause these breaches and things; it’s really just the state of software and where security lies as a priority in a lot of these organizations and managing their risk and so forth.

But, by the same token, Chris, you know what? First of all, thank you, thank you, thank you, for being involved enough to do this report every year, and, whether the metrics paint a rosy picture or a sad picture, nevertheless, it’s a picture that needs to be told, right? We need this kind of insight; we need the metrics to understand where we are and where we need to be. And so great job, once again, in putting this together. You know, I think it’s as important for non-security people to read these reports as it is for security people. Right? To really –

Eng: Yeah, we hope –

Shimel: To –

Eng: Yeah, we definitely hope people will get something out of it and, hopefully, kind of use that to think about how they are building their own programs, right – how they’re remediating, what do their numbers look like – and, hopefully, kind of informs how different organizations think about scaling up their programs. That’s what we’re hoping, anyway.

Shimel: Good. All right. Chris, let’s call it a wrap on this; it’s a Friday afternoon here. And, again, thanks for your insight on the report. Thanks for managing this report year in, year out and being involved with it and helping. And we’ll see you soon. We’re coming into conference season; I’m sure you’ll be busy presenting and getting more –

Eng: I’m sure, yeah. I’m sure we’ll be bumping into each other pretty soon.

Shimel: Absolutely. Chris Eng, VP of research at CA Technologies Veracode, our guest on this episode of DevOps Chat. This is Alan Shimel, everyone. Have a great day.

— Alan Shimel

October 24, 2018
DevOps Chat: CHAOSSEARCH New Spin on Big Data Analysis

Analysis is the dark cloud overhanging big data — it’s difficult without the right tools. Elasticsearch by Elastic has become ubiquitous in helping organizations, but it can get unwieldy, as storage combining with compute is tricky. CHAOSSEARCH is an early stage startup designed to make searching massive amounts of data using elasticsearch and Amazon S3 easy.

Pete Cheslock, VP of Products at CHAOSSEARCH and a good friend of devopsy.kinsta.cloud, joined me in this DevOps Chat to discuss his latest company’s technology and how it might make big data searches less daunting. The key for big data search and analysis is separating storage from compute, using Amazon S3 and Elasticsearch.

As usual, the streaming audio is immediately below, followed by the transcript of our conversation.

Audio

Transcript

Alan Shimel: Hey everyone, it’s Alan Shimel, devopsy.kinsta.cloud, and you’re listening to another DevOps Chat. Today’s DevOps Chat features a returning guest to DevOps Chat. He’s actually probably been on DevOps Chat a number of times, right from I think the very beginning of when we first launched devopsy.kinsta.cloud. He’s pretty well-known in the DevOp community and really needs no introduction from me. It’s Mr. Pete Cheslock. Pete, welcome to DevOps Chat.

Pete Cheslock: Hey, thanks a lot. Thanks for having me.

Shimel: Yep. Pete, always a pleasure to have you, man. You know, I remember speaking with you – actually it was probably before I even launched devopsy.kinsta.cloud, up in Boston. And you kind of gave me your views on what was then a very kind of just starting to blossom DevOps community and DevOps movement. And since that time obviously a lot of water under the bridge and many different hats that you’ve worn, Pete. Here to introduce today though with your latest position in helping a company grow, and that company is called CHAOSSEARCH, correct?

Cheslock: Yeah, that’s correct.

Shimel: And Pete, you’re VP of Products?

Cheslock: Yeah, yeah, VP Products, Products, VP of Products? I don’t know, it’s a working title. We’re still trying to figure out what the best way to –

Shimel: As long as they don’t call you a DevOps engineer, it’s all good. [Laughs]

Cheslock: Exactly.

Shimel: I hear ya. So Pete, well let’s start off with this. I’m going to assume a lot of people listening to this today don’t really know or are not familiar with CHAOSSEARCH, so why don’t we start with that? What’s CHAOSSEARCH?

Cheslock: Yeah, absolutely. And yeah, I think that’s definitely a good assumption. You know, we’re an early-stage startup. We’re just going through seed funding right now, so we’re just going through an A round of fundraising, so then we’re going to be coming out to the market in a few months here. But basically what CHAOSSEARCH is is they’re trying to make it easier to access and make query-able your data as it exists on object stores. So if you think of an object store like Amazon’s S3 service. And so they developed basically a technology that allows you to store data. They basically process your data from S3 and store it back into your bucket. So what’s really interesting is they don’t hold any of your date. They’re simply – you know, you give them like a read-only to your S3 data. They’ll read in that data and shove the compressed, processed data back into your bucket. And then from there they expose APIs to allow you to access it.

So for example, they’ll extend the S3 API to give you some really interesting capabilities of slicing and dicing your data. But the thing that they’ve exposed that’s most interesting, which got me excited about the company, is that they’ve exposed the elasticsearch APIs on top of your data on S3. And so it’s really compelling, because I spent a very long time as a really early user of elasticsearch, and it’s a great technology; it’s a great open source database. But it gets unwieldy over time. And so what really blew me away when I saw the product and what they’d built is that I can point open source log tools like Kibana to the CHAOSSEARCH APIs, accessing my own data on S3. No elasticsearch required. And so the kind of short pitch, we’re trying to make that long tail of data available to people, whether it’s log data or event data of some kind, where customers can run much smaller elasticsearch clusters of their log data or event data, and then they can essentially push the data up to S3. We’ll process it, and now they can get access to that data for weeks, months or even years.

Shimel: Yeah. And Pete, that really highlights sort of a soft-white underbelly or a dark corner around the whole big data issue. It’s great to have – I mean, big data and – big data is only as good as the analysis we can do with it, right? That we can perform on it. And with big data we could certainly get some really keen insights to show us relationships, trends, facts that we might otherwise miss. But the dirty little secret was, for big data, to really get the value out of it, generally you needed big money kind of search to perform that kind of analysis. Because the bigger your data got, the harder it was to actually manage it, search it, analyze it. And so CHAOSSEARCH seems like a real solution for those people who are literally drowning in data.

Cheslock: You’re exactly right. I worked at a company eight or nine years ago called Sonian, which – not a lot of people have heard of Sonian. They were one of the earliest users of Amazon’s S3 and EBS. At the time we were probably the top-three user of S3 and maybe the top-five user of EBS. What’s amazing about Sonian is the number of companies that actually came out of that place. Companies like CloudHealth, which just got acquired by VMware. Sensu was a technology that was built inside of there. Stackdriver was another one. And the reason all these startups came out of Sonian is because they were so early in the cloud space, they had to build out the technology to support it.

One of the other ones that kind of came out on the side, not really related to us, was elasticsearch. We were trying to index customer e-mail data, like an e-mail archiving product. And so we got connected with Shay from elasticsearch before he really started the company because we were so blown away by what he had built. And elasticsearch at the time made sense, right? Using Lucene to store indexes of customer e-mail data allowed the indexes to be much smaller than the sourced e-mail files. But then over time, over the last say eight years or so, elastic’s been around, there’s been this explosion of using elasticsearch for log data. And in many cases, like structured JSON that we’re shoving into elasticsearch. And due to the inverted index nature of Lucene, the data blow up inside of Lucene where you might actually store four or five times more in disc storage on elasticsearch.

So if you shove in a terabyte of JSON logs into your elasticsearch cluster, that could be three or four terabytes of Lucene indexes. And then you have to think about like, “Well, I want durability and reliability, so now I need to like run a replicant index.” So now you’re talking for every terabyte into elasticsearch you need like eight terabytes of physical disk. What was really amazing about what the founders and the early team here has created is the format, kind of solving for the Lucene problem. They’ve actually been able to reduce the size, yet still making the data query-able, using kind of the same APIs, which is what really blew me away, in that it’s now – you know, so many people have to make the choice. Do I keep my data for analysis, at a really high price of elasticsearch? Or, do I throw it away? And I think a lot of people just throw that data away, because they just can’t make it worthwhile in any sort of cost effective way.

Shimel: Right, not in an economic way. Pete, we’re far from a big data solution here at MediaOps, between our seven or eight websites, but I will tell you, just pure search, like an index search on devopsy.kinsta.cloud has become a struggle. Because a lot of the native WordPress search – you know, especially if it’s on server, is killing resources on your web server and so forth. So you want to do it sort of off-server search and it’s just a pain in the butt. Take my word for it. So this is great. You said the company’s launching in a few months though? Not that it’s launching – it’s looking for seed money now. Product will be in market. Where exactly is it now in terms of – is it in beta? Is it alpha?

Cheslock: So yeah, we’re probably going to be closing our A fundraising round over the next 30 days or something like that. The company actually started about two and a half years ago, and so they’ve been essentially building this technology out over the past couple of years and really trying to refine it as they were kind of were coming up with this like product market fit. Like what’s the market we’re going into? And as the founders really looked at the space, they realized that log and event management is where a lot of pain is right now. What’s interesting is when people start using the product, they realize that they can process any kind of data with this and query it. And that’s where there’s a lot of interest, where maybe it’s not just log and event data. Maybe it’s like some long tail of data that you want to get access to or to monetize for your customers.

But where we’re at right now, and what was exciting about me coming onboard is not only did my role change completely – I used to run Technical Operations for Threat Stack, which is a pretty successful cloud security company that’s still just crushing it. I changed my role over to become the Head of Product. So working with customers to understand their use cases and how they’re using tools like elasticsearch. So where we’re at right now is I’m basically looking for what we’re calling design partners, where – the best way to describe the product is in that beta phase, where we’re looking for people with really interesting datasets and maybe people who are on Amazon and have data on S3 or are using elasticsearch, and getting them into our proof of concept setup right now, where we can have them run queries, test the analysis, test how they might use it and really help us refine the final bits as we kind of _____ things up towards GA.

And then, once we go to GA, towards – I would say October timeframe I think is what we’re aiming for, it’s really to continue to refine these really specific use cases like, “I’ve got data on S3 and I want to process this data.” We’ve had some customers that want to push their elasticsearch indices up through the service, which is a really interesting idea, because those loosing indexes already have a defined schema. And so that way, instead of storing seven days of data in elasticsearch, maybe they just store one day and that’s in their hot cluster in elasticsearch on EC2 or something. And then days two through 365 are up on S3 and are available through the CHAOSSEARCH service. So yeah, that’s kind of what the next month or two looks like for me. You know, really trying to work with customers, people with interesting datasets. You know, people who have just way too much elasticsearch and they want less of it. You know, they want to kind of run less servers, spend less money – which I feel like is kind of everyone out there right now.

Shimel: Excellent. Very cool stuff. I mean, Pete, look, for you to get up and leave Threat Stack and come over here, obviously it had to be something really appealing and compelling, to bring you over there. Pete, it’s interesting. My background prior to devopsy.kinsta.cloud was founding and building companies, and one of the things I learned is it’s great to have these grand – come up with the idea of Docker for containers and change the way – you know, the basic platform that we use, right? For our apps and for IT. But those revolutionary kinds of things are really kind of few and far between, especially the ones that succeed. Really IT moves along in evolutionary kind of ways, where it’s a thing like this. It’s thing like with CHAOSSEARCH is doing, that makes it easier, better, faster, more economical to really get a hold of your big data and do the analysis and get a handle on it that you need to make it valuable. Right?

Cheslock: Yeah. Exactly right.

Shimel: And so, you know, this just feels that way to me – that this is a lot – some people may say, “Well, it’s not earthshattering.” But no, but it’s the logical step. It’s the glue that you need to – you know, big data is a big concept and it’s great and it’s going to revolutionize things when we get it. But you need the CHAOSSEARCH or sort of the glue to make these things happen, right?

Cheslock: Yeah, what’s amazing is, as I’ve talked to people running a lot of elasticsearch, a lot of data on S3, it’s a really easy sell, because I can go to them and get them away from running these huge clusters that – they may not even take a lot of queries too. That’s what’s interesting is maybe they’re only querying their data in an automated way the last hour or 24 hours. But then you start looking out seven days, a few weeks, maybe a few months; those queries are significantly less. And so you have to either build out like a tiered storage. But at the end of the day the storage and computer are coupled so tightly that your scalable units on Amazon or any cloud provider is a lot of disk and a lot of CPA all together. And what CHAOSSEARCH has done, it uncoupled storage and compute, where the storage is on S3, specifically your S3 account, and the compute is the CHAOSSEARCH service.

And so it’s a totally different way of thinking about, “How do I access my data? How do I make it query-able?” And now, because it costs so little – and we’re talking like go-to-market pricing a tenth of the price of running elasticsearch yourselves, and significantly less than a lot of the other kind of hosted management, log manager vendors. You know, it’s like now I can keep my data for month and a year and I can run machine learning models cost-effectively, because I can keep so much data. That’s the part that’s super interesting, is, how does this change data science and those companies that really want to have the long tail of data for ML/AI testing, and how does this change how they work their models, how they build it in? And we’re even trying to build in like tenser-flow integration into the application so that you can just access it from within without needing to have all this crazy data science experience.

Shimel: Excellent. It really is. Pete, you know, 15 minutes goes quick. We’re coming up on our thing here. You’re VP of Product. Where do you see this going six months, a year, two years out? It’s a long way to look, but where do you see yourself taking the product direction here?

Cheslock: Yeah, no, I always like to joke that the next six months are clear and in focus. Things get really fuzzy really quick after that. But the nice thing is I think we have a pretty clear plan going out of it. Through the end of the year, reaching GA, bringing some customers onboard that we can solve a problem for them. But really, early next year we’re going to start beta-ing a self-service kind of model, where people can come in and try it out. And probably doing even a freemium model, because since the data is in your S3 bucket and we’re just processing that data, cost is very low. So being able to provide someone a couple of gigs a day of data processing, it’s pretty easy for us. And then that way people can kind of get their feet wet and try it out and we can really start building a community around it.

And then from there, I think the sky’s the limit. You know, we’re looking at some time end of next year, the ability to run it within your own data centers. Right now we want to be kind of pure SaaS so that we can continue and build it out and improve it in our kind of very quick, iterative development cycle. But really it’s a data platform we’re building. I mean, we’re going after that log and event management space because it’s so painful for so many people and it costs a lot of money. But the long-term for us is, what other APIs could we expose on top of your data on S3? Elasticsearch seemed to make a lot of sense, given the market. We’ve extended S3 with some interesting features. But like what else is there? What are different ways people are trying to access different data? And so I think that will be a lot of fun to kind of chat with people, understand their pain and maybe actually bring them a nice solution to that pain.

Shimel: Yeah, absolutely. And I’m sure you’ll have a ball doing it, Pete. Anyway, we’re over time here. Pete Cheslock, newly minted VP Products, CHAOSSEARCH. Thanks for joining us on this episode of DevOps Chat, and we look forward to hearing more about CHAOSSEARCH and about your adventures, Pete.

Cheslock: Yeah, thanks a lot. Thanks for having me. This was a lot of fun.

Shimel: Always a pleasure. Pete Cheslock, CHAOSSEARCH. This is Alan Shimel for devopsy.kinsta.cloud. You’ve just listened to another DevOps Chat.

— Alan Shimel

October 19, 2018
DevOps Chat: All Day DevOps Preview

All Day DevOps, the virtual event featuring more than 100 sessions from practitioners around the world, is happening very soon. Now in its third year, this year’s event will take place Wednesday, Oct. 17, starting at 3 a.m. Eastern. But if you don’t feel like getting up, each 30-minute session will be available on demand as well.

In this DevOps Chat, All Day DevOps founders Derek Weeks and Mark Miller, both also from Sonatype, offer a preview of what’s in store during this year’s virtual event, which is free to attend.

As usual, the streaming audio is immediately below, followed by the transcript of our conversation.

Audio

Transcript

Alan Shimel: Hey, everyone. It’s Alan Shimel, devopsy.kinsta.cloud, and you’re listening to another DevOps Chat. Really happy to have two friends of mine join us on the chat today. We have none other – those crazy guys from All Day DevOps and Sonatype, Derek Weeks and Mark Miller. Mark, Derek, welcome to DevOps Chat.

Mark Miller: Thanks, Alan.

Derek Weeks: It’s good to be here. Always happy to chat with you.

Shimel: Yeah.

Miller: Thank you, Alan. Welcome to New York, welcome back home.

Shimel: Yeah, it’s good to be home. So Mark, you know, this is an audio only, people don’t know, but you have a Fighting Okra hat on, is that correct?

Miller: You know, some of my followers from around the United States are sending me hats and right now the Delta State Fighting Okras have sent me a hat so I’m gonna be giving them a good callout during All Day DevOps. Derek, you haven’t seen this one yet.

Weeks: No, I haven’t. That’s pretty cool-looking.

Shimel: All right, a shout out to the Fighting Okras.

[Laughter]

But Mark, you mentioned All Day DevOps. We’re getting close, right? What’s the date?

Miller: We’re next – we’re gonna be on Wednesday, October 17, and as we looked at this morning, as we looked at it, we’re over 19,000 registrations this morning. Once again, the community has gotten behind it. Derek, you’ve got a bigger hand on it, too. You’ve got some ideas about what’s going on with the viewing parties even, right?

Weeks: Yeah, I think the – you know, we’re at 19,000, I think we’re easily headed over 30,000 by the time we get to the conference, and I think part of that is what we’ve seen with the viewing parties this year. So, you know, it’s an interesting kind of evolution of the conference where we started this thing as, hey, let’s have this virtual conference so that everyone can attend wherever they sit, and then people said, “You know, I want to participate but have others in my company or my community participate as well, so we’re just gonna gather in a place to have this viewing party.” In the first year that we did it there might’ve been 10 because it wasn’t a formalized thing. Last year we kind of encouraged people to do it but didn’t have anything on the site that really mentioned it too much, and I think we had 35 viewing parties.

This year we have 115 viewing parties that have registered in 24 countries, and the interesting thing is not only has that changed the conference and how it evolves and how people share this education within groups but we’ve heard some amazing stories just come out of this community where organizations like RBC in Toronto, they’re planning for up to 1,400 people in RBC to attend their viewing party. State Farm in Dallas has 500 people that they’re planning in their viewing party. National –

Miller: They went up to over 800 this weekend.

Weeks: Yeah. National Australia Bank; you know, they’re planning, I think, 250 people in their viewing party. And when you think about the size of some DevOps conference that all three of us have been to, anyone having 500 people at a DevOps conference has a successful DevOps conference. The fact that one company, you know, is setting up a viewing party just for our conference that’s as large as some DevOps conferences out there it’s pretty interesting. But, you know, the same point – and this is a conversation I had with someone this morning in Munich, he said – you know, we’ll get into talking about Club 20 – but he said, “Hey, do we have 20 people registered yet because we want to get our company into Club 20,” and I hadn’t heard of this organization really before.

And I looked; they had 16 people. Now they actually have 20 so they’ll make the list. But I said, “I’m curious; how many people in your organization could attend All Day DevOps that, you know, are DevOps practitioners?” And he said, “You know, like I want to get on the Club 20 list, but I think we could get 100 people at our viewing party and in a perfect world we could have 1,000 people participate in All Day DevOps.” And that – you know, Mark, as we started this whole thing, that was really the aim, like how do you get to those 1,000 people and give them some free education?

Miller: Yeah. One of the topics that you just touched on – and you and I talked this morning too as we were putting together how our day was gonna go – Alan, when you think about it, can you imagine 20 people showing up at a live conference? I mean think about that. If you get four or five people at a live conference from one company you’re blowing out the budget, right, already. In this case, we can literally allow dozens and dozens of people from the same company to attend 120 sessions in a live conference. It’s a pretty phenomenal way to look at it.

Shimel: And, you know, so one way of looking at it, Mark, is wow, what would the cost be to fly in a dozen, two dozen, three dozen people to a physical conference? The other thing, you know, to make it a truly win-win is what are the benefits of a team that size sharing that common experience, right? It’s almost priceless, right? It’s like the MasterCard commercial: $45 for the conference ticket, $60 on Uber, $20 on lunch. Sharing it as a team? Priceless.

Miller: Yeah.

Shimel: And that to me is the real one plus one equals three of it.

Weeks: Yeah, I think – and that’s something that we see – you know, as we’ve seen the past two years that we’ve done this and I think we’ll see again in spades, is that we have the Slack channel set up during the conference for conversations, and last year during the 24 hours of the conference we saw 30,000 conversations on Slack, and that ability to exchange information, whether it’s with people on your own team and you have your internal Slack channel set up through your own company because, you know, you’re State Farm and that’s what you did and how you shared information, or you’re participating in the global channel, you know, I think it’s all about that kind of exchange of information and also the anonymous – somewhat anonymous level of that communication.

We all know it’s a lot easier to ask a question in a conference if no one can see you’re the person raising your hand or asking the question; you’re just one of many on a Slack channel saying, “Hey, I have this question,” and there’s – you know, it opens people up to sharing a little bit more or asking questions they might not otherwise get to ask.

Shimel: Absolutely, absolutely. Guys, I want to turn the conversation a bit if I can from numbers and viewing to content. Let’s talk – you know, people out listening to this, some of them may already be registered, some may not, some may not even know of All Day DevOps, right?

Weeks: Yeah.

Shimel: Mark, if they register – and registration’s free. You don’t have to be part of Club 20, you don’t have to have your whole company. If you’re just one person who’s digging DevOps you can register for free for All Day DevOps and sit in on 24 hours – up to 24 hours of DevOps, right? They don’t have to listen to all of it; you pick and choose what you want. But give us some of the highlights. What are some of the big hot topic speakers that you’re looking forward to?

Weeks: Mark, why don’t you go –

Miller: One of the things that you just – sure. One of the things that you just said is interesting to me, Alan. You said 24 hours of DevOps. Actually, when you do the math it’s more like 50 to 70 hours of DevOps because we have simultaneous channels happening. The things that I’m excited about seeing is, one, we have David who runs a CRE team and an SRE team at Google coming to talk about site reliability engineering. That’s a new topic for us this year, and to have Dave step in from Google, who is actually running that whole initiative for Google, it’s gonna be an exciting one for me. The other one that I’m – how do I say it?

I’m exciting that Cindy’s coming, Cindy Healy. She is the one that wrote some of the code that went on the Mars Pathfinder to send to Mars. Now that’s – the personal story behind how Cindy did that and how she became involved in it, what’s she’s doing now to encourage women in technology, that is a story that I really want people to be able to hear, so we’ve got her set up as a keynote. Derek, you’ve got some interesting things that you’re looking forward to, right?

Weeks: Yeah. You know, one is a conversation I just had last week with one of our keynote speakers, George Swan. He’s at Autodesk. Super interesting story of DevOps in the enterprise. So these guys knew, you know, the needed to transform the entire company – and I think they have about 200 products, 4,000 developers in their organization, and they needed to transition from an Agile organization to an organization that had DevOps practices top to bottom. At the same time that they were doing this they needed to transition into the cloud. They also needed to transition from a traditional license business to a subscription license business.

They had, like, four or five transformations going on at the same time – you know, become cloud-based infrastructure, transition to SaaS, transition to DevOps and some other things, and it was like, oh my gosh, one, this is a multiyear journey he’s gonna tell his story on, so it will be fascinating to kind of get lessons learned and also understand it’s not just hey, you know, transition to DevOps; it’s there are a lot of other things that are moving parts of our business that have to change, you know, along with this as the marketplace evolves for them and other participants. The other one that I’m really excited about, and a guy that I’ve got to see a couple of times at live conferences, is Navin Vembar. He’s the CTO over at GSA, the Government Services Organization.

They do all the main procurement for the U.S. government. But he is going to talk about how they transformed their organization into a DevOps practice and kind of looking at the maturity models that they use and kind of used as milestones for measuring their progress within their organization and also, you know, coming from an organization that loves silos, they love regulation, [laughs] they overly love governance sometimes, but to hear his journey, he’s a fascinating speaker and someone that I quote often, so always looking forward to hearing people like that join in.

Shimel: Sure.

Miller: Alan, the other thing is as Derek was talking I went over to the All Day devopsy.kinsta.cloud site and clicked the speaker link. We’ve got 124 speakers this year. We had so many good submissions that we actually went above our quote. We had set it at 100 and we’re at 124 right now. It’s hard to go and cherry pick out of those. What we want to make sure people understand is two things; No. 1 is we want to bring 124 practitioners to everybody’s desktop. We want you to be able to pick and choose what you’d like to hear.

No. 2, Derek and I guarantee to the community is there are no vendor pitches here. Alan, you’ve been with us long enough you know. We really hold to that creed, that what we want is practitioners, and when you come here I think that you can trust us to provide that to you.

Shimel: Absolutely. That’s another aspect I want to just highlight too, guys. You know, we – you both highlighted what – some of the ones you’re – some of the talks you’re looking forward to most. Well, Mark, there’s 124 as you mentioned. All Day DevOps is a global phenomenon, and though we may have GSA and we may have some U.S.-based practitioners, Mark, I’ve been around the world with you, right? You and I have traveled – [laughs] literally traveled the world, and DevOps is a worldwide thing, and I think the speaker lineup reflects that. Have you had any stats, like how many different countries are represented?

Miller: Yeah, Derek’s got that, but before he jumps in with the stats I got a story that you’ll recognize. You and I went to Singapore, Bangkok, Sydney and Melbourne about two months ago.

Shimel: Yep.

Miller: While we were having dinner with Rob England in Sydney, I went, “Man, you’ve got a great story, Rob. Why don’t you speak at All Day DevOps?” Meaning because of our travel, because of our ability to meet people in the community, it gives all of us the ability to say this guy’s got a story that deserves to be told, and I think that that’s one of the things when Derek and I were first putting this together we actually talked to – amongst ourselves and said what we want to do is give exposure to people that are doing DevOps that deserve more recognition for what they’re doing, and our global platform has allowed us to do that. Derek, how many countries are we dealing with now?

Weeks: I think, you know, we’re around 130 different countries participating.

[Laughter]

It’s amazing, you know, to see that list grow because some of these – you just imagine like who’s practicing DevOps, and in my head like maybe 22 countries and maybe if I stretched it like maybe 40, right? But to see participants from all over the place – you know, and I just did a quick look at the registrations in the top cities. So No. 1, Bangalore, second, London, Toronto is high on the list, Melbourne, Australia; Sydney, Australia, Atlanta, New York, Istanbul, Chicago, Singapore, Amsterdam, San Jose, Seattle, Santiago, Jakarta.

Shimel: That’s amazing.

Weeks: You know, in the top 20, right? So it’s really, you know, a global kind of phenomena that we’re seeing. You know, where is the DevOps kind of center of expertise? Where are these people around the world? It’s pretty interesting to see where they all come from.

Shimel: Agreed. You know, Mark, listening to you talk –

Miller: I don’t want to leave anybody out but when we –

Shimel: Go ahead. You go, Mark.

Miller: Go ahead, Alan, go ahead.

Shimel: No, no, you go.

Miller: Yeah, I just – I don’t want to leave anybody out, but there’s some names I’d like to give out a call to, is like Midu in India, who’s helped us so much with the community in India. Look at Bangalore; it’s No. 1 on the list. When you look at Michael Mann, you look at Mark Cluett in London; those are community people that have said this is something worthwhile and I want my community to know about it, and that’s kind of – I don’t know if heartwarming is the word, but it means that we have actually resonated with various communities, cross-cultural around the world, and that means a lot to me.

Weeks: Yeah, I just – you know, curious, I just looked at the list of what cities have 40 people registered – Alpharetta, Georgia – that’s near Atlanta – Johannesburg, South Africa, Istanbul, Dubai, Barcelona, Seoul, Sao Paolo, and McLean, Virginia, where I’m sitting.

Shimel: Seoul, I haven’t heard Seoul yet.

Weeks: Yeah. So fascinating, just that list of only cities with 40 is extremely diverse.

Miller: Wow.

Shimel: You know, that’s really – that’s the power of the internet and it also speaks, Mark, to the power of DevOps, is if it was all centered in San Francisco or London, great. All the cool kids, right, are doing it. But it’s not; it is – it’s diversified, decentralized. You couldn’t snuff it out if you tried. You know, Mark, listening to you talk though a little bit about we met this one here and this one there, I can’t help but think of you as sort of the Anthony Bourdain of DevOps, venturing into parts unknown.

Miller: You know, I’m a story guy. I gotta jump in there. When I was in Manila, we got Anthony Bourdain’s driver to take us around to all those places. It was fascinating.

Shimel: And I’ve been on those trips with you. You like to off the beaten path. But, you know, this is a PG-rated show and, you know…

[Laughter]

Weeks: I’m gonna give Mark a new title of the Anthony Bourdain of DevOps.

[Laughter]

Miller: I’ll take it, I’ll take it.

Shimel: Anyway, hey guys, we’re about out of time here, but I wanted to check in ’cause we are getting close. We would be remiss if we didn’t at least mention where can people go to register for All Day DevOps? Derek?

Weeks: AllDaydevopsy.kinsta.cloud. It’s that easy. It takes 15 seconds to register. You’ll get a link to the schedule once you register so you can start to build your own schedule for yourself and your organization. And like we always say, once you register yourself encourage your team to register so that everyone can watch together.

Miller: Alan, the last thing that I’d like them to look at on the site too, is once they register go to the viewing party page. With over 110 viewing parties there has to be something local, that’s relatively local, if you would like to participate with other people.

Shimel: Fantastic.

Weeks: You know, the other thing just really quick before we go is one way that we do this conference for free for everyone is we have a lot of sponsors, like devopsy.kinsta.cloud, Scaled Agile, Cisco, Puppet, Chef, GitLab, Sonatype, CloudBees and other organizations that help fund the conference to actually put this on and make this happen. We also have 245 community groups around the world that are helping us spread the word to say here’s free education for our members, and that’s really at the heart of this conference is the financial support to make it free for everyone and the community’s support to really spread the word to everyone that this is available to them.

Shimel: Excellent. Absolutely, Derek. These – as someone who’s done enough conferences, it really – it doesn’t happen without the sponsors, and a lot of people say, “Oh, the sponsors just want to spam me or contact me.” Yeah, you can talk about that all you want but they make the world go round on these things, right?

Weeks: Yeah.

Shimel: We can’t do it without them and don’t lose sight of that.

Weeks: Yeah. It doesn’t cost $10 million to run but it’s not free to run. [Laughs]

Shimel: Exactly. But we all know that. Anyway, guys, two of my good friends in the DevOps – well, not only in the DevOps space, in the world in general. Mark Miller, who I have traveled around the world with and eaten in some very –

Miller: Thank you, Alan, so much.

Shimel: – fun places – Anthony Bourdain there – and Derek Weeks.

Weeks: Thanks, Alan. Always a pleasure.

Shimel: Always a pleasure, man. I haven’t seen you in too long.

Weeks: Well –

Shimel: But hopefully we’ll see each other soon.

Weeks: Yeah.

Shimel: All Day DevOps coming to a computer near you Oct. 17, right?

Weeks: Yep.

Shimel: All day.

Miller: Starting at 3 a.m. New York Time, Alan.

Shimel: Excuse me?

Miller: Starting at 3 a.m. in New York. I’m sure you’ll be with us for the opening.

Shimel: Yes, I’ll be up bright and early at 3:00. But if you want to register, go to AllDaydevopsy.kinsta.cloud, guys. It’s free. Jump on it. It’s a great event. Have a great day, guys. Thanks for joining us. This is Alan Shimel for devopsy.kinsta.cloud and you just listened to another DevOps Chat.

— Alan Shimel

October 11, 2018
DevOps Chat: DevSecOps with ShiftLeft CTO Chetan Conikee

Security in the DevOps process has gotten a lot of attention lately, as companies grapple with ways to include security further up the application development process. The industry is still very much a nascent space, as players remain engaged in the security-versus-speed conversation—one always is sacrificed for the other. ShiftLeft, however, doesn’t believe that’s the case.

In this DevOps Chat, Chetan Conikee, CTO of ShiftLeft, discusses his company’s approach to DevSecOps and how it can help keep the security department from being the “department of ‘no.’”

As usual, the streaming audio is immediately below, followed by the transcript of our conversation.

Alan Shimel: Hey, everyone it’s Alan Shimel and I’m here for another DevOps Chat – DevSecOps related. I’m joined by Chetan Conikee, a CTO and cofounder of ShiftLeft. Chetan, I hope I didn’t mess your name up too bad.

Chetan Conikee: Absolutely not, you did a great job. Thank you Alan, appreciate it.

Shimel: Thank you. So Chetan, as I mentioned you’re the cofounder CTO of ShiftLeft – we’re going to talk a little bit more about ShiftLeft later in the show – but I wanted to start off talking about what is really one of the hottest topics in our world today and that is this whole whether you want to call it DevSecOps or DevOps and Security or Security Shifting Left – hence the name – talk to AppSec, some other people do – you know what about in your world? Are you seeing this kind of emphasis like never before on what some are calling DevSecOps?

Conikee: Absolutely. Great question, Alan. Let me first start with the term shift left because it has two definite representations; one is a word and the other is a noun. Formerly ShiftLeft started as a way of doing things; meaning taking a practice and moving towards the left in order to reduce what they call as the meantime to repair. And that something could be testing or perhaps measurement of security as well.

So when we adopted that name or changed the word to a noun we wanted to take this seriously; meaning how can we effectively build a product that can measure the security posture of another product at the left, informed from the right as well because often DevOps is nothing but a pipeline – an affective pipeline that is operating both at automation and high velocity. DevOps is all about taking code, building, packaging and thereafter deploying into your production ecosystem.

But it doesn’t just end at deploying: it ends at measuring and feeding back how to optimize your systems thereafter. Often security does not fit well into this high velocity motion because most of the security products today are legacy or antiquated. These technologies like static analysis, dynamic analysis or perhaps a web application firewall finds it hard to fit into this streamline DevOps biplane. And I use that term emphasis with heart simply because these tools are producing outcomes and these outcomes are nothing but logs, information, alerts and all of this is sent back to humans in silos. And it is upon the human to figure out how to correlate this information and take affirmative action.

So in essence this is impeding velocity rather than accelerating velocity, which DevOps is all about acceleration. As a consequence security gets left behind in organizations.

Shimel: Can I tell you – I’m sorry – that was one of the best ones I’ve ever seen – one of the best definitions I’ve ever heard.

Conikee: Thank you. Appreciate it. So we take this seriously as well because what we are trying to do is convert that definition into automaton, meaning how can you measure security thereafter quantify threats to vulnerabilities because often vulnerabilities are measures on the left, threats are measured on the right, so how can a human comprehend both these vectors, associate them and then address and fix these issues, because security is a huge concern these days.

Shimel: Yeah. Agreed. So Chetan this brings up something I wanted to spend some time talking about here today and that is it’s a – not a survey but a study that ShiftLeft’s done – I think you guys had a partner on it – why don’t you share a little bit with our audience kind of just laying the groundwork there.

Conikee: Absolutely. Today we are announcing the industry’s first real world benchmark of what we call a continuous application security solution. And in order to conduct this benchmark test we partnered and collaborated with Cobalt which is a leading provider of integration testing. They have some of the world’s best penetration testers working for them.

So in order to conduct this benchmark test we submitted two instances of an application – a vulnerable application. By vulnerable I mean it essentially contains a certain set of vulnerabilities in it which we often test against in order to measure the efficacy of our platform.

In this test we took that application and we deployed two instances in production. One instance was unpredicted: the typical instance that represents these vulnerabilities and have no protection applied to it. Whereas the other instance was protected by ShiftLeft’s one-time agent.

So the premise of this test was to essentially let these penetration testers conduct tests against these two applications and thereby come back with results. And both these results had a certain degree of focus. The results were to identify what vulnerabilities can be declared by these pen testers upon the application that is not protected by ShiftLeft and thereafter subject the same set of tests on the application protected by ShiftLeft and come back to let us know whether they could penetrate that application or not, or perhaps could exploit that application or not.

So this was a 14-day exercise and we defined the scope for this exercise – meaning what we essentially did was identified the most vulnerable or relevant applications called and racked by OWASP top ten and thereafter opened these applications to the Cobalt pen testers.

Now as they began this exercise and Cobalt had hired essentially three white-hat hacking experts to attack both these applications. Cobalt was able to find and exploit all the six vulnerabilities in the unprotected application which is a validation for the fact that this was a beta application, it had the vulnerabilities and these exercises were successful from the pen tester’s perspective.

Now they took that same exercise and applied it to the application protected by ShiftLeft and the outcome was they were not able to exploit any of those scoped vulnerabilities that we had described in the application.

So the proof point for this exercise was to make ensure that ShiftLeft took the onus of protecting the application from any existential breaches or exploits applied by both potential white hats or black hats in the real world.

Shimel: Got it. So fascinating stuff here. Talk to me now – so how – and forgive me and forgive our audience – we may not be knowledgeable about ShiftLeft – how does ShiftLeft in your company – not as the noun or verb here – how does what you do at ShiftLeft play into this Chetan?

Conikee: Absolutely. Great question Alan. ShiftLeft is the industy’s first convert solution that seamlessly integrates into DevOps pipeline and protects the application by identifying its attack surface without reacting to threats. And let me explain what I just described in a moment.

Often security solutions that have been thriving over the past few decades are built to essentially identify and understand exiting threats and thereafter protect the application. And by that I mean often when a threat is applied or an exploit is applied on the application that threat is published in public databases like missed CBEDBs and NBDBs.

Then after all of these instruments that protect the application are validating every request applied on this application informed of the threat landscape. Now often threats have a lifecycle, meaning an attack or attacks an application by identifying potential breach paths. Now after they attack it is titled as an unknown vulnerability and when it is discovered it becomes a zero day vulnerability and thereafter when it’s published in a database it becomes a known vulnerability.

Now note that often organizations are exposed in that journey from unknown to zero day known because it’s yet to be published. And for all these instruments that are protecting applications there is yet to update its policies to thereby protect the application.

So we at ShiftLeft have devised a convert solution that first of all understands the applications evolving at that service. Now in the world of DevOps we deploy application multiple times a day, a week or a month. Now as the application evolves to serve a company’s business needs the application’s attack surface is evolving as well. It’s either heading in a positive drift or a negative drift. Often more times it exists or traverses in the negative side because engineers are coding at high velocity and when they code they don’t take security as a function into consideration.

As a consequence attackers can figure out more creative ways to exploit the evolving attack surface of the application.

So ShiftLeft has devised a technique that automatically processes byte code into the CI portion of the CICD of DevOps and extracts what we call is the attack surface of the application. Now from the attack surface we extract a policy: the policy that defines how to protect the application in one time because we do not want to feed these alerts back to humans and ask humans to correlate and address and prioritize these vulnerabilities.

So in the spirit of respecting the fast motion of DevOps we extract the security profile from this attack surface and bootstrap an agent that is instrumented with the running application.

Now the job of the agent is to act like one-fold full-time employee. The agent has taken the onus of identifying what is a potential imminent threat that is applied on the application by a black hat hacker and thereby understand better this particular track that’s taking this application into an inadmissible state because often attackers want to first take the application into a state that they wish to take it to and thereafter either exfiltrate data from the application or perhaps install some malicious payload on the host.

So the job of the agent is to identify and segregate malicious intent and thereafter prevent that particular intent from manifesting.

Shimel: Now Chetan I sit here listening to you – and keep in mind I have 20 years – 18 years in the ______ site space and I remember writing articles and papers on when is a zero day a zero day, when is it less than a zero day, you know, this whole discussion around what we can do to make our applications less vulnerable.

And what amazes me is – I think we all agree, right, application security – at least for the last three to five years, maybe more – has really been at the forefront of cybersecurity research and so forth. And it’s only now that we’re starting to see solutions like ShiftLeft which are kind of not the same old same old taking a different view to an old problem or a different solution to an old problem.

And yet, you know, I just got back from Black Hat – I’m not sure if you were there – we hear about one breach after the next and they’re still spinning money out of ATM machines. Are we finally on the right track to you think? Are we making progress?

Conikee: Great question Alan. I would say that we have begun to make progress. You know, there is a journey, we have to embark on this journey and that journey means taking a practice and embedding into our typical pipeline – a pipeline of how we build applications, deploy applications and protect applications.

So this is a beginning but a great beginning. But what has enabled this beginning is inspiration. You know often we learn from other systems and what I’m sensing is security is learning from DevOps. DevOps is all about automation, meaning before Amazon – the advent of Amazon or any public cloud like Azure or GC there was – the notion of provisioning a host took more than one step: we had to write a multitude of scripts in order to provision a simple physical appliance.

And with the advent of companies like Chef Puppet and the public cloud they created this concept called this cloud formation; meaning how do you provision a virtual appliance in a matter of seconds and make it predictable thereafter? Because there is a system of record of your provisioning script and its evolution thereafter – that is one part. The other part is the predictability: how can you assess that particular script and understand its posture? Posture from a resource consumption perspective and resource provisioning perspective?

Now if you take these two same principals to the antiquated world of security – and I use the term antiquated because in the past security was all about reacting to threats and alerts and then we had to deal with how to prioritize alerts because alerts can contain mixes of true positives, false negatives, false positives, et cetera. So how can we steal a page from the book of DevOps? How can we automate security? Meaning you’re creating code, from this code how can we automatically extract its security posture?

Because in the world of security we are assessing code with three particular filters – or what they call SOSAs, sanitizers and syncs – SOSAs are where consumers interact with our application; sanitizers are where we are validating that input; and syncs are all communication points outbound from an application perspective.

Often attackers are attempting to create new syncs after they successfully exploit the application. So how can we extract this particular fabric or script and thereafter take a security function into consideration?

So this is our goal at ShiftLeft and we are hoping that this becomes a norm and other companies like us are adopting the same principal as well.

Shimel: We can hope can’t we? But it’s interesting. You know I – what’s interesting Chetan is no developer raises their hand and says, “I want to develop insecure code.” No ops person says, “Boy, I want us to be the victim of a breach or have a security incident.” Nor does any security person say, “I hate the rest of the IT team so much that I hope they get hacked.” Right? None of this is true.

But yet we have problems – and some of it granted is not just from a technology or toolset perspective, a lot of it frankly is cultural, right, as I’m sure you’ll agree as well. You know, and any time we could use tools that help foster culture, that help foster greater communication and teamwork – I mean I think we all benefit from it, right?

And I hear a little bit with ShiftLeft, you know, this is something that I think is a byproduct if you will of your solution, is that true?

Conikee: Absolutely. Well said, Alan. For the most part security should be an enabler rather than a department of no.

Shimel: You’ve got it.

Conikee: And security should – there should be a common lingua franca between various departments: meaning security, engineering, quality assurance and operations should speak the same language. The language should be comprehensible so that it becomes actionable.

Shimel: Amen. Amen. You know, Chetan when we first started I think I mentioned to you the time goes so darn quickly.

Conikee: Absolutely.

Shimel: We’ve blown past our 15 minutes onto 20 here so we may have to call it an end but for people who may be interested in the recent study you did with Cobalt – and my friend Carolyn Long is a VP at Cobalt and she is one of the smartest people I know in security – where can they get more information?

Conikee: Absolutely. Please visit blog.shifleft.io and on our blog we have a recently trending article that gives you the necessary detail about this exercise. And at the end of the article is the published benchmark’s test for those that want to delve into the detail – which I highly recommend – and thereafter visit www.shiftleft.io and – or contact us; we are happy to onboard your application and thereafter protect your application as well.

Shimel: Fantastic. Chetan Conikee CTO co-founder ShiftLeft. Thanks for being our guest in this episode of DevOps chat and continued success with ShiftLeft – your success is all of our success.

Conikee: Thank you Alan. Pleasure, and thank you for hosting us.

Shimel: Okay. This is Alan Shimel for DevOps Chat. You’ve just listened to another DevOps Chat. Have a great day and see you soon everyone.

— Alan Shimel

September 17, 2018
DevOps Chat: Cloud-Native Security with Twistlock CEO Ben Bernstein

The rise of containers and microservices has given rise to a need for a different type of security. Ben Bernstein, CEO of Twistlock, believes cloud-native security is the answer. Fresh off another round of VC funding, Twistlock has big plans to advance cloud-native security in the container space.

In this DevOps Chat, we talk funding, serverless and cloud-native security with Ben. As usual, the streaming audio is immediately below, followed by the transcript of our conversation.

Alan Shimel: Hi everyone, it’s Alan Shimel, devopsy.kinsta.cloud, and you’re listening to another DevOps Chat. I’ve got a really exciting chat guest and subject lined up today. I’m happy to be joined by none other than Ben Bernstein. Ben is the CEO founder of Twistlock. Ben, welcome to DevOps Chat.

Ben Bernstein: Thank you, Alan. I’m really excited to be here. Thanks for inviting me.

Shimel: Not as excited as I am to have you here. You know, as we were talking off camera, we’ve done other podcasts with Twistlock, but I don’t think we’ve had the pleasure of having you on. It’s a real pleasure. Ben, we’ve got some big news we’re going to break today. Twistlock has raised – is it a round C?

Bernstein: Absolutely. Yes, round C.

Shimel: Tell us a little bit about it.

Bernstein: We’re really excited about it. It’s a $33 million round, which bring us to about $63 million in total money raised. It’s led by Iconic Capital, which is pretty a unique family office that’s probably the largest family office in the world, headed by some of Silicon Valley’s top entrepreneurs, which is a very – something that really made us happy to see that we got the interest of such an awesome funds. So we’re pretty excited about this, and having them join our board. It’s also kind of unique because we sort of reached a growth stage, which is first for me, as a CEO, it’s kind of an interesting to see how the company evolves – sort of the level of maturity we’re starting to hit, which is just exciting. It’s just like seeing your kids go to college or something, I guess. [laughter]

Shimel: Well, look, my oldest son is leaving Friday for college. I know how that feels, Ben. So $33 million in this round, about $60-something million in total. As we were talking off-mic, I was telling you my experience with raising money. Raising money is never easy per se. When you have a good product, a good company, and the right space, it gets easier maybe, but it’s never totally easy. So congratulations on a sizable round here. It also coincides with a new release for Twistlock. Is that correct?

Bernstein: Yes, we actually recently released 2.5, which was a pretty interesting release for us, because it updated some of the – or reinforced our investment in Cloud-Native, which is not just container specific, but also augments our complete suite for Cloud-Native computing. So there were some pretty extensive serverless additions there, AWS Fargate addition there as well as some interesting capabilities around forensics, but overall it was a pretty extensive addition to our suite. So we’re very happy about that as well.

Shimel: So Ben, you mentioned an interesting thing and that is this concept of the whole Cloud-Native space is exploding. We have the Cloud-Native foundation, and all Cloud-Native tools and so forth. Originally, when I heard Cloud-Native, it was very cloud computing, AWS, Azure, that kind of cloud, but with the advent of containerization and Kuernetes and Docker and so forth, it’s really almost I don’t want to say dominated – or synonymous, but it’s so closely linked. From a security point of view though, Ben, where do you see the overlaps, and then where is the difference from Twistlock’s point of view?

Bernstein: You mean between specifically serverless and –?

Shimel: Yeah.

Bernstein: Yeah, so at the end of the day, I think there are two types of application security companies. There are box companies and there are cloud companies. When you think about cloud, it’s basically – it embeds in it the concept that you don’t own the infrastructure. It embeds in it the fact that the people that get the resources get everything in a box. They get the networking. They get the storage. They get everything with a click, whether it’s a public or a private cloud, and it embeds in it the fact that you’re using microsegmentation. It’s about code reuse. It’s about a lot of good things that we all are sort of learning in the past I’d say 8 to 10 years, unless you’ve been working in Google or some of the companies that probably invented this space in house.

Everything I just said is not about a very specific technology; it’s about specific concepts. These concepts are very disruptive to the way security is being built, and we at Twistlock, believed initially that _____ be the poster child of how existing security cannot protect containers, but when we’ve seen that, it’s not just about containers. It’s about other technologies that are being used by DevOps teams in modern organizations going through so to speak a digital transformation. We decided that we should really expand what we do to go beyond containers pretty early on, and go to really focus on Cloud-Native, which is a suite of technologies, that even includes VMs, but is all about giving or empowering these groups, DevOps groups, to be able to move fast and still secure their work loads.

This is something that’s hard for them to do with seven different legacy products, where they’re trying to script their way somehow into putting something on the hardware firewall, plus virtual appliance, plus something that needs to – that has a concept of a server, or was built around a concept of a server, where in reality, it should have protected the app, not the server. But anyway, that’s – and so in this – when you think about it in these terms, serverless and containers, and even VMs, if you use them in a certain manner it’s pretty much similar. If it’s immutative, it’s declarative, it’s predictable, a lot of things that really change even the level of security you could wrap around it.

Shimel: You know, Ben, I also wanted to make the point that so much of this technology is wrapped around the – it’s intrinsically linked with DevOps and a new way culturally of how companies approach not just application development, but specifically application security. I’m wondering – some Twistlock guys, 2.5 guy, your new version out, how does the whole DevSecOps thing movement manifest itself into real features and functionality that we see in the Twistlock suite?

Bernstein: So when we created Twistlock, we realized that existing security was sold to say seven different people in the company. You take the storage person and sell him storage security. You take the network person and sell him network security. You take the server person and sell him server security where when in fact, everything you were trying to do is to protect your app. Maybe there were other people who you sold different security solutions. What we’ve seen is these five people are actually one person in the Dev Ops area. It’s one person that does a single click and – or several clicks and gets all these good things, but he’s in charge of that because he gets all these things from a single source – the cloud.

In our bet, which went beyond just specific technologies, and it is kind of counterintuitive to security in general, we said that security had to actually contain multiple layers of security from the same suite. So there’s no point anymore in getting vulnerability assessment from one vendor and east/west firewalls from a different vendor, and anomaly detection from a third vendor, and _____ from a fourth vendor, obviously unless you have a point in it. Most DevOps team, as long as you could – as long as the product is good, they would be happy to get a suite that allows them to get all of these things from a single source.

That’s one of the bets that we’re taking. It’s a pretty dangerous bet because security has not been that way so far. Luckily for us, it seems to – it seems to be paying off so far. We have hundreds of customers and are doing pretty well.

Shimel: To say the least. I think you’re being humble. $33 million dollars more in the bank. The usual thing is you’re going to use it for sales or marketing and engineering and support, but specifically what do you – with this kind of rocket fuel in the tank, what we can we look for from Twistlock?

Bernstein: Well, to be frank, if – I was at the last DockerCon and I saw that people are actually looking for real cloud-native. There’s a real need. If you look at the language, you see that people understand the need for cloud-native security, and what we really want to create is the – a complete solution that people would love and continue to use. So we need to continue to invest in the product. The product is the key. We don’t know how to play golf with executives. We haven’t been very successful at it generally. We really think that what really shines is a good product and a good support team to actually make sure the customers are happy and are using the product right, and that customers are smart, and that they can evaluate things and buy through the channels that they want to use.

But we’re going to keep doing that, keep trying to make sure that our product is really the most complete solution possible. It gets challenging over time as customers move more workloads into the cloud. They have more demand. We need to excel at doing that, but we’re going to be very focused on making sure our products are – really excels and from that, we believe that given intelligent customers and a supportive team that supports the customer will continue to succeed. So not a lot of huge news there, but just given the demand and given the sophistication of Fortune100 customers, we need to keep – even keeping a product as is requires you to continue and grow just because of the requirements coming from customers over time. We need to do more than just keep the existing product as is. We need to really grow and support new technologies. So anyway, it’s a complex task and we’ll continue at it.

Shimel: Absolutely. Congratulations, Ben. Over the years, I’ve had a chance to work with many entrepreneurs, VCs, successful business folks. Any one of them will tell you doing what you’ve done here with Twistlock, regardless of where the end game actually winds up playing out at some point, is really something to be proud of. So congratulations to you and John and the rest of the Twistlock team on both the new version as well as the significant new raise.

As I mentioned to you off-camera, Ben, the time goes so quick where we’re pretty much right on the edge of our time frame here. Wanted to just quickly end up, for people who maybe aren’t as familiar with Twistlock, want to get a little more information, where could they go?

Bernstein: We have tons of information on our site, Twistlock.com, about use cases, about customers, about cloud-native in general. I encourage people who’d like to learn more to go there.

Shimel: Fantastic. Well, Ben Bernstein, CEO and founder of Twistlock, thanks for being this episode’s guest on DevOps Chat. Congratulations on all of the good stuff, all of the good news coming out of Twistlock, and continued success.

Bernstein: Thank you, Alan, for having me. Thanks everyone for listening.

Shimel: All right, a pleasure. This is Alan Shimel for devopsy.kinsta.cloud. you just heard another DevOps Chat. See you soon, everyone.

— Alan Shimel

September 10, 2018