devopsy.kinsta.cloud

Category: Podcasts

  • Approaching Security as a Fundamental Element of Software Development

    Approaching Security as a Fundamental Element of Software Development

    How we really integrate security into the SDLC? What are some of the best practices around secure software?

    In this episode of TechStrong TV, Jill Britton, director of compliance, and Stuart Foster, product manager at Perforce, joined us to discuss how to improve application security by making security a fundamental component of the software creation process.

    The video is immediately below, followed by the transcript of the conversation. Enjoy!

    Transcript

    Mitch Ashley: I have the pleasure of being joined buy a couple of folks that I’ve talked with before on a webinar and a few other videos. Jill Britton and Stuart Foster from Perforce. Welcome to you both.

    Stuart Foster: Thanks for having us, Mitch. 

    Jill Britton: Thanks.

    Ashley: Let’s start off with some introductions. Since you’re ahead in the time zone here, Jill, how about you go first?

    Britton: Okay, thanks. I’m Jill Britton. I’m the director of compliance for static analysis tools here at Perforce. I look after areas regarding coding standards, functional safety standards, security standards, anything in that area.

    Ashley: Excellent. Stuart, you want to introduce yourself? Tell us about Perforce, too. 

    Foster: Sure, yeah. I’m Stuart Foster. I’m a product manager for our static analysis products, both Klocwork and Helix QAC. And a little bit about Perforce, we help with enterprise development and DevOps at scale. We have a lot of various tools that help accelerate development projects in terms of your planning. So we have project and portfolio management tools, requirements management tools. We have things that help in the create area of the DevOps infinity circle for issue management version, asset management, code review. We have static analysis. We have various functional test case management and things like that that also fit in the verify. And we have a lot of tools that are here to drive your product at a high quality to market. 

    Ashley: Excellent. Excellent. Well, very timely conversation at least here in the U.S. with security of software, in this case distribution with the SolarWinds situation. We’re not going to be talking about that, but strongly the topic of secure applications is right in there, right at the top of the list. Let’s first start, I know we’re going to get into some coding standards and some things like that to equip people with some things that they can use. Maybe we first start with DevOps is more than just – and shift-left is more than just, okay, let’s start having meetings or talk on Slack with the security people and then suddenly we’ve shifted left. You know, there’s a lot more to how we really integrate security into the process of creating software. Maybe, Stuart, if you want to start sharing some of your thoughts about that and of course, Jill, get your perspective as well. 

    Foster: Sure, yeah. I mean you think about DevOps, you think about agile development and it’s the ownership of code; but outside of simply delivering functional requirements for your products, your applications, you know, a lot of teams are just thinking about security. So there’s a necessary to build that in directly. And what that leads you in to is selecting tools, selecting processes to ensure that you’re kind of baking in the security into your pipelines and making it automatic. That starts with thinking about coding standards and things like that. So there’s plenty of secure coding standards that you can focus down on. I think Jill has a couple of words for that.

    Ashley: I’m sure you do. Go ahead, Jill.

    Britton: Oh, I have many, many words on that. The important thing is that you pick the standard that you are going to be comfortable with, that you are going to be able to apply. If you’re supplying a product to a supplier, you will probably be told which ones to use. But you’ve got I think that cert – oh, where, it’s from Carnegie Mellon University. They’ve been looking at security for many years now and it builds up and it also takes experience from both academia, tools, developers, actually anybody who wishes to contribute and then of course it is looked at all the contributions. So this good security standard because it is there.

    But connected to that you’ve got the Common Weakness Enumeration so administered MITRE, and that is actually just a collection of vulnerabilities that have been seen and proved in the field. So you’ve got this whole wealth of experience and you want to avoid doing what the problems other people have seen. That’s where these kind of standards come in. CWE I would argue isn’t a coding standard. It’s more a measure. You know, have I seen this problem? Have I got this problem? And there’s another related set of numbers, let’s put it that, which is common vulnerabilities and that’s even further down the line. Before you even get to see weaknesses, there are vulnerabilities and you just have to use this information in your own developments. Don’t do what somebody else has already done is kind of the way to look at it in security in particularly. 

    Ashley: It isn’t like we don’t know how to create secure software. We do but we kind of do the same thing over and over, the same mistakes, right? 

    Britton: That’s right. 

    Ashley: Sometimes just out of not knowing and sometimes just human error. Sorry, Stuart, you were going to jump in there too. 

    Foster: I was going to say, so effectively one of those, CWE is effectively describing issues that are not necessarily related directly to a language; and the other ones are specific rules that you follow to ensure that you don’t run in to these types of vulnerabilities or weaknesses, right?

    Britton: That’s right. That’s it.

    Ashley: Yeah, there are specific things that are in Java, PHP, you know specific languages that are very well used, very popular of course. Go ahead, Jill.

    Britton: No, the thing is when you look at the CWEs, they are categorized to languages or to systems or just to general sort of areas; but there are then what they call the lists. So you’ve got a list for C and a list for C++ and there’s a list for Java. Then you also have a top 25. Now a top 25 may or may not apply to your own product, but it’s always worth having a look. If those ones don’t apply, there’s another sort of secondary list. It’s emerging, and you could look there. There’s always something to check against for any sort of software.

    Ashley: Are most of the things on that list you find have been there for a while? SQL injection, buffer overflows, things like that? Or do they tend to be newer things? Are we seeing the same things over or would we find a lot of new things?

    Britton: It’s a variety. You do see the same ones over and over again. There are some weaknesses that were on the 2011 list and are still in the 2020, but there are new vulnerabilities as new types of development come in and you get more skilled hackers really. You’ve always got to consider that the hacker is always one step ahead.

    Foster: And you kind of see that these standards or these top 25 lists, they change after every couple of years and they spend their time kind of reviewing the vulnerabilities that have been found in opensource projects and other places and kind of give them a weighting. So while you may see the top 25 have a pretty similar overlap between multiple years, their rankings change. So it kind of shows to you that in general software is becoming more secure but not as secure in these new areas. So you see some vulnerabilities lower and the new ones come up at a higher rate. I think it also depends on where software development is currently is in terms of market. What’s the new hotness? Is it Internet of Things connected to software? Is there a new language coming out that has never been attacked? So you see the priorities change on these top 25 lists.

    Ashley: Let’s step back for a moment. How would a development team or developer – I assume most folks are probably familiar with at least CBEs or some things like that, but maybe they haven’t dived into the top 25 for CWEs. What’s the best way to do that? Is it just go out and read about them, educate yourself? Do people, do development teams like, okay, I’m gonna dive into this and research a little bit more and I’ll share with you or how do people apply this to their teams?

    Britton: Well, there’s several ways they apply it. Sometimes they just hear about it, something like we’re doing talking today; and they go, oh, is this going to be relevant to me. And look, it’s a few things. As I said, sometimes a supplier specifies the angles to have proof that you’ve checked against this. And of course the static analysis tool, we can map checkers and messages against these vulnerabilities and these weaknesses so that we can show your code is exhibiting these features. But it’s really important to know what you’re going to be looking for and then do something about it as the static analysis tools can show you, yes, you’ve got this vulnerability but really it’s not off point just leaving there. You must do something about it.

    Ashley: You know, it’s interesting. So many, not everyone of course, so many of the vulnerabilities are about input, right, to a user interface of some type, web, OS command, mobile device. It’s a lot about neutralizing, dealing with the input that you’ve received, making sure that they’re not injecting something that they shouldn’t, getting access to a directory, all kinds of things like that. So it’s things that apply across multiple technology stacks and multiple applications. So generally speaking it’s not like this is a skill you’re going to use on one app and then you have to relearn it for the next app. You carry that stuff with you. It’s part of your skillset you build as a software engineer. 

    Britton: Yeah, it’s education, isn’t it? You know, it’s just everybody gets more and more experience of this type of issue. It’s not that long ago that security wasn’t at the heart, but now it is.

    Foster: And if you think about it, you know there’s kind of different levels of maturity in terms of security, right? You start developing because you know you have some types of vulnerabilities and you start working towards that. But if you think more longer term about your projects and how they might be very important IP or pieces of software that are embedded over time, it becomes more important to think about security because, you know, being able to fix some issues that you know about versus following a secure coding standard to ensure that the reliability, the strength and hardening of your software to begin with means that years down the line when somebody has integrated your piece of software into some type of embedded system that can never be touched, then you’re lowering the risk of there being vulnerabilities permanently out in the market for a very, very long time. So that becomes kind of the second level maturity when you’re developing and thinking about developing more secure software.

    Ashley: Now you mentioned, I forgot which one of you mentioned something about you might encounter something under a contract with the customer that requires specific security standards. One example I know that you all deal with quite a bit is ISO 26262 for automotive industry. Here you’re dealing with safety of course as well as security. Talk a little bit about how something – do the industry-specific standards really kind of go off the reservation to do something really different or are they building on what we do generally in software and then applying it to their specific domain?

    Britton: We could take ISO 26262 which is for vehicles. It’s very much good practice. It’s nothing totally off the scale. They recommend for software the use of a coding standard. You are allowed to use any coding standard you like, but then there are specific categories of software violations that you need to be able to detect so whatever you do. MISRA is sort of the classic for 26262 because it was developed for vehicles. But you have other industries with very, very similar requirements from software and now we’re moving into the scale of security. Back to vehicles again, it’s ISO 21434 which isn’t out yet; but that also requires the use of a coding standard and the violations they’re looking for are still very, very similar. It’s the same type of things. Quality software is what we’re always looking for.

    Foster: Because as I kind of mentioned a minute ago, you think about a car, that’s a big rolling embedded system that’s more or less permanent once it’s complete and it’s really hard to fix later. So you can understand how security as an oops, we’ll go back and fix it versus security as a process becomes more important. That’s exactly why ISO 21434 is becoming a thing or will becomes a thing once it’s releases. But we already have customers in industries looking at it and following along and understanding what that impact is going to be because to connect the cars was a big rolling computer.

    Ashley: You know, this is a little bit kind of parallel to just creating software but not too long ago I went through a web trust audit and very much like a standard like you’re describing it is do you have a process, do you have something that you follow, should follow, and do you follow it and do you have evidence of that. So many of the standards are just the technical details of what you should do to write pure code. It’s do you have the processes around that and are you following those and in some cases you do an audit. Necessary if you want a WebTrust, for example, certification or others. So it’s not just writing secure code, it’s if you say you’re going to do it, then you have to do it and demonstrate that you’re doing it and have some form. So what I’m saying is it’s really an investment that you’re making by writing secure code following some of these standards, but then that’s going to help you down the road meet other requirements. 

    Foster: Yeah, it makes it easier when you’re trying to implement say ISO 27001 which is information management security management requirement compliance thing. And there’s controls in there and what you’re talking about per the audit is putting in controls for your business and there are various areas in those controls that in many ways relate to the software that’s developed within the company. So yeah, by following some of these security standards you’re in essence helping with those controls later when/if you go and certify your company as that type of ISO requirement.

    Ashley: You know, Jill, I know you deal a lot with governance and some compliance and things like that. What are some of the best practices around secure software? We’ve touched on quite a few already but are there some other thoughts on other best practices to help people move up that maturity scale?

    Britton: Well, it’s a very basic part of any process really. Choose your metrics. How are you going to actually measure and see how you are progressing through your process? And similarly to software weaknesses, when you get your metrics, what are you going to do with them? There’s no point in collecting a whole collection of them and saying, yep, we’ve got our metrics. What does it mean? Does it mean that we have got 27 security violations last week? Oh, great, we’ve got a number. But what are we going to do about those 27 or so security violations? Again, it’s back to process. What is your process? What is an acceptable level of these metrics? And you can have metrics working all the way through the cycle right from requirements to development and test. You can actually have measures of how you are progressing through your cycle.

    Foster: And also what you find when you look at a lot of these security standards and their vulnerabilities, they have kind of the impact likelihood, cost to remediate, potential issue or risks that you see in the market when one goes out. So alongside that, a lot of the standards as well have kind of a risk scale or a prioritization that helps you – gives you an idea of where to start. And then in terms of the process, it’s not always a matter of running a tool or trying to follow these standards and trying to fix all the bugs. It’s important to start somewhere. What we typically call it is baselining where you kind of know your current quality of your software as it is right now, and the point is to start following a security standard and stopping new vulnerabilities from being introduced, and then you can go back and start working on the prioritization of what to fix to start cleaning up. Right?

    And also when you look at it and it seems insurmountable, you might want to spend some time looking at your past kind of customer weaknesses that were found or bugs that were found and kind of understanding in addition to what the standard is important, looking at what types of vulnerabilities you’ve produced or have reported back and focus there. That’s a good start, you know? You know kind of what your developers’ training weaknesses are and then you can implement kind of teaching, learning, training processes as well to help getting your development team more comfortable, more experienced in dealing with these types of issues. 

    Ashley: Sometimes also building on what you said, it can help bring to light other issues along the process like, for example, we keep getting libraries introduced into code that we don’t realize are there and they show up as a security vulnerability. Okay, well, let’s go fix that part of the process too so we know that those things are there and when updates happen they go through some defined process. Another one might be it’s not just new things we find but also repeats, things we consistently find, occasionally over and over whatever frequency. Okay, if we’re fixing them but how do we stop introducing them? What are good ways to help that? It might be education tools, et cetera. Lots of ways to do that. So to both of your point, to your point too, Jill, don’t ask questions you’re not going to do something about the answer, right? 

    Britton: That’s right. That’s exactly right.

    Ashley: So you have to take action on it and of course that’s ultimately why you’re doing it. You’re not just going through the motions to meet some paper standard because that will catch up with you very quick if you do something like that. 

    Britton: _____. It will. You’re absolutely right.

    Foster: Just buying yourself technical debt and risk.

    Ashley: Yeah, exactly. And it will catch up with you. How do you introduce this to some people words like process, standards, compliance, audits, anything with an ISO in front of it? Those kinds of things just sort of kick up the you’re cramping my style here, right? You’re putting a damper on my ability to be creative and write code. I’m bring a little bit hyperbolic, but not everybody reacts to those things favorably. How do you position this with organization so that it’s not just a dreaded thing we have to do, there’s actually really business value to it and it’s good for you and your career?

    Britton: Well, the whole thing with following a process is the developers can be involved in this process and then go through their careers excel help develop this process. But the process shouldn’t be stifling. It should never stifle the creativity. The whole idea of the process is tell me what you do and how do you prove that’s what you do? Okay, that’s fine. Now maybe how can we tweak it and do a little better? It’s the pro thing of continuous improvement that comes up in all the audits.

    We’re actually ISO 9001 certified. So we’re all going through this on a regular basis. You know, but we’re not trying to stifle our developers. They will say, “Well, okay, we have this process. It’s not quite working for us. Can we make some modifications?” We go, yeah, looks fine. We talk to the necessary people and usually it’s absolutely fine. So we involve the developers all the way through and they will then follow it because it’s their own process. 

    Foster: Yeah, I think the goal is the least amount of friction and introducing it over time. And as Jill said, there’s continuous improvement, right? It’s a long-term goal. It’s not something that’s done once and you get over the big hump of pain and it’s a ton of crunch work. It’s just something that you do over time. You look at agile development, right? People do standups. There’s processes, you know. There’s a level of documentation and planning that isn’t overly annoying. You know, the goal of DevOps is to automate a lot of those finicky processes so that they’re fast, they’re repeatable, they give time back, right? They report information back as quickly as possible so that they can be resolved quickly, right? That’s just lowering the friction, the pain of going back and fixing a feature or some sort of development commit, right? And then you’re just layering on another check layer of security, right? And it should be painless. It should be easy.

    As you mature over time, then the quality is going to get better and that goes back to the metrics that Jill was mentioning, not just having a number but seeing that number go down. And then having your teams be able to see that kind of dashboard or information as a feedback mechanism to say, you know, you can visually see the qualities improving. You know, customer reports of issues from the field are going down because our quality is going up. Why? Not just because of security but because of all the automation and the unit tests and checks that you have written and you’ve made the effort to produce better quality code and it’s going to pay dividends. 

    Ashley: I like the idea of tying it a continuous improvement process whether you label that as quality or other things because you can apply that to your DevOps process, to your tool chain, to the quality of your code, safety, security, making compliance easier for you, just improving your overall how you write software, maybe workflows and things like that. So you can make some of these compliance activities or standards activities really part of just maturing your organization and getting better at what you do, working on the organization as well as in it.

    Question for you, how do things like the privacy laws, you know, GDPR, CCPA for California, do those fall into this realm as well? Do they impact the developers as we’re talking about the use of customers’ information and of course we’re handling that in the applications? Is that something developers need to be that familiar with or is that more of the marketing department that are sending e-mails to people?

    Britton: It’s not greatly down with the developers, but it really depends on the type of data that they’re handling. Looking at the tools that Stuart and I work with, we’re not getting/bringing in external tools because it’s a static analyst tool. So we’re not looking at that. But when you get to things in the field, the developers have to be aware if they are going to be handling personal information and they then do have to have GDPR and that has to be included into any process – how are we going to handle this.

    Foster: Yeah, and there are coding rules that, you know, try to help with these types of things. You know not having hard coded secure passwords, not using magic numbers, various levels of encryption that should be written, or warnings that you’re using an outdated type of like encryption in your software, the ability to handle unvalidated inputs and outputs. Things like that are going to help cover some of the requirements for GDPR. So you need to kind of understand how your system is interacting with other systems and securing your APIs and all things like that. So it’s not just your piece of software at that point. Your developers need to be aware as to how your software is going to be used and implemented and how it’s going to communicate with other pieces of software or systems. 

    Britton: That brings us back to the weaknesses we were discussing earlier because there are weaknesses that actually cover that type of data.

    Ashley: Exactly.

    Britton: What are you doing with it.

    Ashley: Vulnerabilities in our software could lead to the loss of data. That could be a violation of those privacy laws of course, as well as other impacts, whatever it was. Well good. This has been a lot of fun. I’ve really enjoyed talking with you about this aspect of creating secure software. We’ve mentioned static code analysis and tools a little bit. I know this isn’t a commercial about Perforce, but would one of you just kind of position for us where in the process of what we talked about today where Perforce really plays in kind of your specialty in this area?

    Foster: Yeah, sure. So in terms of our product which our static analysis products, both Klocwork and Helix QAC, we fit in many different areas of kind of a development pipeline and process. We exist to make sure vulnerabilities are found while the developer is currently typing their code on their desktop. We also fit in for gated checks and quality checks in your SI/CD pipelines so we can quickly report back issues and also your typical if you’re running a nightly on a server or if you’re deploying checks and building your software in the cloud and provision instances and any sort of thing like that. We’re basically available anywhere. So we help you shift left as far as you want. We provide those security checks anywhere just for the ability to drive what I would call continuous compliance. 

    Ashley: Excellent. Great. Well, Stuart, Jill, it’s been a lot of fun talking with you and hope everyone – wish you the best in your journey on creating even more secure software. We’ll speak with you again soon. Take care. 

    Foster: Awesome. Thanks, Mitch. 

    Britton: Thanks.

  • The Use of Open Source in Mainframe Environments

    The Use of Open Source in Mainframe Environments

    Open source is a critical element of IT computing. As companies modernize the mainframe and start to add components to mainframe applications that are off the mainframe, they should to be able to build, test and deploy the components of that application that go across the mainframe and different computing platforms in the same manner. Organizations need open source tools to run on the mainframe to be able to drive those processes in a unified manner across all the different components of that application.

    Tim Willging, chief architect and strategist, and Peter Fandel, mainframe open source evangelist of Rocket Software, join Mitch Ashley to discuss advancements in the use of open source software with mainframe apps and tools. Tim and Peter challenge some long-held beliefs around mainframes, open source software and the increasing role open source is playing today.

    The video is immediately below, followed by the transcript of the conversation. Enjoy!

    Transcript

    Mitch Ashley: Hi everyone, I’m very pleased to be joined by a couple of great gentlemen here to talk about some interesting open source, security, mainframe, DevOps, all kinds of good things. So I’m pleased to be joined by Tim Willging who is chief architect and strategist for the mainframe business unit with Rocket Software and Peter Fandel, senior director of product manager mainframe open source. Welcome, guys.

    Tim Willging: Hello, thank you.

    Peter Fandel: Hello.

    Ashley: I think I might have said your name correctly, Tim. Is it Willging? I don’t know if I said that right. 

    Willging: That’s correct. That’s fine. 

    Ashley: Okay, good. Would you start? I’ll have you both introduce yourselves. Maybe you can also introduce the company, Tim, and then we’ll have Peter do the same?

    Willging: Sure. Yeah, I’m Tim Willging. As Mitch mentioned, I’m the chief architect and strategist for the mainframe business unit at Rocket. I’ve been with Rocket about 15 years and spent the majority of my career developing commercial solutions for the mainframe and mostly around database tools, DB2 tools. Rocket Software has been in existence for 25-plus years and is a privately held company and specifically we do software that’s not just on the mainframe but let me talk mostly about the mainframe software.

    Rocket has a development partnership with IBM and many of the solutions we develop are IBM branded; but we also have many things we’re doing Rocket direct; and open source, particularly the subject of this call, is one of those things that we also sell direct or partnership with companies direct to provide that open source. So Rocket has a long history with the mainframe and we believe in its future and in its sort of importance in the computing divisions across many of the largest companies of the world. 

    Ashley: Excellent. Great. Peter, if you would introduce yourself.

    Fandel: Sure. Peter Fandel, again. I’ve been at Rocket for 19 years. The great majority of those years in engineering management, and I took over management of the open source porting team about, oh, four or five years ago and transitioned to product management about a year ago. And I manage both the open source porting portfolio as well as the Zowe portfolio, and both of those portfolios are critical components to the modernization story for the mainframe in large part because of the huge amounts of open source that are out there that you can _____ can get them running well on the OS as well as attracting talent, the next generation of developers. So that’s really our focus area for open source.

    Ashley: Excellent. You know, maybe you ought to start – and I started my career in the mainframe, too, and so I understand/kind of get where you’re all coming from in the market. There’s a lot of built up beliefs about mainframes and applications and whether you should think about porting or not or, you know, replatform them, leave them alone. Probably like you, I’ve learned it’s very costly and also fraught with danger to just go rebuilding applications, you know, because you think you want them on a different platform. You have to have a really good reason. But open source is actually a really important part of the mainframe ecosystem. I think maybe folks don’t really realize how important it is. How would you guys kind of set that up to give people an idea for the role that open source is playing in the mainframe environment today?

    Willging: Well, I’d start off by saying open source is incredibly important part of IT computing. I mean it’s something early career professionals are learning. It’s a huge way companies are running their business today and rather than writing things from scratch, many are starting with open source and then augmenting or augmenting existing systems with open source to modernize that open source because of its collaborative nature, how different companies and individuals contribute to that open source based on the needs of the industry. You really can’t separate the needs of our organizations that are running mainframes from that open source. Open source is a critical component of running your compute division. So without it, I think the mainframe would be, you know, would have a big missing part in what companies need to run their business today. 

    Fandel: I think also there’s a huge cost savings associated with that because you take any recent graduate and even not-so-recent graduate of any comp sci program and you ask them to start developing applications, they’re going to expect to build those applications on open source building blocks. And if they’re not there, your cost goes way, way up. As well, another area where open source is a must have is DevOps where if you want to – do you want to have a different DevOps tool chain for the mainframe than the rest of your organization or do you want to unify that? And if you’re going to unify it, it’s got to be open source based.

    Ashley: It’s a massive source of innovation and I don’t know if we’re here yet, but if we aren’t, we’re quickly reaching a point where you can’t operate a company, a software team without open source software. And if you aren’t using it, I can promise you your competitors are. So take advantage of it, right? Well, let’s talk a little bit about I know Zowe is of course a big important part of your strategy. Break it down for us. Go into a little bit more detail about where the places, roles that open source does play a role in the software development process and operational systems of mainframes, kind of paint that picture for our folks, our listeners. 

    Willging: I think there are a lot of different ways it plugs in, but the one that I hear most of the companies I talk to is precisely what Peter just said. You know, the mainframe through many years of existence has been very waterfall-ish I’ve heard some companies because of the reliability. It’s the security, the maintainability of the mainframe that there’s a process which changes are rolled out in large batches. As companies modernize the mainframe and they start to add components to those mainframe applications that are off the mainframe, like in the hybrid cloud or have a web frontend or a mobile app frontend to a backend service or application that’s running on the mainframe, they want to be able to build, test, deploy the components of that application that go across the mainframe and those different computing platforms in the same manner.

    And to have tools, pipeline building tools, pipeline organization tools that have tools that build these environments, run the tests, destroy these environments, they need those open source tools to run on the mainframe to be able to drive those processes in a unified manner across all the different components of that application. So many companies today are just starting – you know, mainframe companies are starting on this journey. Many of them are hand rolling over the years have hand rolled sort of tools to try to have them plugged in, but that can get expensive as we’re talking about. So to get that open source running on the mainframe it’s important for them to be able to deploy those DevOps principles and practices in that application management lifecycle process. 

    Fandel: I would –

    Ashley: Very much so. Of course there’s also more with that too. Go ahead. I’m sorry, go ahead, Peter.

    Fandel: I was going to build on what Tim said by introducing a really important distinction and that is that there is three very different ways you can get open source onto the mainframe. Most of the open source we’re talking about is units based. And you can do it by a Linux on Z, you can do it by a zCX Docker container or you can do it by UNIX System Services. And it used to be that the low cost way of doing that was Linux on Z and now more recently zCX because the porting is not much work to do because it’s all automated just port to the hardware which is done by compiles on those systems.

    But doing it on UNIX System Services is a lot more time consuming if you do it manually. But the advantage if you do port well on UNIX System Services is you’re close to the data. I mean if you’re running open source on Linux on Z or on zCX, it’s akin to running it on another machine on the network. You don’t have direct access to the data. Everything has to pass through TCP/IP. But UNIX System Services is part of z/OS and so you have direct access to all of the important data in your organization that MVS data. And that’s a key distinction. Where Rocket plays is open source on UNIX System Services on z/OS. So we are close to where the data is and we have solved the problem of high cost of porting through this GCC glibc technology where we essentially automate the port of open source by simply compiling and linking in GCC and it injects the z/OS specific changes directly into the binary output of the build. 

    Ashley: Talk about that process. Is that something that you’ve been doing a while? Of course going and changing binaries, most folks probably don’t do that as a natural first thought. But it does sound like a pretty straightforward way of minimizing the steps that you have to go through to get to begin using this. You can actually do it _____.

    Fandel: Yeah, it’s just we’ve been working on this GCC glibc port in this manner for over two years, and it’s just coming to fruition now where in first quarter we will be releasing new versions of several of our ports that are not ported by hand. They are ported simply by building them with GCC, and we hope to have transitioned our entire portfolio by the end of 2021 to be built in this manner. And that means we no longer have to modify the source code because upstreaming is no longer a question, an issue. It means that we can turn around security vulnerability fixes much, much faster than we could otherwise.

    Ashley: I don’t mean this in a hyperbolic way or kind of playing up to you in any way, but that kind of an approach can be a pretty big game changer because you’re really asking people, it’s a low lift instead of a heavy lift to port if you will.

    Willging: I’ve talked to many through the z/OS community who believe that due to the effort of going through these ports in the past, they felt that a better strategy was to run your open source close the mainframe – I say close with parentheses – or on a partition on the mainframe that’s running zLinux because the porting effort to get open source running on zLinux is minimal. And so they felt that the version currency, the security fix as Peter said, and the breadth of the packages available, open source source packages available on z/OS proper would always be behind. So they felt that was a better strategy to run it on zLinux.

    We don’t believe that at Rocket. We believe, as Peter said, that there are technical advantages to running that open source both from a DevOps standpoint but even more so from somebody who is looking to augment a legacy application say via some Python library that they’re looking to deploy where they’ve found some open source with a particular function. Running that Python on z/OS and accessing the data directly in concert with the legacy application that’s running or via some subfunction in that modernization effort is vastly different than running that Python in zLinux or on a machine off-platform and reaching into the data. The latency of what you can do, trying to embed that open source sort of in transaction, in the code in transaction becomes much more difficult to do. So we feel that Z customers as they’re looking to modernize will benefit greatly from open source running on Z proper. 

    Ashley: And as I understand, I think as you were describing it Peter, really you are talking about that compile process, right? That’s where you’re introducing GCC glibc libraries. So that’s where this process is happening, changing things in a binary to adapt it to be able to use open source. Is that pretty much it or what else do you have to do to verify or test it or what are folks going to want to go through as they start to introduce this into the process?

    Fandel: Once we build it, we obviously have to run it through the test suites and we have two forms of testing. We have the built-in test suites that most open source products come with. I mean if you go to Git or Python and you download the source code from the community, it will be the product plus a huge volume of test code, self-test code. And so we execute that test code to make sure that it performs with the same results as on a test UNIX System. I think we used Ubuntu as the comparison and we compare the results. And if there’s any divergence on z/OS, then we inspect those divergences.

    And then in addition to that, we have a series of tests that test the z/OS specific capabilities that you want in open source. That’s mostly relating to the ASCII/EBCDIC conversion because z/OS and including UNIX System Service assumes EBCDIC is the default. Most of the data that you’re dealing with is in EBCDIC. And so our philosophy in porting on UNIX System Services, ASCII is the default internally. We compile with ASCII. Communication between open source packages and programs is in ASCII. But when you reach to the operating system, that’s where the conversion has to take place. So we have extra test suites for all of our ports that make sure the ASCII/EBCDIC conversion is taking place correctly both in file IO and pipes. So that adds to our testing process. So it’s not just build it and ship it. We do thoroughly test as well. 

    Ashley: That’s great. I’m glad you explained that too because there certainly are some fundamental differences in the operating system and environments down to the characters that –

    Fandel: Sure.

    Ashley: – that we’re using. How about are there any kinds of applications that are more easier to port this way? That’s not the right way to say it. It’s not porting it but, you know, understand starting to use open source this way or others that maybe are a little more challenging, you might want to wait to take on that kind of an app? How would you recommend people get started?

    Fandel: Well, I mean we found that the porting a language is the most challenging because it just dips into every aspect of the operating system. So our conversion of the Python and Perl ports will be the last ones we release at the end of 2021. Our Python and Perl today are still, the ones we’re releasing are based on manual porting efforts.

    Ashley: Okay, good. Any thoughts on that, too, Tim?

    Willging: I thought your question originally was like talking about a customer application they were looking to augment and add open source to what might be easier.

    Ashley: I would like to go there, too. That actually was my – but I’m glad you answered it the way you did, Peter. I’m curious from a customer perspective data intensive, things that are ultra-high security, high volume, lots of parameters can go into deciding what kind of an application you might use first to go down this path. Any thoughts on what you’d recommend to look at, to pick what app you might try this with?

    Willging: I mean there are a lot of mainframe applications that leverage CICS for online transaction processing and CICS is a great environment to deploy open source. It’s been very forward thinking in its development to take and say a CICS application that’s written in COBOL and say you wanted to augment that via some modules that are written in open source language. That’s very possible and to make that communication really between the first thing you think about is there a runtime environment that’s involved, how do I want to lower that runtime environment so it doesn’t load every time that transaction is moving or it’s moving from the legacy code to the modern language?

    If there’s not, if it’s just a binary without a runtime environment, then you don’t have those types of concerns. So thinking about that and thinking through how you’re laying that out from an architectural standpoint and making sure you think through that. The speed of transition between legacy and open source language environments, that’s probably the biggest concern depending on what you’d like to do. But many companies have successfully done this and gone off and said, okay, we don’t want to necessarily go and rewrite my 30 million lines of COBOL. 

    Ashley: In a billing system probably not where I would start.

    Willging: But there’s a new function we would like to add that potentially goes and augments via reaching to another system where customer data is available that you certainly can do that and many companies have done that successfully. Again, leveraging more modern languages and having that available for that augmentation is much cheaper, much less risk than is involved in a total rewrite and a re-platform which is again even then much of your language that you’ve written or much of what you’ve written won’t even necessarily run on a new system in the cloud. So replatforming is also very expensive or very risky depending on, again, the number of lines of code.

    Anyway, I probably went too far there but that’s sort of how I think about it from an online transaction standpoint. But a batch, different again, that you might just have some data that you want to reprocess and write a new output from that batch and augmenting there with open source is much easier because, again depending on the language can it run in batch, does it require to be hosted within a web server? But even then you can still have a batch process that alerts a web server to now go read this files so you can embed your open source within a say WebSphere Liberty or some web server that can go and even augment batch processes that way. It’s really getting the right architecture down first to say how you want to do that. 

    Fandel: Another area is data science and machine learning. You have for the past 20 years or so you’ve had academics and scientists and corporate engineers developing machine learning extensions, data science extensions, and they haven’t been writing those extensions in COBOL. They’ve been writing it in Python. And so that’s the go-to language for data science.

    Ashley: It does give you access to a lot of new innovation just because of the language as well as the platform –

    Fandel: Exactly.

    Ashley: – and software stack differences. Two other areas I’d like to explore in our kind of time remaining, you mentioned DevOps and the role that open source plays in this. How does it help facilitate moving a mainframe application or developing a mainframe application now into more of a DevOps fashion? Are there some things inherent about because it is, you know, libraries that have a compile time process and there’s other tools? There’s Zowe and things like that can help you with interfaces and tool chains, workflows, things like that. What are your thoughts on that topic?

    Willging: Go ahead, Peter, you start.

    Fandel: Well, I was just going to say that Git, since we release Git I think it’s three years now for z/OS, I think it was our number one download within 30 days of release. Everybody in the world is using Git now –

    Ashley: It starts with Git, doesn’t it?

    Fandel: – or Source Code Control. And so it, like I said before, just enables you to have a unified DevOps pipeline and that’s just huge.

    Willging: Yeah. I mean to add to that, to have all of your source code in Git and then kick off a build process but then thinking of the mainframe and some of its legacy components like the parts of an application that help that make up the database. You know DB2 is very big on the mainframe. DB2 for Z. So to say that you can take those parts of the application touch the tables or smooth the tables in DB2 and extract those and then store those in a source code management repository because in DevOps you really want to store everything as code and represent everything in code so changes can be tracked over time. So taking open sourced, taking some even commercial tooling, taking some open source scripting languages, open source to allow you to interact with a DevOps pipeline that’s running off the mainframe that’s driving the build and reaching in. You know, DevOps is really, again, not a product. It’s a culture and it’s a –

    Ashley: It’s a process. It’s a way of creating software, right?

    Willging: And so to build that DevOps pipeline using those tools but then have the tools – the DevOps tooling and scripting and running on the mainframe to be able to interact with the mainframe sometimes is a combination of open source, purchased tooling, and then hand rolling the pieces in between to fit your company’s needs. That’s really not possible. Again having a Git client for the mainframe and a lot of the other tools there are really important for that process.

    Ashley: I’m curious about the security side of this. You know, a lot of application of air gap requirements things like that or the frequency or timeliness of security fixes. I don’t know if there’s a lot of difference between mainframe environments versus open source. That tends to happen of course more transparently. Maybe what are some of the advantages, pros and cons from a security standpoint of taking this approach?

    Fandel: Yeah, I’m glad you mentioned that because that’s one of the major changes that we have put in place this year with the introduction of the Rocket Open AppDev for Z Solution Bundle. We have switched from sort of old style, one at a time download of tarball and FTP it and then run through ten steps to set environment bevels and get it installed for each source tool or language. And as an example, GIT, which has four dependencies meant four or five downloads. Then you FTPed them over, and then you do the ten steps four or five times. It’s like _____ way. We’ve switched to conda as the system for downloading, installing, and deploying open source on z/OS. That has made a huge difference in terms of the user experience and the ownership experience.

    So it’s now once you have conda installed on your mainframe, that takes about 30 minutes including the download and the setup, then it’s a single command to install Git and all of its dependency from the command line; and further, conda has this concept of a channel which is their word for a software repository or repository of software you download from. Conda supports something called a file channel as well as internet channels.

    So in addition to this, we have set up two internet channels at Rocket, one public one at anaconda.org that anybody in the world can use to download from and a secure channel server on our network that is authenticated for our customers on support. And then additionally you can set up a file channel which is on your premise which is ideal for air gap systems. So if you have an air gap mainframe, you don’t want it to have a connection to the internet, you can set up a file channel, populate it with the full contents of our bundle and then all of the developers within your organization can download at will from your on premise channels. That’s a big advantage in the area of security. 

    Ashley: Those some really big ones, big changes. 

    Willging: It is a big change. I mean there are a lot of companies that have said that the only delivery mechanism for software to my mainframe is through IBM Shopz. So you know, I have some feelings on that. I think that’s old school thinking as far as I’m concerned, but there’s still a culture amongst many mainframe customers that that’s it. I think that is changing and to make it simple, to guarantee that it’s secure and to provide options is what Peter just went through I think is important to help increase adoption of open source on the mainframe. And so Rocket is really leading in that area to help make that possible. 

    Ashley: It is culture. It is sort of tried and true. You know, if it’s not broken don’t fix it. But on the other hand, when you start to demonstrate new capabilities, new benefits, speed improvements, more security, whatever it might be, access to innovation, suddenly that becomes maybe a compelling reason to start to break down some of those traditions or barriers or kind of think about adding this as a capability. You’re not talking about changing everything, right? We’re talking about adding capability to how you create software in a mainframe and expanded environment, correct?

    Willging: Yep, exactly. 

    Fandel: Sure. 

    Ashley: Good, good. Well, where can folks learn more about this? I don’t know if you have any trials or free downloads or you’ve been in beta on some things that are GA or coming out in GA. What’s the best way for folks that want to engage with you and find out more and kind of give this some –

    Fandel: I think you can Google z/OS Miniconda. You can Google Rocket Open AppDev for Z which is the product release from September and that will lead you to our product pages, our download page, documentation on how to install z/OS Miniconda which is the bootstrap to get you started. There is a couple of published videos. If you look at the Open Mainframe Projects Summit from September and look for, under the video there, “Demonstrating a Secure System for Downloading and Installing Software on z/OS,” you’ll find a video there that shows how it’s used. 

    Willging: The only thing I’d add to that is go check out zowe.org.

    Fandel: Thank you, Tim, yes, absolutely, zowe.org.

    Ashley: Don’t want to look past that, of course. 

    Willging: It completes everything else Peter said. 

    Ashley: Very important. 

    Willging: Important stuff. We haven’t really talked much about Zowe, but yeah. 

    Ashley: We’ll have a chance to do that. Would love to have you back and we can delve into that. I really would like to explore some more about this delivery of software into production and how do we start to evolve and expand helping more organizations consider how they might do that so there will be some great areas to explore further. So look forward to doing that with you. It’s been a lot of fun talking. Any parting thoughts before we wrap things up here?

    Willging: Just real quick, I’d encourage those mainframe z/OS shops, if you don’t have a policy for downloading, consuming open source, I encourage you to get your thought leaders together, you know think about the advantages that open source will provide in your modernization efforts. Check out zowe.org but create a policy to not only to consume but contribute. Get involved in the projects. It’s a way to expand your employees’ interest in the mainframe to make that open source there available. So if you haven’t done it, you really should do that. 

    Ashley: Access to more talent I think as Peter was mentioning earlier. It really is opening up new pathways as opposed to replatforming and a lot of not very attractive options sometimes that you now have because of open source. So folks definitely should consider this, look at it, get into it, get involved. Great. Well, gentlemen, it’s been great talking with you, both Tim Willging and Peter Fandel from Rocket Software. Take care, gentlemen, we will see you next time. 

    Willging: Thank you.

    Fandel: All right, thank you. Take care. Bye-bye. 

    Willging: Bye.

  • Managing Data Risk in 2021

    Managing Data Risk in 2021

    You can’t protect your data if you don’t know where it is stored. The first thing to consider when creating a risk-based approach to data protection is the ability to identify and prioritize data, and control who has access to it. But, how do we do this? How do we prioritize data?

    In this TechStrong TV episode, Ron Bennatan, senior vice president and general manager of data security at Imperva, joins Mitch Ashley to discuss privacy, compliance and managing data risk in data lake environments in 2021.

    The video is immediately below, followed by the transcript of the conversation. Enjoy!

    Transcript

    Mitch Ashley: I’m very happy to be joined by Ron Bennatan, who is GM of Imperva Data Security. We’ve talked before—Ron, it’s good to be speaking with you again.

    Ron Bennatan: Hey, Mitch. Nice to see you again.

    Ashley: As always, yes. Data and security—two of my favorite topics. [Laughter] Welcome to 2021.

    Bennatan: Yeah, 2020 finally is over. [Laughter]

    Ashley: Ooh! You know, it takes a pretty big rearview mirror to look at that, but I’m not sure I wanna look back too much. [Laughter] Well, we’re gonna be kinda talking about looking forward into this new year. First, would you introduce yourself, tell us a little bit about you, and also a little bit about Imperva?

    Bennatan: Yeah. So, Imperva is kind of the market lead for data and application security. We secure all paths to data. So, it kind of straddles between both covering how data is accessed and where data is accessed, kind of covering everything regardless of on prem and cloud, hybrid cloud.

    Ashley: Mm-hmm.

    Bennatan: And I am the GM for the data security side, and really glad to be here.

    Ashley: No small problem you’re solving. [Laughter] Data is everywhere, you know?

    Bennatan: One problem, yeah.

    Ashley: [Cross talk] data. At least not very many of ‘em. So, let’s talk about it. I mean, you know, we have the whole government breach in 2020 Russia—you know, Russia, Russia, Russia, that whole thing.

    Bennatan: Yeah.

    Ashley: It certainly brought a lot of attention to supply chain attacks. Of course, you know, that being an intelligence brief, it’s collecting a lot of data, right? You know, that could, of course, happen on a—it is, did happen on corporate networks, it could also be for financial gain; it doesn’t mean it has to be a national state attack. So, I imagine that’s gonna be a—supply chain is a word we’re gonna hear a lot about this year.

    Bennatan: Yeah, I think so. I think, you know, as a, just an individual, my fear, by the way, is—you know, there’s a really big project coming for the entire world, which is the delivery of the vaccines, and that itself is a supply chain. I just hope nothing messes up there, you know? Like, you know, knowing that whatever vaccine you’re getting is actually the valid one and not—you know, there’s a lot of…everything is in data and everything at the end sits in some kind of a, like, what’s the lineage of something? What’s the providence of something?

    Ashley: Mm-hmm.

    Bennatan: You know, it doesn’t necessarily—you know, we in the data governance area talk a lot about providence and proving the correctness of data, but you know, it goes all the way to the vaccine as well. So, I think this issue that we’re seeing, whether it’s breach—breaches get headlines, okay? And, you know, so this SolarWinds thing gets a big headline, the thing that these attacks on MySQL with ransomware, that gets headlines. But really, the bigger issue is getting control of our data and creating kind of a complete risk oriented approach, not just plugging a hole here and plugging a hole here and plugging a hole here. It’s—that’ll never work, right? It’s much easier to create a new hole than to plug all the holes.

    Ashley: The whack a mole strategy. “Here’s another mole to whack.” [Laughter]

    Bennatan: The whack a mole—yeah, the whack a mole doesn’t work. Whack a mole doesn’t work. And especially today when the complexity of the data environment is so much greater.

    Ashley: Mm-hmm.

    Bennatan: You know, 20 years ago—okay, 40 years ago, we just had some dashboard on the mainframe. Twenty years ago, we just had Oracle sitting on some Unix server. Today, it’s such a complex environment, data moves from place to place, you know, you look at what a data lake implies. It’s no longer a single stack, it’s a combination of stacks. Whack a mole doesn’t work.

    Ashley: Mm-hmm.

    Bennatan: So—and most companies understand that. Most companies are in this process of investing in a non-whack a mole approach, in a risk based approach that says, “Okay, I need to know where my data is. I need to prioritize which. You know, not all data is created equal. I need to scan, I need to understand, I need to know who has the ability to access something. I then need to know who is accessing something. I need to control what they’re accessing.”

    So, you know, I think there’s a better understanding that whack a mole doesn’t work.

    Ashley: Mm-hmm.

    Bennatan: I think 2021 is still gonna be a transition year. I don’t think we’re gonna necessarily get there, but we’re gonna transition.

    Ashley: We won’t have universal enlightenment [Laughter] on day—

    Bennatan: It just takes time. I mean, it takes time.

    Ashley: It does.

    Bennatan: There’s a lot of data.

    Ashley: I wanna ask you about—you did a nice job of kind of outlining, you know, identifying what data you have, prioritizing it, access to it. One of the things that I’ve always struggled with is that second step. How do you prioritize? What are the things to consider? You know, there’s obviously sorta crown jewels of my business—this is the customer data or secret sauce or that kind of things, but is there a good sort of stratification of how to prioritize data?

    Bennatan: Okay, there’s a technical answer and there’s a human answer.

    Ashley: Okay.

    Bennatan: Okay, technically, there are very good ways to do this. It takes some effort, okay? You can—you really do this based on a scoring of a bunch of dimensions, okay? So, one dimension could be the sensitivity level of the data, okay? Another dimension could be, what is the impact if something happens, okay? Which—but the reality is that we’re all people, and people are driven by incentive, and incentive is not necessarily, doesn’t always necessarily create the right prioritization. It creates whatever prioritization the incentive is structured around.

    So, I’ll give you an example. A lot of projects going on around privacy—a lot of projects. Because it’s a very, it’s a relatively new thing, less understood, and people don’t always equate the fact that we have been doing the work, almost the identical work, we just didn’t call it privacy, right? We called it classification.

    Ashley: Mm-hmm.

    Bennatan: But when you look at what privacy implies, it’s the same, but very new terminology—different semantics, different terminology. So many people don’t start from an investment they’ve already done in the last five years, they start from scratch, and the incentives are created because there’s a new role in the company. There may be a new—you know, a data privacy officer, okay, and that person has certain mandates and they drive things top down.

    And so, I think that the reality is, prioritization is driven by the projects and the funding that these projects have, and they’re also driven by breaches, okay? Because breaches are very dramatic.

    Ashley: [Cross talk] response, right? [Laughter]

    Bennatan: Dramatic, immediately created. So, I think that the reality is gonna end up somewhere kind of in the middle where things are gonna be driven bottom up and top down and they’re gonna meet somewhere.

    Ashley: Mm-hmm.

    Bennatan: But it’s not terribly complex to create a good prioritization strategy, but it’s not trivial, either.

    Ashley: Yeah, you definitely have to put some thought into it, and it’s not a textbook exercise, right? It’s all contextual.

    Bennatan: Yeah.

    Ashley: What’s important to the business.

    Bennatan: And I think you’ll find in—I mean, my prediction in 2021 is that there’s gonna be a lot of, you know, privacy is not gonna just drive a lot of these from a compliance perspective. Okay, until now, privacy is really—you know, when a company embarks on these privacy and classification projects, or at least what I see from our customers, is that it’s driven a lot by compliance, okay? They need to comply with a CCPA, they need to comply with GDPI.

    Ashley: Mm-hmm.

    Bennatan: I think we might end up having also, like, an attack flavor, okay, around privacy, which is not intuitive, not necessarily intuitive. And it has to do with the fact that, you know, part of the mandate is, you know—so, say that I am, you know, year one of my consumers and you store, I store some of your data. That means I—you have the right to demand some things from me. Like, you have the right to demand that I delete you from my system.

    Ashley: Mm-hmm.

    Bennatan: You have the right or portability, you have the right to ask me what data do I own or do I store about you? And you can come to me with these requests, okay? Different terms, different subject rights request or data access requests. You know, you can easily think about what is the effort that I, as a big company, have when you ask me to do this? It’s not a small effort. So, okay, if you—now, the nice thing is, most people don’t do that, okay? I’ve never called somebody up and told them, “Hey, delete me, okay?”

    Ashley: [Laughter] No, I think that’s probably very much an exception—maybe a rare exception.

    Bennatan: Yeah, but what happens is some people realize that this is the way to cripple a company.

    Ashley: Mm-hmm.

    Bennatan: What happens if they organize and they say, “Hey”—

    Ashley: Or protest, right?

    Bennatan: – protest, you know?

    Ashley: Moving of Facebook onto whatever.

    Bennatan: “We don’t like you, let’s organize 10,000 of our friends and we’ll all hit them at the same time.” What do you think that’s gonna do from a work perspective? It’s pretty serious.

    Ashley: Yeah, I’m guessing it’s not just press a button—that’s gonna consume some resource at the company, people resource, things like that.

    Bennatan: Yeah.

    Ashley: You know, one thing I was wondering, too, is, with this year, there’s discussion now about do we need a national cybersecurity strategy, things like that, just to mention politics, the Biden administration is talking about working with other countries to establish sort of norms of what we do when it comes to cyber whatever with each other and what is considered an attack.

    It seems like that is—maybe not this year, but that’s certainly potential for regulation of those kind of discussions, whether it comes out of any national strategy or not or could be another country that says, “Okay, now we’re gonna take this to the next level.” Like, what is our position around data? Not just privacy, but if it’s compromised, then what happens to you as a firm? I mean, it could get kind of interesting.

    Bennatan: Yeah, it could. You know, I mean—again, we’re delving into, like, opinions and politics.

    Ashley: [Laughter] Yeah, I know.

    Bennatan: I’m not a fan of—I’m not sure that something like this could go that route. But I do think that it goes other routes. So, for example, like, I’m reading a book on kind of the history of the Mossad, okay, the Israeli intelligence service.

    Ashley: Mm-hmm, mm-hmm.

    Bennatan: And you can see—and whenever you read any of these books, you see that all of the intelligence community, they have back channels, right? They agree on things.

    Ashley: Right.

    Bennatan: So, to me, if we wanna have something like that, it should be, like, the cyber czar of the U.S. and some other—you know, the big players, they have to agree. It doesn’t need to be regulated, it doesn’t need to be at the politician level, it has to kind of stay at the—

    Ashley: Kinda behind the scenes with the cyber czar.

    Bennatan: Behind the scenes with the experts at the—but, you know, there’s definitely an issue with state nations, okay? This does need to be addressed. Now, you know, it’s not necessarily that every—you know, we always think that, you know, we’re the good guys, they’re the bad guys, you know?

    Ashley: We would never do anything like what they did to us, right? [Laughter]

    Bennatan: Well, you know, there’s examples in history, okay, of things. So, I do think this is coming. I don’t believe it would happen at a politician level, it will happen at the actual subject matter experts. And by the way, there are good examples of things that have happened like that, especially in the financial services community. I mean, for years, maybe even decades, you know, FS-ISAC, they work together to create some level of standards across investment banks or banks. These things work, I just don’t think it’s political.

    Ashley: Interesting. Well, I wanted to kinda tap your thoughts on that. I know I threw the line pretty far out there in the water, [Laughter] just to kinda get your thoughts on that.

    Well, let’s reel that back in a little bit and talk—you know, there’s GDPR and CCAPE for California. A lot of people are still in this, I think in this kinda implementation stage, even though we’re supposed to have those things all covered, right? Maybe a little farther along with the GDPR, of course, but is that something, this year, you think it’s—we’re in this, okay, let’s now institutionalize some of those things in the organizations and really take care of our knitting there or do you think there’s more things on the horizon that are gonna come at data people?

    Bennatan: I think it’s gone into an implementation phase, for sure—which is good.

    Ashley: Mm-hmm.

    Bennatan: You know, there’s a lot of things, many things are not terribly clear, by the way, even though it’s in implementation. I’ll give you an example, right, this notion of being—my ability to request that you delete me, okay? But let’s take something else, which is a different type of compliance requirement, which is for you to keep logs for a certain period of time, activity logs. There are a whole bunch of other compliance requirements that require you to do that.

    Ashley: Data retention requirements, yeah.

    Bennatan: Yeah. Okay, so—now, what happens if part of the activity logs has my identity in it and I request you to delete it?

    Ashley: Mm-hmm.

    Bennatan: Should you delete it, or should you not delete it? One, in order to comply with one thing, you should delete it. In order to comply with another thing, you can’t delete it, maybe for three years. So, there’s even things like this, which are conflicting messages. And at the end of the day, it’s going to depend on some level of interpretation.

    Ashley: Mm-hmm.

    Bennatan: Which, at the implementation level, I can tell you, I can see two approaches. One is—okay, this interpretation needs to be given by legal or it’s okay for the business to give the interpretation to that. So—and you know how it is. If it goes to Legal, it takes a longer time; if the business can make the decision, then it goes much faster. And I’m seeing more and more of this moving to the business, taking ownership, and making the decisions, in which case, it goes much, much faster.

    Ashley: Well, and oftentimes, it seems that that always happens with laws and regulations, right? There’s what it is and there’s the interpretation of it, and there’s sorta those middle tier organizations, the consultants, the larger firms to sort of establish the frameworks that, “This is our interpretation of it,” so if you do that, you’re sort of in the norm with what the industry is doing. Who knows if that’s the right interpretation or not? It’s always subject to change, but that’s usually where that help comes from, at least on the big thing.

    Bennatan: Well, yeah. I mean, it starts from there and then usually what happens or what it evolves to, which makes it really go much faster, which is what’s happening now is that if—and by the way, it goes back to your question about this nations coming together or an industry coming together. The minute you have some critical mass within a certain peer group and you say, I don’t know, maybe I’m an insurance company, if my peers have taken upon themselves a certain interpretation, I usually don’t even care where they got it from.

    Ashley: Mm-hmm.

    Bennatan: Maybe they got it from these consultants, maybe they didn’t. But if three big players in my industry are doing the same thing, I’m gonna do the same thing.

    Ashley: Yeah.

    Bennatan: And that’s what’s happening now is that, you know, I know the insurance industry well and I know the banking industry well, and I can see that they’re now, “Oh, we’re gonna do like them” and then it’s much easier to follow, like, a track that seems to be proven than to try to invent it. And that’s happening and that’s why I think there’s an acceleration for a lot of these things.

    Ashley: Yeah, I agree with you. That makes a lot of sense. What are your other thoughts about 2021? What are we gonna be thinking about that we didn’t realize is gonna be on our radar for this year?

    Bennatan: I think we are going to kind of understand this notion of, you know, look at the data from every way that you can get at the data, not try to—like, historically, there have been a focus on pillars or silos, which were more and more, or in the past were driven really by tool specialization, okay? Like—okay, I’ve got the DLP and I’ve got this and I’ve got FAM and I’ve got databases, and I’ve got big data.

    And today, the lines are so blurred, especially on the cloud services, they’re even more blurred that there’s starting to be a realization that we really need to look at all paths to the data and all ways to secure the data and look at it instead of individual silos, looking at it as, “This is my data fabric.” Okay, so you’re starting to hear things like, you know, data lakes, data fabric, data services, and the data service could be structured by different implementation stacks. But you’re no longer trying to secure each stack by itself, you’re trying to secure the entire fabric.

    Ashley: Mm-hmm.

    Bennatan: And it’s really the right thing to do. It’s the right way to think about it. It is something that requires a transition, because even organizationally, many companies had one group doing this and one group doing this and one group doing this. And this notion of, you know, looking at a single overlay control layer, okay, which can drive policy down or can bring things up is, you know, it’s time for that. And I think 2021 is gonna be, part of this year is going to be starting to deliver things at that level of abstraction, and it’s also enabled by the fact that there is technology that can kind of bubble everything up into one level, right? You don’t need to look at it as 20 different tools, you need to look at it as kind of like a federated layer, and at least it’s doable, right? Ten years ago, it wasn’t doable to do any of it.

    Ashley: You know, it’s interesting. What it makes me think of is maybe some norms that we’ve had for a while of thinking about data owners, and that sort of puts the blinders on of, now I create that vertical view of, “This is that data for that purpose for that business unit for that application or process, whatever it might be.” And what you’re saying is looking at it more systemically, right? The entire collection of data, and securing it, but also uses of that data and sort additional intelligence and things you can mine from it.

    It’s interesting—so much, as we continue to move more and more into the digital economy, digital world for how we operate, all of those digital activities generate their own data, too. So, now, we’re adding that to the fabric of the data of, you know, we have a lot more usage information, we have a lot more telemetry that we never had. You know, it seems to be expanding exponentially. So, our ability to even just fathom what’s there and how it’s—what uses it might have and how it might be correlated in new ways sort of boggles my mind. [Laughter]

    Bennatan: Yeah, yeah. And, you know, I mean, it used to be this—one of the things people used to talk about when they started building big data platforms was that, you know, until recently, we just, we couldn’t use our data. We just threw 97% of our data away, and now we don’t wanna throw it away, but you’re right, it’s a beast that feeds itself.

    Ashley: Mm-hmm.

    Bennatan: But yeah, I think it’s, I think looking at the risk profile of the entire thing and not cutting into separate projects, which then have no tie to each other, right? I mean, if you want to reduce the risk, you asked about prioritization. Okay, prioritization needs to drive everything, right? Needs to drive reducing service area, needs to drive access control policies, needs to drive audit. If you start separating each one of them, not only is it not very effective, but it means you’re working 10 times harder, because you need to do this many, many, many times.

    So, this notion of a single kind of data security platform which kind of can have this internal feedback loop and use whatever effort you’re doing, it doesn’t matter if your effort is driven by compliance or security or privacy, whatever investment you make needs to build into this platform, not do itself over the next time you do anything. Otherwise, we’re just wasting our time. We’ll never get done. Because you said the data’s growing exponentially—if we’re chasing, if we’re not leveraging what we’re doing for one purpose when we wanna do the next purpose, we’re gonna lose. It’s gotta be a single platform.

    Ashley: Well, and to your point, it seems that once you sort of make that shift, right, to that platform view of data, now you can apply that to how you look at the data across all sources, all uses, all former kind of vertical siloed—you know, you may still want to look at it that way, but it’s not like, “Okay, now we’ve figured it out for compliance, but we’ve gotta go start over to do it for something else for operational process or whatever.”

    Bennatan: Exactly.

    Ashley: No, it’s there, you kinda take that—

    Bennatan: Exactly.

    Ashley: – those same tools and look at the data, analyze it. Now you’ve got that capability to do that for lots of things.

    Bennatan: Yeah, it’s exactly—it’s exactly right. I mean, you know, it’s like, in the business world, there was always this notion of a customer 360 where, you know, what’s the point that I have one system for quotes and one system for support and one system for my policy and I never know what’s going on, and I can’t upsell, and I can’t cross sell. It’s the same thing in security, okay?

    There’s this notion of data 360, data security 360 where, if it’s there, you know, then prioritization becomes easier, right? If you can check your configs and your vulnerabilities, okay, and you get this huge list—what are you gonna do? But if you know that, of this huge list, your consumer data is here and that, you know, the crown jewels are here—great, you don’t have to boil the ocean.

    Ashley: Mm-hmm.

    Bennatan: And if, at the same time, you can bring in and you can check your entitlements and you can say, “Well, you know, I have this vulnerability, and it is important data, but I did a really good job at narrowing it down and access to it is only by these individuals and access is Kerberized and—I don’t know, and maybe I also regulate the policy very tightly—these are all related, okay? So, if you have that single platform, you’re in much better shape than…you know, it’s like, I’m trying to think of a metaphor of, you know, when you have a single viewpoint, that viewpoint may be wrong. When you have—it’s kind of like, you know, these days, this discussion about whether news means anything any more, because it’s so tailored to what you wanna hear.

    Ashley: Mm-hmm.

    Bennatan: But it’s kind of like somebody forcing you to go and watch Fox and CNN and NPR and—you must watch all of these before you make a decision. It’s the same way. We can’t reduce risk by just looking at one thing at a time.

    Ashley: Yeah, we certainly have our own selection bias and data as we do our news programming as well, right? [Laughter]

    Bennatan: Yes, exactly.

    Ashley: Well, this has been a lot of fun talking with you. I just have a sense 2021 is already off to a good start with vaccines and the COVID situation. Hopefully—

    Bennatan: Hopefully.

    Ashley: – you know, we’ll round that hump and folks can really figure out new ways of collaborating with each other without worrying about COVID. Maybe that day’s down the path not too far, but we hope, anyway.

    Bennatan: Yes.

    Ashley: So, it’s been fascinating talking with you. So, we didn’t talk about, you know, go into product stuff, it wasn’t that conversation, but tell us a little bit about where folks can find out more about Imperva.

    Bennatan: So, Imperva.com, we are—you know, if we’re talking about data security, then there’s a whole section there, and you’ll see that we’re investing a lot in, you know, things that we believe 2021 and our customers are implementing. So, a lot of kind of complete security, complete security, whether it’s for compliance, security, privacy—a lot of effort in privacy, a lot of effort around data lakes, everything around cloud. Any cloud that you can think of or any data service, any data-centric workload on the cloud is something that we’ve worked very hard with the cloud vendors to provide support for. And yeah, I think 2021 is gonna be an exciting year because of this modernization that everybody’s embarking on.

    And my hope and what we’re trying to do with Imperva is that the path of modernization for the applications and for functionality also brings with it modernization of security, updated security. That’s really what we’re working on.

    Ashley: I would definitely suggest folks check you out. In thinking about our conversation before of the response to regulation and policies and things like that of looking at what other leaders do, Imperva works with some very impressive companies who are solving the data challenge from a platform standpoint. So, there’s a lot of good reasons to check out Imperva, because there’s a lot of problems you’re solving on the advance that could help a lot of organizations, so I hope people will do that.

    Bennatan: Yeah. Well, thanks, Mitch. Always great to talk.

    Ashley: You bet. Good to talk with you, Ron, and thanks, everybody, for listening in. It’s been a great conversation, and I wish all of you a great 2021 as well.

  • Shifting Left and Static Code Analysis with Perforce

    Shifting Left and Static Code Analysis with Perforce

    Perforce Product Manager Stuart Foster, and Evangelist Steve Howard, join Mitch Ashley to discuss the importance of creating security software from the beginning of the development process. We discuss shift left, SAST, source code scanning and other pursuits towards this goal.

    The video is immediately below, followed by the transcript of the conversation. Enjoy!

    Transcript

    Mitch Ashley: I’m pleased to be joined by Stuart Foster, who’s product manager with Perforce, and also Steve Howard, who is static code analysis evangelist, both with Perforce. Welcome, gentlemen.

    Stuart Foster: Hey, Mitch, how are you doing?

    Ashley: Very good. How about if we—let’s start off with just a little bit of an overview of Perforce, and then, Steve, why don’t you do that and also tell us a little bit about yourself, and then we’ll toss it over to you, Stuart, to give you introduction.

    Steve Howard: Right, yeah. Well, here at Perforce, really, our role as we see it is to help developers write better code more efficiently and more quickly. So, these days, we tend to talk about DevOps and the whole idea of developer operations and how they put that code together, how it’s constructed, how it’s managed, how we control the processes around it as well, of course. There’s more and more regulation, it seems, these days in software development and we’re making that whole process from start to finish, of writing code to delivering it, that whole cycle. It’s hopefully being made easier, more comfortable for our developers that use the products.

    So, that’s really where Perforce sits, and we think of that is really enterprise DevOps, and doing it at scale in large scale organizations.

    So, my role, really, is specific on the Static Code side or Static Code security testing side, SaaS tools, if you will, trying to make sure that that piece of writing the code, actually developers writing the code itself is done to the highest possible standard from both the security and the quality perspective and also, at the same time, getting maximum productivity from our development teams.

    Ashley: Wonderful. Stuart, introduce yourself?

    Foster: Yeah, absolutely. I’m Stuart Foster. I’m the Product Manager or Perforce’s Static Analysis products, both Klocwork and Helix QAC. So, I work with Steve, he’s kinda my partner in crime, amongst a number of other sales engineers and developers and everything, and we’re just out to produce a really solid static application security testing tool.

    Ashley: Excellent. Well, one of the great things about talking with you both is, since you have products in market, you talk to a lot of customers, especially in an evangelist role, too, Steve. It really helps us kinda get an idea of where the state of security is in applications, what people are doing today. The whole idea of shift left, I think, has evolved a great deal. I think when I first heard it was “move left in the workflow.” And then it became, “Well, move the when the security people get involved.” And you’ve really taken it much farther than that to really thinking about how we start writing secure code.

    Maybe one of you could kinda give us an idea of where you think we are as a community on getting to that point of when the code’s written, it’s written securely.

    Foster: Yeah, so, I mean, from my perspective, you know, you kind of look historically at people using the term shift left and, you know, from our perspective, that would’ve been, you know, doing some sort of code analysis on a server and then moving that left in terms of the software development life cycle into the developer’s IDEs.

    What you see nowadays is that, you know, historically, a lot of those responsibilities would’ve been siloed. You’d have a security team, a QA team, various other teams that, in many ways, because they’re siloed and in different stages of the process, you’d run into kind of bottlenecks. What you’re seeing a lot more with the concept of DevOps, DevSecOps, a lot of development teams themselves are becoming owners completely of the code that they write. And what I mean by owners is that, you know, they are writing unit tests, they’re writing more functional tests, they are ensuring that it’s part of their responsibility that the code is secure. 

    So, that’s a shift left, but then you have to think how developers work—they’re working in IDEs, they have their own CI/CD pipelines, they may be using cloud services, various containers that are all getting spun up and broken down. So, it’s not just, to me, shift left any more, it’s analysis anywhere. It’s basically having something that can work anywhere in a workflow, right? And in this sense, you know, the Security teams still exist, the Quality Assurance teams still exist, but what it really is, is a culture of security collaboration that are coming together, right?

    Steve, I think you may have a little bit of a perspective on this, too.

    Ashley: Yeah, I’m interested in your from the field perspective, too.

    Howard: [Laughter] Yeah, no, it’s—well, it’s exactly that. I think it’s a really good point that no, we’re not just talking about moving to the desktops, we are talking shifting left throughout the whole cycle. I think that is a key point. And it is really an opportunity using tools like ours in Static Analysis security testing, it’s about being able to enforce the requirements and the needs from that expert security team where there were bottlenecks previously, trying to put that into rules and regulations that we can then push down to the development tasks and processes that are to the left of that kinda final sign-off.

    So, yeah, that—in that form, very much, I’d say it’s shift left, and it’s about that optimized process now where we can be doing that checking all of the time. So, it’s making sure we’re continuously compliant to those security requirements or quality requirements, as the case may be, yeah.

    Ashley: You know, it’s interesting, we talk about the workflow and we think about the whole tool chain and pipeline, but if you really break it down into much more granular pieces, developers are working on a linear piece of development, right? They’re in multiple places jumping back and forth, solving bugs, adding features, developing new capability—whatever it might be. So, the ability to concentrate on any one aspect of an application, sometimes, is not even possible, right?

    So, you need an environment where you’re aided in how you write and create secure code as well as think about writing secure code. You agree with that?

    Howard: Yeah, absolutely. I think that’s the point. We’re trying to make sure that all of those pieces that they’re working on at different times and, you know, our colleagues in the Version Control System team know this very well, but obviously, there are tens or hundreds of branches running in parallel, as you’re sort of saying, and developers are jumping between, you know, fixing problems on the current release, maybe, and then carrying back on with their feature work for the next release as well. 

    And all this time to try and manage what security requirements each one of those had or quality requirements as well would be a really nightmare without sort of tooling and regulation around that, around the whole process, which is a significant part of developer operations as a whole, the idea of automating a lot of that and building it into the system. So, I think it’s just enforcing that with making sure we’re adding all of the extra pieces that we can also include in those processes.

    Foster: And also, if you think about those developers, too, they’re experts in their fields or experts on some of the functionality that they’re developing, but they’re not also necessarily experts as a security team might be, right? So, part of the trick, too, is to offer kinda help, knowledge, learning via the tool set so that, you know, the developers can keep the velocity up and develop to those standards more easily. 

    And I think Steve mentioned a little bit of a buzzword that kinda we like to think about is, it’s continuous compliance, right? It’s having this throughout the process and enforcing it and continuously monitoring, remediating, fixing, and reporting that become important for these processes to run really cleanly.

    Ashley: You know, it’s interesting, there was—kinda stepping back in time, and again, I think about the myth of developers don’t wanna, don’t care about writing secure code. Well, of course, they do. They don’t wanna write insecure, vulnerable code. But I think there’s a period where we thought training was—you know, let’s teach them about how to avoid writing code with vulnerabilities in it.

    But when you talk about a continuous governance environment, you’re kinda talking about a whole layer of things, right? It’s from the person doing the work to the environment that it’s being checked in and integrated and being tested into the workflow, and then really, the data, the information about that, that’s happening on a continuous, ongoing basis. And that’s really your data set for producing the compliance information, right? We don’t have to go ask you or check your code, it’s already been done, it’s done every step of the way all the time, and here’s the evidence of it, and here’s what we learned from it and here’s where we’re doing well and here’s maybe where we can improve. 

    That’s how I think of continuous compliance. Yes, it shows up in a report down the road, but it’s continuous, because we have all that data.

    Howard: Yeah, I think that’s a really important point from the efficiency as well. I mean, the Security team or the Development Management that want to know when this product or project will shift obviously need to know where they are relative to that compliance requirement. And if we can keep it compliant from day one, we don’t ever have that risk of building up technical debt at the end of the process that we then have to solve, and I think that’s a really important evolution that we’ve seen over the last few years, that we have that knowledge to hand immediately at our fingertips, as you say. So, that’s a really important piece.

    And the other point you made there about the developer education, I think, again—yes, you can teach developers and I know myself, in the past, I’ve learned the right way to do things, wrong way to do things, but there’s nothing quite as good as actually showing the examples of when things are potentially risky within the code you are writing, because you really understand, at that point, what you’re writing and how this could then be a problem, and it is the classic on the job experience that you really want for a perfect education, almost.

    Ashley: Let’s take—I’d love to hear your kinda definition or how you describe Static Code Analysis SaaS to people about, if you’re embracing that, what does that mean to a development team, to a software project team?

    Howard: I think from my perspective, it’s the idea of having an automated code reviewer, almost, looking at the work you’re doing in a fairly pleasant and passive way. [Laughter] Not being critical in any way, shape, or form as well. So, from a cultural point of view, I think the developers perhaps accept it more when it’s from a machine rather than a colleague that perhaps is picking holes in the code they’re writing, which is another good point about having a computer do that initial check.

    But yes, it’s like having that guard at the end of the process, and then we often joked historically about the value of the tools and saying, “Well, you know, if you had two developers that were both going for the job, you know, you’re advertising for a new development role, and one of those developers is, you know, the cost of the license is more, but they never make a mistake in the code, they check in clean, compliant code with every commit, whereas the other developer, they both have the same experience, they’re both writing good code, but the other one could make mistakes and could check in, potentially, security vulnerabilities. Which one would you pick as a manager?” That’s quite an interesting way of looking at it, but of course, you would always want to have that assurance that, essentially, it’s clean code from day one.

    Foster: Yeah, and also, you know, in terms of the perspective, Steve talked a little bit about the developer perspective or the hiring perspective. From the business perspective, you have to think about the costs of reputation, lives, revenue, anything of a product being in the field and discovering an issue. You know, the costs to fix a bug exponentially gets more expensive as you go further down the software life cycle, right? So, it’s cheapest to fix it then and there, right at the start. And, you know, in terms of kind of pain points that the software is trying to fix is, you know, providing that knowledge, providing that quick remediation as developers are working, ensuring the velocity stays up, ensuring that the continuous compliance enforcement throughout the process means that there’s not a heavy lifting of work later in the process, because you know, you weren’t reaching compliance requirements. 

    You know, the ease of using the software however a developer wants to work, it’s all about velocity, you know? And in terms of the business having that oversight and understand, you know, what are our risks, and how soon are we gonna be able to ship this product? Because, you know, we know with many products, the faster to market you get, you know, you have that first mover advantage, and that’s what’s needed in this fast-paced software market.

    Ashley: Well, that’s very tough to recovery from reputation damage and quality issues or perceived quality issues around your products, too.

    You know, one of the ways that I explain it to folks outside of being a developer or within the security world is, it’s kinda like taking your spell check of a document and adding in the grammar checking—so, this is the higher order, more complex, specific things around grammar. In this case, it’s around security and helping you as you’re developing or creating code, right, make sure that you do it the best way, you know? Avoid introducing security issues.

    I know that’s a gross oversimplification, so don’t worry, I’m not saying Perforce is like a grammar checker. [Laughter] But that is sort of a non-technical way of describing what we’re doing. I mean, we want it to be that natural, as part of the process.

    Howard: Yeah, absolutely. And I think that’s very true. I think, you know, we’ve seen the evolution of Static Analysis tools very much in the same way we’ve seen the evolution of things like the Microsoft Word, you know, spell checker to grammar checker and so on, and it’s kind of evolved to be more and more all of the time, you know, of the systems similarly.

    But on top of that, there’s also the opportunity, then, to, what we find quite interesting is to write your own checkers and rules that are specific to your application. So, we can find in some places where there have been security problems in a certain application for some special reason—not necessarily related to the constructs of the code, as most of the coding standards check, but something that’s specific to the project because of the way the project has designed all the code, and that project interacts with outside external interfaces. 

    And it has been interesting to work with customers in some cases. I worked with very large telecom customer on one occasion and he said, “Well, you know, we have this translation module between sort of the application and the platform, and nothing should ever, you know, go direct between the two unless they’re going through this one piece in the middle.” And it’s kind of the golden rule we’ve taught to all the developers from day one and nobody will ever break this rule.

    So, you know, obviously, I was slightly challenged by this and said, “Well, it’d be nice to check the rule. Let’s put that enforcement in just to see if we can, you know, use a custom rule to see if there are any cases and, of course, as you say, it should be clean.” And obviously, you know, a couple of hours later, once the analysis is run, it’s a very big system, we found a couple of cases—only a couple, but there were a couple of cases where exactly what they said should never, ever, ever happen for the security of the entire system, it happened. 

    And so, we were able to then remediate those problems and keep that rule in for future development, and of course, it will always be enforced. So, that was a really interesting case where there was also that, where you have something specific, it’s not just generic grammar issues, if you will, or spelling issues, but something very specific to that document.

    Ashley: Mm-hmm.

    Foster: Yeah, you can imagine some of these things coming out of requirements or an internal coding standard or things that a Development team might be following and be able to enforce these. Especially when you start looking at legacy code, giant monoliths, you know, that have been around for 5, 10, 15, 20 years, decades, you know, and you don’t have those original people who are developing, right?

    So, you know, it’s—we sometimes think of our tool also as, you know, kind of like a quality assurance on a product or a code base that, you know, it’s legacy and all the new people have no idea how to use it, but you need to expand on it, you need to develop it, because it’s valuable IP, and how do you manage developing new functionality and moving forward when, you know, it could be as brittle as a glass house, right?

    So, you know, that’s the other benefit you get out of being able to enforce and write your own standards, I’ll call them, and rules, yeah.

    Howard: Another classic case there, Mitch, is—and you’re well aware of this as well from the field is, this system was never intended to be connected to the Internet, you know? This was something that—

    Ashley: [Laughter] You’re guaranteed it will be, then.

    Howard: Yeah. [Laughter] It was an embedded system that was entirely supposed to run in an embedded environment with no connection to an outside world at all, and all of a sudden, you know, it or something around it that’s connecting to it is now connecting to the outside world and there’s open attack interfaces, and now we have to make sure that they’re not gonna compromise the original legacy code base.

    Ashley: Mm-hmm. Yeah, there’s just sort of the rule of unintended uses always occur, right? The thing we don’t want to happen is usually what happens. [Laughter] So, yeah, the demo becomes the production system is another great example.

    I’m really interested in your perspective on where you think we are. If I made an analogy to the network security world, the best time to be a network equipment or network security salesperson is the day of or the day after a compromise, and automatically, you’ve got budget and we need to solve this problem. I mean, I think we’re well past that in most cases, but where are we in the software world? Do you see this as a, it’s just a natural thing, “Okay, what’s our SaaS strategy, how are we gonna be doing security within the application, what tools should we use for that?” Or is it something where you have to mature to a certain point or have experienced some issues before you’re really gonna take it seriously? 

    Where are development—and I’m asking for a generalization I know, but what’s your sense of where people are today?

    Foster: I’d say, you know, in general, it seems as though most people don’t take these kinds of things seriously until it bites them. I think, you know, there’s some kind of safety feelings of some types of products that you can use where, you know, you find out about a security vulnerability or intrusion and you’re using a piece of software that basically just tells you that, you know, the barn doors have been opened and all your horses are gone, right? But that’s too late in the process. I think we all need to understand that, right?

    So, the point of dealing with it sooner and using tools like security application and security testing tools like Klocwork or QAC is that, you know, you’re trying to build the risk reduction, the known unknowns that you can at least deal with sooner, right? So that, you find out later, in the worst case, that the doors were open, but, you know, nothing happened, you still have your horses, because the stables didn’t open at the same time or whatever, right? Just to make a kind of analogy to it, there.

    Howard: Yeah, I think in the—obviously, we do a lot of work in the embedded space, and in that area, having seen it over the last 10, 15 years, I think, you know, there was a time where we were educating people a lot more about what it was, you know, what it was that we could do with Static Analysis to actually find some of these problems. Whereas I think—and there’s a lot more awareness now of the risks in the security field and, obviously, what Static Analysis can do to help. So, most people are starting to think about it if they’ve not already got it built in.

    So, I’d say it is becoming more of an expected insurance policy against those kind of things within the tool environments. And obviously, the other kind of driving force there is the regulation. So, we’re starting to see standards themselves creeping into the market like the new ISO 21434 standard for automotive, for example, where they’re actually going to require some level of looking at the security of the system and making sure that we’ve done some of those things correctly and, you know, probably that will include what we expect it to include, some level of Static Analysis and coding standards and so on and so forth.

    So, that stuff is really becoming almost regulated into the process just like functional safety regulation has been in that process from the beginning.

    Ashley: Definitely.

    Foster: Yeah, you’d imagine—

    Ashley: Sorry, go ahead.

    Foster: I was gonna say, just on that topic there about cyber security for automotive, you think about it, the cars are effectively rolling entertainment systems, right? So, there’s a lot of new kind of vectors for attack, right? An insecure Bluetooth module inside a car and its infotainment system that is also connected to the bus controller that’s driving all the data from every sensor on the car that’s controlling your fly by wire driving and your fly by wire steering and all those things, right? It’s very important—very important.

    Ashley: And usually, you have, I believe, you know, both cars and trucks that have starters can start by Bluetooth as one of the mechanisms. So, yeah, you have potentially the same attack vector, you need to be able to isolate that, and now you’re into the function of the engine versus the entertainment system, and there’s a lot of work going in, I know, to the automotive systems. They’re very complex. I mean, just as complex as many, you know, many business applications—maybe even more so because they’re integrated from so many different suppliers, and they are, of course, largely based on, built on embedded systems.

    It seems like embedded systems, though, are much more accepting of an update process, of keeping the software current and requiring some way of being able to update software in an embedded system or maybe making it an automatic process so it’s kind of touchless. Is that your experience as well?

    Howard: Yeah, I think so. I think that is coming into many systems now. I mean, there are certain manufacturers driving that kind of process—automotive, for example, is a prime case. And it is interesting, because if you go back 10 years, that just was never even considered. You know, once you kind of flashed that system, it was staying as it was until you climbed up the mobile phone master or whatever it was that the embedded system was doing and flashed it again with the latest version. So, yeah, it is, the over the air updates is creeping in, and that will lead us to a more continuous delivery type architecture with the delivery of the software as well, that we’ll be able to go to that.

    So, yeah, it’s gonna be an interesting time. And as part of that, I guess now the focus has moved slightly from the cost of fixing an issue out in the wild to just, we don’t want to have the issue because of the potential impacts on having that issue even for a day, you know? [Laughter] The idea that a car might be possible to take control of from an external source, for example.

    Foster: Yeah, and I think you’re seeing a lot more emphasis on the security around a lot of these embedded systems because a lot of them are so, you know, requirements driven, compliance driven. They have ISO requirements that are needed to be validated by external bodies to make sure that these things are safe to use, because you know, a lot of our infrastructure is built on the backs of these things, that they’re, in some ways, leading the charge of seriousness for these reasons, right?

    Ashley: Mm-hmm. And it seems, also, that industries like, I mean, we were talking about the automotive industry, are putting together their own standards, I think in part of fear out of, “We don’t want government standards to dictate to us, we’d rather us solve the problem, you know, industry solve it” in a way that they’re more accepting and flexible to them. But do you see more regulation coming at a kinda geographical country, government level? Is more of that happening? You talk about ISO standards, of course, being more industry.

    Foster: Yeah, I’d say so [Cross talk]—

    Ashley: You see it happening? Like, our privacy standards, now we’re talking about software security standards?

    Howard: [Laughter] Well, yes, there is that as well. Perhaps in the European level, for example, of something we see on this side, but yeah, mostly, it’s industry standards, from what I can tell that we’re experiencing and seeing come downstream at the moment.

    Ashley: Mm-hmm. [Cross talk]

    Foster: Yeah, I think it depends on the level of kind of sensitivity of the data. So, you do see requirements as they relate to finance and, like, PCI, DSS, and things like that. You see security in terms of—I’m running a blank, here, but they typically come from when the data is so sensitive, you know, the U.S. government has their STIGs, DISA STIG rules.

    Ashley: Mm-hmm.

    Foster: So, it tends to come down from the level of sensitivity. I wouldn’t be surprised if we do start seeing more requirements, perhaps, for public data, but right now, we have things like the HIPAA compliance and things like that, as they relate to very, very sensitive data.

    Ashley: Yeah, the defense industry is also a good one to start to drive standards, too, and I know they’re doing a lot in security.

    So, just to change, shift focus here, we have a few minutes left. We were talking earlier about where people are in their maturity level of adopting SaaS and writing more secure software using those kinds of tools. If you were someone leading a project, maybe an architect or a lead developer or somewhere in the management chain that said, “You know what, we’re almost there, I just need to do a little bit more convincing, or I need something else to help my organization get there to see why we need to do this, and I’d like to do it before we have a major incident”—what’s your best suggestion for kinda doing that? I don’t know if it’s internally selling or if it’s demonstrating the value of it or how do you see people successfully getting the organization to begin to support adopting it?

    Howard: Hmm. I think that’s a tough question, just because there’s so many industries and so many different kind of topics of particular interest, it always helps to have experience of a well-publicized kind of issue [Laughter] that’s gone out into the—

    Ashley: “We don’t wanna be like those people.”

    Howard: Exactly, yeah. That always helps drive things a little bit when you’ve got something tangible to say, “We don’t want to end up there.” I think, again, the regulation piece is where I see a lot of that being driven. You know, the industry bodies are actually saying, “As a whole, the industry doesn’t want to end up there. That’s not somewhere we can afford to be, and so let’s try and make sure we get this right from day one” and that trickles down into individual projects and so on. [Dogs barking]

    Ashley: Well, speaking of dogs [Laughter]—okay, hold on.

    Howard: Okay. Yeah, so, that’s really where we’re—

    Ashley: Okay, just start your answer again.

    Howard: Yeah, so, that’s really where we’re seeing it, predominantly, from the—you know, first of all, we don’t want a case where there is an incident, because, you know, if we can  use that as a tangible case and say, “We don’t wanna be like those guys, because that didn’t end well for them.” So, that’s, obviously, always a good driving force. But then it’s regulation from the perspective of the industry bodies themselves getting together and saying, “We really can’t afford for this kind of thing to happen, and we want to make sure that, you know, across the industry, we start to follow a best practice that’s gonna do our very best to prevent that,” I guess, is a good way of describing it.

    Ashley: It seems this is also an area where security team CISOs can be a real colleague asset for making the case to the rest of the organization, because if you’re _____ on the software team, you have one set of justifications or definitions for why you need that. Another is, there are corporate—you know, the CISO is trying to accomplish these goals, these things with the organization. And it’s always easier to fit your objectives within a larger objective and get that supported and get funding—that kinda thing. So, that’s another suggestion I would throw out there, too, that also helps folks—and plus, it brings security and software teams closer together.

    Howard: Yeah. [Cross talk] Sorry.

    Foster: Yeah. I think we do see—we do see, we do have, you know, security evangelists that are, as you just said, who are championing it, right? And when we talk to those people and we go over Static Analysis with them and this is the security focus, right? It comes down to, you know, being able to help understand their size of team, what kind of current development practices they have, and what that means and how you can deploy, roll out the tools and, you know, not slow down that velocity, because that’s always the goal. Don’t slow down the velocity, but add more value. 

    So, that’s what we try to help them with, help them understand. And also, you know, help—sometimes, you know, Steve is sitting with these people and we almost help them mature in their software development process to, you know, move from a couple guys who are just doing nightlies and they have no version control, they’re not using quality gates, they have no type of automated pipelines to setting them up so that they can scale their development groups, they can speed up their testing, their development processes, and that all comes out of engaging with vendors like us.

    Ashley: Great. Well, gentlemen, this was a lot of fun. I really enjoy talking security and creating secure applications and, you know, it’s something that I think everybody wants to improve at to different levels and degrees, and hopefully, before those bad things happen. It’s always better to do it before, but if you have to do it after, you do. Maybe sometimes that’s what it takes.

    So, it’s been great talking with you. Again, this is Stuart Foster, who is Project Manager with Perforce—thank you, Stuart—and Steve Howard, Static Code Analysis Evangelist with Perforce as well. Take care, gentlemen.

    Foster: Great, thank you.

    Howard: Thanks, Mitch.

  • Change Management in a Fast-Paced DevOps World

    Change Management in a Fast-Paced DevOps World

    While DevOps speeds up software release cycles, change management often slows them down. So, how do you do change management in a fast-paced DevOps environment? Change control enables organizations to meet compliance and governance while managing changes at the speed of business.

    In this interview for TechStrong TV, Greg Sanker, former CIO, IT Operations specialist, author, speaker and contributor to the “ITL4 Change Enablement Guide,” joins us to share his views on the tremendous impact of Agile, DevOps and CI/CD on traditional change control teams. Greg talks about the new role of change control and his contributions to the “ITL4 Change Enablement Guide.”

    The video is immediately below, followed by the transcript of the conversation. Enjoy!

    Transcript: Change Management in DevOps

    Mitch Ashley: Well, I have the pleasure of being joined today by Greg Sanker who is former CIO, operations specialist, he’s a speaker/author, a bit of a futurist in leadership and digital transformation. He’s also a contributor to the “ITIL4 Change Enablement Guide” and a reviewer of a lot of other content—so, long introduction, there. Welcome, Greg, to [Cross talk].

    Greg Sanker: Yeah, wow—thank you. That’s kind of awesome. Thanks for having me. I appreciate being on the show. I think we’re gonna have—

    Ashley: It’s always—always good to have a fellow musician. Anybody with a Telecaster in their background can be on this any time you want.

    Sanker: Yeah.

    Ashley: So, tell us, [Laughter] in addition to your wonderful guitar sitting behind you, tell us a little bit about yourself.

    Sanker: Well, you mentioned, I am—I specialize in IT Operations. That’s what I’ve done for more decades than I’d maybe like to admit. But service management has been a core part of that, and it’s become really clear to me that my view of service management is a lot more streamlined and agile than a lot of people perceive it as. And so, I’ve begun speaking at conferences and whatnot. I have a book of my own that’s published—I’ll let you guys go find it if you’re interested—about change management.

    But most recently, I was asked to work on the ITIL project on a number of ways, but most notably, the change, what we now call “Enablement Guide.” And one of my prerequisites to get engaged with the project was, if you’re gonna ask me to write it, I’m gonna write it the way I see it, which is very different than what we’ve been doing in the past, and we’re gonna be pushing it forward, so. So, that’s a bit about me.

    Ashley: That’s wonderful. That’s definitely a consistent theme with ITIL4 and other folks, Mark Smalley and others that we’ve talked with is really recognizing—I think ITIL’s had its sort of process moniker, right?

    Sanker: Yeah, pretty much.

    Ashley: But there’s a huge emphasis on, you know, how do you fit ITIL and all of the aspects of operations into something that is more of a dynamic DevOps-y agile kind of world, if that’s a way to describe it. And not to say that it didn’t fit in before, but I think it’s making it more prescriptive in that kind of setting is what a lot of the good work that you and others have done.

    Sanker: You know, I was an early embracer of DevOps, and part of the reason for it is the mindset that explores possibility that ignores the boundaries in an organization to achieve outcomes. That’s, like, music to my ears—I’m all in. So, it’s a match made in heaven, but I can mention, it’s like, there’s a lot of organizations that have adopted service management, they have used ITIL in ways that are very prescriptive, very heavy weighted, very bureaucratic and slowed down. And it was never intended to be that way.

    Ashley: Mm-hmm.

    Sanker: It just is, it got adopted that way, right?

    Ashley: Well, and I think a lot of the work that’s gone into ITIL4 will help make some of that shift.

    Sanker: Yeah.

    Ashley: Certainly, it’s positioned that way, and I know from other authors, that was the mindset of how the contributions were made. Now, you specialize around change management and we were chatting when we first got introduced about, you know, how do you do change management in a rapid development, agile, DevOps kinda world? If you were gonna be ask that at a conference by somebody, how would you—what guidance would you give them?

    Sanker: Wow. So, I’ve got an hour and a half platform, right, for that?

    Ashley: [Laughter] At least an hour and a half compressed into 20 minutes, yes.

    Sanker: Twenty minutes, right? Yeah, absolutely. Well, see—well, let me put it this way. I think it really comes down to one fundamental paradigm that we need to change is, traditional change management, which is what I call your CAB-based, you know, Razor request for change, blah blah blah, is based on the idea that we inspect every single change. And as you probably know, Edwards Deming’s said many, many decades ago, “We’ve got to cease reliance on mass inspection to get quality products.” Well, that’s exactly what we’re doing when we say, “Hey, I need to make a change. Well, let’s just take a look at it,” you know, so on Thursday, we’re gonna get together and look at that. But you know full well that, in a DevOps environment or a continuous integration, I mean, we are moving all the time. We don’t slow down and stop.

    And so, that mindset, just the mindset of stop and let’s inspect is completely incompatible with a flow-based, whether it’s agile or DevOps or CI/CD or any of the others that are out there that are more flow based—you know, rapid feedback loops, short cycles, et cetera—it’s just incompatible.

    And so, what we have to do is elevate out of the inspect mindset into the expectation mindset. And that is—what is it that this organization needs from change management? And some of it’s governance and compliance, some of it’s operational stability and recoverability and some of those kinds of things. But let’s state what that is, we’ll state what those things are for the organization so that those that are working in the pipelines, the value streams can actually optimize how well we achieve those expectations and don’t dictate how they do it, let’s just dictate what the outcomes that are required by the organization. That’s the fundamental shift, and it’s big. I mean, it’s more than a 180-degree shift. It’s very significant.

    Ashley: Definitely a different mindset for sure, you know, the inspection versus not. It’s not unfamiliar, though—that’s what security is doing about shifting left and moving ______.

    Sanker: Yeah.

    Ashley: Iterating the process, DataOps is also—you know, everyone wants to move early in the process, but it’s more than that, because you’re talking about a pretty different mind shift change. So, talk about compliance. Talk about governance from a change management standpoint in this new paradigm.

    Sanker: Yeah. So, you jumped right to the jugular. That is the million dollar question.

    Ashley: [Laughter] No, but it needs to be done and, you know, we all sort of pooh-pooh it.

    Sanker: Right.

    Ashley: But you know what? If you don’t do it, it’s really not fun doing it at the end, right?

    Sanker: So, maybe you remember, because it certainly rattled my world—Dr. Forsgren said on stage at DevOps Enterprise Summit 2018, “CAB is useless.” And everybody in the audience laughed and cheered and duh duh duh duh duh.

    Ashley: You’re not talking about Cabernet, you mean change advisory board.

    Sanker: No. [Laughter] I bet you probably would enjoy a good Cabernet, and I’d—

    Ashley: CABs, yeah. Those others are still good, but anyway.

    Sanker: Merlot, et cetera, they’re all good. But the thing is, she’s not wrong. But she was looking at it from a very specific lens, and that lens was about failed changes—in other words, the success of changes and the recoverability of changes, two things that are really important to the stability of the business. And what she found in the data, she’s a data expert, she has a Ph.D. in this, so there’s no mistake, here.

    She’s absolutely right—however, CABs frequently are part of the compliance equation. In other words, it’s useless for those things, but it has a use elsewhere in the organization, and for many organizations, that’s a very scary thought. Because that’s where we’re able to document and demonstrate for internal and external auditors and validators—are we doing the things that we’re required to do? PCI DSS, HIPAA, GDPR—you know, all of those other compliance framework, NIST 800-53, which hopefully we’ll get to in a minute, but they all have requirements around managing changes within your organization. So, if we don’t do it in the CAB, how do we do it? And that’s precisely the point.

    So, what I’ve sought to do is help organizations build those compliance points into the pipeline itself. In other words, we can point directly to where that’s being done in the pipeline and don’t require a “let’s slow down and actually check the box, here.”

    You know, the governance is—well, can I just say this? There’s a difference between governance and compliance. They’re two very different topics, right?

    Ashley: Yeah.

    Sanker: So, governance is about how organizations make decisions, and making sure that those decisions that are made throughout the organization align with or help fulfill the purpose of that organization, whereas compliance really has to do with the conformance. Are we conforming to applicable standards or expectations, either that the organization has or we have externally?

    So, but when we talk about governance and compliance, we think about performance and conformance. I’m sure you’ve heard that—COBIT and lots of other frameworks have talked about that for years. But performance is about, you know, your strategy and are you utilizing your resources in pursuit, or are you producing value or value creation? And I think that’s fascinating.

    I’ll bet you’ve read or heard of Mark Schwartz’ new book, “The Art of Business Value.” It’s a fascinating read.

    Ashley: I actually haven’t read that one. I’ll have to check it out.

    Sanker: I would put it on your reading list, and I know that sometimes he rubs people the wrong way. I found out that he rubs me exactly the right way, because he asks some really challenging questions, like, “What is business value?” You’d think that’s an easy question to answer, right? It’s like, you know, for a for-profit corporation, it’s like, you know, making your market windows, making your profitability, adding features that attract and retain customers, those kinds of things.

    But what if you’re in—as I was as a CIO—what if you’re in a government agency? That’s not a for-profit organization, so what is business value? Delivery of services to citizens, upholding environmental standards? There’s a lot of things that governments do, but they’re not profit motive. In fact, in some cases, it’s not even financial, so what is business value?

    And so, when we’re running our backlog, you know, we’re scoring our user stories, like, how do we score that in that kinda context? And then, just when he, when I thought that he had confused me enough, he asked, “Well, what if your business strategy”—and I don’t wanna give away his book. Go buy his book. But—

    Ashley: No, just tell me the last chapter, that’s all I need.

    Sanker: Exactly. [Laughter] I actually sent him an e-mail. He’s a great guy and he thinks very, very big. But one of the things that he said was, “Well, what if your strategy is, we want to move customers off of this platform and onto one that’s more profitable to us, or one that’s in alignment with our roadmap and our strategy?”

    So, if we’re down in the value streams or in the pipelines, scoring our user stories against customer satisfaction, customer value, things that customers really, really demand and want, we’re actually creating negative value—these are my words, not his. We’re creating negative value because we’re making our customers like the platform that we want to get them off of.

    Ashley: Mm-hmm.

    Sanker: So, business value would actually be—no, do things that make them less likely to like this product. Make them want to migrate. Make it easy for them to say, “You know, this is really getting old. I’m gonna try your news service.” And so—

    Ashley: Right, that behavior change.

    Sanker: Yeah, exactly. And so, when we really start thinking about, we’re getting, you know, your governance, especially your governance and your compliance, we have to have adaptive, rapidly adaptive systems that allow us to, at any point in time, understand what the current strategy is and how it affects the decision making that we’re doing at any point in time—all without, of course, getting your board of directors involved with every single decision throughout the organization.

    Ashley: Well, let me ask you this. I’m really interested, I’m really curious about how you take this kinda shift in mindset and how you’ve woven that into the “Change Enablement Guide“ that you were the author of. So, you know, it isn’t a process discussion.

    Sanker: No.

    Ashley: You’re talking about viewing things a different way, valuing some things maybe we didn’t value before, maybe some other things less, and maybe delivering a different set of things. Let me just share, briefly. I remember a day, probably not too long ago, where CABs saw their role as saving us from developers and all the bad decisions and lousy code they were gonna create, so that’s what—

    Sanker: [Laughter] Oh, yeah! Absolutely.

    Ashley: – that’s what they say in those CAB meetings.

    Sanker: Yes. Oh, absolutely.

    Ashley: But, you know, that’s not what we’re talking about. Tell me how—so, how do you build that into, in a way that’s gonna help people make that shift through ITIL4?

    Sanker: So, thank you for saying that, because there is some unspoken or lightly spoken or spoken only in factions kind of frustration, there. It’s like—let’s be honest. I used to be a network engineer, I did network security, I did all things network. So, I used to be a change manager at one point in time, and who the heck are you, Greg, to cast judgment on this piece of code and decide whether that’s gonna work appropriately? Everybody knows that I can’t add value to that, and everybody knows that this external body who’s not familiar with it, certainly not at the pace—you can’t add value, and yet you act like you do. And that just—that doesn’t sit well with anybody.

    So, we really need to, if we shift our focus to, “What is it that we’re trying to achieve?”—so, I talk about putting the right measures on the pipeline. How well is this pipeline producing outcomes, those outcome expectations? Is it producing consistently high quality changes? Is it consistently not delivering failed changes—or, for the failed changes that do go through, how quickly and easily can we recover from that with minimizing impact to the customers?

    Those are the things that your CAB or whatever your body is should care about, not the individual changes. You’ve got to get away from the individual changes. You can’t add value to that.

    And so, I get asked a lot of the times, it’s like, “Well, what do we do with our CAB?” since, you know, they’re very unpopular. And in some cases, the best answer is—now I’m gonna say it publicly, so I’m gonna be quoted on this, but—“Blow it up. Just get rid of it. And don’t ever use the C-A-B word again, because it has such a bad name.”

    But it isn’t without value. So, if you have a CAB and the CAB starts saying, “Look, developers, we can’t add value to what you’re doing, but what we really care about is some of these things. So, let’s talk about those things.” And if you’re talking to, particularly DevOps, agile, automated pipeline folks—they were gonna welcome that conversation because they’ve already got it covered. They just do. You know, like you know, I’m the change manager—“Well, have you tested that code before you released it?” Well, dude, that’s the only way it gets in production, if it passed tests. There’s no way for it to make it into production. So, not only is it insulting, but it’s a dumb question, and undermines your credibility.

    Ashley: Well, let me ask you this, then. I think where you’re going—and tell me if this is right—is, “We have so much automation in this pipeline process, right? That’s the whole idea of being able to produce software more reliably, more rapidly.”

    Sanker: Oh, yeah.

    Ashley: There is so much data that’s generated and captured, but not leveraged.

    Sanker: Yep.

    Ashley: We treat it as data exhaust, right? That’s sort of this fancy term for it.

    Sanker: Yep.

    Ashley: In that is a lot of compliance, is a lot of governance information—

    Sanker: Yeah.

    Ashley: – that you can then elevate and connect, and basically save developers time and not having to produce reports at the end. “I’ve already got three-fourths of what I need or what you need, and let me help you put the finishing touches on it so you’ll sail through the next whatever NIST whatever it is.”

    Sanker: Yeah.  So, there, you get to write a chapter in my next book, because that’s exactly, exactly what we want to focus on is, we want to be—you know, alright, this is potentially controversial, but change management is a feedback loop. It helps us understand the quality of changes that are coming through our pipeline. Sitting there and passing judgment and saying, “Nothing passes until I say so”—that’s not adding value. That’s absolutely antithetical—you know this—to the DevOps continuous flow kinda mindset.

    Ashley: So, they’ll go every way around you you can think possible, you know?

    Sanker: Yeah, and the other thing, as we well know, is like, develop is most frequently used in the flagship products. “This is the core business. Get out of our way. We have to deliver these changes to maintain our profitability, maintain our market windows—all of those things that are core to the business, and you think I’m gonna stop for three days when I went from, you know, a user story to a value in under 24 hours in some cases? There is no way I’m gonna stop. I’m just not going to. I will take the risk of it failing and you saying, ‘You shouldn’t have done that.’” And that’s—

    Ashley: You can talk to the chief product officer about—

    Sanker: Exactly.

    Ashley: – the stock’s direction. [Laughter]

    Sanker: And I helped an organization who was, you know, they had agile coaches and they were doing all that. It’s a large company, but I’d rather not name them. And I was asked to come in and look at their change management.

    Well, what they had done was, rather than having one CAB a week, we were having daily CABs. And we were having daily pre-CABs and then the CABs. And one of the things that I tried to get them to understand is, these two are entirely incompatible. If you’re going towards agile DevOps as a development methodology, then you must move away from CAB. CAB has got to go.

    And what was interesting is, their flagship products, the ones that really made money for the organization, they could get away with murder. Why? Because it was so critical to massive revenue in the case of this organization.

    Ashley: Mm-hmm.

    Sanker: And so, we have a big outage, we recover, we move on, that’s okay, it’s an acceptable risk. And change management will still know, we can’t have any defects or whatever in production.

    And so, we’ve got to get ourselves wrapped around the idea of managing business risk. You know, you mentioned something about the compliance, and just this last week—like I said, I’m working on a new book, but I was looking at the NIST 800-53. Everybody’s familiar with it, but they finally released version 5. And so, I’m reading through and the author’s made a very fascinating statement, and I encourage everybody to go look at this. Version 5 is phenomenal, because what they said was, “We have eliminated or reduced the specification of who actually does the activity, and we’ve instead focused on the outcomes that need to be achieved.” In other words, some of the compliance requirements or the controls said, “IT will review” or, “The change manager will review.” And so, if you’re gonna comply with NIST 800-53 previous versions, there’s not a lot of wiggle room around that.

    But I’ve been saying for over 10 years and others have been saying, it’s like, if we focus on the outcomes, then who’s doing it, how they’re doing it is open for us to do it in the most effective way. And that gets us into the pipeline space. And it’s a major step forward, because we struggle with that. Like you said, we have to meet our compliance, you know? If you’re dealing with sensitive information, you know, pharmacy has HIPAA and credit cards and all that sort of stuff, you have to. It’s just not optional. But you don’t have to do it the way we’ve always done it, and that’s the piece that I like to bring to organizations. It’s like—how are you doing it, and can we demonstrate that we’re doing it? Those are basic audit business controls minimums.

    Ashley: I’m curious about, we’re sort of talking also about the whole issue of agility and speed, and on the other end of it of the belief of the best way for something to be stable is don’t change it.

    Sanker: Oh, yeah, absolutely.

    Ashley: Which is antithetical to the environment we’re in, for sure. And so, it’s kinda like you have to embrace, you just have to say, “We love change, and how do we be super effective at it?” instead of, “How do we not change so we don’t mess it up?”

    Sanker: So, there’s two places, two things that I’m gonna mention briefly, but we won’t have time to explore them. There’s the field of study around complex, you know, and Cynefin is one of the most notable frameworks. And the idea that there’s simplex, complex—or complicated, complex, and chaotic, and we treat changes frequently like they’re either simple or complex. Meaning, we need to get some eyeballs on this thing.

    But the fact of the matter is, most systems, especially systems that we’re DevOps-ing, are more like a complex adaptive system. There’s a level of complexity that we’ve not seen in days of old, like back in the ‘80s and ‘90s and early 2000s where these systems were, we could actually try to understand them. Modern system—

    Ashley: They’re not linear. They’re more, you know, dynamic and self-creating, if you will, in run time.

    Sanker: Yeah, exactly. So, that hit me a number of years back. It’s like, this whole mindset, it’s around the idea of, it either is simple or it should be simple, and we can understand it, and we can predict the outcomes, and you can’t. You cannot do that.

    And so, we need to build adaptive systems that say, “Hey, look, this is complex.” And you’re not gonna send it to a board of examiners, you know, a CAB who are gonna say, “Let’s take a look at all the risk,” because it would take you years to do that in a complex system. That’s just—that’s not gonna happen.

    So, we need to adopt practices that are more adaptive. And one of the things in that complex area within the Cynefin framework says that we need to run some experiments to better understand the mechanism. And how that applies to change management—see, this hit me between the eyes because it was like, “No failures in production. No failures will be allowed”—that’s antithetical to learning how the system works. If you don’t make some changes to see how the thing works in operation, you’re denying yourself learning and you’re probably deluding yourself that you’re actually actively controlling it, right?

    The other piece of study that I’ve brought to the table is—and you’ve probably heard of it—this study of anti-fragile. There’s a number of authors whose understanding is astronomically higher than my IQ, you know, Ph.D.s who actively debate such things. But I take complex things and make ‘em simple. Here’s the thing—some things, like a tree, when the wind is blowing, the tree is waving, but there’s more than that going on. It’s the actually fibers, the plant itself becomes more resilient to wind if it’s been exposed to wind.

    Here in Oregon, often there’s logging operations that have a lot of controls for the environment—trust me on that. It’s changed a lot since the good, ol’ days. But one of the things that we do hear is, we leave a segment of trees at the edge of a piece of property that’s being logged, and we do that for the animals, for the environment, we protect the riparian areas. But one thing that you’ll notice is, within one year of leaving that stand of trees that used to be next to a large strand of trees—they break. A storm comes, and they get hit. Why? Because they’ve relied on the strength of their neighbors up to this point, and they’re not suited to stand the wind alone, and so those trees break.

    Here’s the thing—if we don’t inject some failure into the system, for a system that is truly anti-fragile, we’re actually making our systems more fragile. And so, a mindset of change management that says, “No errors, no problems, no failures ever, ever, ever”—you are making your system more fragile, and you’re treating a complex system like it’s simple. And so, we’ve got to change our mindset.

    Now, you know full well—and let me just blast into this—you know full well that, as we take these mammoth changes that we used to make in the old days, you know, it took 6, 8, 12 months to produce, and we’ve turned it into these microscopic things that can go through your pipeline and ours and produce values is your blast radius. What bad thing could happen in production has materially changed. The risk has become very, very small. If that—

    Ashley: And your ability to fix it—

    Sanker: And your ability to fix it skyrockets.

    Ashley: – your responsiveness, it isn’t six months to fix it or three, it’s three minutes.

    Sanker: Yeah. So, one of the change outcome expectations that I encourage people to establish is, what are your expectations for recovery from any given change, and can we realistically do that?

    So, for instance, if I put something in my flagship product that doesn’t work as desired, if I can recover within two minutes, no harm, no foul, is that okay? In a lot of cases, it is. If it’s a defibrillator keeping somebody alive—probably not, that’s probably not okay. But in a financial transaction or a video streaming service or whatever, you know, modern digital services, it’s probably okay, especially if you’ve engineered the release in such a way that you do it like that. You know, blue/red switches, on/off—all of that kind of stuff that we’re engineering in become, you start seeing why we’re able to do that and what that actually does in production. It’s wonderful.

    Sorry, you got me excited about that.

    Ashley: Yeah, boy, I could take you a lot of places with that. [Laughter]

    Sanker: Yeah.

    Ashley: So, let’s take it back to someone who’s gonna pick up a digital version or a printed version of the “Change Enablement Guide” author.

    Sanker: Yeah.

    Ashley: How is that approachable for folks? Let me be honest. I’m probably not gonna go download that to get my heart pumping to go work out, get a better workout—or maybe I will.

    Sanker: [Laughter]

    Ashley: Maybe after listening to this, I will—but how do you make that engaging for people?

    Sanker: Yeah. Well, so, one of the things—well, I’ll put it this way. It’s like, we’re not selling a product, we’re selling an opportunity to solve some problems. That’s why we call it best practice, right? And there’s a lot of best practices, and I don’t feel like they’re nearly at odds as maybe some of the industry might feel like.

    But change enablement is—and I’ve been saying for years that this day has come and it’s now here is, change management is in the crosshairs of modern digital organizations. It’s a pesky remnant of days past. And what the “Change Enablement Guide” is—and there are was more. I mean, for production reasons, I had to kind of limit it. But what I wanted to do was point in this direction of what I’m talking about. It’s like—let’s get people moving towards that. Because like you said, we’re not gonna change overnight. I’m not gonna read the “Change Enablement Guide” and go back to work the next day and say, “Alright, I can save all the world’s problems.”

    And so, what I’ve subsequently started doing is writing guidance around, “But what do I do today? You know, I went to a DevOps conference, I’m excited, but what do I do when I go back to work? How do I start moving in that direction?” And blowing up the CAB, while we might all celebrate that, that’s not a good first step. Instead, what we need to do is start moving ourselves in a better direction. It’s not more of same, it’s—what do we actually need to be doing here, and what’s the best way to accomplish that?

    And it’s difficult, culturally, because it becomes more of a culture. Like you said, there are CABs galore who despise developers probably as much as developers despise CAB. It’s a match made in—

    Ashley: Cats and dogs, living together.

    Sanker: – yeah, exactly. [Laughter] Good reference. And so, that group of people who are widely known for having that kind of mindset are not gonna be welcome when they show up to your spring and say, “Hey, we’d really like to be involved earlier in your change management so that we can help you.” It’s like—yeah, sorry, bud. Let’s start somewhere else.

    So, if change management starts with, “We’ve gotta get out of your way. We’re gonna start delegating more and more of your changes into your value stream, and instead, we’re gonna be focusing on, we’re gonna partner with you on, let’s look at the results of what you’re doing. Because we believe that you are optimizing your delivery and we believe that you have the right tools and controls in place.” So, we are moving more towards, “Are we achieving all the outcomes that the organization has? Is the organization even clear on what those outcomes really, really are?”

    You know, the one that always—I love this one—“All changes must be tested before going into production.” It’s like—well, how many shops have you been in where that’s not true, at least in modern times?

    Ashley: Mm-hmm, yeah. I call that the “meh” policy. “We’re gonna test it? Meh.” No, of course we’re gonna test it.

    Sanker: Yeah, we call it blow testing, right? You know, it’s like, “Eh, okay, I think we’re alright.” That is true. In modern terms, though? For flagship products, for things that are really, really core to the business, most organizations have some form of automated testing, or that’s just part of the process. So, rather than make everybody say, “Did ya test it?” at a CAB, just, like, “Hey”—

    Ashley: “Of course you did.”

    Sanker: Yeah, exactly. Is that happening in the value stream? If it is, I’m not—don’t worry about it. We’ll just move on. Is somebody with the appropriate authority approving that? And that’s—I am gonna take a minute to talk about this one, Mitch, is that everybody talks about change approval in terms of who’s gonna approve this idea going into production, this change going into production? Because if something goes bad, we wanna know who to blame, right? In fact, we used to say, “Who has enough—who is sufficiently authority that they’ll get fired if something goes bad?” Which is, of course, the wrong question.

    Ashley: Yeah, [Cross talk].

    Sanker: Because the right question is, “Who has enough authority that, if it goes bad, they don’t get fired, because they have the authority to put on that level of business risk?” Right?

    But if we move—and we could talk for hours, but the idea of product ownership or taking a product view of services or modern systems is, “Isn’t it true that the product owner is accountable to the organization to decide what things we’re going to move into production?” So, to me, that’s the change approval. We’re approving that we want this change to happen.

    So, nobody’s adding value by making approval at the end of the chain. We’ve already decided that. We already know that we’re gonna make this change.

    So, those are two different approvals. One of them is about a product decision or a governance decision, the other is about a business risk decision of, “Is it actually going to achieve the results that we set out to do?” I believe that most of the latter can be automated. If it’s passed its test, if the testing systems are appropriate, if we’ve done—in some cases, I love peer reviews. It’s like, you know, the code needs to be reviewed by a peer, stamp it, and we ship it out. It happens in a minute or two, you know, not hours or days or whatnot like that.

    So, I think as we kind of retool or thinking around it, I think that’s where we can start really solving actual problems that organizations have.

    I am reminded, though, I saw this—you’ve seen these before, these overhead views of an Indy car or a Formula One car coming in for a pit stop and all this stuff happens and off they go. I saw one this week that was 1.8 seconds. It was awesome, right? And people like thus always share those on social media of, like, “That’s an agile organization, doing the things” and blah blah blah.

    And so, you can’t really argue with 1.8 seconds—four tires, top off the gas, and out you go.

    Ashley: Impressive.

    Sanker: But the question, to me, is—like, I’m a bit irreverent. Why isn’t it zero? Why can’t we make it zero? If faster is better, let’s make it zero. And the answer is, we won’t make it to the end of the race unless we have fuel. We won’t make it, because [Laughter] when you’re racing at 200 miles an hour around corners, your tires literally don’t last that long. It’s not a matter of preference, it’s a matter of life and death, right? So, we have to change tires.

    So, the question isn’t, “Are we balancing speed and agility against risk mitigation?” It’s what it is that we’re trying to accomplish here. And in the case of Formula One, we’re trying to win the race. And to do that, we have to take on certain activities that cause delay—that’s a requirement.

    So, the question isn’t, “Do we have to take a pit stop?” We do. We need tires and we need fuel and sometimes minor adjustments. So, the question then becomes, “How quickly can we achieve those outcomes?” And putting four tires on in under two seconds—that is phenomenal. Hats off to those folks that are doing that. And how do you get however many gallons of fuel we put in a Formula One car in two seconds, I don’t know, but that’s amazing, right?

    And I feel like that’s the mindset that we need to adopt. We need to quit asking, “Why do we have to do change management?” We have to do change management because part of it is to be compliant. That keeps us in business. That keeps us out of trouble.

    Ashley: But it’s a really good example, because it’s not an absolute, it’s not, “Do we need to fuel?” but, “What’s just enough fuel for us to win?” right?

    Sanker: Exactly. Oh, and trust me, you’ve watched it. I know, I can see it in your eye, you watch Formula One or Indy and there’s like a thousand computers over there and stopwatches and gauges and dials and they are calculating exactly how much fuel based on the game strategy or the race strategy, the conditions, who they’re up against, what they think their strategy is—that’s how much fuel we need to take on, right? So, it’s—

    Ashley: Great model for us to think about of, you know, it’s not absolute quality or absolute whatever.

    Sanker: Right.

    Ashley: It’s what’s appropriate for the organization. We’re gonna have to leave it at there.

    Sanker: Okay.

    Ashley: And I want you to come back some time, because the next topic is Stevie Ray Vaughan or Eric Clapton. But I’m gonna—

    Sanker: Oh!

    Ashley: You can’t answer it. You have to come back and answer that, so we’ll have you here another time. It’s been a real pleasure talking to you, Greg Sanker, who is the author of the “Change Enablement Guide” of ITIL4. So, I think you really gave us some compelling reasons to go check it out. So, thanks a lot—it was great talking with you.

    Sanker: Thanks for having me, Mitch.

    Ashley: You bet.

  • Mainframe and DevOps in Traditional Enterprise, With ASG Technologies

    Mainframe and DevOps in Traditional Enterprise, With ASG Technologies

    The assumption that large, established enterprises—from insurance companies to government agencies—can’t adopt Agile processes or DevOps is based on the falsehood that legacy technology stacks won’t allow for it; that existing traditional mainframe applications or legacy applications that large enterprises are built on are incapable of adapting to these approaches.

    Accelerated Strategies Group recently released the new research report, “5 Crucial DevOps Strategies For Cloud and Mainframe 2020,” sponsored by ASG Technologies. In this episode of DevOps Chats, Jeff Cherrington and Anna Murray with ASG Technologies joins Mitch Ashley, CEO of Accelerated Strategies Group, to explore how mainframe app teams embrace DevOps, its processes and tools to equip software teams so mainframe apps continue to provide businesses with vital systems and capabilities far into the future.

    The audio file of the conversation is below, followed by a transcript. Enjoy!

    Transcript

    Mitch Ashley: Hi. I’m joined by some special people, some folks that I really enjoy working with from ASG Technologies—different from the ASG that I work with. [Laughter] So, I’m joined today by Jeff Cherrington and Anna Murray. Thank you, both, for being here.

    Jeff Cherrington: Well, thank you very much, Mitch. I’m Jeff Cherrington, I’m Vice President of Product Management for Systems with ASG Technologies. Many of you may know ASG. We’ve been in enterprise infrastructure management software for over 40 years, and certainly hold a prominent position as well in enterprise content management. We’re very keen to have the conversation today around how traditional infrastructure—meaning the mainframe—aligns with new DevOps initiatives in the enterprise.

    Anna, if you would introduce yourself.

    Anna Murray: Sure. Hi, my name is Anna Murray, and I’m a Senior Product Manager with ASG Technologies, and I work with Jeff Cherrington. And I have extensive experience with automation across platforms and data centers, as well as agile transformation for teams.

    Ashley: Excellent. Jeff was saying before we started that he used to work in radio—you can tell. [Laughter]

    Murray: [Laughter]

    Cherrington: [Laughter]

    Ashley: You’re very comfortable behind a microphone—no doubt, no doubt. We were talking about everybody’s on video these days, so that’s awesome.

    Murray: Right. [Laughter]

    Ashley: Well, great. You know, we’ve been doing some work together and we do have a report coming out to kind of pre-announce that. We’ll be talking about that a little bit later. But there’s a lot of myth about DevOps and mainframes. I know even some analysts—I won’t name any names—have said they’re two different paths, you shouldn’t try to do both. I think the jury has started to change their mind on that.

    Why do you think some of those myths have existed of, “No, we can’t do DevOps on mainframe environments?” Do you wanna start, Jeff?

    Cherrington: I think I will, and then I’m sure Anna’s got some thoughts that she’ll want to contribute. You know, it’s certainly a reality within the industry that we’re going through a generational shift around our leadership or IT, whether it’s infrastructure or application development. There are those who are still working daily, like myself, who represent the traditions of IT from the very beginnings back in the 1960s, and then there are others who are certainly much younger and certainly have much different experiences, which may not include any exposure to the mainframe. And, as a consequence, there might be some resistance, there might be a little bit of anxiety about the technology, and it can be formidable to approach if you don’t have the appropriate background.

    The things that we’d love to talk to, though, is that the mainframe as traditional technology still performs incredibly important roles within the enterprise. And at the end of the day, DevOps is much more about technique than technology, and it can apply as easily to mainframe and should as it does to any other part of the IT infrastructure.

    Anna, do you have some thoughts to build on that?

    Murray: Sure. You know, as I think about the generation gap that you’re talking about, I kind of come on the other end, but I’m Gen X. And so, we were the generation that said, “Oh, the mainframe’s dying,” right? [Laughter] So, that—obviously, I’ve learned that’s totally wrong, and the generation coming after us is coming in to learn the mainframe and realize how powerful it is, right?

    And so, I think a lot of the leaders that are coming in are coming from this generation that didn’t even think about mainframe for a while, right? And so, they did all their planning and their agile transformation and their DevOps transformations thinking about the cloud and all of their Windows and Linux and Unix servers and didn’t even think about the mainframe.

    And so, I think that’s the other side of the gap that happens. As the younger leaders are coming up, you know, especially, I would say blame my generation the most, right? [Laughter] Because there’s this gap, and—learn more and make sure you’re talking to both ends of the spectrum and learn about crossing the gap.

    Cherrington: And actually, the way I like to phrase it now is far less to use jargon like “generational gap,” but instead talk about generational bridge, because that’s what I’m seeing.

    Murray: That’s so much better, yeah.

    Cherrington: We’re seeing that the younger IT professionals are identifying that there’s great opportunity to work with mainframe in the coming years. And so, we see Millennials, or is it Gen Y—the generation after yours, Anna—who are stepping forward and saying, “This is really interesting. The mainframe does all of the technologies, it supports all of the technologies that I learned in uni, university. I can do Java, I can do TCP/IP connections, I’ve got great sources for strong cryptography, it supports Unix, I can put Linux onto a mainframe LPAR—this is a good space to be in.”

    And so, more and more, we’re seeing that the people we engage with are a mix of people that are my age with the white beard or the gray beard and a mix of young, fresh faces that are really eager and really excited to get involved, and who come predisposed to new techniques such as DevOps.

    Murray: Mm-hmm.

    Ashley: You know, there’s so much important things that both of you said. One is that in DevOps, there is no technology definition of what it applies to, and it’s very easy to associate it with Cloud Native and maybe newer applications, because that’s where it sort of grew up initially.

    But DevOps is really about how we create software, not what technology we use to create software. And it’s about how teams collaborate, it’s about how you share information, it’s about how you get different functions from the business together in a collaborative team to kinda do things earlier in the process, do automation. All of that applies to the mainframe, of course, and there’s tools to do much of what we’ve talked about. Maybe you can say a little bit about how people approach DevOps in the mainframe environment.

    You wanna go, Anna?

    Murray: Right? Well, you know, when you approach DevOps in the mainframe environment, it’s not so much just about the mainframe environment, but the company. The whole organization has to be behind DevOps, and there has to be leadership saying, “Hey, this is something we need across silos,” right? And then you have to look at the tools available to you. DevOps is a lot about the toolchain itself. There’s hundreds of tools in that toolchain, and so, you have to start looking for solutions that are gonna meet your needs and help you cross those silos.

    Cherrington: You know, it’s certainly true that everybody resists change. I think that’s an innate part of human nature, but both Anna and I have seen that things of this nature can come forward, can be used with teams that are mainframe-centric, perhaps have been doing things the way they’ve been doing them for 10 years or 20 years or 30 years.

    And there’s natural resistance. “Gosh, you know, I’ve done things the way I’ve done them successfully for so long, why should I change?” Certainly, I’ve seen agile adopted by mainframe development teams, sometimes kicking and screaming, but once the resistance curve is overcome, it becomes very productive and becomes very natural.

    And I think we’re at that stage now in terms of adoption of DevOps for the entire enterprise infrastructure, from Windows all the way to the mainframe and onto the cloud and even onto mobile, even as there are initial reactions of going, you know, “How can this apply? This is not something I learned when I went to my classes to learn more about the mainframe at SHARE in 1995 or 1985.” But we have proof points.

    We have large international enterprises, we have small regional enterprises that are coming forward and saying, “I must have mainframe. It plays a critical role for things that I need to do, particularly around payments processing, anything that needs highly available, real-time online absolute reliable performance. And it is only part of my infrastructure, and I need to treat how I develop there the same way I treat developing anywhere else.”

    Ashley: You were talking about, also—so many apps are not just a mobile app, right? There’s a whole back end to it, oftentimes sharing data, even talking transactions between multiple systems, one of those or multiple may be mainframe applications. You know, and if I step back for a minute and think about our current environment and how much things have accelerated in doing sort of these digital transformation, digital experiences for customers, organizations are looking to both strategically move quicker, but also be able to react opportunistically and sometimes based on conditions. And it seems like some of the flexibility with DevOps makes it easier to coordinate some of those activities across multiple applications. So, if you do have to do three different release to make up one new capability, that comes together much easier.

    Cherrington: It does. You know, the thing that I always look to as I engage with anything around business is, to use an old phrase, “why is the juice worth the squeeze?” And while we talk about DevOps a lot because of its elegance, because of its collaboration, because of the ways in which it makes development easier, the real juice of the squeeze is time to market. The whole reason to consider DevOps is how can we get something we need to get in terms of new technology enablement to market quickly, more quickly than we did before, but not sacrifice quality.

    And that’s what we see time and time again is, we have something we need to get to market. I want to present new capabilities to my customers through an app on the phone because that’s where people live their digital lives. But at the back end, it’s dependent upon the mainframe for things such as account balances, customer statuses—an unending list of capabilities that still remain on the platform as the best place for that type of activity to occur.

    Ashley: Mm-hmm. You know, I’m thinking about just the recent experiences with the reactions to COVID and contactless delivery, inventory, product inventories are often something that’s still on the mainframe, has been for a long time, and now, you know, retail stores, as well as a lot of others, have to be able to maintain a really accurate enough inventory and also present that to the customers so they know, what store can I go at to pick up whatever I’m looking at buying.

    I’m curious, too—I know that ASG Technologies has some of your own products, your own technology around this idea of orchestration across platforms because you work both mainframe and non-mainframe environments. Talk to us a little bit about how you approached coming up with that product and maybe a little bit about how it works and what it does.

    Murray: Sure, I’ll jump in for a little bit here. You know, you talk about orchestration—that’s combining your toolchain, right? You’ve got all the little pieces of your DevOps toolchain, and you need to figure out how to connect all the dots and bring it together and, you know, ASG has been doing automation and orchestration for a long time, but we’ve changed our focus into bringing in the development part of that toolchain, right?

    Operations has been kind of a focus of automation for a long time, and now we have to pull in all of the development pieces that happen and the configuration management that happens, and all of those—there’s lots of little tools there, right?

    So, we haven’t tried to come at the market as trying to provide all the tools, but we’re trying to provide some of them, right, and we definitely want to orchestrate as many as possible, right, and cover the entire environment for the customer. On the mainframe, the most popular thing we’ve been working with recently is JCL Management. Every mainframe’s running JCL, right? Nothing happens on the mainframe without it, and so making sure that there’s a DevOps interface to your JCL Management solution is really important. And so, we’ve made an investment in making sure that ours has both an Eclipse plug-in that makes it easier for the developers to interface with that JCL as well as a RESTful web service API so that it can be integrated with building of the products and crossing across the silos to other products.

    Ashley: Mm-hmm.

    Cherrington: And certainly, in the approach that we’ve taken, we look to be a good citizen of the DevOps enterprise community, and the work that Anna and her team have done, we ensure that we support the most popular integrated development environments for doing this sort of work. So, the JCL quality assurance and management offerings that we have for the mainframe are also certified for the Compuware Topaz integrated development environment and for the IBM—what’s the name of it again, Anna? It’s—

    Murray: IDZ.

    Cherrington: IDZ. What was the rational development environment that’s now rebranded?

    Murray: Yes. [Laughter]

    Cherrington: As well as any other Eclipse compatible IDE that a customer may want to use.

    Ashley: You know, it’s interesting, just thinking about all the, a number of the terms that you both used in just those last two comments, you know, mainframe 10 years ago, you wouldn’t have been talking about, I don’t think, RESTful IDE—all the tools that we kinda think of as sort of in cloud applications, much of it is the same toolset, a lot of the same ideas, just working with integrating some other different tools that are specific to the mainframe environment.

    Cherrington: It is, and you know, I certainly want to encourage those who are listening and watching—and particularly, those who may be ASG customers now or are considering becoming ASG customers—to join us at our worldwide customer conference, a virtual conference this year, EVOLVE, that will be occurring on October 6th and 7th, because we will be making some significant announcements about extending support in this area, including opportunities for creating interfaces to mainframe applications that can extend all the way to mobile, if that’s of use to a customer.

    We’ve had a lot of excitement coming into this, we have a lot of partners and customers who are going to be presenting, and there’s a great deal of energy that we’ll be putting forth around our DevOps support and DevOps enablement.

    Murray: And can folks sign up? How do they register to attend the virtual conference? Is that on your website?

    Cherrington: It’s on our website if you come to www.asg.com, I’m sure you’ll find something right on the home page that says EVOLVE 2020 and it’ll give you the opportunities to get, well, involved.

    Ashley: [Laughter] EVOLVE and involved at the same time.

    Cherrington: Exactly. [Laughter]

    Murray: [Laughter]

    Ashley: You know, the fact that you get to work with so many organizations that are adopting DevOps, I’m kinda curious about two things. How does the mainframe teams approach wanting to learn about DevOps and coming up to speed and trying it, and then how do the existing practitioners maybe outside of the mainframe group that are also using DevOps—how do they see what’s happening in the organization and how do they…do they want to participate, do they support it, are they kinda not sure? What do you see happening, there?

    Murray: So, yeah, you know, there’s a mixed feeling. You know, there’s definitely differences between, you know, if it’s led from the top down and people are being told they have to change, right, it depends on how you’ve adjusted the expectations in your team, right? Helping them learn what’s coming and why instead of just telling people what to do makes a huge difference, right?

    Ashley: Mm-hmm.

    Murray: We’re all much more interested in solving problems and learning new things together. So, it’s really important conversation is held properly, right?

    So, we have some that are really excited that they want to learn and they want to jump in from the mainframe side and grow, and then there’s others that are really just struggling and pulling their mainframe teams along, right? So, one customer I think of, you know, is standardized on Compuware Topaz Workbench, but the developers are less interested in switching over. They have some of the older generation who’s very comfortable with the green screen and they think, “Why bother switching to Topaz?” You know? They just don’t even get it.

    Ashley: Mm-hmm.

    Murray: And so, you know, the truth is, we really need to have solutions that meet the needs of all of the people that are part of the DevOps environment, right? So, it isn’t—you know, a very experienced mainframe programmer doesn’t have to use Eclipse, right? They can be quite happy on the green screen and we should let them stay there, right? [Laughter] And we do, our JCL Management solutions have a very robust green screen interface, and then we bridged the gap, as Jeff was talking about earlier, with having those APIs, right?

    And so—yes, so the DevOps toolchain is still brought together with something like our ASG Enterprise Orchestrator pulling in the code and grabbing with the API the JCL and making sure that the standards are applied and then bringing them across for compiles and distribution into another environment, right? Not that you compile JCL, but there’s other parts that are related.

    Ashley: Mm-hmm.

    Murray: So, you know, you asked—it’s kind of a mixed bag. And we see that, too, right? And as the conversation becomes more common, right, I think we’ll have more people who are more comfortable having that conversation and learning together.

    Ashley: Excellent. Well, Jeff has been bitten by the Zoom bug, so he had to call in from his call dropping, so glad you’re still with us, Jeff.

    Cherrington: Mm-hmm.

    Ashley: You know, we were just talking about the adoption process, and Anna, you were talking about how mainframe teams kind of approach this. I’d like the other side of it. DevOps teams that maybe aren’t in the mainframe group that have been practicing this for a while, maybe a few months, a few years—how do they see this happening, and is there collaboration? Do they pitch in, do they tend to work together or not? How does it go?

    Cherrington: Well, some of the things I’ve seen actually goes to either end of the spectrum. For the teams working with the more modern technologies and have to adopt DevOps very frequently, we find evangelists, those who have gotten to the point where they believe in the techniques and the approaches so passionately that they’re out there actually promoting them everywhere they can. It’s much like what we saw with the adoption of agile in the last decade. We’re seeing that same sort of passion surround this approach to development in this decade.

    You know, it’s one of those things, we have to keep in mind—DevOps was not even a term within the industry until 2009. I mean, it’s barely a decade old, and it’s in these last five years that we have seen the acceleration for adoption.

    Now, having said that, you know there certainly are always going to be pockets within any enterprise where, again, there’s resistance. There are those who are not looking for change and those that, perhaps, while they have embraced the DevOps approaches for their distributed systems or cloud development, look at the ideas of what they have to change or adapt to on the mainframe side and really don’t want to go there. The idea that there might be a long compile that has to be done on the mainframe before the next step in the DevOps process can occur will cause some—a little bit of agita.

    And that’s why it’s so important we need to take a look at the strategies that are necessary for DevOps success. I think that’s something that you speak to very eloquently, Mitch. And out of those steps, the one that I think is so critically important is culture shift. Just like with agile adoption, DevOps adoption needs support from the top, and it needs to be constantly vocal support to help people pull through these resistance curves. Resistance curves are natural and the quicker an enterprise gets through them, the quicker they get to a point that they’re delivering new capabilities to market much more quickly.

    Murray: Right.

    Ashley: Thanks for your kind words about the culture, too, and I want to get your thoughts on this, Anna. Culture is a lot of things, but I think we’re all human at the end of the day and none of us want to look dumb, right? We don’t want to tackle something we don’t know if we can learn or maybe not kinda look so good in front of our peers if we’re the ones behind the curve.

    And I think something that can help combat that is, one of the principles of DevOps is sharing, is really—it’s collaborating, but it’s also sharing. Sharing scripts, process, tools, content, methods—all of those kinds of things which, if you truly adopt DevOps, as you said, Jeff, you can become an advocate and then also help with that transition and very quickly, you know, they kinda get over that curve of, “Okay, alright, I’m not—I’m familiar with this,” as you mentioned, Anna, I can still use my green screen or many of the tools that I’m already using and I’m just adding to kinda what I’ve been doing.

    So, your thoughts, Anna?

    Murray: Right, well, I’ve recently been having other conversations with partners about this culture shift and it’s become really clear with other companies that are supporting DevOps and those initiatives that you really have to have not just buy-in from your senior management, you’ve got to have active support. You know, they have to sponsor the change.

    Because we’ve also seen companies who don’t go through the change, they fail. If it’s from the bottom up only and the senior leadership says, “Okay, yeah, sure, that’s a great idea,” but they’re not actually investing in it themselves, the team, when they hit road bumps—because they will. It’s not, you know, switching to DevOps doesn’t happen overnight. I think we all know that. And over time, you’re gonna hit roadblocks and you’re gonna get discouraged. And who is it that’s gonna lift you up?

    Now, hopefully, there’s evangelists that’ll help keep pulling you through and encouraging you, but at the end of the day, if your senior management isn’t coming back in to say, “Hey, guys, we believe in this process, it’s worth fixing, we’re gonna invest in whatever it takes to solve this problem,” then they’re encouraged to keep moving forward, right?

    But, you know, there is the difference happening out there where the leadership isn’t always invested in it. And so, that’s what we’re really encouraging when it comes to that culture shift that has to happen, make sure that you get not just buy-in but investment from your senior leadership.

    Ashley: Mm-hmm. Hugely important and it goes back to the, “Why are we doing this question?” that you talked about also, Jeff. You talked about really responding quickly and getting software out to market or quickly in today’s situation with COVID and uncertain as well as very quickly changing conditions, you know, something could very easily come down from Product Management or whoever to the development teams and say, “Okay, we need to add this capability or change how we’re doing something.” And the team’s ability to react quickly—I mean, they already do today, but even more quickly under these circumstances, I think, demonstrates the value not only of DevOps, but also those teams.

    Cherrington: Absolutely. Absolutely. I mean, it’s always a virtue to be able to get to market quickly. And in the current period of the pandemic, it’s a virtue that actually can impact lives on a broad scale.

    As the different national governments are taking the steps that they need to take to provide stimulus and secure the safety of their populations during all of this disruption, one of the things we’ve seen is that there have to be changes to critical traditional mainframe systems for all of this to happen. Certainly, the unemployment compensation that was distributed here in the U.S. touched legacy traditional systems at the state level that perhaps have not been actively developed for some period of years.

    And certainly, those government enterprises or commercial enterprises that were pre-prepared to apply their DevOps techniques to making those changes to their traditional offerings, traditional applications were better prepared to deal with this and to deal with the ongoing change as our government’s tried to figure out how to stabilize economies and get us through this challenging time.

    Murray: Mm-hmm.

    Ashley: You’re absolutely right, and that’s one example of many. I mean, there’s insurance claims processing to get your request fulfilled quickly, you know, new medicines—all of that kind of thing. There’s so many areas where mainframe technology is involved today.

    Well, I wish we could keep going. Maybe I have to come to the conference to continue the conversation. [Laughter]

    Cherrington: [Laughter]

    Murray: Yeah, we would love that.

    Cherrington: You would always be welcome.

    Ashley: Well, I appreciate that very much. And so, just a reminder for our audience, the EVOLVE conference is on October 6th and 7th. Go to the ASG.com website to find out how to register; I absolutely recommend doing that. And as I mentioned earlier, we have a research report coming out and the working title of that report, I think, was “Five Critical Strategies for DevOps and Mainframe.” So, we’ll see, maybe that title will stick or something close to that. I’m excited to get that out because that’s been a great effort working with you.

    Well, Anna Murray, thank you very much; Jeff Cherrington, also—thank you for joining us today.

    Cherrington: Thank you so much.

    Murray: Thank you, Mitch.

    Cherrington: And folks—stay safe.

    Murray: Yeah.

    Ashley: Alright.

    Murray: Alright, thank you.

    Ashley: You bet.

  • Bringing DevOps to SAP, With Basis Technologies

    Bringing DevOps to SAP, With Basis Technologies

    DevOps has not only reached mainframe groups in enterprise but also to large applications such as SAP. Organizations are moving away from waterfall methods and mega releases that take many months to deliver. What’s needed are many more rapid release cycles and deployments that support the accelerating pace necessary to do business.

    In this conversation on TechStrong TV, David Lees, CTO of Basis Technologies, joins MediaOps CEO Alan Shimel to explore how organizations implement DevOps into their SAP integration and deployment cycles. Having lived through this transition, David shares his unique perspectives on the necessity to break through bottlenecks and make changes in supply chains, finance and many other business processes needed to support the business.

    The audio file of our conversation is below, followed by the transcript. Enjoy!

    Transcript

    Alan Shimel: Hey, everyone. Thanks for joining us on another segment here for TechStrong TV. In this segment, we are joined by David Lees. David is the CTO at Basis Technologies. David, welcome to TechStrong TV.

    David Lees: Thanks, Alan. Great to be here.

    Shimel: Great to have you here. So, David, I don’t think we’ve done an interview here with Basis Technologies before, so I think we should start right there.

    Lees: Okay. So –

    Shimel: Go ahead.

    Lees: Basis is a company that’s been around for about 20 years. We are a global organization that works with hundreds of customers around the world, and the thing that they all have in common is that they run SAP software, as I’m sure many of your audience do.

    Shimel: Yep.

    Lees: We specialize in solutions around the application life cycle management space. We kind of started up the whole trend in terms of the DevOps for SAP by really helping people to manage their change in a much more Agile way than SAP is traditionally known for.

    Shimel: Excellent. And of course, you know, David we were talking off-camera before we got started, I’m doing devopsy.kinsta.cloud seven years, right? And when I first started everyone said, “Oh, DevOps is something for startups; I don’t know about large organizations.” And then sure enough every large – not every, but most large organizations in the world started recognizing the benefits of doing things in a DevOps fashion, and we fought this battle with DevOps in the mainframe, DevOps on things like SAP. How could something like SAP benefit from DevOps?

    But of course they can, of course they can, and I think SAP themselves has recognized it and they’ve come out with a lot of DevOps-related solutions, products, services, but also companies like Basis are recognizing that, hey, my customers are asking me about DevOps but they’re not going to throw away tens of millions of dollars and years and years of tribal knowledge that they’ve developed in doing things on SAP. It’s just impractical. And so, you know, they have to put a little chocolate in the peanut butter and it’s got to work together, right? I would imagine, you know, Basis has – this is a real specialty that the market’s calling for and Basis has a real role to fulfill here.

    Lees: Yeah. I think it’s as relevant in the SAP world as it is elsewhere for exactly the same reasons; customers are looking to become more Agile, they want to be able to deploy in kind of just in time, moving away from the whole annual or six-monthly mega release delivered in a very waterfall way in order to respond to market changes, customer needs, in order to drive more efficiency within their application development and application management space and to deliver that with quality that a business-critical system like SAP mandates, to be honest.

    Shimel: Sure. So David, I always like to ask our guests a little bit about their own personal background because people out here are wondering, right? So they look at David Lees, the CTO of Basis Technologies, they’ve been around 20 years in the SAP space, they deal a lot. David, what’s your personal background like?

    Lees: So I actually joined Basis only about 18 months ago, and prior to that I spent the majority of my career on the other side of the fence. I was with Proctor & Gamble, a large global consumer goods company, and pretty much most of that time was connected to SAP in some shape or form from implementations through support, upgrades, getting ready for transformation to things like S/4HANA, and it was in that role as a customer that I first came across Basis a few years ago and actually brought in one of their products into Proctor & Gamble. So I kind of first had that experience as a customer of Basis before deciding to kind of make the step in to working here as CTO.

    Shimel: Excellent. So you literally have a career in SAP that you bring to Basis, and as you said, from the other side of the fence; not as a vendor but as a user, a consumer, a customer of SAP.

    Lees: And like you were saying before, I think many of the customers of SAP that have been running with SAP software for now 10 if not 20-plus years have developed the knowledge, they’ve developed their solutions and their processes exactly how they like them, but that’s come with massive investment, massive investment in custom code, custom solutions, investment in the way they’re managing those developments, those projects and maintaining them on a going basis, and so I lived through firsthand this whole lifecycle, let’s say, plus the last few years where the old way of managing things was simply not going to be the right way to continue to manage in the future because at the end of the day, while the rest of the architecture around SAP is evolving at a much faster rate you can’t kind of retain the bottleneck image that SAP has and you have to find a way of breaking that bottleneck and being able to move at a similar pace to the other elements in the architecture. It’s not as easy as saying, okay, well let’s just not touch the back end and keep that totally frozen. It’s simply not possible. You need to make as many changes in your supply chain and finance and legal entity structure, et cetera, as you do in your ecommerce platform.

    Shimel: Fair enough. Excellent. Again, the parallels to what we saw in the mainframe space is also – but I think there’s something else you gotta remember. People don’t use SAP just because of their prior investment or just because it’s what they’ve done before, right? Organizations of a certain size use SAP because it offers real advantages and allows you to do things in such a way that you can’t have or you can’t get with just about any other solution out there. So it’s not just because this is what my grandpa did or this is what we’ve done for 10 years, 20 years; it’s because we are an organization of a certain scale that we require this.

    Lees: Yeah. I think they’re a market leader for a reason.

    Shimel: A reason, right.

    Lees: They have the products that, like you say, run some of the largest enterprises in the world; many of the Fortune 500 or FTSE 100 or what have you. And it’s in spite of that that we need to find a way of being able to manage that in a much more modern way. There’s certain elements of SAP that I’m sure some of your kind of more pure DevOps listeners or audience would struggle to understand in terms of, for example, the way version management is handled in SAP inside like a single database, hard to manage branches in terms of parallel developments, the way changes are deployed through the use of things like transports, create to some extent an ecosystem in SAP that is different than a lot of the cloud native and more modern applications, but at the same time we want to be able to help customers to manage the risk, to manage the agility, and to be able to move things through with confidence and with speed in keeping their changes across their architecture orchestrated in a controlled way.

    Shimel: Absolutely. So David, if you don’t mind, I think we’ve laid the groundwork here well, let’s look at, if you will, the anatomy of an SAP-DevOps engagement, right? We spoke about maybe it’s an existing SAP customer who – right? – doing that big yearly release or twice-a-year release is not going to cut it anymore. They need to be more agile – nimble agile, not necessarily Agile software development, though that too, but they need to be more nimble, they need to be more responsive, they need to – they want to utilize a modern CI/CD pipeline type of delivery system.

    Lees: Yeah. I think as with anything here you’ve got the three normal components. It’s about the people, the process, and the tools. So there’s no kind of silver bullet that’s going to say, yeah, I’m suddenly DevOps in my SAP space. You need to have similar best practices around automating as much as possible within that pipeline, and some of that’s available natively inside SAP to do things like automated unit testing, automated code inspection, but the piece of the jigsaw, let’s say, where we come in is to create effectively the pipeline portion for SAP using one of our products called ActiveControl, and that’s about being able to deploy in an automated way within the landscape to the point where the customer is comfortable and being able to kind of highlight and manage all of the interdependencies or identify where it is safe or isn’t safe to move changes in a more accelerated way.

    So I think some of the best practices I would say are common across the SAP and the cloud native world, and when it comes down to some of the tooling you need to start looking at SAP-specific tooling where it’s not possible to leverage some of the kind of off-the-shelf open source best practice tools like best-of-breed ones such as Jenkins or Git, but equally this is where we can integrate and we can help a customer in managing their SAP side of their estate, their architecture, in a coordinated or orchestrated way with the non-SAP world that they’re managing using some of these more traditional DevOps-based pipeline tools or CI/CD tools.

    Shimel: Yep. Agreed. So David, when Basis comes in – let’s talk about the engagement now, right? Is it more of like a classic consulting where we come in with bodies or is it more of an architectural and here’s the processes and kind of maybe software we’re going to introduce or both?

    Lees: Yeah. So we are primarily a software company in terms of providing the solutions, although we do recognize and we do work with some of our customers that they also need help on what are the process changes or architectural changes that they should make in order to go further, in order to become more Agile, what are some of the, again, best practices that they should be doing without switching on or without investing in? They won’t be able to go as far as they want to or need to. So whilst we are today primarily a software company and we will come in to help our customers activate those solutions and deploy them, we are starting to kind of branch out a little bit more into the consulting side to help customers on that journey, to define what that journey looks like for them based on where they are today and based on where they want to get to.

    Shimel: Fair enough. So I’ve got to kind of ask it only because you have to in these days – how’s COVID affected all of this?

    Lees: I think we’re affected to some extent like many companies; however, we are to a certain extent well insulated. We have a subscription-based model, so the companies that tend to deploy our solutions continue with those solutions. We have an incredibly high renewal rate which we’re very proud of, and that gives us a very strong cashflow and a very strong platform in which we can continue to invest in new products and solutions as well as to continue to invest in identifying and adding new customers. I think to some extent there is – some of the pursuits, some of our prospects have been impacted by budget restrictions, but I think at the same time COVID is accelerating significantly the pace at which companies realize they need to transform. So I think it’s probably as much as one side kind of slowdown and budget restrictions as it is kind of companies coming in saying we have to modernize, we have to change the way we’re doing things and we need our SAP application development to be as flexible as we have elsewhere in our organization.

    Shimel: Agreed. And, you know, I kind of suspected that but I wanted to kind of confirm it with you, is yeah, you know, on the one hand it definitely – everyone working remote and everything that went on you could see it slowing down, but on the other hand it becomes very much a catalyst for transformation, right? There’s no waiting for tomorrow; we’ve got to do this now, and so it happens in there. What about working remote though, David? I mean you’re obviously in your home office right now.

    Lees: I am and I [laughs] was always set up in a home office kind of gig here at Basis, so I’m based in Spain whereas the headquarters is in London. So even pre-COVID I was kind of used to working a few weeks from home and then going into the office for a week or so every month. Now it’s just all at home for the foreseeable future. We’ve had to adapt. Development teams that used to be very – used to kind of working together, whiteboards and daily standups and that sort of thing have found ways of doing that remotely like everyone else has had to do, and yeah, I think we’re all getting used to a very interesting new world, and I would be very surprised if we go back to exactly how it was previously.

    Shimel: I don’t disagree with you. But the good news is, you know what, you’re still able to do your work, stuff gets done, and it could be worse, as they say. Hey, we’re about out of time but I wanted to mention one other subject. You guys are doing a webinar here on devopsy.kinsta.cloud. It’s probably about a month from now, maybe even a little more than a month from now. It’s on SAP and DevOps.

    Lees: Yeah.

    Shimel: I’m not going to put you on the spot here and spill the whole beans, but who should be – you know, talk to us about it.

    Lees: I think the intended audience for that is people who are traditionally on the DevOps side of the fence who’ve grown up outside of SAP who need to or are realizing that SAP should become part of what they’re including in their plans and in their strategies. Now I would still say SAP kind of long-timers would get some value out of that webinar, but it’s more about helping people understand how SAP is relevant, why it’s relevant, and how we can help customers bringing the SAP world together with the more modern side of the application house for customers, and so I think it’s primarily tailored towards that audience.

    Shimel: Cool. More information on this will be available on devopsy.kinsta.cloud under our webinar section. David, I look forward to that webinar. I want to thank you for joining us on TechStrong TV today. Where in Spain are you based, by the way?

    Lees: Near Málaga on the South Coast, so a nice little town.

    Shimel: Oh, it’s terrible, terrible, terrible.

    Lees: Like yourself in Florida.

    Shimel: Yeah, I was gonna say, in Boca Raton I really can’t complain either. But anyway, hey, thanks for joining us, look forward to the webinar in October, and keep up the great work.

    Lees: Great. Well, thanks a lot, Alan. I look forward to coming back on, talking to you again in the future.

    Shimel: Absolutely. David Lees, Basis Technologies, SAP and DevOps. It’s real. It’s something you need to be thinking about, especially if your organization is already an SAP shop. This is Alan Shimel. We’ll be right back with our next guest here on TechStrong TV.

  • DevOps Chats: Easing WFH with strongDM

    DevOps Chats: Easing WFH with strongDM

    strongDM wasn’t founded to help with remote workers forced out of the office by COVID-19. They have been around for five years now. But, the company does excel at allowing you to manage and audit remote access to all of your assets including DBs, servers and clusters (as well as apps and services) anywhere from anywhere.

    In this DevOps Chat we speak with strongDM Justin McCarthy, CTO and co-founder about how the company’s technology works and how it is helping companies move to remote access given current conditions.

    As usual, the streaming audio is immediately below, followed by the transcript of our conversation.

    Transcript

    Alan Shimel: Hey, everyone, this is Alan Shimel, and you’re listening to another DevOps Chat.

    Really happy to be joined on this chat by Justin McCarthy of strongDM. Hey, Justin, welcome to DevOps Chat.

    Justin McCarthy: Hey, Alan — thanks.

    Shimel: Pleasure to have you on here, Justin. You know, as we were talking a little bit off mic, I don’t know if we’ve introduced our MediaOps community, meaning DevOps and Security Boulevard and Container Journal, we haven’t introduced strongDM to them before. So, why don’t we start with that, Justin, and if you wouldn’t mind, in telling the story, talk about a little bit about your own personal journey.

    McCarthy: Yeah, sure thing. Alright, so, first, I guess I’ll just briefly introduce myself. So, I’m the co-founder and CTO of strongDM. And the product is actually, strongDM itself is the best way to manage and audit remote access to your servers and databases and really even Kubernetes clusters. So, that’s what we focus on. We are a remote access product.

    And the way we came to it is basically just a sort of typical startup story. We’ve been around for five years now, and at the beginning, it was scratching an itch just like so many other infrastructure and DevOps products out there. So, in my career, I’d always been involved in growing the teams that built the product and, inevitably, when you build a product and when you ship and when you have production data, you need to grant access to it. To analyze it, you need to grant access to it to administer it, to debug it.

    And that process always felt a little bit somewhere between inconvenient and scary, right? [Laughter] And so, basically, strongDM — yeah, is the response to all those feelings of, that it wasn’t quite as smooth as it could be, it wasn’t quite as safe as it could be.

    Shimel: Excellent, excellent. And you mentioned the company’s around now about five years.

    McCarthy: Mm-hmm.

    Shimel: That’s cool. Alright, let’s talk about — and you’re right, a lot of companies do grow out of one particular scratch to itch, and I’ve done my share of startups over the last 25 years. And what generally happens is, you find out that that particular itch may have associated itches, right, and there’s ancillary things — well, once I scratch that itch, the next itch pops up, right? To paraphrase “The Phoenix Project” or something and bottlenecks and the goal.

    Talk to us—what’s the itch, and then what have you guys found over the five years about kinda getting to that itch and associated itches?

    McCarthy: Yeah, yeah, definitely. So, at the core, the technical approach that it probably takes is that we are an identity-ware proxy, okay? And the first application of that was for database access. So, that’s where we started, right?

    Shimel: Mm-hmm.

    McCarthy: And in fact, at the beginning of time, we thought maybe that was all there was to it. But as we —

    Shimel: Hold on a second, though.

    McCarthy: Yeah, sure.

    Shimel: So, an identity-ware proxy, primarily for databases. For some of our audiences — our security … actually, our DevOps audience probably gets that, some of our security folks do. But break it down for me, Justin — what exactly do you mean by that?

    McCarthy: Sure thing. So, this is an architecture that I think a lot of folks might know as Zero Trust, a lot of folks might know as BeyondCorp, and there were a lot of examples of it out there in sort of an HTTP only case. When we originally conceived of and built the product, it was kind of even prior to a lot of that nomenclature existing.

    So, what an identity-ware proxy is, it is a proxy for a given type of service. Let’s continue to use databases as the example. So, if somebody needs to interact with a Postgres database or a Microsoft SQL Server database — so, let’s say Alan needs to connect your local copy of Tableau to one of those systems, right? So, Alan would log into strongDM and then strongDM would then conduct all of the traffic, essentially, forward it to the target systems. And the identity-ware aspect is that, essentially, we know it’s Alan the whole time and we’re proxying that traffic and we’re handling, essentially, all of the routing of that traffic.

    Shimel: Got it.

    McCarthy: And what’s interesting about that is that you kind of get a lot of the sort of — you get a separation of sort of Alan’s identity from, you know, the credentialing of the underlying system. So, you can actually credential that separately, which means you can rotate the underlying credentials without bothering Alan.

    Shimel: Now, in and of itself, can that be a security kinda setting up kinda almost a man in the middle type of situation?

    McCarthy: Yeah, so, in a lot of enterprises and environments, you at some point, you have to answer the question, how are we gonna audit this particular traffic stream, right? And so, there are also a lot of products out there that are dedicated to, for example, breaking TLS, right? Being the official man in the middle, right?

    Shimel: Yep.

    McCarthy: So, our product has an element of that built in, although it’s not a generic TLS breaker. What it is, is it’s a protocol specific way to answer the question, you know, given MySQL database selection, what would you want to know about what Alan did yesterday? And so, our product is focused on extracting that kind of rich information about what Alan did yesterday with respect to this MySQL connection.

    Shimel: Sure — great, great. Now, what made databases sort of, you know, the prime example of where to use this or as the first use case?

    McCarthy: Yeah, it was the first use case, largely because, again, we were scratching our own itch. [Laughter] So, that was the one that I most often encountered.

    Shimel: Okay.

    McCarthy: You know, and the appeal is, it’s phrased as, you know, “Can I please have access to this because I need to debug X?” Right? And that always, whoever is saying yes or maybe or no to that request, you know, there’s a lot of hope and trust that when you say yes it’s gonna be okay. So, what our product does is, it takes all that, the sort of emotions around that event [Laughter] and tries to give you an answer for how you make that convenient and how you make that safe for everyone.

    Shimel: Excellent, man — good. Alright. So, first ditch is the database and we figure that out and go from there, Justin.

    McCarthy: Sure. So, as I’m sure, as you found with many products, you know, why adopt 10 different products when you can get your existing vendor to do one more thing for you and then maybe reduce and concentrate more of your attention on one tool and so, that’s where our customers pushed us. And so, the next very strong demand that we saw was, now that all of our data systems are granted and revoked access through strongDM, could you also begin to do some of the interactive session and shell-based protocols? So, the first two there were obviously SSH and Windows Remote Desktop. So, that’s where we went next.

    And the theme, if you can sort of imagine it is, you know, everyone in my organization that maybe has engineer in their title or even analyst or scientist — essentially, all of those folks, they need more than just webpages to do their job, right?

    Shimel: They do.

    McCarthy: And so, if it were just webpages, then existing single sign-on would work great, right? But since it’s more than that, it’s deeper than that, it’s really that whole collection of use cases that our customers began asking us to take on.

    Shimel: Excellent. Good stuff. And so, let’s talk a little bit about — so, how, is this sort of distributed as a SaaS kind of thing or a web-based kinda interface? How does that work?

    McCarthy: Sure, yeah. I would say we are neither traditional SaaS, nor traditional on prem. So, in terms of how the product deploys, our customers end up running a bunch of it, but not all of it. So, with any system that involves credentials, any system that involves cryptography, operating it successfully and carefully is actually a huge part of getting it right, okay?

    Shimel: Mm-hmm.

    McCarthy: So, what we’ve tried to do is, we’ve tried to unify the sort of operation configuration. So, if you think of it like centralizing the policies, centralizing the key management.

    Shimel: Yeah.

    McCarthy: So, that’s the part that we host. But the part that actually runs the proxies and therefore essentially the actual conduits for all the data, that runs entirely in customer environments.

    Shimel: And what kind of environment are you talking about?

    McCarthy: Yeah, so, I would say our average customer, the answer is, like, several. [Laughter] So, there are very few environments where you have just one cloud, where you have just one prem data center. So, it ends up actually being several. And so, that’s another strength of our architecture and our approach is that our product, from day zero, was designed with that in mind, so you can run our proxies in Azure and in AWS and in your existing legacy data center. They all unify into one sort of access plane, and it doesn’t matter the product. The same grant access gesture applies to all of those environments.

    Shimel: Oh, okay. Very cool. Very cool. And, you know, from what you’re describing, to me, it doesn’t really make a difference whether I’m a cloud native company, let’s call it, right, or maybe an old line company that had my database running in the closet, right? This kind of identity by proxy is still something I can utilize to make my life easier.

    McCarthy: Yeah, absolutely. And of course, we would hope that it’s especially suited to a hybrid environment like that, because the ability to, from an end user point of view, from one of your data scientists, they just know the three logical things they need to access to do their job. They definitely don’t care what cloud it’s hosted in. They don’t care what data center it’s hosted in, but from the point of view of that end user, it just looks like one thing.

    Shimel: Okay. Now, let’s look at it from the lens of COVID, which we kinda always have to look at everything today through this lens — it seems like this would be a help, right, if I just had to move my whole workforce home, you know, remote. What have you guys been seeing on that?

    McCarthy: Yeah, I think our story there is pretty similar to what a lot of companies that are part of the sort of next generation of IT and — we’re sort of seeing the same trend that everybody else, more people moving remove, therefore any remote access products are, there’s a lot more interest at the moment.

    So, we were always about granting access remotely. It was always a workforce that was going to be distributed. That was our typical customer, anyway. So, it’s definitely the same theme, but just more than we were seeing pre-COVID.

    Shimel: Excellent, man. Now, one thing — it’s funny, I just had a conversation with another company within the IDM space, kind of in like an ED directory service, let’s call it, and this popped up. With the people working from home, with the explosion of how many apps and SaaS services, you know, we’re all accessing right now, it’s great to be able to have just a single proxy, if you will. So, I only have to worry about really going once and then on the back end, it connects to all of these services, let’s call them, or apps.

    And so, from my point of view, that’s a great thing. From your point of view, Justin, right, now you’ve gotta hook into all of these different, new apps and services. How do you do that? Is that via API? Is it a one-off for every one you’ve gotta go into? You know, what’s involved there if I bring you — you know, some app you guys haven’t encountered before?

    McCarthy: Yep, sure. So, I do, there is an important distinction here. There’s a generation and a category of products that you might call them cloud brokers, right? So, cloud access security brokers, that category where you have a workforce that needs to access Salesforce, needs to access another SaaS application. We consider that sort of part of the market to be something that’s necessary but not our specialty.

    Shimel: Not your place.

    McCarthy: Yeah. So, our specialty is, again, it’s that staff where there’s probably an engineer or somehow technical, there’s a technical name in their role somewhere, and they need access to things that aren’t webpages. And so, for that, the procedure is — hey, I’ve got this maybe let’s say even legacy database or I’ve got this type of terminal access that we do that, let’s say we’ve got a lot of machines that are connected via VNC or something. So, and let’s say that’s not on our official supportive protocols list yet. It’s as simple as, we do an intake, we understand the protocol, we add it to the product — that’s it.

    Shimel: Okay. So, you guys do that. I don’t do that myself. I don’t have to kinda write to an API or something and make —

    McCarthy: Yeah, no, exactly. Any system that feels like any sort of client server access paradigm, we can model it in our, essentially in our platform, and then begin granting access to it.

    Shimel: Okay. Almost out of time, Justin. One more area I wanted to hit on this one, though, and that is — talk to me about what to set up here, right? Whether I’m doing it on the cloud, prem, a little of both — how big a job is it to get this thing up and running?

    McCarthy: Yeah. So, we also, of course, have the advantage of — like I said, we’re five years old, but that means we’re only five years old. [Laughter] So, we knew it had to be easy, right? Yeah.

    So, yeah, the act of deploying our proxies is essentially roughly turning it on. Because what then the proxies do is, they take over, for example, topics like discovery. So, they discover what host names they can access and they begin exposing it back to it in the administrative console. So, that’s the one aspect that, you know, deploying it is pretty much turning on the software.

    Additionally, because so many of our customers rely heavily on automation for their DevOps, another way to think of it is, if you already have, if you already have a great Terraform setup that you’re happy with, you have a great cloud formation setup that you’re happy with, we are gonna integrate natively with that setup as well. So, it’s going to be, you’re essentially the overlay, your proxy overlay on your existing infrastructure is gonna go in and it’s gonna feel exactly like anything else that you’re familiar with deploying via Terraform or inside your Kubernetes cluster, for example.

    Shimel: Perfect, man. All good stuff. Justin, I promised you we were gonna do 15 minutes, but I blew it — but not too bad. We’re not too bad. Well, we’re gonna have you back on. It’s gonna be a little bit of a continuing series, right? So, look forward to our next conversation.

    You know what I realized? We didn’t even mention the website, though. So, for people who want more information, it’s strongDM, S-T-R-O-N-G-D-M-dot com, right?

    McCarthy: Yep.

    Shimel: And then can get, if they wanna go check stuff out there. And we’ll have you back on real soon and continue this, man. Sounds great.

    McCarthy: Great. Alright, thanks a lot, Alan.

    Shimel: Alright. Justin McCarthy, CTO and co-founder of strongDM here on DevOps Chat. Hey, this is Alan Shimel, and you’ve just listened to another DevOps Chat.

  • DevOps Chats: Shadow Code Security with PerimeterX

    DevOps Chats: Shadow Code Security with PerimeterX

    When the cloud first caught on, there was a problem with Shadow IT—developers spinning up instances in AWS without the IT team knowing they existed. Now with the ease of CI/DC automated deployments, the problem of Shadow Code—code being added to apps that did not go through the entire team process—has arisen.

    In this DevOps Chats we speak with Elad Koren, VP of product at PerimeterX, about the Shadow Code issue and how his company can help.

    As usual, the streaming audio is immediately below, followed by the transcript of our conversation.

    Transcript

    Alan Shimel: Hey, everyone, this is Alan Shimel, and you’re listening to DevOps Chats. Today’s DevOps Chats features Elad Koren, who is VP of product at PerimeterX, and Elad, welcome to DevOps Chats.

    Elad Koren: Thanks, Alan. It’s an honor being here. Happy to join you.

    Shimel: I appreciate you being here, and it’s our honor to have you. So, Elad, we’ve never—I don’t believe so—we’ve never interviewed or covered PerimeterX before on Security Boulevard or DevOps. And why don’t we start with, why don’t you give our audience a little background—what’s PerimeterX about?

    Koren: Cool. I believe you’re right. PerimeterX has been around for a bit more than five years, and its main goal is to secure businesses, digital businesses across the web. It can be e-commerce, SaaS, and other online businesses, essentially providing solutions to the online digital challenges they have.

    We started with our Bot Defender solution about four years ago, and this targets the bot problem around the web. I’m sure you’re well aware of it, being the security professional you are. About a year ago, we’ve also launched our Code Defender product, which looks at the client-side code and highlights vulnerabilities and issues around that. And we will, of course, develop some more in the future as part of our growth plans to continue in securing digital businesses around the world.

    Shimel: Excellent. So, we’re gonna dive into that in just a second, but Elad, I also always like to ask people about their own personal journey. How did you come to be VP of Products at PerimeterX?

    Koren: Wow. [Laughter] It was a long journey. I’m actually, I’ve actually been in security for more than 15 years. I started with the Israeli Intelligence and the IDF. Then moved to my startup where I took all the security and kind of compliance and fraud areas. From there, I moved on to RSA Security, so you’re probably familiar with RSA Security, now part of Dell. And after a short period in Payoneer as their Head of Compliance and Security, I found myself in PerimeterX, leading the product group and now the VP of Product. I’m very proud of what we do with really, a sense of accomplishment with what we’ve been doing so far.

    Shimel: Excellent. Well, RSA Security was spun out of Dell now, right? Didn’t they do a PE deal a couple months ago, I thought?

    Koren: Yeah, yeah, they had a discussion around that. It was actually interesting because RSA was one of the biggest assets EMC had before Dell acquired them. And once Dell stepped in, I think RSA wasn’t that interesting for them, so it was only a matter of time.

    Shimel: Yeah, no, Dell seems to have a love/hate relationship with the security companies they acquire, but that’s for another podcast. Let’s stick to PerimeterX. [Laughter]

    So, client-side code inspections and security—talk to us a little bit about what you’re doing there and then I wanna jump into this concept of shadow code that we spoke about off-mic.

    Koren: Yeah. So, first of all, very important to understand, you know, the first question that comes to mind is, why are we talking about it on devopsy.kinsta.cloud. So, one of PerimeterX’s top targets is to assist DevOps teams to do their job better. And the fact of the matter is that, after we’ve had our bot mitigation solution for quite a while and DevOps was one of our personas, we went out and asked our customers, what’s their next major problem.

    And one of the things that came up for many, many customers was the fact that they have a challenge around their client side, okay? Developers are adding libraries—well, you know, it’s a common practice. However, some of these libraries, JavaScript or others, they’re not that secure, and they’re not that trustworthy. What we did at that point is kind of think of that problem and what it reminded of. And it kind of reminded us of the problem of the shadow IT that we’ve seen about 10 years ago or so, where employees would just come with their devices or do whatever they want with their bring your own device kind of concept, and companies started fighting that. But at the end of the day, they understood that they have no weight.

    It’s kind of similar with code. Developers will find the best weight to add code, even if it’s a library that they can just take and adjust slightly, and even if it’s a GitHub repository that no one maintains, and these introduce significant risk.

    Shimel: Yeah. Absolutely. You know, it’s interesting, I think no one in DevOps questions anymore when we talk about adding security to the mix. I mean, thank God. You know, a little bit of my own personal journey, right? The reason I got into DevOps is because I thought it was the greatest thing for security. I thought this, the way it came down that this whole DevSecOps movement and shifting left and making developers more aware of security—I envisioned that when I first found out about DevOps. And I was hoping it would give us another chance to do it right, to correct some of the mistakes that were kind of inherent in our whole model.

    And I’m not saying it’s a panacea or it’s perfect now, but certainly, we can acknowledge that developers do care about—no one wants to develop insecure or crappy code. People have pride in what code they develop. Now, when they didn’t know—the more they know about security and the more tools we can give them that allow them to kind of do better, develop more secure code, the better off we are, I think.

    But this issue of, as you’re calling it, shadow code, right, kinda growing ________ with shadow IT when the cloud first came.

    Koren: Yeah.

    Shimel: It’s a problem. And the thing about it is, again, developers don’t raise their hand and say, “I wanna sneak around. I wanna sneak a little code in the back door.”

    Koren: Exactly.

    Shimel: Right? They do it because they have time constraints, they wanna get it done, they want it to work faster, right?

    Koren: They’re exactly the reasons that we’ve understood when we looked into it.

    Shimel: Yeah.

    Koren: And it makes sense. If you think about it, developers should be able to add whatever it is that they think can help them develop faster. I mean, it’s the business goal.

    Shimel: Mm-hmm.

    Koren: The main question that we asked is—well, how can you make it more streamlined, okay? How can you—and now, going one step backwards or sideways, there’s a disconnect, okay? The developers, as you said, they wanna make it a secure code, but the business owner or the person who manages the security posture of the application doesn’t necessarily know that somebody introduced a new vulnerability, because the developer thinks that this repository is safe. I mean, it should be, it’s in GitHub. [Laughter]

    Shimel: Oh, no—agreed.

    Koren: So, the thing is—and this is where we asked around, our customers, what are they doing to mitigate this type of risk and what are they finding out. And what we found out is, interestingly enough, you have a lot of performance monitoring tools, but you have zero security monitoring tools on client side in real time. You have static code analysis, you have a lot of other solutions trying to find the vulnerability before it goes out to production. But once it’s out there, nobody monitors.

    And you have to keep in mind—I know you are keeping in mind, but our listeners—the fact that, well, when in run time, you can inject dynamic code, that changes. And if somebody else gets control over it, this is a problem.

    Shimel: Yeah, yeah. Well, and there’s a couple, I think, other complicating factors, Elad, that I’d like you to address. And that is, you know, today, where does that client side code live? Is it in the container somewhere with the whole Kubernetes microservices mesh thing goin’ on?

    Koren: Mm-hmm.

    Shimel: Is it in the traditional hypervisor kind of environment, or maybe something like that or is it on bare metal and serverless or—you know, it’s a great time to be a developer, but it’s a very hard time to be a security person who’s trying to keep guardrails there for these people to do it as safely as possible, as securely as possible. How do you do that?

    Koren: This is correct. You’re spot on. You’d expect that when a new code is introduced, the company or the developers would adopt it and manage it or maintain on their servers or on their microservers or their mesh. However, because they want to keep it updated, because they want to keep it as capable as possible, they sometimes take the dynamic library from their repository and, whenever there’s a CI/CD process going on, it pulls the library from there.

    And that is where the shadow code becomes a problem. Because, unless you have the right processes around, how to make sure that this library is safe? How to make sure that this is, you’re not introducing anything malicious—and we have seen it happening in the past—this is where you have to address the problem. And you know, I can share some of the things that we’ve seen when we had the validation of the product before we launched it. We had a design partner, and they asked for something like this because their developers asked for the security team to allow them visibility into what’s happening on the client side. And we’ve actually—and it’s a big customer. I don’t want to name names, naturally, here, but—and you will share my surprise—when we found out, when we run on their client side that a very, very big vendor of there is using a jQuery from a non-trustworthy source.

    Now, you’d expect something like this would never happen for a well-established company, but it does. And it’s surprising, but it does. And they weren’t the only ones.

    Shimel: Hmm. Interesting. So, let’s now talk—so, I think we’ve defined the issue and the problem, right? How is PerimeterX helping this?

    Koren: So, our approach when looking at this issue or this problem, naturally, we are a Dev first company. So, we came and asked the developers that we were working with—the DevOps, DevSecOps, and some of the risk analysts, some of the personas—what would be most beneficial. And what they said, and this is how, you know, I’m the VP of Products. I’m a Product Manager at heart. And customer input is the number one thing that I’m looking into when I’m building the products.

    They said very clearly, if they can see, before moving from staging to dev to prod, what the code itself, when it’s pulled dynamically, is doing and have the visibility of what’s happening there in real-time on users—this is something that they don’t have today. Because that will introduce that level of visibility whenever something bad happens.

    I don’t know if you’ve seen some of the Magecart attacks that have happened from third-party libraries. Again, don’t wanna name names, but if somebody looks online, they can find out. When we come in and we analyze in real-time on the client side, on every user going into the website, what the scripts are doing there, whether they send information to a bad domain, whether they try to take the information and store it in a cookie and then use it later on—we can quickly highlight the main risks and main problems and vulnerabilities, like I mentioned earlier with jQuery, so that the DevOps or DevSecOps can quickly raise a flag and say to the developer, “Listen, this script you’ve added introduces new vulnerability.” You can’t see that in the static code analysis, but you will see that on run time on the client side. And that is something revolutionary. We don’t have that today in anyone else’s solution out there.

    Shimel: Hmm. Interesting. What—so, I always ask, and you’re the perfect person, VP of Product—now you’ve got a conundrum. You’ve got a really great security product here that’s gonna help developers with secure code. How do you—how do you sell it? How do you bring it into the organization? Do you talk to the security guy and say, “Hey, go talk to your developer team and tell them to use this?” Do you talk to your developers and say, “Tell your security team you wanna do this?” Do you gotta talk to the security team and the developer team? Whose budget does this come out of? Who ultimately is responsible for setting the policies and processes?

    Koren: And this is—

    Shimel: Is the solution hard to sell, maybe, though?

    Koren: You’re correct. So, theoretically, it’s hard to sell. However, the main key here or the key differentiator is the fact that when we went in to look how we wanna sell this solution, before we even built it, we established the fact that it should fit into their processes. If it’s not fitting into their processes, if they have to start defining things like content security policy, if they have to start maintaining this on a day to day basis, it’s not going to work.

    Solutions for DevOps or DevSecOps today are one, two, three. You have to have it in and you have to do it quickly and you have to have minimum manual effort, unless you definitely prove why it’s needed. And so, this is why we’ve added the fact that nobody needs to do the baselining on their own, we are doing that automatically with machine learning capabilities, what every script does, whether it’s ________ or not, and the fact that they can just have a small snippet on their client side and we’ll analyze whatever and flag in our portal or alerting system that they have, it actually fits into their CI/CD process, it fits into their day-to-day process. They can even have it as part of their regular security processes and have it fit into their SIM.

    And this is where it really makes the difference, because you don’t have to do anything manually and you solve a problem and you fit into their processes, and security people are very happy about it. I can tell you that one of the customers that actually already bought this product was very happy with the fact that they have visibility to this area and they don’t have to maintain it on their own, because we are doing that, the product does that out of the box. So, this was a key component in our product.

    Shimel: Huh. Excellent. Excellent, excellent. So—I told you, the time goes very fast. [Laughter]

    Koren: [Laughter]

    Shimel: We’re coming up on time, here. For people who are interested—let’s use our time wisely—how could they find out more information? What path do you suggest for them?

    Koren: So, first of all, they can go out to our website and check out our solutions, both for bot mitigation as well as the Code Defender and other solutions that we have. I think reaching out to our team if you have any questions, if you have any concerns around the client side, I can tell you that even companies that thought everything’s fine were completely oblivious to threats and concerns around the client side that are out there and many people are not aware of. You can also reach out to me directly via LinkedIn or what have you and we can discuss.

    I think in general, for DevOps and DevSecOps to be more aware of the dynamic nature of the code that is added to the client side and what to do. We actually have a blog post listing the things that are best to do if you want to make sure that it’s secure and you’re on top of things. So, it’s a good read as well. And that’s about it. So, we are here. We’re ready to help.

    Shimel: Cool. Last question, just because you have to ask it because it’s out there—with everything going on in the world with the COVID and everything else, how is it affecting you guys and what are you seeing?

    Koren: It’s actually amazing. We’ve seen some trends around the e-commerce landscape that we’ve actually had some blog posts on. And I think, you know, one of the things that kind of symbolizes these times, a few of the e-commerce sites that we are working with said that the traffic they experienced outgrows the Black Friday/Cyber Monday level of traffic.

    I think it symbolizes kind of the boom of online life that we have. I’m sure you’re experiencing it as well. And I think the shift there is not going to go back to the previous levels. It’s definitely going to go down, but we are seeing so many trends, very interesting trends, and we are naturally adjusting what we are doing, but luckily and happily, we’re doing great these times and it’s—we’re very happy with what we have now.

    Shimel: Well, I will tell you, for whatever reason, you know, and it may not all be good, because there is this whole, with the virus pandemic there’s a cyber pandemic of people trying to exploit. I mean, so, for instance, even here MediaOps where DevOps and Security Boulevard, our security site, used to be pretty even in terms of how many visitors—they both get 350 to 400,000 visitors, unique visitors.

    Koren: Mm-hmm.

    Shimel: All of a sudden, you know, Security Boulevard now this month is gonna be at 600,000.

    Koren: Wow.

    Shimel: Last month was 530,000. So, in the last two months since this COVID thing, you know, we’ve seen traffic on the Boulevard site—DevOps has stayed relatively steady. It hasn’t gone up, necessarily, it hasn’t gone down.

    Koren: You know, one thing we’ve noticed, and I think it can also explain, I think it’s wider than that—attacks and in general fraud and security challenges became much more complicated, because the adversaries are out there and they are targeting a whole lot of other new sites that previously didn’t experience. We are seeing much higher levels of complexity in the attacks that we’ve previously seen on giants, now on even medium small—small- to medium-sized sites.

    Shimel: Agreed. Crazy. Just crazy, right?

    Koren: Yeah, indeed.

    Shimel: You know, because that’s what you need when you have a virus pandemic, we need to have more cybercrime and that stuff to worry about.

    Koren: Yeah. [Laughter]

    Shimel: So, Elad, I want to thank you for joining us today on DevOps Chats. It’s been a pleasure.

    Koren: Thanks, Alan.

    Shimel: Be back on soon. Elad Koren from PerimeterX here on DevOps Chats, this is Alan Shimel and you’ve just listened to another DevOps Chats.

  • DevOps Chats: Software Delivery Leadership Forum and DevOps World 2020, CloudBees

    DevOps Chats: Software Delivery Leadership Forum and DevOps World 2020, CloudBees

    DevOps World | Jenkins World has long established itself as a leading venue for DevOps practitioners to learn, share their knowledge and build a top-notch community. But, what about leaders, the people in leadership roles at companies who utilize DevOps as a vital part of their business and technology strategies?

    Sam Fell, CloudBees area VP enterprise markets, joins DevOps Chats to discuss the newly announced Software Delivery Leadership Forum. DevOps World 2020 is expanding its focus and content by bringing leaders together with content specifically tailored for business and technology leaders. Accelerated Strategies Group, where Mitch Ashley is CEO, is leading the programming of the DevOps World 2020 Software Delivery Leadership Forum.

    As usual, the streaming audio is immediately below, followed by the transcript of the conversation.

    Transcript

    Mitch Ashley: Hi, everyone, this is Mitch Ashley with devopsy.kinsta.cloud, and you’re listening to another DevOps Chat podcast. Today, I’m joined by Sam Fell, area vice president, enterprise marketing at CloudBees. And we’re talking about, actually, a subject of interest to both of us. It’s the new Software Delivery Leadership Forum, and we’ll get into that.

    Welcome to DevOps Chat, Sam.

    Sam Fell: Thank you, Mitch. Pleasure to be here, as always—great speaking with you.

    Ashley: Absolutely. I always enjoy talking with you. For folks who don’t know you, I know you’ve been in the industry around DevOps for a while. [Laughter] So, a lot of folks know you, but—

    Fell: Are you calling me old? Is that it?

    Ashley: Actually, I didn’t say that. I’ve got the gray hair; you’re still young, so.

    Fell: [Laughter]

    Ashley: Just to introduce yourself, for folks that may not know you.

    Fell: Alright, wonderful. So, I am Sam Fell. I am area vice president at CloudBees, where I’ve been for the past year. We were acquired in, as part of the Electric Cloud acquisition where I ran marketing at Electric Cloud.

    Ashley: Yeah, a lot of folks know Electric Cloud, so probably know you from that as well as CloudBees, of course.

    Fell: Yeah. That’s right.

    Ashley: How long ago did that acquisition happen?

    Fell: That was literally a year ago in May.

    Ashley: Okay.

    Fell: So, we are just a year, really just coming up on a year. So, all the exciting stuff was happening, all the stuff that made me lose my hair and all that was happening right now. And we were just so thrilled with the outcome to be, you know, the company that Electric Cloud was. We had a pretty nice name in the space. But when you look at CloudBees and you look at the community that they’ve created and you look at the access to the brains that they have, it’s a tremendous opportunity. You know, obviously, I’m really excited to be able to be here. And I’m excited about sharing this journey with the Accelerated Strategies Group, right?

    Ashley: Well, thank you. I’m excited about that, too. Yeah, CloudBees is a great organization, as was Electric Cloud.

    So, let’s get into the—we made an announcement, or actually, you did a press release about the Software Delivery Leadership Forum, and there’s a connection to DevOps World and some other things. Maybe we start with that and then we can talk about Accelerated Strategies’ involvement in it.

    Fell: Lovely. Awesome. Okay, so, you know, we’ve been running probably, people would say, the most successful DevOps conference for years now, DevOps World/Jenkins World. Almost 2,000 people show up to learn more about what’s happening in the space, there’s hands on practitioner based workshops and training for people to really get their hands on and get them dirty about how to implement these changes that have to happen as you’re digitally transforming your organization.

    One of the things that we took a step back and we were thinking about is, what we don’t really have is a place for the leadership to be able to participate. And, you know, we’ve done leadership seminars and workshops before, but we’ve never said that DevOps World needs to be a place where not only the practitioners are coming, but where the leaders are coming to learn not the practitioner level information, but how to lead the practitioners. What are the processes, what are the methods that people are learning and using and trying and failing with and learning from?

    And so, we thought we would add that to the program. And as we were thinking about adding it to the program, we also thought, because of the way that the DevOps World/Jenkins World show has evolved, I as a vendor looked back at DevOps World/Jenkins World and I was always very willing to sponsor that event when I was at Electric Cloud, because it was an open event. There was not a ton of bias towards CloudBees products, necessarily. It was really about the community, and it was about educating people.

    And so, in that same spirit, when we were thinking about how do we make sure that there’s some objective independence in the way that we’re putting out the ideas around, “How do you lead? How do you lead this transformation? How do you make sure that your teams are all on the same page?” We felt that it was really helpful to have a third party help us with that programming. And so, that’s, of course, this is the cue to Accelerated Strategies, this is why we sort of reached out to you. There’s obviously great synergy between the work that we’re doing, and we thought, who better to help us craft this program, the programming for this conference than the fine people over at Accelerated Strategies?

    Ashley: Good.

    Fell: So, that’s pretty much where we’re going, and then you and I have had lots of conversations about what that Software Delivery Leadership Forum should look like at the event, and so maybe I would pitch it back to you and you could talk a little bit about what we’re thinking about for DevOps World Software Delivery Leadership Forum.

    Ashley: Absolutely. And thanks so much, we appreciate you approaching us, and it was exciting to work with you on this. And I agree—you know, DevOps World, yes, it’s CloudBees behind it, Jenkins World, but the community that you talk about really elevates it above any one vendor, to use that word.

    Fell: Yeah.

    Ashley: And just the fact that Electric Cloud, you know, would—and many others would sponsor a conference that’s put on by another vendor.

    Fell: Yep, yep, coopetition. The competitors were there and happy about it because the audience is the right audience. These are the people that are in the weeds making it happen with their fingers, and now what we’re saying is, “Hey, these people over in the weeds that are using their fingers, they need other people to help on the top that can help support and pave the road for them.”

    Ashley: They do.

    Fell: And so, we’ve been trying to do that.

    Ashley: Well, and I would imagine, too, that, you know, one of the drivers behind leadership as a topic is organization scale. You know, trying to do DevOps on more than two or three or four independent projects, that’s when you get into the heavy lifting. That’s when you’ve got to have management support. And now I see—I know you do, too—leaders reaching out and trying to find resources for themselves, educate themselves, lessons learned, pitfalls, community that they can be part of as well as the technical folks be part of their community and that’s in large part from what we’ve talked to why to go after this Software Delivery Leadership Forum.

    Fell: That’s right, exactly. Do you wanna talk a little bit about what the proposed format would be at the—

    Ashley: Yeah.

    Fell: —Software Delivery Leadership Forum? Wonderful, okay.

    Ashley: Yeah, I’m happy to. We have some research that we’re gonna be launching, and talking about around software delivery that we’ll be discussing and presenting some of the findings from.

    But the main point of how we’ve crafted this to be, it’s a parallel track. It’s not a separate venue, separate place, it’s just a set of content that’s specifically directed at leaders, people in SE or VP director software leader kind of role, but it’s also product leaders as well as transformational leaders. And those are the folks that we really would like to attract to attend to this, because that combination is always what needs to come together to achieve some business strategy, disruptive strategy product delivery, some kind of financial effect that we’re trying to have on the organization.

    So, it’s a track of about 22 different sessions, but we’re gonna have a number of things that will be highly interactive. So, we’re gonna have some panels, we’ll have some audience participation, kinda open forum for questions and answers with different experts, and we’ll have people that—yes, there’ll be some folks who are product company, there’ll be folks who are analysts like from Accelerated, but not only Accelerated; we’ll have other analysts there. And then, we’d like to have as many practitioners in a business role technical leadership role as well to get that perspective.

    Fell: Right.

    Ashley: And we’ve planned some workshops and some things like that. So, my goal is that everybody take three things away from this conference. Whatever those three things are, come back to your company, to your team, to your organization and say, “You know? I picked up some really good stuff. This one, I had no idea somebody was facing the same issue and they actually solved it—”

    Fell: And they fixed it, right.

    Ashley: “—a way I was thinking about, so it validated our thinking.” Or,  you know, “Threw it out the window and we’ve got a whole other approach, but I’m sure glad I didn’t march down that Bataan death march.” [Laughter]

    Fell: Waste my time—failing fast is winning, exactly.

    Ashley: Exactly, exactly. So, the call for presentations is open right now.

    Fell: Yep.

    Ashley: You can go to DevOpsWorld.com, and there’s a button there that is a call for presentations, fill out information, we’d love to have folks, and we are reviewing submissions actively right now, so it’s a good time to get this out.

    Fell: Yeah. We’re getting—actually, from that release, we’re getting quite a few submissions, which is really nice. There’s some suggestions around some of the topics that we’re looking to have people talk about—governance, security, those are big topics. So, yeah, definitely interesting to get people’s perspectives and to see what they have to say. We’re excited about that.

    Ashley: It is. So, full disclosure, we both have an interest in this and participating, but it’s good to be on with you to talk about it.

    Fell: Yeah, for sure.

    Ashley: You know, I’m curious, too, because you’re doing this—well, let me ask you, why did you decide to name it the Software Delivery Leadership Forum? Because I know we went around a couple names, a couple of ideas.

    Fell: Yeah. So, well, when you think about the work that the fine people are doing, whether they’re doing it from a DevOps perspective or they’re just in dev or they’re just in ops or they’re part of the executive team who’s not part of development or operations, but they have a vested interest in development and operations getting their stuff done, right?

    Every—this is such a hackneyed expression, it’s very overused, but every business is a software business. And now, in this age we find ourselves in, the digital connection with your customers is more important than ever.

    Ashley: Mm-hmm, mm-hmm.

    Fell: And so, there are other people beyond just dev and ops. And so, originally, it was going to be the DevOps Leadership Forum, but it’s really, we think it’s more about that, delivering that software or delivering those services that are foundationally built on top of software, and you can call it whatever you want, but down at the base layer, somebody’s written code, somebody’s built that code, somebody had to deploy it to a server so that people could get any value from it.

    And so, the software delivery life cycle, the SDLC, is something that we think a lot about in the SDLF, like, the Software Delivery Leadership Forum. Because the SDLC is still alive, it’s still kicking, and application life cycle management matters, even in the age of DevOps. And so, we thought that this would be an appropriate name for what we’re trying to put together with you is a place for people, a community to come together that’s built on the community that we already have of all these practitioners who already understand how a lot of this stuff works, but now, they’re just looking for some guidance and some, “Hey, I don’t wanna go down that road again, because that guy already failed,” right? Failing fast and learning from it is not a failure.

    Ashley: Exactly. There’s so much sharing that happens in the technical community, I’m confident we can build that with the business product and technology leadership.

    Fell: Yep.

    Ashley: It’s interesting. We’re in the midst of, maybe we’re still in the early part of this, I don’t know the whole COVID-19 and being in lockdown and, of course, there’s the health and, in some cases, even losing people that we know in our family, or friends. So, in all due respect to that, you know, I’ve talked—some of the things I’ve talked about on another venue, which is TechStrong TV, is, it’s interesting to see this—

    Fell: Congratulations on that launch, by the way.

    Ashley: Thank you. It’s a lot of fun to do.

    Fell: Yeah, I love it. Yeah, that’s great.

    Ashley: It’s a lot of fun to do. Some great content on there. If you haven’t checked it out, go to TechStrong.tv.

    Fell: Yes.

    Ashley: Two to four hours a day, Monday through Friday, so—keep you company. I call it “the view” for technology people. [Laughter]

    Fell: Nice. Very nice.

    Ashley: So, even in this situation, you know, we’re locked down, right? Everybody’s working from home. Most people are. I’ve noted, you know, restaurants have all gone to delivery. They’ve gone to curbside delivery. Even restaurants around here have now started to come up with new offerings to essentially take home packaged meals, kinda like you would from one of those box delivery services.

    And even places like Best Buy has curbside delivery. I had to build a computer, and the delays of shipping something from a manufacturer were so long, you know, and they had to change their apps. They rebuilt their app or added to their app and their web notifications to have, you know, a button to push to say, “I’m in the parking lot and I’m in a white this car and here I am, and by the way, here’s my QR code or scan code,” or not even have to do that.

    Fell: To validate—yeah, exactly.

    Ashley: And that happened in weeks. That wasn’t—you know, that’s a few weeks that those two examples happened. Now think about—

    Fell: Life finds a way. Commerce finds a way, maybe that’s [Cross talk].

    Ashley: It does, it does. And kudos to those folks and many others who were looking for—maybe we didn’t disrupt it, but disruptive business strategies. [Laughter]

    Fell: Yeah. I mean, they have to respond.

    Ashley: Delivery through software.

    Fell: Yeah. They need to respond. Life finds a way, I love it—DevOps finds a way.

    Ashley: It’ll be interesting, I think, you know, given the time frame, the dates are September 21st through the 24th in Las Vegas. And, you know, we’ll have several months behind us, hopefully not too bad, but several months of experience. I bet there is gonna be a lot of stories, many stories of folks who have reacted and responded and some successes, and—

    Fell: Led through that change, right, exactly. Led through that change.

    Ashley: So, talk about a good test—well, not good, but talk about a test situation for the flexibility and agility of our businesses.

    Fell: Yeah. This is the crucible. That’s really the crucible, right here.

    Ashley: Yeah. It truly is.

    Fell: And so, one of the other things, you know, that we’re doing—and Mitch, you’re part of this as well—is that we’re so excited about the Software Delivery Leadership Forum that we actually don’t wanna wait for the September event to get started.

    And so, what we’re actually doing is, we’re creating an online virtual series of Software Delivery Leadership Forums, with our first one coming up at the end of this month in April. Episode one of this series is gonna be on upskilling. It’s on adapting humans at the speed of DevOps and we’re gonna be joined by our friend, Jayne Groll—our mutual friend Jayne Groll—and Eveline Oehrlich, who is her head of research.

    Ashley: Both are mutual friends, yeah.

    Fell: Right, and Eveline—

    Ashley: Both are analysts with Accelerated, too.

    Fell: She’s with you over at Accelerated Strategies, right. And we’ll be joined even further after they’ve given their little talk about the survey results, and we’re actually gonna get an interesting cut of data for the European, because we’re doing two versions of that, event—one for the U.S. time zone friendly and one for EMEA time zone friendly.

    Ashley: Mm-hmm.

    Fell: And so, for the EMEA one, there’s gonna be a different cut of the data that’s gonna be much more focused on that geography, which we’re really excited about. But we’re having a whole bunch of other folks join us as well to be able to have that conversation, because we’re really wanting to try and create, again, a community where there’s interaction between the attendees and the folks who are on the webinar or on this virtual event. Because so many events, people parade out folks that are saying, “Yeah, here’s how we’re successful” and then maybe you get five or 10 minutes at the end for questions and answers.

    And so, we’re kinda gonna flip that on its head and we’re gonna have 30 minutes up front for the Software Delivery Leadership Forum—30 minutes up front for Jayne and Eveline to go through their findings to talk as subject matter experts about this topic that they certainly know a lot about. And at that point, we’re going to introduce a couple of other panelists to join us on camera and start fielding questions directly from the audience.

    And so, the folks who are going to be joining us this time around, we’ve got Ellen Thorne, who is the head of HR here at CloudBees, so who better than a person who’s in HR to talk about what’s—that’s an interesting perspective. What’s unique about that from a—

    Ashley: Skills and training of their people, yeah.

    Fell: Yeah, training. Robert Reeves, who’s the CTO at Datical, right? So, how is the technical person who used to hire technical people, what is that like? What kind of people are you looking for? What skills do you find deficient? What skills do you find lacking that you’d like to see? Of course, you’re joining us.

    And so, really, really excited about that. And then over in Europe, we’ve got a couple of other people joining as well—Eveline Oehrlich will be on that one as well as Cheryl Razzell, who is over at, I think, PolyCom right now, and she used to be at HSBC. She’s not a customer, I think, any more, but she was a customer ad she definitely has an opinion about this.

    So, we’re really looking for people with different perspectives to have a conversation. I’m gonna be on there, you’re gonna be on there. I’ll be moderating, I’m gonna be looking at the chat window the entire time. All the panelists will be able to respond to the questions that are posed in the chat. There’s gonna be a mechanism for people to vote the different answers up or down, which is pretty cool.

    Ashley: Mm-hmm.

    Fell: And my role is really gonna just be to interject and speak on behalf of the people on the chat and say, “Hey, Bill brings up a good point,” “Hey, Debbie just brought up a really good point, she disagrees with you,” “He thinks that that is wrong,” whatever. I’m gonna try and be a conduit to make sure that the audience feels like they’re part of that conversation.

    So, we’re gonna have 30 minutes of up front, set the table subject matter expert speaking stuff, and then we’re gonna have an hour of live Q&A or as long as that audience has questions, we’re gonna work on trying to get to them.

    Ashley: Wow, an hour. That’s great, that’s fantastic. I mean, you know, community in some ways starts with conversation. That’s what draws people in and say, “What’s happening? What are those folks talking about? What is that? Oh, I have a question, I have a thought.” And I’m sure we’ll get some opinions coming in on those questions. [Laughter]

    Fell: That’s right.

    Ashley: Sometimes, they’re sort of written that way, that, “Hey, don’t you think this is true, instead of the way you said it?”

    Fell: Yeah, exactly.

    Ashley: Which is actually, it’s some of the challenge as well as seeking information that helps advance it for all of us, and—you know, there’s more than one way to skin a cat, if you will.

    Fell: I was on a crowd chat with DevOps Institute, I think it was about a week ago, and we had folks from all over the world chiming in and giving their, some prompts that we were asked. It’s fantastic to see people, like you said, to challenge the status quo, to give their opinion of what’s worked for them to understand. Because that—you know, you get an understanding of how some people’s situations are just different. And you can’t—there is no best practice. I think Manuel Pais and Matthew Skelton, they talk about this all the time. There’s no best practice, there’s just lots of good practices—

    Ashley: Good practices.

    Fell: —that you can apply to various different, apply to different areas, but you can’t just say that there is one best practice.

    Ashley: So, kind of a variation on the best practice theme, I’ll just ask you your—neither one of us can predict the future that accurately. [Laughter] But what are your thoughts about this virtual event, virtual conferences? Is that something you think will stay with us in a meaningful form post COVID and whatever, how we’re working, all that environment?

    Fell: I do. You know, you saw O’Reilly has canceled all of their physical conferences.

    Ashley: Yeah, yeah.

    Fell: And so, regardless of what ends up happening, I think the same way that the businesses like Best Buy and the other folks that you talked about are transitioning and they’re shifting.

    Ashley: Mm-hmm.

    Fell: I mean, I’m seeing—this week, I’m starting to see all the activities that my children were involved in are now, they’re finally acquiescing and saying, “Okay, we’re transitioning, and now, here’s the Zoom link for you to join for the kung fu class and you’re just gonna do it in your living room, but you have to still wear all of your things and you still have to show up and be polite.”

    So, they’re also transitioning, and businesses are transitioning. They’re learning how to work more remote. I’m very lucky to be working at a company that’s, you know, we’re 99, 90% remote already, so it’s not a huge change for us, but it is a huge change for the folks that are working remotely in my company who have, now, children at home.

    Ashley: Mm-hmm.

    Fell: And so, education is gonna be impacted and I’m talking with Charlie Betz, who’s an analyst over at Forrester, he’s a teacher over at the University of St. Paul—or St. Thomas, excuse me. And he’s talking, he and I talk all the time about how is education gonna be impacted by this? What are—as a parent, I’m sitting at home and I’m working, but my children are here, and they should be doing some school things, but it’s very hard for me to keep up with that and keep track of it. And I think it’s hard for a lot of the educators, because they haven’t had the experience in the virtual meeting space that we’ve all been sort of used to for a while.

    And so, I think there’s a lot of learning that needs to come out of this, but I see—and one of the things that Charlie says is, when people have no jobs, they can do a couple different things, but one of the things a lot of different people do is they go back to school, which is a fantastic thing to do.

    Ashley: Mm-hmm. Oh, yeah, absolutely.

    Fell: And if you give people viable options to educate themselves and make themselves—again, all about the upskilling conversation that we were just having. How do we enable these people to be able to make sure that they’re gonna stay relevant, and how do we help them get jobs for the folks that are right now, that they don’t have a job because they’re forced to be in an office that’s closed?

    Ashley: Right.

    Fell: So, I think that there will be quite a bit of focus on how do we educate at a distance, how do we make the collaboration work better. And I think it’s happening already naturally. People are getting more accustomed to being on video calls, right? People who used to never want to be on a video call, now they’re flipping on their video, because it’s like—you know what? We’re all in this together, and one of the things that I really appreciate, before we started scheduling this, I ran around to everyone in my house and I said, “Hey, guys, please don’t scream for the next 30 minutes, because I’m gonna be recording something.”

    Ashley: [Laughter]

    Fell: But the fact is, is that I’m on meetings with people and, you know, my kid or my wife will walk by and we’ll speak. And now, it’s—it used to be where I was embarrassed by that, because it was an intrusion into my workday. And sort of the mind shift that I think is happening is that now, what’s happened is that I’m allowing work into my home.

    Ashley: Yep. And you’re entering into other people’s homes at the same time.

    Fell: And I’m also entering—yeah, absolutely, I’m entering into other people’s homes. But I think there’s a much more understanding and a much more—I’m more comfortable, anyways, with there being life happening outside of me. It’s like the classic BBC video of the girl, the 2-year-old girl who runs in the room. He was—he was, like, appalled that that would happen. And now, if that happened, hopefully, he wouldn’t be, right?

    Ashley: It’s called life.

    Fell: So, I think, as an organization—it’s life, right? And hopefully, as a humanity, we’re getting better about that, and I think we will.

    Ashley: You know, to give you some, a little bit of data to back up part of what you were saying is, we’ve launched—we started an IT, health emergency IT preparedness survey back in February, and things happened so fast that just a traditional survey couldn’t keep up with the changes that are happening with work at home and how this has evolved.

    So, we’ve also launched a flash poll survey, and I’ll be talking about it on TechStrong TV this week. But one of the initial ones before we announced this flash poll, one of the initial ones was, “What is the most critical—you’ve only got one choice—between kinds of services that you use to support your work at home staff? There’s e-mail, collaboration services, meeting, file sharing, phone—all kinds of things.

    Fell: And which ones had degradations, yeah.

    Ashley: Well, yeah. And it’s, surprisingly, collaboration and online meeting services were equal. Those are the top two, within 1% of each other.

    Fell: The top two worst performing?

    Ashley: No, this is of our most critical to my business.

    Fell: Most critical—okay, got it.

    Ashley: Supporting it. We’re gonna do another survey on what have you had issues with that’s actually launched this week.

    Fell: Yeah, I just—that’s why I saw it, because I just saw that one. I was like, “What?”

    Ashley: You just saw it. [Laughter] So, 37 and 36%, I think, online meetings was 37. And so—and email was, like, 20. So, that tells me people have already made this shift into Slack and Microsoft Teams and Hangout and Google and whatever other forms they have for kinda their corporate chat.

    So, I think that skill of collaborating that way is something that will be a natural outcome of the situation we’re in.

    Fell: Yeah, I think so. I mean—yeah, hard times force people to think differently.

    Ashley: Mm-hmm.

    Fell: And we’re a very resilient species. We’ve been around for a while, and we’ll figure this out, but it’s terrible to have to go through it.

    Ashley: It is, it is. We just hope everybody gets through okay.

    Fell: Yeah, exactly.

    Ashley: That’s what we’re hoping for. Well, Sam, it’s a great pleasure, a lot of fun, always, to talk with you. Thanks for being on the podcast and for joining us today, sharing and also sharing with Accelerated Strategies the Software Delivery Leadership Forum. Excited to do all of those events with you and take that dialogue to the next level and really build the community amongst the leadership side.

    Fell: Me, too—wonderful. Thank you, Mitch. I appreciate your invitation and stay safe.

    Ashley: You bet. So, you’ve listened to another DevOps Chat podcast. This is Mitch Ashley thanking everyone for joining us today. Be safe, be careful out there.