devopsy.kinsta.cloud

Category: Features

  • DevOps: Get the Basics Right to Avoid Costly Mistakes

    DevOps: Get the Basics Right to Avoid Costly Mistakes

    It is a truth of both agile and DevOps that speed is of the essence. This is a relative statement—some apps/portfolios/orgs envision speed completely differently than others—but if you are releasing more frequently than you were in the past, thanks to DevOps, that’s kind of the point.

    Yet, we see a whole selection of things that fall by the wayside or are underserved. Security and testing are the easy/obvious ones. Considering the frequency with which we see “Unsecured Amazon storage bucket” (one announcement of this type spurred this blog), should we not see this folded into standard procedure or tools? Well, it is folded into standard tools for checking your cloud environment—I’ve used a couple of them and seen this warning come up. So shouldn’t organizations be using these tools more?

    One thing about speed: It tends to cut corners. And cut corners lead to lapses. You can be a blameless environment that doesn’t punish people for mistakes, but for an organization, a mistake such as exposing hundreds of thousands of users will be punished from the outside, from regulatory agencies and the market in general. They’re not mistakes we should be making, and yet it still happens.

    Another one is rollback versus roll forward. Most DevOps shops focus on the ability to fix an issue and plow forward. This works really well, 90+% of the time, and fails spectacularly sometimes. While it is an added step with some slowing of process, being able to roll back changes is important. And more than, “Well, theoretically, Git will allow us to go back to the last marked release …” because there is far more to an application architecture than just the source stored in a repository.

    In short, no matter how fast things are going, have a plan for when things go wrong. Because they inevitably will. And take the little bit of extra time for preventatives such as warning someone other than the creator about buckets with no security.

    “But there isn’t time …” is the common refrain to blogs such as this one. I get it. The increase in delivery only creates an increased appetite for delivery. But is there time to respond to disaster, should the worst happen? And I’d argue that it must be IT that plans for disasters. When someone gets fired because there was a security breach or a total systems failure, it is rarely an app owner. That puts the responsibility to make certain one doesn’t happen squarely on IT.

    We’re delivering faster, offering increased functionality to users, moving forward on every front. You all are out there rocking it daily. Don’t risk all of that being ignored because you failed to take care of basic issues. Check out the tools to help out; feature flags and cloud security scanners both spring to mind with regard to my examples above. The tools are out there, you just have to work them into your process.

    — Don Macvittie

  • Oracle v. Google and Its Impact on APIs and Code

    Oracle v. Google and Its Impact on APIs and Code

    What, if any, impact will the Supreme Court’s rulings on Oracle v. Google have on the industry at large?

    TL/DR: It is unlikely to impact the vast majority of us, vendor and enterprise developer alike.

    Disclaimer: I am a technologist, not a lawyer. I have done some research, and talked to lawyers, but all that I say here is opinion—even my direct quotes of lawyers are their opinion. The Supreme Court can and has caught people off guard and taken cases in different directions than expected. Though I believe this path is the one it will follow, nothing is assured until the Court rules.

    Throughout this blog, references to The Case are actually using Harvard’s case document.

     

    Background

    On Nov. 15, the U.S. Supreme Court agreed to hear a dispute between Oracle and Google. The Oracle v. Google case has been going on for nearly a decade. And its roots go further back than that.

    The origins of the case go back to 2005, when Google and Sun Microsystems discussed the licensing of Java for Android. This will likely be important because it shows that Google knew it was someone else’s intellectual property (IP). More on this later.

    The crux of the case falls into three parts:

    1. Eight files that were decompiled and used. In one court case, these were found to never have made it into distribution and only used for testing (which limits liability).
    2. Two source files from Java used directly. These were found to be infringing, but damages were set to zero with consent of both legal teams.
    3. The part at issue: the copying of the entirety of Java’s interface from 37 modules. Most tech writers imply this as 37 APIs. It is not—it is hundreds of classes and thousands of methods. The 37 is the number of Java packages.

    The case has been through four court hearings/rulings so far. To sum up the relevant parts of these cases:

    1. The first case (split between a jury and district court, running at the same time) included patent violations. Those were defeated and Oracle has not appealed that defeat. On copyright infringement, the court found that Google did violate copyright on the 37 packages and the directly copied source code, but not on the eight decompiled files. The jury could not settle the issue of “fair use” of the 37 packages.
    2. The district court decided that the eight decompiled files and the copied source code for one function were violations, but remanded the rest back to the jury trial to settle the issue of fair use.
    3. In the end, the district court determined that APIs are not protected by copyright. Their decision involved two parts:
      1. That there is only one way to write a given API.
      2. The interface code and hierarchy can be replicated by third parties without violating copyright because it is an interface and not content.
    4. On appeal, the finding was reversed, and APIs were deemed copyrightable, sending the case back again.
    5. At this point, Google asked the U.S. Supreme Court to intervene. The court chose not to.
    6. Back at jury trial, the court found that while APIs may be copyrightable, Google’s use was fair use in the legal sense. This is the one topic the jury could not originally agree upon.
    7. Oracle appealed this decision, and the District Court ruled that it indeed was not fair use to copy the structure and exact interfaces.
      1. Where the jury and the district court appear to disagree is in the idea of APIs. Calling a publicly documented API is obviously fair use, and the jury latched onto that idea. The District Court appears to acknowledge the difference between calling an API and copying it.
    8. And after some tumultuous hearings and years in the court, the Supreme Court has agreed to weigh in on the issue.

    The real problem here is that copyright law is not a great fit for software. It never has been. If I rewrite your Java application in Python, using the same exact structure and naming, is that protected? All of the “words” have changed. It is akin to rewriting your Hindi book in English without a direct translation  … except that in software there is the added issue of it behaving exactly the same.

    But we can’t fix the misapplication of copyright law to software—only Congress can. So what to expect?

    A fair number of people, from the open source community and from businesses that see opportunity in a ruling that favors their world  view, are advocating that affording copyright protection to APIs is a horrible idea because it would hurt independent developers. This is not quite true. Independent developers are not generally copying Java. While this case does have bearing on coding practices and what developers can do, it does not have a huge impact on innovation, just on copying others’ work and how that is achieved—much like the “reverse engineering” court cases of the ’80s.

    There are good arguments on both sides of this case, but it seems to me that fair use will lose. They are public APIs in the sense that anyone can call them; indeed, some of these packages are so core to the system that you could argue every app must call them. But calling a function is not the same as re-creating it. In that sense, they are not in the public domain. Anyone can read a book, but that does not make it public domain.

    But no matter how this turns out, it is all good for the industry. This uncertainty has hung over the IT sector for nearly a decade. Just like when a new boss comes along and there is uncertainty in the department until everyone adjusts to the new demands/style of the boss, a solid ruling will allow settling to start. The industry can adapt once it knows what the rules are and we can move along.

    The real long-term answer is to have an equivalent to copyright and patent that is designed around the unique parts of computer code. Clearly, software is not the same beast as a book or magazine. The considerations are half-patent and half-copyright, and trying to bend those two to meet the needs of software has created some serious issues. Oracle quoted a 1995 case to showcase this problem—the company is quoting it to win the appeal, but it should be quoted as the reason the protections need to be designed for this medium that is not a publication and yet consists of words whose sequence matters: “Applying copyright law to computer programs is like assembling a jigsaw puzzle whose pieces do not quite fit.” (attributed by Oracle to the concurring opinion of J. Boudin in Lotus Dev. Corp. v. Borland Int’l, Inc., 1995).

    One interesting bit that the courts thus far have ignored, and I would love to see the Supreme Court take up, is this: If Google entered into licensing negotiations, the company knew it was Sun/Oracle’s IP. To then go and copy parts of it without licensing appears wrong on its face. It would be interesting to understand the process that ended in the copying and hear whether that bears upon the legal standing of the case. I can think of both malicious and innocent routes to get there; it would be interesting to hear testimony about motivations and thought processes.

    But even without delving into the licensing negotiations, this case stands to redefine copyright with regards to software. J. Michael Keyes, a partner at the international law firm Dorsey & Whitney, had this (and more) to say:

    “One of the issues that the court may decide is the scope of copyright protection for software. Google had early success below in convincing the trial judge that the snippets of copied code were not copyrightable. Google may urge to have the court take a look at this issue anew. Another huge implication is the scope of fair use. The jury found that Google’s use was “fair” and that it was not liable for infringement. The appellate court overturned that jury conclusion and found that Google’s use was not fair as a matter of law. The Supreme Court will likely wade into this thicket and provide guidance on the scope of fair use in the context of software code and what role an appellate court plays in reviewing jury decisions in that regard.”

    What we (all of us) do not want to do is confuse “good for the industry” with “legal standard.” It is not the court’s job to do what we think is best for the industry; it is the court’s job to determine applicable law and what that implies needs to happen. Presumably, the Supreme Court will do just that. And it may not please any of us. But if we think the current iteration of the law is flawed, that is something to take up with elected representatives, not ask a court to “fix” on the fly.

    For the vast majority of us, though, it impacts our development not at all. Using APIs or forking open source projects are all still perfectly acceptable, and no ruling by the Supreme Court is going to change that—because copying someone’s APIs is the issue here, not using them. And open source licenses allow for copying and improving projects. So carry on. Get the job done. And read about the Supreme Court’s final decision with a smile, because either way, the industry will adapt to the new normal and continue forward.

    — Don Macvittie

  • DevOps Chats: Enterprise Continuous Testing With a Shift-Right Mindset

    DevOps Chats: Enterprise Continuous Testing With a Shift-Right Mindset

    Test automation and CI/CD are evolving very rapidly to achieve great speed and impactful results by DevOps teams and Agile organizations. Our DevOps Chats guest, Tricentis Chief Product Officer Wolfgang Platz, contributes his experiences to the state of the art with his newly released book, “Enterprise Continuous Testing.”

    Automating software testing just for the sake of automation is “doing the mess for less.” What is the business value of each software element you are testing, and what is the right strategy for testing these software implemented functions? Automation certainly brings with it speed, but are we ultimately making incremental improvements to testing which doesn’t tap into the full power and benefits of automated software testing.

    Our discussion with Wolfgang focuses on the need for expanding dev-testing with higher-level integration, system, and end-to-end user experience testing, which are often performed well by shared testing and operations teams. Now, think Shift Right. While Agile and dev teams often focus on the progression of creating new and exciting capabilities for the business and our customers, we still require that regression mindset that consistently validates unit tests, valuable business functionality, system integrity, performance, and operational needs.

    Transcript

    Mitch Ashley: Hi, everyone. This is Mitch Ashley with DevOps.com and you’re listening to another DevOps Chats podcast. Today, I’m joined by Wolfgang Platz, who is Chief Product Officer and Founder at Tricentis. Our topic today is a new book that he has just released, “Enterprise Continuous Testing.” It’s a great topic, talking about agile and DevOps and how we transform testing within that environment. So, Wolfgang—welcome to DevOps Chats.

    Wolfgang Platz: Thanks, Mitch. Appreciate it. Thanks, everybody for listening to this podcast. I’m actually on it to be with you and have the chance to introduce a bit of what Tricentis is about and what this book is about, and thanks for having me.

    Ashley: Absolutely. Well, let’s jump right—and we’re honored to have you. Thanks for joining us. Why don’t you jump into, just tell us a little bit about yourself, maybe why you founded Tricentis, and then we can get into the book.

    Platz: [Laughter] Actually, that’s a great question. If you look us up on the Internet, you’re gonna see our official founding date, which is 2008, but the journey actually and the story, the history starts a bit earlier.

    I founded Tricentis as a services company first in 1997, and bringing some software QA services to the world, but I had to acknowledge, frankly, that all the tooling out there just wasn’t sufficient when it came to test automation. When you do test automation, I always call it, you run into kind of a honeymoon phenomenon. Meaning, you record your first test cases and everything looks bright, but when you try to replay them and when you want to have them up and running even when changes happen, then you fail and things break.

    So, the only way out was to come up with my own tool there, and that was the birth date of our software product offering. Then, in 2008, it was mature enough to switch into a product company, and from there on, it has been a fantastic journey upwards. Now, we are, as you may know, placed as a leader in Gartner’s magic quadrants now for the fourth time in a row, so.

    Ashley: Mm-hmm. Congratulations, yeah.

    Platz: We are about to take it all over, that’s great.

    Ashley: Well, fantastic. You know, it’s a familiar model that’s kinda transitioning, pivoting from that services into a product company—and congrats on that successful transition. It’s not always a successful one. [Laughter]

    Platz: Yeah, that’s true.

    Ashley: So, if you started in 2008, that was probably around the area of agile. DevOps wasn’t really kinda taking off quite yet. I think that’s about when it kinda started. So, you were already, I’m sure, thinking about how teams are starting to work differently in agile methodology, maybe kinda leaning, starting towards a lean kind of an approach.

    Platz: Absolutely.

    Ashley: Yeah, interesting. Well, tell us about why did you write this book. I know continuous testing, CI/CD—all of that stuff is certainly at the heart and center of DevOps and everything kinda starts there. Automated testing is, of course, a key part of that. So, certainly there is a need for this topic. Why did you decide to write a book?

    Platz: So, what I had to acknowledge in talking to customers and coming up with this product was, software testing is treated in a lot of organizations like a stepchild. And I mean, everybody is aware of the relevance of development, don’t get me wrong, and everybody is aware of the good things that IT can bring to use.

    But software testing is kind of living in the shadows, and that was one experience. And when I then talked to customers, I found out that their ask and their intuitive need is always kinda the same. They see that software testing is here, they to some degree accept that there is a need for software testing, but all of a sudden, they say, “Well, I mean, it’s a lot of effort, so we wanna have this automated.”

    Ashley: Mm-hmm.

    Platz: And the first reaction to dealing with software testing in a more professional way is, “Let’s automate that thing.” And automation is a super element aspect of an improved software testing, don’t get me wrong. However, bringing large enterprise customers up to speed on software testing is actually more like a transformation journey. It is a change agenda you need to go through.

    Because it’s not just about automation. A friend of mine said automation is just doing the mess for less, right?

    Ashley: Mm-hmm.

    Platz: But there is so much more potential in optimizing software testing when you have a more comprehensive look at it, when you have a clear understanding what is the business value of each and every functionality you wanna cover with tests. What is the concrete need for test cases in order to address the business risk that may be within certain functionality, and then what is the right strategy of executing these tests? Are you gonna run them through user interfaces? Are you gonna run them through APIs? Are you gonna run them with the use of decoupling, which service virtualization enables you to?

    Only if you have that comprehensive perspective, then you can make continuous testing really work. Otherwise, it’s just gonna be little things, little improvements that certainly help, but don’t untap the full power of a better software testing world, and that is why I’ve been coming up with this book. I think it is overdue to make people aware—“Wait a minute, it’s not just jumping about onto the automation train. This is not gonna be the full story.” And you’re gonna leave a lot of stuff behind if you just think that way. Does that make sense, Mitch?

    Ashley: It totally does, and I think if I kinda pull a couple things out of what you described, one is—automation is part of it, but continuous is also another aspect of changing the kinda whole paradigm of how you think about testing. Because I remember it wasn’t that long ago where you had a QA team and a development team.

    Platz: Correct.

    Ashley: And of course, that created automatic tension because the QA team tried to find every problem and what has to be fixed before it gets released and change control meetings and all that kinda stuff. It was highly manual, even if some of the testing was scripted. But now, in this world of continuous integration, continuous testing that’s automated you know, it’s less an argument between teams and people and that kinda friction and more about how do we make sure we’re automating the right things? How do we make sure that we escalate when problems need to be fixed, when builds break and tests fail and make sure that we’re testing the right kind of functionality?

    So, it’s engaged the developers—and tell me if you agree with this—it’s really engaged the developers much more heavily in the creation and embracing that testing rather than being a separate function someone else does. True?

    Platz: Yeah, absolutely. I mean, you pointed it out already, Mitch. In former days, it was like Dev, Test, and Ops providing different teams to software development life cycle. And what has happened, though, is that these big, we call them test centers of excellence where you would have a large number of manual testers, usually concentrated with some management capability, a little bit of automation maybe, even.

    This piece, these large TCOEs, these test centers of excellence actually, they have either vanished or they are—or everybody’s questioning themselves if they should keep going with this model. And fully right so. Because what we now see with the need for speed in development is that the cooperation between Dev and Test needs to be so much closer, needs to be so much more fluent.

    So, what we see is that these test centers of excellence are torn apart. And what we call out there as a recommendation is that you, if you have a very slim, so to say, team, a best practice team or we call it digital TCOE, that on one hand has this tight connection between Dev and Test, but on the other hand, provides some guidance on how to do testing in the best way.

    Ashley: Mm-hmm.

    Platz: And we see a lot of companies switching into Dev Testers where you have this hybrid form of developers and testers established directly within the agile teams.

    Ashley: Mm-hmm.

    Platz: And I think that’s certainly the right way to do. However, on the other hand, we wanna be very much aware that complex system landscapes require higher levels of integration testing and some flavor of an E2E user acceptance test, which tends now to be forgotten when people go into an agile and DevOps practice.

    So, I think—yes, we see the TCOEs erode. They got into Dev testers, which makes a lot of sense, but please, guys, be aware that there is a need for higher levels of testing which has not vanished overnight, and someone needs to take care of that. What we see is that these needs tend to shift more towards the operations team towards a kind of shared services team now in large enterprises, but it is a, what I would say contra streaming, like the shift left pushing testers into development. We now see kind of a shift right movement also, which makes sure that some higher levels of test are still covered in these shared services organizations.

    Ashley: Yeah, it’s interesting you pointed that out that way, because you know, you do hear a little bit about shift right and you think about the world that we’re operating and it’s much more complex, because we’re operating in cloud environments, maybe doing cloud native, maybe not, but we’re working maybe in multiclouds, private data centers, private clouds, et cetera.

    All these combinations of environments—and, you know, some apps might be in one place, some might be in multiple. There’s a lot of permutations of that environment that an Operations team is going to say, “How do I know this stuff is gonna work, and when it breaks, how do I know you’re gonna be able to fix it quickly? Diagnose it, test it, and apply the fix quickly.

    Platz: Absolutely.

    Ashley: And that automation is also the automation of—yes, unit and functional tests, but also getting into being able to do some performance regression testing, those kinda, that may be specific to that environment, maybe even specific to an iteration release or kinda backing up and reapplying changes or backing out changes.

    So, it’s in this complex environment that relying on manual processes seems like almost an impossible task. You have to have automation, you have to have a philosophy about doing testing. I hate to wax on here. Maybe I’m singing your song here, [Laughter] but that seems to be the environment we live in and that we have to figure out the best ways to do it. Am I on track here?

    Platz: No, no—that’s cool. And I’m glad that you mentioned regression, because what we have to acknowledge is that the entire mindset of agile development, per definition, is progression rather than regression. And that is good, because agile development is about pushing out new releases, pushing out new capabilities, new features, new functionality faster than ever.

    But what that mindset means is that all the agile Dev teams and the Dev testers that are within the agile teams, they also have this progressive mindset. It means that yes, if you work with them, they will accept the need for unit tests, and some of them will create great unit tests. We have seen this within our customer base. But at a certain point of time, when this is shifting towards regression, then you might find out that having unit tests set up once is one thing, but keeping unit tests up and running is a different game. And what we see is that as soon as you shift from progression into regression, you need to have people with a different mindset that really accept regression as the key aspect of their software testing behavior. If you don’t do that, you, time over time, are gonna run into issues in complex system landscapes.

    And we see that, actually, I call it the evaporation of unit tests over time, which we have done a great survey with some Swiss banks, and we found out that having unit tests set up is something you rather easily can do. But over time, with more releases, you’re gonna see that their coverage, their potential, their power is eroding. And people are then referring more to the higher levels of integration to also catch these kinds of errors going forward, which makes a lot of sense.

    But I just want to make you aware—don’t see agile development and Dev testers as the sole instance of your software testing. If you do that, you’re gonna run into issues in a complex, multi-system landscape with a high extent of regression with a heavy backbone with a true enterprise system landscape.

    Ashley: So, Wolfgang are you saying that—you know, we used to say that you had to have separate QA teams, so it was people who didn’t create the software that could more fully test it and not kinda, you know, testing your own software is hard because you assume things.

    Are you saying that, even in this automated world, continuous testing world, we still need folks who have that sort of systemic thoroughness QA kind of approach to testing, even though we’re automating it, don’t strictly rely on developers to create a self-created unit and system kinda test. Is that what you’re saying?

    Platz: Exactly. That’s what I wanna get to.

    Ashley: Good. Well, tell us a little bit about your book. You know, there are a lot of ways to approach a technical book about a technical subject. Is this a prescriptive how to? Is this kind of a strategy guide to lay out how you put together a full program of continuous enterprise testing in agile and DevOps? Tell us a little bit about the approach of how you went about this book.

    Platz: The intent of the book is to provide a comprehensive overview of what you need to think about when you go for continuous testing. As I pointed out at the very beginning of our conversation is that just going for an automation of your software tests is not gonna be the solution. You need to have a way more comprehensive look onto the subject.

    And so, what the book really is gonna do, it’s gonna introduce the perspective of a value-slash-risk based testing to you, which is the foundation. Before we wanna do a software test, we wanna make sure that it’s worthwhile doing the test, right? If this is just a functionality with very, very minor relevance to the business, you might be good with just one test case, you know what I mean?

    Ashley: Mm-hmm.

    Platz: However, if it’s an absolutely mission critical capability of your software, then you want to make sure that all the different flavors of use cases and all the different procedures that may be within this specific use case are really covered, otherwise it’s, from a risk perspective, not bearable.

    So, having a clear understanding about the business risks-slash-the business value that is associated with each and every functionality that your software provides is the foundation. If you understand that clearly, you will be able to present kind of a map, your map of where you wanna put your dollars into. And guess what? This map is not gonna be just for your budget allocation, but it’s also gonna be relevant for your reporting, because your reporting should go towards business risk covered and not just for counting test cases.

    And guess what? Your management is gonna love it. All of a sudden, they see that somebody has a plan on how budget is allocated in the software testing space. So, it’s real cool stuff.

    As soon as you know that, you wanna make sure you have the right test cases at hand. We’re gonna give you an overview about different techniques of how to create the most meaningful test cases—meaningful in terms of what is the extent of additional business risks that I can cover with the specific test case, right? And when we have done that, we’re gonna make you aware that there’s different approaches to go towards automation. You can go through a user interface, you can go through an API. When you go through this, make sure you decouple systems so that you always know exactly what is the cause of a failure.

    Ashley: Mm-hmm.

    Platz: And when you have come up with that, you wanna make sure that you keep your test data stable, meaning that the system always has a reliable set on basic data at hand so that you’re not gonna be surprised by failures of tests which are just a matter of inconsistent or incorrect test data.

    So, all these things are gonna be introduced in the book. Of course, not into the very, very level of detail, but I think at least in a way that you know what it is about and you have a good basis from where you can start digging further into stuff. But ideally, you take the book and you walk home with it and say, “Uh huh. Now I understand what it is all about.” And I’m not gonna just go out there and jump on the next open source framework for test automation and say, “This is gonna be the holy grail of my whole journey,” you know? It depends on what you wanna achieve.

    Ashley: It’s about kinda creating a systemic, prescriptive framework for how to do this kind of work, because testing is, it sounds like such an understandable word, but it’s also so overloaded, because, as you talked about, there’s lots of ways to test, whether it’s through the API, the interface, there’s security testing, there’s black box testing, and you brought up the whole topic if test data. It’s hard to acquire in the first place and you have to main that, you know, evolve it with the functionality and the application—

    Platz: Oh, yeah.

    Ashley: – part of the regression, also. It’s really a very complex topic, and you get into performance testing, load testing, sort of chaos monkey-style resilience testing, all kinds of things that can happen. And then behind the scenes, now that we’re automating so much, we have this data that’s super valuable that we can communicate to the business. We plugged in code in here and it made it into production, and here’s how we know it was thoroughly tested and made sure that it’s gonna be reliable when we get it out in production.

    Platz: Yes, and this is actually where the journey is gonna go looking forward. What we’re gonna see happening is that more and more tracing information, logging information from production will be fed back into the testing loop, right?

    Ashley: Mm-hmm.

    Platz: I’ve pointed out the business risk courage. I mean, how do you get to a clear understanding of business risk? You get through that by knowing how often systems are used and what is the potential damage. These things—and nowadays, if you have a proper logging out there—can be obtained from production logs. And these production logs can be fed into, back into the loop by now influencing your test awareness on specific functionality from the very beginning, from the development perspective already, into the higher levels of testing.

    So, what we’re gonna see is that data, the use of data, even with some artificial intelligence involved, is gonna be a tremendous source for driving the further efficiency of the testing cycle. So, we’re gonna see, actually, the ideas of the value based testing in conjunction with having the right test cases at hand being even more optimized and speeded up by this data loop that we’re gonna get into the future.

    Ashley: Mm-hmm. So, your book is available, I assume, through normal channels like, you know, an Amazon and other places like that?

    Platz: Yes, correct. Just type it in, “continuous testing,” and you may wanna put my name in there also, and then you’re gonna be good to go.

    Ashley: Awesome. Excellent. Well, congratulations on the launch of the book. That’s exciting. I appreciate all the work. I haven’t written a book, but I’ve talked to many people that have, and I know it is a painstaking love of passion for the topic that kind of sees you through to the end and getting the book published. And I know you had some help with Cynthia Dunlop, so shout out to her for helping you on the book.

    Platz: Yes, yes, yes. It actually, that was a dramatic understatement, Mitch. Cynthia has not been of just some help. I would say she is the true author of the book. I was the one laying out the concepts, and it was stunningly how little time I had to provide to give all the basic information for her to come up with it. All the kudos actually need to go to Cynthia. I’m super excited to have my name on the book, but I feel a bit ashamed because it should be her name more prominent than mine, frankly.

    Ashley: Well, it sounds like both are well deserved, and your appreciation is certainly transparent to Cynthia.

    Platz:[Laughter] I hope so.

    Ashley: And, you know, we never go alone, right? We never get there alone, so it’s through working with other that always—it gets all of us there together, so. Well, congratulations on the book and thanks for being on DevOps Chat with us today.

    Platz: It was my pleasure. Thanks, Mitch.

    Ashley: It was my honor to have you. I’d like to thank Wolfgang Platz, who is Chief Product Officer and Founder at Tricentis, and of course, wishing him the best of luck with the book. Thanks to you, all of our listeners, for joining us today. This is Mitch Ashley with DevOps.com. Have a great day and be careful out there.

    — Mitchell Ashley

  • The Road Ahead for Security, DevOps Transformation

    The Road Ahead for Security, DevOps Transformation

    Qualys hosted its Qualys Security Conference 19 this week in Las Vegas. Hundreds of attendees from around the world have gathered at the Bellagio hotel to learn more about the current state of cybersecurity and what the future has in store. As you might expect, given the technology landscape today, many of the sessions and discussions have revolved around DevOps and how cybersecurity can adapt to DevOps culture.

    Scott Crawford Qualys Security Conference
    Scott Crawford, Research Vice President at 451 Research presents “The Road Ahead for Security, IT, and DevOps” at the Qualys Security Conference

    The Shifting Technology Ecosystem

    Scott Crawford, research vice president at 451 Research, opened day two of the Qualys Security Conference with his keynote, “The Road Ahead for Security, IT, and DevOps.” Crawford talked about the dramatic changes over the last decade or so—from monolithic to microservices, from standalone applications to integrated software, from Waterfall to Agile, enterprise networks to IoT / OT, from standard networks to 5G wireless, and from traditional IT to DevOps. He discussed the challenges organizations face and the dilemma of doubling down on past success and missing out on innovation.

    With all of the changes from this shifting technology ecosystem, there are consequences. Organizations no longer have a single point of control and IT teams are faced with managing and protecting an increasingly complex web of interconnected platforms, applications and services.

    Security and DevOps

    Crawford also talked about the rise of zero trust. He described it as a sort of mashup of the concept of least privileged access, combined with behavioral analytics and an on-demand, just-in-time approach to managing access. The zero trust paradigm is based on the premise of never trust, always verify—and that means determining who is requesting access, under what conditions they’re requesting access and what actions they intend to perform with that access on a case0-by-case basis.

    He shared the Xebia Labs Periodic Table of DevOps tools to illustrate the overwhelming number of options available and stressed the importance of automation. CI/CD (continuous integration/continuous deployment), RPA (robotic process automation), and SOAR (security orchestration, automation and response) solutions have been developed to help address and manage some of the complexity of both DevOps deployments and cybersecurity.

    Xebia Labs Periodic Table of DevOps Tools
    Xebia Labs Periodic Table of DevOps Tools

    DevOps and Digital Transformation

    Crawford’s keynote was followed by a presentation titled, “The DevOps Transformation of the Qualys Platform: Lessons Learned.” Dilip Bachwani, senior vice president of engineering and cloud operations for Qualys, talked about how the company has addressed the “innovator’s dilemma” and steps it has taken to deal with growing pains as it goes through its own digital transformation.

    Bachwani explained that the engineering team at Qualys has expanded exponentially in a relatively short period. That comes with several benefits in terms of how quickly they can develop and implement new solutions, but the team also faced challenges with balancing that growth and trying to maintain velocity at the same time without sacrificing stability or reliability.

    Just like every other organization going through a DevOps transformation, Qualys has had to break down silos with people, processes and tools. Bachwani explained that they needed to do things differently to automate as many functions as possible. They decided to view infrastructure and operations as software problems and find ways to fully automate deployment and configuration.

    He also emphasized that security must be built in throughout the DevOps life cycle. Qualys adopted a hub-and-spoke DevOps model and developed a robust DevOps toolchain designed to combine and orchestrate an effective set of tools for developing, delivering and maintaining software. Having a standardized toolchain ensures that there is a common platform available to everyone.

    Accelerating Transformation With Citizen Developers

    At the end of Crawford’s keynote, he talked about the lack of people with the right knowledge and skills. Cyber Security Ventures predicts that there will be 3.5 million unfilled cybersecurity jobs by 2021. That’s just cybersecurity. What about developers and other key personnel? Crawford posed the question, “Where will we find the people to do it all?”

    The answer, he said, is citizen developers. What is a citizen developer? It’s a movement to empower employees who are not developers by trade with the tools and support to implement ideas. They are the ones closest to the problem and, arguably, best equipped to understand how to solve it. They don’t need to be professional developers or need experience with coding environments. Low-code and no-code platforms give them the power to turn ideas into solutions.

    A few years ago, George Hulme wrote an article on the growing citizen developer movement. Hulme noted, “The organizations that succeed with citizen development will be those that wholeheartedly embrace the movement and cultivate it, with IT’s help and guidance.”

    The DevOps revolution isn’t going away. Companies that don’t embrace digital transformation will likely fade into oblivion as they are left in the dust by more innovative competitors, and organizations that don’t figure out how to integrate security into the fabric of their culture will struggle against a rapidly expanding and evolving threat landscape.

    At this point, these things are like the ocean tide. It is coming whether you like it or not, and it will destroy those who are not prepared. Crawford wrapped up by asking the audience to consider what their role is going to be in building and maintaining the future of DevOps and cybersecurity.

    — Tony Bradley

  • 2019 DevOps Dozen Finalists Announced

    2019 DevOps Dozen Finalists Announced

    We are proud to announce the 2019 DevOps Dozen finalists. When we first launched five years ago, I hoped the DevOps Dozen would grow into an award that would honor the best of the best in the maturing DevOps space.

    This year, we have included three new categories: Best DevOps Practitioner of the Year, Best DevOps.com Podcast of the Year and Best DevOps Conference of the Year.

    Our finalist are definitely the most outstanding leaders in DevOps. Congratulations to each and every one of the finalists in the 12 categories. It is quite an accomplishment to make it this far into the process. Whether you’re chosen as a winner or not, you should be proud to be part of such an impressive list, and don’t get discouraged as there will be more opportunities in the future.

    Since the nomination process was opened to the public, our team has gone through thousands of votes to pick out the finalists in each category. It was not an easy task, as we couldn’t automate (DevOps) it—we had to do it by hand. Many thanks to the everyone here at DevOps.com for doing this.

    The next step in the process is for the finalists in each category to submit a presentation on why they deserve the award. Please email your final presentation to dd@devops.com by Dec 15, 2019. That information will be passed onto our panel of judges who will then select the winners. The 12 (it is the DevOps DOZEN, after all) winners will be announced in December.

    Here Are the 2019 DevOps Dozen Finalists

    1. Best DevOps Solution Provider

    2. Best DevOps Practitioner of the Year

    3. Best DevOps Solution Provider Executive

    4. Best DevOps.com Podcast of the Year

    5. Best DevOps Book/eBook of the Year

    • The Unicorn Project: A Novel about Developers, Digital Disruption, and Thriving in the Age of Data – Gene Kim
    • War and Peace and IT: Business Leadership, Technology, and Success in the Digital Age – Mark Schwartz
    • DevOps Paradox: The truth about DevOps by the people on the front line – Viktor Farcic
    • Team Topologies: Organizing Business and Technology Teams for Fast Flow – Matthew Skelton and Manuel Pais
    • Project to Product: How to Survive and Thrive in the Age of Digital Disruption with the Flow – Mik Kersten
    • Engineering DevOps: From Chaos to Continuous Improvement … and Beyond – Marc Hornbeek

    6. Top DevOps Evangelist

    7. Best DevOps Transformation

    8. Best DevOps Transformation Leader

    9. Best New DevOps Solutions Company

    10. Top DevOps Conferences

    11. Best DevOps Presentations of the Year

    12. Best DevOps.com Article of the Year

    Thanks to everyone who voted and nominated candidates. Good luck to all of the finalists!

    Stay tuned and, as always, thanks for supporting DevOps.com and the rest of the MediaOps family of communities. If you have questions, please write us DD@devops.com.  Also if you want to be included in next years DevOps Dozen awards you need to subscribe to DevOps.com and look for announcements next September.

    — Alan Shimel

  • Microsoft Extends Azure Reach to On-Premises IT Environs

    Microsoft Extends Azure Reach to On-Premises IT Environs

    Microsoft today at its Ignite 2019 conference previewed an Azure Arc management service that extends the reach of the Microsoft cloud to on-premises IT deployments.

    Azure Arc will be made available on both local Windows and Linux servers as well as Kubernetes clusters or an instance of Kubernetes that Microsoft is making available on Azure Stack for on-premises IT environments. With these announcements, Microsoft is, for the first time, extending its hybrid cloud computing strategy beyond systems running a version of the Azure operating environment on an on-premises IT environment.

    Jason Zander, executive vice president for Microsoft Azure, told conference attendees Azure Arc creates an opportunity for IT organizations to extend the best DevOps and cloud management practices employed on Azure to on-premises IT environments. Rival cloud service providers, he noted, are only paying lip service to hybrid cloud computing.

    In addition to extending the Azure management plane to on-premises IT environments, Microsoft today extended its portfolio of edge computing platforms capable of running instances of Azure Stack to include previews of commercial series and rugged series for Azure Stack Edge. Configurable with graphical processor units and field-programmable gate arrays (FPGAs) capable of running machine learning algorithms in harsh environments, the ruggedized instance of Azure Stack Edge even includes a portable server designed to be carried in a backpack.

    To complement those systems, Microsoft is also previewing a range of additional services for Azure Stack, including a fully managed, real-time data ingestion service designed to stream millions of events per second from any source to build data pipelines the same way the company does on the Azure cloud; a cloud database management service for Microsoft SQL Server and Postgres; and an Arc-enabled application programming interface (API) management platform.

    While Microsoft generally would prefer organizations to adopt Azure Stack as an on-premises extension to Azure cloud, not many organizations have the robust modern server infrastructure in place required to run Azure Stack. Azure Arc is an effort to extend the reach of Azure to multiple types of edge computing platforms without requiring organizations to deploy yet another operating system. Alternatively, organizations can employ a diverse range of Kubernetes distributions, of which there are now more than 70 that have been certified by the Cloud Native Computing Foundation (CNCF). Most organizations at this stage have yet to standardize on a single distribution of Kubernetes.

    It’s unclear what the appetite is for extending the management plane Microsoft created for its Azure cloud to on-premises environments. However, as the scope of the hybrid cloud management challenge facing organizations grows as various new forms of edge computing manifest, Microsoft clearly sees an opportunity to increase the scope and reach of its cloud platform strategy. Each edge computing instance is essentially a smaller instance of a cloud platform for which most IT organizations will not have the tools required to manage at scale. In fact, many IT organizations might not even attempt to deploy workloads at all on those platforms without something equivalent to Azure Arc to manage and secure those environment first being in place.

    — Mike Vizard

  • Privileged Access Management at the Speed of DevOps

    Privileged Access Management at the Speed of DevOps

    Privileged access management (PAM) is clearly a very important element of effective cybersecurity. PAM has been ranked number one on Gartner’s Top 10 Security Projects for two consecutive years, and it is number four on the CIS Top 20 Critical Security Controls. The struggle for many organizations, however, is they have also embraced DevOps culture, and the traditional approach to PAM clashes with the principles of a DevOps environment. Thankfully, it is possible to effectively manage privileged access in a DevOps environment—it just takes a new approach.

    Challenges of Privileged Access Management

    Despite advances in identity and access management and the existence of things like two factor or multi-factor authentication and biometric authentication, most organizations still rely on a simple username and password to authenticate an individual and grant access to network resources, applications and data.

    The concept of least privileged access—ensuring that users only have access to things they really need access to—has been preached since the dawn of cybersecurity and, when done right, can be an effective strategy. The problem is it’s tedious to do right and it’s very easy to lose track of who has privileged access to what.

    That’s where a PAM solution comes in. The whole point of privileged access management is to provide a way to track and manage who has access to what—to grant access when it is needed and revoke access when the need no longer exists. Unfortunately, there is a lot of confusion about PAM and the effort to manage privileged access is challenging for many organizations.

    Enterprise Management Associates (EMA) conducted a survey on the state of privileged access management. The survey included responses from companies representing different company sizes and industries around the world and focused on IT and cybersecurity professionals—most of whom are familiar with PAM and are directly responsible for managing or granting privileged access to users.

    There were some concerning findings from the survey regarding the perception of PAM. Twenty percent of organizations that reported using a dedicated PAM solution believe it somewhat or greatly reduces user productivity. Many respondents also expressed frustration about the amount of time required to manage privileged access. Half of the survey participants cited manually granting temporary privileges as either very (29%) or extremely (21%) time consuming, and half also said manually revoking temporary privileged access as very (33%) or extremely (17%) time consuming.

    PAM and DevOps

    As challenging as traditional PAM solutions might be for businesses around the world, they pose an even greater hurdle for organizations that have embraced DevOps. A recent episode of the Defense in Depth podcast, hosted by David Spark and Allan Alford, focused on some of the challenges of PAM. A listener had written to complain that “the traditional approach to PAM breaks everything that is DevOps. Regardless of what your PAM provider says about their ability to scale with your dev cycles, they’re lying about the real level of effort.”

    That seems like a fair assessment. Pretty much every company is somewhere on the cloud journey—and working within a DevOps framework on some level. Companies often face backlash from the DevOps team—with some going so far as to boycott efforts to introduce security tools that can’t keep up with the pace of DevOps in the cloud.

    DevOps developers express frustration about trying to build an automatic scaling, horizontal infrastructure and being dragged down by demands from the security team to install agents and change authentication modules. They complain it often means rewriting all of the scripts, resulting in significant and unnecessary effort for a solution that ultimately can’t scale properly in the DevOps environment.

    That is, unfortunately, true when it comes to traditional PAM solutions. Next generation PAM solutions, however, are built with the cloud and DevOps in mind. They are lightweight, automated, scalable and capable of integrating seamlessly with common tools and platforms.

    Rethinking Privileged Access Management

    Based on the survey, organizations understand the importance of privileged access management, but they struggle with finding a way to implement it effectively. The question from the podcast illustrates how PAM is also a significant issue for DevOps environments because traditional PAM solutions do not integrate or scale effectively.

    At the same time, we all know that PAM is a crucial element of effective cybersecurity. It’s not acceptable to just not manage privileged access, but it also doesn’t make sense to suffer through the headaches and frustration introduced by traditional solutions. Organizations need to find new privileged access management tools that work in the cloud and can scale automatically to provide PAM at the speed of DevOps.

    — Tony Bradley

  • Survey Suggests DevOps Drives Digital Business Transformation

    Survey Suggests DevOps Drives Digital Business Transformation

    A survey published this week by New Relic, a provider of IT monitoring services, suggests there is a high correlation between organizations that have embarked on digital business transformation initiatives and adoption of best DevOps practices.

    Revealed at the Futurestack conference hosted by New Relic, the survey 750 global senior IT decision-makers of enterprises with 500 to 5,000-plus employees in Australia, France, Germany, UK and the U.S. conducted by the market research firm Vanson Bourne finds roughly 50% of organizations in the U.S. have completed or are close to completing their digital transformation journeys.

    The same survey finds nearly two-thirds of U.S. respondents in the region claim they’ve adopted DevOps practices.
    Buddy Brewer, group vice president and general manager for client side analytics at New Relic, said the challenge most organizations face in terms of DevOps is they lack access to real-time analytics. DevOps teams may have access a raft of metrics, but all the data they have only tells them how many times something occurs on average. Roughly half of end users may be having a satisfactory application experience, but the half that are not are the company’s most important customers, noted Brewer.

    Overall, the survey finds that that there’s nothing easy about modernizing IT environments to drive digital transformation initiatives. Nearly three-quarters (72%) expect the technology team to deliver more innovations and updates. However, more than 50% said they find their complex new software and infrastructure difficult to manage and monitor for performance issues.

    Nearly half the respondents (48%) also admitted end users or customers are likely to tell them about a problem before they know about it and 46% said they are told about these issues before they know how to fix them.

    At the same time, 79% said the rest of the business has higher expectations in how digital systems perform and 63% admitted that because of pressure to respond to business needs, they work longer hours to observe and manage software performance. Almost half (46%) said C-suite executives want daily updates about how software systems are performing and 40% said CEOs also want more answers when outages or performance problems happen.

    More than half (56%) of all respondents acknowledged that it is impossible to properly assess all the machine data being collected. Just as troubling, a third said they are challenged on business benefit metrics for their digital transformation projects.

    The survey identifies the top five digital transformation challenges:

    1. Separate parts of the organization are moving at different speeds to embrace digital transformation.
    2. Shortage of skilled employees.
    3. Restricted budgets.
    4. Understanding and measuring business benefits.
    5. Continued resistance to shutting down legacy systems.

    As always in the case of IT professionals, however, hope springs eternal. A full 89% said they believe artificial intelligence and machine learning algorithms will become important element for managing digital systems. That may indeed come about one day. However, until that day arrives, DevOps teams should count on the fact that every digital business transformation project that gets launched equates to a lot more hard work for them.

    — Mike Vizard

  • Kubernetes and Service Mesh With StackRox

    Kubernetes and Service Mesh With StackRox

    DevOps and Kubernetes enable organizations to “go with security,” adding security to the code instead of on top of it. In other words, security is built-in rather than bolted on.

    This is one of the aspects of security in the DevOps world with a Kubernetes system that Ali Golshan, co-founder and CTO of StackRox, explained during this exclusive interview with Alan Shimel, Media Ops CEO and editor in chief. The other aspect is security has become continuous instead of snapshots, which provides “a lot more visibility into various stages, various deployments, as well as running environments,” he said.

    During the interview, he explored containers, orchestrations, Kubernetes and service mesh.

    “Leveraging this cloud-native infrastructure, Kube and service mesh the right way reduces your dependency on third-party tools, allows you to hand off things you typically handed off to vendors and does that inside your infrastructure, which makes it actually more secure and highly scalable,” said Golshan. “You can automate that and reduce your lock-in and your sort of dependency.”

    Check out the interview below to learn more about the best and emerging practices in Kubernetes, service mesh and the security feedback loop.

    — Veronica Haggar

  • The Relationship Between Feature Flags and DevOps

    The Relationship Between Feature Flags and DevOps

    From 2017 I’ve started most DevOps conversations with, “DevOps is now boring, and I love it!” What I mean by boring is that we are arguing less and less about why you should embrace DevOps principles and practices and having more and more conversations about how to implement DevOps in the real world. As a result, we are getting even more specific about various implementations. One of those is the use of feature flags to increase application velocity, ability to roll back and maybe even canary releases.

    Feature flags seem obvious, but are still a relatively new endeavor—and, for now, still pretty much isolated to the unicorn dev teams. Feature flags allow developers to logically group features and create a toggle for them that can be switched in testing and even prod. The benefits of feature flags go beyond just getting features out faster:

    1. Launching complex feature sets incrementally versus in bi-weekly sprints (fast waterfall). Sometimes features are more complex and connected to multiple services, back-end changes, etc. With modern microservices-based architectures where small dev teams operate on code bases that are independent, this can be difficult. By grouping all the various changes across the entire application as a feature, it means that teams can release functionality they have built without waiting for everyone else and only after everything is in place to turn on the feature.
    2. Recover from risky releases. Part of having feature flags is you can release functionality to production faster. But this also means that you can release functionality that has not been vetted fully across the entire application. With feature flags, an initial rollback does not need to be related to infrastructure at all; it can just be a matter of toggling a feature in production.
    3. Create a value-based information architecture for your application. Developers still are very guilty of thinking of the application as their code base and their immediate teams. But the application from the user’s perspective is based on value. Feature flags, assuming you use some basic best practices for determining your flag structure and names, can group features by value and thus better align everything in the delivery chain to the value the application provides.
    4. Quality (QA and Q)  teams have more power and can provide more value with feature flags. With feature flags, developers can throw more functionality over the fence to QE faster and enable them to have more control of functionality being tested and released without engaging with Dev.

    I would say the initial driver for feature flags has been for A/B testing and canary releases. However, the reality of A/B testing and canary releases don’t make sense for everyone because their application may not have enough users or transaction volume to be statistically accurate to see the value.

    But the value of feature flags goes well beyond that. The challenge is, if you are not a DevOps and automation-focused organization, these benefits are not going to be there. Feature flags are an outcome to DevOps, but my hope is they also are a driver for DevOps. Organizations are guilty of looking at DevOps from the process improvement perspective only. But the real business benefit of DevOps is the value you provide your application users. And if you have a value-first mindset, not only are feature flags an obvious design pattern but also they necessitate that organizations think about their delivery chain in a highly efficient and critical way. Put another way, if feature flags are an outcome you seek, then DevOps processes are not a choice.

    — Chris Riley