Tag: testing

  • 4 Mobile DevOps Must-Haves

    4 Mobile DevOps Must-Haves

    Mobile DevOps requires a robust and streamlined strategy for success

    The mobile revolution of the last 20 years has radically changed our everyday lives. That’s not just the way we communicate, but also the way we find information, purchase goods and services, entertain ourselves and more. This has gone in tandem with great technological advances elsewhere, including small-business VoIP. Now it’s never been easier to organize a business teleconference or simply catch up with friends and family, wherever they happen to be.

    Recent years have seen mobile apps scale new heights of technological achievement. This plethora of new apps has changed the way we work, play and chat. But that’s not all. It’s also changed people’s expectations—and it’s up to businesses to respond to that. Consumers expect apps tailored to suit the different aspects of their lives. This puts a great deal of pressure on businesses and other organizations to meet these higher and more demanding expectations.

    That’s why so many are taking advantage of the opportunities offered by mobile DevOps. In a nutshell, the purpose of DevOps is to facilitate enhanced collaboration between project managers, mobile developers and ordinary employees. This helps businesses to meet their goals more effectively, while also ensuring that users enjoy the best and most convenient experience possible. It also means that businesses don’t have to go through the hassle of using native apps to create new hybrid products, greatly simplifying their tasks.

    But without a robust and streamlined strategy in place, mobile DevOps won’t automatically deliver the goods. You need to have a solid understanding of a range of different factors. These will include user preferences, development trends and software delivery. One thing that is clear is that consumer expectations are changing at a remarkable rate and you need to ensure that your offering is up to par. Agility is the watchword, and your DevOps strategy needs to have this flexibility built-in.

    Elements for Successful Mobile DevOps

    Here we’ll take a look at four must-haves of any streamlined mobile DevOps strategy to help you get started. Read on to find out more.

    Planning

    It might seem like an obvious point to make, but thorough planning is the indispensable foundation of any effective mobile DevOps strategy. Having said that, you’d be surprised just how many firms fail to take this basic requirement properly into account. It doesn’t matter exactly what your specialism is—whether it’s HR software or telemedicine app development—your project must be planned in fine detail.

    Get everyone who’s working on the project together. That means all stakeholders, from your developers and testers to your operations team and others. It’s then that you can start to map out what form your project will take. It’s a good idea to get clients involved at this early stage as well, because it’s imperative to keep their needs in mind at all times. Think about what everyone requires from the project (including quality control requirements) and how to deliver it. Consider the potential risks involved and what steps you might take to mitigate these.

    People and Communication

    It’s the same with any project—unless you’ve got the right people on board and the right chemistry between them, you’re almost certainly doomed to fail. Your team, therefore, is of the utmost importance. Mobile DevOps inevitably involves a wide array of different people with unique skill sets, who will make their own particular contributions to the project. You should prioritize finding the right mix and maintaining morale across the board. This will go a long way to ensuring harmonious working between the different wings of your team.

    Part of keeping team morale high is making sure that communications between different parts of the team are smooth. DevOps projects, as we’ve noted, tend to have quite a large number of people working on them. There is a danger that different teams can become somewhat siloed. That’s where each smaller element of the team fails to work coherently with those around it. You should, therefore, make a concerted effort to keep the different groups working on your project collaborating closely with one another. In this way, your people are sure to add up to more than the mere sum of their parts.

    Testing, Monitoring and Delivery

    Continual testing, monitoring and delivery are all essential to ensure that deployment is smooth and free of avoidable errors. Every build should be subject to rigorous testing so that any snags can be discovered and ironed out. Monitoring must be similarly thorough at each step of the way. This helps to keep the production process stable so that any unexpected hitches can be dealt with more effectively when they arise. Continual testing and updating are more important than ever, given the advent of the internet of things and its always-on nature.

    It’s also important to remember that speed is of the essence when it comes to mobile DevOps. The more effective your testing, monitoring and delivery processes are, the quicker you’ll be able to roll out new apps—in other words, increasing software velocity—and you’ll be able to do so with confidence that they’ll be reliable, user-friendly and secure. This will help to improve the overall efficiency of the development process, as well as boosting user and client satisfaction. It will also improve return on investment while helping to minimize the risk of cost overruns.

    Quality and Efficiency

    Striking the right balance between quality and efficiency in mobile DevOps may be top of mind for you. We’ve already alluded to the importance of efficiency in DevOps processes. Maximizing quality in the development of new apps can also do a lot to improve efficiency, save money and cut out unnecessary expenses.

    After all, there’s no getting away from it: development can be an expensive business. A laser-like focus on quality can prevent avoidable complications in the development process. Crucially, it also helps to maintain user satisfaction (always a vital consideration). Think about telemedicine app development, for example—a particularly fraught area. A telemedicine app that’s unreliable, prone to errors or generally shoddy could have a devastating impact on a company’s reputation. Robust DevOps tooling not only allows for better quality but also enhances efficiency by freeing up developers and testers to work on other tasks.

  • Micro Focus Updates LoadRunner Testing Tools

    Micro Focus Updates LoadRunner Testing Tools

    Micro Focus has revamped its venerable line of LoadRunner testing tools as part of an effort to both unify code bases and provide tighter integration with third-party application performance management (APM) tools from AppDynamics, a unit of Cisco Systems.

    Vicky Giavelli, director of product management for performance engineering at Micro Focus, said Micro Focus in now in the first stages of unifying the code base of instances of LoadRunner that run on-premises and the cloud to enable DevOps teams to more flexibly test applications regardless of where they are deployed.

    DevOps teams can already share licenses across LoadRunner products and rapidly scale up or down based on seasonal or ad hoc demands, she noted.

    As testing continues to shift left, Giavelli said, Micro Focus continues to invest in aligning the LoadRunner family of application testing tools with multiple DevOps platforms. LoadRunner tools currently support more than 50 application protocols and technologies and more than 52 scripting technologies in addition to providing integrations with open source continuous integration/continuous delivery (CI/CD) tools and data visualization platforms such as Grafana and InfluxDB.

    The alliance with AppDynamics extends that strategy by making it easier for DevOps teams that employ LoadRunner to consume and share data with users of one of the most widely employed APM platforms, she noted.

    Giavelli said Micro Focus continues to enhance the user experience of a testing platform that has been widely employed now for a quarter-century. While competition has increased considerably during that time, the LoadRunner family of tools has maintained its base among application testers, she said, while extending its appeal to developers, who increasingly are taking responsibility for testing as part of an integrated set of DevOps processes.

    Of course, it’s still not clear how far left toward developers application testing might shift. While the consensus might be that more testing on the part of developers is good, application testers may provide a more unbiased opinion about how well an application accomplishes the ultimate business goal.

    Regardless of the role of testing within the organization, the one thing that everyone agrees on is it should be easier to generate meaningful tests that reflect real-world use. No matter who does the testing, the goal is to not have to roll back an application once it is deployed in a production environment. As applications become more complex, Micro Focus continues to invest in machine learning algorithms and other forms to advanced analytics to identify potential issues sooner than later, noted Giavelli.

    It’s too early to say how the battle between application testing vendors will play out in the months ahead. The one thing that is certain is that in the wake of the COVID-19 pandemic, application testing has never been more critical. More organizations than ever are building digital business applications to make it easier to engage with their customers, at a time when many of them might not be returning to an office anytime soon. The issue, of course, is that not many of their end users will be especially tolerant of issues such as a mobile computing application that doesn’t live up to expectations the first time it’s deployed.

  • GitLab Survey Sees DevOps Gains as Developers Exercise More Control

    GitLab Survey Sees DevOps Gains as Developers Exercise More Control

    A global survey of 3,650 software professionals published by GitLab respondents indicates developers are exercising more direct control over IT environments.

    More than a third of respondents (35%) said developers define and/or create the infrastructure their applications run on, with 14% actually monitoring and responding to infrastructure events. Additionally, 18% of developers instrument code for production monitoring, while 12% serve as an escalation point when there are incidents.

    More than 25% of developers reported feeling solely responsible for security, while 33% of security team members say they own security. A total of 29% said they believe everyone should be responsible for security.

    A quarter of respondents said their organizations have been practicing DevOps for three to five years, while 37% said they have been practicing for one to three years. Nearly 60% of respondents said their organizations deploy code either multiple times a day, once a day or once every few days, a 15% increase over the previous year.

    Brendan O’Leary, senior developer evangelist for GitLab, said that massive jump in the number of organizations capable of deploying code at faster rates indicates many more organizations are starting to move further along the DevOps maturity curve. For example, the survey finds 70% of developers can now provision their own IT environments.

    Well over a third of respondents (38%) said their DevOps implementations include a continuous integration/continuous delivery (CI/CD) platform. Almost 59% of those surveyed use GitLab, while 23% use GitHub and 11% use BitBucket. For builds, 60% use GitLab and 38% use Jenkins. A total of 29% said their DevOps implementations include test automation, while 16% said they have embraced DevSecOps.

    Almost 40% of survey takers said they “partially” use microservices, while a quarter (26%) said they fully use them. Well over a third (38%) have also adopted Kubernetes.

    Despite that progress, however, the survey makes it clear there is still much work to be done. More than 42% said testing still happens too late in the life cycle. In fact, 47% identified testing as the top DevOps bottleneck for the second year in a row. Only 12% said they have completely automated testing.

    When it comes to security, well over a third (36%) find it hard to understand, process and fix any discovered vulnerabilities. About a third (31%) said prioritizing vulnerability remediation is still an uphill battle.

    O’Leary said when it comes to security the most challenging issue remains determining what vulnerabilities to prioritize alongside a long list of other bugs and issues that often need to be addressed. Cybersecurity teams all too often continue to send developers a list of vulnerabilities they have discovered with no context, he noted. Many developers are only made aware of an issue after their code has been merged, which O’Leary said makes it more difficult for them to address a specific security issue.

    It’s not clear any organization will ever achieve complete DevOps nirvana. However, as organizations become more proficient it’s clear the return on investment in DevOps is starting to be achieved as more organizations deliver code faster. The challenge now, of course, is making sure the right secure code is delivered at the right time.

  • EP 1: A Brief History of Testing

    EP 1: A Brief History of Testing

    DevOps’ sister site Digital Anarchist has launched a bi-weekly video series and companion monthly roundtable on DevOps, Automation, CI/CD and testing. Featuring leaders in all of these areas, we explore the challenges and issues software delivery and IT teams face every day. How do we go faster, smarter with better quality? DevOps Unbound!

    The video is immediately below, followed by the transcript of our conversation. Enjoy!

    Transcript

    Alan Shimel: Hey everyone. Welcome to the very first episode of DevOps Unbound. I’m your host, Alan Shimel and joining us for this initial episode of DevOps Unbound, I’m proud and happy to introduce you to James Bach and Grigori Melnik. Grigori, James, welcome to DevOps Unbound.

    Grigori Melnik: Hey everyone.

    Shimel: Before we jump into things, let me give each of you an opportunity to kind of introduce yourselves, your backgrounds, your positions, etcetera, so that our audience kind of knows a little bit, if they don’t already, who you both are. So, Grigori, you’ve already been a guest in our show, so if you don’t mind, I’m going to ask James to go first. James? Introduce yourself. Go ahead.

    James Bach: My name is James Bach. I’m a consulting software tester. I started as a developer in my teen years after I dropped out of high school. I’m the proudest high school dropout you will ever meet. I later wrote a book about dropping out of high school and my particular self-education method, but my big opportunity came when Apple computer hired me to be a test manager, and at the time, I was the youngest manager in the R&D division of Apple computers.

    So that’s kind of something I’m proud of. And I fell in love with testing and I decided to devote my career to testing. I studied the epistemology of testing, the mathematics of testing, the sociology of testing. I am a bit obsessed with testing.

    Shimel: To say the least.

    Bach: And I have a book about that called “Lessons Learned in Software Testing” with a couple of coauthors. And now I teach and consult.

    Shimel: Excellent. And you know what, high school dropout or not, you’ve done pretty darn good, so congratulations and not saying it’s an example for our younger people in the audience to necessarily emulate. Of course, James grew up –

    Bach: My son did better than me. He never went to high school, so I’m –

    Shimel: Really?

    Bach: He’s already outdone me.

    Shimel: Absolutely. Very good. Grigori? Why don’t you tell them a little bit about yourself?

    testingMelnik: Sure. So I’m Grigori Melnik and I am the chief product officer of Tricentis, focusing on building the testing tooling of the future, testing tools of the future. Previously, I guess I spent almost two and a half decades focusing on the developer audience and building all kinds of platforms, Microsoft and Splunk and MongoDB and you know, always been kind of customer obsessed in the sense of developer obsessed quite a bit, making sure that whatever we were focusing on would help drive the productivity and alleviate some of the pains that developers in our target audience had.

    And while a few of those things that I’m particularly proud of, my team at Microsoft was actually the first one that built the very first auto-scaling for what was back known as Windows Azure, now Microsoft Azure, but also focused on common patterns and practices in implementing the crosscutting components that represent crosscutting concerns that later would make into the .NET framework.

    So again, a lot of work identifying those patterns, capturing them, externalizing them, testing them out and then helping drive their use. When it comes to testing, I first kind of discovered it in my days in academia when I was doing research and teaching at university, and it was quite interesting because testing was predominantly viewed through the lens of formal method groups. It wasn’t really viewed as an intellectual activity of discovery, as an intellectual activity of exploring or experiencing something.

    And those were actually the days when I’ve encountered the group of testers from the context driven school of testing, including James and then we kind of kept in touch ever since, even with me gravitating away from the testing more into the developer world and now doing like the full loop of coming back to testing and I have to tell you, I felt, when the opportunity came, I felt that the world of testing, at least from the perspective of how the discipline itself has advanced would be way further developed in the last decade than I found after kind of leaving it for a while.

    So I’m now on a mission of kind of reigniting that whole discipline and putting it more on a pedestal and giving it more view and empowerment and leveling up all around. So, yeah.

    Shimel: Excellent. So, thank you both, gentlemen. And I’m Alan Shimel, I’m the editor in chief, CEO for MediaOps, the folks behind devopsy.kinsta.cloud, Container Journal, Security Boulevard, TechStrong TV, et cetera. And just quickly before we jump into today’s agenda, DevOps Unbound though, today’s focus is on testing. It’s not just about testing. It’s really about DevOps kind of unbound, setting us free, being able to leap tall buildings and bound into space, sort of freeing ourselves of gravity, whatever you want. It’s about speed. It’s about acceleration. Right? It’s about automation in a lot of ways.

    And each – we’re going got be doing these every other week and each episode will focus on particular aspects of this unbound nature, if you will. Today’s is testing. And then once a month, we’re also going to have a roundtable with about six or eight people of Grigori and James’ stature and we’ll open it up to the public for live audience questions and discussion. So I’m really looking forward to that. DevOps Unbound is sponsored by our friends at Tricentis and we thank them for it and with that out of the way, let’s jump into things.

    So, Grigori, I wanted to start off – obviously, you and James take very different paths to our – and both of you have been very successful in your lives and careers. So it just goes to show, there’s more than one way to skin the cat. Right?

    But, let’s take a look – look, I’m probably older than both of you, though not that much I’m sure, but let’s look at the history, the historical context of where you guys come from to be sitting here today and how that, you know, vis-à-vis kind of the whole testing industry, the whole testing, you want to call it a profession, the guild, the whatever you want to call it, right, how has this evolved over the years and let’s put some context in with timeframes. James, when you first went to Apple without saying you’re old, what kind of – what were the timeframes for that?

    Bach: Well, I am old.

    Shimel: So am I.

    Bach: But you’re older, which is nice. It’s nice to be a young guy in the room again.

    Shimel: Uh-huh. I can’t tell you how well that makes me feel, but go ahead.

    Bach: First, 1987, May 21, 1987 was the first day that I ever professionally tested something as a tester. Before that, I was doing videogame development and I had done informal testing, and I hated testing because testing just got in the way of me shipping my product. So, I didn’t like testing until it became my job and then I realized I’m perfect for testing ’cause I love complaining. And I’m much more interested in finding how things don’t work than to figure out how to make them work.

    Shimel: That’s interesting. You know, my background’s in cyber, back then we called it InfoSec, and in InfoSec, most of the people got into it ’cause they liked to break things. Right? Yeah. Same thing. Grigori, how does that timeframe-wise and everything else, how does that match up with your own experience?

    Melnik: So I like to actually build things. _____ you guys who are breaking things. And occasionally I break things and actually this also speaks about this overall I guess optimism of most developers who think that what they write is perfect and always underestimate the amount of buggy code that we put in there.

    But to me, if I think about my first day when I actually had my job as an engineer, actually, it was September, I think 5, 1990, so a few years after James and at that time, I was actually writing a C-library for a PC converting the data from mainframe into the visual format and I had to deal with all kinds of quirks of Fortran and the 80 characters limit and the asterisks for, you know, for bringing – the continuation of the line, all kinds of exciting stuff. But back to the beginning, to make sure that I don’t go too far on the tangent, we actually didn’t have, for a very, very long time – again, I’m giving you my angle into the testing, if you will.

    We actually didn’t have a dedicated thinking about testing as an activity, as a pursuit of excellence. It was just something that you kind of were doing and then at some point later, you’ve heard and learned that oh wait, look, there’s a QA department, apparently, right? And they’re the ones that were mostly viewed as gatekeepers or police of some sort. Right?

    And then again, if I think about again, once I got more into the research and academia and studying testing specifically because, you know, I’m probably one of the very few people who has testing in the title of my PhD dissertation. But, I remember looking back and thinking, “But where does this come from?” Right? Where did actually software engineering come from?” Right? And of course, this is where the history of the conference deck, you know, after the _____ _____ where all the stalwarts of at that time computer science and well, not even computer science, mathematics departments because that’s who were doing all the programming back then at the point the software engineer discipline and later on the association of debugging, right?

    It was the infamous story of how the moth was found in Harvard University’s scientific lab and the term bugging, you know, was coined. I almost think like the – in the early days, it was very much associated with this era of debugging. Right? Then later on, the era of the ability to demo, the fact that the software was matching the stack or the requirements, I think it almost feels like there were at least a decade when people were fixated and some would argue that it still continues.

    Shimel: What decade was that?

    Melnik: Hmm?

    Shimel: What decade are you referring to?

    Melnik: The debugging or the – the debugging I’d say –

    Shimel: Are you talking about the 70s? Are you talking about the 60s? Are you talking about the 50s?

    Melnik: I’m talking about 50s and 60s. I’m talking about the 50s and 60s as almost the era of debugging the way I viewed it. Then, after that, the era of more like 70s and 80s where you were focusing more on matching the spec and demo and that the software would actually approve it. And this is where this big, big, big focus was on formal methods and trying to even mathematically prove that a software worked according to the spec.

    And then after that, I feel like the next era was the era of breaking or, you know, something that probably is still continuing. And again, all these elements I feel are present in today’s profession of testing, but the later years, the last 20 years was everything that I was kind of studying, watching from the sidelines, getting myself into. I like to call it the era of enlightenment, if I may. I know that’s super pompous of me, but this is where I view more testing in the way how James and other members of the context drew in school of testing had formulated it which is, you know, lighting the way for the decision-maker, not for the developer necessarily, not for – again, it could be for the developer, but somebody who at the end of the day makes the decision whether what is being produced is acceptable or not, or if it’s good enough to ship. Is it, you know, what risks associated there and informing me as a decision maker about all these different factors and then leaving that decision up to me.

    So the testers are not the police. They’re not the quality gate. They’re not _____ anything. It’s really up to the stakeholder who is commissioning all this effort and who makes the ultimate decision about releasing something, putting something into production.

    Bach: I’d like to offer some historical notes on this.

    Melnik: Please.

    Bach: Okay, first, are you aware what the first book was on the subject of software testing, the first known book? A little quiz for your Grigori?

    Melnik: Is it the Myers book?

    Bach: It’s before that. It’s Program Test Methods from 1972 by Bill Hetzel. Now that book was a report on a conference a testing conference which was the Chapel Hill conference, which happened in ’72. That seems to have been a historical watershed event in testing. Before that, the first chapter on software testing that I’ve been able to find was written in 1961 by my teacher, Jerry Weinberg, who wrote it when he was 25 years old. It’s a brilliant chapter on testing.

    What’s fascinating to me is how differently Weinberg talks about testing in 1961 compared to how they talked about testing in 1972, although there is a reference in the 1972 book back to Weinberg, praising Weinberg’s attitude about testing. It’s fascinating to read that because the book in 1961, it’s Computer Programming Fundamentals and the chapter about testing was written by Jerry, he told, although it said it’s co-written, the book’s co-written with Herbert Leeds and Jerry Weinberg, but Jerry wrote the chapter on testing.

    And Jerry talked about testing as a matter of imagination, not automation, as a matter of being suspicious and critical, as a fundamentally unalgorithmic process. And he develops this beautiful story in his chapter about a programmer who’s sure that he’s found the last bug and then finds out that he’s wrong. And then he adds another check for that.

    And then he’s sure now he’s found the last bug and then he sees something else go wrong. And Jerry’s conclusion is look, this is just, we’re working on complex systems and this is always going to be like this. Now, Jerry was on the first project in 1958 that ever had a dedicated test team.

    Now, the reason why he knows that is that in 1961, IBM held a conference of all the computer programmers in the entire world of which there were 400, apparently, and Jerry said his team went to that conference. He was at IBM at the time, and he said everyone thought they were crazy ’cause they had a dedicated group of testers and no one else did.

    They all were just testing their own stuff. So it seemed like the idea of having a dedicated test team caught on and then in 1972, the way they talked about testing was all about test cases, automation, formality and that kicked off the era, I think, of the factory, the test factory. We can have technology and we can have procedures that make it so you don’t rely on human thinking.

    So this goes against what Jerry Weinberg was preaching, not only back in ’61, but all through his career, I became a student of Jerry’s in around 1989 and took all his classes in the mid-nineties and he became a mentor to me. That’s why I’m a real acolyte of Jerry Weinberg, because he always put the human in the center of everything. But, testing – when I joined testing, all the testing textbooks were about how you have to define test cases and if possible, automate them.

    And I felt like that was a fool’s idea. I think, well, the problem is is that there’s too many things to define and far from being unbound, we’ve become super bound by all of these documents and all of this code. And so if you want to be unbound, we have to find a way to temper that. We have to find a way to interact with our tools so that we’re not constrained too much by them. And that, in the nineties, the urge to unbind ourselves from that waterfall world and that heavy documentation world created agile, but it also created the context driven school of software testing, which is a humanist approach to software testing.

    And that was in the mid-nineties that we started creating that. And then DevOps and agile continued to sort of take over the world except that it seems to me that agile in a lot of places has lost it’s humanist foundation and it’s turned into tools, tools, tools. And so if we’re going to really talk about Unbound, DevOps Unbound, it seems to me that we have to be specific about what is it we’re unbinding. I hope what this means is it’s people being unbound, using DevOps as a tool rather than –

    Shimel: But not just a tool. I’m sorry. I didn’t mean to jump in. But not just – you know what, James, we don’t want to make the same mistake that Agile made. And you just said, one of the problems with Agile is we went from humanist to tools, right, too much of a reliance on tools. Let’s not make the same mistake in DevOps. I think that’s something I’ve advocate for for a while too is we can’t lose the human in DevOps.

    Bach: Actually, I want to hear you say more about that, Alan, if you’re willing to say a little more about that.

    Shimel: I’m not shy. I’ll give you my two cents. Look, I started devopsy.kinsta.cloud. I cover this place. I cover this area pretty heavily and I have for seven or eight years now. And I think there’s been this – and I don’t want to say agile failed ’cause agile is far from a failure. But there has been a reliance on tools and maybe overly processed. When we think about things like ITIL and ITSM versus DevOps, let’s say, right, we get in – we tend to point fingers at ITIL. Excuse me and ITSM _____ too much process, too heavy, too much of that, that whole change management thing.

    But there are good things there. Nothing’s all bad and all good. The world’s not black and white. It’s grey. Right? And so there are things we take out of ITIL and things and ITSM that we put into DevOps. Btu DevOps, to me, anyway, and it’s just my humble opinion, is fundamentally about people and teams and culture, not the tools- there’s a lot of tools we could use and there’s overlap in tools and what’s the best tool for any given job.

    We could fight about it till the cows come home, but at it’s nitty-gritty, DevOps is making sure that devs and ops and tests and security and these people work with a higher degree of communication, right, with a higher degree of cooperation getting jobs done faster. And we do that by adding things like automation to the mix. But automation for automation’s sake alone is not enough. Right? And to do automation, yes, you’ve got to use tools sometimes to make that automation work. I get it. But if we lose the fundamental aspect that DevOps is, at its core, about humans, I think we all – we lose what DevOps is and I don’t hold myself out to be as accomplished as either one of you, but I –

    Bach: I still love you, Alan. I love what you just said. I think this is – if you can hold on to what the principle and the feeling behind what you just said, we can do anything with tools and we’ll be okay. We’ll be okay. If we’re talking to each other, if we’re caring about each other, I keep seeing people losing that and of course, any technology and any methodology, when it’s applied sociopathically, will become abuse. And so we just – those of us who can, shall do – should do whatever we can do keep it non-abusive and keep us talking. And then I think DevOps can be a wonderful thing.

    Shimel: Absolutely. I’m sorry to take us down that. Go ahead, Grigori.

    Melnik: No, it’s super encouraging to hear what you said, Alan and again, I’m a very big believer into the greatness of human mind, but it feels to me that in this kind of history of our industry, the pendulum keeps swinging back and forth, right? remember the whole infatuation with the case tools, you know, back in the eighties and nineties and then later on, there was this whole big movement of the software factories, like the term was actually software factors where the idea was that you can go and abstract away at some high level and then the loop, y, the tools will generate all the code and everything for you and make it all highly maintainable, all that, and of course that failed too.

    But I think out of this desire for some reason to do – to come up with the sort of boosters. So I don’t know, some kind of the way of equipping the humans to deal with the inherently complex problems where today there’s no better mechanism of dealing with them other than human brains. So this whole vision of, you know, computers writing computer code, I mean, that’s been promised in the first era of AI, not today, but I’m talking about the sixties, right, and seventies were all –

    Shimel: Yeah, yeah, I mean Spielberg does pictures.

    Melnik: Then it kind of died out. Right? Same thing again, you know, having the computers entirely come up with the test strategy, the thinking about testing automatically, entirely automatically, I think is – I think it’s utopical, at least if you think about what the tools are capable of doing today and for a very, very long time in the future. I think the human will remain to be at the center of this activity, while again, all the tooling is great because tooling gives you additional powers and additional ways of perhaps automating the most rudimentary, the most, you know, boring elements of the job and lets you actually free up time to think about new theories, new hypotheses, new models, new approaches –

    Bach: But you’ve got to consider different kinds of tools. Like, have you noticed the difference between let’s say a tool like a programmer’s IDE. When I use a programmer’s IDE, I don’t feel oppressed. I feel empowered. I feel like I’m riding a magic carpet and I have laser cannons mounted on it. I feel wonderful. But when I use most testing tools, I feel oppressed, like I have just a few possible things I can do and if I have any –

    Shimel: Like you’re castrated.

    Bach: that’s outside of what the test tool designer has thought that I might want to do for testing, then I’m not allowed to do that. This is what I feel like when I encounter something like Cucumber, which is very restricted and the Gherkin language, it’s very restricted. It’s stopping me from going on flights of imagination. And yet, when I use the tools that programmers have created for programmers, unsurprisingly, programmers have created for programmers tools that make them feel powerful and free and able to do all kinds of wonderful things. I would like to bring the same sense of freedom to testing that programmers already enjoy. That’s part of what we’re doing with the transformation work we’re doing at Tricentis.

    Shimel: Yeah. I love that.

    Melnik: In all fairness, not all of the programmer’s tools are that liberating. Some of them feel like you are operating in a straightjacket too, but in general, at least there are some examples, there are instances of tooling that are widely accepted, widely adopted today that are supporting and kind of accelerating your abilities to do your job without constraining you. That’s for sure.

    And you know, the piece that James was alluding to, this is actually the new initiative, that we are driving inside Tricentis, you know, about mentioning this whole new platform of the test tools for the future where we are building something akin to the IDE, we’re actually not calling it IDE, we’re talking ITE, integrated testing environment where actually it’s the human being, the human tester who is in the very, very center of the ITE, and all these other snippets and utilities and services and everything around the human tester, around the human that manages to support the individual in the way, wherever his line of thought, line of exploration, line of test execution may lead to. Okay?

    So yeah, we’re kind of envisioning this to be as a super configurable kind of morphing tool, almost like the chameleon style. We’re depending on the context and the environment, the different capabilities of the ITE, the integrated testing environment will represent themselves and then the human will actually pick whether this is actually something that they want to take advantage or not as opposed to again being forced into a specific kind of mold or one way of doing testing. And I don’t think – we don’t think actually anybody has attempted something of this caliber, specifically in the space of test tooling before. That’s why we’re so enthusiastic and so passionate about this project. Aren’t we, James?

    Bach: Yeah. I also don’t think that anyone is going to be able to copy is easily because the only way that they could actually compete with us is by hiring someone who loves testing, which test tool companies hate doing because testers are troublemakers. I still can’t really – it’s hard for me to fathom that Grigori really knows what he’s done by hiring me because I’m a constant gadfly.

    Melnik: [Laughs]

    Bach: I’m constantly complaining about the state of test tools in the world and at Tricentis, frankly, and I want things to be fundamentally better, but that means we have to fundamentally change how we think about what is a tester. Which might get us into the role of testing a bit, but the – to me a tester is a puzzle solver, a problem finder. I’m not a button pusher and I’m not a verifier. I don’t verify things. Tools can verify things. I’m not looking to prove that something works because I can’t prove that anything works. What I can do is collect evidence and then draws inferences from that evidence with a critical mindset.

    And of course, lots of tools are collecting facts about the products that I’m testing and those – that can be automated and so we’re pulling in these facts. And we can say, “We got this output and we expected to get this output, so apparently there’s no problem here as far as we can tell so far.” But then the tester layers onto that a critical mindset saying, “But how do we know and how can we be fooling ourselves?”

    And so I’m bringing that mentality into Tricentis and I’m really surprised so far they’ve been actually quite welcoming of me, but for years and years, I’ve tried to help other tool companies with this and basically, you know what reaction I get? They basically say, “Well, our customers just want to have automation that just pushes a bunch of buttons and we don’t really care about test design. We don’t really care about empowering testers. So that’s not going to help us make any money, James, so we’re really not interested in supporting the skilled testers that you keep talking about.” So Tricentis has a different kind of vision that I think is hard to match.

    Shimel: There’s also a little bit of you can’t make wine before it’s time. Right? And I think right, I think you need the right macro conditions, that things like DevOps have brought about and the whole progress -0 we’ve gone back here now to the fifties and sixties and all the way to the present time.

    Certainly the world we live in today may be a – I want to think so anyway, we’re a little bit more open to the idea of saying, “Look yeah, we can automate some stuff. We can have – we don’t need a human to just push buttons. Right? We have tools that can go push the buttons for us, but we need humans to do blank. We need humans to do some of this stuff and so we need to enable and empower them to do that within the context of let’s automate what should be automated, let’s say. But we still need those humans in there.

    You know, when I was growing up, much like you guys, right, the only animal in the world that used tools was humans. Isn’t that – that’s what I learned in school, right, that’s what separated us from animals, one of the things, right, was our ability to use tools. Well, then we found out chimpanzees use tools and other monkeys use tools and we still were okay with that ’cause they were primates. Now we found birds use tools, right, to crack nuts and stuff like this.

    So tool use in itself does not connotate one’s intelligence or higher being. We need to – so to me, that’s the ultimate case. Right? It’s not about what tools we use. It’s about how we use our brains. And I think that’s apropos to what we see in testing today. Yes, we could come up with tools, but it’s creating an environment for people to think in and use their brains in and apply it that I think will ultimately provide us the highest payback, if you will. Right?

    Bach: It’s interesting what you just said about you’re implying that the history of how we got to where we are now is important, that we have to have a certain set of – a certain historical context, technological context in order to take the next leap. And I think one thing in which that’s really true is we now have decades as an industry, decades and decades of experience trying to eliminate people from engineering and we can’t do it. I remember, maybe you do, there were these advertisements in Byte Magazine in the eighties about case tools and all your marketing people can just define your product and you don’t need developers anymore because it automatically generates all the code.

    And I remember sitting there at Apple Computer reviewing case tools, advertisements that had been sent to me, thinking, well, could I buy some of these tools and actually use them to help with the testing? And it turned out, no. You couldn’t. It was just all – it was all hype and nothing. And so I shrugged. And so a lot of – and even today, now we have AI, AI is the new case tool and the AI will do all this stuff for you. Then it turns out the AI is biased and racist and confused and if you put kitten ears on a whale, it says, “That’s a kitten.”

    Shimel: Yep. And it’s no more than ML, the machine learning anyway, but you know, yes, we’ve been – we have been, you know, chasing this white whale for most of my adult life and at the same time, if it was a nation, the fastest growing nation on earth are developers. There’s 40-something million is the estimate growing what, 20 percent a year or something like that. So I think it’s time we stop probably chasing that white whale and move on to empowering people.

    Bach: And one way to do that, I suggest a definition of programmer or perhaps developer. And the definition I use is a developer is someone who stands at the gateway between the world of the machines and the world of humans. And the developer’s purpose is to operate competently in the world of people and then operate competently in the world of machines.

    And if you don’t have that person standing at the gateway, then you don’t have a connection between the world of humans and the world of machines. You just have people kind of praying to the god of technology and hoping that good things happen. And so no matter how complicated technology gets, no matter how much AI we have, we still have the need to operate competently in the world of people and the developers will always live right at that threshold. And testers need to live there too and not get mowed under by automation.

    Melnik: Yep.

    Shimel: And you know what guys, we’re almost out of time, but this also brings up something that I thought we might be able to get to on this show, but we’re running out of time, and that is that this caste system, if you will, caste system of developers being a higher caste than the tester who’s higher than the, what we call today the SRE, the ops person or what have you. And this whole IT stack of humanity, I think we need to examine that too with the develop – when we’re asking the developer to do the security and the testing, is he any better than the tester? Is there a difference? But we’re going to save that one I think for another show, if you guys would like to come back on, and we can explore that a bit.

    Melnik: _____ back, but I categorically reject that whole hierarchy and I also do not like the whole, you know, notion when people say, “Hey, but the tools and everything that would really help with the low skilled tester.” That’s another notion that kind of gets _____ _____ –

    Bach: No, we reject that.

    Melnik: So annoying and, you know, whether it’s development or testing or any other, you know, data science modeling, the UX design, it’s the discipline, it’s something that requires dedicated effort of studying, perfecting, thinking about the way how people do that work and it requires intellect. That’s the most important thing. You know, using some very powerful tools doesn’t give you the license to turn off your brain and I’m happy to come back on the show and talk about the different career letters and progressions because again, it’s going to take us a longer –

    Shimel: That’s a whole other show.

    Bach: There’s a lot to –

    Shimel: _____ and vocal about that.

    Bach: say about that. But I just want to note that this whole issue has been studied in great depth in another field, in the field of the sociology of science. For instance, in physics, theorists are in the higher caste and experimentalists are in the lower caste. So if you’re an engineer, you’re in the lower caste and they’ve studied this. How do these people cooperate? How do they assign prestige to each other? It’s a well-known phenomenon in many fields of human endeavor.

    Melnik: And the same in math, right, the theoretical math versus applied math, the theoretical math seems to always think that they’re the gods and they’re the ones that are, you know, creating something that, you know, will take centuries for the world to truly appreciate and then you guys are in the applied math solving the problems of today.

    Shimel: Exactly. Guys, we’re out of time, but I’m going to hold this – I’m going to hold you both to it. We’re going to come back on – not in the next show because I think we have something scheduled, but in one of the ones coming up, we’ll have you both back on, or maybe this’ll serve as part of our first roundtable in about a month from now when we’ll open it up to the audience as well. But for now, Grigori, James, thank you. Yes

    Melnik: Thanks, Alan.

    Shimel: James, thank you.

    Gentlemen, it was a pleasure. This is Alan Shimel. You just watched the very first DevOps Unbound. We’ll be back in two weeks with a fresh panel, fresh topics, stay tuned. Thanks again to Tricentis for sponsoring and thank you for watching. Bye-bye.

  • Survey Sees Little Progress in Drive to Shift Testing Left

    Survey Sees Little Progress in Drive to Shift Testing Left

    It looks like the drive to shift testing left toward developers to accelerate the development and deployment of applications has stalled. A survey of 102 IT professionals involved in application testing published by Perforce, a provider of testing tools, finds only 18% of organizations test applications solely within their development feature teams. The rest continue to rely on a separate team in one form or another.

    More troubling still, the survey conducted by Gateway Research finds more than half the respondents (56%) are automating less than a quarter of their test cases.

    Eran Kinsbruner, chief evangelist for Perfecto by Perforce, a set of testing tools, said the survey results make it clear that when it comes to automating testing within a DevOps environment there is still a lot of reliance on manual testing. In fact, the survey finds the rate of test automation is less than 50% on average. Almost 40% of teams are automating fewer than a quarter of their test cases, while only 9% automate three-fourths of their test cases, the survey finds.

    On average, only 14% of respondents said they are releasing code daily. Another 39% said they release code weekly, while 46% release code monthly.

    Kinsbruner said it’s clear most organizations are still engaging in a “Big Bang” approach to testing that results in most testing being conducted after large amounts of code has been merged versus being conducted by developers concurrently as code is developed. A total of 66% said they test code after a development project is ready, while 54% said they test when each build is ready. That compares to 43% who test upon each code change and 41% that test at the start of a software sprint.

    A major reason many organizations are not releasing code faster is clearly manual testing. The survey identifies manual testing as the most time-consuming element of the testing process (72%), followed by test environment setup (28%), script creation/maintenance (27%) and test cycle analysis (19%). In terms of biggest challenges organization face, achieving complete test automation (47%) tops the list, followed by test instability/false negatives (44%) and lack of scripting/coding skills (41%).

    In terms of overall priorities, survey respondents placed reducing regression testing time (46%), reducing the number of escaped defects (43%) and automated testing (43%) at the top of the list.

    It may not be possible to automate every part of the testing process, but Kinsbruner said it is clear many more organizations could be automating lower-level testing processes to accelerate application development and deployment. In fact, the survey notes 46% of respondents are looking to invest in test automation in 2020. Among organizations that are using testing tools, two-thirds are using open source software (68%), while 66% are using some type of automation framework.

    Regardless of the approach to testing, Kinsbruner said it’s clear testing teams need more visibility into the DevOps pipeline. Organizations are clearly underestimating how long it takes to thoroughly test an application, so the sooner those teams can see what is being developed the faster they can develop test suites, he said.

    Of course, there are still plenty of development teams that are giving short shrift to testing in the name of achieving delivery deadlines. The trouble with that approach, however, is the end customer all too often winds up being dissatisfied to the point where the application winds up being reject altogether. After the end of the day, nobody really wants more bad code faster.

    — Mike Vizard

  • Launchable Aims To Improve Delivery Velocity, With Kohsuke Kawaguchi and Harpreet Singh

    Launchable Aims To Improve Delivery Velocity, With Kohsuke Kawaguchi and Harpreet Singh

    A new company, Launchable, aims to improve your delivery velocity by helping you be much smarter about which tests you run and when. By applying machine learning (ML) and better use of the data output by your software development work, the two co-founders think they can make a meaningful impact on development efforts.

    Sure, you say, lots of developers say they can change the world. But in this case, the co-founders have some very impressive résumés to give you confidence they just may do it. The dynamic duo is none other than Harpreet Singh, former GM of Atlassian BitBucket and VP of Product and Design at CloudBees, and the one and only Kohsuke Kawaguchi, founder of Jenkins and CloudBees Chief Scientist, most recently.

    Wait a second, you may be asking, you’re saying KK from Jenkins/CloudBees has started a new company? Yes, that’s what I am saying. In a podcast that will be available very soon, KK and Harpreet talk with us about Launchable’s why—and why now. Stay tuned for that.

    It’s worth noting that KK has been at CloudBees for 10 years and working on Jenkins for 15 years. In the tech business, that is nearly a lifetime. During our podcast, KK said he felt that with Jenkins being passed to the Continuous Delivery Foundation (CDF), coupled with a healthy group of leaders within the Jenkins community, now was a good time to move on to the next adventure.

    Beyond Jenkins, CloudBees has grown into its own next phase, with lots of exciting developments announced and more coming. The scientist in KK has led him to explore solutions to problems he sees in the way so many companies are doing development. His longtime friendship with Harpreet, dating back to their days at Sun, has energized them both to once again work together and tackle the real problems that development teams are facing.

    What will come out of Launchable? I don’t know. But the company has already attracted some blue-chip VC investment, as well as an impressive list of angels including Sacha Labourey, CEO and founder of CloudBees.

    They (like everyone else, it seems) are hiring development engineers and data scientists. So if working with KK is something that appeals to you, head over to the Launchable site and check out the Careers section.

    I am sure we will be hearing more about Launchable in the days, weeks and months ahead. In the meantime, best of luck to Harpreet and KK, and many thanks for a job well done with Jenkins.

    — Alan Shimel

  • DevSecOps: 10 Best Practices to Embed Security into DevOps

    DevSecOps: 10 Best Practices to Embed Security into DevOps

    For companies that employ the agile approach, DevOps seems like a natural extension. Traditionally, enterprises started with integration, development and test automation early in the product lifecycle. Gradually, the agile delivery team took care of iterative development and monitoring practices that increase code quality.

    Today, organizations work in unison to bridge the gap between development and operations (DevOps). They want to offer a faster market delivery with the least human interaction. 

    But what about security integrations? Is there a way to mitigate vulnerabilities early in the development lifecycle?

    The answer is DevSecOps.

    Exploring the Concept of DevSecOps

    DevSecOps is short for development, security and operations. It brings together people, processes and technology to pursue a shared objective. 

    The objective of DevSecOps is to implement security decisions on the same scale as development and operations and make everyone in the product lifecycle accountable for security. 

    Why Should You Adopt DevSecOps?

    People adopt DevSecOps because they are seeking:

    •  Modern alternative to traditional security engagement.
    • Transparent collaboration and workflows during development.
    • Security that’s built into the product, not applied at the final stage. 
    • Reduced expenses and faster delivery rate. 
    • Faster recovery speed in case a threat is detected.

    Steps to a Typical DevSecOps Workflow

    1. A developer starts by writing code within a version control system.
    2. Any required change is committed to the version control system.
    3. Another developer analyzes the code to identify any security defect that may weaken code quality. 
    4. An environment is created to deploy and apply security configurations to the system.
    5. Next, a test automation suite is executed to evaluate the newly deployed application.
    6. After it passes the automation test, the application is deployed to a production environment.
    7. This new production environment is actively monitored for security threats. 

    While there is no right way to transform organizational culture, below are a few components necessary to sustain a DevSecOps environment: 

    Let Developers Get Security Right: Developers are responsible for security. Therefore, you must keep them on top of cybersecurity best practices through continuous training and learning activities. 

    Promote an Open Culture: Openness in communication within the enterprise environment can drastically improve development and security. One way to keep information transparent is by using metrics and dashboards wherever possible. 

    Get Experts on Board: It is extremely difficult to transition from DevOps to DevSecOps without the supervision of expert security professionals. Hire people who understand security within the development and operations environment and let them train your DevSecOps team for the big transition. 

    Tempted to embed security into DevOps? How do you ensure that the best practices are followed? We have answers.

    The Best Way to Implement the DevSecOps Process

    Gather a single group of professionals (admins, developers, security engineers and testers) that are aware of your product from start to end. They should know your requirements and should be experts in deploying, monitoring and implementing new changes. 

    Once you have your team ready, here’s what you need to do next.

    Plan

    Planning is crucial. Do not just stick to feature descriptions. Instead, go for detailed user stories that include:

    • Functional and nonfunctional requirements (e.g., security and performance).
    • UI and UX designs.
    • Acceptance test criteria.
    • Threat models.

    Develop

    Start by evaluating your existing practices. Choose the best resources to build a development model in coherence with security guidelines. 

    Build

    Automated build tools can do a lot more than compile codes. Use them to conduct test-driven development, enforce quality standards and ensure that the best security practices are implemented through static code analysis. 

    Test

    When it comes to a DevSecOps environment, test automation is not limited to UI-focused Selenium tests. Optimally, your security practice should include the following:

    • Unit testing.
    • Front-end testing.
    • Back-end testing.
    • API testing.
    • Database testing.
    • Passive security testing.

    Secure

    Because development, operations and security go hand in hand, only a few issues are left unattended toward the end of the development process. 

    When vulnerabilities are identified, there is a better chance of determining if they are potential exploitations or false positives. 

    Deploy

    Automated provisioning and deployments can be utilized to accelerate product delivery and add consistency in the development process. Using an infrastructure-as-code tool, one can audit properties across the IT infrastructure and enforce secure configurations in a system. 

    Operate

    Routine maintenance and upgrades should be an indispensable component of your operations team. Leverage infrastructure-as-code tools to patch zero-day vulnerabilities and apply updates to the entire organization’s infrastructure. 

    Monitor

    A continuous monitoring plan should be in action to generate real-time stats of how your system is performing. In case any exploitation is recorded, it can be addressed immediately. 

    Scale

    Traditional data-center operations cannot entirely replace a compromised environment. Today’s ability to scale infrastructure through virtualization and the cloud, while addressing the demands of modern-day IT user base, should go a long way. 

    Adapt

    When it is about sustaining an agile practice, continuous improvement is key. This is also true for DevSecOps practices, as you improve and adapt throughout the software development lifecycle.

    Conclusion

    DevOps isn’t going anywhere, anytime soon. It is the new phase of developing, releasing and updating products in a software lifecycle. 

    That’s why high time security professionals let go of the traditional security stack and embrace security solutions at the speed of DevOps.

    — Deepak Gupta

  • DevOps Places Software Testing in the Crosshairs

    DevOps Places Software Testing in the Crosshairs

    To keep up with the pace of innovation, testing must play a larger role in the software development life cycle

    The speed of digital transformation is already staggering, and it’s only going to increase. To put this into some very concrete terms, consider that:

    • There are 7.7 billion people in the world.
    • 5 billion have regular access to a toilet.
    • 5 billion own a mobile phone.

    All of a sudden, a huge number of people jumped from a very provincial lifestyle straight into digital times—creating a tremendous demand for more, and more innovative, software.

    The traditional ways of developing and delivering software aren’t adequate for meeting this new demand. Not long ago, most companies were releasing software annually or bi-annually. Now, iterations commonly last two weeks or less. While delivery cycle time is decreasing, the technical complexity required to deliver a positive user experience and maintain a competitive edge is increasing.

    testing

    In terms of software testing, this brings us to an inflection point. In most organizations, testers were already racing to keep pace when delivery cycles were longer and application complexity was lower. Every quarter or so, the development team would pass a release candidate over the wall to QA, who would then scramble to validate it as thoroughly as possible in the allotted time—largely with manual testing.

    Now, digital transformation initiatives such as Agile and DevOps are pushing traditional methods to their breaking point. Organizations are releasing much more frequently—from monthly on the low end to multiple times per hour on the high end. And as organizations increasingly edge toward continuous delivery with automated delivery pipelines, intermediary quality gates and the ultimate go/no-go decisions will all hinge upon test results.

    Is your organization’s testing process up to the task?

    We Have a Testing Problem (Two, Actually)

    In most organizations, testing delays application delivery while providing limited insight into whether the applications are meeting stakeholder expectations. It’s not fast enough to help teams find and fix defects when it’s optimal to do so. And it’s reporting on low-level failures (e.g., 78% of our tests passed) rather than providing the business-focused perspective needed to make fast release decisions (e.g., Only 38% of our business risk was tested … and 25% of that didn’t work properly).

    Let’s take a quick look at each of these problems in turn.

    The Speed Problem

    DevOps is all about removing the barriers to delivering innovative software faster. Yet, while other aspects of the delivery process are streamlined and accelerated, testing consistently emerges as the greatest limiting factor.

    A recent GitLab survey that targeted developers and engineers found that testing is responsible for more delays than any other part of the development process.

    Where in the development process do you encounter the most delays?

    testing 2

    The same conclusion was reached by a DevOps Review survey that polled a broader set of IT leaders across organizations practicing DevOps. Again, testing was cited as the No. 1 source of holdups in the software delivery process. In fact, it “won” by a rather wide margin here—63% reported that testing was a major source of delays, while the second-highest source of delays (planning) was cited by only 32% of the respondents.

    Where are the main holdups in the software production process?

    testing 3

    Why is testing such a formidable bottleneck? That could be the topic of an entire book. For now, let’s summarize some key points:

    • The vast majority of testing (over 80%) is still performed manually—even more at large enterprise organizations.
    • Approximately 67% of the test cases being built, maintained and executed are redundant and add no value to the effort.
    • At organizations that have significant test automation, testers spend 17% of their time dealing with false positives and another 14% on additional maintenance tasks.
    • Over half of testers spend five to 15 hours per week dealing with test data (average wait time for data = 2 weeks).
    • 84% are routinely delayed by limited test environment access (average wait time for test environments = 32 days).
    • The average regression test suite takes 16.5 days to execute, but the average Agile sprint is two weeks, from start to finish—including planning, implementation and testing.
    • The average application under test now interacts with 52 dependent systems—which means that a single end-to-end transaction could cross everything from microservices and APIs to a variety of mobile and browser interfaces, packaged apps (SAP, Salesforce, Oracle, ServiceNow…), custom/legacy applications and mainframes.

    The software testing process wasn’t working perfectly even before the advent of Agile and DevOps. Now we’re asking teams to “just speed it up” at the same time that modern application architectures are making the process even more complex. Given that, it’s hardly surprising that speed expectations aren’t being met.

    The Insight Problem

    Only 9% of companies perform formal risk assessments on their requirements/user stories. Most attempt to cover their top risks intuitively, and this results in an average business risk coverage of 40%. Would you feel comfortable driving a race car with blinders on? That’s essentially what you’re doing if you’re rapidly delivering software with insight into less than half of your total business risk.

    Moreover, most organizations can’t immediately differentiate between a test failure for a trivial issue and a business-critical failure that must be addressed immediately. Most results look something like this:

    testing 4

    What insight does that really provide? It’s clear that …

    • There’s a total of 53,274 test cases.
    • Almost 80% (42,278) passed.
    • Over 19% failed.
    • About 1% did not execute.

    Maybe the test failures are related to some trivial functionality. Maybe they stem from the most critical functionality: the “engine” of your system. Or, maybe the most critical functionality was not even tested at all. Trying to track down this information would require tons of manual investigative work that yields delayed, often-inaccurate answers.

    Today’s go/no-go decisions need to be made rapidly—even automatically and instantaneously. Results that focus on the number of test cases leave you with a huge blind spot that becomes absolutely critical—and incredibly dangerous—when you’re moving at the speed of digital business.

    Closing the Gap

    How do you evolve from the slow, burdensome testing that delivers questionable results to the lean, streamlined testing that provides the fast feedback required to accelerate innovation and delivery?  That’s what I aim to explain in my new book, “Enterprise Continuous Testing: Transforming Testing for Agile and DevOps.” This book is targeted to senior quality managers and business executives who need to achieve the optimal balance between speed and quality when delivering the software that drives the modern business. It provides a road map for how to accelerate delivery with high confidence and low business risk.

    — Wolfgang Platz

  • Synack: DevSecOps Being Accelerated by Cultural Shifts

    Synack: DevSecOps Being Accelerated by Cultural Shifts

    The 2020 State of Compliance and Security Testing Report from cybersecurity testing platform vendor Synack claims that some of the world’s largest organizations are encountering a significant cultural shift within their development teams, and that bodes well for those seeking to build DevSecOps teams. For the report, Synack surveyed leaders from more than 300 organizations representing a number of industries and verticals, including technology, government, health care, information technology and financial services. 

    According to the report, a large percentage of organizations and institutions are moving toward a rigorous, continuous testing model to ensure compliance. As part of this shift, organizations are utilizing crowdsourced security testing to achieve regulatory compliance and real security, with adoption expected to increase four-fold in 2020.

    Perhaps those companies are being driven by numerous new regulations on the horizon, including the  California Consumer Privacy Act (CCPA), PCI  DSS 4.0, New York’s SHIELD Act and  other proposed data compliance regulations. Or perhaps the shift toward continuous crowdsourced security testing is being driven by the failures some organizations have encountered with traditional security testing methods, which were performed less frequently and failed to keep up with the CI/CD (continuous integration/continuous delivery) pipelines that many organizations have embraced. 

    Simply put, Agile and DevOps application development and delivery methodologies were poorly served by the traditional, siloed security testing methodologies of the past. A realization that has driven many organizations to embrace the DevSecOps model, in which cybersecurity becomes part of the overall team, having equal standing with both development and operations team members. 

    Organizations pushing ahead with the DevSecOps model quickly realized that security toolsets and team member interactions must change to be successful. Synack’s research bears that out, and the report offers that the adoption of continuous security testing platforms is expected to increase by 4X in 2020.

    “The rapid embrace of crowdsourced security testing has happened because it is proven to work better than traditional security testing methods and addresses the ever-growing talent gap within organizations,” said Synack CTO and co-founder Mark Kuhr. 

    The report found security testing is becoming part of an organization’s normal routine, rather than a once-a-year check of the box focused only on compliance. According to the report, 44% of the organizations and institutions surveyed perform security tests on a monthly or weekly basis, suggesting that they are moving toward a continuous model, which is further enabled by crowdsourced solutions.

    The report also revealed that some 63% of organizations commonly use external vendors to identify and reduce vulnerabilities. However, 52% experience unwanted cost and complexity due to overlap in functionality from using multiple security vendors. Those two data points demonstrate that simplification provided by leveraging a single security testing platform could deliver savings. Synack claims crowdsourced security testing solutions provide 147% higher ROI than a typical pen test and can decrease the burden of testing on organizations by reducing the signal-noise ratio. 

    “This shift toward continuous crowdsourced security testing will allow organizations and institutions to have the best of both worlds by procuring technology that offers efficient and effective results while fulfilling best practice standards such as NIST 800-53 to meet compliance objectives,” said Kuhr.

    “Although we are seeing a move toward a 24/7, 365 security culture at organizations in a wide variety of industries and geographies, there is still ample room for improvement,” said Aisling MacRunnels, Synack’s CMO. “Our survey found that on average, most security tests are lasting just 20 hours. As the number of cyber incidents continues to increase, it will be imperative for decision-makers to implement security testing solutions on a continuous basis with 1500-2000 hours of testing a year.”

    Those increased testing requirements will ultimately become the foundation to build a DevSecOps culture in most any organization bound by compliance regulations.

    — Frank Ohlhorst