devopsy.kinsta.cloud

Tag: agile

  • In DevOps, Figure Out Why Before Considering How

    In DevOps, Figure Out Why Before Considering How

    It has been fairly well-established that DevOps is primarily a function of the underlying culture—about breaking down barriers and shifting roles so that teams and individuals are empowered to come up with innovative ideas and execute them without the burden of traditional corporate bureaucracy. It is also understood that there are a variety of tools and platforms that enable and automate processes. However, before an organization can even get to the “How”—the culture and tools of DevOps—it should be able to answer the question, “Why?”

    I reached out to organizations that have embraced DevOps and leaders in the field to see what they had to say about the “why.” What benefits can businesses expect, and what goals can adopting DevOps help them achieve—or achieve faster, or achieve cheaper?

    “Delay in delivering innovation comes with an increasingly high price tag, perhaps the loss of an entire market to competitors,” says Forrester Analyst Charles Betz. “DevOps accelerates and improves the quality of software delivery, but that is not an end in itself. The increased speed provided by DevOps is essential to achieving the fast product feedback needed for digitally transforming enterprises to stay relevant and competitive.”

    As Betz points out, companies should implement DevOps for the competitive advantage. Dave Lindquist, IBM Fellow and VP of Private Cloud for IBM, puts it another way: “DevOps is at the heart of the digital transformation. It’s how the next wave of disruptions are being developed and delivered.”

    DevOps bridges the gap between development and operations teams. Inspired by agile development and continuous integration, and incorporating their principles, it addresses the challenges of today’s condensed timeline for application development.

    In that way, the line between the “Why” and the “How” blurs into a bit of a gray area. Companies should adopt DevOps with specific goals in mind—a vision for what they expect it to do for them—because simply eliminating barriers or implementing tools and platforms without a strategy in place for what that is expected to accomplishment is likely to be a waste of time, money and effort. On the other hand, it is very difficult to execute or achieve those goals without also successfully navigating the “How.”

    “Enterprises are navigating major technology shifts and demand is therefore higher than ever for simple and consistent automation and management across hybrid infrastructure and application portfolios,” says Marc Holmes, VP of Product and Revenue Marketing at Chef. “Organizations should adopt DevOps practices for both the culture and tooling to create a consistent and collaborative workflow so enterprises can build, manage and deploy software quickly and safely in any environment.”

    “DevOps can have a direct effect on your bottom line. According to a recent study from Enterprise Management Associates (EMA), ‘Companies that excel at DevOps interactions are 11 times more likely to see double-digit annual revenue growth. It doesn’t stop at revenue growth either. According to EMA, 49 percent of organizations also agree that accelerated delivery frequency leads to higher customer satisfaction, and 35 percent find that it enables them to be more agile and competitive,’” notes Chris Lippi, VP of product at NGINX.

    With its focus on automation, DevOps also helps to ensure predictability, efficiency, accuracy and the standardization of operational, development and testing processes. This, in turn, saves time and money by shortening the release cycle, reducing errors and lessening the need for time-consuming fixes. It also frees up resources to focus on more creative endeavors and on tackling larger problems as they arise.

    Lippi sums it up this way: “Once implemented, DevOps tends to prove its business value in terms of time and cost savings. Making the business value of DevOps clear—to the newly unified team and beyond—helps DevOps culture take root.”

    There are lots of tools and practices out there to choose from, and plenty of guidance and advice on how to implement a DevOps culture. Before you go down that path, though, step back and figure out what you hope to accomplish by adopting the culture, tools and practices.

    — Tony Bradley

  • Culture Change: Where It Matters in DevOps

    Culture Change: Where It Matters in DevOps

    Red SquareLori and I were in Russia in the late ’90s. It was an interesting dichotomy of life in a changing country. While we were there, the government devalued the ruble—just cut three zeros off the end of everything—to bring it into historical line with other world currencies. It was odd, because there was a mass of old money and a smattering of new, and most systems were not yet reconfigured to handle the new prices. Thus we paid 180,000 rubles to fly from Moscow to Rostov.

    But the physical implications were not the issue. The cultural implications were. We stayed with a very nice host family in Moscow, and talked long with them about pretty much everything. The older generation tended toward the old system, while the younger generation toward the new. Thus, the father of the family, in his 50s, blamed the west (specifically the United States, but since we were from the United States, I think the vodka made him stress that part) for losing his job as an astrophysicist. He also blamed the west for their currency woes. If yesterday, you had half a million rubles in the bank, today it was 500, so his angst was understandable. He blamed trade issues for that. His wife agreed with him, and scolded us for buying them nice things, since they would have to pay to keep them usable.

    Their adult son—and indeed most of the younger  generation—was the polar opposite. The younger generation thrived in the new environment, saw a bright future and laughed at today’s misfortune. It would be better tomorrow, and they were adapting to the new open environment to make sure it was.

    No, this is not a “get out of the way with your old thinking” blog. It’s a “be smart with your new thinking” blog. One problem endemic to the youth of Russian culture when we were there was they believed the rules no longer applied rather than believing there were new rules and new ways to interpret the old rules. It got many into trouble.

    To Quote a Friend: Culture, It’s a Thing

    There are some in the DevOps community who like to talk only about culture, because the non-automation benefits of DevOps are most felt in culture change. This is a not-unreasonable stance, and though I am definitely an aficionado of the automation side of DevOps, I see the benefits of culture change and embrace them where they are most impactful. I also acknowledge that in a full DevOps implementation culture change is inevitable.

    One place where most organizations could put more forethought—regardless of how the project is developed/deployed— is in releases. Think this through as a customer, not as a DevOps engineer. You have been burned by companies horribly missing a deadline after announcing features that you were waiting for (most of us have, anyway). By this point you have also been burned by companies releasing with the feature you wanted, only to find that it barely (if at all) worked, but now that it was released, the company would make it better over time.

    Now switch back to being an IT professional. We know the pressures and issues around releases.

    I think the biggest improvement commercial enterprises can make is to decouple marketing releases from Dev releases—to watch a project go through CI/CT/CD/ARA until it is solid, and only then start planning a release date. While this isn’t fully in tune with rapid delivery, the marketing team has a whole set of activities to perform that are dependent upon a certain level of functionality. Weekly stand-ups and continuous delivery do not make that functionality real on the date it was promised, everyone involved in the process knows that. So planning like it will be is as inefficient as writing code before you have a product idea.

    For internal teams, it’s a little bit different. There is the opportunity for Dev/Release engineers to stay in contact with representatives of the customer (normally managers or team leads, but whoever represents the customer), and adjust everything. Expectations can be managed if a certain functionality prove difficult to get working right, or release dates can be managed except in the most politically charged environments, and slight changes can be made to requirements—things such as, “Well, if we do it this way, we can meet deadlines, but it will impact this other thing,” can be brought to the customer. Having a representative of the customers on the team is sort of like disassociating marketing release from DevOps delivery: There is the chance to make sure it’s right before it hits users’ hands.

    Oh Yeah, Culture

    But I’m not here to tell you how to manage the process, I’m really here to talk about the culture. It used to be that if a needed feature wasn’t ready, the feature was cut or the release was delayed. All that Agile and DevOps have really added at that late stage in the game is a third option—release it with a substandard feature.

    In the end, either users (in an enterprise setting) or someone looking out for users (in a commercial setting) need to make that call. Far too often, Dev and Ops are making it, and users are suffering for it. Change that. Give the power to those you are working for. The app you’re developing is aimed to serve a purpose for a set of users. If a new release does not offer them what they need—even if you can check a box and say it’s in there—then don’t release.

    The thing combined Agile and DevOps does generally give us is more predictive ability. It’s not perfect, but we do know sooner if there will be problems. Generally speaking, it is easier to slip release dates early in the process, simply because the release is not as anticipated.

    It’s About Why

    Some teams that used to cut features because they couldn’t meet deadlines are now teams that put in sub-standard features so they do meet deadlines. There are good reasons to plan a graduated rollout of a feature or feature set, but “couldn’t get it implemented” isn’t one of them. The granularity of Agile means that estimates should be tighter, because you’re estimating smaller chunks at a time. The ability of DevOps to deliver a more consistent end product and configured environment should mean there is less delay in the deployment cycle. So the correct answer is to find out what users want, go that route and then examine where you went wrong so you can do better next time. Sometimes “do better” will mean better estimates, sometimes it will mean using less new or questionable technology, sometimes it will mean controlling change in the environment outside the project—or accounting for it in estimates. But improvement should be the goal, all of the time.

    In the end, you’re creating a tool designed to serve the needs of a certain set of users. I don’t want to be sold a snappy laptop with an advertised 1TB SSD drive only to get delivered the same laptop with an external, full height, MFM encoded drive touting a whopping 110 MB. So why would it be okay to do the equivalent in software (or, in a couple cases I know the details of, firmware)? Don’t be silly, of course that’s extreme. Maybe the machine comes set up to boot from 5.25-inch floppy instead, to reduce shipping overhead.

    — Don Macvittie

  • Continuous Testing vs. Test Automation: 3 Key Differences

    Continuous Testing vs. Test Automation: 3 Key Differences

    Testers have been wrestling with test automation for years, yet most teams are not satisfied with their current level of test automation or the overhead required to maintain it. Additionally, the past few years have brought a sea change in the way that applications are architected, developed and consumed—increasing both the complexity of testing and the business impact of software failures.

    How can software testing professionals help the business control risk in light of the increased complexity and pace of modern application delivery? Enter continuous testing.

    What is Continuous Testing?

    Continuous testing is the process of executing automated tests as part of the software delivery pipeline to obtain feedback on the business risks associated with a software release candidate as rapidly as possible.

    Test automation is designed to produce a set of pass/fail data points correlated to user stories or application requirements. Continuous testing, on the other hand, focuses on business risk and provides insight on whether the software can be released. To achieve this shift, we need to stop asking, “Are we done testing?” and instead concentrate on, “Does the release candidate have an acceptable level of business risk?”

    Why Do We Need Continuous Testing?

    Today, changes across the industry are demanding more from testing while making test automation more difficult to achieve (with traditional tools and methods, at least):

    • Application architectures are increasingly more distributed and complex, embracing cloud, APIs, microservices, etc., and creating virtually endless combinations of different protocols and technologies within a single business transaction.
    • Thanks to Agile, DevOps and continuous delivery, many applications are now released anywhere from every two weeks to thousands of time a day. As a result, the time available for test design, maintenance and especially execution decreases dramatically.
    • Now that software is the primary interface to the business, an application failure is a business failure—and even a seemingly minor glitch can have severe repercussions if it impacts the user experience. As a result, application-related risks have become a primary concern for even non-technical business leaders.

    How Is Continuous Testing Different From Test Automation?

    The main differences between continuous testing and test automation can be grouped into three broad categories: risk, breadth and time.

    Risk

    Businesses today have not only exposed many of their internal applications to the end user, they also have developed vast amounts of additional software that extends and complements those applications. For example, airlines have gone far beyond exposing their once-internal booking systems. They now let customers plan and book complete vacations, including hotels, rental cars and activities. Exposing more and more innovative functionality to the user is now a competitive differentiator—but it also increases the number, variety and complexity of potential failure points.

    Large-scale “software fails” have such severe business repercussions that application-related risks are now prominent components of a business’ public financial filing. Given that notable software failures resulted in an average -4.06 percent decline in stock price (which equates to an average of negative $2.55 billion loss of market capitalization), it’s not surprising that business leaders are taking note—and expecting IT leaders to take action.

    If your test cases weren’t built with business risk in mind, your test results won’t provide the insight needed to assess risks. Most tests are designed to provide low-level details on whether user stories are correctly implementing the requirements—not high-level assessments of whether a release candidate is too risky to release. Would you instantly stop a release based on test results? If not, your tests aren’t properly aligned with business risks.

    To be clear: We’re not suggesting that low-granularity tests aren’t valuable; we’re stating that more is needed to stop high-risk candidates from going out into the wild.

    Breadth

    Even if a business manages to steer clear of large-scale software fails that make the headlines, even seemingly minor glitches can still cause trouble these days. If any part of the user’s experience fails to meet his expectations, you not only risk losing that customer to a competitor—you also risk brand damage if that user decides to take his issues to social media.

    Just knowing that a unit test failed or a UI test passed doesn’t tell you whether the overall user experience is impacted by recent application changes. To protect the end user experience, you need tests that are broad enough to detect when an application change inadvertently impacts functionality which users have come to rely on.

    Time

    Now that the speed at which organizations ship software has become a competitive differentiator, the vast majority of organizations are turning to Agile and DevOps to accelerate their delivery processes.

    When automated testing emerged, it focused on testing internal systems that were built and updated according to waterfall development processes. Systems were all under the organization’s control, and everything was completed and ready for testing by the time the testing phase was ready to start.  Now that Agile processes are becoming the norm, testing must begin in parallel with development; otherwise, the user story is unlikely to be tested and deemed “done done” within the extremely compressed iteration time frame (often two weeks).

    If your organization has adopted DevOps and is performing continuous delivery, software may be released hourly—or even more frequently. In this case, feedback at each stage of the process can’t just be “fast”; it must be nearly instantaneous. If quality is not a top concern for your application (e.g., if there are minimal repercussions to doing a rollback when defects are discovered in production), running some quick unit tests and smoke tests on each release might suffice. However, if the business wants to minimize the risk of faulty software reaching an end user, you need some way to achieve the necessary level of risk coverage and testing breadth—fast.

    For testing, there are several significant impacts:

    • Testing must become integral to the development process (rather than a “hygiene” task tacked on when development is complete)
    • Tests must be ready to run almost as soon as the related functionality is implemented
    • The organization must have a way to determine the right tests to execute at different stages of the delivery pipeline (smoke testing upon check-in, API/message layer testing after integration, end-to-end testing at the system level, …)
    • Each set of tests must execute fast enough that it does not create a bottleneck at the associated stage of the software delivery pipeline
    • A way to stabilize the test environment is needed to prevent frequent changes from causing an overwhelming number of false positives

    Continuous Testing> Test Automation

    If you only take one idea away from this article, we hope that it’s this:

    Test automation ≠ continuous testing

    Continuous testing > test automation

    Even the teams that have achieved fair levels of success with traditional test automation tools hit critical roadblocks when their organizations adopt modern architectures and delivery methods:

    • They can’t create and execute realistic tests fast enough or frequently enough
    • The constant application change results in overwhelming amounts of false positives and requires a seemingly never-ending amount of test maintenance
    • They can’t provide instant insight on whether the release candidate is too risky to proceed through the delivery pipeline

    It’s important to recognize that no tool or technology can instantly “give” you continuous testing. Like Agile and DevOps, continuous testing requires changes throughout people, processes and technology. However, trying to initiate the associated change in people and processes when your technology is not up to the task will be an uphill battle from the start … and ultimately a losing one.

    If your organization is starting or scaling your continuous testing automation efforts, we encourage you to review two new pieces of research by Forrester and Gartner. Both reports provide insight into continuous testing and test automation trends as well as how the top continuous testing tools compare.

    About the Author/Wayne Ariola

    Wayne Ariola (1)Wayne Ariola, CMO of Tricentis, is a recognized thought leader on software testing topics such as continuous testing, risk-based testing, service virtualization, and API testing. Wayne has created and marketed products that support the dynamic software development, test, and delivery landscape. He has driven the design of many innovative technologies and received several patents for his inventions. He has been a contributor to the software testing space for fifteen years and in the software industry for more than 20 years. Connect with him on LinkedIn.

  • DevOps Chat: Waterfall Will Not Disappear, with Flint Brenton, CEO Collabnet

    DevOps Chat: Waterfall Will Not Disappear, with Flint Brenton, CEO Collabnet

    One of the perks of being editor in chief of devopsy.kinsta.cloud is I get a chance to meet with so many leaders of the DevOps and IT industry. You can really learn a lot by listening to what these folks say. This DevOps Chat is a perfect example. I sat down with Flint Brenton, CEO of Collabnet. Collabnet is not some johnny-come-lately. The company has been helping developers and IT professionals for a long time. Flint talks about the fact that while DevOps is a great path for many organizations, it may not be the right way for every situation. Waterfall is not going away entirely.

    Have a listen to my discussion with Flint and hear for yourself what he has to say on the subject. As usual, the streaming audio player is immediately below with the transcript below that. Enjoy!

    Streaming Audio

    Transcript of Audio

    Alan Shimel: Hey, everyone. This is Alan Shimel, devopsy.kinsta.cloud, here for another DevOps Chat. Happy to be joined in this DevOps Chat by Flint Brenton, CEO of CollabNet. Flint, welcome.

    Flint Brenton: Hey, Alan. It’s great to see you again.

    Shimel: Great to see and hear you again. Flint, the last time I think you and I talked was back at the DevOps Enterprise Summit in San Francisco. I guess that had to be October or November.

    Brenton: November. Actually, it was the same week that our new president was elected, if you remember.

    Shimel: Yes. Yeah, a red letter day.

    Brenton: Right.

    Shimel: So, Flint, without getting any further into that, let’s turn the conversation to CollabNet. What—well, first of all, maybe some of our audience is not familiar with CollabNet in general, so why don’t we start with that? Can you give us a little CollabNet background?

    Brenton: Yes, absolutely. So CollabNet is a SAS-based, cloud-enabled development platform focused exclusively on enterprise that delivers application lifecycle management capability for both Agile and waterfall development processes.

    Shimel: Mm-hmm.

    Brenton: The version technology to structure those builds and to maintain high quality and high quality control throughout the process. And then we also provide the DevOps infrastructure to build and deploy automatically whatever gets created in Azure or waterfall, with high quality, on time, and, to the extent we can help, under budget. So that’s really our focus.

    Shimel: Sure. And you know what, Flint? I remember this about CollabNet. You guys have been in business now for a number of years, certainly before DevOps was what it is today, let’s call it, whatever it is today. How long have you been—

    Brenton: Yeah, we’ve been in business—this is our 18th year.

    Shimel: Absolutely. So, you know, DevOps is another technology innovation, if you will, that’s come down the road, but I think what—not unique, but what is one of the special qualities of CollabNet is is that you’re not out here saying that the whole world needs to go DevOps today, right, that everything—we should be all DevOps all the time. But you have customers who are still on waterfall and may very well remain on waterfall for the foreseeable future.

    Brenton: Right. And we really invest very heavily in both waterfall and Agile development platforms and—although the growth rate for Agile’s higher than waterfall in enterprise, we still see roughly 70 [percent] to 75 percent of enterprises using a waterfall methodology. But the fastest growing segment in that area is clearly Agile.

    Shimel: Sure. And I—obviously. But I think what’s important for our audience to recognize is even the biggest DevOps and Agile advocates or biggest—whatever you wanna call ’em, should recognize that there are still projects and places where waterfall might be the best way to skin the cat, to complete a project correctly. And it doesn’t have to be an all-or-nothing type of environment, if you will.

    Brenton: That’s exactly right. You know, the industries that tend to be highly regulated, with a high need for security, compliance, and scalability, tend to lean more towards waterfall development. And, of course, once you started a process where you’ve got an established team that’s been using waterfall for a long period of time, they tend to prefer to continue with that. But it’s new projects, new projects where maybe time to market outweighs the need for compliance and security and scale—those tend to go Agile in an enterprise. But, either way, regardless of development methodology, the things you need beyond the tool chain or the process of building software is fairly similar, right? The way you use version control is fairly similar. The way you use DevOps technologies is also very similar.

    And what we found is customers who are well grounded in Agile and waterfall—typically, in enterprise, they have both—getting your act together in DevOps and making sure you have a strategy, you created a culture to embrace and execute that strategy, then having the processes and tools in place to bring it to reality is critical because you don’t really get the return on investment for anything you develop, regardless of the methodology, until you put the build into production. And that’s what DevOps, at the end of the day, is all about.

    Shimel: Sure is. Excellent. Excellent point, Flint. Flint, let me turn things a little bit ’cause, as I mentioned to you off camera, off mic, 15 minutes here goes pretty quick. Want to talk about some recent news around CollabNet in the industry. I understand you guys were just included in a Gartner Magic Quadrant, correct?

    Brenton: Yes. Yes, thank you for bringing that up. Yeah, we’re very excited about it. It’s always a reflection of the investment our team has made in Agile planning and execution. So we’ve always been considered a leader when it comes to waterfall development, but, as we just mentioned, for enterprise, our enterprise customers are really demanding that we provide both, both equal value in waterfall and Agile. So we invested heavily in the last few years in Agile, and, as a result, we were just included in their Magic Quadrant as a visionary for Agile planning and execution, and we’re very excited about it.

    Shimel: That’s nice. That is great. And I know everyone always wants to be in the upper right, but, really, having my background, being entrepreneurial and having started businesses, I’d almost rather be in that visionary quadrant than the upper-right leader quadrant and certainly in the visionary quadrant over, let’s say, the challenger quadrant, which is the upper left, because I think, with visionary, I mean, the recognition is that you’re setting a—you’re pushing the envelope, right? You’re pushing the vision—hence, “visionary.”

    Brenton: Right.

    Shimel: So congratulations to you guys. I know, in speaking with your team, you have rights for download of the quadrant, so what we’ll try to do is put into the show notes here a link to that where people can download the quadrant report and see if—check it out for themselves.

    Brenton: Yeah, that’d be perfect and I’m sure that your listeners would very much appreciate that.

    Shimel: Sure will. So, Flint, beyond the quadrant inclusion, though, what else do you see that’s happening with CollabNet or in the market in general?

    Brenton: We’re starting now to get a number of customer wins that are embracing our motto of providing a cloud-based service where you can consume our integrated ALM, version control, and DevOps solution either in Amazon or on our own private cloud and, in many cases, a combination of on-prem and off-prem.

    Shimel: Right.

    Brenton: So we now have a number of customers who have completely moved their development process into one of our cloud-enabled environments, and it’s very exciting.

    Shimel: That is exciting.

    Brenton: It’s very exciting. And the great (thing) about that is, if it’s in the cloud, it’s actually easier for us to include, Alan, both open-source and compare it to components that we don’t own or control because then the need to test to get everything in your data center  is lessened and our ability to innovate to your advantage is much greater, so we’re very excited about that.

    Shimel: Absolutely. So, Flint, interestingly enough, I was reading an article this morning—I forgot where I saw it but it was a recent survey where 65 percent of enterprises still keeping the majority of their infrastructure on their own data center, not on a public cloud, not a hybrid situation. Is that about what you guys are seeing but is the arrow pointing up to the cloud? What’s your take on that?

    Brenton: Now that’s a great question, Alan, and it’s absolutely what we’re seeing. So a thing to keep in mind, though, is “What is the variable that’s changing and what is the velocity of that change?” Right? So I would say, two years ago, that 65 percent number was probably 90-plus, right? So, as enterprises get more and more comfortable with the security and compliance options in the cloud, whether it’s in a public cloud or private cloud, they can see the tremendous advantages of driving down the cost of development and actually increasing the speed of innovation by going to the cloud.

    I’ll give you an example. U.S. government, we do a tremendous amount in federal, and so both the Obama administration and now it’s transitioned to the Trump administration has really been pushing the department of defense to do a couple things. One, innovate faster, and so, if you just think about the U.S. government is barraged by cyberattacks from all over the world. Right?

    Shimel: Mm-hmm.

    Brenton: And so your ability as a professional, in NSA or CIA or the FBI, to deal with that is you want a complete toolkit of things you can use, and you want to choose the tools you need to deal with those cyberattacks, but you also need to do it quickly. So if you can use open source, it helps. If you can deploy it in a cloud, it dramatically increase—or decreases the time to deployment. And, three, it also tremendously helps you on the economics side. So even something as highly regulated and structured as the US federal government, there are advantages sometimes in doing things in a public cloud to help, and so we’re starting to see variations of that theme also in industry, primarily around enterprise, where there’s flexibility, increased ability to innovate, and improved economics by moving workflow to the cloud.

    Shimel: Yep. You know, Flint, living in the tech world and you lived the world—you live that life; I live that life—I think sometimes we get  jaded in that we’re in this bubble of, “What’s the hot new thing?” and it’s DevOps and it’s whatever’s next after DevOps, and then sometimes you get a reminder—65 percent still not in the cloud. And you say, “But I thought everyone’s in the cloud.” And I think it does good for us to remember that there’s a big world out there that moves at their own pace and their own—

    Brenton: Right.

    Shimel: Same thing with waterfall. Right? I think people would be—people in our audience from devopsy.kinsta.cloud would probably be surprised to find out how many organizations still either haven’t adopted DevOps or are primarily waterfall shops.

    Brenton: You know, the key, when you’re a solution provider like CollabNet, is you need to meet people where they are. Right? So, if they wanna be on-prem, you need to be able to deliver them superior value on-prem. If they want to move part of their workload to the cloud, you need to be able to help lead them and sponsor that transition at that pace by which they’re comfortable. Right? So I learned a long time ago the key to success in this business is to give people what they want, right? He who can do that best with a quality solution at a great price is gonna win. Right? So that’s just the way it works, right?

    Shimel: Absolutely—and that hasn’t changed, by the way, through all of these innovations, and I think it was true 100 years ago and it’s true today. Or maybe even more than 100.

    Brenton: You bet.

    Shimel: Anyway, Flint, listen, I think we’re about out of time here. I wanna be respectful of your time, so what we will do is let’s have you on again soon when we have some more time and we can delve into this. Are you gonna be at DevOps Enterprise Summit maybe in London?

    Brenton: You bet. Yeah.

    Shimel: Or San Francisco?

    Brenton: Well, I always enjoy speaking with you, Alan. You’re a great guy and you’re—

    Shimel: Thanks.

    Brenton: —you’re a fun guy off camera too, I will add.

    Shimel: [Laughs]

    Brenton: So I look forward to our next adventure together. Thank you for the—

    Shimel: Absolutely. Hey, Flint—

    Brenton: All right.

    Shimel: Flint Brenton, CEO of CollabNet, this week’s guest on DevOps Chat, thanks for joining us. This is Alan—

    Brenton: My pleasure, Alan. Thank you.

    Shimel: Thank you.

    Brenton: All right. Buh-bye.

    Shimel: This is Alan Shimel for DevOps Chat.

    — Alan Shimel

  • ShiftLeft & DevOps: A Narrative Within a Narrative?

    ShiftLeft & DevOps: A Narrative Within a Narrative?

    I procrastinated watching the season finale of “Westworld” just because I ended up reading a day later how Dolores ended the episode! I don’t know who to blame: my inquisitiveness or that guy who couldn’t hold himself. It kind of feels stale when you know the ending.

    Finally, I finished weeding through the multiple narratives across the episodes, and the flashback narrative merged seamlessly with the current one. That got me thinking about using that as an analogy and maybe understand whether the concept of ShiftLeft is really actually a “narrative within a narrative” of #DevOps.

    Ok, so what is ShiftLeft?

    I came across the term at my fav go-to source, @devopsy.kinsta.cloud, where Alan wrote on how much to Shift Left and its diminishing returns, and another very old article on the continuous testing approach, which primarily seeks to test early and often, to catch defects, yeah, early!

    Then it was thrown at me in the context of identifying repeatable activities, often manual with consistent results, performed by the Support & Operations team, and moving that work left from Level-3 all the way to L-0. Empowering the help desk, I would say!

    In both the scenarios, the underlying objective was to catch defects and or reduce life support activities early in the cycle (below) which costs more and more as we move up the food chain, which in turn makes a release and subsequent maintenance cheaper.

     

    As Alan also referred in the same article to the IBM Study, it’s almost 100 times more expensive to fix an eventual production defect which cost about six times more if caught at Implementation stage or about 15 times more if found during testing. Thus, “Early identification and prevention can improve reliable software delivery” would be the mantra.

    Now, to the crux of this article, Is ShiftLeft not what we aim to achieve via DevOps?

    DevOps basically creates a collaborative, non-siloed cultural change to bring both Development and Operations together early in the game.

    Throw in a bunch of tools to facilitate CI/CD, pull in the security guys early, expand automated testing coverage, include change and release—who are the catalysts, as I call these functions in my earlier article here—squeeze in a few monitoring tools and dashboards and voila! You have a complete and mostly successful DevOps team to dish out the most appetizing feast of quality software repeatedly.

    A table for two (Dev & Ops) has become a party of six! Yaaay!!

    So, from DevOps standpoint, ain’t we doing the same as ShiftLefting (If I may coin the word) with the resulting cost effectiveness, more efficiently?

    We do when we as a team make shorter and frequent sprints; gain insight, not just on how to code better or leaner but also by learning from the masters in Operations who run that software on the infrastructure they monitor and support daily. Or perform the VA scan early, or push a scenario as unit test instead of regression, and/or raise flags on potential capacity challenges.

    So again, isn’t DevOps team already doing that ShiftLefting as part of its charter?

    There is a limit to moving things left, but then, if your organization is still doing wireframes under a waterfall or is in the process of setting up a DevOps team, then this approach could be a stop-gap arrangement, an eventual merging of a narrative within a new narrative.

    But then my argument would be, Why stop-gap and why not a full frontal? Why not embrace the DevOps model, which comes preinstalled with all the ShiftLeft possibilities?

    Even if we are aware how an episode ends, procrastinating raises the risk of being left behind in the race to deliver quality, cost-effective product quickly.

    Technology is redefining businesses: Transportation is an app and soon probably will be driverless; banks are rebranding as tech companies in FinServ business; a fridge can reorder milk; trash cans can be Wi-Fi-enabled! Ah, the internet of things. Is Alexa listening?

    My other relevant posts are here and here.

    Thank you for the time.

    — Abdul Salaam Khurram

  • How Much QA is Enough in Software Development?

    How Much QA is Enough in Software Development?

    People working in quality assurance as testers, managers or engineers know what it’s like having to justify their existence; it’s been part of the job for many years. QA experts have had to fight the perception that what they do is a bottleneck, overkill and it slows everyone down.

    Finally, however, QA is getting the attention that it deserves. Companies are gradually investing more in QA skills and tools. This began with the onset of Agile and DevOps—methodologies which have now infiltrated mainstream software development practices everywhere. Since those practices focus on speed, efficiency and quality, engineering leaders have grown to understand that great products are not achievable without adequate QA checks and balances. Another risk is the unflinching nature of high user expectations: Thanks to social media, one user complaint can quickly escalate into others across the web, creating an uncomfortable perception problem for the software provider.

    Robust testing practices can prevent egregious flaws in the software that result in poor user experience and performance—or, worse, a breach. True, testing can sometimes get in the way of speed. Yet no matter how fast you release a product, if it doesn’t work well, no one’s going to buy it, much less use it.

    Nevertheless, we can continually improve the QA practice to fit into modern development teams’ workflow. Testers need to work with developers and master automated testing, while developers need to sometimes engage in testing; everyone should think about the appropriate level and amount of testing. You can have too much of a good thing.

    The Right QA Balance

    The below ideas can help your organization reduce conflict between QA aficionados and those who would rather code and release as fast as possible.

    1: Assess, analyze and report. Developers and testers alike are highly technical; they like numbers. Therefore, the more that the QA team can quantify the risk of an application, provide evidence as to the effectiveness of current QA practices and use metrics to show the need for more QA resources when warranted, the faster that everyone will get on board.

    There are plenty of tools that can deliver common quality metrics such as total number of defects, defect leakage, test coverage, test pass/fail rates, test completion rates, code coverage and application uptime. It’s important to share and discuss these metrics throughout the development team and with product and business managers. What constitutes high quality for the development team may not be the case with executives who are closer to the end user/customer.

    2: The gut check. After metrics, take into consideration the comfort level of your team and company concerning product flaws. In regulated spaces, or for applications that collect sensitive private data, the tolerance for bad code and security gaps is usually low. Gather feedback from different perspectives and roles in the organization for a balanced perspective on the QA practice and areas for improvement.

    3: Get people involved. Quality assurance is no longer a separate activity from development; especially when you are practicing Agile and DevOps, the responsibility for quality is shared. Therefore, developers, business analysts and operations managers all should be doing their work with an eye to quality, delivering feedback regularly to the QA team. Provide opportunities and incentives for any team member to attend events and training related to software quality.

    4: Streamline testing. Test automation tools are helping product teams be more comprehensive in testing, by automating low-risk test cases so that testers can focus their exploratory and manual methods on high-risk or new features. This is a positive development, although testers must take time to learn how to use the tools. Some other ways that can help teams be more efficient in testing include:

    • Conducting more testing early in the sprint. This avoids the 11th hour disaster of finding a major flaw right before the release deadline.
    • Per Agile methods, dividing development tasks into smaller chunks with team members from different disciplines to ensure testing is being done all the time and with visibility by all.
    • Using test driven development (TDD) or behavioral driven development (BDD), which requires that QA, developers and business stakeholders meet at the beginning of a project to agree upon requirements and test cases. Developers then write code against the test, until it passes. New tools are helping bridge the gap by writing requirements and user stories in plain English.
    • Outsourcing part of the testing load during a major project, which can help increase coverage and reduce stress on your team. However, understand that too much outsourcing will undermine any efforts to develop QA as an internal best practice and “mindset” for software teams.

    If your team is committed to QA as a best practice, not an afterthought, those at the top will need to invest properly in the right tools, training and people to get results. Misconceptions about QA still exist; contrary to the view of some DevOps experts, we can’t (and shouldn’t) automate every test. As well, even though developers are now testing the code they write, you still need experienced testers on your team. With the pace of innovation and the high stakes in the marketplace, organizations need more—not fewer—people engaged in testing.

    For testers who love their job, as many of them do, it’s an exciting time to be in the field. Companies increasingly are seeing the value of QA and testers continually are gaining more tools and tactics to excel at helping their organizations produce really great software.

    About the Author / Kyle McMeekin

    Kyle McMeekin is Senior Sales Engineer at QASymphony. Prior to QASymphony, he was a programmer analyst at Cognizant Technology Solutions. Connect with him on LinkedIn.

  • Test Driven Development, Continuous Test and Code Quality

    Test Driven Development, Continuous Test and Code Quality

    We’ve gotten a lot of good out of agile development methodologies. I’m never a “One True Way” kind of person, but agile has a ton of benefits that development needed (and Ops increasingly needs). Extending that to the Dev side of DevOps is painless, since much of agile is wrapped in DevOps clothes these days.

    From my perspective, the most impactful bit to come from Agile was test driven development. By writing the test to fulfill the requirement, then writing the code to pass the test, our small team (I was on a team of developers with no Ops people supporting us—or more accurately, we were the Ops team when I first used TDD) needed the assistance it provided. Without dedicated QA staff, we were able to perform as rigorous a test suite as our changes warranted, simply by running the tests that we’d used for development.

    Of course, this is old news to some of you, TDD has largely been consumed in the larger Agile/DevOps movements and you’re at it daily. But the shift to having developers do more of their own testing is something that I’m still pretty iffy about, to be honest.

    Devs Aren’t Necessarily Testers

    While it can work, devs certainly can understand the weak points in their code, I would argue that the person writing the code is not the correct person to write the tests for it. Particularly as you get into more the integration phase of testing, and the complexity of code inter-relationships heats up. Some developers will do well at it. My experience is, many will not. Why? Because if they don’t know of a weakness in their solution when writing the solution, they won’t know of the weakness when writing the test case. And it is a truly rare shop that has 100 percent code coverage for testing, so those weaknesses could end up being time bombs.

    Simply put, QA needs to work with Devs to write the tests. Two heads are indeed better than one when looking for faults with solution designs. But the reality of development often doesn’t allow for this type of cooperation. When you’re on a sprint and this one bit is slated for half a day, setting aside time to get together can be difficult for a problem the developer has already written the test for and solved in their head. After all, the whole point of Agile and DevOps was to speed delivery, and making the simple more complex isn’t generally part of that equation.

    Have a Plan, and Use it

    I know of a couple of shops where integration testing is pretty much an afterthought. One has a QA team that fights to make certain it is done and done well (though still as an afterthought), rather than cobbled together as a PoC (I was thinking Proof of Concept; take either use of that acronym, both are accurate) not too long before release. The others muddle through, figuring out what doesn’t work in fits and starts (too often in production), rather than following a stream. The result is, unsurprisingly, missed deadlines and ever-extending sprints.

    For all of the good that DevOps in general, and CI/CD in particular have brought us, there are a few areas—one of them being testing—where speed and/or quantity has replaced quality. We run tests a LOT. But are they the right tests? For unit tests in a TDD environment, yes—for lines that have test coverage. Outside of that narrow space, I think the answers are much more iffy, and it depends upon a lot of things.

    The best hedge is to have QA involved in writing the tests for TDD environments, even for small bits. It’s not an overhead if it shortens the overall timeline to delivery; even if it slows a piece here or there, it’s the timeline to finished product that really matters, not the timeline of a given piece of it.

    If you don’t have a QA team—as many shops do not—I recommend devs doing it for each other. That’s not to say that QA is not needed, but it does utilize the “Two heads are better than one” mantra, and keeps the ideas flowing. Otherwise, the Ops members of a DevOps team are great for that. Historically, production woes fell on them, so no doubt they’ve honed their skills in detecting problem areas.

    Don’t Be That Person

    No matter how you go about it, do not go light on the testing. Do TDD, and keep it up. Make every change request start with test modification, and then fix what fails. Document the tests, since they’re code also, and stressing what is being tested and why is as important as the code talking about what it’s doing and why.

    After all, you don’t want to end up with the difference between ‘==’ and ‘>=’ placing you in the headlines for a week, mostly with bad news. That difference is an obvious TDD test case if ever one was written; don’t be that person.

    Continuous testing works, test driven development works. You just have to think about it when implementing them, and stick to a plan that tests beginning to end. It’s about quality code quicker, not about any code quicker. Note that I’ve been focused on the process end of testing, not the smoke/load/regression end. Those are important too, but their issues are different, so I opted to throw them into a separate blog.

    So go forth, iterate quickly, but test thoroughly. I doubt most of us will achieve 100 percent code coverage anytime soon, let alone 100 percent integration nested coverage … But test what’s important, and bring what’s important to test.

    — Don Macvittie

  • It Is Not All Choice and Speed in DevOps

    It Is Not All Choice and Speed in DevOps

    One of the stumbling blocks starting to show in the agile/DevOps world is that speed of delivery and developer choice are asserting themselves as the primary worthy goals.

    I’m a developer. I have that “look what I did” attitude when I tackle a particularly tough problem and I love it when an environment lets me choose a toolset that helps me achieve my goals faster (or lets me play with tools I have been eyeing for a while). I want to solve problems quickly, and have no issue with stand-ups to show my progress.

    Creative Commons Attribution: winnifredxoxo, flicker
    http://www.flickr.com/photos/61056899@N06/

    But we’ve lost sight of the whole longevity issue, and far too many of us have lost sight of the whole “necessary restrictions” thing, too, which is related. The security environment did not change just because your team went to DevOps with Agile, and what must be supported to keep your cool app up and running didn’t change—at least it didn’t change conceptually. And that’s where we start seeing issues.

    Lori and I were talking this morning, and I came to the conclusion that Ops/DevOps teams would be warranted in saying, “Okay, you want that cool gizmo new thing that is totally outside our architecture, YOU support it.”

    Think about it. Between Agile and “Developers are Kingmakers,” we have set different Dev teams using different environments, from languages to databases to deploy targets to even load-balancers. And while there is a lot of talk about “chuck it over the wall” being gone, the truth is that the responsible Devs will be responsible for code in production, but not the infrastructure that we’re allowing them to inflict upon the Ops/DevOps team. And all of this is done with “timely delivery” as a mantra.

    The problem today is different for security. In security, it is seen by many in the DevOps crowd as inherently slowing down the process. And this, in our current fever dream of rapid delivery, is seen as inherently a problem. Time to step back and see where security can benefit from and add to DevOps. That movement is, thankfully, starting, but needs to be more prevalent. My blog about DB monitoring is an example of DevOps benefitting security, and increasingly the SecureOps movement is showing promise in integrating security with DevOps.

    Do yourself a favor. Look around your IT org. Are there apps running that are 20 years old or older? If yes, speed is still important to business agility, but you have to acknowledge that every unique database/load-balancer/whatever that you introduce is potential technical debt. If you’re working on a core system replacement and have decades-old apps running in your world, chances are pretty good that 20 years from now, someone will be figuring out what you were doing.

    Don’t make it harder than it has to be for future-you. Stick as close to standards as your development effort will allow. It’s been a long time since companies successfully enforced database standards, for example, but having a first choice means that support and training can focus on the first choice. And that 20 years from now, some poor new developer will be able to go “Oh, it’s another MariaDB app.” instead of “What the heck is SuperFastDBForMe, and why are these queries written in pig-latin?”

    And don’t make it easier for hackers by demanding that security not slow down the process. Encourage them to move faster, but it’s not all about speed. In fact, both security and ops were never about speed; they were about thoroughness. The environment and the apps that run in it need to be as secure as is reasonable, and the production environment needs to be as stable as possible. DevOps tends to find itself at odds with both of these facts, sublimating them to the desire to be “agile.” Yes, automation helps with stability, and I’m a huge fan of that. It also helps with the knowledge crunch, but it is more of a shifting of tribal knowledge than an elimination. Someone, in the end, has to be able to modify tools to face the new reality, and that person needs much the same knowledge that they need to do it manually.

    The goals and results of DevOps are great, but if you’re working for an enterprise that will have long-lived apps, just make certain you’re not sacrificing more than you’re gaining. Business agility matters, but so do both security and long-term technical debt.

    Balance, as always, is the way forward. Strive for it early, or struggle for it later.

    — Don Macvittie

  • CD 101 Podcast: The What, How and Future of Continuous Delivery

    CD 101 Podcast: The What, How and Future of Continuous Delivery

    Now that we’ve arrived at Episode 4 of our CD 101 Podcast, let’s address some basic concepts—like the what and how of it all and the future.

    Wait … perhaps we could—or should—have touched on these key topics earlier. Or not. At least not the future part? Anyway …

    While this (mostly) backwards-looking orientation may seem a bit odd, I’d argue that we’re actually navigating the conversation on precisely the correct path. Let’s rewind for a moment to look at where we’ve come from.

    Episode 1: We highlighted many of the fundamentals of this thing called CD, yet we did so from more of a conceptual and best-practices standpoint. While the ideas and processes involved do resonate in the real world, the topics we cover in this episode are bit closer to the water-cooler conversations we’re hearing about CD as it grows in practical adoption.

    In Episode 2: We covered the whole how do you “realize agility” piece of the conversation. This is important and a precursor to some of these other basic questions. The entire process of digital transformation begins with getting agile and you can’t truly adopt continuous delivery without doing so; therefore, this was another good place to feed today’s discussion.

    In Episode 3: We looked at the prospect of re-inventing the “Modern Software Factory” with agile development and continuous delivery. OK, so, perhaps we were getting a bit ahead of the basics here, but I’d maintain that this is an important piece to consider upfront that builds on these other building blocks. As with the topics covered in Episode 2, it is critical to understand the big picture challenges and benefits before you begin to get your hands dirty.

    And now for some nitty gritty.

    In Episode 4, we hear from Shamim Ahmed, CTO of Continuous Delivery Solutions at CA Technologies, as he answers the nine burning questions he most often hears in speaking to enterprise practitioners as they look to employ DevOps and Continuous Delivery practices in the real world.

    So, while these previous conversations were material, this is a crucial set of topics to consider as you invoke your own strategy, including:

    1. What is the difference between DevOps and continuous delivery?
    2. What are the key impediments/obstacles to achieving speed/quality in CD?
    3. How do we overcome these obstacles?
    4. How do we get started on the CD journey?
    5. What are the key top-level metrics you want to track in a CD program?
    6. What org changes do we need to make to implement CD?
    7. What the key capabilities we need to implement?
    8. How do we integrate all the different tools we have in our environment?
    9. How will the future of continuous delivery evolve?

    OK, that last part is a bit more postulation on our part—though we like to think on the practical side, based on the answers to all these other important real-world adoption dynamics.

    Tune in and hopefully learn a good deal about where your approach stacks up, and needs to evolve.

    As always, you can find said Episode 4 of the Continuous Delivery 101 Podcast by:

    • Listening to it on your computer on YouTube
    • Downloading it from iTunes store
    • Downloading it via the Podcast App by searching for “Continuous Delivery 101 podcast”

    Enjoy!

    — Scott Edwards

  • The Postman Delivers More

    The Postman Delivers More

    Most of us use Postman during API development. The company claims 3 million users, and as one of them, that number doesn’t raise my eyebrows any. But if you are like me, you’ve figured out collections, how to back up your data and perhaps looked at tests, but not done much more. As a tool for API development, Postman is powerful and useful, but that means many of us don’t look very far beyond how quick and easy it is to set up an out-of-browser test environment to evaluate request/response.

    We use collections to separate projects, but really only as a way to group calls together and keep a single copy of each call close to hand while the history list scrolls through multiple copies.

    But for DevOps, there’s more. And they’ve just added to it again.

    Packing in the Features

    Groups can be used for grouping calls with tests, for example, and a command line tool can perform automated testing. If, like us, you use Postman to test as the API is developed, and use groups to keep those calls pulled together in one place, then testing from the source that already contains your entire API makes sense.

    And as of this week, Postman also includes the server side. Postman Mock service allows you to mock up the API calls and specify a valid return value. That means client-side developers can use Postman Mock for testing their app/page, while server-side developers are using the original Post

    man to develop the APIs that Mock emulates and testing is running tests from the real-world APIs that already are in Postman for development purposes.

     

    Dev, DevOps, Testing and Ops: Something for Everyone

    So a quick diagram shows that the entire development process has a bit of Postman in it if that’s your tool of choice:

    While all assist with agile development, for DevOps, the pieces that are the most beneficial are Test combined with Groups and the new Mock service. As long as the Mock service is configured for a group of calls, test can call them and the logged output can reflect that the test succeeded but ran through the mock service. Once the service is completed and the group is directed to point at the server, testing will run tests against the real API. Using Newman, the command line tool, these tests then can be integrated with your build toolset.

    The Mock service can take imports from SOAPui, if you are converting from SOAP calls or use SOAPui for REST and are moving to Postman. I’m told the Postman toolset also supports SOAP calls, but truth in writing, I’ve never even looked to see how that would be done. I’ve used both tools, but never on the same project.

    There’s a Cost, but There’s Also Value

    This next bit sounds a bit salesy, and I try not to do that. But for the price, there aren’t a lot of value-add tools out there that deliver this kind of value. Lori and I are talking about going Pro, as we both use Postman in projects. And considering we’re regular users, I figured if we didn’t know about most of these features, many of you wouldn’t, either.

    Mock is part of Postman’s Pro package, along with sharable team collections, and API documentation tools. That means it will cost you a little bit of money, but if you’re on a team and have been cobbling together a way to share collections, team collections and associated collaboration tools are worth the price—particularly since built-in auditing will show all of the changes made to Postman collection items. That means fixing a change-based error on the Postman side is easier.

    As of this writing, the cost for Postman Pro was $8 per user per month. My guess is sharing groups (or worse, each team member re-creating/maintaining them) costs more than that per month in many cases. Consider that documentation can be generated and shared to a private URL and the fact that Mock service enables parallel development, and $8 sure seems like a steal. Finally, Pro offers API monitoring, so performance and uptime can be tracked to better adjust to the realities of production. There’s more (such as integrations with some great Dev/Ops/DevOps tools), but I’ll stop.

    Summary

    Design, Development (both client and server, in parallel), Test, Monitoring … as long as you can get it deployed, Postman can help with the rest. Postman Pro has a reasonable price for the functionality it offers, and the introduction of the mock service makes agile development, tight timelines and consistent “start to deployment” testing easier. I’d venture to say it is worth taking a look at. And if you’re developing APIs without it, pop out and get the free version, you’ll quickly find why so many of us use it.

    You can find out more about the various editions of postman on Postman’s Product page.

    — Don Macvittie