Tag: software development

  • What Developers Should Take Away from the Iowa Caucus App Debacle

    What Developers Should Take Away from the Iowa Caucus App Debacle

    In the wake of the recent Iowa caucus, vote-counting software has fallen under suspicion after a smartphone app built by Shadow, Inc., brought chaos and delays to the first 2020 Democratic caucuses. Stopgap measures, such as Nevada officials canceling the application’s use in its upcoming caucus, are not enough to ensure vote tallies the electorate can be comfortable with. What’s needed is to correctly identify and fix the underlying problem. 

    The flaw does not lie with a lack of end-user sophistication, dataset complexity or inappropriate app security. Remember, government officials around the world are transforming urban areas into connected, responsive smart cities. Students are live-streaming interstellar space flights to Mars and Saturn. Uber drivers navigate past traffic jams, while seniors use apps to make time-sensitive healthcare decisions and adjust their retirement portfolios. People of all ages and experience levels do this and more, on the fly, with handheld mobile devices. 

    The real problem lies in the way leadership teams drive software delivery. The Iowa app failed for a number of reasons: It was released with no product testing after a short, two-month development runway conducted under a cloud of secrecy. It was not built to scale. Worse, there was little to no input, transparency or communication among the application’s most important stakeholders—the Shadow management team, the software developers and the end-users, in this case, precinct captains running the Iowa caucuses and party.

    The IT professionals in charge could have avoided these problems by following three simple principles of successful software development: 

    • Collaborate With All Stakeholders: Applications are no longer stand-alone widgets. They are digital experiences. With even the most sophisticated design, experiences that don’t serve their target audience are failures. A development process that prioritizes collaboration with end users can prevent such failures. Feedback from those with a vested interest in the outcome would have quickly raised red flags for developers. In Iowa, collaborating with intended users would have highlighted a need for accessibility training and connectivity issues, since the app was deployed in buildings with high-speed internet along with remote, off-grid locations. 
    • Build to Scale: Every developer knows to build for performance at an order of magnitude larger than you expect. This includes ensuring performance on the widest possible range of handheld devices and phones. From a developer’s perspective, the phone model will impact application performance due to different hardware specs. It’s imperative for designers to build apps that are native, so that all users, devices and performance profiles will function as intended. A useful software solution should reap success, not failure, when more people use an application than initially expected. 
    • Be Transparent: Valid concerns over security risks and election integrity unfortunately led Shadow’s team to prioritize secrecy over transparency. That didn’t work. Problems are not detected and solved when siloed or when small groups of people work on a solution in secret. Proper vetting and testing falls by the wayside. The risk that the app will not be suitable for its intended users increases. The app may have additional, exploitable security vulnerabilities that can only be found when teams ensure proper openness and transparency during the development process. Proper governance and guardrails that protect core data mitigates the security risk for election-based applications (and software in general). There’s little to be gained and much to risk by closing off input and vetting at any point during the full application development cycle. 

    Developers Move From Stressed to Empowered 

    One simple, effective way to solve or avoid these issues is by adopting an enterprise-grade, low-code software development platform. The “low-code” label refers to visual, model-driven software development, which employs an unambiguous visual language throughout the application development lifecycle, enabling nontechnical stakeholders, end users and professional developers to work together. 

    Such a collaborative platform can surface issues with senior leadership before it’s too late. Feedback from citizen developers—that is, not citizens in the voting sense, but rather, non-traditional developers—can capture leadership’s careful attention, precisely because the feedback is from nontechnical users. The right low-code platform would have enabled the Iowa caucus volunteers and stakeholders to collaborate on every step of the process without slowing it down.

    Another benefit of a robust low-code platform is having more people on deck to meet product goals with tight deadlines. Insights from citizen developers with unique backgrounds can also help when testing applications. With suitable governance controls, low-code functionality speeds up a developer’s ability to deliver software without taking away their ability to dive deeply and extend functionality with coding when needed. 

    Market analysts have evaluated the strengths and merits of different low-code platforms. In multi-experience use cases—such as Iowa’s, particularly because the eyes of the world are watching—it’s important to use a low-code platform that enables applications that are truly device native. This allows the software to take advantage of core device capabilities, typically delivering better performance and supporting the interactivity users are familiar with. Additionally, continuity online and offline would address connectivity issues in areas where signals are weak or nonexistent; the data can be properly synced when a signal is available. Lastly, a robust low-code platform provides tools to enable apps to scale automatically, avoiding the coding work required to allow an app originally built for 1,000 people to service 100,000 or a million. 

    An Urgent Role for Software Developers

    When roughly half of all eligible voters sit out general elections, talented developers should not lose sight of the many ways technology can aid and expand civic participation. Eventually, we may live in a world where election-based apps for registered voters are accessible and secure, regardless of the user’s device, age, location, physical ability or educational background. Building in redundant systems of paper-based vote confirmation and other offline systems will add integrity, inspire greater confidence and drive adoption of these innovations.

    What happened in Iowa is, unfortunately, not an isolated case. These complications are endemic to how most leadership teams drive software delivery. Development teams of all types will continue to run into similar issues until their leadership makes transparency, accessibility and scalability a priority by investing in the appropriate resources. Organizations can leverage the strength and impact of fully robust low-code software development platform to that end. 

    Since the introduction of low-code in 2004, platform capabilities have matured at an accelerated rate, moving to widespread adoption for enterprise use. During the same period, elections have increasingly relied on software-driven electronics, leading to consistent issues. Citizens casting ballots is one of the most sacred mechanisms of our republic. Any software used to tally votes must be bulletproof. Truly open and holistic low-code software development platforms can make this happen by proactively bringing people together to make sure software works for those who need it most.

    — Jon Scolamiero

  • Observability-Driven Development: From Software Development to DevOps and Beyond

    Observability-Driven Development: From Software Development to DevOps and Beyond

    In today’s world of cloud platforms and applications, typically delivered as SaaS (Software as a Service), it is critical for organizations to ensure efficient collaboration between software development and operations teams. The software development team codes the software application or platform, while the DevOps team streamlines and automates the entire production lifecycle, from development and testing to deployment.

    Today, applications are updated anywhere from weekly to daily, requiring fast turnaround. In the old days, every change request would go to the different stages–software design, development, testing and release. This can’t be sustained in an agile environment where updates must be as close to instantaneous as possible. 

    How can organizations achieve this, and where does observability fit into it all?

    What Is Observability and ODD?

    Observability-driven development (ODD) adds another layer to software development by encouraging the development team to think about the application availability and uptime throughout their development process and similar to unit-testing development, wrap their code with more verbose logging, metrics and KPIs. This gives the IT operations team more data on the application and improves overall observability, allowing an organization to detect predictable and unpredictable permutations of failures that may occur in the future. ODD observes the behavior of a system throughout the entire development cycle in order to learn what behavior is normal and abnormal, and to detect potential weaknesses.

    Observability provides everything with a real-time status of a product–this includes software developers, DevOps and any other department that requires it. Each role has different needs. Software developers need to know why a certain feature has to be added or developed in order to understand its business value and prioritize it over other features or updates. If a certain feature is likely to require many future updates, this also needs to be taken into consideration in the development phase so that the code can be adjusted easily without negatively affecting DevOps teams when they test and deploy the feature.

    These are just examples of things that must be taken into consideration during the development process. Organizations must be able to create transparency and effective communication between the different departments and roles.

    Inefficient Collaboration in the SaaS Market 

    DevOps need to ensure zero downtime, fast updates, constant monitoring and more. This can be done by maintaining an operational mindset during the development process. Traditionally, developers write the code, QA do the testing and operations take care of deployment, but there needs to be more collaboration between these three stages with each doing their part with an understanding of everyone else’s roles and needs pertaining to the product. When this isn’t done, numerous problems arise. Operational inefficiencies can ultimately lead to a delay in delivery time and even affect system availability. When there are multiple teams involved, especially in different locations and even time zones, there can be ineffective communication and hand-offs, particularly between software development and operations teams.

    Meanwhile, ad hoc changes must be implemented, negatively affecting product stability and release cycles. Miscommunication and inefficient operations lead to confusion over who is responsible for what task, feature and target. These are all issues that can result in significant downtime, losses, wasted resources and frustration within the organization. Effectively streamlining the software development cycle using full transparency and seamless collaboration between development and operations solves these problems.

    ODD Can Improve Your SaaS Product

    A strong proactive observability platform can predict and address issues before they occur, thus increasing effectiveness and speed when it comes to updating and tracking changes, as well as releasing new features. It simplifies complexities and management, and allows to automate operations so that organizations can maximize efficiency.

    Benefits of ODD include:

    • Common goals.
    • Shared metrics.
    • Execute on goals and recognize results.
    • Performance guidelines.
    • Application governance strategy.
    • Alignment between IT and business perspectives.

    Employing ODD when developing cloud applications is critical. This is due to the highly distributed nature of cloud architectures, increasing the chances of a failure exponentially compared to traditional systems. As a result, developers should be educated to think and implement observability to create highly available and resilient applications and systems.

    How to Integrate ODD in Your R&D Practices

    Today, development teams and practices are under constant pressure to meet scrum deadlines and release features on time. They are measured on their code effectiveness and meeting deadlines, while ODD adds another layer of measurement of serviceability and observability metrics to their considerations. 

    Implementing such an approach requires organizations to provide their development teams with the right toolset and knowledge on how their code affects the SRE and IT operations teams. From a technical standpoint, development teams should be familiar with all levels of monitoring for application infrastructure (for example Kafka, RabbitMQ, Kubernetes), application performance metrics (user experience, transactions per second, etc.) visualizations and their respective remediation processes. 

    ODD culture changes the way organizations think about their development process, and essentially injects an operational mindset to their daily practices and processes which should increase the availability and performance of their cloud application while streamlining collaboration and work processes, and improving teams’ satisfaction and productivity.

    — Avi Shalisman

  • EP 1: A Brief History of Testing

    EP 1: A Brief History of Testing

    DevOps’ sister site Digital Anarchist has launched a bi-weekly video series and companion monthly roundtable on DevOps, Automation, CI/CD and testing. Featuring leaders in all of these areas, we explore the challenges and issues software delivery and IT teams face every day. How do we go faster, smarter with better quality? DevOps Unbound!

    The video is immediately below, followed by the transcript of our conversation. Enjoy!

    Transcript

    Alan Shimel: Hey everyone. Welcome to the very first episode of DevOps Unbound. I’m your host, Alan Shimel and joining us for this initial episode of DevOps Unbound, I’m proud and happy to introduce you to James Bach and Grigori Melnik. Grigori, James, welcome to DevOps Unbound.

    Grigori Melnik: Hey everyone.

    Shimel: Before we jump into things, let me give each of you an opportunity to kind of introduce yourselves, your backgrounds, your positions, etcetera, so that our audience kind of knows a little bit, if they don’t already, who you both are. So, Grigori, you’ve already been a guest in our show, so if you don’t mind, I’m going to ask James to go first. James? Introduce yourself. Go ahead.

    James Bach: My name is James Bach. I’m a consulting software tester. I started as a developer in my teen years after I dropped out of high school. I’m the proudest high school dropout you will ever meet. I later wrote a book about dropping out of high school and my particular self-education method, but my big opportunity came when Apple computer hired me to be a test manager, and at the time, I was the youngest manager in the R&D division of Apple computers.

    So that’s kind of something I’m proud of. And I fell in love with testing and I decided to devote my career to testing. I studied the epistemology of testing, the mathematics of testing, the sociology of testing. I am a bit obsessed with testing.

    Shimel: To say the least.

    Bach: And I have a book about that called “Lessons Learned in Software Testing” with a couple of coauthors. And now I teach and consult.

    Shimel: Excellent. And you know what, high school dropout or not, you’ve done pretty darn good, so congratulations and not saying it’s an example for our younger people in the audience to necessarily emulate. Of course, James grew up –

    Bach: My son did better than me. He never went to high school, so I’m –

    Shimel: Really?

    Bach: He’s already outdone me.

    Shimel: Absolutely. Very good. Grigori? Why don’t you tell them a little bit about yourself?

    testingMelnik: Sure. So I’m Grigori Melnik and I am the chief product officer of Tricentis, focusing on building the testing tooling of the future, testing tools of the future. Previously, I guess I spent almost two and a half decades focusing on the developer audience and building all kinds of platforms, Microsoft and Splunk and MongoDB and you know, always been kind of customer obsessed in the sense of developer obsessed quite a bit, making sure that whatever we were focusing on would help drive the productivity and alleviate some of the pains that developers in our target audience had.

    And while a few of those things that I’m particularly proud of, my team at Microsoft was actually the first one that built the very first auto-scaling for what was back known as Windows Azure, now Microsoft Azure, but also focused on common patterns and practices in implementing the crosscutting components that represent crosscutting concerns that later would make into the .NET framework.

    So again, a lot of work identifying those patterns, capturing them, externalizing them, testing them out and then helping drive their use. When it comes to testing, I first kind of discovered it in my days in academia when I was doing research and teaching at university, and it was quite interesting because testing was predominantly viewed through the lens of formal method groups. It wasn’t really viewed as an intellectual activity of discovery, as an intellectual activity of exploring or experiencing something.

    And those were actually the days when I’ve encountered the group of testers from the context driven school of testing, including James and then we kind of kept in touch ever since, even with me gravitating away from the testing more into the developer world and now doing like the full loop of coming back to testing and I have to tell you, I felt, when the opportunity came, I felt that the world of testing, at least from the perspective of how the discipline itself has advanced would be way further developed in the last decade than I found after kind of leaving it for a while.

    So I’m now on a mission of kind of reigniting that whole discipline and putting it more on a pedestal and giving it more view and empowerment and leveling up all around. So, yeah.

    Shimel: Excellent. So, thank you both, gentlemen. And I’m Alan Shimel, I’m the editor in chief, CEO for MediaOps, the folks behind devopsy.kinsta.cloud, Container Journal, Security Boulevard, TechStrong TV, et cetera. And just quickly before we jump into today’s agenda, DevOps Unbound though, today’s focus is on testing. It’s not just about testing. It’s really about DevOps kind of unbound, setting us free, being able to leap tall buildings and bound into space, sort of freeing ourselves of gravity, whatever you want. It’s about speed. It’s about acceleration. Right? It’s about automation in a lot of ways.

    And each – we’re going got be doing these every other week and each episode will focus on particular aspects of this unbound nature, if you will. Today’s is testing. And then once a month, we’re also going to have a roundtable with about six or eight people of Grigori and James’ stature and we’ll open it up to the public for live audience questions and discussion. So I’m really looking forward to that. DevOps Unbound is sponsored by our friends at Tricentis and we thank them for it and with that out of the way, let’s jump into things.

    So, Grigori, I wanted to start off – obviously, you and James take very different paths to our – and both of you have been very successful in your lives and careers. So it just goes to show, there’s more than one way to skin the cat. Right?

    But, let’s take a look – look, I’m probably older than both of you, though not that much I’m sure, but let’s look at the history, the historical context of where you guys come from to be sitting here today and how that, you know, vis-à-vis kind of the whole testing industry, the whole testing, you want to call it a profession, the guild, the whatever you want to call it, right, how has this evolved over the years and let’s put some context in with timeframes. James, when you first went to Apple without saying you’re old, what kind of – what were the timeframes for that?

    Bach: Well, I am old.

    Shimel: So am I.

    Bach: But you’re older, which is nice. It’s nice to be a young guy in the room again.

    Shimel: Uh-huh. I can’t tell you how well that makes me feel, but go ahead.

    Bach: First, 1987, May 21, 1987 was the first day that I ever professionally tested something as a tester. Before that, I was doing videogame development and I had done informal testing, and I hated testing because testing just got in the way of me shipping my product. So, I didn’t like testing until it became my job and then I realized I’m perfect for testing ’cause I love complaining. And I’m much more interested in finding how things don’t work than to figure out how to make them work.

    Shimel: That’s interesting. You know, my background’s in cyber, back then we called it InfoSec, and in InfoSec, most of the people got into it ’cause they liked to break things. Right? Yeah. Same thing. Grigori, how does that timeframe-wise and everything else, how does that match up with your own experience?

    Melnik: So I like to actually build things. _____ you guys who are breaking things. And occasionally I break things and actually this also speaks about this overall I guess optimism of most developers who think that what they write is perfect and always underestimate the amount of buggy code that we put in there.

    But to me, if I think about my first day when I actually had my job as an engineer, actually, it was September, I think 5, 1990, so a few years after James and at that time, I was actually writing a C-library for a PC converting the data from mainframe into the visual format and I had to deal with all kinds of quirks of Fortran and the 80 characters limit and the asterisks for, you know, for bringing – the continuation of the line, all kinds of exciting stuff. But back to the beginning, to make sure that I don’t go too far on the tangent, we actually didn’t have, for a very, very long time – again, I’m giving you my angle into the testing, if you will.

    We actually didn’t have a dedicated thinking about testing as an activity, as a pursuit of excellence. It was just something that you kind of were doing and then at some point later, you’ve heard and learned that oh wait, look, there’s a QA department, apparently, right? And they’re the ones that were mostly viewed as gatekeepers or police of some sort. Right?

    And then again, if I think about again, once I got more into the research and academia and studying testing specifically because, you know, I’m probably one of the very few people who has testing in the title of my PhD dissertation. But, I remember looking back and thinking, “But where does this come from?” Right? Where did actually software engineering come from?” Right? And of course, this is where the history of the conference deck, you know, after the _____ _____ where all the stalwarts of at that time computer science and well, not even computer science, mathematics departments because that’s who were doing all the programming back then at the point the software engineer discipline and later on the association of debugging, right?

    It was the infamous story of how the moth was found in Harvard University’s scientific lab and the term bugging, you know, was coined. I almost think like the – in the early days, it was very much associated with this era of debugging. Right? Then later on, the era of the ability to demo, the fact that the software was matching the stack or the requirements, I think it almost feels like there were at least a decade when people were fixated and some would argue that it still continues.

    Shimel: What decade was that?

    Melnik: Hmm?

    Shimel: What decade are you referring to?

    Melnik: The debugging or the – the debugging I’d say –

    Shimel: Are you talking about the 70s? Are you talking about the 60s? Are you talking about the 50s?

    Melnik: I’m talking about 50s and 60s. I’m talking about the 50s and 60s as almost the era of debugging the way I viewed it. Then, after that, the era of more like 70s and 80s where you were focusing more on matching the spec and demo and that the software would actually approve it. And this is where this big, big, big focus was on formal methods and trying to even mathematically prove that a software worked according to the spec.

    And then after that, I feel like the next era was the era of breaking or, you know, something that probably is still continuing. And again, all these elements I feel are present in today’s profession of testing, but the later years, the last 20 years was everything that I was kind of studying, watching from the sidelines, getting myself into. I like to call it the era of enlightenment, if I may. I know that’s super pompous of me, but this is where I view more testing in the way how James and other members of the context drew in school of testing had formulated it which is, you know, lighting the way for the decision-maker, not for the developer necessarily, not for – again, it could be for the developer, but somebody who at the end of the day makes the decision whether what is being produced is acceptable or not, or if it’s good enough to ship. Is it, you know, what risks associated there and informing me as a decision maker about all these different factors and then leaving that decision up to me.

    So the testers are not the police. They’re not the quality gate. They’re not _____ anything. It’s really up to the stakeholder who is commissioning all this effort and who makes the ultimate decision about releasing something, putting something into production.

    Bach: I’d like to offer some historical notes on this.

    Melnik: Please.

    Bach: Okay, first, are you aware what the first book was on the subject of software testing, the first known book? A little quiz for your Grigori?

    Melnik: Is it the Myers book?

    Bach: It’s before that. It’s Program Test Methods from 1972 by Bill Hetzel. Now that book was a report on a conference a testing conference which was the Chapel Hill conference, which happened in ’72. That seems to have been a historical watershed event in testing. Before that, the first chapter on software testing that I’ve been able to find was written in 1961 by my teacher, Jerry Weinberg, who wrote it when he was 25 years old. It’s a brilliant chapter on testing.

    What’s fascinating to me is how differently Weinberg talks about testing in 1961 compared to how they talked about testing in 1972, although there is a reference in the 1972 book back to Weinberg, praising Weinberg’s attitude about testing. It’s fascinating to read that because the book in 1961, it’s Computer Programming Fundamentals and the chapter about testing was written by Jerry, he told, although it said it’s co-written, the book’s co-written with Herbert Leeds and Jerry Weinberg, but Jerry wrote the chapter on testing.

    And Jerry talked about testing as a matter of imagination, not automation, as a matter of being suspicious and critical, as a fundamentally unalgorithmic process. And he develops this beautiful story in his chapter about a programmer who’s sure that he’s found the last bug and then finds out that he’s wrong. And then he adds another check for that.

    And then he’s sure now he’s found the last bug and then he sees something else go wrong. And Jerry’s conclusion is look, this is just, we’re working on complex systems and this is always going to be like this. Now, Jerry was on the first project in 1958 that ever had a dedicated test team.

    Now, the reason why he knows that is that in 1961, IBM held a conference of all the computer programmers in the entire world of which there were 400, apparently, and Jerry said his team went to that conference. He was at IBM at the time, and he said everyone thought they were crazy ’cause they had a dedicated group of testers and no one else did.

    They all were just testing their own stuff. So it seemed like the idea of having a dedicated test team caught on and then in 1972, the way they talked about testing was all about test cases, automation, formality and that kicked off the era, I think, of the factory, the test factory. We can have technology and we can have procedures that make it so you don’t rely on human thinking.

    So this goes against what Jerry Weinberg was preaching, not only back in ’61, but all through his career, I became a student of Jerry’s in around 1989 and took all his classes in the mid-nineties and he became a mentor to me. That’s why I’m a real acolyte of Jerry Weinberg, because he always put the human in the center of everything. But, testing – when I joined testing, all the testing textbooks were about how you have to define test cases and if possible, automate them.

    And I felt like that was a fool’s idea. I think, well, the problem is is that there’s too many things to define and far from being unbound, we’ve become super bound by all of these documents and all of this code. And so if you want to be unbound, we have to find a way to temper that. We have to find a way to interact with our tools so that we’re not constrained too much by them. And that, in the nineties, the urge to unbind ourselves from that waterfall world and that heavy documentation world created agile, but it also created the context driven school of software testing, which is a humanist approach to software testing.

    And that was in the mid-nineties that we started creating that. And then DevOps and agile continued to sort of take over the world except that it seems to me that agile in a lot of places has lost it’s humanist foundation and it’s turned into tools, tools, tools. And so if we’re going to really talk about Unbound, DevOps Unbound, it seems to me that we have to be specific about what is it we’re unbinding. I hope what this means is it’s people being unbound, using DevOps as a tool rather than –

    Shimel: But not just a tool. I’m sorry. I didn’t mean to jump in. But not just – you know what, James, we don’t want to make the same mistake that Agile made. And you just said, one of the problems with Agile is we went from humanist to tools, right, too much of a reliance on tools. Let’s not make the same mistake in DevOps. I think that’s something I’ve advocate for for a while too is we can’t lose the human in DevOps.

    Bach: Actually, I want to hear you say more about that, Alan, if you’re willing to say a little more about that.

    Shimel: I’m not shy. I’ll give you my two cents. Look, I started devopsy.kinsta.cloud. I cover this place. I cover this area pretty heavily and I have for seven or eight years now. And I think there’s been this – and I don’t want to say agile failed ’cause agile is far from a failure. But there has been a reliance on tools and maybe overly processed. When we think about things like ITIL and ITSM versus DevOps, let’s say, right, we get in – we tend to point fingers at ITIL. Excuse me and ITSM _____ too much process, too heavy, too much of that, that whole change management thing.

    But there are good things there. Nothing’s all bad and all good. The world’s not black and white. It’s grey. Right? And so there are things we take out of ITIL and things and ITSM that we put into DevOps. Btu DevOps, to me, anyway, and it’s just my humble opinion, is fundamentally about people and teams and culture, not the tools- there’s a lot of tools we could use and there’s overlap in tools and what’s the best tool for any given job.

    We could fight about it till the cows come home, but at it’s nitty-gritty, DevOps is making sure that devs and ops and tests and security and these people work with a higher degree of communication, right, with a higher degree of cooperation getting jobs done faster. And we do that by adding things like automation to the mix. But automation for automation’s sake alone is not enough. Right? And to do automation, yes, you’ve got to use tools sometimes to make that automation work. I get it. But if we lose the fundamental aspect that DevOps is, at its core, about humans, I think we all – we lose what DevOps is and I don’t hold myself out to be as accomplished as either one of you, but I –

    Bach: I still love you, Alan. I love what you just said. I think this is – if you can hold on to what the principle and the feeling behind what you just said, we can do anything with tools and we’ll be okay. We’ll be okay. If we’re talking to each other, if we’re caring about each other, I keep seeing people losing that and of course, any technology and any methodology, when it’s applied sociopathically, will become abuse. And so we just – those of us who can, shall do – should do whatever we can do keep it non-abusive and keep us talking. And then I think DevOps can be a wonderful thing.

    Shimel: Absolutely. I’m sorry to take us down that. Go ahead, Grigori.

    Melnik: No, it’s super encouraging to hear what you said, Alan and again, I’m a very big believer into the greatness of human mind, but it feels to me that in this kind of history of our industry, the pendulum keeps swinging back and forth, right? remember the whole infatuation with the case tools, you know, back in the eighties and nineties and then later on, there was this whole big movement of the software factories, like the term was actually software factors where the idea was that you can go and abstract away at some high level and then the loop, y, the tools will generate all the code and everything for you and make it all highly maintainable, all that, and of course that failed too.

    But I think out of this desire for some reason to do – to come up with the sort of boosters. So I don’t know, some kind of the way of equipping the humans to deal with the inherently complex problems where today there’s no better mechanism of dealing with them other than human brains. So this whole vision of, you know, computers writing computer code, I mean, that’s been promised in the first era of AI, not today, but I’m talking about the sixties, right, and seventies were all –

    Shimel: Yeah, yeah, I mean Spielberg does pictures.

    Melnik: Then it kind of died out. Right? Same thing again, you know, having the computers entirely come up with the test strategy, the thinking about testing automatically, entirely automatically, I think is – I think it’s utopical, at least if you think about what the tools are capable of doing today and for a very, very long time in the future. I think the human will remain to be at the center of this activity, while again, all the tooling is great because tooling gives you additional powers and additional ways of perhaps automating the most rudimentary, the most, you know, boring elements of the job and lets you actually free up time to think about new theories, new hypotheses, new models, new approaches –

    Bach: But you’ve got to consider different kinds of tools. Like, have you noticed the difference between let’s say a tool like a programmer’s IDE. When I use a programmer’s IDE, I don’t feel oppressed. I feel empowered. I feel like I’m riding a magic carpet and I have laser cannons mounted on it. I feel wonderful. But when I use most testing tools, I feel oppressed, like I have just a few possible things I can do and if I have any –

    Shimel: Like you’re castrated.

    Bach: that’s outside of what the test tool designer has thought that I might want to do for testing, then I’m not allowed to do that. This is what I feel like when I encounter something like Cucumber, which is very restricted and the Gherkin language, it’s very restricted. It’s stopping me from going on flights of imagination. And yet, when I use the tools that programmers have created for programmers, unsurprisingly, programmers have created for programmers tools that make them feel powerful and free and able to do all kinds of wonderful things. I would like to bring the same sense of freedom to testing that programmers already enjoy. That’s part of what we’re doing with the transformation work we’re doing at Tricentis.

    Shimel: Yeah. I love that.

    Melnik: In all fairness, not all of the programmer’s tools are that liberating. Some of them feel like you are operating in a straightjacket too, but in general, at least there are some examples, there are instances of tooling that are widely accepted, widely adopted today that are supporting and kind of accelerating your abilities to do your job without constraining you. That’s for sure.

    And you know, the piece that James was alluding to, this is actually the new initiative, that we are driving inside Tricentis, you know, about mentioning this whole new platform of the test tools for the future where we are building something akin to the IDE, we’re actually not calling it IDE, we’re talking ITE, integrated testing environment where actually it’s the human being, the human tester who is in the very, very center of the ITE, and all these other snippets and utilities and services and everything around the human tester, around the human that manages to support the individual in the way, wherever his line of thought, line of exploration, line of test execution may lead to. Okay?

    So yeah, we’re kind of envisioning this to be as a super configurable kind of morphing tool, almost like the chameleon style. We’re depending on the context and the environment, the different capabilities of the ITE, the integrated testing environment will represent themselves and then the human will actually pick whether this is actually something that they want to take advantage or not as opposed to again being forced into a specific kind of mold or one way of doing testing. And I don’t think – we don’t think actually anybody has attempted something of this caliber, specifically in the space of test tooling before. That’s why we’re so enthusiastic and so passionate about this project. Aren’t we, James?

    Bach: Yeah. I also don’t think that anyone is going to be able to copy is easily because the only way that they could actually compete with us is by hiring someone who loves testing, which test tool companies hate doing because testers are troublemakers. I still can’t really – it’s hard for me to fathom that Grigori really knows what he’s done by hiring me because I’m a constant gadfly.

    Melnik: [Laughs]

    Bach: I’m constantly complaining about the state of test tools in the world and at Tricentis, frankly, and I want things to be fundamentally better, but that means we have to fundamentally change how we think about what is a tester. Which might get us into the role of testing a bit, but the – to me a tester is a puzzle solver, a problem finder. I’m not a button pusher and I’m not a verifier. I don’t verify things. Tools can verify things. I’m not looking to prove that something works because I can’t prove that anything works. What I can do is collect evidence and then draws inferences from that evidence with a critical mindset.

    And of course, lots of tools are collecting facts about the products that I’m testing and those – that can be automated and so we’re pulling in these facts. And we can say, “We got this output and we expected to get this output, so apparently there’s no problem here as far as we can tell so far.” But then the tester layers onto that a critical mindset saying, “But how do we know and how can we be fooling ourselves?”

    And so I’m bringing that mentality into Tricentis and I’m really surprised so far they’ve been actually quite welcoming of me, but for years and years, I’ve tried to help other tool companies with this and basically, you know what reaction I get? They basically say, “Well, our customers just want to have automation that just pushes a bunch of buttons and we don’t really care about test design. We don’t really care about empowering testers. So that’s not going to help us make any money, James, so we’re really not interested in supporting the skilled testers that you keep talking about.” So Tricentis has a different kind of vision that I think is hard to match.

    Shimel: There’s also a little bit of you can’t make wine before it’s time. Right? And I think right, I think you need the right macro conditions, that things like DevOps have brought about and the whole progress -0 we’ve gone back here now to the fifties and sixties and all the way to the present time.

    Certainly the world we live in today may be a – I want to think so anyway, we’re a little bit more open to the idea of saying, “Look yeah, we can automate some stuff. We can have – we don’t need a human to just push buttons. Right? We have tools that can go push the buttons for us, but we need humans to do blank. We need humans to do some of this stuff and so we need to enable and empower them to do that within the context of let’s automate what should be automated, let’s say. But we still need those humans in there.

    You know, when I was growing up, much like you guys, right, the only animal in the world that used tools was humans. Isn’t that – that’s what I learned in school, right, that’s what separated us from animals, one of the things, right, was our ability to use tools. Well, then we found out chimpanzees use tools and other monkeys use tools and we still were okay with that ’cause they were primates. Now we found birds use tools, right, to crack nuts and stuff like this.

    So tool use in itself does not connotate one’s intelligence or higher being. We need to – so to me, that’s the ultimate case. Right? It’s not about what tools we use. It’s about how we use our brains. And I think that’s apropos to what we see in testing today. Yes, we could come up with tools, but it’s creating an environment for people to think in and use their brains in and apply it that I think will ultimately provide us the highest payback, if you will. Right?

    Bach: It’s interesting what you just said about you’re implying that the history of how we got to where we are now is important, that we have to have a certain set of – a certain historical context, technological context in order to take the next leap. And I think one thing in which that’s really true is we now have decades as an industry, decades and decades of experience trying to eliminate people from engineering and we can’t do it. I remember, maybe you do, there were these advertisements in Byte Magazine in the eighties about case tools and all your marketing people can just define your product and you don’t need developers anymore because it automatically generates all the code.

    And I remember sitting there at Apple Computer reviewing case tools, advertisements that had been sent to me, thinking, well, could I buy some of these tools and actually use them to help with the testing? And it turned out, no. You couldn’t. It was just all – it was all hype and nothing. And so I shrugged. And so a lot of – and even today, now we have AI, AI is the new case tool and the AI will do all this stuff for you. Then it turns out the AI is biased and racist and confused and if you put kitten ears on a whale, it says, “That’s a kitten.”

    Shimel: Yep. And it’s no more than ML, the machine learning anyway, but you know, yes, we’ve been – we have been, you know, chasing this white whale for most of my adult life and at the same time, if it was a nation, the fastest growing nation on earth are developers. There’s 40-something million is the estimate growing what, 20 percent a year or something like that. So I think it’s time we stop probably chasing that white whale and move on to empowering people.

    Bach: And one way to do that, I suggest a definition of programmer or perhaps developer. And the definition I use is a developer is someone who stands at the gateway between the world of the machines and the world of humans. And the developer’s purpose is to operate competently in the world of people and then operate competently in the world of machines.

    And if you don’t have that person standing at the gateway, then you don’t have a connection between the world of humans and the world of machines. You just have people kind of praying to the god of technology and hoping that good things happen. And so no matter how complicated technology gets, no matter how much AI we have, we still have the need to operate competently in the world of people and the developers will always live right at that threshold. And testers need to live there too and not get mowed under by automation.

    Melnik: Yep.

    Shimel: And you know what guys, we’re almost out of time, but this also brings up something that I thought we might be able to get to on this show, but we’re running out of time, and that is that this caste system, if you will, caste system of developers being a higher caste than the tester who’s higher than the, what we call today the SRE, the ops person or what have you. And this whole IT stack of humanity, I think we need to examine that too with the develop – when we’re asking the developer to do the security and the testing, is he any better than the tester? Is there a difference? But we’re going to save that one I think for another show, if you guys would like to come back on, and we can explore that a bit.

    Melnik: _____ back, but I categorically reject that whole hierarchy and I also do not like the whole, you know, notion when people say, “Hey, but the tools and everything that would really help with the low skilled tester.” That’s another notion that kind of gets _____ _____ –

    Bach: No, we reject that.

    Melnik: So annoying and, you know, whether it’s development or testing or any other, you know, data science modeling, the UX design, it’s the discipline, it’s something that requires dedicated effort of studying, perfecting, thinking about the way how people do that work and it requires intellect. That’s the most important thing. You know, using some very powerful tools doesn’t give you the license to turn off your brain and I’m happy to come back on the show and talk about the different career letters and progressions because again, it’s going to take us a longer –

    Shimel: That’s a whole other show.

    Bach: There’s a lot to –

    Shimel: _____ and vocal about that.

    Bach: say about that. But I just want to note that this whole issue has been studied in great depth in another field, in the field of the sociology of science. For instance, in physics, theorists are in the higher caste and experimentalists are in the lower caste. So if you’re an engineer, you’re in the lower caste and they’ve studied this. How do these people cooperate? How do they assign prestige to each other? It’s a well-known phenomenon in many fields of human endeavor.

    Melnik: And the same in math, right, the theoretical math versus applied math, the theoretical math seems to always think that they’re the gods and they’re the ones that are, you know, creating something that, you know, will take centuries for the world to truly appreciate and then you guys are in the applied math solving the problems of today.

    Shimel: Exactly. Guys, we’re out of time, but I’m going to hold this – I’m going to hold you both to it. We’re going to come back on – not in the next show because I think we have something scheduled, but in one of the ones coming up, we’ll have you both back on, or maybe this’ll serve as part of our first roundtable in about a month from now when we’ll open it up to the audience as well. But for now, Grigori, James, thank you. Yes

    Melnik: Thanks, Alan.

    Shimel: James, thank you.

    Gentlemen, it was a pleasure. This is Alan Shimel. You just watched the very first DevOps Unbound. We’ll be back in two weeks with a fresh panel, fresh topics, stay tuned. Thanks again to Tricentis for sponsoring and thank you for watching. Bye-bye.

  • What Is DevSecOps and How to Enable It on Your SDLC?

    What Is DevSecOps and How to Enable It on Your SDLC?

    For the past three to four years, all the companies around the IT world have adopted agile and different application development methodologies that leverage the work for different departments or areas and helps them to develop new products and release new features to improve their processes and infrastructure.

    In this new Agile and DevOps world where everybody on a team is involved in the rapid-changing and evolution of their application, we are promoting accountability for everybody in terms of security—this is when DevSecOps joins the party.

    What is DevSecOps?

    DevSecOps is a new model that provides accountability for the security implementation in the application; from the planning, design, development, QA/testing, to release and when operating on a production environment.

    When implementing DevSecOps on the Software Development Lifecycle (SDLC), an organization will experience the continuous integration and will notice that the costs for compliance are reduced, code is constantly being analyzed, tested, delivered and released properly.

    DevSecOps enables the process of implementing security to everybody and makes them accountable.

    Why Is It important?

    As I stated previously on this blog, on this rapid-changing era, everything is evolving at a very accelerated pace. We continue to discover vulnerabilities and breaches across platforms and operating systems, patches are released constantly but we—as part of the operating team of a company—cannot afford the risk of having a vulnerability on any side of our IT system/application.

    Main Benefits

    • Reduces vulnerabilities present on your code.
    • Reduces vulnerabilities present on your IaC technologies.
    • Reduces the number of ways to exploit your application
    • Reduces downtime.
    • Improves your application stability, availability and security.

    How Can I Enable DevSecOps on My Current DevOps Pipeline or SDLC?

    DevSecOps is a must-have methodology that needs to be integrated into your DevOps process/pipeline to help you improve your security on your SDLC.

    There are five important phases that need to be followed in order to enable DevSecOps on a current DevOps pipeline or in the SDLC. Here are the crucial phases to enable it:

    Phase 1: Secure Local Development. Start by implementing secure working-environments. When you are developing an application, in most cases you will use open source technologies. Docker is a great helper at this phase since it automates the infrastructure and services deployments on local machines. So when you are using this ready-to-go docker environment, make sure that you are using the most recent/updated versions of the Docker Images and scan them for vulnerabilities. Even the images from official providers have vulnerabilities that need to be patched.

    Phase 2: Version Control and Security Analysis. Enable Vulnerability while uploading your source code.  Having multiple hands or people working at a piece of code can lead to vulnerabilities, especially when they are remote. Git systems have been a great improvement for collaboration between team members and code. When a team member uploads a piece of code, I strongly suggest that you enable automated testing for security on your code dependencies and core. Some good alternatives to do it are Snyk or Sonatype’s Nexus.

    Phase 3: Continuous Integration and Build. When creating the development image/package, you’ll need to make sure that your build tool or system has the proper security in place. It uses https:// protocol, it’s properly hardened and secure, it’s available and protected for attack mitigation or even not accessible via the internet. The tools that you can use here are Jenkins, Circle CI, AWS CodeBuild, Google Cloud Functions, Azure DevOps.

    Phase 4: Promotion and Deployment. When deploying to an environment, insert the environment variables through your CI/CD tool and try to manage them as secrets. Proper encryption and management of these are recommended in order to enhance your security protocols.

    Phase 5: Infrastructure Security. When your app is deployed, make sure that you have an IDS (Intrusion Detection System). Tools such as OSSEC or Wazuh will help on this matter to protect all your hosts.

    Once your code gets to the production, it doesn’t mean that it will be 100% secure. New vulnerabilities are disclosed every day but this cycle will help you and your team to test your code against all the repository of known vulnerabilities, at the time of monitor, configure, reconfigure, adapt and deploy solutions to the environment.

    Tools and Processes

    These are the tools you will need to enable in your DevSecOps process:

    • CI/CD Tool.
    • Secrets Manager.
    • Version Control System.
    • Docker Orchestration tools.
    • Security Analysis tools.

    And these are the processes:

    • Automated regression testing.
    • Dynamic testing.
    • Automated Vulnerability Assessment.
    • Automated Integration.
    • Promotion.
    • Automated Deployment.

    What Is the Ideal Workflow for DevSecOps?

    1. A developer creates a new code and integrates it into the VCS.
    2. Members from the QA team retrieves the code to perform the static code analysis to identify security flaws or functional tasks.
    3. A test environment is created automatically using IaC such as Terraform, CloudFormation, Chef or Puppet and the security configurations are added to the system.
    4. The test automation suite is performed to the application with a tool usually Selenium or any other tool that performs backend, UI, integration, security and API tests.
    5. After the test suite is performed and successful the new changes are sent to the production environment.
    6. The new version of the code is now going to be monitored in the production environment using an APM or Cloud-native monitoring tool.

    Following these points you are ensuring that your application is following TDD practices improving the code quality, compliance, increasing the number of releases of code to production and reducing the time to market which is essential for any organization.

    What Are the Challenges When Enabling DevSecOps?

    • Enabling too Many Tools: Enabling too many tools can become a problem on your SDLC, especially when your team is not used to work and relate with DevOps/security tasks. The main recommendation here is to start slow. Start by enabling only the necessary tools to get your team familiar with the process and add more when you feel your team is prepared for it.
    • Getting Used to the Methodology: It will take some time for all the team to get used to the DevSecOps methodology/culture and also to keep following it in order to be compliant with the normative that your business demands. Try to always stay up-to-date and coaching your team for the newest technologies.
    • Chasing Perfection on the Process: Keep in mind that all the DevSecOps process won’t be perfect, but it will get mature over time. Teams always try to chase perfection and this only leads to more problems with even more integrations or dependencies.

    Conclusion

    In the end, I think every organization must make the effort to shift to a DevSecOps methodology or process and come up with a multidisciplinary team with a focus on security. That’s how an organization will transition from doing DevOps to DevSecOps. Allowing all their collaborators to have accountability on the part they are actively developing.

    — Guillermo Vélez

  • SaltStack Extends IT Automation Framework

    SaltStack Extends IT Automation Framework

    SaltStack today added three open source modules to its portfolio that make it easier to automate the management of IT at scale.

    Heist is a plugin that allows organizations to create agents that will dissolve after a specific amount of time. Umbra is a plugin that gathers data streams to make it easier for machine learning algorithms to consume.

    Finally, Idem is a declarative programming language that enables IT teams to create workflows based on stateful constructs that can be used in conjunction with any IT automation framework or systems management tool.

    All three modules are based on the Salt Plugin Oriented Programming (POP) model. Company CTO Thomas Hatch said SaltStack created POP to make it easier to manage modules and libraries of code within a monolithic application without having to transition to a different microservices-based architecture based on containers.

    That POP approach also serves to make it easier to test modules within a monolithic application environment, he said.

    SaltStack used POP to create Salt, an open source IT automation framework that serves as the foundation for the commercial IT automation platform provided by SaltStack. Organizations that have embraced the open source Salt project include Juniper Networks, Cloudflare, Nutanix, SUSE, LinkedIn and eBay.

    Heist extends the Salt project by giving DevOps teams an alternative to having to commit to either an agent or agentless-based approach to managing IT. Instead, DevOps teams can create an agent to collect data that will run only as long as it’s needed.

    Umbra, meanwhile, prepares data to be consumed by machine learning algorithms transparently, which Hatch said is critical in providing visibility into how artificial intelligence (AI) models are constructed. That capability is going to be a requirement in any regulated industry that requires organizations to comply with IT audits, he noted.

    IT automation frameworks are, of course, foundational to any set of best DevOps practices. While most of the initial IT automation management frameworks were proprietary, the entire category has gained more traction as open source implementations of these frameworks became available. Now DevOps teams have a plethora of open source and commercial IT automation frameworks on which to define a set of DevOps processes.

    The biggest challenge in getting IT teams to embrace these frameworks historically has been the preference of many IT professionals for scripts they develop. However, as the scale of the IT management challenge has increased in the age of the cloud, relying on scripts that are often undocumented has become problematic. Industrial-scale IT organizations require IT automation frameworks that allow them to create reusable playbooks that are easily documented. Otherwise, every time an IT professional who authored a script leaves the company, the IT organization soon discovers the script they’ve been relying on that has broken can’t be fixed easily.

    The day when all IT organizations rely on some form of an IT automation framework is fast approaching. In fact, the remaining IT automation challenges have a lot more to do with the culture within the IT organization than they do any of the technical capabilities of the platforms being considered.

    — Mike Vizard

  • A Developer’s Task: Minimizing Challenges Created by Fragmentation

    A Developer’s Task: Minimizing Challenges Created by Fragmentation

    Over the past decade, there has been a wellspring of apps, services and platforms to help developers build custom solutions for businesses. Some are great, some not so much. Regardless of quality, the quantity of options and each one’s varying level of sophistication has made it difficult for developers to separate the wheat from the chaff.

    A recent survey from IDC supports this notion. It contends that the current landscape of software development languages and frameworks is highly fragmented. Having too many options—as opposed to a single, unified environment—leads to holes and disconnects in the development process, not to mention the potential for significant complications in the long-term support of today’s apps.

    Here are some ways developers can navigate the challenges created by this high level of fragmentation. 

    Don’t Be Afraid of Low-Code (or No-Code) Solutions

    The IDC survey found that less than half of developers—44%—have used low-code development tools professionally at one point or another. As one would surmise, low-code programming brings with it a host of advantages, such as easy-to-use and intuitive interfaces, and the ability to deliver applications faster. 

    Some low-code platforms also enable developers to build applications that can be used on any device while also reducing dependency on outsourced IT. A built-in script builder can allow even relatively inexperienced developers to turn complex business logic into functional workflows—such as automating a repetitive task—within a matter of minutes. Even after apps are launched, workflows can be updated and customized to fit specific business needs nearly instantly.

    Achieve Scale with Serverless Applications

    When building on a serverless platform, developers prevent infrastructure maintenance from inhibiting applications from scaling globally. This also allows developers to build mobile apps without managing the backend infrastructure. Cloud computing using public clouds typically involves renting virtualized servers across the technology stack. But when looking to scale, the onus is on developers to deal with the operating system and database while also managing necessary updates. This can lead to a work-intensive process on the app infrastructure level.

    A serverless infrastructure, on the other hand, abstracts databases and file servers, shifting focus to the app logic level. This allows developers to build and manage microservice-based apps that integrate with other services, making it easy to scale applications.

    Make Things Easy on Your Team

    According to the IDC survey, just 20% of developers claim they are “extremely familiar” with containers and microservices. With such expertise hard to find, using flexible tools that allow teams to quickly deploy code in scalable, compartmentalized components is vital.

    Organizations should aim to help their developers focus more on designing top-class user experiences of serverless applications, rather than spending (or wasting) time and resources on setting up and maintaining components on the server side. This gives developers the ability to focus on doing what they love—developing—as opposed to other tedious and mundane activities such as manually managing servers and storage solutions, or tweaking the underlying infrastructure of their microservices.

    Of course, making things easier for developers means deploying a powerful tool built on a stable foundation that can offer SDKs and APIs for both mobile and web apps, the ability to test early and often in a sandbox environment and analytics that monitor and report performance in real-time.

    Developers know just how challenging their jobs can be, and the fragmentation that can occur due to the vast number of options on the market can make it even worse. There is nothing wrong with using multiple solutions, of course, but having a sturdy and unified underlying infrastructure goes a long way toward avoiding unnecessary headaches.

    Finally, organizations should look to those who know best when deciding which company-wide tools to implement: the developers themselves. As the IDC study states, developers should be viewed as key stakeholders in IT purchasing within any organization moving to the cloud accompanied by an internal digital transformation.

    — Raju Vegesna

  • Developers Are Gaining Clout, and We Need to Pay Attention

    Developers Are Gaining Clout, and We Need to Pay Attention

    I’ve been watching the ascent of developers for quite some time. A few years ago, I wrote an open letter to developers in which I noted their power and collective influence over the direction of technology across industries. As creators of our digital future, developers hold incredible power—and with that power comes great responsibility.

    Developers now directly influence technology decisions within their organizations. They have achieved widespread recognition for their expertise and, as such, have frequently acquired a broader role in their company’s digital trajectory.

    While developers have always been first and foremost in feature and functionality development at the Cloud Foundry Foundation, we’re starting to see this approach bubble to the surface across the business landscape and in other open source communities. Cloud-native technologies often focus on operators, but it’s just as important to empower developers.

    If our goal is, as it should be, to enable developers to be more productive and give them the freedom to create more applications to production as quickly as possible, a developer’s workflow needs to be seamless and intuitive.

    Developers Need Widespread Support

    The long-tail impact of developers gaining a seat at the table and influencing technology decisions is a positive shift that demands widespread support. It’s critical that we recognize the important role developers play today and give them accompanying responsibilities to meet our common goals.

    Think about who is creating the software that’s changing how organizations operate and interact with customers: It’s the developers. Developers, above anyone else, know the technologies that make them most productive and impactful. Putting them in the driver’s seat is paramount because we all benefit when they are given space and time to be as productive as possible.

    Enterprises of all shapes and sizes are taking notice and shifting strategies as a result. Indeed, a significant majority of organizations are empowering developers with the freedom to choose the tools they use. In our most recent research, we found that 59% of respondents said their company is giving development teams more authority to choose their tools.

    Perceptions Are Evolving

    It’s no coincidence that this change is happening when software development becomes more critical for companies across all industries. Organizations realize the biggest hurdle to improving application development productivity is culture. More than a third (37%) of survey respondents feel that way today. We’ve been tracking this sentiment since 2016 and only a quarter of respondents felt that way then.

    Just three years ago, 39% of respondents said technology was the biggest hurdle for their organizations to overcome. That number has dropped to 29% today.

    That’s quite the inverse from 2016, and it indicates the levels at which software development is becoming a greater priority for businesses. As such, it’s increasingly common for organizations to give developers resources and influence to shape the direction of their business.

    Let’s not become complacent. We all need to double down on giving developers the best possible experience. It’s the developers who are driving transformational change in our organizations. The more power we give them to do so, the better off we will be as a result.

    — Abby Kearns

  • Maintaining Exceptional Quality Despite Shrinking Delivery Deadlines

    Maintaining Exceptional Quality Despite Shrinking Delivery Deadlines

    My first article in this series focused on the need for continuous improvement across three key dimensions of software development–velocity, quality and efficiency. While these are all equally important, in this piece we’re going to focus more specifically on quality.

    The past few months—what you might call the summer of outages—have been troublesome for many of the internet’s biggest players. Google, Apple and Cloudflare, as well as Facebook, Twitter and LinkedIn, have all experienced bouts of unplanned downtime, sometimes even multiple bouts in quick succession. Is this all just a matter of coincidence and bad luck? Perhaps, but peeling back the layers reveals an increasingly painful reality of modern DevOps life.

    While the root causes of these outages were varied, several were blamed on software glitches. This suggests software rollouts may be happening before they are comprehensively battle-tested and ready for production-level systems. This is never a good idea, especially when you’re dealing with high-profile services supporting millions of users worldwide. In these cases, software-related stumbles are that much more damaging.

    When software is rolled out prematurely, a cardinal rule of development is broken in that we, the end-users, become the unwilling guinea pig for poor-quality applications. No matter the pace of rollouts, an unwavering focus on quality needs to be maintained, and automation, integration and measurement can be the keys to ensuring this.

    Automation

    Automation refers to a pre-programmed process that executes tasks with minimal human intervention. The more you automate, the higher your quality should be. Avoiding manual intervention decreases the likelihood of less optimal techniques or human mistakes, both of which can lead to poorer application quality and customer satisfaction. Essentially, automation enables the best techniques and practices to be quickly and easily replicated.

    Consider the example of automated unit and functional testing. If you can automate the execution of the best testing scripts incrementally across code units and functions, you can test more accurately, not to mention more quickly. The benefits will increase exponentially with the additional layers of testing you can automate. We recently worked with a customer who realized an overall ROI of 467% from their investment in automated unit testing.

    Integration

    Modern applications are increasingly componentized, spanning multiple platforms. Consider an e-commerce application, where the front-end may reside on a system of engagement (such as a web server) while the back-end transaction ultimately occurs on a system of record, such as a mainframe.

    High-quality, end-to-end digital services ultimately depend on all critical components being universally and automatically included and prioritized in DevOps pipelines. Similar to automation, this is because any time manual effort is introduced into the integration process, you increase the chances for error and consequently, the risk of a particular code segment lagging behind others.

    The end result? More defects making it into production in the sprint to the finish line. It’s like the famous I Love Lucy chocolate assembly line episode, which offers a humorous visual interpretation of the chaos that erupts once human intervention is introduced into a process.  Ideally, seamless toolchain integrations, across multiple application-leveraged platforms, can be utilized to maximize quality outputs leveraging DevOps technologies you’re already using, such as JIRA, Jenkins and SonarQube.

    Measurement

    Improving quality depends on development teams constantly improving their methods. This can be achieved through ongoing measurement against key performance indicators (KPIs).

    The challenge is establishing relevant KPIs, and we have seen many development teams fail at this. The most effective KPIs are those that translate to real business impact. For example, “number of bugs released into production” is a commonly used metric, but it does nothing to convey the bottom-line cost of those bugs. A more effective metric would be the cost of resolving defects, with success being declared when the organization reduces the amount of money spent on fixing bugs.

    The cost of fixing a bug is much less when it takes place earlier in the software development lifecycle (for example, in the coding and unit testing phases) than when the software is near or already in a production state. When a development team can consistently show it is reducing costs through earlier identification of bugs, this puts them in a stronger position to justify investments in better testing and analysis tools.

    Conclusion

    We’ve long known about the challenges DevOps teams face to balance velocity and efficiency with a high degree of software quality. It’s like asking a circus performer to sprint across a tight rope. It may seem unfair, but the need to release high quality software faster and faster, while staying within budget, is not going to abate any time soon.

    The recent spate of outages has taught us that even the biggest and the strongest in the industry are not immune to immense speed pressures, and the quality stumbles that often result. Automation, integration and measurement can help development teams in their ongoing quest to meet the seemingly conflicting requirements of improving quality, velocity and efficiency—at the same time.

    — Rick Slade

  • Common Pitfalls of Software Development and Tips to Avoid Them

    Common Pitfalls of Software Development and Tips to Avoid Them

    In business, no one wants to plan on failure. However, when it comes to software development, failure is more common than most managers want to admit. In particular, many managers and executives fear large software-based projects are especially likely to fail due to the complex nature of these projects and the nuanced reality of the deliverables.

    While there is no doubt a software development plan is complex, there is no reason to be so readily consigned to failure. In fact, it might be argued that software is so important to your operations that failure should be viewed as unacceptable. However, this aggressive approach in theory needs to be backed up with proper execution in practice. That is why you need to know the common pitfalls that befall most software projects. This is the only way to make sure your company avoids them.

    Vague Project Requirements

    One reason so many projects fall short of the mark is that these projects fail to define the mark adequately. Your software development project cannot succeed until you know what you really want it to do. This means your project specifications need to be clear from the very start. Your goals should be defined first, and then these goals should be consistently reiterated to team members throughout the process. This includes the scalability of the project. The development team needs to know what features should be included in order to produce a final software solution that actually meets your company’s needs.

    This is all part of developing a versatile software development plan. A good software development plan will anchor your project from start to finish. A poor one will inevitably leave you vulnerable to failure.

    Unrealistic Timeline

    The timeline matters to the company, which is why some companies set aggressive timelines as part of their software development plan. However, while these timelines sound good in planning sessions, they may not make much sense in the daily development process. It is pivotal to think carefully about a timeline in order to account for development, testing and more. All parties should be consulted in the development of the timeline. This includes high level executives, project management and developers. This varied input should allow the project to accommodate the needs of management alongside the needs of developers. This is a form of risk management because an insufficient timeline will lead developers to cut corners, leading to a lackluster result.

    Unbalanced Staffing Commitments

    As you define your software requirements, you will need to think about the team members who will bring your plan to fruition. If you do not assign enough people to the project, production will fall behind. However, if you put too many people on the project, things can start to get messy. Overstaffing a project can easily lead to miscommunication, and it is harder to keep a large team of people unified. This can also lead to redundancy and a lack of consistency if too many duties are spread out. For these reasons, it is important to assess your staffing situation correctly from the start. Be sure to include your project management team in the final staffing decision.

    Insufficient Testing Procedures

    In all your planning, it can be easy to neglect testing. The reality is testing should be a critical part of your plan, accounting for a significant amount of your timeline and resources. Testing is important because it allows the development team to iron out any problems with the software. All software will have bugs in its early versions, and these bugs can be caught and fixed with sufficient testing. Testing is the only way to make sure your software works the way it needs to work. If you implement deliverables that do not perform, then your workers will be flummoxed, your consumers will be disappointed and your security could be at risk. Therefore, allot plenty of time for testing throughout the entire development plan, allowing for each component to be tested prior to going live.

    Poor Hands-On Leadership

    If you really want to make sure your project goes well, then make sure you have good people in place to lead it. Your project management team is absolutely critical to successful software development. A good project manager will internalize all parts of your plan. This includes the budget, the timeline, the goals and the staff. Your project managers need to facilitate communication, manage schedules and oversee adjustments. This key leadership role is your software development plan in action. Make sure you hire the person with the right skills and the right experience. Then, make sure that person knows exactly what your goals are to achieve success.

    These pitfalls are common for a reason. However, with a little foresight, it is possible to keep your project and your company from disaster.

    — Shane Zilinskas

  • How to Transition to Become a DevOps Engineer

    How to Transition to Become a DevOps Engineer

    You’ve either heard the buzzword or you’ve seen the extensive list of rewarding aspects that come with being a DevOps engineer. Whatever the reason you’re here, this guide should help with taking the right steps toward becoming a DevOps engineer, whether this is your first job after graduating or an opportunity to reskill leveraging your prior industry experience.

    Immerse Yourself

    Learn the fundamentals, practices and methodologies. Understand the why behind DevOps before jumping into the tools. I believe the key goal of any DevOps engineer is to increase speed and maintain or improve quality across the entire software development life cycle, in order to provide maximum business value. Read articles, watch some YouTube videos and go to one of your local Meetup groups or conferences–it’s useful to become a part of the welcoming DevOps community where you’ll learn from the mistakes and successes of those that have been here before you.

    Your Background

    There are likely two key times where you’ll be transitioning into the role of DevOps engineer; either reskilling during your career or as your first job after graduating. You may have prior experience in technology, such as working as a software developer, systems engineer, systems administrator, network operations engineer or database administrator, all of which will bring broad insights and useful experience for your future role as a DevOps engineer. If you’re starting your career, maybe you’ve finished your degree in computer science or any other STEM field which will give you some of the basic stepping stones you’ll need in this transition.

    I think it’s important to breakdown the broad spectrum of what DevOps engineer roles entail. Most enterprise companies today will use DevOps engineers in three different positions.

    • DevOps Engineers, with a Dev Bias: Work in a software development role, building software applications. The key point is they leverage CI/CD, shared repositories, cloud and containers as part of their everyday work, but they are not necessarily responsible for building or implementing tooling. They will understand infrastructure and in a mature environment, will be able to push their own code into production.
    • DevOps Engineers, with an Ops Bias: Could be compared to systems engineers or systems administrators. They understand software development but do not spend the core parts of their day building software applications. Instead, they are more likely to be supporting software development teams to automate manual processes and increase efficiencies across the human and technology systems. This could mean breaking down legacy code and using less cumbersome automation scripts to run the same commands or it could mean installing, configuring or maintaining infrastructure and tooling. They ensure the right tools are installed and available for any teams that need them. They enable teams with learning how to leverage CI/CD and other DevOps practices.
    • SREs (Site Reliability Engineers): Think of site reliability engineering as having software engineers solving operations and infrastructure problems. SREs focus on creating scalable, highly available and reliable software systems.

    In the ideal world, DevOps engineers will understand all the above, which is the norm at some mature technology companies. However, if you knock on the door of top tier banks or many Fortune 500 companies you’ll realize oftentimes these roles do usually have biases towards dev or ops.

    Operating Systems

    Whichever background you’re coming from, you should start with the fundamental technologies you’ll need to use and understand as a DevOps engineer. Linux is most likely the operating system you’ll be using daily, although some of you might be using Windows. This is where everything runs, and fundamental knowledge is important. You can install Linux at home where you’ll be able to break as much as you want and continue learning along the way.

    Scripting

    Next up you’ll need to pick a scripting language. There are many to choose from, such as Python, Go, Java, Bash, PowerShell, Ruby and C/C++. I suggest starting with Python, it’s one of the most popular for a reason, it’s easy to learn and interpret. Python follows the fundamentals of object-oriented programming and can be used for web development, software development, creating desktop GUI apps and even business applications.

    Cloud

    After Linux and Python, my next pick would be cloud, where you could choose from Amazon Web Services, Azure or Google Cloud Platform. I’d start with AWS, as they have an extensive collection of free learning tools available to  take you down any track from learning about AWS as a developer to operations and even the business facing components. In fact, you might even be overwhelmed by how much is on offer when you first get setup with AWS. Maybe start with EC2, S3 and VPC and see where you want to go from there.

    Programming languages

    For those of you with a passion for software development. You should keep on improving your programming skills. Some good languages commonly used in this role include Python, Go, Java, Bash, PowerShell, Ruby and C/C++. You should also become familiar with tools such as Jenkins and Git/GitHub which you’ll be using frequently as part of the CI/CD process. Infrastructure is no longer left to the operations guys so you’ll need to have some exposure to a cloud platform such as AWS, Azure or GCP.

    Containers

    Finally, start learning about containerizing code using tools such as Docker and orchestration platforms such as Kubernetes. All the above have extensive online learning resources for free and most cities will have local Meetup groups where you can turn up and learn from experienced people in a friendly environment.

    For those of you with less experience in development, you can still get involved using your passion for automation, increasing efficiency, collaborating with others and a never-ending desire to improve on your own work. I would still suggest learning the tooling outlined above, but with less emphasis on the coding/scripting languages. It will be useful to learn about Infrastructure as a Service, Platform as a Service, cloud platforms and Linux. You will likely be setting up the tools and learning how to build systems that are resilient and fault-tolerant while writing your own code.

    Reskilling

    If you’re transitioning into a DevOps role during your career, you should start within your company to explore opportunities. Can you reskill by working with an existing team? There might be an option for you to shadow other team members, ask for advice and reskill without even leaving your current job. This might not always be possible and some of you will need to move into another company.

    If you can learn some of the practices, tools and technologies listed above, you’ll be in a good position to demonstrate relevant knowledge during interviews. The key is to be honest and to not set yourself up for failure. Most hiring managers understand you don’t know all the answers but if you can show what you’ve been learning and explain you’re open to learning more, you should be in with a good chance.

    Starting Your Career

    You can apply to direct open opportunities at companies who are hiring junior DevOps engineers. You will often find these companies say they’re looking for more experience and you should apply back in future when you’ve gained some commercial experience. It’s the typical, frustrating scenario of “we want more experience” but nobody is willing to give you the first chance.

    It’s not all gloomy though, there are a few companies focused on training and upskilling graduates directly out of the university. MThree, where I currently work, delivers exactly that. We hire fresh graduates and train them for eight weeks in our Alumni Academy. Training covers everything from theory through to hands-on projects with Python, Jenkins, Git, Ansible, Docker, Kubernetes and AWS. By the end of the training you’ll have some solid exposure to the entire software development life cycle and a good understanding of how this applies in a Fortune 500 environment. The best part is once qualified, our teams then leverage their extensive list of clients to provide job opportunities for junior DevOps engineers. MThree pays these associates on a full-time salary with benefits for the first 18-24 months before they join the client as direct employees at the end of the agreement. This is a great way to bridge the gap from university into your technology career.

    In summary, there are many ways to transition to become a DevOps engineer. This is a very rewarding career route which will likely keep you engaged and challenged as well as increasing your earning potential.

    — Conor Delanbanque