devopsy.kinsta.cloud

Tag: agile

  • Agile and DevOps: Where the Rubber (Literally) Meets the Road

    Agile and DevOps: Where the Rubber (Literally) Meets the Road

    Do Agile and DevOps methodologies apply outside of technology? More specifically, can we apply them to sport? I decided to answer these questions by analyzing the application of Agile and DevOps principles within my road-cycling team.

    For those of you who don’t know, cycling is a very data-driven sport. While we are training and racing, we have a live feed showing data on our computers, which are attached to the handlebars. We see data such as power (watts), heart rate (bpm), cadence (rpm) and much more. Monitoring and analyzing this data allows us to work within our limits to ensure we aren’t overloading ourselves while performing to our highest capabilities. We measure our limits through tests over time periods such as 30 seconds, five minutes and 20 minutes. The results of these tests form specific thresholds that we work within to maximize the benefits of training and racing. We can set up automated alerts in our bike computer to warn us if we are overexerting ourselves or we can keep an eye on the screen to measure this ourselves.

    In a mature DevOps environment, we work toward monitoring at the service level to understand usage, performance, environment and application health and more. This doesn’t just help us to find the problem, it allows us to set up self-repair protocols within our infrastructure if the monitoring tools report any alerts.

    We run the cycling team like a startup, where everyone wears many hats. We operate in small, loosely coupled units; everyone rides together without silos, but then we also have specialties which come out during the sessions. For example, the climbers are better at riding fast uphill, so they work together at times, as do the powerful sprinters with the sprints and so on. The important point is that we don’t segment ourselves solely to these responsibilities – the sprinters still have to get themselves over the climbs and the climbers still have to practice sprinting. We never know when one of us will have to try to fill in for a teammate. We understand each other’s roles and the importance of that person within the team.

    We learn as we go. We can’t rely solely on one person in the team, even if they are the strongest. Let’s say the A rider has a crash 2km before the finish – what do we do? The lead out train all move one place up the line, so the guy who was supposed to be the last man left to support the sprinter in the final move, will now be the sprinter. We have setup a self-healing system with very low mean time to repair. Flip back to DevOps – some bad code has made it through your pipeline. You can’t solely rely on the person who wrote the code to solve this problem. In the ideal world, we use version control to assist other people within the team to figure out exactly how to repair the issue. We can design the system to self-heal by automatically reversing the failed change to an earlier state.

    Everyone in the team understands that they have a cross functional role. This doesn’t mean that we don’t have specialisms or areas that we like more than others, it just means we are a team of doers. This compares to a team working effectively with DevOps principles. It doesn’t matter if you started your career as an applications developer or systems administrator—you must get the work done and solve problems for the greater good of the organization.

    Bicycles and DevOps Cycles

    Essentially, your entire role as someone who practices DevOps is to make everyone else’s life easier through implementing processes and tools that help to increase productivity and reduce errors. In a cycling team, everyone is working for the success of the entire organization, not just for themselves. The DevOps role can compare to that of a domestique in cycling.

    For context: In cycling we ride directly behind the wheel of the person in front, with maybe an inch between our front wheel and the rear wheel of the other rider. We do this because it uses approximately 30 percent less power for the person behind. The domestique is responsible for making the other person’s job easier. They don’t work for individual glory but sacrifice themselves for the greater good of the team.

    The DevOps architect or engineer in your organization (disclaimer: Yes, I believe that it’s everyone’s responsibility to practice DevOps methodologies, but often there is a person who has the strongest knowledge of DevOps and how to implement this across the organization) does not spend their day thinking about how to show off about some awesome code they wrote or the system they built. They are thinking about how to make your day-to-day processes easier and solve problems for the good of the team.

    Pulling the pace at the front of the peloton. This is called the ‘lead out.’ The pace is kept so high that other teams can’t attack or ride ahead. The aim is to drop the final rider off ahead of the other sprinters and as close to the finish line as possible.

    We train, fail fast, continuously learn and succeed together. In the words of Kiichiro Toyoda (founder of Toyota Motor Corporation), “They do not have the expertise gained from the failures it took to produce the original. … The thieves may be able to follow the design plans and produce a loom, but we are modifying and improving every day.” So, you can steal their plans but without knowing their failures and ongoing development, you won’t create the same product. In comparison, another cycling team could find the blueprints of how we train and try to do the same, but they won’t understand how we operate as a unit. They won’t understand our failures and what we have learned to advance to where we are now.

    We practice continuous improvement and continuous learning. We constantly try to better our previous performances. If we win a bike race, we don’t go home, put our feet up and say, “We are the best team, nobody else will beat us”—we analyze our performance and it’s more than likely there are many things we could have done better, even if we did win the race. Why do we do this? Because on a different day, if minor mistakes are made again, the other teams might get the better of us. In the tech world, we may have achieved our fastest, most secure deployments to date with very few failures, but are we going to rest on our laurels and cease to improve? This is the antithesis of what we should be doing in DevOps environments—we should be constantly looking to better our previous work. At the organizational level, the business itself needs to stay competitive and ensure that customer demands are being met; technology allows the business to stay on track with these targets.

    Our processes worked very well, and they continued to improve. This was helped by the implementation of chatOps to develop our culture and collaboration. When I joined the squad, we communicated via email. This was slow, clunky and generally not the best way for a team of 16 people to communicate. Our inboxes were constantly full, and we sometimes missed messages. Enter Slack. At first, some of the guys were closed off to the idea of change—“Why don’t we just use email? It’s so much easier,” they said. It really wasn’t easier to use email instead of Slack; it’s just that they didn’t want to change and learn to use a new tool, even if it only took a few hours to get comfortable with it.

    This is a common problem I see when trying to implement DevOps tools in enterprise organizations. Even if working with Docker or Kubernetes is going to bring huge benefits for people, sometimes they just don’t want to change from what they know. We practiced empathy and explained the benefits, while showing the improvements in a short space of time. Similarly, we do this as DevOps consultants when guiding cultural and procedural transformations at enterprise organizations. Anyway, we ended up using Slack and before I knew it, the entire team culture had changed for the better. We used the various channels to keep things organized such as “racing,” “team rides,” “ICE” (In Case of Emergency), “coaching,” “random” and others. The culture improved because we were able to bond, have fun and joke with each other without fear of disrupting important conversations in other channels. Our coach is based in Santa Monica and we are based in New York, yet we feel as though he is with us on every training ride.

    I’ve seen Slack, Trello and other collaboration tools being used to improve collaboration in remotely dispersed teams. So why does improving culture matter? Shouldn’t people just focus on doing their job? Yes, of course they should. I’m not saying that this should be a distraction. But bringing your teams together means that they are more likely to collaborate in times of need. This will help your global teams with communication and collaboration. It will also help with employee satisfaction because people will feel a greater sense of belonging.

    Conclusion

    To summarize, Agile and DevOps methodologies can be applied outside of technology. These principles and practices can be implemented across entire organizations. They can help manage change and transformation while increasing productivity through automation and the adoption of modern technologies for cloud computing. Don’t get caught up in the DevOps tools.

    Crossing the finish line @ The New York State Criterium Championships – March 2018

     

    — Conor Delanbanque

  • The Human Touch: The Key to Optimizing Your Software Value Stream Network

    The Human Touch: The Key to Optimizing Your Software Value Stream Network

    Software has no intrinsic value until it’s working in production and supporting a customer’s business initiatives. With that in mind, it’s imperative that organizations focus on how they reduce the time to value, not just time to deploy. And that’s achieved through people.

    A common misconception is that technology underpins a software delivery value stream. This is only half true. Tools add immense value and help people to be more productive—whether through advanced functionality (such as the ability to visualize and manage workflow) or by automating repetitive activities (such as test cases and releasing code). All of this boosts throughput. But it’s actually the human touch that drives a product.

    It’s all about how practitioners work together to understand and address a discerning customer’s needs with software. To that end, organizations must ask themselves, “How do we optimize practitioner collaboration?”

    It’s All in the Artifacts

    Collaboration optimization lies in how organizations handle product-critical information—artifacts such as features, epics, stories and defects—that are created for the very purpose of sharing with collaborators.

    Therefore, the follow-up question that organizations should ask is, “What system do we have in place to ensure these artifacts are flowing in real-time, so the information is always up to date, ensuring that all work is always accurate and adding value?”

    Fortune 100 companies have already asked these questions, making a number of significant discoveries:

    1. The software value stream is actually a social network of collaborators who are connected by artifacts—hence, “value stream network.”
    2. This network consists of two different, but equally important, phases: “plan and design” and “build and deploy.”
    3. The network by default is not explicitly connected because the tools collaborators use do not naturally integrate with each other and automatically flow artifacts between them.
    4. This disconnect forces collaborators to waste precious time chasing up information via manual means such as email, status meetings, spreadsheets, tool-switching and duplicate entry.
    5. This manual communication is non-value work that costs organizations up to $10 million in productivity overhead (based on a typical 1,500-person IT department).
    6. Such archaic knowledge-sharing practices significantly compromises the fidelity of data as it goes from one practitioner to the next.
    7. The “plan and design” phase—the part of the software delivery process that determines what product/feature is to be built/fixed—is detached from the “build and deploy” phase.

    Through these discoveries, the last one in particular, organizations came to an important realization: They have no real way of knowing if they’re building the right product because there are six degrees of separation between the people who design the product, and those who build and deploy it.

    The result? Disgruntled customers and a big loss of competitive advantage. To address this, they’ve created a new infrastructure that provides a framework to connect and support both phases.

    By fusing the value of people and technology into a defined value stream network that is visible and traceable, they have a seamless flow of work that they can see, measure and control. They’re now in a position to identify where value is created and lost, enabling them to make informed business decisions to make better products.

    The Software Value Stream Network

    The enterprise software delivery process has long been misunderstood as a traditional manufacturing system that is linear and sequential—thanks in part to the industry’s failed adoption of the Waterfall model. The negative influence and impact of that experiment still lingers today.

    The process is linear in the sense that there are two clear bookends: A product/feature/fix is requested (A) and then that product/feature/fix is made (B). There are also established phases across the production line: conception, initiation, analysis, design, construction, testing, deployment and maintenance. But that’s where any real similarities end.

    In reality, the process is more of a latent network of critical interactions between collaborators. These bilateral working relationships exist at key stages of the software delivery process, such as developer and tester, developer and operations, etc.

    When you actually lift up the floorboards of the process and peak inside, it looks more like a social network connected by the flow of artifacts, which looks something like this:

    Most of these relationships have developed over time as organizations have introduced a higher degree of specialty in key stages across the value stream. This evolution has been vital in helping organizations to:

    •   More deeply understand customer requests/issues
    •   Clearly articulate the requirements/defects
    •   Enhance product management/optimization
    •   Acutely break down requirements into epics and stories
    •   Write more accurate code faster
    •   Automate and accelerate higher levels of testing
    •   Continuously deploy
    •   Introduce tighter feedback loops
    •   Continuously improve and innovate

    While all these new roles, disciplines, purpose-built tools, processes and methodologies have helped different practitioners to perform better, it’s also introduced huge technical complexities. How do you flow those all-important artifacts if the tools are disconnected by design, and when collaborators are working in silos, trapped in “administrivia” hell? It’s tantamount to having a sports car with no gas and a confused driver. Or an airplane flight network with no air traffic control. It’s a banana republic.

    Building the Right Product via Value Stream Integration

    Agile and DevOps illustrate just how vital integration and automation is to this flow of work. Agile brought in prioritization, feedback loops and sprints to build the most important features as fast and accurately as possible.

    DevOps, meanwhile, has accelerated speed of deployment. In that sense, the build and deploy phase is optimized. Automation has work flowing seamlessly between Dev and Ops teams, and eliminated/mitigated cross-team issues, such as miscommunication regarding what products need to be deployed, and when.

    But what’s the point of faster delivery if it’s not what the customer wants?

    We need to consider how wrong features are developed. If you turn to look upstream, you realize that the teams planning and designing products are affected by the same misalignment that used to plague DevOps teams.

    The disconnections between collaborators and their tools in the creative software design process mean that often what’s implemented isn’t right or is delivered too late. In either case, it doesn’t add value.

    But that can be radically changed through sophisticated toolchain automation, which shares artifact data across tools and automates the hand-offs between owners in the “plan and design” stage.

    From the sales to product to PMO to business analysts, by bringing these customer-centric, product-focused teams together, you dramatically improve the product articulation process and the speed at which its performed.

    You build features closer to the customer’s needs. You contain that precious value. And then through automation you can flow that value downstream to developers to create that value. It is as big a revelation as DevOps.

    How is this all done? Enter value stream integration. This process is much more than just connecting two tools, say Jira and HPE QC. It’s about connecting all tools, practitioners, disciplines and processes to define and create a free-flowing network that is automated and scalable. Think of it as the plumbing in a house. If even one pipe isn’t connected, there’s no flow of work (value).

    Value stream integration connects the pipes to:

    •   Automate Information Flow Across Value Stream: It enables the frictionless flow of artifacts, as well as information from events across the value stream. This removes non-value-added work and bottlenecks; increases velocity and capacity; enables automated traceability; and dramatically improves employee satisfaction.
    •   Enable Value Stream Visibility: When managers want to see metrics and dashboards to understand project status, to optimize the process or to ensure compliance, it has been nearly impossible to get a real-time, holistic view across non-integrated tools. Value stream integration unlocks lifecycle data from these application tool silos by automatically compiling lifecycle activity data into a single database. This data can be used to create consolidated, full-lifecycle reports and dashboards, as well as for traceability reporting.
    •   Create a Modular, Agile Toolchain: Value stream integration enables organizations to use the products that best support each discipline while getting the benefits of a single, integrated toolchain. This drives more value from each tool and allows organizations to easily add, replace and upgrade them, creating a proactive environment for innovation. It creates that value stream network.

    Finally … Manageability

    With this network, organizations can finally see, manage and optimize one of the most important processes in their business: the engine that drives their prosperity in a digital-centric world. You cannot underestimate how important it is to have this network, this complete system. This is the state where innovation thrives, and where continuous improvement can be executed. And, of course, where you gain the essential visibility to be confident you’re always building the right product.

    — Nicole Bryan

  • Time to Rethink Infrastructure Operations in Enterprises

    Time to Rethink Infrastructure Operations in Enterprises

    “Business is technology and technology is business” seems to be the new mantra as the rules of corporate strategy are being rewritten across industries. According to IDC’s survey in 2017, “Two-thirds of global CEOs will have digital transformation as part of their corporate strategy.”  That’s CEOs—not CIOs—who will have digital transformation as part of their corporate and, not IT, strategy.

    time for change

    While digital transformation is not a new phenomenon, making digitization core to the corporate strategy is a fundamental shift for technology teams. Delivery units, or “change the enterprise” teams, have warmed up to new ways of working over the last few years and are already on the learning journey. Most enterprises today have digital or agile transformation initiatives that tend to focus on the “change” aspect significantly. However, the less glamorous “run the enterprise” dimension tends to be forgotten or side-lined in these initiatives.

    The rise of DevOps was meant to bridge the gap between “change” and “run.” However, many DevOps transformation initiatives seem to be taking the same old path of least resistance that agile transformations have historically adopted: by focusing on the “change” and delivery teams.

    As a consequence, the infrastructure and operations teams—or, “run the enterprise” units—are now waking up to the new reality and have far less time to adapt than their “change” counterparts. The big question really is, What does it mean to infrastructure and operations teams when the CEO is now embracing digital transformation?

    This means that the technology infrastructure required to support business strategy requires a combination of the usual consistency and quality that infrastructure and operations have historically focused on—but only with greater speed, innovation, flexibility and customer intimacy. For the first time, business is looking at infrastructure as equal partners on this journey, not as a subservient IT function.

    What Caused This Shift in Thinking?

    Since 2001, there have been four game-changers in technology that have caused this shift in thinking: the Agile Manifesto of 2001, adoption of Cloud by enterprises, rise of startups and the DevOps movement. The world of technology as we know has been going through a tectonic shift as a consequence of these overlapping waves of change.

    Agile and DevOps

    Version One’s 11th “State of Agile” report highlights that 71 percent of organizations have current or planned DevOps initiatives in 2018. While Agile ways focus on effective team structures, value focused mindset and greater business alignment, DevOps has taken the principles to the logical next steps of delivering value by focusing on flow, amplified feedback loops and building a culture of learning and innovation. In addition, full stack automation covering infrastructure, application delivery and release orchestration has really catapulted DevOps into the top initiatives in organizations. In terms of architecture, new constructs such as evolving design, microservices, containers and serverless computing are all redefining technology infrastructure. The rise of these new ways of working is only piling immense pressure on traditional infrastructure organizations to reform and rethink every aspect of their operating model and challenge processes built on a foundation of perceived stability.

    The Startup Way

    Enterprises increasingly are being influenced by the ways of startups and how they are driving rapid innovation. There is a cross-fertilization of people moving to enterprises who are challenging the thinking in these enterprises, especially infrastructure area. Teams are now embracing once considered absolutely alien, if not impossible, concepts such as chaos engineering and anti-fragility.

    Cloud

    The recent RightScale “State of Cloud” report reveals that 97 percent of enterprises are using cloud in 2017. However, only a small share of their workload has actually been moved to cloud, indicating the potential for cloud in the enterprise context. This indicates that enterprises are past the quick wins such as digital front-ends and will now struggle with the difficult areas of their applications—the bulk of the enterprise apps where the traditional infrastructure constraints and mindset rules. Enterprises can tackle this by truly embracing change, thinking big but starting small with experiments and continuous refinement of the every single dimension of the way infrastructure teams operate.

    “Nothing is permanent but change.” The wise and timeless words of Confucius ring true!

    — Aditya Vadaganadam

  • DevOps and Low-Code/No-Code Tools

    DevOps and Low-Code/No-Code Tools

    Helping out a friend, I had reason to revisit low-code/no-code solutions and poke around a bit. The industry has been relatively quiet on the topic, but changes in systems and marketing made me feel the need to offer an update on the relation of current solutions to DevOps.

    Low-code/no-code solutions have been a panacea in the eyes of some both in and outside the industry for decades. 4GLs were the earliest widespread attempt to make an environment that eliminated the need for developers, and while some years are quieter than others, the movement has not slackened.

    There are even some things we should consider adjuncts to low-code/no-code environments that normally are not considered. Traditionally the drive for low-code/no-code solutions is to move faster than IT normally turns out solutions. This would force us to consider the spreadsheet in accounting with 50,000 lines of special functions as a companion solution—faster than waiting for IT and more agile.

    The thing is that low-code/no-code environments have increasingly realized that IT is going to be their best customer, adjusting their offerings to focus more in that direction. At the same time, agile development and DevOps have made low-code/no-code solutions more palatable to an IT shop that is looking to continue the gains brought about by these technologies.

    The future appears to trend toward low-code/no-code solutions being an adjunct of agile and DevOps. I don’t say this lightly, but the signs do seem to point that way. Agile shops, having become more responsive to business needs, will hit a limit on what they can deliver, while the appetite for faster and more responsive systems delivery will not abate. Low-code environments that offer the following will become a very real option to extending the gains of agile and DevOps:

    • A clear and understandable way to code when needed in the language of choice for the IT environment.
    • A high level of feedback on what is wrong, possibly including suggestions on how to fix it.
    • Transparency in what is actually happening behind the scenes with the “no-code” part of the system.

    The idea is simple. By handling tasks that we, as developers, do over and over and over, we are freed to spend more time doing the truly innovative and environment specific work. This would seem to offer the next step in the growth of agile.

    Those tools that offer options such as automated deployment will step into ARA land also, making the Ops side of DevOps happy—in shops that haven’t invested a fortune in setting up ARA already, anyway. If low-code can offer the ability to say, “Here is a problem,” then the developer can submit a fix, and it is submitted to test, and if tests pass is sent to production (and some of the tools do), then the only questions are, “Does the code generated by the low-code part perform? and, Is it easy to integrate existing libraries, etc.?” If the answers are yes, then we also have an add-on for DevOps that delivers value.

    We live in interesting times. It will be very cool to see this all play out, and interesting to see where it heads after this. At least one vendor is betting on artificial intelligence (AI) and machine learning (ML) to offer increased productivity.

    Of course, it isn’t a panacea. I read where a vendor said, “We automatically update your queries when a table is modified,” which spawns another dozen questions for me, starting with “Really? What do you do about code that was counting on that field much later in processing? Do you check that screens are displaying defaults, or that variables used to make decisions aren’t always the default value? Do you flag this stuff so devs can ‘fix’ what automated query changes ‘broke’?” I’ll be digging in some more, because now they’ve got my attention. So you’ll probably be hearing more from me about this topic.

    Meanwhile, let the future worry about the future, and keep kicking it today. Just stay aware of what else is going on out there that might be handy in your toolbox.

    — Don Macvittie

  • DevOps: What a Wonderful World it Would Be

    DevOps: What a Wonderful World it Would Be

    The promises of DevOps are clear. They offer faster iterations, allowing ops to keep up with agile development and make an IT organization more responsive to business needs.

    DevOps doesn’t make less code. It doesn’t reduce configuration files. It doesn’t eliminate deployment issues (though it might reduce them). It doesn’t eliminate testing; indeed, it creates more if Agile and DevOps are taken to their logical conclusion:  More testing, earlier. DevOps doesn’t reduce the need for security.

    In short, it doesn’t reduce workloads. It might automate some of them, freeing up man-hours, which is absolutely a positive result—one of the more positive results, in my opinion. But it also introduces the overhead of the DevOps toolchain, so some of that automated freed-up time is already spoken for.

    And automation of testing (for example) is something that’s been around for decades. The first UI-based automated testing system I recall was Microsoft Test. I managed a team that used it extensively to automate test runs of the software we delivered to financial services. It worked well, and there was much gnashing of teeth when Microsoft discontinued the product.

    The same is true of most of the time-saving benefits of DevOps: We’ve tried them before, but DevOps approaches them from a different perspective—and, in many cases, a more sustainable perspective.

    I’m a DevOps aficionado, make no mistake. The reason I’m stressing what it won’t do here is because misperceptions drag it down. Don’t go into it looking to reduce costs, because at least in the short run you won’t. Don’t go into it looking to save work, because at least in the short term those savings will be minimal. Going into it for the wrong reasons means the things you are measuring will ultimately look like failures, even if all the good DevOps offers is reaped.

    Instead, go into it looking to be more responsive. Go into it looking to beat your competition to market. Go into it looking to increase communications across the team. Delve into DevOps for the right reasons, and you’ll be measuring the right outputs.

    And remember, it is an iterative process. Don’t overstate expected benefits on the first pass. Low-hanging fruit makes it look as if DevOps will do a lot of things, but low-hanging fruit often has tight ties into firmly rooted trees, making it harder to grab than originally thought. Use a measured approach and look for consistent improvements. In some organizations those improvements might be small at first. But use your knowledge of the organization and application portfolio to create a measured success plan.

    Reaping the benefits is worth the change DevOps brings. The ability to tell a line-of-business leader that the new functionality they are considering could be mocked up and ready for review in a week or two is a huge benefit to the organization. And it doesn’t hurt at all that such responsiveness will raise the rest of the organization’s opinion of IT.

    So it is a wonderful world, if your goals are the right ones.

    — Don Macvittie

  • Defects: Pay More Attention to What, Less to Who

    Defects: Pay More Attention to What, Less to Who

    Software defects have been with us forever, and while DevOps and agile can reduce the number of defects generated, it is unlikely this number will ever get to zero for a given organization—and even if it does, it won’t likely stay there. Simply put, we’re developing increasingly complex systems on increasingly short timelines. The reduction possibilities that DevOps offers cannot completely combat this combination.

    But the quest to keep defects down is necessary. The goal should always be “Less, not more.”

    One of the things we have historically done is associated defects with developers. It is natural to want to know “who broke the entire payroll system,” but quite often “who” is the wrong question.

    A huge benefit of both agile and DevOps is that it allows us to look closely at where the process fails and determine why. While every developer in a shop knows “you should avoid modifying that module,”—and those are the easy ones to target for early bug reduction—there are other modules and source files that might be a hidden source of a certain percentage of defects. Using the defect tracking toolset to keep tabs on where bugs are occurring might show that the problem is a given developer, but more likely it will show that the problem is a given set of source.

    So if you are not already, it is time to start tracking defects closely and looking for patterns. There are some great documents out there about relatively common break points that might be useful. However, reviewing your own data is more useful, as it isn’t about common—it’s about actual problems in your source pool.

    This is all part of the process for agile and DevOps, I’m just saying it a different way than the usual “blameless” discussions because I think they fail to connect for some shops. Of course blameless is great, the point here is focus on problems—whether that is blameless or not depends upon which problem you find you have.

    Meanwhile, keep cranking it out. If you’re working on a viable product, then defects are an annoyance, but not something to gnash your teeth over. You are developing a viable product, after all.

    — Don Macvittie

  • Can Agile and DevOps Get Along?

    Can Agile and DevOps Get Along?

    There is a common misconception that Agile and DevOps cannot work together, which does a disservice to everyone in the software development and delivery field. In fact, both approaches can significantly add to the Scrum pillars of transparency, inspection and adaption to better enable product teams to develop, deliver and sustain complex products. Both Agile and DevOps embody great ideas and enthusiasm that can add value to how teams work, ensuring the success of their organization in a world where business is both driven and disrupted by software.

    For businesses to take full advantage of the opportunities presented by Agile and DevOps, both communities—the Agilists and DevOps professionals—need to rectify the misunderstandings between the two approaches that amplify the differences between them. They must understand why these myths permeate and learn how to overcome these barriers so Agile and DevOps can finally play nice in the sandbox.

    Agile and DevOps: The Misunderstandings

    Both Agile and DevOps pursue the desire to deliver value to customers in a more effective way, but from different starting points—Agile with the developer, and DevOps from operations. Their emphasis also varies, leading people to believe that they are different things: Agile emphasizes team interactions, culture and values, while DevOps emphasizes delivery pipelines and flow.

    These apparent differences are further amplified by misunderstandings that do nothing but to pit the two approaches against each other:

    • Some Agilists think all process is bad, while DevOps is steeped in a history of stability through process.
    • DevOps uses tools and automation to increase speed, while some Agilists view tools as unavoidable at best and a distraction at worst.
    • DevOps uses holistic systems to solve problems, while Agile uses people to drive change and discover it through work.

    It is disheartening that these misunderstandings exist because Agile and DevOps share more in common than most realize. For instance, both include faster delivery cycles, smaller increments or releases and use feedback to improve and remove waste and impediments. Both approaches can also influence a cultural shift in an organization that promotes small changes that are easily manageable so they lead to bigger and better business outcomes over time.

    If they are more alike than dissimilar, then what exactly is holding these two methodologies back from getting along?

    Politics, Organizational Silos Widen the Gap

    Organizational changes, such as new hires and managerial moves, etc., attract strong personalities who get recognized for driving change and who get rewarded when those changes produce results. When their success is tied to the success of either their Agile or DevOps program, competing political goals are deemed a threat and lead to confusion:

    • Different tool strategies: How many times do you hear, “The Agile teams use JIRA, but the DevOps teams use Service Now”? These tools and the vendors selling them focus on a change initiative to drive purchases. This makes it even more difficult to integrate Agile and DevOps, as their data and processes are hidden within different tools.
    • Politics rather than value to make decisions: The saying, “It is not what you know, it is who you know,” is not just true in national capitals, but in any organization. Politics is the reality of most organizations and that means that the associated initiatives not only come with baggage of the work, but also with the politics of who is driving them.
    • Increased division and organizational conflict: Ironically, the DevOps and Agile movements call for unification and collaboration between different groups. However, when practiced, they can sometimes lead to even deeper gaps within traditional groups due to their different terminology and ideas.

    Different Cultures

    Agilists consider process to be “emergent” in response to the needs of the situation. This means teams are empowered to make decisions about how they work via a light framework (such as Scrum) and then add any other process they think are needed to achieve more value and faster feedback.

    DevOps traditionally takes a different tack, with a focus on systems thinking with flow models and value streams.

    Despite the culture differences, Agile and DevOps need consistent language and concepts to work together, or else communication breaks down and collaboration suffers. This leads to alienated stakeholders, increased fear and defensive behavior, poor product and service quality, lack of transparency and visibility and, more painstakingly, never getting anything done.

    Scrum Breaks Down the Barriers to Agile and DevOps

    Delivering value to customers through software requires elements from both Agile and DevOps coming together in a comprehensive process that is fluid and enables the teams to work in a dynamic manner. Through Scrum that empowers cross-functional, business-oriented teams to deliver software continuously, work flows seamlessly between all the right stakeholders, in real time.

    However, simply having cross-functional teams in your organization isn’t enough to make sure all involved are working in a holistic way to deliver customer value. Organizations need to make the right decisions about who should be in or out of a team, since different technologies require specialist skills and infrastructure to support it.

    A cross-functional team must have all the competencies needed to accomplish the work without depending on others who are not part of the team, as described in the Scrum Guide, to deliver a working product.

    As technology, market and environmental complexities and their interactions have rapidly increased, Scrum’s utility in dealing with complexity is proven daily—Scrum teams are highly flexible and adaptive so that they can deliver products of the highest possible value. When misunderstandings are put aside, and people learn to communicate with each other, cross-functional teams comprised of Agilists and DevOps with all the right skills can deliver done software, faster for your organization.

    — Dave West

  • ‘Let the Developer Choose the Language’: Too Much of a Good Thing

    ‘Let the Developer Choose the Language’: Too Much of a Good Thing

    As more and more enterprises drive into the DevOps space, this is less of an issue, but you still hear stories that make one cringe. The idea of DevOps (and, by extension, agile) is to get products to market faster and spend less time managing/fixing. Some take the idea to be freedom to choose—and it is to some extent, as long as an organization remembers that it can be (and often is) an increasing management nightmare to pile on product after product. DevOps already demands a fair number of new products to manage, so standardizing those products is imperative to the long-term success of DevOps in the organization.

    Technically, each team could choose its own toolchain, programming languages and data stores in a vacuum, ignoring the rest of the organization. But that is a terrible direction to take in almost every scenario. Yes, we have a lot more choices for languages and environments, and most of DevOps is still in development stages, so there are a ton of products to choose from. But because you can have them all in production doesn’t mean you should. I still see some pundits saying, “Let the developer choose the language.” That’s fine, as long as part of “choose the language” is “with our current architecture as the default.” Five languages or three CI/CD tools or several ARA tools is overhead—overhead to maintain, to keep skills current in and to troubleshoot. And it is overhead that isn’t going away without a project to replace it.

    I’ve written about this before, and yeah, this is the agile/DevOps technical debt. It happens more than you’d think, and yes the product gets to market fast, but if the life expectancy of the product in question is more than a year, you’re going to be maintaining it quite a bit. People who point to microservices or serverless and say, “Each could be in a different language!” have never worked in a long-term maintenance setting, I would guess. It’s hard enough to eliminate redundancies that had to happen in the normal course of business/application growth; making it worse by creating redundancies based on developer whim is just a bad idea.

    That could be my experience talking. I’ve lived through the “No, we can’t do that because the backend is written in (insert obscure toolset), and it won’t integrate with (insert technology du jour)” conversations. Both as a developer and as a manager. It creates future problems that didn’t have to be.

    Dalibor Siroky hinted at this problem in point number three of his “Five Biggest Ways to Fail at DevOps” article, but I thought it needed a bit of an expansion. Again.

    Keep kicking it, bring in new tools that add value, but don’t just build redundant chains because Developer Don is most comfortable with the programming language emoji.

    — Don Macvittie

  • Digital Transformation: How DevOps Could Have Revamped Toys R Us

    Digital Transformation: How DevOps Could Have Revamped Toys R Us

    The story you’re about to read and its recommendations are real; they contrast with what you have heard so far about digital transformation and DevOps implementation. Its objective is to convince you about that the time to go digital is now; it’s about transforming your business, not only IT; and to do it you’ll have to implement DevOps entirely.

    In September, Toys R Us, one of the world’s largest toys store chains, filed for bankruptcy protection. A better understanding of how disruption in the retail industry impacted the company would have helped to prevent its downfall and to reinvent its business model. For example, providing Toys 2.0 (a generic term referring to a new generation of toys built on cloud, mobility, IoT and AI technologies) gaming services could have helped the company survive.

    Shifting to Toys 2.0 demands a revolution in thinking, vision, practice and value that business leaders, CIOs and IT vendors haven’t done yet. They must forget the obsolete notion that technology and speed determine business success. Adopting paradigms that stress innovation and solve competitive constraints is the safest path toward fast and durable revenue.

    The biggest problem Toys R Us faced was the fact that consulting firms and IT vendors either didn’t understand the challenge as an industry disruption issue—or if they did, they improperly addressed it.

    Let’s clarify: To tackle your industry’s disruption, you must adapt your business model including the set of interactions, staff and skills, processes and practices, beliefs and values and infrastructure and tools used across your company to generate revenue. Industry disruption is the challenge, and DevOps through its many assets is the unique business capability companies need to quickly transform and win their digital battles.

    The problem, as Alan Shimel, founder, CEO and editor in chief of devopsy.kinsta.cloud, puts it:

    “While so many companies have already started to use DevOps or planning to use DevOps and related methodologies soon, there is only a small percentage of companies that are end-to-end DevOps. This means there is still tremendous opportunity in bringing DevOps to market.”

    How should have Toys R Us leveraged DevOps to tackle its industry’s disruption? Why did investing in that global digital e-commerce platform prove insufficient? How well did they transform their business model? How would have implementing DevOps 100 percent helped to overcome Amazon and Walmart competition?

    The principles, ideas, practices and solutions in this article are part of a case study we developed at ITaaSNow to see how to help companies facing similar challenges.

    What is DevOps?

    Alan, who is a major contributor to DevOps, is right: Only a small percentage of companies have fully adopted it. DevOps is much more than the automation toolchains consultants and vendors talk about. If they want to help lines of business survive and keep their job, CIOs must get a grasp of DevOps’ business facets.

    So, what is DevOps? Why do I think, it would have saved Toys R Us from digital downfall?

    DevOps Beyond the Operations Automation

    From my experience implementing it, DevOps is a business capability that seeks to optimize business operations (versus the sole IT operations). It provides companies with four competitive advantages needed to meet the digital business competitive challenges:

    1. Enterprise collaboration
    2. Business agility
    3. Short time-to-value
    4. Innovation culture
    Figure 1 – DevOps Overview

    As the picture shows, DevOps is a set of concepts aggregated around the notion of value stream to form a digital business management capability. It’s structured around four stages that format the company’s interactions and workflows into a powerful digital business management capability. These stages include:

    • Plan & Measure – addresses strategic watch, service prioritization and revenue generation through innovation portfolio management practices and infrastructure.
    • Develop & Test – takes advantage of agile and lean practices and infrastructure to ensure innovations design, development and testing.
    • Release & Deploy – leverages continuous delivery practices and infrastructure to timely deliver the right innovations to the right markets.
    • Monitor & Improve – stresses continuous improvement to guarantee not only premium customer experience but also customer value.

    Four business value catalysts are used to act as revenue generators:

    • Culture & Values
    • People & Skills
    • Methods & Practices
    • Tools & Infrastructure

    As you can see, DevOps transcends application development issues; it covers the entire business life cycle.

    DevOps Framework as a Facilitator of DevOps Implementation

    So, what is a DevOps Implementation framework?

    Concretely speaking, it’s a bundled DevOps platform combining managed PaaS infrastructures and optimized value streams to help companies boost their digital business:

    Figure 2 – DevOps Implementation Framework

    The platform seeks to help businesses implement DevOps end to end and builds on three principles to guarantee digital profits:

    1. Innovation as the revenue driver – It’s the belief that business can resist the competition imposed by technology companies such as Apple, Google, Amazon and Facebook when they use innovation to keep customers loyal, attract new ones and generate revenue.
    2. Value stream agility as the foundation of market responsiveness – It’s the demonstrated notion that companies are market-responsive when they rely on agile and lean value streams.
    3. Managed cloud as the catalyst of business performance – It’s the proven idea that when companies are focused on their core business, they innovate and offer superior customer experience.

    The 5-Step Digital Transformation Blueprint

    The 5-Step Digital Transformation Blueprint makes DevOps implementation fast and easy. As illustrated, it’s a comprehensive business transformation framework I developed to help clients with their digital transformation initiative:

    Figure 3 – The 5-Step Digital Transformation Blueprint

    It’s an integrated and inclusive approach to business transformation that involves the company’s relevant contributors. It structures the transformation effort into a five-stage transformation journey:

    • Digital strategy: In this step, the transformation team uses tools like the SWOT analysis matrix to help business and IT executives understand the changes in the disrupting industry along with the impacts on the company’s business model.
    • Value stream: In this step, the transformation team uses tools such as the value stream mapping to identify the required changes and get business and IT executives agree on the company’s future value stream. Interactions, staff, skills, processes, practices and tools are at the heart of the discussions.
    • Digital platform: In this step, enterprise and solution architects work together to identify and implement the technology and tools selected to support activities in the value stream.
    • Digital organization: In this step, a panel of business and IT users at the operational and executive levels experiment the new digital business model under actual business conditions. The tested digital business model is then deployed as the new digital organization.
    • Digital business management: In this operation phase, the value stream and the digital infrastructure are extensively used to run the business.

    How Chief Digital Officers Set Up Top Digital Businesses

    It would be remiss on my part not to recommend the online digital transformation course that gives DevOps practitioners—executive and operational from both the business and IT—the insights about implementing DevOps as the foundation of digital business.

    “How Chief Digital Offers Set Up Top Digital Business: Establish DevOps as the Foundation of Your Digital Business” is a 15-module course that offers concise, practical and powerful tips to help practitioners learn and apply strategies and tactics to effectively plan, monitor, facilitate and lead business transformation including IT to digital.

    Analyzing Amazon and Walmart Disruptive Impacts to Spot Guidelines for Reinventing the Company

    Many bankruptcies happen because business leaders don’t take the time to understand their industry’s disruption and the impacts on their overall business.

    The reality on the ground is, industry analyses and competitive strategies are either overlooked or totally ignored. Understanding the toy industry’s disruptive effects would have helped to identify Toys R Us’s transformation as the safest solutions to avoid bankruptcy.

    DevOps implementation framework makes the difference, it sees DevOps as the foundation of digital business success. It considers industry and competitive analyses essential steps of the DevOps transformation initiative. Figure 4 outlines the outcomes that resulted from our industry analysis:

    Figure 4 – Toy Sector’s SWOT Analysis

    As illustrated, it was shown that Toys R Us should have exploited Amazon’s vulnerabilities and take advantage of the technological opportunities to reinvent itself as a Toys 2.0 gaming service provider. These vulnerabilities include little product differentiation while technological opportunities refer to the new toy generation built on cloud, mobile, IoT and AI technologies.

    The company’s strengths—reputation, longtime industry presence, brand equity—combined with the cost savings resulting from managed cloud adoption would have not only simplified its transformation but also challenged Amazon on costs.

    Changing the Value Stream to Establish Innovation and Invent the Company’s Digital Business Model

    Business leaders don’t take the right transformation decisions because they lack insights into the changing competitive environments. Aggressive marketing and sales campaigns supporting short-term financial gains. Toys R Us failed because it bet on its debt reimbursement at the expense of its business model reinvention.

    A DevOps implementation offers a value stream mapping framework that not only facilitates innovation culture implementation but also speeds up digital business models development.

    Figure 5 – Value Stream Mapping, Agile, and Lean

    The objective of the value stream mapping I performed was to outline the features of work environments that take advantage of DevOps Three Ways to equip the company with the competitive advantages needed to resist Amazon and Walmart. The following lines summarize the outcomes.

    Implementing DevOps Flow Principle to Defeat Amazon and Walmart on Innovation: The value stream mapping recommended to leverage Agile and Lean principles to properly implement the flow principle and defeat Amazon on its little product differentiation. The Agile’s features to stress included collaboration and prioritization while focus on people engagement and customer value were the Lean’s benefits to emphasize. Their combination were likely to not only foster innovation culture but also accelerate time-to-market and time-to-value.

    Implementing DevOps Feedback Principle to Guarantee Superior Customer Experience: As to the feedback principles, the value stream mapping recommended to use Lean’s value stream assessment and Agile’s review mechanisms. The objective was to institutionalize continuous improvement practices as a means to ensure superior customer experience.

    Implementing DevOps Experimentation and Learning as the Foundation of the Innovation Capability: As to the experimentation and learning principle, the recommendation was to take advantage Agile’s prototyping and learning curve-lessening features to institutionalize innovation and risk-taking cultures. They were likely to not only help the company challenge its competitors on innovation but also ensure durable revenue and customer loyalty.

    Enabling the Value Streams with Managed PaaS Services

    Using technology ingeniously to boost business activities is clearly one of the keys to succeed in the digital business. Today’s DevOps-timid approach focused on speeding up IT operations alone not only is ineffective, but it also doesn’t make sense. It’s the entire value stream that needs optimization, not just IT.

    The findings of the digital platform assessment I performed, showed as illustrated, that the global e-commerce platform wasn’t enough to compete against Amazon and Walmart. The company was at the embryonic stage of digital transformation maturity practices:

    Figure 6 – Toys R Us Possibly at the Embryonic Stage of Digital Transformation Maturity

    The findings suggested that using technology to innovate or creating the operational conditions to cut cost, accelerate time-to-market and be responsive to market opportunities were either overlooked or improperly addressed.

    Using a digital platform engineering framework allowed me to design in a record time the following architecture:

    Figure 7 – Possible PaaS Platform for Toys R Us

    The digital platform would have built on two cloud services supporting the value stream: managed software as a service (SaaS) and platform as a service (PaaS). Let’s discuss them.

    How Managed SaaS Services Would Have Helped to Fix the Cost Leadership and Business Agility Issues

    SaaS services would have supported the Plan & Measure stage. In addition to supporting varied management activities, it would have provided two fundamental tools that possibly could have enabled the Toys 2.0 strategy: customer relationship management (CRM) and innovation management portfolio. CRM tracks, measures and improves customers’ experience, keeps them loyal and guarantees revenue. Innovation portfolio management identifies innovation ideas, supports their implementation into digital services and monitors generated value. It gives the company opportunities to disrupt and control its industry.

    The on-demand and metered billing features of SaaS services would have allowed Toys R Us to get the needed software in record time while enabling time-to-market, time-to-value and responsiveness competitive advantages.

    How a Managed PaaS Platform Could Have Helped to Address the Express Delivery Issue

    PaaS services would have supported activities covering the Develop & Test, Release & Deploy and Monitor & Improve stages. In addition to providing the technologies needed to implement Toys 2.0 gaming services, it would have built on microservices to set up the automation tool chain and accelerate innovations development and delivery.

    Wrapping Up

    The recent IT innovations are disrupting several industries and expanding the digital economy. To avoid troubles, businesses must urgently start their digital transformation. Toys R Us, Sears and many others learned it the hard way.

    It’s urgent to understand that the digital transformation of your business isn’t only about migrating your IT to the cloud, implementing automation tool chains or deploying artificial intelligence tools across the company. It’s primarily about transforming with a strategic mindset your company’s business model from the ground up.

    Digital transformation is a complex process that engages the entire company, and requires specific methodologies, practices and tools. I renew my recommendation to take the online course, “How Chief Digital Officers Course Set Top Digital Businesses“—you’ll get from it the practical vision, approach, processes, practices and tools that’ll help you create the conditions for successful digital business with DevOps.

    — Philippe Abdoulaye

  • No-Ops: It’s Been 5 Years

    No-Ops: It’s Been 5 Years

    As I hinted in this post, the state of no-ops is not anywhere near no-ops for most use cases. The closest we have today would be software as a service (SaaS), which handles the majority of operations for us, but still requires some interaction by operations staff.

    The next closest I can think of is the extreme form of hosted. My company uses a hosting provider to handle its website and blog, and the provider does a ton of the heavy lifting. But it would be a fabrication to imply that this mean no operations were performed. The simple fact is that backups, software updates and user management all are in our operational ballpark.

    About That Five Years …

    Why did I mention five years? That’s when the “No-op flame war” erupted. PCWorld has a pretty good summation of it, with links to the blogs in question. The short version is Netflix claimed No-Ops (later it became pretty clear the company just renamed a DevOps-reduced operations group), and Etsy responded with, “That’s because you’re doing it wrong.” Then people jumped on the “no-ops is not a good phrase” bandwagon, and it was … fun to watch.

    But Again, That’s Not the Point

    We too often get caught up in buzzwords. What we want to do is reduce operational overhead significantly per app. This reduces time to delivery, and it frees up time to work on new apps that are useful to the business or spur new business. If we can operationally manage three times the applications because we’ve reduced operations overhead, then we have time to do more. One of the things we haven’t done real well in the past is phase out technology. Perhaps with the free time we could reduce the operational overhead caused by five different solutions to the same problem across our application domain, offering even more time to work on new functionality or product lines.

    Don’t Get Sucked In

    I would argue that both Netflix and Etsy were “doing it right.” You start from where you are, and you streamline both operations and development. You start to ignore boundaries where the ROI is measurable, and you make your environment better/stronger/safer/more agile, whatever the business needs. And then you iterate again. Who cares what you call it?

    No No-Ops

    If No-Ops is coming, it is a loooonnnnggg way off. That’s okay. Because we are improving. You may iterate through improving a zillion times, with different goals each time, implement advanced monitoring and recovery, and wake up one day to realize that you’ve reached “No-Ops.” But as long as you are improving the cycle time/maintainability of the environment you work in, worrying about no-ops is pretty much a waste of time in the current environment.

    But You Knew That

    Here at the end, I’m kind of repeating what most of you already know. But when the hype hits, it doesn’t hurt to have a reminder. Most of you are keeping your business running and cranking out new apps as you’re able, and doing so at a faster pace than you were five years ago while others were arguing about no-ops. So keep doing that. Push Agile/DevOps further on each project. The more time you free from repetitive work, the more you have for doing/learning cool new stuff. So keep cranking, and one day you’ll have apps that are near no-ops.

    — Don Macvittie