devopsy.kinsta.cloud

Tag: automation

  • Release Automation: Delivering the Big Quality Payoff

    Release Automation: Delivering the Big Quality Payoff

    Have you ever gone down the path of testing the latest release of an application only to find out that you had the wrong version?

    Maybe you accidentally grabbed the wrong build or you didn’t have the correct configuration file for the environment you were testing against—or your database wasn’t updated correctly, or the script you were using was still pointing to a prior version of the software.

    Whatever the case, we can agree that keeping track of all the various artifacts, database schema changes, configuration parameters and other components needed to accurately test and deploy an application is both time-consuming and prone to errors.

    Today, many companies are looking to DevOps, continuous delivery and agile development methodologies to shorten their application delivery cycles and reduce the number of errors and the costs associated with application deployment processes. Nearly all enterprise companies are software companies, and their mere existence depends on delivering differentiating products and services via software to their customers.

    It’s imperative to be able to react to customer demands and competitive pressures quickly, if not immediately, to stay ahead of the competition. Gone are the days of yearly release cycles.

    Today we see companies deploying new capabilities into production environments on a monthly, weekly or even daily basis. At some companies, hundreds, if not thousands, of deployments are occurring across dev, test and production environments daily. The old methodologies we once used simply can’t support this type of volume and cadence.

    Employing Release Automation

    So, how are these companies transitioning to this new way of thinking? In a word—automation; specifically, by automating early and often.

    Everything about your release process is a candidate for automation. This includes environment provisioning and configuration automation, build automation, test automation and release and deployment automation.

    A customer of ours recently implemented CA Release Automation, and within eight weeks had automated 90 percent of its deployments. Deployments that once took more than five hours and a team of people now take a matter of minutes.

    “This not only saves us a huge amount of time, but it also mitigates the risk of human error being introduced by manual processes,” said Tim Dallmann, platform engineer at Jewelers Mutual.

    Another customer, a major airline, implemented CA Release Automation and immediately reduced their production outages from an average of 237 over a three-month period, down to 15, seven, six and five over the following four months.

    And Molina Medicaid Solutions reduced their cost of human error by 90 percent, all the while responding quicker to emergency deployment requests and improving deployment quality.

    Automation is critical, but I believe these results prove that implementing a framework to manage and deploy release processes in addition to automation works even better.

    Increased Flexibility and Visibility

    Continuous delivery helps break down barriers often seen between development and operations teams. It provides, even forces, a more collaborative approach and automation is a key enabler of this.

    “One of the best things about automation is that it gives us the flexibility to skill-up end users—deployments in our test environments are no longer solely the responsibility of the operations or IT teams,” explained Dallmann.

    With an application release automation (ARA) solution, artifact, build, configuration and release processes are no longer tracked with spreadsheets or embedded in complicated scripts.

    Increased visibility into these processes ultimately enables all stakeholders to take a more active role in the release process, and access to real-time information can significantly reduce the amount of time and resources needed to resolve and recover from problems when they do occur.

    Automation leads to:

    Integrating with the Ecosystem

    An enterprise ARA solution can help you create a single control point to streamline all your application delivery processes for better predictability and reduced risk. A critical component of this is the ability to integrate and orchestrate tools and technologies required along the continuous delivery pipeline, including source control, automated testing tools, provisioning and configuration management solutions and change management and performance management solutions.

    Integration with an application lifecycle management solution can provide even further alignment, governance and traceability between development projects and what is ultimately being tested and promoted to production.

    “By integrating the two solutions, we will have a complete audit trail of deployments to help us understand how, where and by whom stories have changed,” said Phil Nickolai, director of application services at Jewelers Mutual.

    “What we really needed was a solution to streamline our whole way of working,” he said. “We had numerous people working on projects, competing for resources and causing complexity. Now we’re back in control and getting smarter every day.”

    The Time Is Now

    Let’s get rid of the problem that exists between the keyboard and information kept in spreadsheets, scripts or emails. Not only will you stop wasting time testing the wrong version of an application, but you can also ensure you have the right test data, the appropriate test cases and the correct configuration parameters for your environment.

    Stress-inducing release weekends and nights become a thing of the past. Developers can spend more time on developing innovative capabilities that generate revenue and keep customers coming back for more.

    — Tim Mueting

  • Why Financial Companies Love Continuous Delivery

    Why Financial Companies Love Continuous Delivery

    The other day I was on the American Express website paying bills and browsing around the benefits and services afforded me when I ran across a “who we are” section. It read:

    “We’re a global services company that provides customers with access to products, insights and experiences that enrich lives and build business success.”

    Surprisingly, the term “financial” was missing from the description. This led me to wonder if other financial institutions are evolving how they think about themselves. Here are some other examples:

    • Bank of America: “Here to help connect you to what matters most.”
    • ING: “Empowering people to stay a step ahead in life and in business.”
    • MasterCard: “As a technology company, we connect individuals, businesses and organizations around the world, creating greater opportunities for all.”

    Financial Companies Love Their Customers

    My unscientific assessment of various financial services companies suggests that they are centered on delighting their customers. They are transforming who they are by altering their customer engagement models and essentially becoming better software companies. MasterCard has even reclassified itself as a “technology” company. These companies are transcending the role of transaction engine to offer seamless, intuitive digital experiences for their customers.

    And the proof is in the performance. According to Forrester’s Customer Experience (CX) Index, financial services firms, including retail banks, credit card companies, investment firms and auto and home insurers, are some of the top brands worldwide when it comes to CX.

    You might think that large financial institutions would be digital laggards because of legacy infrastructure, regulatory compliance pressures and overall risk-avoidance. But you’d be wrong.

    In another study, 81 percent of financial services firms said they are embracing digital to redefine their businesses, and 87 percent are on the DevOps bandwagon. Sixty-one percent even said they are disrupting their industry with advanced digital initiatives.

    To stay competitive and improve their customer experience aptitude, financial services firms are undergoing significant digital transformation.

    Getting the Pulse of Fintech Continuous Delivery Adoption

    But how progressive are these firms in terms of their application delivery practices? Working with CA, Gatepoint Research recently surveyed more than 100 IT executives from financial services companies to take the pulse of where they are in continuous delivery adoption. Results can be found in this report and infographic.

    First off, to see if these companies might be considered unicorns or stragglers, they were asked the total volume of monthly deployments in all environments. Forty percent fell at the low end with fewer than 50 deployments per month. Now, lower deployment numbers could mean that these are less agile organizations or that they have fewer applications in general. Somewhat correlative to lower deployment numbers, the survey found that 48 percent currently are not using an application release automation (ARA) solution. Without automation, it’s tough to scale release processes—particularly to keep up with agile development.

    Moving up the spectrum, 41 percent are moderately prolific at 50-500 deployments per month, and then 15 percent are what I would consider rock stars with more than 500 (7 percent of those over 1,000). Those doing hundreds of deployments a month often have agile development teams and more modular applications. They likely have continuous delivery practices in place that push more frequent, iterative deployments left, helping higher-quality applications to move through the pipeline to production.

    The primary application delivery goals for financial services firms for the next 12 months are to get better control of release processes, tools and environments; to reduce risk and failures; and to release applications more frequently. All of these needs typically stem from companies undergoing digital transformation—it feels more chaotic and disconnected, manual processes are breaking under the weight and there’s intense pressure to move faster.

    Compliance was not a top goal, and the majority (83 percent) claim that their current release process is adequate to meet regulatory requirements. However, they did acknowledge that they could improve in terms of knowledge safekeeping and auditability of the release process.

    The data certainly shows that financial services companies are adopting continuous delivery practices in some form and exploring solutions, including many open-source tools, to help them get the job done. They currently are focusing continuous delivery practices on tracking and reporting releases, planning and scheduling releases and automating test deployments. These practices ultimately help financial services companies to release high-quality applications anywhere at any time.

    How Continuous Delivery Helps Fintech Win

    Continuous delivery sits at the heart of many digital initiatives because of what it enables these companies to achieve for their customers:

    • Delivering features and updates that customers need when they need them
    • Delivering applications that meet performance and quality expectations of customers
    • Delivering secure, compliant applications (for external and internal customers)

    Continuous delivery enables continuous devotion to the customer experience.

    Continuous Release: Fast Customer Response

    With continuous delivery, you establish an application release workflow that automatically connects a developer’s update to parallel testing to staging and ultimately to production release. Because of the underlying end-to-end automation technology, these steps can happen quickly, seamlessly and with little to no manual intervention. Some might call it “push button release” or “zero-touch release.”

    It’s a controlled, tracked, governed, collaborative process. Application stakeholders stay aligned as application changes traverse the pipeline. Your customers can have access to fixes, updates and new innovations in days instead of weeks or months.

    Continuous Testing: Building in Quality

    In contrast to the significant time penalties and shortfalls that can happen with manual testing, in the continuous delivery pipeline, testing is shifting left and happening in parallel. Hand-offs and feedback loops are automated and efficient.

    Testing can happen earlier and without delay with the help of test environment simulation. Testing is on target through automatic test case creation, even from requirements. Testing is real-world with on-demand test data generation. And testing is rigorous with developers able to performance test apps at their end. The continuous testing that occurs as part of the process helps to build in quality and ensure top performance before reaching the customer.

    Continuous Compliance: Audit-Ready Processes

    Ensuring applications are compliant and secure is critical for your customers. It’s making sure customer information is safe and that application changes are tracked sufficiently for SOX or other reporting. It’s also about helping out those internal “customers” like auditors to prove compliancy and easily build reports.

    An automated continuous delivery pipeline can track releases, apply role-based governance, orchestrate proper data masking, deploy security policy updates and provide an audit trail for application releases—who did what, when and why. All this data can be collected and reported easily, which saves time and effort for auditors. You essentially stay compliant and secure all the time.

    Customer Satisfaction Through Automation

    Financial services companies are laser focused on using technology to delight and retain customers. And continuous delivery is the application delivery practice that’s underpinning these efforts. The companies that are exceptional at this, with thousands of deployments per month, are using automation tools such as Application Release Automation (ARA). These tools are the backbone of continuous delivery, providing the automated orchestration, governance, tracking and reporting for your pipeline—development through production.

    That’s why I’m not surprised that leading financial services companies make up a large portion of CA’s release automation customers, including Euroclear, GM Financial, ING, Jewelers Mutual and Western Union. Our customers put high value on delivering excellent customer experiences and are implementing modern software delivery methodologies, practices and associated tools to support their efforts. GM Financial recently said it is “the software company that does auto finance.”

    If you are serious about customer satisfaction and staying ahead of nimble competitive threats in the market, you must get serious about end-to-end automation. It’s time to truly embrace the practice of continuous delivery to be able to consistently release applications that are progressive, always compliant and completely customer-satisfying.

    — Patricia Johnson

  • Boosting Time-to-Market by Improving Software Development

    Boosting Time-to-Market by Improving Software Development

    Today’s business environments require “agility” in terms of being able to quickly understand and adapt to information, trends and new developments. For example, a company might employ a business intelligence suite that provides them with real-time information on product sales, which allows them to dynamically adjust content or strategy.

    Set a Baseline

    You can’t measure improvement in a business process without first establishing a baseline. The first step in reducing the time-to-market for software development is to look closely at the current development process. Review the steps involved in turning an idea into a live product or product feature and identify any bottlenecks, inefficiencies and constraints. Once this review is complete, set realistic goals that reflect the current situation and the improvements you want to see within a certain time period.

    Eliminate and Optimize

    During the review and baseline phase, you might come across activities that do not add value and do not speed up the process. Perhaps there are some unnecessary meetings during the mid-phase of the development process that typically do not uncover any problems and hold no real value. Or, there’s a documentation step that is a drain on human capital and takes away valuable time from testing and innovation.

    Identifying problems and bottlenecks is vital, but the efforts will be wasted if there aren’t open lines of communication. Poor communication causes software release delays, so establish efficient processes, set up the right team structure and adopt tools and technologies that allow transparent and fast communication between cross-functional teams. When it comes to teams, you want to find the right balance of staff. Generally, a winning team includes the right analysts, marketing managers and DevOps resources who can work together towards a common goal.

    Streamlining and Developing Incrementally

    As the team moves into development, there are steps you can take to increase efficiency. Independent tasks should be performed in tandem; for example, test scenarios built while various features are defined. Do not wait until development is completed before creating test environments; encourage the team to split parallel tasks that best match with their individual talents.

    When it comes to enhancements for business apps, there’s an inclination to go for the “home run” by greatly changing the application and including many new features with every update. For reliability and speed to reaching the market, you instead want an incremental approach, by which you develop controllable enhancements and bug fixes that provide immediate value to the users. Prioritize the “wish list” of feature sets so you can better manage and plan the updates that will go into future releases. Get buy-in from other departments and then build and create an enhancements schedule.

    Test Often and Embrace Automation

    In addition to having testers test the application, you should actively engage with clients and select users early in the development process so they can provide input on the overall functionality and user interface as well as help discover bugs. Your time-to-market can be greatly impacted if bugs aren’t caught early on, and you might even need to pull a newly launched application if the glitches impact users adversely.

    Improving this process requires defined and repeatable testing procedures and, to reduce time-to-market, these procedures must be automated at least partially.

    Consider these other benefits of software testing that will improve your DevOps:

    • Discovering bugs and software errors can improve the system’s capacity
    • Testing can improve the security of your applications
    • Improve customer engagement and satisfaction through a high QoE score
    • Reduce the total cost of ownership
    • Better meet industry standards and compliance regulations

    Other automated processes should be introduced into the development cycle, including environment provisioning, data or API mocking; builds management; and standards enforcement. For instance, automated deployment solutions give DevOps a repeatable process so engineers have more time to focus on crafting new features. Another example is release automation tools that manage workflow as software moves between development, production and testing. These tools not only speed up the process, they also eliminate potential for human errors.

    The modern business relies heavily on applications to communicate, complete sales and operate more efficiently. Companies strive to bring these apps to the market quickly so they don’t miss opportunities and can adjust to market conditions. To accomplish these goals, companies must adopt the right software development processes and tools to improve their time-to-market speed and capture a competitive advantage.

    About the Author / Ajay Kaul

    Ajay Kaul is managing partner at AgreeYa, bringing more than 27 years of experience in sales, consulting and managing small to large IT projects worldwide. Kaul has led AgreeYa through 16+ years of success, leading the company in highly competitive and complex markets and driving significant profitable growth with global workforce of more than 1,500. Prior to founding AgreeYa, he was responsible for managing engagements for Deloitte serving private and public sector clients across United States. Connect with him on LinkedIn.

  • The Right Automation Tool for the Job

    The Right Automation Tool for the Job

    My father was an antique dealer, and we did a lot of furniture repair in the workshop. All of us kids took turns working for the business as we grew up, and we learned some solid lessons in the process.

    One day, when I was still pretty young and very new at working on things, he pointed out a set of screws someone had set into an otherwise beautiful oak chair and told me to remove them while he worked on another piece for the same customer. I grabbed a screwdriver from the workbench and went to work, but it was tough going: The screwdriver kept slipping, and I was new at it, but I knew enough that gouging the wood was certainly not on the list of things one wants to do with antique furniture. I took my time, got all four out, but the screw heads were pretty mangled—close to stripped—and the end of my screwdriver was also showing marks. Being new at it, I thought nothing of it, and was proud of how few scratches I’d put into the wood.

    My father was a patient man for the most part, and I was proud of having done my part. So it was a complete surprise to me when he could barely contain rage as I showed off my handiwork.

    After he pulled himself back under control, he took the time to teach me that there was a difference between a screwdriver and a wood chisel, and that I should have seen it didn’t look like a screwdriver. It turns out I had ruined his best chisel, and the “few marks” I had left in the chair were gouges that we would have to carefully repair.

    Are You Using a Chisel?

    I learned an important lesson from that day. Always be certain, despite what you think you know, that you have the right tool for the job.

    I’ve certainly said that enough in my career—in development, in ops and in my blog. It is easy to see the tool in your hand as good enough to get the job done, but it is worth stepping back for a second and making certain it is.

    Not too long ago, I worked with an automation company whose largest competitor was not any competitor; it was, “We can do that by hand.” Evidence over time showed that the company could get partway there, and then would find itself with issues that kept it from getting the features/reporting it wanted and a metric ton of code to maintain. Unsurprisingly, some called to have the tool brought in three to six months after embarking on a build project.

    Using bash scripts, Maven/Gradle/Cmake and a dose of Python to do release automation is a good example of that syndrome in action. So is cobbling an application provisioning tool into a release automation system. Like a chisel, these are all astounding tools that rock in their given area of expertise, but are far from optimal—and can even cause problems—when used to automate software releases.

    Familiar Questions

    There is almost always someone willing to say that building is better than buying. These people thrive on the creation aspects of IT and are valuable assets. But the equation has to make sense. In software development, we’ve come to the point of “core business function” as the differentiator in the build/buy equation, and it has cut down the number of systems for which we have to maintain source. But DevOps is new, and a lot of DevOps is scripting, be it command line (bash scripts/batch files/ps) or Python/Ruby/php, straight commands built in-house or scripts for use through tools such as Jenkins. We’re generating a lot of artifacts, and those artifacts will require maintenance. It’s the technical debt of the near future, because environments are not static things, and those scripts will need to change over time.

    So the questions to ask will be familiar if you went through previous build/buy cycles and include:

    • We can build it here, but should we? What long-term costs and risks are we taking on by doing so?
    • If we build it here, what are the tradeoffs? What functionality could we have today that will go missing? What functionality will we likely never have in a home-brew solution?
    • If we build it here, what flexibility will we have, and how does that compare to available tools?
    • What are the timelines? Generally speaking, CI/CD tools can be up and building in a week or less, be truly productive in six weeks or less, and be a standard part of the build/deploy process within six months, even at cautious companies. What would building it have for a timeline, and are the resources required to complete the implementation constrained?
    • DevOps is growing and changing. A dedicated product will incorporate those changes through the upgrade process. Will time be available to staff to do the same on a home-built system?

    There is a Right Tool

    My experience is that IT shops who allow someone who claims, “We can do CI/CD/ARA with the tools we have today, it’ll just take a little work” to run with that sentiment end up feeling more like my father than happy that they made the choices they did. The complexity of the process, combined with the growing functionality and integration of toolsets from different vendors (via plugins or APIs), have made the purpose-built toolsets more effective than cobbled together is likely to be. And when the home-built tools fall short or needs change, the cost in IT staff resources is not something to be dismissed lightly. While blocking off a chunk of time to make IT more effective is important, it is also important to make the most efficient use of that time possible. Having a toolset dedicated to CI/CD and even release automation means less time maintaining source that just manages builds and application deployment. Save that time for things that improve efficiency even more, not keeping the status quo. There will certainly be scripts for your server and application provisioning tools to maintain, because that’s how those tools are designed. And if you are using a tool such as Jenkins as part of your toolchain, it generally has scripts as part of the process too. Increasing beyond that just to avoid implementing the rest of the CI/CD/ARA ecosystem would be a poor investment of time.

    As I told a friend recently: Yes you can implement a full-blown end-to-end ARA system in an application provisioning tool. By the same token, you can implement a fully functional responsive web app in assembler, too.

    — Don Macvittie

  • Dispelling the Myths Around Continuous Testing

    Dispelling the Myths Around Continuous Testing

    You may have seen the term “continuous testing” and thought it was just the DevOps flavor of the month. Some might even think it isn’t part of DevOps or isn’t for cloud or hybrid cloud applications. Well, think again.

    As it turns out, there are a few more fallacies around the idea of continuous testing floating around the application development community. Ready to find out?

    Chapter five in the new, free eBook “Continuous Testing for Dummies, IBM Limited Edition,” lists 10 myths. Here are a few to ponder:

    Continuous testing is only about executing test scripts

    Limiting the term “testing” to only the execution of test scripts is a misnomer. As my co-author, @Al_Wagner and I point out, testing and checking application functionality are not equal. There’s a long list of testing items to check off one’s list, the least of which is collaborating.

    Continuous testing is merely a buzzword

    Guess what? It’s a core part of that essential DevOps practice, continuous delivery. And, with customers demanding higher quality software faster, it isn’t going anywhere soon. In short, invest in continuous testing.

    Only agile teams do continuous testing

    False! Some or all parts of continuous testing are critical for any type of team as well as all players. This includes test automation, production-like test environments and realistic test data. A related myth—only testers contribute to testing—is also not true at all. The idea of continuous testing includes all team members. The book describes the roles everybody has to play. For example, did you know that designers can improve testing? They can work with testers to help them understand where the fragile parts of the application are.

    Automating tests means we need fewer tests

    No, the machines are not taking over. Automating tests does not mean cutting back on testing jobs. To the contrary, automation is reducing testers’ needs to act like cyborgs and spend more time doing what they do best: learn and experiment.

    Continuous testing isn’t for large, complex systems

    Au contraire! The best thing you can do for large or complex systems is leverage continuous testing practices and tools such as service virtualization to conduct end-to-end testing earlier and more frequently, rather than having each subsystem do all its own testing in isolation until just before deployment to production.

    To read more myths about continuous testing, check out “Continuous Testing for Dummies, IBM Limited Edition,” here.

    About the Author / Marianne Hollier

    MarianneMarianne Hollier is an IBM and Open Group Master Certified IT Specialist in application development. She has strong, practical expertise in measurably improving the software development life cycle and driving the necessary cultural changes to make it work. She is instrumental in architecting, tailoring and deploying the IBM DevOps best practices as well as appropriate software development tools on many types of projects, from large to small, long to fast-track, agile to traditional. Marianne is especially passionate about all things testing—process, tools, culture and automation. Her experience is broad-based, spanning both custom projects as well as standard software packages that apply to many industries, including refining, pharmaceutical, telecommunications, health care, financial, automotive, government and retail. Connect with Marianne on LinkedIn or Twitter.

  • Be Here Now – Agile and DevOps in the Current State of Synergy

    Be Here Now – Agile and DevOps in the Current State of Synergy

    If there’s one thing that our culture does not lack, it’s a litany of pithy catch-quotes meant to remind us to focus on the task at hand; yet, this often runs counter to our tendency to fixate on matters of the past and future.

    While we are so often encouraged to “live in the moment” and “seize the day”, this is hardly an easy task, especially as related to any sort of emerging phenomenon that compels us to improve on historical missteps, and predict how things should change for the better.

    Register today for the 3rd Annual DevOps Virtual Summit.

    With the methodologies of agile and DevOps the tendency is, of course, to do the same, harping on the inefficiencies of previous software delivery models and painting an elaborate portrait of how related workflows are being transformed overnight.

    The simple truth is that it’s easy to get excited about concepts that hold such tremendous potential in recasting an increasingly critical vital set of processes, and lose track of where things actually stand.

    To note, while a recent research project published by Coleman Parkes found that over 80 percent of organizations have adopted some manner of agile and DevOps practices, and see them as critical in transforming their internal software factories, only a third have adopted the methods on a broad scale.

    Point being, we’re still pretty early.

    That said, we in the business of helping organizations adopt related culture and tooling have gone to great lengths to encourage everyone that this change should be pursued with dogged intensity. We have sliced and diced the data to illustrate how those organizations at the forefront of so-called digital transformation, enabled by agile and DevOps, are realizing the results in the form of increased velocity.

    The same research finds that “advanced” adopters of the two concepts are, in fact, seeing significant improvement in time-to-market, customer satisfaction and new business growth.

    But research is just that, and often not completely aligned with real-world scenarios. Someone who takes the time to answer a survey about such matters, one could reasonably expect, is the type of person far more likely to be involved in agile and DevOps activities.

    My personal conversations with everyday practitioners [aka other parents on the sidelines at my kids’ sports and social events, many of whom work in IT] tell me that we’re much more oriented to the “pockets of innovation” phase at present, than any sense of pervasive adoption.

    So this begs the two-part question – where do we really stand, and what do I really need to care about?

    The only reliable source of record in my mind is the perspective of people already doing the work, beyond what I can discern amid basketball games and middle school theater. Real-world practitioners are the voices of reason that allow us to “be here now” and address the specifics of practical agile and DevOps implementation.

    Over the past several years, I’ve found one of the best sources of such input to be the customer panels available at industry conferences and on webcasts. When it’s time for me to sit down and crank out some related content, I often go back to these recordings to remind myself where the rubber actually hits the road.

    On Tuesday Mar. 7, we’ll host just such an event, the 3rd Annual DevOps Virtual Summit, where agile and DevOps leaders from organizations including CNN, GM Financial and United Airlines will share their thoughts on how this ongoing transformation is playing out, and where the remaining challenges lie.

    Along with those experts, we’ll also hear from Dr. Nicole Forsgren, one of the leading researchers in the space who is in seemingly constant contact with people leading the charge and working in the trenches putting agile and DevOps into practice.

    As a marketer, and previously as a journalist, everything that I know comes from direct interaction with those colleagues willing to share their hands-on experience garnered via doing this work every day.

    Register for the event and join us to find out where agile and DevOps actually stand today.

    — Matt Hines

  • CD 101 Podcast Episode 2: The Agile Enterprise

    CD 101 Podcast Episode 2: The Agile Enterprise

    And we’re back with another episode of the “Continuous Delivery 101 Podcast.”

    In Episode 1, we covered the three waves of IT disruption—and how these disruptions, combined with the move toward digital transformation, are drastically changing every industry.

    This, of course, underlined how accelerating speed-to-market is clearly more important than ever before.

    In this second episode, we talk about Agile methodologies with expert Dan Rice, who addresses how these practices can enable business strategies to be implemented at greater speed.

    Listen closely and you’ll learn more about what Agile actually means across various teams, roles, product owners and scrum masters, along with related points on DevOps and CD tooling.

    All of these topics are great food for thought as you think about how you can move to build software faster and with increased quality. Yet, if enterprises only focus on these issues, they aren’t tackling or talking about the real problem in their organization that Agile can solve.

    It’s time to take the conversation beyond the team level and talk about how work flows to every involved team, enterprise-wide—specifically, about how initiatives can expand across multiple teams, how to manage shared resources and how to pull in other dependent teams.

    For Agile to work, it has to expand universally and be ingrained further at program level, to understand precisely how work is coordinated and flows across every domain.

    You can listen to Episode 2 of the “Continuous Delivery 101 Podcast” via:

    • YouTube
    • iTunes store
    • Or by searching for it in the iTunes Podcast App under “Continuous Delivery 101 Podcast”

    Thanks for tuning in to the “Continuous Delivery 101 Podcast.” We hope you’ll continue the conversation by posting your comments below or in the comments section on YouTube or iTunes.

    Meanwhile, keep tuning in: We’ll be back soon with another episode!

    — Scott Edwards

  • Reality Check: Bringing Continuous Delivery Dreams to Life

    Reality Check: Bringing Continuous Delivery Dreams to Life

    Every developer shares the same dream: to write a piece of code or app that changes everything.

    Whether it makes your company do things more efficiently, attracts hordes of new customers, simply makes existing clients happier or maybe even changes the world a little bit, that code is what we strive for. Some of us don’t merely harbor that dream; many of us actually achieve that moment of brilliance. For those who have, it’s remembered as a moment of pure joy.

    So, where does this magical process begin? First, you talk to your manager and, if you’re lucky, she or he agrees that it is a fantastic idea and starts to sell it internally. Sometimes the idea is so good that you get the go-ahead on the same day and even get a small team to work on it!

    Then, soon thereafter, reality kicks in.

    You need data, inevitably from applications running somewhere in the cloud and/or some other data from on-premises data stores. You need to interface with APIs, some well-documented, some not. You need some user experience work done, and the standards your company has adopted don’t really match your vision.

    You also need to interface with many of the standard services your company has already implemented because, after all, your app doesn’t run in isolation—you work with existing billing systems, a CRM system, a fulfillment system … and all of them run on servers that can be anywhere. Soon you find yourself fighting for test data, systems to test with and applications and APIs to interface with. Before you know it, you’re banging your head against the wall.

    Fortunately, your teammates are flexible, creative people. And because your internal systems seem to work against you, you go around them. You download a trial version of some testing software; you install a copy of some internal systems on a virtualized test server (if you can find one) and start working with these.

    Since your app needs to be snazzy and snappy, you download some open-source software to help you test the performance on mobile devices. After three sprints, you’re now ready to move your app to an environment where you can start performing real tests!

    Most times this is the moment when the hardest reality kicks in. The copies of the systems you tested with were not up to date. Data has changed. APIs have changed. The real system responds three times slower than the copy that sat on the server right next to you. The data in the cloud has changed. The performance testing tool you used is not available in the official testing environment. … The list goes on.

    This doesn’t even include the difficulties of moving all the components of your new app to different environments. Trust me, this is not a fictional nightmare scenario; it is reality and I see it happen all the time.

    Now, what if you had the test-data generators necessary to create the data you need and a server that ran virtualized services, mimicking all the (up-to-date) external systems you need including the APIs to talk to them—services that would allow you to say, “What if this application responds three times slower when I request data?” or, “What happens if this system suddenly goes down?” What if, when you’re done building, you could simply migrate the app and all the components that go with it to another (test/QA) system with one click?

    Everybody wants to support people with good ideas and turn these ideas into working software without the frustration that very often distracts and demotivates people. To do so, you need to create the infrastructure that supports this. Is it difficult? Not really, the tools to automate services, create test data (and test scenarios) and migrate complete applications from A to Z are available.

    However, just like a great app, this too starts with a dream—the dream of allowing people with great ideas to work on these ideas without the overhead of an infrastructure that allows them to focus on the coding and not on the infrastructural issues; the dream to create a software factory where the creation of the app takes 90 percent of the time and the infrastructural burden only 10 percent, not the other way around.

    If you’re serious about developing software and want to create an environment where the infrastructure allows people to develop and release software faster instead of working against them, find the tools that support it and jump in, because they exist today.

    Enacting this change now and maybe the next time someone comes to you with a great idea you can be confident that they will have all the support needed to get there efficiently.

    — Marcel den Hartog

  • Automation’s Impact on a Structured Life

    Automation’s Impact on a Structured Life

    A few years back my sister’s husband died after an unexpected illness. My sister, who was in her early 50s, experienced a devastating loss at a relatively early age. To her good fortune, my brother-in-law left behind an adequate pension and their mortgage had been paid off, so she didn’t have to bear the burden of having to work to make ends meet. And she didn’t—for a while, anyway.

    After a period of mourning and adjustment, however, my sister decided to return to work. I asked her why, and this was her response: “There were only so many movies I could go to, only so many TV shows to watch, only so many trips to take. I just needed something to do.”

    Now you may ask, what does my sister’s loss have to do with the impact of automation on human employment? Consider this: As automation replaces more human labor, fewer people will be required to make the economy function. Many of those whose labor is no longer needed will be left unemployed, sitting at home deprived of the thing that gave their lives meaning and structure: a job. And without a job, many will be left adrift—not just economically, but also psychologically, living a life without meaning or purpose. That which made their lives worth living will be gone. The implications of a population put aside by automation and living a life without structure or meaning are significant.

    Allow me to elaborate.

    The Need for Structure in Day-to-Day Living

    Human beings are not born with the instinct to organize their waking hours. Anybody who has a 2-year-old can tell you, a child will run around all day until he or she drops. It’s up to mom or dad to say, “It’s time for breakfast, “It’s time for lunch,” “It’s time for a nap,” “Bedtime’s coming; put on your pajamas and let me read you a story. Good night.”

    Children need structure imposed on them from an external source. That’s how it is. This role is filled by a parent, and later in life the school in the form of structured activities performed at usual times. By the time a youngster enters high school, most will have a firm sense of how a day is organized.

    This sense of organization transfers well to the workplace: A good employee respects the structure of the day. She arrives on time, does the tasks for which she is responsible, communicates the status of her tasks, respects the time of others and meets the time demands made upon her. Whether you work in a factory or a law office, being able to operate effectively within the structure of the workday is essential to one’s economic and psychological well-being. There is a certain comfort that comes from knowing where one is going to be on Tuesday and knowing what one will be doing.

    So, what happens when the job goes away? How does one organize himself or herself without something explicit to do? As my sister said, “There are only so many TV shows to watch.”

    Let’s take a look at the extreme.

    Providing Structure Where None Exists

    When I was a young man, before I got into programming, I was a counselor at a secure detention facility for adolescent boys. This a nice way of saying I was a guard in a kiddie jail. Most of my day was spent supervising the movement of teenagers who had committed very serious crimes. When I say supervising their movement, I mean that in the most literal sense. The job of my squad was to make sure the residents woke up on time, dressed in their rooms and were ready to go to school. Then, we called for lineup. The boys lined up and were counted. After count, our squad moved the line to breakfast and then onto the facility’s school. Once in school, we stood at the back of the classrooms to supervise the residents. If a student acted out, we removed him from class. At the end of the school day, we did the lineup again and went back upstairs. The residents went to their rooms. Then they lined up for the exercise yard. At 6 p.m. we lined them up again and went to dinner. After dinner, we moved the line upstairs. At shower time, we moved the line to the shower room. The day was very structured, to say the least. Day in and day out, everyday was predictably the same.

    And you know what? It worked.

    The residents responded to the structure. Most grew to accept the routine. They went to school and got a little smarter. Three square meals a day made them a little healthier. These were boys who had been left to their own devices since early on, mostly because of neglect. They had little sense of how to organize a day. They just did whatever popped into their heads. And sometime what popped into their heads was an armed robbery or felonious assault. In terms of the ability to organize themselves, most were at the level of a 2-year-old—a scared 2-year-old. Not surprisingly, the predictability of events and presence of supervision made them feel safe. Once, I asked one of the residents why he was there. He said with surprising candor, “Because I want to be.”

    When a Job Goes Away

    Granted, adolescent incarceration is an extreme example of imposing structure on someone’s life where no structure exists. But, what happens to a person when a job—the basic agent for organizing one’s daily life—goes away?

    Here’s what happens.

    Some people die prematurely. A study of Shell employees found that those who retire at 55 are 89 percent are more likely to die in the 10 years after retirement than those who retire at 65.

    Others have increased rates of substance abuse. A study conducted by the Substance Abuse and Mental Health Services Administration indicates that 17 percent of unemployed worker suffer from substance abuse—twice that of those who are employed.

    Or, they spend more time sleeping and watching TV, as reported by the Wall Street Journal.

    Clearly, having a job is better than not having a job. Financial considerations aside, it’s the way by which most people organize their day. As more people become displaced from the workforce because of automated labor and are living without the demands of daily work to structure a day, more of the population will feel adrift. If the trends described above stay true, there will be an increase in substance abuse, TV watching and death rates. They also may look to other external forces for structure, such as political movements that promote order, rank and predictability.

    So, what are we to do? In earlier versions of this article, I included a call to action to revamp, maybe even revert, the educational system to focus more on the arts and humanities rather than the baseline skills required to get a job. I planned to point out that artists, athletes, scientists and entrepreneurs don’t need to a job to structure themselves. They always have something to do. They have an internal ability to organize themselves in the pursuit of short- and long-term goals that are their own. As we move toward full automation, these folks are going to do just fine.

    Anyway, that’s what I planned to propose in the earlier versions. Now I just don’t know. I am not sure we have the social or political will to address the problems that are on the horizon. The highly educated people who have good analytics skills and a creative outlook will find paying employment. But, as trends indicate, those who are paid to follow instructions and perform redundant tasks will be replaced by robotic automation. They will go home. They will watch unlimited amounts of movies and TV on Netflix, Amazon and YouTube. They will play games on the internet, for a while anyway. Then, one day they will come to realize that there are only so many movies to watch and so many games to play. The appeal of an unstructured life will wane. They will want something to do and have no idea about how to make that something happen.

    So the questions at hand are these: Will we fortunate ones—particularly those of us in IT, the people who are implementing the automation—watch from the sidelines? Or, will we come to understand the consequences of automation on human employment—and in doing so, will we be become part of the large-scale solution, devising ways that help those that are going to be displaced think different and be different?

    I hope so.

     

    This article is the last of the series, The Impact of Automation on Human Employment. Here is a list of the previous articles:

    What Do We Do When Everything is Automated

    Rethinking Unemployment in the Age of Automation

    The Impact of Automation: Considering 3 Facts

     

    — Bob Reselman

  • 4 Top Tips for Developing a New Feature

    4 Top Tips for Developing a New Feature

    As a backend developer at CA BlazeMeter, I recently had the chance to work on a big feature for our web app. The feature required a massive change on our backend code, a big change on our frontend and multiple migrations. You know what I’m talking about—it was one of those projects that developers fear.

    But, I gained a lot of valuable experience from the project. Not only did we deliver a kick-ass feature, I also learned the importance of teamwork and patience when developing something new. Armed with those two values, and the right tools and technologies, you can deliver a new feature quickly and efficiently.

    Here are a few more things that I learned that should prove helpful whether you’re developing a major feature or one that only requires a minor code change.

    Tip 1: Plan to understand the development requirements

    The process of development starts way before you start coding, with planning. After the product manager decides what needs to be developed, R&D needs to understand the cost: how many days (or weeks) of development the feature will take, how many people will be needed and which APIs are required.

    To do that, the product manager needs to conduct at least three “grooming” sessions with backend, frontend and QA leaders. The grooming sessions start with the product manager presenting the product flow and requirements.

    Since no one wants to read a lot of text (except perhaps the awesome readers of this article), the product manager should prepare mockups. They don’t have to be designed or colorful, but they do need to show the team what the feature looks like in his or her vision. During our grooming sessions, we used Balsamiq, which lets you create screen sketches for how the product is supposed to look, feel and work.

    Editing screen sketches using Balsamiq

    After the initial presentation, the sessions should include a lot of open discussion—asking questions, checking for answers and examining compliance, etc. —until everyone is satisfied and understands what the feature requires from them and from others.

    This is the time and place for frontend, backend and QA to give their opinion about the schedule and to alert and flag any potential problems in development or extreme use cases. It’s the role of the product team to answer questions and change the flow and requirements until there are no R&D rejections.

    Afterward, the frontend and backend leaders need to create a feature development plan, sitting down to determine what data they need to transform the product vision into a real thing. Or in other words, which APIs are required for each flow or button, how long each step is supposed to take, including buffers, and the order of development. APIs are the agreed “contract” between the development teams. After determining the contract terms, each team can proceed individually.

    I recommend you write your APIs in Swagger, which turns YAML files into detailed HTML pages with all the APIs.

    Swagger Editor

    This plan needs to be transparent to everyone working on the project. In fact, it’s even better if each project participant can add and describe scenarios. Tools such as TestLink and Atlassian do just that. Then, when the code is ready, developers can go over the list and see everything is covered.

    We will need to stick to this schedule later on, especially when new demands unrelated to the feature come in, such as fixing bugs. Sticking to this plan is the best way to distinguish the wheat from the chaff of the development process.

    Start writing automation ASAP

    After planning, we want everyone to start working, and not only the backend. To do that, the backend can prepare mocks, either through outsourcing services such as JSON servers, which show inputs and outputs, or by writing dummy code.

    I mentioned earlier that we used Swagger to write APIs. We also used it to generate a server in NodeJS/Python, like this:

    Generating a JSON Server through Swagger

    From this moment on, frontend can begin writing and building screens and QA can start writing API automated testing, by using tools suh as JMeter. At this point, frontend and QA have the response codes and response bodies; they know which features to put where; and which APIs they need and how to use them.

    QA also know which data they’re missing, such as field names and field data. Sure, the backend code isn’t ready yet and all of the automated exams will return “red” instead of “green,” but the main issue is making sure everyone is as prepared as possible.

    This includes load and performance testing. You can use open-source tools such as JMeter, which is enhanced by CA BlazeMeter. By uploading our test script into, configuring the number of virtual users and ramp-up time, determining geo-locations to test from and analyzing KPIs to determine trends and the health of our system, we can ensure our product is delivered to our customers without crashing.

    This is also true about sending code to QA: Developers should test their code themselves and ensure that when they commit their code, it works 99.9 percent of the time.

    By starting automation as soon as possible, you can minimize future bottlenecks in the development process.

    Ensure the team communicates and works together

    Like I said before, teamwork is the most basic requirement when creating a new feature. This is because new questions arise during development and because requirements change. Therefore, everyone in the team needs to be in sync all the time, from the product manager and the designer to the developers and QA.

    Open communication lines are crucial because everyone is working on the same product at the same time, but each developer or QA should be working on a different part of the product and on different functionalities. Communication ensures each person’s work is compatible with the rest of the team’s work. This saves time, prevents frustration and redundancy and ensures maximum professionalism.

    To make sure everyone knew what was happening, we used Slack and opened a dedicated channel for the creation of this feature. We also had daily (short!) standups where we discussed what we have and what is missing.

    Opening a Slack channel

    Communication also includes team and assignment management. JIRA, our primary issue tracking tool, offers Kanban-style, Scrum and fully customizable boards, which show the progress of each development task, including the developer assigned to it, and his or her exact status or phase.

    This helps you understand what other people are working on, and just as importantly, where people are stuck. If you can, get everyone to work in an open space and put a large screen in the middle of it. This lets you stay in sync at all times.

    We used JIRA to open and manage bugs as well. This was helpful when multiple QAs found the same bug. It’s important to carefully choose which tickets are opened and put them under one epic, so everyone stays focused.

     

    JIRA Dashboard

    Another important aspect of communication is ensuring everyone uses the same terminology, as sometimes people might use the same term to describe different things or different terms to describe the same things. To prevent mixups that could waste your time, make sure everyone is on the same page. For example, do you say “system admin” or “<name (ex. BlazeMeter)> admin”?

    Work on the same development environments

    It’s important that everyone works on identical development environments and even on the same version of the tools and libraries. Otherwise, it could cause an (in)compatibility mess, including a lot of unnecessary back and forth and even might cause bugs to pop up. This environment needs to be as similar as possible to production, but only on a smaller scale.

    Conclusion

    Now look at the feature that came out of all that work here. I’m so grateful that I had the opportunity to work on it with an awesome team.

    I hope these best practices help you the next time you develop a new feature. Don’t forget to keep going agile after development and delivery by running tests in a continuous environment such as Jenkins for every commit.

    CA BlazeMeter works with Jenkins, ensuring you can load test in the cloud and analyze your data while shifting left. You can also use different APM tools, such as New Relic, AppDynamics and CA APM to alert you about errors before QA, which saves time.

    — Dor Atias