devopsy.kinsta.cloud

Tag: Lean principles

  • DevOps for Salesforce: CI/CD Challenges

    DevOps for Salesforce: CI/CD Challenges

    DevOps encompasses a wide variety of technical and social practices for improving the development, delivery and operation of IT systems. Salesforce has become more than a CRM system; it evolved into a full-featured ecosystem and a low-code platform that made development far simpler and more approachable than traditional code-based platforms. Cross-functional teams build on Salesforce to create custom apps and integrations with other enterprises systems without requiring the rigorous approaches that are common with traditional technologies like Java.

    Now, DevOps for Salesforce has emerged to enable teams to build applications faster. And as teams have built more and more customizations onto the Salesforce platform, the level of complexity has exploded. DevOps has emerged as a critical capability for teams to bring this complexity under control.

    The Primary Challenge of Deployments

    The primary challenge that most Salesforce teams face is deployments. Deployments on Salesforce are not just a matter of moving files, they require sophisticated management of thousands of XML files and careful management of dependencies. A number of commercial companies have appeared to help manage the Salesforce development life cycle, and the Salesforce development experience (DX) initiative has unlocked vastly more possibilities for teams to automate existing processes and adopt new workflows.

    The need for more structured and rigorous approaches to Salesforce development has increased the need for teams to learn from traditional software development and DevOps teams and understand how traditional techniques like modularization and continuous integration can be adapted to the Salesforce platform. Technology and knowledge transfer from the broader DevOps community to the Salesforce community remains an active area of exploration.

    Working in Parallel

    It was the adoption of continuous integration by the extreme programming (XP) community in the 1990s that drove intense interest in the need for automated testing, especially unit testing. This allows teams to create exact copies of each other’s work and develop in parallel. Working in parallel allows for far faster innovation since work doesn’t have to be precisely sequenced. 

    But there’s a cost to working in parallel; these workstreams eventually need to be integrated into a cohesive whole. Software projects following a “waterfall” methodology may delay integration until near the end of the project. But these parallel work streams diverge naturally over time, meaning that integration can take months and require enormous amounts of rework.

    Continuous Delivery: A Distinct Practice

    Continuous integration (CI) avoids this issue by merging work early and often (at least daily). “If it hurts, do it more frequently,” is a mantra in the CI world, where the challenge is to resolve conflicts in small batches rather than letting them accumulate.

    CI creates a constant risk of failure for software, since teams working on one part of the system may be making changes to shared components that can cause other teams’ code to fail. Automated testing provides a layer of protection by allowing functionality to be checked and rechecked thousands of times with minimal human intervention. Creating automated tests requires more effort than executing a manual test, but the cost of re-running each test becomes negligible. Thus, automated testing is a perfect solution for high-velocity teams who need fast and ongoing ways to ensure they are not regressing as they build new capabilities. 

    Continuous delivery (CD) is a distinct practice that’s often mentioned in the same breath as CI. Continuous delivery was popularized by the 2010 book of the same name by Jez Humble and Dave Farley. The practice of CD extends the idea of CI by aiming to deploy code to production in small batches on a regular basis; ideally at least daily. Whereas CI addresses the risks of developing in parallel, CD addresses the risks of delaying feedback from end users.

    Software is built to solve specific problems. But each attempt at a solution by developers is a hypothesis that needs to be tested and confirmed. Only the people who actually experience that problem can tell developers whether these solutions are sufficient. So developers must get fast feedback from end users to have confidence in what they’ve built. Teams that neglect this practice are operating on assumptions and optimism; the longer they delay delivery the more the risks of those assumptions compound. Poor adoption and user experience is to be expected when teams don’t practice continuous delivery.

    Increasing Pressure

    CI creates added pressure to test quickly, CD increases that pressure significantly. Delivering small batches of changes reduces the risk of each change and makes them faster and easier to roll back. But by definition, we are changing production systems that may be used by thousands or millions of people each day. A failure caused by CI may waste a few hours of a developer’s time and delay delivery of a change. But a failure caused by CD can prevent thousands of people from doing their jobs, corrupt production data, expose security vulnerabilities and worse.

    When developers integrate their work in CI, the goal is to run a fast set of tests that confirms nothing major has broken. But work being delivered to production needs to go through a battery of comprehensive tests to ensure it meets all functional and structural requirements. In a waterfall-style project, pre-release testing may involve scores of testers working for months. In continuous delivery, teams must be able to execute tests at every level of the system and have robust confidence that no failures have occurred.

    Comprehensive testing requires testing the user interface, testing processes that span multiple systems, loading and confirming data using APIs, security testing and more. Continuous delivery has driven the need for automated testing to new levels and led to entirely new approaches like canary deployments. Canary deployments are a form of shift-right testing where a feature is initially delivered to a small subset of end users to check for issues before rolling it out to everyone.

    The Essence of Lean

    It’s often said that the main driver of success in DevOps is culture. The acronym CALMS is used to summarize the five aspects of DevOps: Culture, automation, lean, measurement and sharing.

    Culture refers primarily to creating a culture of continuous learning and experimentation. The practices described above are not formulaic and require a growth mindset and a culture that prioritizes psychological safety and information flow.

    Automation is the aspect of DevOps most commonly emphasized (including in what’s been written here). Automation enables dramatic efficiencies and is the central miracle of technology, but automation is not a panacea without addressing these other aspects of DevOps.

    Lean refers to the manufacturing practices pioneered in the 1930s at Toyota and further developed and built upon in the late 1980s and 1990s. The essence of lean is to do more with less by understanding and optimizing the value stream by which work is actually performed.

    Measurement refers to the importance of taking a data-driven approach to optimizations. Measurement provides teams with a way to gauge whether process improvements are effective or not.

    Finally, sharing refers to the importance of sharing information, practices and resources between individuals, teams and organizations.

  • Principles Systems and Tools – What Shingo Can Teach Us About DevOps

    Principles Systems and Tools – What Shingo Can Teach Us About DevOps

    In my last post, The Power of a Game: What I Learned from Facilitating the Phoenix Project Simulation, we explored how DevOps and Agile practices are built on underlying Lean principles that go back decades to a time before the Toyota Production System. When you look into the history of Lean, it is almost impossible not to encounter the great thinker Shigeo Shingo. Shingo worked closely with Taiichi Ohno (the father of the Toyota Production System) to advance Lean thinking while building the Lean production system at Toyota.

    As someone who attempted to apply Lean to IT long before DevOps came into existence, I have always said DevOps is about creating an environment where it is safe to look deeply, question what is happening, think crazy things, experiment and learn based on outcomes. Ten years ago, I was invited by The Shingo Institute to serve as a teacher, coach and assessor. I had recently been awarded the Shingo prize for my book, Lean IT – Enabling and Sustaining Your Lean Transformation. What I learned from Shigeo Shingo’s writings heavily deepened my understanding of the role of Lean and flow in IT. These ideas are reflected in the Lean Enterprise Pyramid below and it’s amazing how effective they are with Agile and DevOps.

    Principles and IT Excellence

    A principle is a timeless universal truth that controls outcomes whether you believe in it or now. Think of it as a fundamental truth or proposition that serves as the foundation for a system of beliefs and behaviors. Principles govern consequences and outcomes–both positive and negative. “Create customer value” is a core principle in business success. If you don’t believe in customer value, or don’t deliver it, your prognosis is pretty bleak.

    In Lean IT, I introduced some key principles that drive IT success:

    Let’s begin with a working definition of DevOps—a mindset, tool set and set of practices that combine software development and IT operations to shorten the development/operations life cycle to deliver small iterative features, fixes and updates often and in close alignment with business objectives and value creation.

    Consider the Lean principles pyramid from a DevOps perspective:

    • Foundation principles build an intentional culture of trust, respect, focus and learning.
    • Behavior is how we show up and engage.
    • Perspective is a fresh new way of looking at IT work.
    • Flow is the outcome of effective DevOps in action.
    • Capstone of the pyramid fosters a regenerative culture of trust, safety, experimentation, collaboration and innovation.
    • Outcomes are higher quality IT solutions and services, in less time, with less cost with healthier teams.

    For DevOps, Agile and High-Velocity IT of any flavor to be successful, each of these principles must be addressed. You can’t address them all at once, so I suggest beginning at the base while taking steps to enhance flow. If you focus on these two elements, respect for people and flow, you will always be progressing in the right direction.

    Systems and IT Excellence

    Shingo defined a system as a collection of tools and tasks highly integrated to accomplish a specific outcome. The Shingo Model captures the important relationship between principles, systems, tools and results. It also supports three key insights which clarify how to achieve excellence in IT (or any organization). IT needs to get good and then continuously get better with its systems, tools and results (no kidding). Understanding how these elements are informed and reinforced by principles is essential to making this happen.

    The Insights

    1. Ideal Results Require Ideal Behavior
    2. Beliefs and Systems Drive Behavior
    3. Principles Inform Ideal Behavior

    For DevOps and Agile, the Shingo Model provides a logical sequence for change. Put another way, what are the key questions we need to answer?

    1. What do the ideal behaviors of DevOps look like?
    2. What beliefs do our people have today and what beliefs do they need to embrace DevOps? What systems (think: work systems, procedures, policies and technology) do we have in place today, what behaviors do they drive and what needs to change?
    3. How can our understanding of lean principles clarify our understanding of question #1?

    Take a moment and reflect on the role of principles and systems in your transformation. The more you ground your approach in the stability of principles, the greater your chances of success. If you or your organization does not truly believe in these principles, a conversation around which principles you do believe in would be a good place to start.

    — Mike Orzen