Category: DevOps Practice

  • Best of 2020: Don’t Do Agile and DevOps by the Book

    Best of 2020: Don’t Do Agile and DevOps by the Book

    As we close out 2020, we at DevOps.com wanted to highlight the five most popular articles of the year. Following is the fifth in our series of the Best of 2020.

    In software development, we talk a lot about the importance of Agile and DevOps. Agile aims for rapid releases, positively encourages involvement from customers and other stakeholders and splits work into smaller iterations, releasing as soon as possible. DevOps extends it by introducing cross-functional team collaboration and automation, enabling teams to always have a stable build ready to deploy.

    That’s the short version and there’s a huge range of books and frameworks out there to read so that anyone, anywhere–apparently–can just start doing it. The danger is that if you follow them too closely, processes can actually become too rigid, so you end up losing the agility you’re striving for. I always get suspicious when theories in books are read and regurgitated completely without thinking about the actual situation on the ground. I’d much rather have a conversation, write up our notes, try it out and see how it can be improved.

    Clearly, I’m not saying that you shouldn’t have boundaries and rules. I worked for a company that moved from no processes at all to adopting Agile methodologies. It needed to put in place a framework to guide people in the right direction, particularly initially.

    As companies mature though, they need to look at what works best for their particular situation–otherwise the danger is that common practice masks commonsense. You end up following processes, such as two-weekly reviews, that don’t necessarily match your needs–why wait two weeks for a review, for example, if something obviously needs fixing today? Where did Agile go?

    What Does Agile Mean for You?

    The best place to start is to define Agile for your organization. On the positive side, definitions of Agile are very broad–but the flip side of that is people get confused about what terms actually mean and whether they should be using them.

    For me, Agile is about being able to change the way you work, reflect on things and continuously improve. The biggest goal is to bring in very short feedback loops and be in a position where you can do something when you get that feedback. After all, there’s no point having a quick feedback loop if you can’t adapt or react equally quickly. 

    I’ve deliberately kept this definition high level–it’s important not to mandate exactly how you deliver Agile. For example, we use a variation of Kanban in our team, based on the original methods adopted by Toyota, having switched from a flavor of Scrum. I’ve had people come up to me and say we’re not really doing Kanban properly, as we’re not following every step of it rigorously. That’s not an issue–we’re using it in a way that matches our needs and goals.

    The Components of an Agile Philosophy

    One of the reasons that people follow Agile books and processes is that it’s easier than looking at your culture and mindset. In software, I believe we’re more interested in the output of our minds, rather than the output of our hands. We have bodies, but they’re primarily there to move our minds to meetings. What becomes difficult is if we try and measure productivity by looking at the wrong metrics–essentially focusing on the output of our hands instead of our minds (how many lines of code have you written, for example).

    You need to move away from this and create a culture with four pillars:

     

    • Everyone is committed to constant improvement: Even the best bit of code or greatest process can be improved, so teams need to be open to always try to get better at what they do and how they work (focusing on where improvement will have an impact). It isn’t a negative or an admission that you’re not up to the job. In fact, it’s the exact opposite–wanting to learn and improve is fundamental to Agile, and generally being a good developer. 

     

     

    • Everyone is free to share their ideas: I talked about the need for fast feedback loops, and that extends to everyone being able to give their opinions, without fear or worries about ridicule. There really is no such thing as a bad idea, as it can spur further thoughts that get you to a solution. For sharing to actually work, you need psychological safety in your team–people have to be confident that whatever they say, it won’t be held against them come review time. Forget the “I know best because I’ve been doing it for longer” approach as well. I’ve often seen young inexperienced developers come up with great ideas–because they’re young and inexperienced.

     

     

    • Everyone is encouraged to question everything: Building on the first point, people have to be prepared to constantly review the way they, and the wider organization, works. Why do we do it this way? Is there a better solution? This shouldn’t be a recipe for constantly changing working practices and not getting anything done though. Go back to your original objectives of what you’re trying to achieve and then question whether current methods are the best way of getting you there.

     

     

    • Teams don’t have to follow rigid methods: No one should be forced to work a certain way (that’s the antithesis of Agile). Instead, teams should be encouraged to find the way that works for them. There’s a whole range of Agile methods out there, and the combination you adopt should be focused on your particular needs. That’s where my suspicion of Agile books comes from–for me the perfect read would be one that just fills your brain with thousands of ideas.

     

    Agile is at the heart of efficient, high-quality software development–just don’t risk losing its agility by following processes too slavishly and making everything you do too rigid and inflexible.

    — Dave Rook

  • Best of 2020: Dear Staging: We’re Done

    Best of 2020: Dear Staging: We’re Done

    As we close out 2020, we at DevOps.com wanted to highlight the five most popular articles of the year. Following is the fourth in our weeklong series of the Best of 2020.

    Dear Staging,

    I’ve been thinking a lot lately, and I’ve come to the decision that it’s time for us to go our separate ways. Our time of working together in the software delivery process has come and gone, and I just can’t look past your shortcomings anymore. I would say it’s not you, it’s me, but let’s not kid ourselves: It is you.

    You’re Too Expensive to Keep Around

    For starters, you’re way too much of a burden on my wallet. With your memory-based cache, web servers and application servers that query a database, you’re costing me more than $100K per environment—and that’s just for your most basic architecture. After factoring in your maintenance fees, necessary certifications, and infrastructure costs, you’re just not practical anymore.

    Businesses regularly review their expenses with a fine-tooth comb and cut out anything that doesn’t provide efficiency and value. You, Staging, offer neither of those, and it no longer makes sense to invest in a dummy environment.

    You Can’t Fulfill My Needs

    When I test my code, I want to know that it works in production. Staging, you can’t give me that. Your environments and your data are different than that of production, which causes inaccuracies in my test results.

    As a tester, this means just because my code works in staging, it won’t necessarily work in production. It’s beyond frustrating when my code passes its test in staging but then fails when it reaches production. You’ve broken my confidence too many times here, and it’s just unacceptable.

    You’ve Lost Your Significance

    I need to test in an environment that makes an impact and provides meaning to my work, and you don’t give me that. If my staging environment is down, it’s not a big deal, and people don’t care to fix it.

    Yes, there was a time when you were relevant, but that was years ago when software development was still gaining traction and staging was the default rather than a choice. Now, I can’t even get anyone else to care about you. When your system goes down, no one will give you the time of day because you’re old, outdated, and not a priority.

    You Slow Me Down

    My time is valuable, and I’m wasting it when I spend it with you. For starters, it’s hard to create an exact replica of production, which means I have to create new data when I test in a staging environment. Because of this, my test results from staging don’t give me the same level of accuracy, so I end up doing more work for less quality.

    Additionally, most companies have more than one staging environment, and it takes time to individually test in each one. Most importantly, end users are not going to use my product in staging; they’re going to use it in production. Why not cut out the middleman and go straight to where the action is?

    I’ve Found Someone New

    Her name is Production, and she gives me what I need. One of the biggest factors is that I’ll never know the differences between you and production until I test there.

    With feature flags, I can safely test in production without fear of breaking something or negatively affecting the customer experience. They allow me to target specific users, which means I can choose exactly who sees the changes I make to the feature. All other users must wait for me to give the green light before they can see the new feature.

    As you can see, Staging, it’s time for us to go our separate ways. We’re no longer compatible, and Production more than fulfills my every need. I’d say good luck in finding someone who considers you valuable, but the truth is, it may be time for you to hang it up permanently.

  • Best of 2020: Cybersecurity: What Programming Language Is Better for Your Career?

    Best of 2020: Cybersecurity: What Programming Language Is Better for Your Career?

    As we close out 2020, we at DevOps.com wanted to highlight the five most popular articles of the year. Following is the second in our series of the Best of 2020.

    As a novice developer, you may be thinking about learning a programming language and starting a fantastic career.

    In fact, this question on Reddit inspired the whole article:

     

    Source: Reddit.

    The thing is, programming languages represent certain philosophies of its authors and tailored to certain types of problems, i.e., there is no such thing as “the best programming language.”

    Source: Reddit.

    Every Tool Has Its Time and Place

    In this article, I describe major programming languages according to its popularity, usage, demand and ease of learning. 

    There is market demand and a potential salary range for each language. These are linked to specific personal goals and prior programming experience.

    What Questions Should You Ask Yourself Before Diving in?

    • What kind of projects do you want to work on? If career flexibility is a priority, then learning Python or C++ will allow you to work with different types of programming. If your passion is web development, learning JavaScript or PHP is a smart choice.
    • How much experience do you have? You won’t want to switch to an advanced programming language, such as Rust, if you don’t have previous programming experience. If you’re new to a language, you’ll want to start at a higher level and be intuitive.
    • What are your career goals? Do you want to become a freelancer, working for an existing company or work with a startup? Are you trying to be better in your current role?

     

     

    If you plan to get a job in a large technology company, learn the programming languages they use, e.g., NASA, Google and Facebook use Python.

    Many programming languages have similar syntax and qualities—so if you learn one of them, it will be easier for others. The more languages you know, the more flexibility you have in your career and development projects.

    In the list below, we will look at the best and most popular programming languages for many of the most common uses, including web development, mobile development, game development and more.

    Source: Reddit.

    Top Five Programming Languages to Start Your Career

    Java

    It is probably the best programming language if you are looking for a common language that will be useful in various situations. As for choosing a programming language to learn, there are many resources for people who are thinking about learning Java. Java was released in 1995, which makes it one of the oldest programming languages. It is used on a wide range of platforms and operating systems, including Microsoft Windows, macOS, Linux and Solaris. 

    That means there will always be jobs for Java developers, making it (perhaps) the best programming language to learn.

    Research shows that Java developers can expect to earn between $80,000 and $130,000 per year. If you are thinking about learning your first programming language, think about taking an online course, for example, CodeGym. This course, which contains 40 levels of game-play classes, will teach you all the basics you need to know to start programming in Java. It’s easy to follow—even if you don’t have programming experience—and it is worth checking out.

    It is also best used for back-end web development, mobile development and desktop applications.

    JavaScript

    This programming language works on several platforms. Therefore, it is ideal for websites that will be accessed through various devices and browsers. It is suitable for anyone who designs websites with effects, interactive features, animations and pop-ups—standard features on virtually any brand website these days. If you want to work in a creative environment and add more experience to the user, this is a coding language to learn.

    JavaScript is the number one language on the web. The growth of frameworks such as jQuery, Angular and React JS has made JavaScript even more popular.

    If you just can’t stay away from the internet, it’s better to learn JavaScript sooner than later. I consider this JavaScript Masterclass an excellent place to start. For cheaper alternatives see this list of free JavaScript courses.

    It is also best used for front-end and back-end web development

    Learning PHP

    PHP—another old language, first appeared on the stage of programming around the same time as Java in 1995. It is a general-purpose language widely used for web development from the early days of the web.

    Unlike other languages, which can be quite tricky to use with HTML, PHP can be built directly into the HTML code block. Start and finish instructions are used to enter and exit PHP, making it very easy to use. Although PHP is not as popular as Python or Java, it can be a suitable language to learn because of its simplicity. It is convenient for beginners and you can write your first scripts just a few hours after you start your PHP course.

    Again, the popularity and wide use of PHP means that work opportunities are relatively easy to find. An average PHP developer can expect to earn around $80,000 per year. However, there is potential to make much more as a freelancer.

    Take an advanced PHP course, which will teach you everything you need to know to find your first job as a programmer. It doesn’t matter if you’re studying your first language, or if you’re an experienced developer looking for the best programming language to add to your resume—PHP is a great choice!

    It is best used for back-end web development.

    Python

    Source: Reddit.

    Python is a useful step toward more advanced forms of programming languages. Python provides a high degree of website readability and is used by companies such as Reddit, Google and even NASA.

    It is seen as entry-level programming that does not require too much prior knowledge. It may be ideal for relatively new professionals to help small businesses that want to expand their presence on the internet without having to resort to advanced professionals. Python is a relatively simple programming language that is very easy to pick up if you already have programming experience. That makes it one of the best to learn if you are already working as a developer and want to increase your resume quickly.

    The code on Python is designed to be readable. It uses a lot of white spaces, which makes it a pretty easy language to learn while you’re studying it.

    One of the most useful Python’s features is its compatibility with data analysis systems and scientific applications. It is a popular language in scientific communities and is often used by researchers to write their programs.

    An average Python developer can expect to earn about $100,000 per year. However, there is little room for wage growth.

    If you think Python is the best programming language to further your career, consider enrolling in a BitDegree course. The Python Basics Course includes everything you need to know to start writing programs on Python and contains less than four hours of lecture material.

    It is best used for back-end web development, desktop applications, and data analysis.

    C#

    C# is another object-oriented, general-purpose language based on C. It is considered a suitable programming language for building applications native to Microsoft platforms. Microsoft initially developed it as part of its .NET framework for building Windows applications. C# uses a syntax similar to other languages such as C++. Therefore, it is easy to pick it up if you come from a different C family language. 

    C# is not only a suitable language for developing Microsoft applications, but also a language used by mobile device developers to build cross-platform applications on the Xamarin platform. Besides, anyone interested in VR development should think about learning C# as it is the recommended language for building 3D and 2D video games using the popular Unity game engine, which produces one-third of the best games on the market.

    If mobile application development or virtual reality is your business, consider learning C#. Many mobile device developers use C# to create cross-platform applications on the Xamarin platform. Although it is not the most convenient programming language for beginners. 

    If you are interested in any of the above areas, please see the section “Learning the code when making games – Udemy’s Complete Unity Developer in C#.” I see that more than 200,000 students have enrolled in this course, which indicates its popularity.

    It is best used for mobile and game development.

    What Are the Best Programming Languages to Learn?

     

    Source: Reddit.

    As was mentioned at the beginning, there is no best programming language to learn. The best language for you will depend on your experience, your current knowledge and reasons to learn a new language. 

    While there are many options, you should be able to narrow down your language choice by what you want to learn from it. 

    Remember to consider the following:

    • Programming experience, because some languages, such as Python and Java, are more suitable for beginners who have never programmed before.
    • What you want to do with the language. Different languages are used for different things.
    • How much time do you have? Some languages are much more demanding than others.

    Make your choice based on your interests and the type of software development you want to get into. Here is a brief summary:

    • External web development: JavaScript.
    • Internal web development: JavaScript, Java, Python, PHP.
    • Mobile development: Java, C#.
    • Game development: C#.
    • Desktop applications: Java and Python.

    To Sum Up

    Programming and developer communities are evolving faster than ever.

    Various new programming languages are emerging that are suitable for different categories of developers (beginners and experts) as well as for other applications (web applications, mobile applications, game development, distributed systems, etc.).

    This list simply scratches the surface of existing programming languages. Still, the simple fact is that with the right coding skills, you will be perfectly prepared to offer valuable technical support and become an asset to any organization.

    — Dmitrii Borodin

  • Best of 2020: 5 Project Management Skills Every Developer Should Have

    Best of 2020: 5 Project Management Skills Every Developer Should Have

    As we close out 2020, we at DevOps.com wanted to highlight the five most popular articles of the year. Following is the first in our series of the Best of 2020.

    The days of the siloed project manager and “skilled” development team are becoming a thing of the past, thanks to the increasing complexity of dev projects. In many cases, the knowledge gap between the project manager and the developer is large, making it nearly impossible for any non-technical person to effectively manage the entire scope without significant input from the development team.

    These days, every member has to have a basic level of project management skills to be able to contribute to the overall direction. With the days of Gantt charts behind us and more companies using project management software as they work from remote locations, all team members have better access to all of the components of the project and an expanded sense of responsibility.

    When the developers take an active role in the management of the overall project, the benefits are numerous. Not only do the developers have a better sense of how long each step of the process will take—and their contribution to the larger project—but they can also more accurately prioritize the crucial elements needed to take the project across the finish line or recognize where errors, scope creep or delays may derail a project. In short, individual team members become more active collaborators in delivering better, tighter code and applications more quickly.

    Project Management Skills to Know

    That said, project management doesn’t happen magically, and it would be foolish to assume that just because someone is a skilled developer they know how to manage a project. But familiarity with a few critical PM skills and requirements can go a long way. Here are five aspects of project management every developer should understand.

    Estimation and Scheduling

    Like any sector of business, project management has its key skills and lingo. The most significant areas of focus for project managers tend to be scheduling and estimation. Scheduling is critical, as you have to assign tasks and accurately manage the workload across the team.

    Estimation is also necessary and important to get right, especially for the product development team. The estimation of a completion date triggers many other teams to perform their functions across the company and is often used by the executive team to target success or failure. But delivery dates are notorious for being missed. Working within date ranges versus single-point deadlines is highly encouraged. Ranged estimates capture and help manage the uncertainty so common in software development today, giving the entire team more opportunity for success and allowing for the unexpected roadblocks that occur with nearly any project.

    Critical Communication

    As a team, it’s essential to keep the lines of communication open. We’re not talking about unnecessary chit chat; we’re referring to critical updates that may affect the overall project. This includes having strong interpersonal skills, namely the ability to maintain clear team communication, complete document reviews and have succinct interactions about project deliverables and expectations.

    Technology and online collaboration skills are also important. Ensure all team members know how to effectively estimate their time and keep their progress updated in software the entire team can access. Ensure document sharing and team updates are enabled, so there are no excuses for missed or mixed messages.

    Problem-Solving

    You can tell a lot about a project team by how the team members solve the major and minor technology and business-related problems that occur—often daily. You might expect that Agile development and DevOps teams have institutionalized troubleshooting in development and engineering. However, teams still need to treat problem-solving as a skill that should be continuously strengthened, especially as technologies evolve. Appointing a lead to coordinate and direct others when problems arise can be an efficient way to take control of a spiraling situation quickly.

    Technical Documentation

    Writing technical documentation can be a neglected task but a valuable skill for project teams. Too often, documentation isn’t part of the team’s workflow, but it should be. Whether a project team has a staff or contract technical writer assigned to its project, documentation is an important skill. It’s also important to note that documentation doesn’t necessarily need to take place in traditional-type documents. Today’s project teams are often better served by wikis, enterprise social tools and online project management tools. What matters is that the documentation is centralized, online, secure, auditable and searchable by the team and by management.

    Risk Management

    Many of the above skills lead to an end-goal of managing risk—in this instance, the risk of not delivering a stable, feature-complete project on time. Risk management is a skill that is most effective when distributed among the project’s team members. This is especially important with teams that consist of cross-functional specialists and those with no clear oversight or direct leader. The key to distributing risk management across a project team is to make individual team members accountable and to explain the potential risks in their project tasks to the greater group. This “ownership” of their function and deliverables allows for the most realistic planning possible, requiring each team member to take accountability for precise timelines for their piece of the project. These timelines then roll up to the greater project plan and provide a more accurate estimation of completion date and potential pitfalls.

    The Well-Rounded Developer

    Project management skills are useful for all employees across an organization. There are no downsides to embracing proven tactics from the project management playbook as they’ll only serve to increase efficiency and promote better team organization and cohesiveness. Developers who have technical skills in collaboration with project management knowledge are always in high demand. Dedicating time to honing these skills is a worthwhile endeavor for any developer.

  • Why Boring Tech is Best to Avoid a Microservices Mess

    Why Boring Tech is Best to Avoid a Microservices Mess

    Microservices are all the rage. A Forrester study found that 76% of enterprises were rearchitecting applications for microservices.

    At the same time, microservices are definitely not a silver bullet. Among those already using microservices in production, one study found that 59% said each microservice added increased operational challenges such as data management. Even more concerning, the same report revealed that when asked to compare the difficulty of troubleshooting different environments, 73% said microservices were more difficult, and only 21% said they were easier than monoliths.

    But how did we end up like this? According to Matthew Sinclair, VP of engineering at BCG Digital Ventures, it is because that, as technologists, “we want to use the most modern tech, and sometimes we even want to do it with tech we’ve got no experience in. We see this with engineers, not because it’s the right thing to do but because they want to learn something new.”

    Sinclair, who was speaking as a part of an Indorse Engineering Leaders Online Panel, went on to point out how the problem becomes self-perpetuating: Customers hear about new tech—in this instance microservices—that will fix the problems they have with monoliths. They rave about microservices to software engineers, who also get excited about it and then go off and start adding microservices in software engineering projects.

    The problem is, there is just as much chance of getting into a tangle with microservices as there is with monoliths. That’s because people often start breaking things up into microservices too early—they’ve heard it will be a good idea and they’ve seen some of the benefits, so they try to apply it to everything. In doing so, they create what some might call a “micromess.”

    Avoiding the Micromess

    So, how do you avoid ending up in a micromess?

    You need to go back to the fundamentals. One way to look at it is understanding that microservices are distributed systems, something many people will have experience with. They should also be familiar with what another panelist, Oswaldo Hernandez Perez, engineering manager at Improbable, called the first law of distribution: “If you don’t need to do it, don’t do it.”

    So that means focusing on why you are building what you are building. What are you trying to achieve? This is a fundamental question that’s applicable to businesses of all sizes. What problem are you trying to solve, and how will your solution remove friction from its users’ lives?

    That’s what people care about. Even if you’re developing a niche app for a highly technical audience, they are unlikely to care too much about how it got to them, only that it did and it is fixing a problem for them.

    If the only way to achieve that is with microservices, then yes, you should definitely use them. If there is an alternative, then consider using that. Do not simply start breaking everything up into microservices just because that is what everyone is currently talking about.

    Don’t Stick Your Head in the Micromess

    Ultimately, microservices are an architectural pattern to reduce complexity. It does this, but it also adds complexity elsewhere. If used in isolation, then you’ll fix your complexity in one dimension and have it proliferate elsewhere. So, it is vital that you use the many other tools out there to minimize complexity across the entire organization.

    It goes back to the question: What problem are you trying to fix? The majority of businesses can do that with boring tech. Granted, the likes of Amazon and Google, which are cloud-native to their core, can use microservices, but even large enterprises should be asking themselves if it’s really the right choice for them.

    Established businesses are likely to have a significant legacy codebase, which needs integrating with. Again, microservices might not be appropriate in this situation—it might be worth saving them for cloud-native projects, and use boring tech to resolve the challenges within legacy IT.

    Do You Really Need Microservices?

    Time and time again, we come back to the fundamental question: What are you trying to fix with this product and/or technology? Not how, but what? What friction will you banish from the user’s life if they engage with your product? Only once you answer that question should you consider whether microservices is for you. Chances are, you may find you don’t need them by that point.

  • Up Your DevOps Game: It’s Time for NoOps

    Up Your DevOps Game: It’s Time for NoOps

    NoOps is here and it’s more than just a buzzword or an ideal. The next generation of platform-as-a-service (PaaS) platforms are evolving to minimize the developers’ operational burden and reduce dependency on DevOps for deployment. Dev teams that don’t want to deal with operations or don’t have in-house operations resources should consider deploying applications on a NoOps platform.

    To understand the concept of NoOps, we need to start with an understanding of how deployment management has evolved over the decades.

    From Hardware and Software and Back Again

    It’s been a long trend of back and forth between developers and “deployers.”

    It was once mandatory for software developers to be fluent in the hardware for which they were creating software. They wrote the code and then worked with the operations team to deploy it on in-house servers. Everything was tightly coupled. The first mainframes—IBMs and SiliconGraphics machines—ran on custom-built software, and the developers were responsible for the deployment and management. The developers had the machine first, and then had to create the code to make them run—assembly or low-level language or even hardware code; rope memory, in some cases.

    Then came the x86 revolution, when hardware became more standardized and was decoupled from the complexities of deployment. Developers started focusing on the software and organizations had separate teams that would manage the deployments.

    With the advent of virtualization and infrastructure as a service (IaaS), the deployment of software now depended on APIs for infrastructure provisioning, and we returned to an integrated development and deployment function—in other words, DevOps.

    The Problems DevOps Created

    The problem then became one of choice: As APIs multiplied, developers had to decide which APIs would be most relevant, efficient and useful to deploy their software—and on what infrastructure: bare metal, virtual machines such as EC2 and VMWare, Kubernetes, Swan, OpenShift, serverless, Lambda, Knative, etc.

    Not only did the teams have to decide on what IaaS to deploy the actual software, but they needed to also choose how and where the layers and logs would be monitored—and even where those logs would be stored. They need to consider things such as traceability and horizontal or vertical scaling issues, to name a few.

    This has resulted in the creation of bespoke permutations and combinations that work great … until vendors’ upgrades require them to keep up with tens or hundreds of configuration changes. Most teams are keeping it together, but it definitely can be done more efficiently.

    Now It’s time to Up the Game to NoOps

    It’s time for the next approach: Limit the number of choices to create standard best-in-class operations that deliver economies of scale and easily evolve with minimal hassle.

    NoOps simplifies cloud operations—everyone can do things the same way. NoOps aims to “completely automate the deployment, monitoring and management of applications and the infrastructure on which they run,” according to Forrester, which coined the term.

    NoOps is about standardizing the approach to deployments and reducing the number of variables, bringing simplicity. At its core, NoOps is focused on automating deployments and executions that are predictable and repeatable.

    NoOps & Standardization

    The development and increasing adoption of containers are critical to the entire NoOps philosophy. Containers provide the ability to independently deploy services and applications, automating and standardizing the process to deploy anything, anywhere. Using containers delivers the tremendous portability that hasn’t been seen since the development of generic hardware.

    With encapsulation within the container, whatever is running inside will behave the same no matter where it is deployed.

    The NoOps-containers movement will transform the entire DevOps industry.

    Let’s go back even further. In the 1800s, few homes had electricity. They hooked up whatever appliance they had to whatever generator they were running and kept their fingers crossed. Once governments took over the role of bringing electricity to every home with standardized voltages and outlets, the appliance industry took off.

    A true NoOps platform will provide a standard “socket” that works for everything. The developer will simply build the app, and it will “plug in” anywhere. With NoOps, developers don’t have to worry about how the application will work, deploy or scale, nor do they have to consider reliability or elasticity. The platform will deliver it automatically.

    The “Gift” of Containers

    Containers free developers from managing infrastructure and reduce the operational burden associated with building and deploying applications. They also minimize application conflicts among different environments and improve portability and team collaboration.

    Containers make application deployment easier than ever before. Continuous integration doesn’t change as part of this process. Adopting containers as part of NoOps simply automates continuous delivery—developers write the code, commit the changes and push it to the container’s remote repository. Based on the Docker file, an image will be built and the containers that run your application will be deployed. Since all deployments are rolling, no downtime occurs.

    Leveraging containers provides speed, agility and flexibility to any infrastructure, thus bringing cost savings and dynamism. Containers can flexibly host microservices, which can be scaled individually, as necessary. Using CPU, memory or even custom business rules, developers can set auto-scaling policies so applications are ready to serve an increased number of requests whenever needed. As the platform monitors application performance, it can use these insights to intelligently architect a right-sized infrastructure.

    Putting NoOps Into Play

    If you’re considering adopting NoOps, look for a PaaS provider with the expertise to make the right platform choices for you based on your application development language of choice. For example, if you are looking to host a Ruby application, ensure that the provider has the requisite Ruby expertise.

    A true NoOps platform will streamline deployment, automatically scale based on your business requirements and provide outstanding performance. The automation dramatically boosts productivity and reduces costs associated with deploying and managing applications.

    NoOps platforms monitor operations and automatically resolve issues using built-in observability mechanisms and intelligent alerts. Centralized logging, metrics and traceability make it easy to identify issues and initiate fixes without expensive downtime. Moreover, the platform should ensure strong security boundaries and adhere to applicable compliance regulations.

    When “going NoOps,” also make sure that the solution provides comprehensive documentation and 24×7 support for emergencies that would normally be handled by an in-house DevOps team.

    Wrapping It Up

    NoOps can eliminate much of the work associated with deploying and monitoring applications by simplifying and streamlining operations and reducing the need for extensive in-house DevOps support. While it’s not a one-size-fits-all solution, it delivers great value helping companies of all sizes from start-ups to enterprise teams save time and money and better redirect their resources toward high-value activities, creating the best code to deliver the highest quality products.

  • 6 Tips to Survive the World’s First Holiday Lockdown Gaming Crunch

    6 Tips to Survive the World’s First Holiday Lockdown Gaming Crunch

    ‘Tis the season to be jolly … especially if you’re a gamer. The upcoming holiday gaming release schedule is a joy to behold for locked-down, socially distanced or just plain pandemic-fatigued game enthusiasts.

    For game studios and game developers, however, holiday releases have an entirely different connotation—the infamous crunch time. While views differ on the evils or benefits of crunch time, there is one thing that the most strident detractors and the most ardent supporters of the development practice can agree on: This holiday season’s crunch is unique.

    How the world’s first-ever WFH gaming crunch time will affect actual release schedules, workflow, collaboration and game quality (not to mention coffee consumption) remains to be seen. To better prepare for the challenges of WFH crunch, here are six tips we’ve gathered from our gaming partners:

    Gaming Developer Tips

    Don’t pretend it’s business as usual, but don’t reinvent the wheel

    Developers (like everyone else) are most comfortable with what they know. So, while it’s crucial to adapt your development process and workflow to WFH, you don’t need to re-create them.

    To modify processes, put together a multidisciplinary team of your top people in each department. Bring them together (in person, if at all possible) for a short yet intensive meetup and let them brainstorm possible WFH scenarios and create a detailed playbook.

    This revised playbook should focus primarily on changing only what needs to be changed for remote working while keeping everything that already works intact. It should also include a comprehensive list of experts to be contacted for each scenario or question. And if a developer encounters a scenario for which there is no process, the playbook should help him/her identify the right team to notify.

    Once the revised playbook has been established, disseminate it to everyone then follow it religiously. Instruct employees to follow process, even if they feel that doing so may cause minor delays. Shortcuts are not the right move for WFH—it’s too easy to get confused.

    Create working groups (and avoid email like the plague)

    Email is a notoriously inefficient and confusing collaboration tool in the best of times. Late responses, missed messages in the thread … these and similar email issues can easily translate into WFH chaos.

    Instead of email, adopt a collaboration tool and strictly enforce its use. This enforcement is crucial, because the most powerful tool is useless if not used.

    Then, create multiple 10-person “smart work” groups with well-defined scopes and a domain-expert moderator to lead the discussion in each. Task these teams with addressing, discussing and solving companywide issues in their relevant arena.

    A list of domain-expert moderators should be shared with employees along with instructions on how to contact each moderator when unanticipated scenarios arise.

    Do VPN smart

    WFH employees frequently face VPN hardships such as low response times. Any latency is a nightmare when working remotely, and this can be compounded by low-speed home network connections.

    Aside from upgrading internet infrastructure (for employees at home, too) and using cloud-enabled DevOps tools (instead of on-prem solutions), implement companywide network traffic protocols to ensure smoother and more predictable VPN connection speeds.

    For example, instruct employees to avoid uploading large chunks of data through the office VPN at peak times. Instead, split these into reasonable-size chunks or schedule large commits during low-traffic hours such as lunchtime or at night. To ensure the success of this method, continuously monitor network traffic and alert employees that inadvertently violate traffic policies.

    … Or don’t use VPN at all

    Alternatively, you could trade your VPN for another acronym – FTP. When multiple teams working on multiple projects need to share artifacts, instead of uploading them through the overloaded office VPN, have them use a shared, secured FTP outside your network VPN. And consider using Steamworks or a similar solution to share testable artifacts.

    And there’s another brilliant solution you can choose, beyond VPN and FTP. The Google Stadia game streaming platform (for which many games are developed, anyhow) allows frequent uploads of private game builds. This enables developers to quickly see changes made throughout development in a live runtime environment. This little-known trait can save a massive amount of time and (especially) VPN bandwidth when working from home.

    Automate what you can

    When time loss is compounded by the inherent disconnect of WFH, any time savings are that much more valuable. Automation is well-known to save time and effort in office-based development, but in WFH its contribution can be mission-critical. As a rule, in WFH anything that can be reasonably automated should be automated.

    For example, try automating uploads to different storage locations (Steamworks, FTP, etc.). Then, you can integrate this upload as part of your build process—effectively automating the process. Allow each developer to choose in advance whether the build should be uploaded automatically, and enable email notifications when the upload is complete. This little time-saver (and others like it), multiplied by multiple development teams and multiple developers, can really add up.

    Try to avoid software updates

    In the office, when a little Windows software update goes wrong (which is not infrequent), it’s an annoyance. But in WFH, when no one is in the office to restart the affected machines, it can easily cascade into a serious productivity crisis.

    Try to avoid any unnecessary software upgrades or changes so as to not jeopardize your stable environment.  If you must update, make sure that someone is in the office to manage unexpected issues.

    The Bottom Line

    WFH is going to be the new normal, and development processes need to evolve accordingly. Crunch time is the time when processes are pushed to their limits, even in the best of times. By taking the time to ensure that the wheels are as greased as they can be, gaming companies can eliminate workflow and collaboration friction points. The end result—aside from on-time deliveries of high-quality versions—is developers who are happier, more productive and even a little bit jolly.

  • Split Allies With AWS to Advance Feature Flagging Adoption

    Split Allies With AWS to Advance Feature Flagging Adoption

    Split today announced it has integrated its namesake platform for managing feature flagging with the Amazon’s Simple Storage Service (Amazon S3) as part of an effort to streamline the process of adding new capabilities to applications.

    Company CTO Pato Echagüe said one of the most challenging aspects of feature flagging is providing a way for development teams to access data required to test those capabilities. Integration with Amazon S3 will now make it possible for Split to ingest millions of customer events per minute as Parquet files from Amazon S3.

    That capability will make it possible to have enough statistically relevant data to analyze how a feature is being employed within minutes, said Echagüe. Previously, software engineers would have had to spend hours manually configuring the Split feature flag management platform to ingest gigabytes or terabytes of data, he noted. The integration with Amazon S3 makes it possible to now analyze terabytes of data in near real-time.

    Split next year also plans to make it possible to send feature flag data to Amazon S3, where it can be combined with other types of data that could be analyzed using a third-party application.

    The end goal is to make it easier for product and engineering teams to collaboratively iterate new features and capabilities faster in a way that aligns better with a product road map, added Echagüe.

    Interest in feature flagging has been on the rise as organizations invest in digital business transformation initiatives to better engage end users that are working from home to combat the spread of the COVID-19 pandemic. Feature flagging capabilities make it easier for those organizations to test new capabilities without having to expose every ens user to a capability that ultimately may not have enough appeal to be included in the application.

    Feature flagging, of course, is already employed within many DevOps workflows. Split, however, is providing a platform that is designed to make it easier for DevOps teams and application owners to collaborate on the rollout of features based on feedback from end users regardless of what DevOps platforms application development teams are employing, said Echagüe.

    As IT becomes more complex, Echagüe noted it’s more important than ever for organizations to have a consistent approach to change management. The bulk of issues many IT organizations wind up addressing usually have something to do with a recent change to an application, he said.

    There’s no doubt that as organizations begin to rely more on digital processes, organizations will want to have more confidence in the capabilities being added to these platforms. As a result, many application owners who don’t have a lot experience working within the context of a DevOps workflow still want to be able to exercise and maintain control over what features are added to an application.

    Regardless of the approach to feature flagging, it should be apparent to all involved by now that this capability going forward is going to play a much larger role in application development and deployment.

  • Leading Effective Incident Response Without Interminable Bridge Calls

    Leading Effective Incident Response Without Interminable Bridge Calls

    There are easier ways to manage incident response without creating war rooms and packing IT staff onto bridge calls

    Your phone vibrates at 11 p.m., and you know that can only mean another major incident with one of the business’ critical systems. You get geared up for the war room, dial into the bridge call and start reviewing the major incident report. You do this instinctively because it’s the third major incident in as many weeks, and you wonder if there’s a faster, easier, less stressful way to keep critical systems running, especially during periods of operational uncertainties.

    You’re not the only one recognizing that having too many major incidents is a significant issue. In a recent survey on the future of monitoring and AIOps, 94% of respondents stated that issue resolution is critical to their business, but only 28% are satisfied with their handling of major incidents. Too many organizations feel that their only path toward improving their incident handling capabilities involves redesigning their critical business applications, facing down their technical debt or building out an SRE practice.

    There are easier ways to manage major incidents without creating war rooms and packing IT staff onto bridge calls. When used well, AIOps techniques can address the pain points in resolving major incidents that drove business leaders and CIOs to establish war rooms and bridge calls in the first place. To understand what AIOps capabilities are needed, it helps to review the history of war rooms and bridge calls and why they are inefficient in solving today’s operational challenges.

    Furthermore, IT organizations can take advantage of AIOps platforms much faster than re-architecting applications, addressing technical debt,or hiring more site reliability engineers.

    How Bridge Calls Became the Status Quo in Incident Response

    No one wants unreliable or poor performing business systems. It’s why IT teams have become extremely proficient at recovering from incidents, especially the ones that are easy to diagnose and easy to resolve. Problems such as web servers going down, databases running out of storage or services stuck in deadlocks are relatively easy to diagnose with today’s monitoring tools. In fact, over the last few years, many IT Ops groups have used tools to automate the recovery from these common issues.

    But more complex issues are harder to solve. Issues such as:

    • Problems causing a cascading failure of dependent systems that are all sending frequent alerts.
    • Issues right after major application deployments or infrastructure changes.
    • Bottlenecks in customer-facing applications that are experiencing unusually heavy loads.

    IT’s history of solving these complex incidents isn’t great. The incident management team responds and calls in help from Tier One support. With so many things going wrong, operational teams have no choice but to call in higher levels of support, including developers. By the time someone communicates to the business on the incident’s status, leaders are irate over the lack of communication, and the time it is taking to recover from the issue.

    CIOs and IT leaders dislike being yelled at and seeing long outages. Their easiest management response is to get all the experts in the room, often called the war room, in the hope that having more people involved is better. War rooms often include bridge calls to allow remote people to attend, and bridge calls are standard practice for major incidents occurring off-hours.

    Bridge Calls Don’t Solve Incident Management Problems Better

    It surely isn’t better for the operational engineers to be responding to off-hour incidents regularly. It’s not better if a seasoned major incident manager is needed to oversee the bridge call and ensure differing opinions don’t escalate into arguments or finger-pointing. It isn’t better if the recovery times miss business objectives and if root causes are never identified. It isn’t better if the resolution times to recurring problems don’t improve.

    It’s also really bad if the number of complex issues is increasing. This is likely to be the case as application architectures based on hybrid cloud or containers and microservices add complexity while frequent releases from DevOps teams add to risk. Also, global events such as COVID-19 create usage uncertainties and network bottlenecks during periods of increased business importance.

    Solving for the Root Cause of Inefficiencies

    Bridge calls and war rooms are inefficient practices to solve complex problems. They require too many people, take too long and require too many more follow-ups to identify the root cause.

    IT has built up an arsenal of tools to better manage myriad operational domains. We’re using one set of monitoring tools for the data center and a separate one for the public cloud. Every database, API, microservice, application component and type of IoT device has different tools to monitor performance. In fact, according to the future of monitoring and AIOps survey, about 20% of respondents reported having 25 or more monitoring tools. In addition, there’s also all the system, network and application log files that often have the most critical information when troubleshooting a complex issue.

    That’s many tools, considerable data and countless alerts to sift through under the pressure of a major incident. These tools often require different skills and subject matter expert participation, which is another reason why bridge calls and war rooms are so crowded.

    Now, AIOps can mean many different things to different people. But if it’s going to improve major incident recoveries without requiring bridge calls or war rooms, then it must:

    • Aggregate all the data from these monitoring tools.
    • Aggregate data from all of your change and topology tools, so you can find out which changes led to an incident or what’s affected by the incident.
    • Sequence the data into a time series showing what issues came first.
    • Correlate the hundreds or even thousands of alerts into discrete events that operators should review.
    • Present the information in an easy console for operators to assess during major incidents.
    • Enable engineers to automate steps in the recovery.
    • Integrate with different workflow, collaboration and communication tools to automate incident response steps.
    • Share the event sequence with engineering and development teams.

    During incidents, the differentiator is that machine learning models have gotten a head start in processing all the data. Instead of dozens of tools requiring lots of people, the start of the incident review can begin by examining information in a single platform. Instead of IT Ops being inundated with too much noisy data and alerts, machine learning has already processed and correlated events into a more easily decipherable storyline. Instead of manual recovery steps, automated steps can be centralized and orchestrated from a single console. Lastly, since the organization’s workflow, collaboration and communication tools are integrated, no one should feel out of the loop on issue status and postmortem steps to make improvements.

    IT teams often celebrate when a legacy system is shut down because operations have fully transitioned to a modernized platform. Isn’t it time to do the same with war rooms and bridge calls?

  • Change Management in a Fast-Paced DevOps World

    Change Management in a Fast-Paced DevOps World

    While DevOps speeds up software release cycles, change management often slows them down. So, how do you do change management in a fast-paced DevOps environment? Change control enables organizations to meet compliance and governance while managing changes at the speed of business.

    In this interview for TechStrong TV, Greg Sanker, former CIO, IT Operations specialist, author, speaker and contributor to the “ITL4 Change Enablement Guide,” joins us to share his views on the tremendous impact of Agile, DevOps and CI/CD on traditional change control teams. Greg talks about the new role of change control and his contributions to the “ITL4 Change Enablement Guide.”

    The video is immediately below, followed by the transcript of the conversation. Enjoy!

    Transcript: Change Management in DevOps

    Mitch Ashley: Well, I have the pleasure of being joined today by Greg Sanker who is former CIO, operations specialist, he’s a speaker/author, a bit of a futurist in leadership and digital transformation. He’s also a contributor to the “ITIL4 Change Enablement Guide” and a reviewer of a lot of other content—so, long introduction, there. Welcome, Greg, to [Cross talk].

    Greg Sanker: Yeah, wow—thank you. That’s kind of awesome. Thanks for having me. I appreciate being on the show. I think we’re gonna have—

    Ashley: It’s always—always good to have a fellow musician. Anybody with a Telecaster in their background can be on this any time you want.

    Sanker: Yeah.

    Ashley: So, tell us, [Laughter] in addition to your wonderful guitar sitting behind you, tell us a little bit about yourself.

    Sanker: Well, you mentioned, I am—I specialize in IT Operations. That’s what I’ve done for more decades than I’d maybe like to admit. But service management has been a core part of that, and it’s become really clear to me that my view of service management is a lot more streamlined and agile than a lot of people perceive it as. And so, I’ve begun speaking at conferences and whatnot. I have a book of my own that’s published—I’ll let you guys go find it if you’re interested—about change management.

    But most recently, I was asked to work on the ITIL project on a number of ways, but most notably, the change, what we now call “Enablement Guide.” And one of my prerequisites to get engaged with the project was, if you’re gonna ask me to write it, I’m gonna write it the way I see it, which is very different than what we’ve been doing in the past, and we’re gonna be pushing it forward, so. So, that’s a bit about me.

    Ashley: That’s wonderful. That’s definitely a consistent theme with ITIL4 and other folks, Mark Smalley and others that we’ve talked with is really recognizing—I think ITIL’s had its sort of process moniker, right?

    Sanker: Yeah, pretty much.

    Ashley: But there’s a huge emphasis on, you know, how do you fit ITIL and all of the aspects of operations into something that is more of a dynamic DevOps-y agile kind of world, if that’s a way to describe it. And not to say that it didn’t fit in before, but I think it’s making it more prescriptive in that kind of setting is what a lot of the good work that you and others have done.

    Sanker: You know, I was an early embracer of DevOps, and part of the reason for it is the mindset that explores possibility that ignores the boundaries in an organization to achieve outcomes. That’s, like, music to my ears—I’m all in. So, it’s a match made in heaven, but I can mention, it’s like, there’s a lot of organizations that have adopted service management, they have used ITIL in ways that are very prescriptive, very heavy weighted, very bureaucratic and slowed down. And it was never intended to be that way.

    Ashley: Mm-hmm.

    Sanker: It just is, it got adopted that way, right?

    Ashley: Well, and I think a lot of the work that’s gone into ITIL4 will help make some of that shift.

    Sanker: Yeah.

    Ashley: Certainly, it’s positioned that way, and I know from other authors, that was the mindset of how the contributions were made. Now, you specialize around change management and we were chatting when we first got introduced about, you know, how do you do change management in a rapid development, agile, DevOps kinda world? If you were gonna be ask that at a conference by somebody, how would you—what guidance would you give them?

    Sanker: Wow. So, I’ve got an hour and a half platform, right, for that?

    Ashley: [Laughter] At least an hour and a half compressed into 20 minutes, yes.

    Sanker: Twenty minutes, right? Yeah, absolutely. Well, see—well, let me put it this way. I think it really comes down to one fundamental paradigm that we need to change is, traditional change management, which is what I call your CAB-based, you know, Razor request for change, blah blah blah, is based on the idea that we inspect every single change. And as you probably know, Edwards Deming’s said many, many decades ago, “We’ve got to cease reliance on mass inspection to get quality products.” Well, that’s exactly what we’re doing when we say, “Hey, I need to make a change. Well, let’s just take a look at it,” you know, so on Thursday, we’re gonna get together and look at that. But you know full well that, in a DevOps environment or a continuous integration, I mean, we are moving all the time. We don’t slow down and stop.

    And so, that mindset, just the mindset of stop and let’s inspect is completely incompatible with a flow-based, whether it’s agile or DevOps or CI/CD or any of the others that are out there that are more flow based—you know, rapid feedback loops, short cycles, et cetera—it’s just incompatible.

    And so, what we have to do is elevate out of the inspect mindset into the expectation mindset. And that is—what is it that this organization needs from change management? And some of it’s governance and compliance, some of it’s operational stability and recoverability and some of those kinds of things. But let’s state what that is, we’ll state what those things are for the organization so that those that are working in the pipelines, the value streams can actually optimize how well we achieve those expectations and don’t dictate how they do it, let’s just dictate what the outcomes that are required by the organization. That’s the fundamental shift, and it’s big. I mean, it’s more than a 180-degree shift. It’s very significant.

    Ashley: Definitely a different mindset for sure, you know, the inspection versus not. It’s not unfamiliar, though—that’s what security is doing about shifting left and moving ______.

    Sanker: Yeah.

    Ashley: Iterating the process, DataOps is also—you know, everyone wants to move early in the process, but it’s more than that, because you’re talking about a pretty different mind shift change. So, talk about compliance. Talk about governance from a change management standpoint in this new paradigm.

    Sanker: Yeah. So, you jumped right to the jugular. That is the million dollar question.

    Ashley: [Laughter] No, but it needs to be done and, you know, we all sort of pooh-pooh it.

    Sanker: Right.

    Ashley: But you know what? If you don’t do it, it’s really not fun doing it at the end, right?

    Sanker: So, maybe you remember, because it certainly rattled my world—Dr. Forsgren said on stage at DevOps Enterprise Summit 2018, “CAB is useless.” And everybody in the audience laughed and cheered and duh duh duh duh duh.

    Ashley: You’re not talking about Cabernet, you mean change advisory board.

    Sanker: No. [Laughter] I bet you probably would enjoy a good Cabernet, and I’d—

    Ashley: CABs, yeah. Those others are still good, but anyway.

    Sanker: Merlot, et cetera, they’re all good. But the thing is, she’s not wrong. But she was looking at it from a very specific lens, and that lens was about failed changes—in other words, the success of changes and the recoverability of changes, two things that are really important to the stability of the business. And what she found in the data, she’s a data expert, she has a Ph.D. in this, so there’s no mistake, here.

    She’s absolutely right—however, CABs frequently are part of the compliance equation. In other words, it’s useless for those things, but it has a use elsewhere in the organization, and for many organizations, that’s a very scary thought. Because that’s where we’re able to document and demonstrate for internal and external auditors and validators—are we doing the things that we’re required to do? PCI DSS, HIPAA, GDPR—you know, all of those other compliance framework, NIST 800-53, which hopefully we’ll get to in a minute, but they all have requirements around managing changes within your organization. So, if we don’t do it in the CAB, how do we do it? And that’s precisely the point.

    So, what I’ve sought to do is help organizations build those compliance points into the pipeline itself. In other words, we can point directly to where that’s being done in the pipeline and don’t require a “let’s slow down and actually check the box, here.”

    You know, the governance is—well, can I just say this? There’s a difference between governance and compliance. They’re two very different topics, right?

    Ashley: Yeah.

    Sanker: So, governance is about how organizations make decisions, and making sure that those decisions that are made throughout the organization align with or help fulfill the purpose of that organization, whereas compliance really has to do with the conformance. Are we conforming to applicable standards or expectations, either that the organization has or we have externally?

    So, but when we talk about governance and compliance, we think about performance and conformance. I’m sure you’ve heard that—COBIT and lots of other frameworks have talked about that for years. But performance is about, you know, your strategy and are you utilizing your resources in pursuit, or are you producing value or value creation? And I think that’s fascinating.

    I’ll bet you’ve read or heard of Mark Schwartz’ new book, “The Art of Business Value.” It’s a fascinating read.

    Ashley: I actually haven’t read that one. I’ll have to check it out.

    Sanker: I would put it on your reading list, and I know that sometimes he rubs people the wrong way. I found out that he rubs me exactly the right way, because he asks some really challenging questions, like, “What is business value?” You’d think that’s an easy question to answer, right? It’s like, you know, for a for-profit corporation, it’s like, you know, making your market windows, making your profitability, adding features that attract and retain customers, those kinds of things.

    But what if you’re in—as I was as a CIO—what if you’re in a government agency? That’s not a for-profit organization, so what is business value? Delivery of services to citizens, upholding environmental standards? There’s a lot of things that governments do, but they’re not profit motive. In fact, in some cases, it’s not even financial, so what is business value?

    And so, when we’re running our backlog, you know, we’re scoring our user stories, like, how do we score that in that kinda context? And then, just when he, when I thought that he had confused me enough, he asked, “Well, what if your business strategy”—and I don’t wanna give away his book. Go buy his book. But—

    Ashley: No, just tell me the last chapter, that’s all I need.

    Sanker: Exactly. [Laughter] I actually sent him an e-mail. He’s a great guy and he thinks very, very big. But one of the things that he said was, “Well, what if your strategy is, we want to move customers off of this platform and onto one that’s more profitable to us, or one that’s in alignment with our roadmap and our strategy?”

    So, if we’re down in the value streams or in the pipelines, scoring our user stories against customer satisfaction, customer value, things that customers really, really demand and want, we’re actually creating negative value—these are my words, not his. We’re creating negative value because we’re making our customers like the platform that we want to get them off of.

    Ashley: Mm-hmm.

    Sanker: So, business value would actually be—no, do things that make them less likely to like this product. Make them want to migrate. Make it easy for them to say, “You know, this is really getting old. I’m gonna try your news service.” And so—

    Ashley: Right, that behavior change.

    Sanker: Yeah, exactly. And so, when we really start thinking about, we’re getting, you know, your governance, especially your governance and your compliance, we have to have adaptive, rapidly adaptive systems that allow us to, at any point in time, understand what the current strategy is and how it affects the decision making that we’re doing at any point in time—all without, of course, getting your board of directors involved with every single decision throughout the organization.

    Ashley: Well, let me ask you this. I’m really interested, I’m really curious about how you take this kinda shift in mindset and how you’ve woven that into the “Change Enablement Guide“ that you were the author of. So, you know, it isn’t a process discussion.

    Sanker: No.

    Ashley: You’re talking about viewing things a different way, valuing some things maybe we didn’t value before, maybe some other things less, and maybe delivering a different set of things. Let me just share, briefly. I remember a day, probably not too long ago, where CABs saw their role as saving us from developers and all the bad decisions and lousy code they were gonna create, so that’s what—

    Sanker: [Laughter] Oh, yeah! Absolutely.

    Ashley: – that’s what they say in those CAB meetings.

    Sanker: Yes. Oh, absolutely.

    Ashley: But, you know, that’s not what we’re talking about. Tell me how—so, how do you build that into, in a way that’s gonna help people make that shift through ITIL4?

    Sanker: So, thank you for saying that, because there is some unspoken or lightly spoken or spoken only in factions kind of frustration, there. It’s like—let’s be honest. I used to be a network engineer, I did network security, I did all things network. So, I used to be a change manager at one point in time, and who the heck are you, Greg, to cast judgment on this piece of code and decide whether that’s gonna work appropriately? Everybody knows that I can’t add value to that, and everybody knows that this external body who’s not familiar with it, certainly not at the pace—you can’t add value, and yet you act like you do. And that just—that doesn’t sit well with anybody.

    So, we really need to, if we shift our focus to, “What is it that we’re trying to achieve?”—so, I talk about putting the right measures on the pipeline. How well is this pipeline producing outcomes, those outcome expectations? Is it producing consistently high quality changes? Is it consistently not delivering failed changes—or, for the failed changes that do go through, how quickly and easily can we recover from that with minimizing impact to the customers?

    Those are the things that your CAB or whatever your body is should care about, not the individual changes. You’ve got to get away from the individual changes. You can’t add value to that.

    And so, I get asked a lot of the times, it’s like, “Well, what do we do with our CAB?” since, you know, they’re very unpopular. And in some cases, the best answer is—now I’m gonna say it publicly, so I’m gonna be quoted on this, but—“Blow it up. Just get rid of it. And don’t ever use the C-A-B word again, because it has such a bad name.”

    But it isn’t without value. So, if you have a CAB and the CAB starts saying, “Look, developers, we can’t add value to what you’re doing, but what we really care about is some of these things. So, let’s talk about those things.” And if you’re talking to, particularly DevOps, agile, automated pipeline folks—they were gonna welcome that conversation because they’ve already got it covered. They just do. You know, like you know, I’m the change manager—“Well, have you tested that code before you released it?” Well, dude, that’s the only way it gets in production, if it passed tests. There’s no way for it to make it into production. So, not only is it insulting, but it’s a dumb question, and undermines your credibility.

    Ashley: Well, let me ask you this, then. I think where you’re going—and tell me if this is right—is, “We have so much automation in this pipeline process, right? That’s the whole idea of being able to produce software more reliably, more rapidly.”

    Sanker: Oh, yeah.

    Ashley: There is so much data that’s generated and captured, but not leveraged.

    Sanker: Yep.

    Ashley: We treat it as data exhaust, right? That’s sort of this fancy term for it.

    Sanker: Yep.

    Ashley: In that is a lot of compliance, is a lot of governance information—

    Sanker: Yeah.

    Ashley: – that you can then elevate and connect, and basically save developers time and not having to produce reports at the end. “I’ve already got three-fourths of what I need or what you need, and let me help you put the finishing touches on it so you’ll sail through the next whatever NIST whatever it is.”

    Sanker: Yeah.  So, there, you get to write a chapter in my next book, because that’s exactly, exactly what we want to focus on is, we want to be—you know, alright, this is potentially controversial, but change management is a feedback loop. It helps us understand the quality of changes that are coming through our pipeline. Sitting there and passing judgment and saying, “Nothing passes until I say so”—that’s not adding value. That’s absolutely antithetical—you know this—to the DevOps continuous flow kinda mindset.

    Ashley: So, they’ll go every way around you you can think possible, you know?

    Sanker: Yeah, and the other thing, as we well know, is like, develop is most frequently used in the flagship products. “This is the core business. Get out of our way. We have to deliver these changes to maintain our profitability, maintain our market windows—all of those things that are core to the business, and you think I’m gonna stop for three days when I went from, you know, a user story to a value in under 24 hours in some cases? There is no way I’m gonna stop. I’m just not going to. I will take the risk of it failing and you saying, ‘You shouldn’t have done that.’” And that’s—

    Ashley: You can talk to the chief product officer about—

    Sanker: Exactly.

    Ashley: – the stock’s direction. [Laughter]

    Sanker: And I helped an organization who was, you know, they had agile coaches and they were doing all that. It’s a large company, but I’d rather not name them. And I was asked to come in and look at their change management.

    Well, what they had done was, rather than having one CAB a week, we were having daily CABs. And we were having daily pre-CABs and then the CABs. And one of the things that I tried to get them to understand is, these two are entirely incompatible. If you’re going towards agile DevOps as a development methodology, then you must move away from CAB. CAB has got to go.

    And what was interesting is, their flagship products, the ones that really made money for the organization, they could get away with murder. Why? Because it was so critical to massive revenue in the case of this organization.

    Ashley: Mm-hmm.

    Sanker: And so, we have a big outage, we recover, we move on, that’s okay, it’s an acceptable risk. And change management will still know, we can’t have any defects or whatever in production.

    And so, we’ve got to get ourselves wrapped around the idea of managing business risk. You know, you mentioned something about the compliance, and just this last week—like I said, I’m working on a new book, but I was looking at the NIST 800-53. Everybody’s familiar with it, but they finally released version 5. And so, I’m reading through and the author’s made a very fascinating statement, and I encourage everybody to go look at this. Version 5 is phenomenal, because what they said was, “We have eliminated or reduced the specification of who actually does the activity, and we’ve instead focused on the outcomes that need to be achieved.” In other words, some of the compliance requirements or the controls said, “IT will review” or, “The change manager will review.” And so, if you’re gonna comply with NIST 800-53 previous versions, there’s not a lot of wiggle room around that.

    But I’ve been saying for over 10 years and others have been saying, it’s like, if we focus on the outcomes, then who’s doing it, how they’re doing it is open for us to do it in the most effective way. And that gets us into the pipeline space. And it’s a major step forward, because we struggle with that. Like you said, we have to meet our compliance, you know? If you’re dealing with sensitive information, you know, pharmacy has HIPAA and credit cards and all that sort of stuff, you have to. It’s just not optional. But you don’t have to do it the way we’ve always done it, and that’s the piece that I like to bring to organizations. It’s like—how are you doing it, and can we demonstrate that we’re doing it? Those are basic audit business controls minimums.

    Ashley: I’m curious about, we’re sort of talking also about the whole issue of agility and speed, and on the other end of it of the belief of the best way for something to be stable is don’t change it.

    Sanker: Oh, yeah, absolutely.

    Ashley: Which is antithetical to the environment we’re in, for sure. And so, it’s kinda like you have to embrace, you just have to say, “We love change, and how do we be super effective at it?” instead of, “How do we not change so we don’t mess it up?”

    Sanker: So, there’s two places, two things that I’m gonna mention briefly, but we won’t have time to explore them. There’s the field of study around complex, you know, and Cynefin is one of the most notable frameworks. And the idea that there’s simplex, complex—or complicated, complex, and chaotic, and we treat changes frequently like they’re either simple or complex. Meaning, we need to get some eyeballs on this thing.

    But the fact of the matter is, most systems, especially systems that we’re DevOps-ing, are more like a complex adaptive system. There’s a level of complexity that we’ve not seen in days of old, like back in the ‘80s and ‘90s and early 2000s where these systems were, we could actually try to understand them. Modern system—

    Ashley: They’re not linear. They’re more, you know, dynamic and self-creating, if you will, in run time.

    Sanker: Yeah, exactly. So, that hit me a number of years back. It’s like, this whole mindset, it’s around the idea of, it either is simple or it should be simple, and we can understand it, and we can predict the outcomes, and you can’t. You cannot do that.

    And so, we need to build adaptive systems that say, “Hey, look, this is complex.” And you’re not gonna send it to a board of examiners, you know, a CAB who are gonna say, “Let’s take a look at all the risk,” because it would take you years to do that in a complex system. That’s just—that’s not gonna happen.

    So, we need to adopt practices that are more adaptive. And one of the things in that complex area within the Cynefin framework says that we need to run some experiments to better understand the mechanism. And how that applies to change management—see, this hit me between the eyes because it was like, “No failures in production. No failures will be allowed”—that’s antithetical to learning how the system works. If you don’t make some changes to see how the thing works in operation, you’re denying yourself learning and you’re probably deluding yourself that you’re actually actively controlling it, right?

    The other piece of study that I’ve brought to the table is—and you’ve probably heard of it—this study of anti-fragile. There’s a number of authors whose understanding is astronomically higher than my IQ, you know, Ph.D.s who actively debate such things. But I take complex things and make ‘em simple. Here’s the thing—some things, like a tree, when the wind is blowing, the tree is waving, but there’s more than that going on. It’s the actually fibers, the plant itself becomes more resilient to wind if it’s been exposed to wind.

    Here in Oregon, often there’s logging operations that have a lot of controls for the environment—trust me on that. It’s changed a lot since the good, ol’ days. But one of the things that we do hear is, we leave a segment of trees at the edge of a piece of property that’s being logged, and we do that for the animals, for the environment, we protect the riparian areas. But one thing that you’ll notice is, within one year of leaving that stand of trees that used to be next to a large strand of trees—they break. A storm comes, and they get hit. Why? Because they’ve relied on the strength of their neighbors up to this point, and they’re not suited to stand the wind alone, and so those trees break.

    Here’s the thing—if we don’t inject some failure into the system, for a system that is truly anti-fragile, we’re actually making our systems more fragile. And so, a mindset of change management that says, “No errors, no problems, no failures ever, ever, ever”—you are making your system more fragile, and you’re treating a complex system like it’s simple. And so, we’ve got to change our mindset.

    Now, you know full well—and let me just blast into this—you know full well that, as we take these mammoth changes that we used to make in the old days, you know, it took 6, 8, 12 months to produce, and we’ve turned it into these microscopic things that can go through your pipeline and ours and produce values is your blast radius. What bad thing could happen in production has materially changed. The risk has become very, very small. If that—

    Ashley: And your ability to fix it—

    Sanker: And your ability to fix it skyrockets.

    Ashley: – your responsiveness, it isn’t six months to fix it or three, it’s three minutes.

    Sanker: Yeah. So, one of the change outcome expectations that I encourage people to establish is, what are your expectations for recovery from any given change, and can we realistically do that?

    So, for instance, if I put something in my flagship product that doesn’t work as desired, if I can recover within two minutes, no harm, no foul, is that okay? In a lot of cases, it is. If it’s a defibrillator keeping somebody alive—probably not, that’s probably not okay. But in a financial transaction or a video streaming service or whatever, you know, modern digital services, it’s probably okay, especially if you’ve engineered the release in such a way that you do it like that. You know, blue/red switches, on/off—all of that kind of stuff that we’re engineering in become, you start seeing why we’re able to do that and what that actually does in production. It’s wonderful.

    Sorry, you got me excited about that.

    Ashley: Yeah, boy, I could take you a lot of places with that. [Laughter]

    Sanker: Yeah.

    Ashley: So, let’s take it back to someone who’s gonna pick up a digital version or a printed version of the “Change Enablement Guide” author.

    Sanker: Yeah.

    Ashley: How is that approachable for folks? Let me be honest. I’m probably not gonna go download that to get my heart pumping to go work out, get a better workout—or maybe I will.

    Sanker: [Laughter]

    Ashley: Maybe after listening to this, I will—but how do you make that engaging for people?

    Sanker: Yeah. Well, so, one of the things—well, I’ll put it this way. It’s like, we’re not selling a product, we’re selling an opportunity to solve some problems. That’s why we call it best practice, right? And there’s a lot of best practices, and I don’t feel like they’re nearly at odds as maybe some of the industry might feel like.

    But change enablement is—and I’ve been saying for years that this day has come and it’s now here is, change management is in the crosshairs of modern digital organizations. It’s a pesky remnant of days past. And what the “Change Enablement Guide” is—and there are was more. I mean, for production reasons, I had to kind of limit it. But what I wanted to do was point in this direction of what I’m talking about. It’s like—let’s get people moving towards that. Because like you said, we’re not gonna change overnight. I’m not gonna read the “Change Enablement Guide” and go back to work the next day and say, “Alright, I can save all the world’s problems.”

    And so, what I’ve subsequently started doing is writing guidance around, “But what do I do today? You know, I went to a DevOps conference, I’m excited, but what do I do when I go back to work? How do I start moving in that direction?” And blowing up the CAB, while we might all celebrate that, that’s not a good first step. Instead, what we need to do is start moving ourselves in a better direction. It’s not more of same, it’s—what do we actually need to be doing here, and what’s the best way to accomplish that?

    And it’s difficult, culturally, because it becomes more of a culture. Like you said, there are CABs galore who despise developers probably as much as developers despise CAB. It’s a match made in—

    Ashley: Cats and dogs, living together.

    Sanker: – yeah, exactly. [Laughter] Good reference. And so, that group of people who are widely known for having that kind of mindset are not gonna be welcome when they show up to your spring and say, “Hey, we’d really like to be involved earlier in your change management so that we can help you.” It’s like—yeah, sorry, bud. Let’s start somewhere else.

    So, if change management starts with, “We’ve gotta get out of your way. We’re gonna start delegating more and more of your changes into your value stream, and instead, we’re gonna be focusing on, we’re gonna partner with you on, let’s look at the results of what you’re doing. Because we believe that you are optimizing your delivery and we believe that you have the right tools and controls in place.” So, we are moving more towards, “Are we achieving all the outcomes that the organization has? Is the organization even clear on what those outcomes really, really are?”

    You know, the one that always—I love this one—“All changes must be tested before going into production.” It’s like—well, how many shops have you been in where that’s not true, at least in modern times?

    Ashley: Mm-hmm, yeah. I call that the “meh” policy. “We’re gonna test it? Meh.” No, of course we’re gonna test it.

    Sanker: Yeah, we call it blow testing, right? You know, it’s like, “Eh, okay, I think we’re alright.” That is true. In modern terms, though? For flagship products, for things that are really, really core to the business, most organizations have some form of automated testing, or that’s just part of the process. So, rather than make everybody say, “Did ya test it?” at a CAB, just, like, “Hey”—

    Ashley: “Of course you did.”

    Sanker: Yeah, exactly. Is that happening in the value stream? If it is, I’m not—don’t worry about it. We’ll just move on. Is somebody with the appropriate authority approving that? And that’s—I am gonna take a minute to talk about this one, Mitch, is that everybody talks about change approval in terms of who’s gonna approve this idea going into production, this change going into production? Because if something goes bad, we wanna know who to blame, right? In fact, we used to say, “Who has enough—who is sufficiently authority that they’ll get fired if something goes bad?” Which is, of course, the wrong question.

    Ashley: Yeah, [Cross talk].

    Sanker: Because the right question is, “Who has enough authority that, if it goes bad, they don’t get fired, because they have the authority to put on that level of business risk?” Right?

    But if we move—and we could talk for hours, but the idea of product ownership or taking a product view of services or modern systems is, “Isn’t it true that the product owner is accountable to the organization to decide what things we’re going to move into production?” So, to me, that’s the change approval. We’re approving that we want this change to happen.

    So, nobody’s adding value by making approval at the end of the chain. We’ve already decided that. We already know that we’re gonna make this change.

    So, those are two different approvals. One of them is about a product decision or a governance decision, the other is about a business risk decision of, “Is it actually going to achieve the results that we set out to do?” I believe that most of the latter can be automated. If it’s passed its test, if the testing systems are appropriate, if we’ve done—in some cases, I love peer reviews. It’s like, you know, the code needs to be reviewed by a peer, stamp it, and we ship it out. It happens in a minute or two, you know, not hours or days or whatnot like that.

    So, I think as we kind of retool or thinking around it, I think that’s where we can start really solving actual problems that organizations have.

    I am reminded, though, I saw this—you’ve seen these before, these overhead views of an Indy car or a Formula One car coming in for a pit stop and all this stuff happens and off they go. I saw one this week that was 1.8 seconds. It was awesome, right? And people like thus always share those on social media of, like, “That’s an agile organization, doing the things” and blah blah blah.

    And so, you can’t really argue with 1.8 seconds—four tires, top off the gas, and out you go.

    Ashley: Impressive.

    Sanker: But the question, to me, is—like, I’m a bit irreverent. Why isn’t it zero? Why can’t we make it zero? If faster is better, let’s make it zero. And the answer is, we won’t make it to the end of the race unless we have fuel. We won’t make it, because [Laughter] when you’re racing at 200 miles an hour around corners, your tires literally don’t last that long. It’s not a matter of preference, it’s a matter of life and death, right? So, we have to change tires.

    So, the question isn’t, “Are we balancing speed and agility against risk mitigation?” It’s what it is that we’re trying to accomplish here. And in the case of Formula One, we’re trying to win the race. And to do that, we have to take on certain activities that cause delay—that’s a requirement.

    So, the question isn’t, “Do we have to take a pit stop?” We do. We need tires and we need fuel and sometimes minor adjustments. So, the question then becomes, “How quickly can we achieve those outcomes?” And putting four tires on in under two seconds—that is phenomenal. Hats off to those folks that are doing that. And how do you get however many gallons of fuel we put in a Formula One car in two seconds, I don’t know, but that’s amazing, right?

    And I feel like that’s the mindset that we need to adopt. We need to quit asking, “Why do we have to do change management?” We have to do change management because part of it is to be compliant. That keeps us in business. That keeps us out of trouble.

    Ashley: But it’s a really good example, because it’s not an absolute, it’s not, “Do we need to fuel?” but, “What’s just enough fuel for us to win?” right?

    Sanker: Exactly. Oh, and trust me, you’ve watched it. I know, I can see it in your eye, you watch Formula One or Indy and there’s like a thousand computers over there and stopwatches and gauges and dials and they are calculating exactly how much fuel based on the game strategy or the race strategy, the conditions, who they’re up against, what they think their strategy is—that’s how much fuel we need to take on, right? So, it’s—

    Ashley: Great model for us to think about of, you know, it’s not absolute quality or absolute whatever.

    Sanker: Right.

    Ashley: It’s what’s appropriate for the organization. We’re gonna have to leave it at there.

    Sanker: Okay.

    Ashley: And I want you to come back some time, because the next topic is Stevie Ray Vaughan or Eric Clapton. But I’m gonna—

    Sanker: Oh!

    Ashley: You can’t answer it. You have to come back and answer that, so we’ll have you here another time. It’s been a real pleasure talking to you, Greg Sanker, who is the author of the “Change Enablement Guide” of ITIL4. So, I think you really gave us some compelling reasons to go check it out. So, thanks a lot—it was great talking with you.

    Sanker: Thanks for having me, Mitch.

    Ashley: You bet.