devopsy.kinsta.cloud

Tag: feature flags

  • Prevent Technical Debt by Knowing When to Remove Feature Flags

    Prevent Technical Debt by Knowing When to Remove Feature Flags

    As more organizations use feature flags and experiments, it’s critical to understand that some of these changes should exist for only a short or specific period of time, and then be removed from your codebase. Removing flags when appropriate avoids technical debt and complexity that can make your code fragile, harder to test and more difficult to support.

    The simplest ways to evaluate when a feature flag or experiment should be removed include looking at how long it’s been in the system and how many different states it’s in. If the feature flag has been in your system for a long time and all of your users have the same state of the feature flag, then it should likely be removed.

    However, evaluating based on just those two factors isn’t enough. It’s smart to also evaluate the purpose of a flag or experiment before finalizing the decision to remove it. The appropriate lifetime of a feature flag or experiment, and when it should be removed from your codebase, depends heavily on its use case. 

    When to Remove Temporary Flags and Experiments

    If a feature is designed to be rolled out to everyone and you don’t expect to experiment on the feature once it’s launched, then you’ll want to ensure you have a ticket tracking its removal as soon as the feature has been fully launched to your users. These temporary flags may last weeks, months or quarters. Examples include:

    • Painted-door experiments: These experiments are intended to be used only in the early phases of the software development lifecycle and aren’t intended to be kept in the product after they have validated or invalidated the experiment hypothesis.
    • Performance experiments: These experiments are intended to put two different implementations against each other in a live, real-world performance experiment. Once enough data has been gathered to determine the more performant solution, it’s usually best to move all traffic over to the higher-performing variation and remove the related flag.
    • Large-scale refactors: When moving between frameworks, languages or implementation details, it’s useful to deploy these rather risky changes behind a feature flag so that you have confidence they won’t negatively impact your users or business. However, once the re-factor is done, you hopefully won’t go back in the other direction.
    • Product re-brands: If your business decides to change the look and feel of your product for brand purposes, it’s useful to have a gradual rollout to gracefully move to the new branding. After the new branding is established, it’s a good idea to remove the feature flag powering the switch.

    How to Review the Lifetime of Permanent Flags and Experiments

    If a feature is designed to have different states for different customers, or if you want to control its configuration for operational processes, then it’s likely the flag will stay in your product for a longer period of time. Examples of these flags and experiments include:

    • Permission flags: These flags are useful if you have different permission levels in your product, such as “read-only” that doesn’t allow edit access to the feature. They are also useful if you have modular pricing like an inexpensive “starter plan” that doesn’t have the feature, but a more costly “enterprise plan” that does have the feature.
    • Operational flags: These flags control the operational knobs of your application. For example, these flags can control whether you batch events sent from your application to minimize the number of outbound requests. You could also use them to control the number of machines that are used for horizontally scaling your application. In addition, they can be used to disable a computational expensive, non-essential service or allow for a graceful switchover from one third-party service to another in an outage.
    • Configuration-based software:  For any software or product that is powered by a configuration file, this is a great place to seamlessly insert experimentation that has a low cost to maintain and still allows endless product experimentation. For example, some companies may have their product layout powered by a config file that describes in abstract terms whether different modules are included and how they are positioned on the screen. With this architectural setup, even if you aren’t running an experiment right now, you can still enable future product experimentation.

    Note that even if a flag is meant to be permanent, it’s still critical to regularly review these flags in case they become obsolete or should be deprecated, which would then add technical debt to your codebase.

    Establishing a Feature Management Process 

    Creating an always-on feature management process goes hand-in-hand with using feature flags and experimentation effectively. Some organizations use a combination of a task tracking system and their feature flag and experiment platform to manage this cleanup process seamlessly and quickly. If the state of feature flags and experiments can be synced with a ticket-tracking tool, then an engineering manager can run queries for all feature flags and experiments whose state has not been changed in the past 30 days, and track down owners of the feature flags and experiments to evaluate their review. Other teams schedule recurring feature flag and experiment removal days to ensure engineers review the oldest items on a regular basis to minimize technical debt.

    The benefits of feature flags and experimentation to help teams ship code and launch new features quickly and safely far outweigh the needed management to minimize codebase technical debt and complexity. When managed as a recurring process, looking at all key factors (lifetime, state and use case), teams can feel confident that their systems will remain efficient and reliable.

    — Asa Schachar

  • Pendulums and DevOps

    Pendulums and DevOps

    I have long noted the trend of pendulums in IT, particularly in organizations with longer histories. Centralized IT will be achieved, its weaknesses noticed and a movement will begin to decentralize. Decentralized IT is achieved, and people remember its weaknesses, causing centralization.

    The same is true with speed of delivery versus quality. Faster will be more important for a while, then the reality that faster causes quality concerns will surface, and reliable becomes the keyword, causing slower rollouts of better product.

    This has been going on since I started in IT, and while it is more evident in enterprises, any company that lasts long enough will see these cycles run through.

    Agile and DevOps, as the Great Disruptors, stood a chance of redefining the tensions that cause these pendulum swings. Centralized versus decentralized is less important if stakeholders are all truly involved and informed—something that the culture side of the DevOps crowd aimed to achieve. Similarly, fast and quality were promised, as the ability to recover from inevitable mistakes would be rapid, while integrated testing would uncover most issues.

    We have enough years under our belts to know at this point that these issues are, at best, resolved with a resounding asterisk that is clarified sometimes.

    One nice thing about DevOps (more than agile) is that it has not stopped evolving yet, which means there is still hope that rapid delivery and high quality can both be answered while involvement of business owners and all the various IT groups will reduce the centralized/de-centralized cycle. But we’re not there yet.

    Split.io—purveyor of feature flagging/tagging tools—has a pretty good blog about what they discovered when they went out and talked to hundreds of shops. It shows that failures are happening at an alarming rate, and at some number of shops, for your average bug, they will still be fixing this released issue when the next release goes out. That is asking for trouble, in my experience.

    Better integrated testing is certainly part of the answer. TDD is good, but not enough, since it is more granular than some tests require. Split believes that feature flags are part of the answer, and I believe they’re right. But this approach is a band-aid to the problem.

    By the time you flip a feature flag to turn off a bad feature, you’ve already released a bad feature. And you will be spending time researching and fixing it. I believe feature flags should be used to catch it when this (inevitably) happens in a complex coding environment, but at the same time the source of issues should be looked into. An ounce of prevention is indeed worth a pound of cure.

    Figuring out what is allowing bugs to slip through into production rather than counting on remediating them (using precious time to do re-work) should be a priority moving forward. Is there a process/policy that is at the source? Is it simply a training issue? Training issues are repetitive—teaching today does nothing for new algorithms/tools put to use after today, and turnover can create a training gap if you’re not staying on top of it. For some shops, is speed of delivery faster than some piece (normally testing, but some shops suffer for development on certain platforms also) of the delivery chain can adequately respond to? Do you need to re-evaluate toolchain and delivery timelines?

    We are delivering more code, faster than ever, and that is exciting. But quality matters, and anyone who relies on end-users to test is playing a very dangerous game. Find ways to ensure quality without slowing down the process more than necessary. At this point, I honestly would implement runtime-flippable feature flags as a starting point, simply to be more responsive to issues. From there, work to resolve the underlying reasons that bugs are cropping up.

    Until then, keep cranking it. Keep working with the business to get quality systems in front of users, and enjoy doing some great work.

    — Don Macvittie

  • The Use of Feature Flags in Code: Too Many Flags?

    The Use of Feature Flags in Code: Too Many Flags?

    It is interesting to watch the growth of an idea over time. The idea of using flags in code to do different things is as old as programming itself. Old-school configuration files used them, and they worked, so they were adapted.

    Thing is, given an idea that works, humanity inevitably takes it too far, and flags to control software behavior seem to have reached that point. If you are using one set of flags for A/B testing, another to segregate code in development at runtime, another to turn functionality on for some users but not others and yet another set of flags to hide data based upon user…You’re buried in unnecessary code branches. If your organization isn’t great about cleaning up after flags are no longer needed, you are building spaghetti code that future developers will not thank you for.

    The best answer to this growing burden is focus. What is the best use of feature flags/toggles in your environment? They certainly speed the delivery process by having the code there, in nightly builds, just not executed until the flag is flipped. This helps in rollout also by allowing operators to turn a feature on when it is ready by changing config rather than redeploying the application.

    For A/B testing, in many cases it makes more sense to handle switching at a load balancer, where again it is a config, but separate instances serve the UI in question. This, like all decisions regarding flags, should be based upon the volume of change. Changing the contents of a list is likely easier to test with flags, but large volume of change is probably better run separately and switched off of a load balancer.

    The same is true with the volume of change in source. A rollout might be the best option for handling large volumes of change, and feature flags may simply create more complexity.

    As with everything, approach the problem with a plan that works best, in this instance, in your environment. There is no denying that the ability to turn on a new feature and then turn it off if something goes wrong is appealing, just use the idea judiciously. And clean up after yourself. No shop that uses flagging extensively will say “yeah, we don’t clean it up,” because the mess of nested checks for flags would slow the code and make it passingly difficult to even read, let alone maintain, over time. So approach it from the beginning with “It is a tool, and when it no longer serves its intended purpose (the feature is fully live or abandoned), we remove it.”

    And keep rocking it. You are the engine of business, and this is what you do. Make it great and keep it running.

    — Don Macvittie

  • DevOps: Get the Basics Right to Avoid Costly Mistakes

    DevOps: Get the Basics Right to Avoid Costly Mistakes

    It is a truth of both agile and DevOps that speed is of the essence. This is a relative statement—some apps/portfolios/orgs envision speed completely differently than others—but if you are releasing more frequently than you were in the past, thanks to DevOps, that’s kind of the point.

    Yet, we see a whole selection of things that fall by the wayside or are underserved. Security and testing are the easy/obvious ones. Considering the frequency with which we see “Unsecured Amazon storage bucket” (one announcement of this type spurred this blog), should we not see this folded into standard procedure or tools? Well, it is folded into standard tools for checking your cloud environment—I’ve used a couple of them and seen this warning come up. So shouldn’t organizations be using these tools more?

    One thing about speed: It tends to cut corners. And cut corners lead to lapses. You can be a blameless environment that doesn’t punish people for mistakes, but for an organization, a mistake such as exposing hundreds of thousands of users will be punished from the outside, from regulatory agencies and the market in general. They’re not mistakes we should be making, and yet it still happens.

    Another one is rollback versus roll forward. Most DevOps shops focus on the ability to fix an issue and plow forward. This works really well, 90+% of the time, and fails spectacularly sometimes. While it is an added step with some slowing of process, being able to roll back changes is important. And more than, “Well, theoretically, Git will allow us to go back to the last marked release …” because there is far more to an application architecture than just the source stored in a repository.

    In short, no matter how fast things are going, have a plan for when things go wrong. Because they inevitably will. And take the little bit of extra time for preventatives such as warning someone other than the creator about buckets with no security.

    “But there isn’t time …” is the common refrain to blogs such as this one. I get it. The increase in delivery only creates an increased appetite for delivery. But is there time to respond to disaster, should the worst happen? And I’d argue that it must be IT that plans for disasters. When someone gets fired because there was a security breach or a total systems failure, it is rarely an app owner. That puts the responsibility to make certain one doesn’t happen squarely on IT.

    We’re delivering faster, offering increased functionality to users, moving forward on every front. You all are out there rocking it daily. Don’t risk all of that being ignored because you failed to take care of basic issues. Check out the tools to help out; feature flags and cloud security scanners both spring to mind with regard to my examples above. The tools are out there, you just have to work them into your process.

    — Don Macvittie

  • 5 Best Practices for Feature Flagging

    5 Best Practices for Feature Flagging

    Feature flags, also known as the feature toggles, are important concepts in the field of DevOps. They have been around for a long time and have been used continually by developers for their convenience. As software development services providers are striving for higher agility, practices such as feature flagging are becoming increasingly relevant. If you are not familiar with the concept of a feature flag, let’s start with understanding what feature flagging is.

    What is Feature Flagging?

    Feature flags are used for the process of branching your code, which means to turn certain code paths on and off to separate the release of new features from the actual deployment of our code. Why do DevOps developers want to branch their code through feature flags? Well, feature flagging is a great software development practice for two reasons. Firstly, for features that are a work-in-progress, developers can use a feature flag to hold their release until they are ready. This ensures the release of other features that are ready is not delayed just because one feature is not. Secondly, features that are being worked on can be kept off using the feature flag and only once it is ready it can be turned on. This allows developers to work directly on the feature without changing the main code, also known as the trunk or master branch. When all is working and ready to be released, turning on the feature toggle releases the feature, just like a switch.

    To understand feature flags in the simplest term, feature flags are “if/else” statements to develop a conditional branch or pathway into your code. For example, you can use a feature flag to state that “if so and so feature is enabled, run new algorithm; or else, run old algorithm.” This allows developers to control the functionality—if the feature flag is enabled only then the new algorithm is executed; else skipped. This is the most basic form of feature flags, however, they could be more complex.

    Feature flags are a practical way for software teams to maintain the delivery and release of features, and there are many advantages of using them. However, too many feature flags can make your code more complex and difficult to manage.

    Here are the best practices of feature flagging that can save your project from becoming a feature flag hell.

    More Feature Flags Equals Complex Code

    Feature flagging is a good practice that can help you maintain the delivery and release of different features, however, using too many feature flags can quickly turn your code into a feature flag hell. The more feature flags you create, the more code paths are going to be created, this increases the complexity of your code and makes the testing process harder. A higher number of code paths can cause a mess and risk the functionality of your code. Therefore, it is best to only create feature flags when it makes sense and always check if a previously created feature flag can be re-used instead.

    Name Your Feature Flags

    For efficient management of feature flags, it is crucial to have a proper flag naming system. Without it, it is possible to confuse the flags, forget why they were created for, or create flags with the same name—all of which can cause disruptions and confusion. When working with a team of developers, it is important to ensure that everyone follows the same labeling method so they can get the information about the flag just by looking at the name. Otherwise, it is likely that a team member might mistakenly trigger the flags, upsetting your code.

    Log and Clean Up Old Flags

    Over time, it can become difficult to keep track of the changes that are made to a feature flag. Therefore, it is important to track changes through logging, especially when working in teams. Using logging, you can keep a track of who created the feature flag, who made a change and at what time. This can save you from wasting time figuring out who was responsible for what change. A log will set the record straight about every change introduced for every feature flag.

    This is also great to know which flag and code paths are obsolete and redundant now. When you know which flags are unused you can clean them from your code to avoid any conflict or confusion.

    Use a Feature Flag Management Tool

    For better management, control and visibility of your feature flags, you can use available feature flag management tools. Like every other management tool, feature flag management tools are enterprise-scale software to manage feature flags. Some of the most popular tools include LaunchDarkly, Rollout and Bullet Train. Tools like these can make the management of feature flags easier for you and increase your development speed.

    Use Feature Flags to Collect Data

    When it comes to scaling your product, feature flags can be of great help to collect data from users. For instance, if you want to introduce a new feature, you can roll out the change slowly to a few users first. Using feature flags, you can make the new feature or update available to a smaller set of users only and collect data about how well they respond to the release. You can then use their feedback to improve or change your features to deliver an improved user experience. Feature flags are used in this manner by companies such as Facebook and Netflix to test their new features before releasing them for the entire user base.

    — Emily Williamson

  • An Embarrassment of Riches

    An Embarrassment of Riches

    History says people rarely appreciate what they have in the heat of the moment. Our highly connected and high-speed world is no exception to this rule. The things that frustrate us are in our faces, while the things that make nearly everything better/safer/faster are taken for granted.

    That is every bit as true in IT as it is in life. We expect our deploys and releases to go easily, because life is far easier in that regard than it was even 10 years ago. We get frustrated when things don’t work out, because they so often do.

    The one area Agile has addressed is that most shops don’t beat the person responsible for an issue until they leave. That’s not to say no one is judgmental, it just isn’t official policy to brow-beat even stupid mistakes. The goal is to get better, not to bash the person responsible for the mistake. Most of the time this is a downright positive change. Particularly since, in complex systems, it is often not one individual’s error, but many.

    Before Agile, I rushed out a fix to a piece of software used by several large banks. They needed the fix, and I was most qualified to do it. I used the file handle for the printer to open a printer, and wrote to it. All went well in test and we shipped it out. The results were catastrophic corruption of one set of databases for several large banks.

    Unknown to me, someone had re-used the printer file handle if certain conditions were met, and opened it on the database. It being reserved for printers, that never occurred to me, and I simple reopened it and used it. The strange conditions that pointed it at the database were never encountered in test, and I was responsible for some serious damage to our reputation and the banks in question’s confidence in our software.

    Thankfully, our development team was already on “do it fast, don’t point fingers,” so it was mentioned that better testing could have been done, and we implemented plans to do just that, using scrubbed customer data. And the root cause–using a reserved file handle for access to something else–was fixed. My own angst as having created the mess made me sit with operations while they rebuilt the destroyed databases one at a time, using some astounding Unix command line wizardry. In the end, the software and the team were better off for the experience. And I got an early lesson in not trusting that code would behave as-expected.

    The point being, bashing me for a mistake that put us all on high-stress because our biggest customers were rightfully angry would not have solved the problem. Letting me work through, identify root causes with the test team and get things straight worked better.

    We’re facing a similar hill in DevOps today. Do you test in production? How much do you test in production? What if you have similar totally unforeseen issues crop up?

    The answer is you must test in production. Inevitably, there will be errors in production, it is far better to find them early and fix them than to test in production because the app is live and there were things you couldn’t test outside of production. Either way, you’re testing in production, the question is how much control do you have.

    This is one of the strong suits of DevOps. Using things such as feature flags, monitoring and AIOps, you can spin up a small percentage of production servers with changes. Or deploy parallel servers in production for low-server-count apps, then just stream a few customers to them to test out new changes. Yes, those customers might have a terrible experience if there is some unforeseen issue, but better a few customers than all of your customers if you rolled out without testing. For some industries, building a list of people willing to be used to test new features like this can be a marketing ploy.

    Our frustration at things going wrong will still be there, but the joy of catching those problems before major catastrophe hits our production servers should be there, too. Not long ago, you rolled it out to everyone and dealt with the consequences. Crashes were big and even more frustrating. Better to have a few customers impacted and the ability to fix issues before it’s an emergency.

    I’ve said it before, we work in astounding times with giant leaps forward in responsiveness. Take advantage of them. And take time away from making and running great software to appreciate them, if only for a moment.

    Then add more features and fix more bugs. We are in the heat of the moment after all.

    — Don Macvittie

  • The Relationship Between Feature Flags and DevOps

    The Relationship Between Feature Flags and DevOps

    From 2017 I’ve started most DevOps conversations with, “DevOps is now boring, and I love it!” What I mean by boring is that we are arguing less and less about why you should embrace DevOps principles and practices and having more and more conversations about how to implement DevOps in the real world. As a result, we are getting even more specific about various implementations. One of those is the use of feature flags to increase application velocity, ability to roll back and maybe even canary releases.

    Feature flags seem obvious, but are still a relatively new endeavor—and, for now, still pretty much isolated to the unicorn dev teams. Feature flags allow developers to logically group features and create a toggle for them that can be switched in testing and even prod. The benefits of feature flags go beyond just getting features out faster:

    1. Launching complex feature sets incrementally versus in bi-weekly sprints (fast waterfall). Sometimes features are more complex and connected to multiple services, back-end changes, etc. With modern microservices-based architectures where small dev teams operate on code bases that are independent, this can be difficult. By grouping all the various changes across the entire application as a feature, it means that teams can release functionality they have built without waiting for everyone else and only after everything is in place to turn on the feature.
    2. Recover from risky releases. Part of having feature flags is you can release functionality to production faster. But this also means that you can release functionality that has not been vetted fully across the entire application. With feature flags, an initial rollback does not need to be related to infrastructure at all; it can just be a matter of toggling a feature in production.
    3. Create a value-based information architecture for your application. Developers still are very guilty of thinking of the application as their code base and their immediate teams. But the application from the user’s perspective is based on value. Feature flags, assuming you use some basic best practices for determining your flag structure and names, can group features by value and thus better align everything in the delivery chain to the value the application provides.
    4. Quality (QA and Q)  teams have more power and can provide more value with feature flags. With feature flags, developers can throw more functionality over the fence to QE faster and enable them to have more control of functionality being tested and released without engaging with Dev.

    I would say the initial driver for feature flags has been for A/B testing and canary releases. However, the reality of A/B testing and canary releases don’t make sense for everyone because their application may not have enough users or transaction volume to be statistically accurate to see the value.

    But the value of feature flags goes well beyond that. The challenge is, if you are not a DevOps and automation-focused organization, these benefits are not going to be there. Feature flags are an outcome to DevOps, but my hope is they also are a driver for DevOps. Organizations are guilty of looking at DevOps from the process improvement perspective only. But the real business benefit of DevOps is the value you provide your application users. And if you have a value-first mindset, not only are feature flags an obvious design pattern but also they necessitate that organizations think about their delivery chain in a highly efficient and critical way. Put another way, if feature flags are an outcome you seek, then DevOps processes are not a choice.

    — Chris Riley

  • Split Software Adds Feature Monitoring to Feature Delivery Toolset

    Split Software Adds Feature Monitoring to Feature Delivery Toolset

    One of the issues that has long plagued both development and test is determining exactly where problem code exists. The increasing rate of change that agile and DevOps have brought to projects makes this problem worse than ever because changes today can impact code from yesterday–or yesteryear.

    The constant flux that modern software is in makes tracking down issues that much harder. This is offset a little bit by the fact that changes are generally smaller and more incremental because there are several iterations to get improvements into place and short sprints are the norm. But not completely offset, because a defect may just now be showing up, even if it was introduced several iterations ago.

    Feature Delivery tools offer the ability to group changes into features, then turn them on and off as needed. In the case of Split, they can also be phased in to see impact upon users. While the name uses “feature”, any code change like “feature enhancement” is included.

    The announcement pertains to new functionality called Feature Monitoring. This new addition to the toolset allows monitoring of features in production to show the DevOps team what is happening with their changes in real time. It also allows for advanced targeting of users so small groups can get the updates and evaluations made on how the changes impact those users in production.

    Feature monitoring captures error data, and feature tagging ties that data to features, meaning defect identification and resolution can both happen at a faster pace. Less looking where there isn’t a problem, and more information about how buggy a new release is.

    Given that I’ve recently written a fair amount about testing and the issues test DevOps team members face with rate of change, this seems custom-designed to address those issues. While there is no silver bullet, the faster issues can be identified, assigned and repaired, the better the overall application quality will be–and that is what this announcement is about.

    Not having used the tools yet, I can imagine some thought has to go into inter-related functionality, but that has always been true–you have to know what works as an overall system and can be placed together in a release versus what has to be held back if a given piece is incomplete. The way I envision Feature Delivery, and more specifically Feature Monitoring, being used would be smaller pieces than an entire release. Even if a new feature is going live, someone knows what inter-related pieces must be complete, so simply shifting that knowledge left will allow effective feature tagging.

    Finally, the ability to turn updates on and off by changing flags means less downtime, and if planning is done well, the virtual or complete elimination of rollback plans. Agile and DevOps talk of fix and re-release, but we’ve all seen scenarios where that simply isn’t an option, and this may be the solution we’ve been looking for–rollback by changing a flag that changes what code is executed. Flag-checking occurs in code with if-then’s though, so after many iterations code clean-up, it would likely need to be done just to keep the code from being overburdened with nested logic for iterations that are known to be stable.

    I think I’ll add this to my list of markets/products to dig deeper into and actually put to the test. We have a set of servers sitting idle at the moment, and a mobile app we use for testing, perhaps we’ll take some time to test out flagging and nesting.

    In the meantime, it is definitely worth a look. Anything that eases test burden and might remove the need for rollback planning is worth a look. Meanwhile, keep kicking out the tools that run business.

    — Don Macvittie

  • How Product Managers Can Take Back Control with Feature Flags

    How Product Managers Can Take Back Control with Feature Flags

    Product managers live in a high-speed world and are always focused on releasing the most impactful features in the shortest amount of time. Since users have grown accustomed to regular UI updates and bug fixes completed in near real time, this frenetic pace at which product managers work will only increase.

    The process of pushing out new features or software fixes can be very stressful on product managers, and the one thing they crave to keep their sanity and ensure a successful release is control. Luckily, that’s exactly what feature flags provide.

    Dialing Up Feature Flags

    At their core, feature flags give development teams the ability to turn features on and off without deploying new code. Think of them as dials in the cloud. When the dial is attached to any feature in a product–whether a new one or an old one–it controls who sees it, who has access to it and how quickly it is delivered to an entire customer base.

    Feature flags let organizations break the tyranny of the Thursday night release by segmenting it into smaller bits to control exposure and enable a continuous release motion. You might start by turning on a feature to only internal users or beta testers so you can test in production to make sure nothing has been missed. From there, you have the freedom to ramp up the feature to your customers gradually before going to 100% availability.

    With this controlled exposure, development teams can lessen the panic and anxiety around the weekly release schedule. Having the ability to completely turn off features means feature flags also increase application stability and remove the need for hotfixes. It’s less often necessary to assemble a war room if there is a problem when you can simply turn off the feature in seconds. By working feature flags into the release process, teams take control of the who, what and when in terms of releases. They can also dictate types of users, demographics, locations and scale the release of the new feature.

    Feature Flags and Product Managers

    Outside of the DevOps team, feature flags can also be really powerful for product managers. Historically, the release of a new feature meant product managers had to coordinate with cross-functional teams to ensure the release was executed successfully. This meant making sure marketing was ready to publish a blog post about the feature, the sales team knew how to use and sell the new feature, and the product marketing team had documentation prepared for how the feature set works. Of course, all of this had to be perfectly synchronized with the engineers’ timeline for releasing the feature.

    With feature flags, product managers now essentially have the keys to the kingdom. It gives them the power to turn the feature on for some or all of their customers without waiting for engineers to complete their next deploy. Not having to coordinate that extra piece of the schedule gives a lot of power and time back to the product manager.

    A Deeper Look at Customer Feedback

    Another aspect of a product manager’s job is collecting feedback. They need to be able to answer questions related to business KPIs, such as “How is this feature performing for my customers?,” “Are customers able to do the action the feature is intended to perform?” and “Are we generating more revenue from the feature?” Feature flags can also help in this area.

    As product managers ramp up a feature, they are simultaneously shortening the feedback cycle. Since feature flags allow for segmentation, where one customer group sees one set of features and another customer group sees a different set of features, and since key metrics can be captured into separate buckets for each group, product managers can now run controlled experiments as part of the release cycle. Seeing KPIs aligned with the different customer groups helps them understand which features are performing the most successfully.

    Creating the Ideal Product Roadmap

    As product managers embrace usage measurement tied to feature rollouts, entire teams can better align with agile best practices to continuously push out the smallest incremental set of new features. By understanding the performance of every feature, you’ll be better equipped to iterate and refine your ideal product roadmap.

    — Dave Karow

  • CloudBees’ Acquisition of Rollout: The CEOs’ View

    CloudBees’ Acquisition of Rollout: The CEOs’ View

    As Mike Vizard reported yesterday, CloudBees, as part of its mission to build “the world’s first end-to-end automated software delivery system,” announced another acquisition: Rollout, a developer of a feature flag application that allows for simplified testing of new modules within an application.

    Not to repeat the gist of Vizard’s article; you can read it for yourself. But this continues what has become something of an acquisition tear by CloudBees. The company has acquired CodeShip and Electric Cloud, among others, while becoming one of the founding members of the new Continuous Delivery Foundation (CDF), under the Linux Foundation and a sister foundation to the Cloud Native Computing Foundation (CNCF) and the Cloud Foundry Foundation. The CDF is now managing the Jenkins project, as well as Jenkins X, Spinnaker and Tekton.

    CloudBees’ Mission

    Make no mistake: These acquisitions, along with the CDF formation, represent a major change for CloudBees. The team there is clearly trying to separate from the pack of “end-to-end” DevOps contenders. CloudBees CEO Sacha Labourey is pushing hard to raise CloudBees above the noise and the rest of the market. The company announced a $62 million round of financing about a year ago.

    You don’t have to be Warren Buffet to see that CloudBees sees an opportunity here. The company is driving toward achieving scale in product and revenue on the path to some sort of liquidity event (perhaps an IPO?) or market leading opportunity.

    For now, though, back to the Rollout acquisition. I had the chance to sit down with Labourey and the CEO and founder of Rollout, Erez Rusovsky. We discussed what Rollout does, its history and how it will fit into the CloudBees product portfolio.

    It is always great to interview a founder about his company being acquired. What a tremendous feeling of accomplishment to found and build a company, and have another entity value it so much that you are willing to sell it to them. This goes so far beyond validation.

    Congrats to Erez and the whole Rollout team and, of course, congratulations to Sacha, KK and the rest of the CloudBees team. This seems like a perfect fit.

    In the meantime, here is my conversation with Sacha and Erez on the acquisition and more. Enjoy!

     

    — Alan Shimel