Let’s face it, debugging is broken.
Whether you wholeheartedly agree or vehemently disagree with that statement, I think we can all agree that debugging, at the very least, has room for improvement. In fact, the way software developers debug their code hasn’t changed much. Of course, there are now a wide array of observability and monitoring tools available to make a developer’s life easier, but when we look at the process developers typically follow when debugging code, it’s stayed relatively unchanged over the years.
Having the right tools available definitely improved the overall debugging experience, but they’re not the only factor. Internal organizational processes are equally as important, and should be adapted to make developers’ lives as smooth and stress-free as possible while still maintaining the proper levels of organizational security and compliance. Inefficient or highly restrictive processes, while great for the compliance team, can adversely affect the velocity of getting to the bottom of issues quickly. The right solution is typically a combination of adapting those processes to fit the company culture, along with a combination of tools that allow for compliance with industry standards and best practices.
How Developers Debug Code
When you talk to developers today about how they debug their applications, the process is typically not all that different than it was many years ago. The standard approach to debugging is still through the use of logging frameworks. Of course, teams are taking advantage of things like application performance management (APM) tools and other tools in the monitoring and observability space, but when you look at the primary method of debugging issues that require developers to dive into the code, logging is typically the main approach.
The process typically looks something like this:
When a defect is found, developers will first go to the log files on the production or pre-production servers in the environment where they are debugging. If they can’t find information in the logs to help them get to the root cause of the issue, they go back to their code and add additional log lines, finally sending their code through the process shown above. For highly advanced organizations, the process can take minutes or hours, but more often than not, this process drags on for days – or even weeks. It becomes especially time-consuming when the defect developers are trying to find and remediate is in production. In these cases, the cost to the business can be staggering; there’s got to be a better and more efficient approach to debugging.
Where Tools Can Help
While new and shiny tools don’t always solve all debugging problems, they can often ease some of the pain developers face and, in the process, make their lives a little bit easier. Debugging, observability and monitoring tools can get developers the data they need to solve problems faster than the more traditional method of digging through logs and using trial and error.
The place many organizations start when considering diagnostic tools is with APM. These types of tools focus specifically on monitoring and management of the application availability, and can help to collect metrics and other relevant application information that can aid in pinpointing infrastructure issues. Most often, these tools are not actually used by developers themselves, but rather operations or production support teams to ensure availability and performance thresholds of applications.
Another critical category of telemetry are tracing tools, which can help developers better understand how requests flow through multiple tiers of applications and even help identify error rates and latency. There are many vendors selling this category of tools, but there are also open source options which allow developers to integrate tracing into their applications via a vendor-neutral approach.
After diagnosing an issue with an APM solution, there is often a handoff to development to remediate the issue. When developers need to debug, they often use their local IDEs or, even better, a remote debugging solution. Remote debuggers are used directly by developers to simplify debugging of applications in remote environments. These tools helps developers get to the bottom of customer issues faster by giving them direct access to the underlying application data in a safe and secure way while the application is running without stopping it or having to redeploy it. A major benefit of remote debuggers is that they can give developers the data they need from production environments without compromising security or requiring developers to have direct access to sensitive environments. While remote debuggers and other debugging tools are not a silver bullet, they can be a critical component that empowers developers to better address critical issues.
Fixing Organizational Problems
If solving customer issues is taking longer than expected or is potentially hurting your brand, it’s critical to sit down with your development team and use a data-driven approach to identify where the problems lie. Do a deep dive into your issue tracking or ITSM system to identify trends or recurring issues within particular application components. Development teams can often become so overwhelmed tackling issues and building out new features that valuable information in these internal systems can go untapped. Customer issues and defects should be reviewed periodically to assess things like business impact, cost to fix and alignment with the product roadmap. There is valuable information that can help focus the team’s effort on time-saving activities as well as improving the product, making for happier customers.
Another area to look at is your software delivery process. Are there bottlenecks in your delivery pipeline that could be made easier or more efficient but have been overlooked? The speed and efficiency of your software delivery pipeline has a direct impact on developers’ ability to debug effectively and can contribute to a broken or slow debugging process. Developers should have the ability to push code into new environments as quickly as possible to remove slowdowns from too much context switching. This means automating as much of the delivery pipeline as possible, removing unnecessary manual deployment approvals and shifting left as many activities as possible.
When it comes to empowering developers and fixing broken debugging processes, organizations should focus on fixing organizational bottlenecks as well as equipping developers with tools that can save them time and improve their daily workflows. Focusing on these areas can increase development team velocity and eliminate costly day-to-day slowdowns which add up over time and negatively impact productivity. With a few key changes, debugging can be a much more manageable and predictable process.
