devopsy.kinsta.cloud

Tag: agile metrics

  • Nobl9 Shares SLO-as-Code Methodology

    Nobl9 Shares SLO-as-Code Methodology

    Nobl9 has released version 1.0 of an open specification for defining service level objectives, dubbed OpenSLO, and, in addition, has defined a repeatable SLO methodology.

    Kit Merker, Nobl9 COO, said the Service Level Objective Development Lifecycle (SLODC) methodology represents an effort to create a set of best practices for achieving and maintaining SLOs that are implemented as code within an application environment.

    A survey of more than 300 IT managers and executives conducted by Dimensional Research on behalf of Nobl9 found only 29% of respondents had no plans to implement SLOs. A full 94% of respondents that have or plan to implement SLOs intend to map them directly to business operations, with 91% reporting they expect that effort to improve decision-making. More than 80% also said their organizations are planning to increase the use of SLOs, with 87% indicating SLOs should improve overall microservices performance.

    The challenge is that most organizations have limited visibility into their IT environments. The survey, for example, found less than half of respondents (46%) had visibility into all their IT environments. Only 45% and 35% claim to have visibility into containers and microservices, respectively. More than three-quarters (78%) said hybrid clouds make observability more difficult.

    Ironically, 45% of companies reported they already employed 11 or more observability and monitoring tools. On the plus side, 31% have hired site reliability engineers (SREs), while nearly half (46%) planned to create that role.

    Nobl9 is trying to spur greater adoption of SLOs by making available an open source SLO specification that defines a common interface for constructing SLOs across a Git-based workflow. Since OpenSLO was initially launched, more capabilities have been added including a DataSource object that makes it easier to reuse connection details and which makes creating SLOs less verbose and three alerts have been defined: AlertCondition, AlertPolicy and AlertNotifcationTarget.

    Nobl9 is also releasing an OpenSLO-to-Nobl9 converter to transition OpenSLO YAML files into Nobl9 YAML when required.

    SLOs, of course, are not a new idea. They have been employed as a metric to track the performance of IT services for decades. However, as more microservices-based applications are built and deployed, it’s becoming more challenging to maintain SLOs across applications that have many more dependencies than legacy monolithic applications. Ultimately, each IT team needs to provide some sort of objective benchmark that assesses their overall effectiveness at delivering application services. SLO-as-code is intended to make it simpler to gather the metrics that confirm whether service levels are being achieved. Those SLOs are then tracked and integrated across everything from financial operations to supply chains, noted Mercer.

    It’s not clear to whether reliance on SLOs has waned over the years or if application environments simply became too complex to track meaningful metrics. However, as applications are increasingly viewed as services, it’s now only a matter of time before SLOs become more widely adopted across a modern application environment. The challenge, of course, is that defining an SLO is a lot easier than maintaining it—especially within today’s highly dynamic application environments where services tend to come, change and eventually go unexpectedly.

  • How To Use Metrics To Measure The Success Of Your Agile Projects

    How To Use Metrics To Measure The Success Of Your Agile Projects

    Ah, metrics. A company’s best friend. Without them, a business can’t be certain it’s on track with deadlines. It doesn’t know if it’s operating efficiently, or even if the last organizational change had any impact at all. According to serial Entrepreneur David Skok, “Great companies are almost always run by great management teams. And great management teams know that the only way to improve a process is to start by measuring it.”

    However, choosing great metrics isn’t always easy. When simultaneously managing multiple agile projects and incorporating new agile initiatives, it can become difficult to measure the impact your changes are making, or if your team is actually improving.

    So what’s a company to do? To gain an accurate picture of what is actually happening, it’s imperative businesses use a combination of both business and agile metrics to keep track of projects. Business metrics should help measure the success of your agile initiatives, and agile metrics will allow you to continuously improve upon processes and deliver exceptional results to clients—which should be the entire mission of an agile approach.

    But which metrics are most relevant for agile teams?

    Choosing the Right Agile Metrics

    Deciding which metrics to incorporate into your business takes plenty of thought. Most importantly, it’s key to select metrics you will actually use. If you don’t imagine you’ll have the resources to measure niche data sets, for example, it’s a waste of time to carve up ambitious metric tracking schemes.

    Our experience has lead us to focus on three key areas: quality, performance and budget. We choose these categories because they offer a robust picture of the team, the company and the client side. Additionally, these will help you to determine whether projects are running on time and/or if you have enough resources to fulfill them.

    It can be daunting to start a project with a whole list of in-depth metrics. Generally what we do is begin each project with a light set of metrics that allow us to monitor the project efficiently without taking away time or energy from the actual project itself. Then, we adjust or add metrics based on the particulars of that project and the needs of the client. Usually, this means defining the project road map with the number of sprints to be completed distributed in blocks or releases. This clearly outlines the time to market of the MVP and the testing needs of the client. From this point, we derive other necessary metrics such as initial budget definition based on high-level comparative estimates and the known speed of the teams working on the project.

    After the first sprint has been completed, engineering metrics such as defect injection rate, testing automation coverage and technical debt analysis can be added based on needs. It goes without saying that all metrics are oriented to exceed client expectations and maintain product quality.

    After establishing general categories, you can begin to use more specific metrics to measure and improve agile projects or products. It’s a great idea to turn to the 12 agile principles for inspiration—these speak to the importance of continuous delivery of impactful software, simplicity and changing behaviors to increase efficiency, among other values. Based on this list, companies may want to consider the core metrics of velocity, sprint burndown and release burndown chart. Obviously there are others that might work for your organization; we recommend selecting at least two metrics to accurately gauge agile projects.

    Tracking velocity can help measure a particular team’s output throughout each sprint, and whether management decisions are stimulating or hindering this output. There are no baselines with velocity, because velocity is particular to each team. If you have a team of five people working on a Java project, the velocity for that team is completely different than three people working with .Net.

    After the first sprint is completed by your team, you can understand the amount of features that can be completed by the team for that project. However, don’t get too caught up with only increasing your velocity. There are other important metrics that must also be used to measure the success and continuous improvement of agile initiatives.

    A sprint burndown is another great way to measure output. With this metric, leaders can see the rate at which his/her team finishes the sprint and create a chart to clearly illustrate where they are and if they are on track.

    A release burndown chart is a great measure of the overall project. This entails creating a chart to highlight which features have been completed and which are yet to be completed. The chart is updated at the end of each sprint by the Scrum Master.

    To further breakdown these metrics—data permitting—productivity can be measured member by member. This entails measuring the velocity of each member to give a better picture of the overall working speed, and provide estimates for the project.

    Choosing the Right Business Metrics

    Whilst agile metrics help measure aspects of the development process and the objectives of your partnership with your client, they do not shed much immediate light on whether you are meeting your overall business goals and objectives. This is where business metrics come in. Keep in mind that correct business metrics should depend on company goals, objectives and the input of your team.

    Software veteran Joel Trammell recommends that all metrics have five key characteristics: They are easily measurable, directly correlated to business performance, predictive of future business performance, isolated to factors controlled by the group it is measuring, and comparable to metrics of competitors.

    Some businesses find it important to focus on performance and project management metrics with clients. These can help reveal efficiency and accuracy of planning, showing how many times a business hit the specified target or improved client relations. We have a number of key performance indicators (KPIs) that are maintained by the company overall.

    When managing budgets, project management metrics are useful, too. To calculate the remaining amount of budget, look at the backlog and what you have already created. The first goal is to complete the initial components or objectives with the client. After you have completed your initial minimum viable product, you can start adding more features to the product based on what’s left in your budget and what you know about the efficiency of your team.

    Other Success Indicators to Take into Account

    Although metrics are important, they should not be the only indicators used to measure performance and success. Alongside quantifiable data are intangible elements such as employee satisfaction, company culture and team motivation. Really, these elements can make or break a project; if your staff is not being challenged or feels burnt out, the project is more likely to fail.

    Measuring these factors is another process altogether; no set of numbers can tell you if your company has a healthy culture or if it is rotten to the core. This requires both instinct and communication.

    Based on our own practices, we recommend conducting a retrospective assessment after every project, including the whole team if possible. This allows time to gather suggestions from every team member, as well as concerns, best practices and areas for improvement. Again, this information is not quantifiable, but is extremely valuable in improving and developing business processes, working relationships and company culture.

    From the information obtained in the assessment, an action plan can be drafted up, potentially implementing a new helpful technology or working to solve an existing problem.

    A great way to carry out this assessment is on a “360” basis, whereby all the team members evaluate a person trying to discover his or her improvement areas. As with any agile initiative, this should be a core component of your strategy.

    Also, regular meetings—every two to three months—between the Scrum Master and each team member can facilitate this process. They can review a “Professional Development Plan”—a format particular to the company where the employee records his expectations and needs. Together, the team member and the Scrum Master can then come up with an action plan for growth.

    Conclusion

    Simply put, metrics are essential to keep track of businesses and teams. Make sure to only choose metrics that are highly applicable, and to resist the temptation of measuring too many—after all, biting off more metrics than you can chew will take away the focus from your agile strengths.

    — John A. Jaramillo