devopsy.kinsta.cloud

Tag: visibility

  • How Visibility Helps Devs Focus on Innovation

    How Visibility Helps Devs Focus on Innovation

    As software engineering leaders adjust to a future defined by remote and hybrid work models, they are shifting priorities in the organization of work to compensate for the lack of in-person collaboration. According to a recent study conducted by Jellyfish, the top priority for engineering leaders is ensuring that each member of their team is focused on the highest-priority work for as much time as possible. To achieve this goal, these leaders are using real-time visibility and data-driven decision-making to guide more effective management.

    Earlier this year, Jellyfish analyzed the work of more than 23,000 engineers and surveyed hundreds of leaders to better understand the current state of engineering management. The research revealed that engineering leaders are shifting resources away from reactive work—such as responding to customer-reported bugs—to the type of work that directly impacts top-line revenue. Additionally, engineering teams are emphasizing growth and innovation, developing the type of infrastructure that will enable their teams and their software to scale long-term.

    Prioritizing Visibility

    To shift from reactive to proactive work, engineering leaders must first have visibility into what their teams are doing. How can you allocate your time and resources more effectively if you don’t know how they’re being allocated today? Before work can be prioritized effectively, leaders must understand how much bandwidth their teams have to take on new projects. Additionally, they need data-backed insights in order to align the priorities of the engineering team with those of the company at large.

    Visibility is also key to avoiding work that wasn’t part of the plan. According to the study, teams spent 22% of their time on unplanned work in 2021, a 15% increase year-over-year. This trend indicates that engineering leaders may not realize when certain types of work are taking attention away from mission-critical initiatives. When asked to name their biggest challenge heading into 2022, 45% of leaders (by far the largest group) selected “making sure everyone is focused on the highest priority work.”

    To gain visibility into their operations, some engineering teams are doubling down on data aggregation and analysis. According to the report, while all organizations tend to devote similar amounts of time to “keeping the lights on” and customer support activities, those that invest in a more robust data organization are able to spend 29% more time on innovation and 48% less time on unplanned work.

    Building Through Data and Metrics

    The report also found that engineering leaders who’ve embraced data-driven decision-making are seeing more productive, healthy and focused teams. These organizations tend to release code more quickly and foster higher collaboration than their peers. Conversely, organizations that don’t prioritize data-driven decision-making are more likely to experience burnout, decreased productivity and less focus on innovative work.

     

    Here again, the difference between embracing data and ignoring data comes down to technology. Aggregating data and metrics manually can be tedious and challenging. However, emerging tools are able to automatically collect, measure and analyze data to offer insights on what teams are prioritizing. As a result, leaders can more easily align engineering work with business priorities, ensure more predictable development cycles, increase financial responsibility and continuously improve operations.

    For every engineering leader, nailing this approach to data will be essential for maintaining efficiency as teams expand. More than 79% of engineering leaders expect to grow the size of their teams in 2022, and respondents named employee onboarding ramp time, virtual onboarding and hiring among the most significant challenges they face.

    Despite the numerous challenges engineering teams have endured in the past two years, the industry has adapted and become more collaborative as remote and hybrid work models multiply. For leaders who seek to increase data-driven decision-making, new tools and approaches will result in more intentional engineering work and improved visibility and operations.

  • Bridging Departmental Divides for Successful Automation Projects

    Bridging Departmental Divides for Successful Automation Projects

    For too long, IT has been largely isolated from an organization’s core business. Any division between IT and lines-of-business (LOB) is even more detrimental when speed is not just a competitive advantage but an existential necessity in today’s accelerating digital economy. 

    As business leaders streamline and turbocharge every facet of their operations, savvy organizations are rushing to eliminate inefficient and damaging gaps between departments by putting more software tools in the hands of more employees. Analysts predict 80% of technology products and services will be built by those who are not technology professionals by trade by 2024. 

    This hits home amidst the booming trend toward robotic process automation (RPA), low-code/no-code and other automated solutions, which have only continued to grow as more businesses realize the vast benefits of automation: Accelerating operations while also reducing costs; elevating customer experiences and shifting employees to higher-value positions and laying the foundation for new data-driven revenue streams.

    Squandering Automation’s Advantages

    But these advantages are quickly squandered if there’s a turf battle between the business unit looking to optimize processes and the IT department responsible for deploying the tools to make that happen.

    When there isn’t collaboration and plenty of trust on both sides, automation initiatives are more prone to disappointments and delays. 

    This may happen if the IT department comes across as too smart for their own good, putting less value on feedback from their LOB colleagues and rushing to implement software with a focus on speed rather than outcomes. On the LOB side, leaders may assume their IT counterparts are too technical and don’t understand business objectives and choose to override important IT input about which vendors are best qualified when it comes to support, integration and security.

    The problem here is not the technology; hundreds of thousands of businesses across the world have successfully deployed process automation solutions. It’s the execution.

    When an automation initiative fails, there’s almost always blame to be shared on both sides of any divide. Likewise, success typically boils down to people across IT and every business department working better together to achieve a common goal. 

    To embrace a collaborative mindset and set up your automation projects for success from day one, I encourage you to follow these clear steps:

    1. Reinforce a shared mission: Whether it’s automating the customer onboarding process or deploying RPA bots to automate a repetitive, time-consuming process, communicating a shared mission is the crucial first step to pulling off a successful project. On the IT side, that involves talking with the LOB about their goals, objectives and business strategy. LOB leaders, meanwhile, can emphasize that IT is an equal business partner that has valuable insights and expertise to contribute.
    1. Establish visibility and transparency: Armed with a full understanding of business objectives, IT leaders can now evaluate tools built to give all stakeholders a full view of the processes that will be affected. By documenting and mapping out each process within an organization, leaders can better understand current processes and gain insight into areas that are ripe for standardization and automation. 
    1. Tailor solutions that match the problem: With heightened visibility across the organization, LOB and IT teams can then determine the scope of each problem and tailor solutions accordingly. Some processes can be kept and optimized; some can be tweaked with automation. Other processes may need to be reconfigured significantly before automating or shut down entirely because they are outdated or unnecessary. 

    A great example of this collaborative approach comes from Coca-Cola Beverages Florida, LLC (Coke Florida): Encumbered with a legacy process repository — a collection of PDFs, videos, pictures and spreadsheets outlining more than 600 processes — Coke Florida standardized and leveraged visual process mapping software to establish an easily accessible document repository that provides visibility and control over key processes, encourages collaboration and increases accountability. 

    Coke Florida continued an organization-wide initiative to digitize and automate its business processes with digital workflows, forms and RPA software, driving further efficiencies across its entire organization. As the company discovered firsthand, shared goals result in shared success. 

    When speed and high-quality outcomes become the norm, companies master the essential arts of urgency, resilience, visibility and collaboration. By bridging the divide between departments, organizations not only drive internal efficiencies but also gain a competitive advantage that is indispensable in today’s burgeoning digital economy. 

  • The Automated Data Center Emerges

    The Automated Data Center Emerges

    Hyperscale and automation seem to go hand-in-hand, at least when it comes to the data center. Synergy Research reported that, at the end of 2017, there were 390 web-scale data centers worldwide, up from 300 a year earlier. IDC identified that hyperscale data centers “have a minimum of 5,000 servers and are at least 10,000 sq ft in size but generally much larger.”

    Today, hyperscale data centers are driving a sense of urgency for automation, but many smaller corporate data centers are just as complex, even if they are less physically expansive. In either case, automation takes data from across the data center, applies analytics and machine learning to that data and helps to identify both inefficiencies and trouble spots in the data center–as well as their origins. 

    Some data center functions are well on their way to becoming automated, making new deployments and changes to existing deployments simpler, faster and more efficient than they are today. Among those functions, orchestration, change management, software-defined architectures and DevOps stand out. 

    Meanwhile, another data center function is lagging: problem resolution. The challenge revolves around the siloed nature of the data center, where the multiple, disparate tools and methods used to sleuth out inefficiencies and pinpoint trouble spots lack a unified view of the data center infrastructure. In addition, silo-centric infrastructure monitoring tools lack application context. Short of automation, today’s data centers are stuck in the past. Especially as a data center grows and must interact with cloud-based services, no amount of human intuition in using those tools and methods can overcome the barriers of siloed systems. 

    However, data center infrastructure planners are making strides in identifying the requirements for accelerating the move to the automated data center. 

    Three Keys to the Automated Data Center

    For years, the automated data center was just a dream. And as recently as late 2018, Forrester spoke of islands of automation.

    “For the enterprises I’ve talked to, it’s not like any of them are completely manual at this stage, but they’re all at varying degrees of trying to integrate all these pieces,” said Chris Gardner, Forrester senior analyst. He added that fragmentation was due to a lack of holistic solutions that could connect all aspects of the data center.

    But now, three forces are shaping up to help make the automated data center a reality. Let’s look further. 

    Real-Time Visibility 

    Real-time infrastructure visibility has become a critical element of automating the data center. Why? Because an issue can occur anywhere on the virtualized, hybrid infrastructure, and the issue can ripple to create problems elsewhere, making issue resolution a perennial challenge. 

    For that reason, your best defense is to build deep, real-time visibility into the compute/virtualization stack and the storage/storage network stack. This will help you quickly ensure the performance of mission-critical applications accessing the data from those stacks.

    For ITOps, it is about proactively pinpointing the source of the issue and resolving it before application users are impacted. That requires stitching data across the full stack of technologies supporting the business: application services and infrastructure services, both on-premises and in the cloud. 

    Real-Time Analytics

    Why is real-time analytics crucial in the data center? In short, it’s because real-time responsiveness is critical for supporting anomaly detection, root cause analysis, remediation, prevention and planning use cases, where processing and correlating massive amounts of information is critical.

    For ITOps, it means distributed real-time architectures must combine cloud-based and on-premises analytics that leverage AI and machine learning. It’s the analytics and correlation that helps you make sense of all that real-time data so it becomes actionable.

    Closed-Loop Systems

    IT organizations are acutely aware of the perils of introducing changes to critical production environments. In fact, they take great pains to establish a governance of change. Closing the loop is about humans delegating decision-making to technology.

    For ITOps, that means ensuring integrated control of change processes across the entire technology stack. Data and analytics should lead to optimal real-time decision-making that empowers the automation for closed-loop operation.

    How Automated Is Automated?

    In 2016, researchers at the University of Pisa argued that hyperscale data centers would have to rely completely on analytical, automated systems to have to any chance to manage the huge number of day-to-day issues their servers generate. They added that “large computer systems in general, and data centers in particular, will ultimately be managed using predictive computational and executable models obtained through data-science tools, and at that point, the intervention of humans will be limited to setting high-level goals and policies rather than performing low-level operations.”

    Now, more than three years later, critical technologies and methodologies are in place to make the automated data center a reality, even if humans are still required for guidance, if not for sleuthing out infrastructure issues.

    — Len Rosenthal

  • Rethinking the APM Paradigm: A New Vision for Enterprise Application Monitoring

    Rethinking the APM Paradigm: A New Vision for Enterprise Application Monitoring

    Traditional APM systems today can’t measure up to tomorrow’s processes

    IT today is inordinately complex; continually pushing the boundaries and comprehension for even the most sophisticated minds. And while modern IT is unlocking new, innovative experiences and capabilities, the advent of cloud computing and its distributed, ephemeral nature has also made it extremely difficult for IT managers to keep apprised of their infrastructure stacks.

    This reality has taken shape so swiftly and with such a vengeance that Gartner estimates by 2021, enterprises will be able to monitor only 20 percent of their applications accurately. In a digital world, that is like a department head saying he or she knows what happens in only 20 percent of your factories or storefronts. Twenty percent, regardless of the industry or circumstance, is both unacceptable and a grave risk to business continuity and success.

    APM Helps, But Not Much

    Enterprises need to know what the other 80 percent is doing. Unfortunately, the traditional application performance management (APM) tools on which enterprises typically rely are no longer up to this task for three reasons.

    First, the modern application suite looks very different today than it did just a short time ago. The monolithic and custom ERPs, CRMs and other enterprise systems that were the hallmark of big businesses a decade ago are giving way to cloud-based, off-the-shelf, as-a-service solutions constantly updated with iterative DevOps and agile development techniques.

    Second, most traditional APM tools rely on embedded monitoring agents installed on the monitored system, or they need code-level insertions tactics to monitor. Either way, that type of burden is unsustainable at today’s enterprise scale and doesn’t work when applications are launched in containerized cloud environments. It also fails to account for the shift to more off-the-shelf or as-a-service solutions created without APM data outputs.

    Finally, the very infrastructure enterprise operates on looks nothing like it did even five years ago. Most enterprises have moved (or are in the process of moving) to cloud, multi-cloud, or at least hybrid infrastructure. The advent of cloud and the ephemeral nature of microservices and containers means that application topology looks very different at 8 a.m. when only early-bird employees are in the office, then it does at 10 a.m. when the remaining employees arrive. That unpredictability is a nightmare for APM tools, which, at best, now offer only a keyhole view of the entire IT enterprise; performance management now relies as much on the infrastructure as it does the application.

    So, What Needs to Change?

    Traditional APM tools are battleships in an asymmetric warfare age: highly specialized and designed for a single mission, but at the cost of the flexibility to adapt to modern environments as needed. We need carrier groups, not battleship armadas.

    The ideal APM solution for modern enterprises is an “agent-less” or “agent everywhere” approach. A modern solution must holistically consider how one application interacts with other applications. For instance, how does online ordering generate data and feed into warehouse processing tickets? And it must understand how those applications interact with the underlying infrastructure. For example, where are databases or applications stored and how reliable is that infrastructure?

    Current APM solutions miss the mark entirely, scratching only the most superficial layer of today’s technology stack when instead, they need to understand the relations from the deepest layers of infrastructure out to the end users’ experiences. Such an approach will allow enterprises to see into the other 80 percent of applications and deliver the insight and automation needed to drive robust and innovative digital experiences.

    However, knowing you need visibility and having it are two very different things. Fortunately, innovation within AIOps through agent and agentless technologies promises to deliver what conventional APM can no longer.

    By making visibility into application layers easier and deploying machine learning-enabled topological mapping between infrastructure and applications, AIOps is allowing for more detailed insights into the health of mission-critical business services than ever before. It allows enterprises to regain the visibility they’ve lost to an ephemeral world and be able to understand, organize and innovate with the agility they need to compete in today’s business environments.

    It is time to leave traditional APM behind and move forward into an AIOps enabled world. Neglecting to do so puts the business at ever-greater risk.

    — Erik Rudin

  • LogicMonitor Extends Reach to the Cloud

    LogicMonitor Extends Reach to the Cloud

    LogicMonitor has been making available performance monitoring tools as software-as-a-service (SaaS) applications for several years now. At the Microsoft Ignite conference this week, the company announced it is adding a module through which IT organizations can gain similar insights into public cloud computing services.

    Steve Francis, founder and chief evangelist for LogicMonitor, says LM Cloud is designed to provide real-time visibility into the performance of Amazon Web Services (AWS) and Microsoft Azure clouds, including resource performance monitoring, detailed billing analysis and cloud provider availability.

    Many IT organizations don’t realize just how different managing a public cloud is compared to an on-premises IT infrastructure environment, he says. Not only is too easy to let a workload waste resources, spiraling cloud service costs out of control, but also cloud service providers have policies that limit consumption of resources in a specific region.

    Capabilities provided by LM Cloud include automated discovery of an organization’s entire cloud environment and dashboards that can be employed to troubleshoot issues.

    Francis says that as an extension of the core LogicMonitor service, LM Cloud provides a means for IT organizations to employ one monitoring tool across what have become de facto hybrid cloud computing environments. In fact, using multiple clouds will require IT organizations to rely more on monitoring tools based in the cloud. Otherwise, Francis notes, IT organization would have to correlate data across multiple incompatible tools.

    One of the things that holds IT organizations back from using public clouds is simply that they are more familiar with their on-premises environments. Francis says tools such as LM Cloud will make it much simpler for IT administrators to explore a public cloud environment using a set of capabilities that cloud service providers are not inclined to provide.

    Thanks to the rise of public clouds, DevOps arguably has never been more complicated, with applications distributed across multiple public clouds. But it’s very difficult to manage what can’t be seen, and it’s nearly impossible to compare one public cloud with another—much less an on-premises IT environment—without access to a common baseline of metrics. All too often deciding where to deploy a workload is based on either instinct or corporate fiat rather than detailed analysis.

    While many IT organizations expect to reduce costs by moving workloads to the public cloud, the truth is more nuanced. Long-running applications tend to be less expensive to deploy on-premises. Shorter-run applications lend themselves well to public clouds. Yet, many IT organizations are simply not fast enough when it comes to deploying applications on-premises. Internal IT organizations that have embraced DevOps processes, however, are now more agile. That, coupled with software-defined infrastructure, is narrowing the dexterity gulf that exists between on-premises and cloud computing environments.

    All things may not be equal just yet. But the day when an on-premises IT environment is just another cloud that needs to be managed is that not that far off.

    — Mike Vizard

  • 5 Ways DevOps Can Help Improve IT Security

    5 Ways DevOps Can Help Improve IT Security

    You already know DevOps can make software delivery faster and more efficient. But did you know it can help to keep software more secure, too? Here’s how.

    DevOps refers to a set of practices and cultural priorities related to the way software is designed, produced and deployed. It emphasizes collaboration between developers (the people who write code), IT Ops (the folks who manage software in production) and everyone in between—such as software testers and designers.

    When done properly, DevOps greatly reduces the time it takes to bring software from idea to implementation to end user delivery. It also adds efficiency to the software delivery process in many ways. It allows different team members to work in parallel, for example. It also ensures that coding problems are found early in the delivery pipeline, when fixing them requires much less time and effort than it does once a bug has been pushed into production.

    DevOps and Security

    But making software delivery faster and less costly isn’t the only thing DevOps does. DevOps also helps make software more secure.

    It does this in several ways. Consider the following:

    • Visibility. DevOps encourages constant communication between all members of the IT organization. That maximizes each team member’s visibility into the software being developed. In so doing, it places as many eyeballs as possible on the code, at all stages of delivery—which maximizes opportunities for your staff to find security flaws before code is released into production.
    • Automation. In the DevOps world, software delivery tasks are automated as much as possible. With automation comes consistency and predictability—and a much lower chance that human error during a manual process could introduce a security vulnerability.
    • Fast updates. DevOps enables a much faster pace of software delivery. That means that security bugs (and bugs of all kinds, for that matter) can be squashed very quickly by rolling out a fix.
    • Tool-agnosticism. With DevOps, you can use whichever tools or programming frameworks you like. You’re not bound to particular platforms or vendors. That’s important from a security perspective because it empowers you to adopt the tools that are most secure for your needs—not the ones you’re locked into because you have an outdated, inflexible software release process.
    • Agile technology. While DevOps doesn’t require you to use any specific type of tool, DevOps teams tend to favor next-generation architectures and technologies, like microservices and containers. These help to make apps more secure by reducing attack surfaces and enabling quicker reaction. If you deploy your app using containerized microservices, it becomes harder for attackers to compromise your entire app, because an attack against one microservice doesn’t give them control of the other ones.

    DevOps isn’t a silver bullet when it comes to cybersecurity, of course. Keeping your systems and data secure requires careful planning on a number of fronts and implementing a variety of tools.

    But DevOps—especially a highly automated, DevOps-based release process—should be one important weapon in your cybersecurity arsenal.

    — Chris Tozzi

  • Visibility Drives Data-Driven Decision-Making

    Visibility Drives Data-Driven Decision-Making

    One of the biggest challenges of DevOps today is lack of visibility—especially shared visibility—into application delivery process and outcomes. A common data fabric allows all stakeholders to share visibility into DevOps processes, so they can more easily collaborate and communicate, use objective data to make decisions, and deliver better code faster.

    The DevOps Visibility Challenge

    A typical DevOps process relies on a loosely coupled tool chain. It includes separate tools for discrete functions including project management, source code control, provisioning and configuration, test execution and workflow automation. A tool chain approach adds much value, but becomes complex quickly. Also, it makes it difficult for stakeholders to get accurate insight into application delivery.

    This challenge is especially endemic to large and distributed enterprises. It’s even more difficult when there are thousands of developers and dozens of data centers across multiple geographies. Having duplicate tools, specialized teams, varying processes, distributed locations and multiple business units makes it even harder.

    As a result of this poor visibility into application delivery, stakeholders from business and IT:

    • Cannot manage the velocity of feature delivery. They cannot detect and fix slowdowns or bottlenecks in the process or allocate and prioritize resources. They also cannot plan for downstream activities or triage slowdowns and stoppages in the software development life cycle.
    • Cannot make objective decisions on quality. They risk releasing error-ridden software, slowing down response times, introducing security vulnerabilities, oversaturating infrastructure, damaging reputation and adding unnecessary costs.
    • Cannot properly collaborate on detection, diagnosis and remediation of production problems. They also cannot share an understanding of the impact of code changes on business key performance indicators. Customer satisfaction, customer signups, cart fulfillment rates and revenue all could be affected.

    Data Mining Enables Visibility

    To overcome these obstacles, successful DevOps teams are using objective metrics mined from machine data. That helps them gain insight into the end-to-end delivery cycle, from idea to release.

    Collecting, correlating and analyzing machine data from multiple sources gives dev and ops (and others) continuous insight. They can see the velocity, quality and business impact of application delivery. It provides individual contributors the data they need to be responsible for their own work, accountable to each other and aligned with business objectives. It enables managers to make data-driven decisions about workload prioritization, resourcing, scheduling, performance and value delivery.

    How to Get to Data-Driven DevOps

    Some of the key actions you can take to start on a journey of data-driven DevOps include:

    Measure your speed of application delivery: The machine data generated by your DevOps tool chain provides a wealth of information about all phases of the development cycle. It shows what is in development, close to release, in backlog and slowing you down. It shows which stages in the process are failing and what teams are most efficient. Consider mining and sharing velocity-related measures such as commit rates, sprint times and “idea to cash” time, to name a few.

    Measure the quality of your application code: Even if you are already using testing tools to understand release quality, there is more to quality than test coverage. Again, choosing the right KPIs is critical, as is sharing those measurements throughout your team(s). Some other key metrics may include the number and importance of defects, defect variance from release to release (or from team to team) and build/integration/deployment failures.

    Measure the business impact of application code: All too often organizations measure activity, but fail to align application delivery with business goals. As one customer recently told me, “Shipping crappy releases faster won’t help.” Business stakeholders especially are looking for business KPIs such as:

    • user signups or cancellations
    • customer satisfaction
    • cart fulfillment or abandonment
    • time on site
    • transaction failure rate
    • sales volume
    • application revenue
    • app downloads
    • offer open rates
    • customer satisfaction; and
    • customer acquisition costs

    Measure the human impact of DevOps: DevOps is mainly about people, so you must find ways to measure culture change, too. You can mine various applications for objective metrics such as number of sick days, “work from home” days and variance in team and individual productivity. However, machine data will not provide all the relevant KPIs; you will need subjective measures too, such as from “stay interviews,” feedback forms or surveys.

    Data-Driven DevOps with a Common Data Fabric

    A data-driven DevOps approach more effectively drives continuous improvement and higher levels of agility. It also helps achieve greater collaboration, improved security and compliance, and better alignment with business KPIs. These all occur while enabling rapid iteration and innovation.

    A common data fabric provides the objective measurement and shared visibility critical for a data-driven DevOps approach. With comprehensive and continuous visibility into key performance measures, provided through a shared data fabric, DevOps teams can:

    • Isolate “waste,” detect and correct slowdowns, and deliver applications faster
    • Correlate test and QA outcomes to find more problems sooner and improve code quality
    • React faster to detect and address problems that do get through to production; and
    • Use real-time insight to measure business impact and iterate faster on good change

     

  • Dashboards from Hell

    Dashboards from Hell

    Even organizations with a mature DevOps services model inevitably will hear from executive sponsors about the “need” for a dashboard for DevOps information. Organizations just starting to embrace DevOps won’t have the time to construct one yet, but over time will reach the same conclusion. The need is real, and the value is significant—it is how you get there that makes the difference. Vendors are keen to promote the idea that they already have a dashboard built into their respective products. And this is partially true.

    But the problem with dashboards is not a matter of information, it is a matter of perspective and scope. Ultimately, all of the information an audience wants can be found in the various DevOps tooling used in the continuum of services your organization created. However, locating some of it takes profound knowledge of how each tool operates, adequate security profiles and an elephant’s memory as you click on multiple windows to try to correlate the data you ultimately are seeking across the various tools.

    Before you know it, executives and perhaps even practitioners are calling for a consolidated dashboard that would present this information cleanly. The danger here, of course, is an attempt to consolidate every perspective from every individual audience into a single dashboard for your company. Instead, it is wiser to begin such an effort by breaking down the number of audiences into categories (by need, perhaps) and examining what is already available and what is missing to make a dashboard project effective, and not one originating from hell.

    Dashboards: The Practitioner Perspective

    There is a keen difference in what a typical engineer wants to see as it relates to DevOps from what the other audiences will be looking for: namely, detail. If you consider this part of a dashboard effort the foundational section (i.e. providing the right detail information to the engineers who ask), the remaining higher-level presentations can be built on top of this data. Most engineers are content to just “look into the given tool” to get what they want. But sometimes consolidating this information into a dashboard begins to make more sense.

    Ideally, your company has only one build tool and, again ideally, only one deploy tool. If you have multiple types of both kinds of tooling in these disciplines, you would be better served to pick a winner there than try to use a dashboard effort to consolidate build and deployment data across multiple build and deployment tools. Allowing multiple kinds of build and deployment tools across an enterprise usually winds up in the demand for multiple dashboards from multiple audiences, and before you know it, the dashboard project from Hades once again has infested your organization. A clear winner in build, deployment and release orchestration cuts out this kind of problem.

    Most of the time the need for a dashboard emerges when users are trying to see information in a way the native tooling did not plan for. For example, I want to look at a given server, or class of servers, and a list of everything that is on them (by version). Or as an example, I want to look at these technology components of this type and version and find out every server in my entire data center that they may have been deployed to.

    Some vendors offer internal reporting capability intended to meet these needs. However, using those reporting features should be labeled with “caveat emptor” signage. Typically, vendors apply their reporting directly against online or production data within their respective tools. So an engineer presses the report button on his build tool, and the build reporting function begins to look at its own data from a production viewpoint, instead of from an offline reporting copy of the data. When you start running reports against “live” tools, you get slowdowns, sometimes significant slowdowns.

    To avoid this issue, start with determining what data the engineers need, how often they need it and to where you can copy this information (to form the basis of your dashboard system). You also may wish to examine how the base data is inherently constructed and join some of it where it makes sense, so you are not issuing very large queries joining key data from many tiny tables together before anything sensible emerges for your new dashboard system. Speed of presentation will matter still, even though the data becomes only near-real-time instead of real-time.

    Dashboards: The Business Perspective

    Once you have ascertained the detailed information to satisfy your engineering or practitioner community, it usually needs to be married to your business segmentation to meet the needs of this audience perspective. For example, I may have 30 servers that I use regularly in my e-commerce products or line of business division. My engineers may care less about this designation, as they only care which servers are content servers or database servers or application servers. But my e-commerce business teams look at this information differently. They only want to see “their” servers, not every server in my company. So to accommodate the next perspective audience, I need a way to marry servers and technologies to the product lines or lines of business my company understands.

    This may require an outside or additional database that tracks the raw server name and marries it with the business unit that cares about it. It may also track the application names the same ways, marrying them with the appropriate business unit. But this marriage now offers the ability to pull reports (or dashboard screens) by line of business—it is nearly certain your basic build and deploy tools weren’t designed to show information in this way. This kind of business segmentation, or filtering (in some tools), provides an entirely new way to examine the data. The need to drill down to the details may still exist, but a picture of change relative to a product line or business line becomes a high value-add for the business community who looks at DevOps data.

    The business community looks at DevOps information to help it make decisions based on the state of change at any point in time. E-commerce customers, for example, might get special company notifications of upcoming changes or improvements in their capabilities, as the business community is made aware, and stays current with the planned release events and quality of testing underway. Or those same notifications might be delayed based on a quick look at the same information presented the same way.

    The point is, business users are an important community or audience of DevOps information, and the lens they use to examine data is from a product or line of business construct rather than the raw data aggregated by a practitioner.

    Dashboards: The Executive Perspective

    Executives, no matter the level, usually are more concerned about trends in the information than point-in-time data. For example, a given executive may want to know how many deployments per month are happening out of the development class environment for a given line of business. If you have constructed your dashboard with the architecture laid out above, you should be able to provide this kind of information. The discreet data is stored offline (copied from the build and deploy and other DevOps tools themselves), then married with the business segmentation data. Now it can be presented in trend-over-time manners that executives invariably want.

    It is much more likely that a business team member will use the drill-down features of your dashboard than your typical exec. But having it available up to the executive layer is a huge win when demonstrating the detail of what is going on where and potentially framing the why. The other significant win of having a single dashboard for presenting trending data is the consistency in the presentation across the enterprise. Consider for a moment how many Excel spreadsheets are sure to spring up around the company when execs task their people to produce reports that aren’t available in a dashboard at their fingertips. Each minion of the exec likely will grasp certain data in unique ways, often resulting in bar charts that never seem to line up.

    Having one dashboard—or, rather, one method of presentation of the same data—eliminates having multiple unique minion versions that lack context and often misrepresent the real story. In addition, without having a central dashboard you construct and control, minions will pull data requests against the production copies of build and deploy and release tooling, perhaps slowing these tools down to a crawl while data is gathered. The act of looking at a thing will certainly change the speed of that thing, in this instance.

    Impact to the Bottom Line

    Meeting the 80/20 needs of your primary audiences of DevOps data can be done without tying up programming resources forever. Once you have delivered a core dashboard system with mainline capabilities, your executive or business teams can decide how much more resources they are willing to fund to continue to update and enhance the dashboard. As long as they fund it, you can dedicate the resources you need to keep making it better (see agile). But for most shops, this kind of dashboard is a three-to-six-month effort from end to end.

    Once the dashboard is up and running, it becomes “the” tool in the arsenal of transparency for DevOps services. Granting access to this kind of tool (i.e., setting up user IDs) is significantly simpler than doing it in the multiple of underlying tools you have to administer without it. The cost of administration in a large firm can be substantial if a dashboard is not present. Imagine having to set up a typical user ID in your build tool, deploy tool, configuration management database, testing tool(s), release orchestration tool and monitoring tool—and that’s just to start. Within these types of tools that provide a real service or function are secondary concerns such as security or permission profiles, group memberships, etc. A dashboard only needs a valid company ID, and you are ready to “see” change in motion via DevOps.

    The work effort typical executives “task” their minions to produce reports is usually no small effort, either. While this work effort is rarely captured or understood by the executives who request it, it definitively puts a drain on company resources (both man and machine). A functional dashboard can eliminate this work nearly altogether, and produce a consistent work-product output. Your DevOps strategies should be created to reduce costs and facilitate innovation at every turn. If you need help feel free to contact me, I am still looking at the moment 🙂 .

    To continue the conversation, feel free to contact me.

  • Interoperability’s Impact on Transparency

    Interoperability’s Impact on Transparency

    We fix what we see is broken. Providing “transparency” into the DevOps-enabled software development life cycle (SDLC) has tangential goals of quickly illuminating what is broken, running slower than it should or working flawlessly.

    Automated testing systems, for instance, are generally good at highlighting pass/fail results from integrated testing in the development process. Once they have proven their reliability, they become a de facto part of the development automation and rarely are thought of again. However, when a post-DevOps legacy build begins to take longer and longer to complete, are we able to quickly examine the runtime of our entire body of integrated tests to ensure they continue to perform within expected execution timing parameters (i.e., could the code be fine and the tests themselves be the culprit)? And is that problem exacerbated when the number of different testing tools begins to explode on the same coding unit of work?

    Connecting the dots has always been the goal of DevOps. In the ideal world, the business requirements—or the epics and stories—are part of the “dots” that remain connected or associated with each change in coding, builds, deployments and ultimately release. But are these “dots” easily transparent in each part of the DevOps continuum?

    Transparency from multiple perspectives

    To think engineers are the only audience of DevOps is too restrictive. Product and project managers are also interested in knowing “the state of change” as it relates to the business issues they are tracking, and in real time. Whether the requirement originated from a bug-fix or as a new feature, the audience of people who want to see what is where within the DevOps set of tools is not confined only to people within the development or operations of IT. The ability to provide sensible, near real-time presentation of what changes are where gives the entire organization transparency to the progress of change. It impacts every organization from customer service, to program management—executives want to know as much as casual viewers in many different business functions.

    Audit is another key driver for full transparency. Regulatory compliance is at one end of this need; corporate quality standards should be on the other. Organizations always should be more concerned with quality standards than the regulations they are subjected to. Therefore, the process of audit always drives the need for greater transparency. Imagine the ability to have “push button audit” enabled by DevOps for any change under inspection. Imagine what further cost delta this represents to organizations that are still bound to only manual methods of examination. With DevOps, not only is this possible, it also should be normal. The entire history of any change entering production should be stored by each of the DevOps tools within the journey, from ideation to implementation. All test records should be immediately accessible (and automatically associated with the particular version of change in question). Everything from failed builds to the exact time it was released into each impacted data center around the world can—and should—be available for a consolidated, push-button recollection for audit. However, the ability to make this happen often is directly proportionate to the interoperability of the DevOps tools used.

    Transparency of key data and processes

    When point solutions are selected for the DevOps tooling (defined as best-of-breed tools, with no inherent interoperability with other tools in the process), the particular functions they perform are thorough and generally complete. However, the only way to integrate key information between different tools is through APIs and/or a homemade amalgamation of the data from each point solution tool into a common dashboard. Amalgamating data is always a challenge, particularly when you consider the point of view of the audience varies by function and business discipline. If you eliminate or reduce the points of view down to only engineers (as is what usually happens), you destroy the value of DevOps transparency for every other business discipline in your organization. If you spend the labor to create a very functional, multi-perspective dashboard, your investment can be much higher than what you make in the tools themselves.

    Integrated solutions—generally pushed by the same vendor as the preferred (albeit more license-cost-heavy) solution—are not the ultimate panacea, either. While a given vendor may be able to offer tools that track requirements, construct builds, integrate testing, execute deploys and orchestrate releases, more often than not, the vendor is stronger in one discipline over the others. They may ask you (the customer) to use a limited number of tools designed for one thing to do more functions than the tools were intended to perform—which creates a recipe for constant customization and lucrative service deals for the vendor.

    What becomes important to measure in any of these instances, is the level of inherent interoperability that will exist between each type of DevOps tool in the offering. If the “connecting of dots” is inherent, and the state of change can be viewed in any of the tools, that is a win. If the point-of-view offered extends to more than just an engineer, but includes other business audiences who may have interest, that is a win. If the reporting is push-button, that is a win.

    Transparency of tool usage and roles

    But beware the dark side. Whether your organization relies on point solutions or integrated solutions, both can be corrupted by an inconsistent methodology of user profiling within the tools themselves. For example, let’s assume we have a business requirements tool, a build tool, a deployment tool and a release tool. Generally, a programmer, who we’ll call “Dave,” has a very specific role in the SDLC process that doesn’t include releasing code (programmers usually don’t release code into production). Generally, the level of granularity in defining tasks and roles within a given tool is proportionate to the rules your auditors require (whether in-house or regulatory). So for a given bank, we might define the role of a programmer “Dave” to be able to see business requirements, execute builds all day long, deploy them to the test class (or first level) of environment in the SDLC and then simply watch what happens as it progresses to production, never directly interacting again.

    One of the biggest problems DevOps tooling has is that each tool handles user management separately. That may allow me to define the roles of a programmer properly within each tool, but it does not enforce that my guy Dave is properly mapped to that role. I could make Dave a business analyst in the requirements tool, a programmer in the build tool, a deployment manager in the deployment tool, and a release manager in the release orchestration tool. This would allow Dave god-like abilities in each tool within the process. He could, in fact, create the requirement to move all our money to his account, build the code to do it, deploy it and release it into production. Dave would not have triggered any alarms in the tools, because each tool thought he was doing what he was supposed to be doing in that role. And I have never seen tools audit what role Dave is supposed to be mapped to given his job in the company or even spot-checking what role he carries in other DevOps tools to see if there is an inconsistency. Perhaps abstracting user management from the entire set of tools may be the answer—but then, who offers that capability today?

    Transparency, when properly managing access controls and particularly at the push of a button, can provide solutions—even if only after the fact—to a great number of problems that exist in our organizations and in our SDLC process. We fix what we can see is broken. If our DevOps solutions can highlight transparency from ideation to production release, we can have confidence that what we see is real and worth our attention. When our transparency varies by tool, or loses connection within the process, our confidence of the state of change diminishes and our certainty about what needs our attention goes away. Using DevOps to shine a light on our true priorities is yet another cultural change enabler we have the responsibility to pursue.

    To continue the conversation, feel free to contact me.

  • Rugged DevOps: Good, Bad & Ugly

    Rugged DevOps: Good, Bad & Ugly

    The Good: DevOps has been a success for Dev.

    With higher expectations, better processes, more management focus and new tools like advanced PaaS and containers, developers are churning out more and more software faster than ever. Success! Or is it?

    The Bad: DevOps has been a success…for Dev.

    For Dev teams, this may result in pats on the back and high fives. But for Ops, this can look like one thankless tidal wave of work after another. The same software velocity that is praised in Dev is currently overwhelming their counterparts in Operations. And frankly, many DevOps tools released in recent years happen to benefit Dev much more than their counterparts in Ops. (My pet theory is that developers at IT automation firms are acting and thinking like…developers – focusing their creative energies on solving Dev problems.) A similar problem holds true for IT Security teams who want to build quality in by “shifting left” – see  https://devopsy.kinsta.cloud/2015/12/10/getting-rugged-devops-right/ . From the vantage point of IT Operations and Security teams, they are doing more work than ever but without any more human or IT automation resources. To add insult to injury amid their increased workloads, many IT Ops teams have to invest countless hours (away from family and personal time) in training courses and migrations to reorient their processes to fit into shiny, new Dev-centric “DevOps” tools and paradigms. 

    The Ugly.  

    Work. Overwork. Glitch. Search. Find. Rework. Repeat.

    A quick glance at IsItDownRightNow.com will show you which IT Operations teams are in a living hell at the moment. The other thing you will notice is that these IT Ops teams are from some of the most reputable companies on the planet. They are talented teams but stuck in rework cycles of web performance problems, security vulnerabilities and outages.

    These problems get the attention of everyone. Operations…and Dev…and Security…and customers…and Finance…and auditors and the press.

    No one likes rework. And no one likes the panicked fire drills that cause panicked rework. So what causes those rework cycles?

    Drama-free IT. Visibility Cures Chaos.  

    Much of this chaos can be tied to miscommunication. Boring? Maybe. Simple? No. Part of miscommunication is cultural. (Hint: OrcaConfig’s “How to Screw Up DevOps” white paper covers the cultural aspects of miscommunication). But miscommunication is also cured by automation solutions that promote transparency and visibility.

    If we want drama-free IT we need IT automation solutions that eliminate miscommunication and missed assumptions between Dev, Ops and Security.

    What better way to do this than with full transparency and visibility?

    Bird’s Eye Views:

    Many IT automation tools cater to SysAdmins who are interested in managing configurations of individual servers. That’s great if your focus is on individual severs. But IT Operations, eCommerce teams, Application Owners, Compliance & Security teams and IT Managers often care more about the health and performance of their application ecosystem (including related applications, middleware, databases and configurations that tie them together). So there is often a mismatch between the tools that SysAdmins and Developers have versus what IT Ops needs.

    Operating at Enterprise Scale Requires at-a-glance Visibility:

    IT Ops teams need bird’s-eye views of applications, databases, middleware and ecosystems to promote ease of ongoing operation. They need to be able to see, at a glance that a configuration compliance problem exists and they need to know right away the location and nature of the error. Large scale IT Ops teams do not have the luxury of toggling back and forth between multiple reports and multiple screens searching for that non-compliant, broken needle in an ecosystem haystack. If they are faced with a glitch or an outage, they need to find and fix it right away. One size does not fit all. Dev, Ops and Security each have their own interests and their own need to view their application through a lens that is most meaningful for them. Multiple ecosystem views eliminate guesswork between Dev, Ops and Security and save precious time in the find-fix cycle.

    Forrester Research spells it out well in their 7 Habits of Rugged DevOps white paper, Habit 1: Increase Trust And Transparency Between Dev, Sec, And Ops:

    “I&O pros pride themselves on understanding the interconnectedness of the hardware and software environment, and they are unnerved by complexities that poorly governed delivery practices create because the firm rates I&O pros on their ability to improve the performance and stability of the production environment. To maintain the environment, they must reduce the chances of an outage or interruption of service, and they must reduce the MTTR when an issue does occur.“

    Understanding that interconnectedness depends greatly on how much intuitive ecosystem visibility we provide to our colleagues in IT Operations. But it’s not just IT Ops that benefits from visibility, Dev and Security do too.

    Developers are able to write more secure code when they get early, in-context feedback. As David Mortman, distinguished engineer and Dell chief security architect, said, “Security issues are product quality issues, and no one wants to write buggy code.”

    Even better than efficiently finding and fixing errors is of course to prevent them in the first place and to proactively eliminate the damage they cause. Here too, the answer is better visibility. With true application ecosystem visibility, Dev and Ops teams would be able to model relationships between applications, middleware and their configurations so users will know that “changing this, affects that”.

    My teenage son has better visibility via his smartphone as to his driving route and weather conditions than I have for my multibillion dollar IT operation.

    Visibility into the Future:

    Ecosystem visibility is a key to communicating between Dev, Ops and Security and running complex application environments in the present. But what about the future?

    Release automation tools and custom scripts are excellent at deploying changes across large environments. Unfortunately they can be just as efficient at deploying typos and other mistakes across large environments, causing the chaos that we discussed earlier. Here again Operations teams would benefit from improved visibility – the ability to preview and pre-validate proposed changes across the entire ecosystem well before those changes are deployed. This is the real promise and savings of ecosystem visibility. Better visibility, with more use-case lenses cuts out the miscommunication that causes too many performance glitches, security vulnerabilities and outages. This chaos takes a toll on our IT Operations teams, our employers and our customers.

    What Next?

    We are soon entering a new season with new budgets. The temptation to “buy something” will be great. In the spirit of Rugged DevOps my recommendation is to take a full, bird’s eye view of your SDLC and to shift Ops and Security leftwards, building their critical considerations into your processes and into your IT automation tool selection.