devopsy.kinsta.cloud

Category: DevOps Practice

  • Le Morte d’Arthur— A DevOps Engineering Journey

    Le Morte d’Arthur— A DevOps Engineering Journey

    The stories of King Arthur have fascinated me from the time I was young. Now that I am old DevOps_the_Gray, I enjoy relating the story of King Arthur to my experiences with DevOps. I hope you find this blog interesting and relevant too.

    Imagine a boy raised in Britain by the wizard Merlin who becomes king because he can pull the magic sword Excalibur from a stone. Once king, he unites warring tribes, forms a brotherhood of knights and creates a round table to be sure everyone who sits around it has equal status as they pursue the quest for the Grail.

    DevOps as the Grail of Software Creation and Delivery

    To engineer DevOps, even the first way of DevOps—continuous flow (harmony)—requires faithful employment of leadership (King Arthur), collaborative team culture (the Knights of the Round Table), disciplined engineering processes (laws of Camelot), technology (Excalibur) and automation (the magic of Merlin, Archimedes and Nimue). The ultimate pursuit of the third way of DevOps—continuous improvement (quest for the lost Grail)—requires persistent commitment to engineering discipline and noble practices.

    DevOps also requires a persistent evangelist (Nimue, the Lady of Avalon) and leadership (Morte d’Arthur) to be forever vigilant and ready to lead DevOps’ evolution into the future (King Arthur is forever ready to rise again with Excalibur).

    Both King Arthur and DevOps depend on the virtuous tenets of chivalry/culture, laws/ processes and use of Excalibur/technology.

    Arthurian tales represent a masterful integration of historical events with misty, imagined magical episodes and, like DevOps, embody tenets of chivalry and respectful moral leadership. While the means can be elusive, the results can be magical. The sword Excalibur and the search for the Grail are powerful symbols that remind me to always try to engineer technology for good and noble purposes.

    The Arthurian Analogy Expresses the Importance of DevOps

    “Then there entered into the hall the Holy Grail covered with white samite, but there was none might see it, nor who bare it.” —Le Morte d’Arthur, BOOK XIII CHAPTER VII

    “…said Sir Gawaine, we have been served this day of what meats and drinks we thought on; but one thing beguiled us, we might not see the Holy Grail, it was so preciously covered. Wherefore I will make here avow, that to-morn, without longer abiding, I shall labour in the quest of the Sangreal.” —Le Morte d’Arthur, BOOK XIII CHAPTER VII

    Like the Grail, DevOps is too often regarded as an ethereal thing, earnestly sought after and pursued. To do DevOps well means treasures and miraculous benefits; satisfying, harmonious production; and competitive advantages in abundance.

    Like Arthur’s battles, hard work and skills are required for success with DevOps.

    “King Arthur rode in the battle exhorting his knights to do well, and himself did as nobly with his hands as was possible a man to do; he drew out Excalibur his sword, and awaited ever whereas the Romans were thickest and most grieved his people, and anon he addressed him on that part, and hew and slew down right, and rescued his people; and he slew a great giant named Galapas, which was a man of an huge quantity and height, he shorted him and smote off both his legs by the knees, saying, Now art thou better of a size to deal with than thou were, and after smote off his head. There Sir Gawaine fought nobly and slew three admirals in that battle. And so did all the knights of the Round Table.” —Le Morte d’Arthur, BOOK V CHAPTER XII

    DevOps, At its Best, is Engineering Greatness

    In my experience, well-engineered DevOps always starts by defining a great goal. Greatness is won by careful, inspirational leadership, measured tactics, superior technology, a culture of collaboration, courage and unwavering persistence in applying skills to the task until the goal is attained. To engineer is to do the noble work of engineering. DevOps engineering requires the skills of elite leaders and engineers that have a profound understanding of DevOps practices.

    DevOps’ Enduring Legacy and Future

    “Yet some men say in many parts of England that King Arthur is not dead, but had by the will of our Lord Jesu into another place; and men say that he shall come again, and he shall win the holy cross.” —Le Morte d’Arthur, BOOK XXI CHAPTER V-VII

    DevOps implementations – and DevOps itself – are not immortal.

    As DevOps leaders, experts, and evangelists come and go, the perceived value of DevOps may wax and wane. The fame of innovative new technologies expires ever more quickly with the acceleration of technological systems facilitated by stacking technologies higher and higher.

    Epilogue

    The stories of King Arthur teach us to lead passionately when needed to triumph against DevOps challenges and to continue the pursuit of perfection. I hope you find this Arthurian DevOps analogy as relevant and interesting as I do, and I hope that it adds a little uniqueness and fun to your DevOps understanding. If you would like to see more on this analogy, I refer you to the only DevOps book with the sword Excalibur on the cover. Now, on with the quest of engineering DevOps!

    Marc Hornbeek – DevOps_the_Gray, esq.

  • Netlify Acquires FeaturePeek to Streamline App Feedback Process

    Netlify Acquires FeaturePeek to Streamline App Feedback Process

    Netlify announced today it has acquired FeaturePeek as part of an effort to make it simpler for DevOps teams to create a portal through which application stakeholders can both inspect a web application developed using JavaScript and leave comments for developers.

    Matt Biilmann, CEO of Netlify, said Netlify will embed that collaboration capability in the next iteration of its Netlify Deploy Previews. That tool makes it currently possible for a front end pull request from a Git repository to automatically generate a live preview of the site with a unique shareable URL.

    The current version simply shares URLs with application stakeholders, versus enabling DevOps teams to create a portal through which the review process can be centrally managed to make feedback more consistent, said Biilmann. Each piece of feedback from an application owner or stakeholder made via the portal will become a comment on the Git pull request, or can be shared via any collaboration tool a development team employs alongside a Netlify workflow, noted Biilmann.

    In the meantime, Netlify is also making available a free user role license for reviewers that want to make use of Netlify Deploy Previews to view recent updates to applications.

    Netlify has been driving adoption of a serverless computing framework based on a JAMstack architecture for building JavaScript applications faster. An application based on a JAMstack architecture isolates the front end and back end end of an application to make it possible to prebuild front ends that are highly optimized around static pages to make web applications faster. The name JAMstack is a mashup of JavaScript, application programming interface (API) and markup language.

    The need for more collaboration between developers and stakeholders has become more pronounced, in part, because organizations have significantly accelerated the rate at which they are investing in digital business transformation initiatives, noted Biilmann. With organization more dependent on software than ever, it’s critical for them to streamline the feedback cycle as the number of application stakeholders within organizations steadily increases, added Biilmann.

    As the relationship between DevOps teams and the rest of organization continues to evolve, the amount of feedback any team is likely to receive will continue to increase. Everyone from sales teams to the board of directors now realizes how dependent the business is on software. DevOps teams will need to way to organize and prioritize that feedback. They should also expect application owners and stakeholders to follow the progress of software development projects more closely than ever because, in many cases, financial projections are now based on assumptions being made about software delivery timetables.

    Many DevOps teams have been making the case to have a more central role in business planning for years. Now that many of them are getting a seat at the proverbial business planning table, however, they may soon realize they don’t really have the processes in place to absorb the overabundance of enthusiasm and anxiety that application stakeholders are likely to generate in equal measure.

  • Our API Mess is Coming

    Our API Mess is Coming

    APIs are the satisfaction of a long and deep need to be able to create consistent and reliable integrations between disparate systems, operating systems and datasets. As we started to use REST-based APIs, we also realized they fulfilled a previously little-addressed gap in automation. Frankly, they are a force multiplier in DevOps delivery.

    Early efforts at system-independent APIs were not great, but we learned lessons and are steadily getting better. While REST is the default solution for APIs, it probably isn’t the end of the road, as we continue to learn and improve. That’s what we in tech do – get excited about an idea, drive it to broad use and then iteratively improve it.

    And that’s the root of the problem. We have been, and are, saturating our applications with APIs. Internal, external, external-that-rely-on-another-external. The thing that comes to mind every single time I look at an API dependency map for even a medium-sized application is “spaghetti code.” Those lines are going all over, quite frequently returning to themselves.

    And we’re going to have to maintain this creation. Maybe not us, individually, but us as in IT and DevOps teams. The only constant is change, and far too many of us are not only failing to plan for API changes, but failing to even think about them. Because things are generally going swimmingly and most IT managers and DevOps teams have more urgent things to think about.

    But risk management – particularly broad risk – is an essential part of future planning. If you lose access to external API X, what will you do to keep your application running? If you lose an entire internal system that hosts API Y, how can you recover in a timely manner? While most organizations will identify and repair single points of failure, the solution is a pool, and the impact of losing Z servers in the pool needs to be understood, if nothing else – questions like, “At what point would user experience be impacted?” need to be asked, and a plan formulated to keep that from happening.

    We are already seeing issues created by using external APIs. All of them are predictable – an API going offline, or even disappearing completely; an API interface or attendant data format change; a change to the algorithms underlying the API, causing a change in data results, etc.

    API usage is far easier than it has historically been – and companies are working hard to make it easier to access APIs, both yours and others’. This is a good thing, as it means less time spent figuring out and tweaking API calls and request data. But it means our API dependence will only grow, and the number of lines in our dependency graphs – each representing some amount of risk – will only grow.

    “But we’re doing things in the cloud, so we can auto-scale, and failures are no longer and issue for us!” some shout. While that can help some scenarios, it doesn’t help others – if a vendor goes under and takes an API you use with it, or a preferred vendor (or inside component) becomes a security risk, you’ll still need to take steps. Knowing what and where your weaknesses are before they become issues is important, and worth your time.

    Most of you are killing it, cranking out solutions faster, iterating to make them better, using APIs to extend functionality and reduce delivery times. I’m just saying, stay aware of the technical debt you’re creating, because debts always come due, and risks eventually become issues. So know what’s a risk in your infrastructure, and have a plan to react when it does cause problems.

  • Perforce Acquires 21 Labs to Advance Mobile App Testing

    Perforce Acquires 21 Labs to Advance Mobile App Testing

    Perforce Software announced today is has acquired 21 Labs, a provider of a suite of tools that leverages artificial intelligence (AI) to provide a no-code approach to testing mobile applications. Terms of the deal were not disclosed.

    Tim Russell, chief product officer for Perforce Software, said the 21 Labs tools complement a set of similar AI-based tools for testing web applications that the company makes available via its Perfecto portfolio. The company also makes available Virtual Devices, a service that makes it feasible to test mobile computing devices at scale using Apple iOS and Google Android emulation software. Perforce acquired Perfecto in 2018.

    21 Labs developed a test automation platform that uses algorithms to observe interactions within a mobile application that are then used to generate tests. That approach sharply reduces the need for a developer or professional application developer to employ scripts to generate those tests, said Russell.

    The next step will be to integrate 21 Labs with the test recommendation engine that Perforce already makes available via its Perfecto testing platform, added Russell.

    Perforce

    As the rate at which applications are developed accelerates, Russell noted, application testing has become a major bottleneck. The shift toward scriptless approaches to application testing will make it possible for organizations to quickly vet applications to ensure quality without having to slow down the rate at which they are being deployed. That’s especially critical at a time when mobile applications are often at the core of any digital business transformation initiative.

    It’s not clear to what degree application testing will be automated, but it’s apparent that as AI becomes more advanced, the need for many of the routine tests conducted by professional developers today will be eliminated. In theory, that should afford professional developers more time to conduct complex tests, while more routine tests are automatically run as an extension of a DevOps workflow.

    Application testing, of course, has always been the first thing to suffer whenever developers are pressed for time to make a delivery deadline. However, the sooner any issue is discovered, the less expensive it generally is to fix. At the same time, end users are not generally patient when it comes to mobile applications that don’t work as advertised. External-facing mobile applications, especially, need to provide a compelling experience the first time they are fired up. At the same time, internal digital business processes that are accessed by end users and business partners now need to be accessible from anywhere.

    Arguably, it’s harder to provide a great application experience on mobile computing devices than it is on a desktop platform, which means there should be even more testing of mobile applications before they are deployed in a production environment. The days when developers experimented on end users by rolling out minimally viable applications are coming to an end, as end user expectations continue to rise. DevOps teams that don’t adjust to that new reality may soon discover just how critical testing has become to their own job security.

  • WhiteHat Security Allies With Bit Discovery on Vulnerability Intelligence

    WhiteHat Security Allies With Bit Discovery on Vulnerability Intelligence

    WhiteHat Security, a subsidiary of NTT, has announced an integration with Bit Discovery to make it easier for developers and cybersecurity teams to discover the extent to which the attack surface they need to defend might be impacted by a vulnerability.

    Bit Discovery maintains a database made up of more than 5 billion internet-connected assets and hundreds of third-party resources to automatically generate an inventory of the assets that make up an organization’s attack surface. Bit Discovery continuously scans internet-connected assets to monitor new domains, phishing sites and connected devices to provide cybersecurity teams with a deeper understanding of the threats their organization faces.

    WhiteHat Security integrated its application security platform for scanning for application vulnerabilities with the Bit Discovery database via application programming interfaces (APIs) to make it possible to search that database via a dashboard. All vulnerabilities surfaced via that dashboard are verified by WhiteHat’s Threat Research Center, a team of application security experts, to reduce any potential alerts that might actually be false positives.

    Craig Hinkley, WhiteHat Security CEO, said the goal is to make it simpler for organizations to shift more responsibility for applications security left toward developers in a way that doesn’t result in them chasing their tail every time a new vulnerability is discovered. Instead, a virtuous cycle of workflows can be created that enables developers and cybersecurity teams to collaborate more effectively, Hinkley added.

    Ultimately, the goal is to make scanning for application vulnerabilities a more natural extension of any quality assurance process. The simple fact that there is so much focus today on DevSecOps best practices only highlights how much work there is to be done before security becomes just another routine gate within an automated DevOps workflow, noted Hinkley.

    In the meantime, unfortunately, cybersecurity teams are finding it increasingly difficult to keep pace with the rate at which applications are being developed and updated. The Bit Discovery database integration provides a way for cybersecurity teams to have more relevant conversations with developers about which vulnerabilities need to be prioritized based on the actual threat they represent to the organization. In the absence of that integration, security becomes a losing battle, simply because developers and cybersecurity teams both are overwhelmed by the volume of vulnerabilities that would otherwise need to be investigated on their own without any assistance.

    The integration between WhiteHat Security and Bit Discovery comes at a time when the focus on software supply chains has increased significantly in the wake of a series of recent high-profile breaches. More attention is now being paid to preventing malware from finding its way into the application development process. The challenge is finding a way to provide developers with actionable insights into risks that need to be mitigated before and after an application is deployed in a production environment. After all, vulnerabilities are just as often discovered after an application is deployed as they are before.

    It may be a while before most organizations are able to implement a robust set of DevSecOps best practices, but the more reliable the vulnerability intelligence being provided to developers, the more likely it becomes they will act on it before a cybercriminal exploits it.

  • Why DevOps Needs Mandatory Vacations

    Why DevOps Needs Mandatory Vacations

    “If a team doesn’t have time for retrospectives, you should hold them twice as often.”

    It’s 5:00 p.m. on Friday, and you’re just shutting down your computer to kick off the weekend. A message pops up in Slack:

    “Congratulations, you’ve been selected to take the next week off! We’re cutting off access to all work-related systems and revoking your credentials for the week. See you the following Monday.”

    What just happened?

    There’s an ancient security practice worth reviving in the age of DevOps: mandatory vacations. In practice today, it typically looks more like cutting off access when an employee is taking an already planned vacation, rather than forcing them to take a spontaneous one.

    But the goal remains the same: quickly assess bottlenecks, critical dependencies and security vulnerabilities. The side benefit? Your employees actually disconnect completely when they go on vacation!

    The Double-Edged Sword of DevOps

    DevOps practices increase a team’s end-to-end ownership, responsibility and autonomy. Team members modify code, deploy infrastructure and handle operational duties – sometimes all in the same day. Bringing Dev and Ops teams closer together results in reduced lead time, faster deployments and higher-quality software.

    But blurring the lines between Dev and Ops requires broad access and permissions. When everyone on the team has access to everything, there’s no forcing function to prevent relying on individual or shared accounts. This access proliferation results from a lack of clear indication on which credentials, accounts or permissions authorize each responsibility. Critical tasks may depend on a single person’s account. Credentials may be shared and never rotated.

    For example, deployment may depend on a service account, or bot, for which everyone has a copy of the API token. Admin or sudo rights may be delegated on a case-by-case basis to individuals, rather than role-based access control or group management.
    The result: a confusing quagmire of roles, permissions and dependencies.

    Reviewing and Remediating

    Explicit review of all accounts and permissions is an onerous, audit-like chore. While it might be necessary, particularly in regulated environments, it doesn’t mesh as well with agile, iterative, automated and just-in-time principles that fuel DevOps.

    Mandatory vacations provide an alternative. It’s a simple concept: cut off a team member’s access when they go on vacation.

    Depending on your setup, you might use this time to disable accounts, revoke privileges, rotate credentials, remove them from groups and even archive any home directories left over in production.

    The danger lies in leaving dependencies on personal accounts for day-to-day operations. From a security perspective, personal accounts are much more likely to be compromised or abused by a disgruntled insider. They’re an automatic back door into your deployment and production systems. From a reliability and operations perspective, a dependence on personal accounts means that changes to personal accounts (such as an employee leaving or losing access) may cause unforeseen outages. When you control the change as part of a mandatory vacation, you’re prepared for a change that could potentially cause things to break and can trace it back to the root cause quickly. Nothing like breaking the build to force a cleanup of technical debt! This technical debt could include critical nightly cron jobs left in home folders or owned by a single user, personal credentials hidden away in build secrets or operational practices that were assuming an individual’s identity or role.

    As a bonus, mandatory vacations also help ensure you keep your on-boarding practices up to date and robust. While re-provisioning your well-rested colleague upon their return, you are forced to acknowledge which privileges and permissions are truly necessary. Depending on the team’s growth and maturity, that may mean creating well-documented practices or enabling full user management automation.

    Finally, this provides an opportunity to promote key security principles. Are you enforcing least privilege where you can, giving users the minimum level of permissions needed to do their job? Are you granting access to shared folders in your password manager? If so, did you rotate the stored credentials when your colleague “left” the company, however briefly? This might provide an excellent example of why it’s time to implement multi-party authorization (MPA), just in case gaps like these persist, even after a few mandatory vacations.

    Mandatory vacations provide a ‘testing framework’ for access proliferation. Repeating difficult agile practices frequently helps streamline the process simply by doing it more often. If your team doesn’t have time for retrospectives, you should hold them twice as often. Testing your access management practices every time a colleague goes on vacation provides a natural, easy cadence to ensure your practices remain up-to-date – without having to scramble after a breach or when a team member leaves permanently.

    Take a Vacation, for Everyone’s Sake

    You might think you don’t have time to deal with the ramifications of random mandatory vacations. But imagine how much more difficult it is to discover a single chokepoint during a live incident! If you don’t have the time to test off- and onboarding accounts, that implies your access management practices are rusty, poorly defined and thus, unavailable when they’re needed in a rapid-response scenario. Laptops and accounts will be compromised. Staff may leave or join for unforeseen reasons.

    So please, take a week off … for your sake, and ours.

  • Private and Virtual Private LTE/5G Challenges Network Visibility

    Private and Virtual Private LTE/5G Challenges Network Visibility

    Digital transformation continues to disrupt the entire IT stack by shifting nearly every paradigm, including the new blurring lines between network infrastructure. New mobile network technologies are at the cusp of adaption in the enterprise networks with a growing number of innovations poised to enable expanded and more flexible connectivity with a gradual displacement of traditional Wi-Fi networks.

    The evolution began as new unlicensed radio frequencies become a part of the 3GPP 5G Release 16 mode of operation. This new mobile mode of operation has the ability to offer the necessary technology for mobile network operators (MNOs) to operate the unlicensed spectrum into 5G networks and make some of it available for corporate networking. New standardization will enable the enterprise networks to gradually adopt these new mobile networks to their connectivity framework, in order to leverage the benefits of new modern applications and alignment to their connectivity, scale and quality of service requirements.

    With the benefits of a new mode of connectivity – in a converged fixed or pure mobile infrastructure – there are always measurable risks that require attention. Specifically, the new mode of connectivity brings potentially new security threats and operational challenges that need to be addressed.

    New Operating Modes

    It is likely that incorporating this mobile technology will require starting from new operational modes; these will be imperative to grasp the direction and the adaption of private mobile networks in reality. Industry hype is certainly driving the trend towards the mode in which the corporate IT teams will be capable to deploy and operate the new private mobile network. However, at this point in time, the more realistic option will be that full control of design, deployment and operation will be from the MNOs, as they have strong experience, knowledge and risk mitigation capabilities. This mode of operation is also incentivized through the business aspects of creating a new revenue stream for the MNOs. This higher-margin business can provide more stringent security operations, particularly in light of better understanding the vector of attacks. Such an approach provides carrier-grade operations with the commitment of a service level agreement and correlation to the required corporate business agility and continuity needed by the enterprise.

    Questions like the who, what and where of network traffic need to be answered to prevent security threats and maintain network performance and reliability—in essence, identifying and maintaining the identity of the mobile subscriber in a complex hybrid model of mobile connectivity. A corporate mobile network can be completely private, well isolated from other networks with no roaming into the public network. It can also feature a less stringent traffic policy using a virtual slice of the private network with a segmented perimeter in which certain traffic will roam into a public network and will be tracked with respect to subscriber-aware policy. This latter option is a greater challenge because of the inherent separation of data or user and control planes in mobile network architecture is virtual rather than physical. Additionally, new mobile 5G architecture includes virtualization and network slicing to offer valuable versatility and scale, but it also makes NetOps and SecOps monitoring and control even more challenging. In light of the complexities, we can assume that the favorable and practical mode of new corporate mobile networks will be outsourced to an MNO that will operate in tight synergies with corporate IT teams. Their work will be based on a defined playbook to drive a gradual evolution of architectural reference models and pragmatic learning curve for corporate “do it yourself” operations mode. Complexities multiply in cases related to IoT and manufacturing applications in which the mobile core is required to be closer or co-located with the edge to support low latency and high throughput requirements.

    5G features a separation of data/user and control planes that is an architectural advantage, particularly helpful for large IoT deployments, dynamic scalability and services and higher data rates with greater bandwidth efficiencies. It offers more distributed processing for greater scale and less susceptibility to a single point of failure. It also represents a shift from best-effort networks to ones that can provide better, more differentiated levels of services and prioritization. This separation requires a new approach for network visibility to understand who is doing what. This is an issue for security as well as networking performance.

    The Challenge of Virtualized Network Functions

    Another challenge is that 5G virtualizes in software the higher-level network functions that traditionally have been performed by physical hardware. This virtualized network slicing means a sharing of physical resources that expand the attack surface by offering the potential for a sophisticated attacker to traverse from one virtual instance to another knowing that it will be difficult for security teams to detect this lateral movement. In addition, 5G radio network deployments may include significant expansions of small cells connecting in a campus geographically distributed environment, machine-to-machine communications and dynamic connections to multiple cells. All of these present a new frontier for attackers and new challenges for IT operation teams. Organizations need visibility and understanding of these new infrastructure resources to ensure a clear design and implementation to prevent security breaches and disruption of business continuity.

    The potential for significant expansion of IoT devices on corporate networks using 5G offers important advances for measuring and monitoring devices or making everything connected. At the same time, these lean devices generally have far less endpoint controls—particularly robust access control—than traditional computing devices, and it is difficult to add such controls directly to each unit. The rise of more IoT machine-to-machine communication and away from a hub-and-spoke model adds additional complexity. A proliferation of such devices also intensifies the need for deeper visibility and more automated security to scale with these deployments without the bottleneck of human intervention for each. This adds risks to the internal IT learning curve, making it an unlikely option and again, favors outsourcing to MNO experienced teams to design, deploy and maintain the corporate private mobile networks.

  • Why QA and Monitoring Need to Merge

    Why QA and Monitoring Need to Merge

    Customer expectations for speed and reliability have upended the software development life cycle. Whereas developers used to develop a product and send it to quality assurance engineers for testing before widespread release, customers now demand new versions and features on a quicker timeline. In an era with little available time to run QA in the traditional development process, what’s the alternative?

    Fortunately, modern automated infrastructure provides a solution. Infrastructure-as-code (IaC) toolchains that deliver continuous observability with AIOps let SRE teams overlook the entire process, from the first piece of code through the software product’s public release.

    This merger of the traditional roles of QA and monitoring is enabling quicker speeds within the delivery cycle, a development that improves the customer experience but also ignites business innovation.

    The Benefits of Speed in the SDLC

    Observability delivers the data necessary to eliminate traditional QA, but teams still need help. With intelligent observability — where you apply AI to the data — teams can better focus their efforts and analyze information much more quickly.

    Finding and fixing software issues after deployment unnecessarily stresses SRE teams. But through observability with AIOps, DevOps practitioners and SREs monitor from early development through daily performance. QA is omnipresent. Teams can identify root causes of issues instantly, and AI surfaces the most actionable steps to quickly resolve impacts on the infrastructure.

    The DevOps team used to leave QA engineers with the tedious and time-consuming task of pinpointing issues amid a sea of data, creating a bottleneck within the development process. The continuous learning and intelligent collaboration approach, on the other hand, merges traditional QA and monitoring and yields a CI/CD pipeline that actually works.

    Integrating observability with AI into the development cycle allows teams to spot change almost instantly. If the platform starts acting oddly after its release, SREs and DevOps practitioners can jump in immediately without the need to wait for a QA team. If the issue is where a change in performance is expected, AI and machine learning often still can reduce what’s needed from your teams. If, for instance, you’re monitoring with an adaptive thresholding algorithm, you can let the algorithm retrain and learn the new behavior instead of relying on the DevOps team to communicate the expected change in performance to the QA team.

    As QA shifts left – as evidenced by the growth of software development engineers in test (SDET) – it’s natural that monitoring and assurance capabilities should shift left, too.

    A Principled Approach to QA

    The merger of quality assurance and monitoring throughout the software development cycle follows the three guiding principles behind DevOps practices.

    • Flow/system thinking. The approach eliminates workflow silos and instead creates a holistic view of the development process. The continuous oversight negates typical quality hazards, such as the handoff that can occur despite a known issue or the optimization for local efficiency only.
    • Amplifying feedback loops. A merging of QA and monitoring eliminates the need to loop feedback through multiple teams with various processes because teams are given consistent feedback throughout software development and delivery.
    • A culture of continual experimentation and learning. DevOps teams, no longer in a position to just pass software down the line to QA, will build constant improvement into their processes as they learn from bugs along the way. This also provides more opportunity to take risks and experiment. The approach is a more holistic product cycle versus quick development.

    Improving the Q in QA

    Removing QA as a singular step in the software development process initially may seem like removing quality assurance itself. But QA is actually heightened by integrating observability with AIOps into the development cycle.

    Observability with AIOps allows teams to see into systems as they are being built but also allows them to gain actionable insights to resolve issues on the overall structure. Actionable insights empower the merging of traditional QA and monitoring to drive a whole new speed of delivery, which yields better customer experiences and gives businesses the ability to quickly launch competitive, innovative services.

  • Five Tips For Creating Design Systems

    Five Tips For Creating Design Systems

    Design systems were all the rage half a decade ago, with an explosion of open source resources made public by Google, Salesforce and Shopify, for example. Not long after, design tools like InVision, Sketch, Adobe XD and Figma featured ways for product teams to maintain design systems within their apps. If you want to build a scalable, consistent product that increases build velocity in design and engineering teams, a design system is integral.

    Throughout my own design career, I have become an advocate for building and maintaining design systems. It has become a niche passion of mine, and I hope to share some of what I’ve learned through the years as a design system nerd.

    But wait – first, what is a design system? A design system is a library of style, components and other patterns documented so that building and adopting products is more efficient and cohesive for designers, developers and end users alike.

    The key benefit of a design system is that it creates consistent, repeatable patterns that make end users think less. A product experience that delights is usually designed with persistent visuals and consistent interaction. Users want to feel comfortable knowing that no matter where they navigate, they won’t be surprised by what they find. Repetition, in the case of product design, is not boring, but welcome. Design systems create trust with users.

    Another benefit is the increased build velocity from design and engineering teams. As designers, we are tasked with solving problems. We want to create a simple understanding of how our users can accomplish tasks in a workflow. Of course, we are tempted at times to invent new patterns to solve design problems. We often forget, in the minutia of design iterations, that we’ve already solved a particular problem in a prior project or in another part of the current product. This inefficiency can lead to wasted time, especially if those existing patterns and components have not been documented. In a single-person design team, the negative effects may not be as visible, but one can imagine the exponential nature of a larger design team consistently duplicating existing work or creating new patterns that, ultimately, create an inconsistent user experience.

    Instead, we want to empower ourselves as designers to focus on solving problems without the need to invent new interfaces every time we are posed with a design challenge. Design systems are always vital, but are especially so when design teams scale up and the product experience reaches a level of maturity.

    Hopefully, you’re now convinced that design systems are the backbone of the best design teams and delightful user experiences. Below are a few tips to help you create a usable design system in your own organization.

    Start Small

    Create that button symbol. Save that primary color or header text style. Audit your designs and see what patterns emerge. You get the idea! It doesn’t matter where you are in the design process; any remnant of documenting a pattern will do wonders for saving your precious time and creating beautifully consistent experiences for end users.

    Prioritize Design System Documentation

    Whether it’s as simple as a shared page, internal minisite or a full-fledged website, documenting your design system and making it easily accessible to the right people is fundamental to its success. You need to determine the level of complexity that is needed for your organization and teams that you collaborate with. Talk to your engineering team to see how to best document and stay in sync between design and code.

    Keep Evolving Your Design System

    Ask yourself and your team if certain parts of the design system are actually helping solve problems. Iterate on a component that needs to be more flexible or add a new one if there is an important use case that needs to be addressed. Audit your component library to see if there are components that are not used, or if there are multiple components that can be merged.

    Have a System for the Design System

    It’s easy to get lost in the moment and forget to create an important, requested component or remind a developer to build a part of the UI as a component. Having a project management system and workflow for your design system can help prioritize needs and keep communication organized. Whether it’s logging tickets in Jira, moving cards over in Trello, or simply annotating or resolving comments in Figma, there needs to be a workflow that is consistent and works for the entire team.

    Collaborate

    I’ve fallen into the trap of being inside a design tool bubble. As designers, we’re often stuck inside software that is great for communicating a user experience but poor at staying in sync with the actual code that is being written. This causes an inconsistency between design and code. Design systems are no exception to that problem. From the outset, advocate for working with a developer to build a design system that works as a fruitful collaboration between design and engineering.

    These five tips will help you start creating a usable design system that will make your collaborators and, most importantly, your users, happy. Ready to get started?

  • Dear Vendor: Letter to Technology Vendors

    Dear Vendor: Letter to Technology Vendors

    While I was writing my “Dear Management …” post, something was going on behind the scenes that inspired me to write another. We all have things we wish to say to vendors; this open letter picks some of the big points that they seem to invariably struggle with.

    Dear Vendor,

    We want to use your products, otherwise we wouldn’t even be writing to you. But depending upon who you are, and what year it is, you are struggling to meet our needs.

    We understand that your insistence that your new product or acquisition will solve all of the problems we don’t have is probably executive-driven, but give us the benefit of the doubt. We know our environment, and just because we use you for one thing does not automatically imply we want you to constantly push this other thing at us. By all means, tell us about your entire product line, but then back off. We will let you know if anything appears to fulfill a need.

    While we’re chatting, can we discuss pricing? Not just cost, but the byzantine structures you create, time after time? If you are going to use 15 different charges and fees to jack up the price there, we’re paying the bill – we’ll see it. You’re far better served by being forthright. Do you have any idea how many stories like, “I am not funding your Ferrari with your commission,” stories (this was actually said; it was hilarious – we went with a competitor) are out there? And they keep occurring. Just this week, the phrase “Your pricing is bonkers!” was submitted to a company. We want a good product at a fair price, not a cheap price up front that will drain all of our budget over time.

    Speaking of good products, let’s talk about security. Security issues with your product are our issues more urgently than they are yours. Don’t mess around; when notified, quantify the threat and notify us immediately, with suggested remediations until you can get a patch ready. Holding off until you can patch puts us in the unenviable position of being vulnerable and, worse, not even knowing we’re vulnerable. Use good coding practices so we’re not all caught off guard, and waive those secure coding requirements for no one. Interns and C-level execs are not professional coders (interns aren’t yet, and C-levels no longer are), and any code by these two groups needs to be reviewed by someone that knows your systems before they are sent to us for deployment. Don’t force bad updates on us. Do your best not to give us bad updates at all.

    Finally, on the topic of forcing things on us – guess what? We have a need for a specific tool in a specific environment. Telling us that starting next month your services are all “moving to the cloud” does not suit all of our needs. We get it, you want that data. It’s all the rage right now. You should understand that, if you stop serving our needs in your quest for our data, we’ll go elsewhere, eventually, no matter how reluctantly. Cloud is a great platform, but most IT tools need the versatility to run where we need them, not just on one platform out of many.

    And, as a side note, your AIOps generally isn’t. We know it’s the buzzword du jour, but honestly, if it’s basically the same product with some logic or a dashboard strapped on, that’s not AIOps, it’s normal product improvement. When you have deep data collection and analysis that uses trained algorithms, then you can join the crowd screaming AIOps. Until then, hush. You sound silly, and it undermines the rest of your messaging.

    Just to reiterate: We have a need, show us how you fill it, and let us decide if that suits our requirements. Then tell us about your other offerings and back off. We’re smart enough to smell vaporware, stubborn enough to resist your desire to collect all of our data and wise enough to judge whether or not you and the product you’re pushing this week will be around in a year or two. You serve a purpose, and we’re grateful you are doing this stuff so we can focus on our business. But please, make life easier on us, not harder. And if one more of you sells a product we don’t need to the C-suite, we may explode. That Sword of Damocles that you raised over us will inevitably fall, and your company’s reputation will suffer for it. But we’ll pay the price for trying to implement something we didn’t need in the first place.