devopsy.kinsta.cloud

Tag: software quality

  • Harnessing the Power of AI Chatbots for Continuous Testing

    Harnessing the Power of AI Chatbots for Continuous Testing

    Artificial intelligence (AI) is revolutionizing the way we develop and test software, especially with the advent of AI chatbots. These intelligent agents have the potential to streamline and automate continuous testing while improving software quality. However, software and QA managers must understand the urgency, opportunities, pitfalls and best practices associated with integrating AI chatbots into continuous testing processes. This blog explains crucial aspects to help your team harness the full potential of AI-driven continuous testing.

    The Urgency of Integrating AI Chatbots Into Continuous Testing

    • Increasing complexity of software applications: Modern software applications are becoming increasingly complex, which makes manual testing time-consuming, error-prone and costly. AI chatbots can analyze vast amounts of data in real-time, identify patterns and learn from the testing process, making them invaluable for continuous testing.
    • Fast-paced development and market demands: Today’s competitive market demands faster release cycles and shorter time-to-market. AI chatbots can help you accelerate testing and development cycles, allowing your team to focus on delivering high-quality software promptly.
    • The growing importance of customer experience: AI chatbots can simulate user interactions and predict user behavior, helping to improve user experiences by identifying potential issues before they become real problems.

    Opportunities for AI Chatbots in Continuous Testing

    • Improved test coverage and accuracy: AI chatbots can execute a large number of tests simultaneously, leading to better test coverage and increased accuracy.
    • Enhanced defect detection: AI chatbots can identify defects early in the development process, reducing the overall cost and time spent fixing them.
    • Efficient resource allocation: By automating repetitive tasks and reducing human intervention, AI chatbots enable teams to allocate resources more effectively, leading to increased productivity.

    Pitfalls to Avoid When Implementing AI Chatbots

    • Over-reliance on AI: AI chatbots are powerful tools, but they should not replace human testers completely. Human intuition, creativity and critical thinking are essential for effective testing.
    • Insufficient training data: To work effectively, AI chatbots require quality training data. Ensure that your chatbot is trained with diverse and representative datasets to minimize the risk of biased or inaccurate results.
    • Integration challenges: Integrating AI chatbots with existing systems and processes can be challenging. Ensure that your team has a clear understanding of the integration requirements and potential issues that may arise.

    Best Practices for Applying AI Chatbots to Continuous Testing

    • Define clear objectives: Establish clear goals and expectations for your AI chatbot to ensure that it aligns with your team’s overall testing strategy.
    • Choose the right AI chatbot platform: Consider factors like cost, scalability, ease of integration and support for programming languages when selecting an AI chatbot platform.
    • Continuously monitor and refine: Continuously monitor your AI chatbot’s performance and make necessary adjustments to improve its effectiveness and efficiency.
    • Foster collaboration: Encourage collaboration between human testers and AI chatbots to leverage the strengths of both.

    Benefits of Applying AI Chatbots to Continuous Testing

    • Improved test coverage and accuracy: AI chatbots can execute a large number of tests simultaneously, leading to better test coverage and increased accuracy. By automating repetitive and complex tasks, AI chatbots can effectively handle edge cases and efficiently test various scenarios, thereby reducing the likelihood of undetected issues in the software.
    • Enhanced defect detection and early issue identification: AI chatbots can quickly identify defects and potential issues early in the development process, reducing the overall cost and time spent on fixing them. By leveraging machine learning and pattern recognition, AI chatbots can predict potential problems and bring them to the attention of developers, allowing for proactive resolution and improved software quality.
    • Efficient resource allocation and increased productivity: By automating repetitive tasks and reducing human intervention, AI chatbots enable teams to allocate resources more effectively. Human testers can focus on higher-level tasks that require creativity, critical thinking, and domain expertise. This leads to increased productivity, as the team can concentrate on delivering high-quality software and improving user experiences.

    Conclusion

    AI chatbots are transforming the landscape of continuous testing, offering unprecedented opportunities to improve software quality and speed up development cycles. By understanding the urgency, opportunities, pitfalls and best practices associated with integrating AI chatbots into continuous testing processes, software managers can effectively harness the potential of these intelligent agents and drive their teams toward success.

  • A Seven Point Checklist for Getting SAST Right

    A Seven Point Checklist for Getting SAST Right

    With so many physical products—from automobiles to airplanes and medical devices to industrial control systems—now being driven by software, product security has become a top-level concern for manufacturers. Software flaws can not only affect security by introducing vulnerabilities that can be exploited by attackers but also impact safety by compromising a product’s functional operation.

    In addition, product security has significant financial implications. For example, fixing a vulnerability in development costs 10 times less than in testing and 100 times less than fixing it in production. This explains why static application security testing (SAST) has become a cornerstone of product security and the engine for implementing code analysis at the earliest stages of development, commonly known as shift left security.

    Unlike other forms of application security testing (AST), SAST scans 100% of the code, including configuration files, not just the code that is executed at runtime. SAST can also provide additional benefits, such as quality and architectural testing. Software quality and security are inextricably linked since poorly written software typically is insecure software.

    With the stakes so high, getting SAST right is the tipping point for any product security program. To evaluate SAST tools, consider the following checklist to make sure they align with your business requirements:

    1. Deep Language Support: C/C++, Java, C#, etc.

    Languages are often selected based on the programmatic requirements of the software producer, the needs of the consumer and the hardware associated with the product. For example, developing software for safety-critical applications like aerospace requires different capabilities than languages used to build web applications.

    2. Compiler and Embedded OS Platform Compatibility

    Compilers used for embedded software development implement unique flags for configuration, such as performance optimizations, code size, error checking and the diagnostic information emitted. Since embedded OS platforms typically include compiler and debugging tools plus the various OS files and configuration options, make sure the SAST tool can integrate with the development environment used.

    3. Actionable Alerts

    For the DevOps team to embrace security, embedding security in the software factory life cycle cannot slow down development velocity or hinder developers’ ability to meet software release timelines. False positives redirect developer attention away from writing software to investigating potential defects. Too many false positives result in developers losing faith in true positives and paying less attention to security alerts of real substance.

    4. Built-In DevOps Integration

    The software development life cycle employs an extensive toolset that provides automation to help developers get products to the production stage faster—not unlike a manufacturing plant or factory. Make sure the SAST tool supports and integrates with distinct tool categories including collaboration and management, source control and repositories, IDEs, compilers, orchestration and automation and business intelligence.

    5. Flexible Deployment

    Depending on the organization and industry, SAST may need to support a variety of deployment options, including on-premises, cloud, hybrid, etc. For example, air-gapped deployments are often required in security-sensitive environments where the data protection uncertainty associated with multitenant public clouds is unacceptable. Meanwhile, cloud-based SAST can provide benefits like scalability and elasticity and reduce infrastructure capital investments.

    6. Developer-friendly UI

    Shift left security is accomplished by integrating security warnings into the integrated development environment (IDE) used by engineers. A SAST tool that works within existing IDEs can reduce the number of errors presented to testers, which subsequently reduces the volume of kickbacks to engineering and leads to faster time-to-market.

    7. Standards

    Many organizations use—and some must comply with—standards for software coding best practices such as SEI CERT, MISRA, etc., which provide a framework for improving the safety, reliability and security of software systems. In addition, a company may need to satisfy functional safety standards such ISE 61508, as well as derivative industry specific standards including IEC 62304 (Medical), ISO 26262 (Automotive),  DO-178C (Avionics) and IEC 62434 (Security). A SAST tool should provide integration with key standards that govern an organization’s products and, where appropriate, be certified for them.

    Using these guidelines for evaluating SAST tools will help ensure that the product you select meets your organization’s business, technology, safety and security requirements.

  • Chainguard Adds Private Edition of Code Signing Platform

    Chainguard Adds Private Edition of Code Signing Platform

    Chainguard today added a private preview of a Chainguard Enforce Signing service, enabled by the open source Sigstore project, that allows developers to generate digital signatures for software artifacts using identities and one-time-use keys they create themselves.

    Kim Lewandowski, head of product for Chainguard, said Chainguard Enforce Signing provides an alternative to relying on a public service to generate those digital signatures. It also allows organizations to bring their own key and certificate so that key usage can be monitored and audited per compliance and privacy requirements.

    In addition, Chainguard added a library of policies that administrators can deploy directly from the Chainguard Enforce web console to their environments.

    The Chainguard Enforce Signing feature is based on Sigstore, an open source project that provides a protocol to simplify cryptographic certificate generation by removing the need to manage keys. At its core, Sigstore creates a set of short-lived ephemeral keys with a trust root based on an auditable public log. Because it creates an immutable log, everyone participating in the software development process, including auditors, can now transparently track the chain of custody for any software artifact.

    Key management, key compromise/revocation and the distribution of public keys and artifact digests are challenging to manage. Developers are also required to determine which keys to trust in addition to learning how to sign them when different platforms are employed.

    Chainguard’s alternative is enabling organizations to host their own immutable log rather than relying on a public service to eliminate the need to manage keys, noted Lewandowski.

    Historically, code signing platforms have been problematic because they were susceptible to tampering. Cybercriminals can, for example, swap out digests to gain access to compromised credentials. Digests and public keys are also often distributed across vulnerable websites susceptible to hacks or inadvertently stored on a public Git repository.

    Code signing—along with software bills of materials (SBOMs)—is one of two technological advances that will improve the security of software supply chains. SBOMs, of course, are going to be considered more reliable when the software components they describe are signed by the developers that created them.

    Less clear is whether the signing of code will improve its overall quality. Most developers take pride in their code, but the overall quality of the software deployed in production environments could always be improved. Developers may decide to review the code they contribute to a build more intently if they know they will be required to sign it.

    In the meantime, it’s only a matter of time before organizations require that developers sign code before it is deployed in a production environment. In addition to increasing confidence in the quality of the code employed, those signatures make it simpler to track down the development team that created code whenever there is an issue that needs to be resolved. After all, the most challenging issue organizations encounter when trying to update software is not so much the creation of a patch as much as it is identifying who should create that patch based on their knowledge of the software components used.

  • Report: Impact of Bad Software on US Economy Reaches $2.41 Trillion

    Report: Impact of Bad Software on US Economy Reaches $2.41 Trillion

    A report published this week by Synopsys in collaboration with the Consortium for Information & Software Quality (CISQ) estimated that software quality issues might adversely impact the U.S. economy to the tune of $2.41 trillion in 2022.

    Cybercrime losses due to a rising number of software vulnerabilities alone are on track to increase 42% in 2022 after growing 64% in 2021, according to the report.

    More troubling still, the number of failures due to weaknesses in open source software components accelerated by an alarming 650% from 2020 to 2021.

    Finally, the report estimated that overall technical debt would increase to $1.52 trillion in 2022.

    Dr. Anita D’Amico, vice president of cross-portfolio solutions and strategy for Synopsys Software Integrity Group, said that despite these issues, there is cause for optimism. The silver lining is that more organizations are aware of software supply chain issues in the wake of a series of high-profile breaches.

    As more responsibility for application security also is shifted further left toward developers, the number of vulnerabilities being introduced into applications should also begin to decline steadily, she added. In addition, more organizations will soon be following the lead of the federal government by requiring software bills of materials (SBOMs) that will make it simpler to remediate vulnerabilities once they are discovered, noted D’Amico.

    Of course, there are still many cultural and technical issues that need to be resolved before DevSecOps best practices are consistently employed across most organizations. However, the expectation is that cybersecurity teams will be able to concentrate more on crafting cybersecurity policies as application development and IT operations teams assume more responsibility for implementing those policies, said D’Amico.

    But while giving individual developers the tools they need to write more secure code is a major step in the right direction, there is also a need to make sure the builds to which they contribute code also remain secure. As such, DevOps teams need to implement guardrails across every stage of a DevOps workflow to make sure vulnerabilities are not created as code is merged. In addition, DevOps teams now need to be aware of any malware added to an open source software component downloaded from a compromised repository. Even once patched, there’s a good chance another developer will have downloaded the same vulnerable component to construct a different application.

    No one knows for certain how many vulnerabilities may have already been introduced into production environments, but it’s unlikely all those issues will ever be completely addressed. Organizations with limited technical resources will need to prioritize the patching of the most severe vulnerabilities in their application environment based on how accessible code may be to cybercriminals.

    Cybercriminals certainly are getting more adept at exploiting vulnerable code. Developers may soon find themselves being required to update software components deployed by someone else many years earlier. In the meantime, more accountability for application security should lead to better quality code that, over time, will reduce the current unsustainable level of technical debt.

  • Why IT Should Focus on Quality Over Velocity

    Why IT Should Focus on Quality Over Velocity

    Quality assurance (QA) isn’t a new phrase for developers. Most know it as a staple part of the software process that is crucial for producing quality code. But throughout the pandemic, with extreme pressure for businesses to digitally transform quickly, it’s a discipline at risk of being dismissed by organizations and tech leaders as a last-minute, box-ticking exercise – but at what cost? Many have neglected QA in search of that quick app launch or new digital service, leading to an unwanted surprise or customer disappointment when the apps and features don’t work.

    We all know the old adage: ‘quality over quantity,’ but when considering software development, CIOs should have quality over velocity in mind. Although producing software at speed is clearly a necessity for business success–whether startups pivoting during a pandemic or enterprises keeping up with these nimbler competitors–QA must become more than a forgotten principle for internal IT teams. To achieve the necessary standards at the speed required, tech leadership must embed QA as a specified requirement of any software project, rather than an afterthought.

    Quality Assurance = Customer Satisfaction

    Quality assurance is about monitoring the software development process and method to ensure that the output adheres to the project requirements, as well as industry security and compliance standards. With QA at the core, businesses can gain a better understanding of the status of their product, which allows them to assess, improve and consistently enhance its output for better outcomes and know whether the output meets the business requirements.

    In order to draw out a strategy that ensures quality at each step, businesses need to understand their customers’ needs, user journey, expected target markets, and how they expect the software to perform. Without this, businesses will struggle to not only meet and ensure customer requirements but also business goals and expectations.

    Without QA, it is difficult for businesses to assess the functionality of the product or service and it becomes extremely difficult to track critical defects that become costly to address once found post-production. Poor QA can also lead to several other challenges, including delayed product releases, unhappy customers, reduced brand reputation and increased technical debt.

    Quality Shouldn’t be Assumed

    In its infancy, software development took on the best practices used in the hardware/manufacturing industry, which largely relies on assessing quality at the end of the product life cycle before release. As a result, much of the technology sector adopted outdated strategies which then became common practice. In fact, there are still hundreds of CIOs and CTOs I speak to that expect QA processes to be an add-on at the end of the software development process.

    Many technical leads believe it’s developers who drive quality in a software project throughout its life cycle. And this is true, to an extent – there are some fantastic, talented developers out there! But this perspective can teach bad habits and can mean QA is an expectation of individual developers rather than a specified requirement for the entire project.

    That’s because it’s easy to forget that developers aren’t quality engineers; those are, in fact, two entirely separate roles that should be working side-by-side throughout any digital project. A developer’s role involves writing and maintaining the source code, which usually includes implementing unit and integration testing to ensure that the code matches the requirements. However, they are not involved in performing functional and non-functional testing or reporting defects to the development team.

    It’s a People Problem

    Any business embarking on a digital project based on software needs both developers and QA testers—you can’t have one without the other. But tech leaders can struggle to hire both good developer talent and QA engineers—whether due to expense or lack of availability of skilled workers. Recruitment is a lengthy and costly process, especially when recruiting skills that are already in high demand.

    In light of this landscape and paired with the impacts of the COVID-19 pandemic, which has caused many businesses to slim down their in-house teams, IT departments are increasingly stretched. This means processes like QA, which many companies don’t view as a necessity or a priority, can be neglected. But putting quality assurance at the heart of the software life cycle can actually save time, helping to avoid clearing code with errors that will end up coming back to be updated.

    Ultimately, businesses don’t just need to build software, they need to improve it and make it work and evolve with its needs and goals. Quality assurance makes this possible.

  • Survey Surfaces Tension Between Software Speed and Quality

    Survey Surfaces Tension Between Software Speed and Quality

    A survey of 600 software professionals at organizations with more than 1,000 employees published today by OverOps, a provider of software analytics tools, finds the need to ensure quality still trumps the need for speed within most organizations.

    According to the survey results, 70% of respondents say quality is their paramount concern and they would rather delay a product road map than risk a critical error impacting their users. More than half of the survey respondents (53%) said they encounter such critical or customer-impacting issues in production at least one or more times a month. Two out of three survey participants report spending at least a day per week troubleshooting issues in their code, with 30% spending anywhere from two days to a full week.

    Nevertheless, organizations continue to invest DevOps (58%) to accelerate application development. Among those organizations, continuous integration (54%) and continuous delivery (42%) platforms were the top priority.

    Other areas of investment include microservices/containers (45%), static analysis (37%) and dynamic code analysis (27%).

    Nearly half of respondents (45%) said pressure to move fast was one of their top software quality challenges. More than half of respondents (59%) said they release new code/features anywhere from bi-weekly to multiple times a day.

    Despite investments in DevOps, however, a quarter of participants also said that over 40% of critical production issues are first reported by end users or customers rather than internal systems.

    OverOps CTO Tal Weiss said the survey makes it clear there needs to be more focus on software quality. The challenge organizations face is finding a way to achieve that goal while still accelerating the rate at which code is delivered. Too many organizations are measuring the lines of code delivered without taking into account how often they run into issues in production environments because of the quality of the code deployed.

    Of course, whenever a project falls behind one of the first places any organization looks to make up ground is by reducing the amount of time allocated to testing. However, many organizations are also integrating testing within CI/CD platforms to make sure critical testing processes are not skipped in the name of making a deadline.

    It’s not clear whether software being deployed today is any higher or lower in quality than software developed using legacy waterfall approaches. As long as humans write code, there will always be issues. Proponents of DevOps will argue that deploying smaller amounts of code more frequently helps mitigate that issue. If errors are discovered, it’s easier for DevOps teams to not only roll back that smaller update but also more quickly discover the source of any issue. Sometimes that issue may have more to do with the platform the code is being deployed on than the code itself.

    Regardless of the approach to developing software, there is clearly more at stake than ever. As organizations embrace digital business transformation the quality of any software update can have a major impact on the business. The challenge is first eliminating as many of those disruptions as possible and then, secondly, minimizing their potential impact on the business as quickly as possible.

  • Maintaining Exceptional Quality Despite Shrinking Delivery Deadlines

    Maintaining Exceptional Quality Despite Shrinking Delivery Deadlines

    My first article in this series focused on the need for continuous improvement across three key dimensions of software development–velocity, quality and efficiency. While these are all equally important, in this piece we’re going to focus more specifically on quality.

    The past few months—what you might call the summer of outages—have been troublesome for many of the internet’s biggest players. Google, Apple and Cloudflare, as well as Facebook, Twitter and LinkedIn, have all experienced bouts of unplanned downtime, sometimes even multiple bouts in quick succession. Is this all just a matter of coincidence and bad luck? Perhaps, but peeling back the layers reveals an increasingly painful reality of modern DevOps life.

    While the root causes of these outages were varied, several were blamed on software glitches. This suggests software rollouts may be happening before they are comprehensively battle-tested and ready for production-level systems. This is never a good idea, especially when you’re dealing with high-profile services supporting millions of users worldwide. In these cases, software-related stumbles are that much more damaging.

    When software is rolled out prematurely, a cardinal rule of development is broken in that we, the end-users, become the unwilling guinea pig for poor-quality applications. No matter the pace of rollouts, an unwavering focus on quality needs to be maintained, and automation, integration and measurement can be the keys to ensuring this.

    Automation

    Automation refers to a pre-programmed process that executes tasks with minimal human intervention. The more you automate, the higher your quality should be. Avoiding manual intervention decreases the likelihood of less optimal techniques or human mistakes, both of which can lead to poorer application quality and customer satisfaction. Essentially, automation enables the best techniques and practices to be quickly and easily replicated.

    Consider the example of automated unit and functional testing. If you can automate the execution of the best testing scripts incrementally across code units and functions, you can test more accurately, not to mention more quickly. The benefits will increase exponentially with the additional layers of testing you can automate. We recently worked with a customer who realized an overall ROI of 467% from their investment in automated unit testing.

    Integration

    Modern applications are increasingly componentized, spanning multiple platforms. Consider an e-commerce application, where the front-end may reside on a system of engagement (such as a web server) while the back-end transaction ultimately occurs on a system of record, such as a mainframe.

    High-quality, end-to-end digital services ultimately depend on all critical components being universally and automatically included and prioritized in DevOps pipelines. Similar to automation, this is because any time manual effort is introduced into the integration process, you increase the chances for error and consequently, the risk of a particular code segment lagging behind others.

    The end result? More defects making it into production in the sprint to the finish line. It’s like the famous I Love Lucy chocolate assembly line episode, which offers a humorous visual interpretation of the chaos that erupts once human intervention is introduced into a process.  Ideally, seamless toolchain integrations, across multiple application-leveraged platforms, can be utilized to maximize quality outputs leveraging DevOps technologies you’re already using, such as JIRA, Jenkins and SonarQube.

    Measurement

    Improving quality depends on development teams constantly improving their methods. This can be achieved through ongoing measurement against key performance indicators (KPIs).

    The challenge is establishing relevant KPIs, and we have seen many development teams fail at this. The most effective KPIs are those that translate to real business impact. For example, “number of bugs released into production” is a commonly used metric, but it does nothing to convey the bottom-line cost of those bugs. A more effective metric would be the cost of resolving defects, with success being declared when the organization reduces the amount of money spent on fixing bugs.

    The cost of fixing a bug is much less when it takes place earlier in the software development lifecycle (for example, in the coding and unit testing phases) than when the software is near or already in a production state. When a development team can consistently show it is reducing costs through earlier identification of bugs, this puts them in a stronger position to justify investments in better testing and analysis tools.

    Conclusion

    We’ve long known about the challenges DevOps teams face to balance velocity and efficiency with a high degree of software quality. It’s like asking a circus performer to sprint across a tight rope. It may seem unfair, but the need to release high quality software faster and faster, while staying within budget, is not going to abate any time soon.

    The recent spate of outages has taught us that even the biggest and the strongest in the industry are not immune to immense speed pressures, and the quality stumbles that often result. Automation, integration and measurement can help development teams in their ongoing quest to meet the seemingly conflicting requirements of improving quality, velocity and efficiency—at the same time.

    — Rick Slade