devopsy.kinsta.cloud

Author: Ragnar Lönn

  • Performance Testing is a Team Sport

    Performance Testing is a Team Sport

    Performance testing is a vital component of any developer’s working process. The performance of your website or app can be judged by a lot of people in your organization, both technical and non-technical.

    Developers, operators and performance engineers are on the front lines when it comes to the development of a functioning site/app, but there will always be people away from engineering with comments, too.

    “Can we make this faster?”

    “Is the UI making this a little too slow?”

    C-level execs, board members and marketers can often make the journey into becoming amateur web performance testing experts and critics, but that’s okay!

    Web performance is widely accepted as a crucial factor in user experience—for example, when it comes to the e-commerce buyer’s journey. Your application better be fast, or you’re going to end up costing yourself conversions and money.

    Instead of having these conversations after the development and testing has been done, we should definitely include all the stakeholders in the process from the beginning. Sound familiar? (ahem, DevOps, ahem)

    Assemble the Team

    Here are a few simple ways you can get everyone involved in your performance testing cycle:

    Test Early

    In the business of collecting data, getting a baseline is key. Test your minimum viable product (MVP) as aggressively as you would test your “final” product. If the MVP can’t stand up to some pressure early on, it might be in deep trouble moving forward.

    Test Often—and Automate

    The idea here is to create a performance trend to show if you’re moving in the right direction. More data points is always better.

    The best way to generate many data points is to programmatically run tests after each code commit, either using continuous integration software or by having a post-commit hook or something similar in the source code control system that makes an API call to fire off a load test.

    This way, you’re arming yourself with the right information to know when new code is impacting application performance, and you’ll know when to jump in and investigate changes right away.

    Give People Access

    We’re living in the era of transparency in software as a service (SaaS), so it’s always a good idea to give your coworkers access to your testing dashboard data. While some people in the organization might not wholly understand the technical side, it won’t be difficult for them to grasp that an increasing load time can be a bad trend.

    There are all kinds of dashboard SaaS companies out there who can help you do this—assuming you want to stay away from pounding numbers into a spreadsheet or table every day.

    At Load Impact, we’ve received a lot of feedback from engineers who were looking for an easy way to share test information with team members, and from that, we created the “Team Members” functionality.

    Testers simply invite team members to their organization, and the new invitee can see past results and even create scenarios to be tested. That way, everyone’s in the loop, and everyone can get involved to the extent of their expertise.

    Play Nice and Share

    Continuous testing is a mindset as much as it’s a methodology. It fits neatly into the ideals of DevOps, and a big part of that is an inclusive, collaborative culture with your team members.

    So remember, play nice with your colleagues—technical and non-technical—and you’ll create a productive, creative environment for everyone to thrive in.

  • SaaS: Building Tools for the DevOps-Minded

    SaaS: Building Tools for the DevOps-Minded

    DevOps, Continuous Delivery, Continuous Integration and Continuous Testing are no longer just methodologies and development strategies — they’re all a part of a growing economy within the tech industry.

    Great Software-as-a-Service (SaaS) companies such as Chef, Docker and New Relic have done an amazing job of spotting the trends early and building their tools and services around a new wave of development processes. Chef and Docker alone have reportedly raised nearly $250 million, and New Relic went public in 2014.

    In fact, this very website is another example of how impactful these new development practices are — as several specialized news outlets have been born out of these revolutionary development strategies. So, not only has the DevOps methodology helped developers create better products, faster; but the process has opened opportunities for technical experts to share their experiences on sites like this, e-books, webinars, conferences and any other way people communicate.

    On devopsy.kinsta.cloud, you’ll see the best articles on the web discussing how DevOps is defined in different organizations, why Fortune-500 companies are shifting old practices toward CD and the best practices of CI from industry experts.

    In the race to be the most DevOps-friendly solution — no matter the niche — hundreds of startups have been founded and funded, which has only enhanced the toolboxes of thousands of developers. You’d be hard-pressed to find a good SaaS company that doesn’t offer integration with Jenkins CI, TeamCity or Chef, and that’s because not only should a SaaS product solve an important problem, but it should have minimum disruption of your current development process.

    With all that in mind, Continuous Testing is one of the most exciting spaces in technology today. Continuous Testing is a process that aims to synchronize testing and QA with dev and ops, which ultimately yields better products more quickly.

    There are several factors that make Continuous Testing successful in a team’s software development lifecycle — such as open-source collaboration solutions, test automation APIs and tools to track the historic performance of applications .

    With the release of Load Impact Version 3.0, the global performance and load testing SaaS company has announced its latest feature that fits right in with the DevOps mindset: Performance trending.

    Performance trending allows users to plot runs of the same test over time. That gives companies a clear view into data that locates patterns of performance degradation or improvement in order to easily validate code and infrastructure changes.

    In the spirit of Continuous Testing, DevOps, CD and CI — performance trending improves users’ ability to gain both a low-level and high-level view into the performance of their systems. And with the increased number of tests run by DevOps professionals, performance trending yields valuable information for companies concerned with their application performance — which should be everyone.

    Between devopsy.kinsta.cloud and plenty of great conferences and meetups around the world, it’s safe to say this economy of DevOps-minded tools will only continue to grow in the coming years.

  • 5 Ways Continuous Integration Reduces Risk in Development

    5 Ways Continuous Integration Reduces Risk in Development

    ***This post is based on themes and ideas from the great e-book, “Continuous Testing,” by our friends at Parasoft.

    There’s a lot of talk about DevOps and Continuous Integration in the tech space right now, but sometimes the implementation of those methods can be tricky.

    An effective Continuous Integration strategy requires DevOps teams to manage risks in software development with a range of practices.

    Here are five processes you can use to make Continuous Integration work for you.

    Automated and Self-Testable Builds

    DevOps professionals can use separate machines for software integration to eliminate the tight coupling between the build process and Integrated Development Environment (IDE) that leads to last-minute integration concerns.

    Everything needed to build the software should be contained in a separate version control repository.

    Dedicated Continuous Integration servers and open source build automation tools can run these builds through continuous testing processes and deploy the product in the required environments.

    Automated deployment further eliminates the bottleneck of waiting for other team members to deploy testable versions of the latest software.

    Synchronize Version Control Repository

    Continuous integration is only possible as long as Dev, Ops, QA and the wider IT team is tightly integrated, collaborative and supportive of each other’s efforts.

    From a tooling perspective, effective collaboration requires a separate-but-unified master database updating changes in real-time.

    All database artifacts — including scripts, procedures and other assets should be placed in this version control repository.

    DevOps teams can enable Continuous Database Integration by using automated database integration, local database sandbox and version control repository to share database assets, as well as empowering developers with capabilities to modify the database and making DBAs a part of the Dev team.

    Build a Stronger Foundation and Expand Test Coverage

    Limiting test cases to individual assets executed at every check-in to the version control repository ensures the continuously integrated builds are defect-free in combined form.

    Tests should be repeatable and performed continuously in virtualized systems replicating unavailable testing environments.

    Continuous Integration methodologies allow you to determine how much of the code was actually tested without much manual intervention.

    Although some code coverage tools can assess the percentage of test coverage by package or class, Continuous Integration systems ensure this coverage is updated in real-time.

    For instance, DevOps teams can run these tools as part of the Continuous Integration server build script every time the version control repository is updated with a code change.

    With this information, untested code would never get through to the next stage and the latest test will be conducted across all areas as required.

    Enhance Project Visibility

    With automated build testing and deployment using a centralized code repository, build feedback mechanisms update Devs and Ops with the latest test results and the necessary progress information.

    In cases when the same level of automation for complex builds seems impractical, DevOps teams using manual procedures must establish a feedback loop to bring everyone on the same page with real-time communication and notification of accurate performance and results.

    These capabilities ensure the same high standard of reporting is maintained for manual testing practices as available for automated build testing systems.

    Follow Consistent Dev Standards

    Quality code forms from the ground up, with adherence to common code and architectural standards established for an organization’s specific requirements.

    Automated inspection tools enable Devs working on short and quick builds to maintain their focus on quality from the very foundation of the software itself.

    DevOps teams should assess every integration build for potential defects that surface after integrating several iterative builds but should be taken care of far earlier during the development stages.

    In Conclusion

    The right toolset is key to Continuous Integration, as is the cultural change that facilitates this DevOps practice.

    Continuous Testing presents several areas of improvement for DevOps teams to develop iterative builds in individual, isolated branches and then merge the latest builds in a unified, single and well-synchronized Master branch.

    The idea is to ensure all risks are mitigated beforehand so DevOps professionals are not bogged down with endless needs to rework the product — and that gives them time to focus on true Continuous Integration.

  • Continuous Testing: What exactly is it?

    Continuous Testing: What exactly is it?

    ***This post is based on themes and ideas from the great e-book, “Continuous Testing,” by our friends at Parasoft.

    The aptly named DevOps practice of Continuous Testing synchronizes Testing/QA with Dev and Ops processes optimized to achieve business and development goals.

    A few years ago when Agile was all the rage, lightning-fast Dev sprints left the Ops and QA personnel playing catch-up due to inadequate and sluggish testing practices with minimal coverage across frequent builds.

    The widespread culture of Agile development has accelerated development while software testing still lags behind, forcing organizations to cut corners in QA or slow down Dev processes entirely.

    Instead of speeding up the conveyor belt beyond control or slowing down development to kill efficiencies, Continuous Testing in DevOps encourages a systematic approach toward process improvement.

    Continuous Testing goes beyond automation and encompasses all practices — including tooling and cultural change — that help mitigate risks before progressing to subsequent Software Development Lifecycle stages.

    Continuous testing and Automation

    The approach to Continuous Testing can vary and follow diverse pathways to ensure the best user experience is delivered, free of defects.

    It’s barely viable to repeat all tests every time a new feature is added, so the Continuous Testing strategy fosters a company-wide cultural change to achieve four capabilities: Test early, test faster, test often and automate.

    End-to-end test automation practices are intended to integrate QA into existing fast-paced Dev and Ops processes as a means to create continuity while maintaining faster development cycles.

    It’s important to understand the role of automation in Continuous Testing. Automation alone does not enable continuity in testing, but it helps provide a qualitative assessment of risk and practice actionable tasks to mitigate these risks throughout the SDLC.

    In essence, automated testing constitutes the detection process for software issues and defect prevention, whereas Continuous Testing addresses the wider challenge of improving the effectiveness of these detection sensors.

    What does Continuous Testing Look Like

    This example demonstrates how collaboration, automation and tooling enable end-to-end testing that’s performed faster, more often and better aligned with Agile Dev and Ops practices.

    Continuous Testing starts within development processes as Devs use open source tools such as Selenium to test the functionality of their code.

    Tools such as GitHub or other internal repositories can then be used to store tests and version together with the software code. DevOps teams can use the same tests to integrate frequent builds of the software with the Load Impact API.

    Once the code reaches preproduction, dedicated QA teams can use these tests by modifying dynamic test parameters as appropriate. In the production environment, Ops can reuse these tests for acceptance testing and ongoing post production monitoring.

    Once the application goes live, integration with existing APM tools help keep tabs on server-side app metrics.

    Continuous Testing in this example is broken up into at least four environments: Development environment, Continuous Integration environment, QA Pre-production and Performance testing environment, and the Staging environment. Additional post-live monitoring further enable end-to-end continuous testing.

    Key Elements of Continuous Testing

    Parasoft, a global software solutions company, is one of the great thought leaders in DevOps and Continuous Testing. The company has laid out some key elements of the process in a fantastic e-book, and they’re definitely worth sharing here:

    Risk Assessment: Covers risk mitigation tasks, technical debt, quality assessment and test coverage optimization to ensure the build is ready to progress toward next SDLC stage.

    Policy Analysis: Ensure all processes align with the organization’s evolving business and compliance demands. Primary objectives include:

    • Identifying trends associated with injection of dangerous patterns within the code
    • Augmenting defect prevention practices for high risk areas
    • Isolating risk in target areas

    Requirements Traceability: Ensure true requirements are met and rework is not required. An object assessment to identify which requirements are at risk, working as expected or require further validation.

    Advanced Analysis: Using automation in areas such as static code analysis, change impact analysis and scope assessment/prioritization to prevent defects in the first place and accomplishing more within each iteration.

    Test Optimization: Ensure tests yield accurate outcomes and provide actionable findings. Aspects include Test Data Management, Test Optimization Management and Test Maintenance.

    Service Virtualization: Early, frequent and everywhere testing requires access to real-world testing environments. Service visualization enables access to the virtual form of the required testing stages, cutting the waste time to test environment setup and availability.

    An effective Continuous Testing strategy incorporates all of these elements and forces an organization-wide cultural change to synchronize Dev, Ops and QA/Testing as part of the true DevOps philosophy.

  • The Business Value of Continuous Testing

    The Business Value of Continuous Testing

    ***This post is based on themes and ideas from the great e-book, “Continuous Testing,” by our friends at Parasoft.

    Enterprise software development models have presented many innovative solutions for IT to align development efforts with business goals.

    Adopting new software development lifecycle methodologies is as much a business decision as an IT decision. From an IT perspective, this means the ability to produce profitable products at lower cost without compromising quality.

    Maintaining quality through the code changes to add features and push new releases has been a struggle until the inception of the DevOps-driven Continuous Testing model, which transforms costly, time-consuming IT practices into profit centers.

    The DevOps Patch to Agile Pitfall

    Having already adopted lean Dev and Ops strategies, Agile enterprise IT seems to have unlocked unprecedented efficiency and productivity by delivering better business value at a rapid pace.

    But in all the mayhem and chaos in jumping on the bandwagon of the Agile development, IT teams ended up stockpiling a mass of false assumptions.

    Simply implementing Agile processes and iteratively adding functionalities doesn’t necessarily streamline development.

    With this approach, teams ultimately ignored the true Agile philosophy of adapting, collaborating and planning, and ended up following an accelerated and reinvented version of Waterfall in hopes to develop fast, deliver early and manage issues effectively.

    A change was needed somewhere between design and development, and release and production phases where IT teams were failing to identify software defects and perform fixes fast enough.

    The lack of early, adequate, parallel and continuous testing was transforming IT into a cost center by drawing unnecessary expenses and time investments in reworking software to meet desired quality standards.

    The DevOps approach aims to address this challenge by facilitating continuous testing as an ongoing effort to identify and fix software defects early and deliver the right quality at the right time at the right cost.

    Continuous testing presents the following value propositions to raise business bottom line and open doors to unprecedented revenue opportunities:

    Informed Decision Making

    Business expectations about the reliability and speed of application delivery vary throughout the SDLC.

    Continuous Testing creates a centralized system identifying failures and defects that present business risk to the organization.

    Continuous Testing as a consistent effort provides adequate visibility and useful, real-time data. This can be a huge advantage to businesses, especially when it comes to application performance monitoring and testing.

    The assessment of stability and validity of software builds enables Ops and business decision-makers to trade off decisions between risk and time of release, cost investments, functionality, scope and quality.

    Business Agility and Competitive Differentiation

    Enterprises across all industries have development shops delivering software applications as core components of their service offerings.

    These departments, often driven by DevOps, must focus on churning out new, stable and high-quality software and race to deliver a stream of features that complement primary service offerings.

    For instance, mobile check deposit was a leading driver for bank selection a few years ago and financial institutions offering this online service (while doing it well) early gained valuable competitive differentiation.

    In the DevOps world, Continuous Testing creates a safety net for DevOps teams to develop, test and release new features to the market faster without compromising quality. This safety net works as a trusted testing environment, suite and tool to assess the impact of code changes accurately and quickly.

    With these capabilities, organizations gain the agility to bring new products to market and compete on innovation and not on the scale of business operations and investments.

    All in all, great DevOps levels the playing field between the eight-person startup and the massive enterprise competing in the same space. Who releases faster with a better product is more dependant on the process, not the funding.

    Stronger Business Performance

    IT performance is business performance. It’s measured in terms of the quality, productivity and functionality of IT solutions delivered for internal business operations or as marketable products.

    The primary objective of Continuous Testing is to ensure high quality software products are delivered on time, cost and scope. To achieve these goals, IT teams need capabilities such as automation-driven, self-testing builds in a clone of production environments.

    QA is expected to leverage frameworks and automation tools that allow testing early during the lifecycle. That shrinks time spent on test setup and maintenance, using service virtualization to replicate unavailable testing environments and establishing a comprehensive quality validation process with continuous integration systems.

    These practices work as core elements of Continuous Testing to test faster, sooner, more often and continuously, thereby boosting IT and business performance.

    These business values not only generate unprecedented new revenue streams but also unlock cash trapped within the business IT.

    The resulting increase in revenue encourages organizations to change their game entirely and pursue new DevOps-driven business models in improving service delivery, scaling operations immediately and competing on fronts of business innovation.