devopsy.kinsta.cloud

Tag: performance testing

Avoid Crashes: 5 Best Practices for Load Testing

Demanding customer expectations and the rapid distribution of information across social media have made website and applications crashes an off-limits path for today’s companies and organizations.

Crashes resulting from traffic spikes cause immediate and long-term loss of sales, as well as tarnishing the brand name. When servers or websites go down during highly anticipated events such as Black Friday, the Big Game or before a major concert, the repercussions can be permanent.

Yet, avoiding the “website temporarily unavailable” screen is also easier than ever before. By performance testing your website continuously and proactively you can easily detect errors, bottlenecks and bugs, and then fix them to ensure your website handles any amount of traffic.

Here are five best practices for load testing your website or app:

Define Your Load Goals

Before starting your tests, you need to figure out what it is that you need to test.

This includes the number of concurrent users, traffic patterns and areas that users can visit on your site. For example, if your site has 10,000 users concurrent at its peak, but 3,000 arrive in the morning and that total slowly climbs up to 10,000 in the afternoon—and these visitors mostly just browse your homepage—you’ll need to run a different test than if you have 10,000 concurrent users on your website all day long, constantly browsing multiple pages.

To determine your load goals, you need to engage directly with your business colleagues. For instance, get your product manager to set the system’s expected abilities, ask the marketing team to inform you about any special campaigns that might increase website traffic, and then look at server logs and tools such as Google Analytics to determine previous customer patterns.

Choose Your Load Testing Tools

There are multiple open-source load-testing tools that you can use to run your tests.

JMeter is the most popular tool, but you can also use Gatling, Locust, Tsung, The Grinder or Selenium. See here to learn more about the differences between these toolsets.

Does this seem complicated? Taurus is an open-source automation test framework that simplifies running these tools, by using a simple YAML script. If you need to run your tests from the cloud, utilizing rich analytics or requiring user collaboration, you might consider using BlazeMeter.

Take Your System to the Limit

To make sure you stand up to your load goals, you need a buffer; specifically, you need to make sure that your system can handle heavier loads.

This ensures that the CPU stays low and you don’t experience memory leaks. However, we also recommend you take your system to the limit and see when and how it fails. This ensures that you know exactly what your system’s weaknesses are, giving you a better ability to fix any errors that might occur.

This is especially important if you come across an unexpected traffic spike—say, if your competitor’s website crashes and all their customers come running to you. Wouldn’t you rather be prepared for that?

Monitor Test Results

Set aside resources for looking at test results.

Examine KPIs including Response Time and Hits/s, as well as the correlations between them, to learn what your system’s abilities and bottlenecks are. This information will help you see what needs fixing.

As you continue testing over time, you will be able to identify KPI trends, which will help you monitor system health. We also recommend you monitor backend KPIs like DB Queries, with APM tools.

Test Again

So, you’ve tested, you’ve analyzed, and you’ve fixed.

Now you have one last thing to do: Run your tests again to make sure everything is working smoothly after the committed changes. That ensures nothing gets overlooked. It would be a shame if all your hard work went down the drain due to a minor oversight.

Now, let’s get testing!

To get more load testing tips, check out this free whitepaper, “How to Ensure Your Website or App Won’t Fail at Peak Times.” To learn more about BlazeMeter, request a demo.

— Noga Cohen

May 16, 2017
Continuous Testing Improves Speed, Quality

We live in a world that expects immediacy. For a society that gets news updates from Twitter in 140-character increments as it happens while we microwave a complete meal in less than five minutes, anything that takes months—or even weeks or days—seems like an eternity.

When it comes to continuous performance testing, organizations and developers need to keep up with the accelerated pace of development common in a DevOps environment—but without sacrificing the quality of the testing. That’s the real challenge.

Twitter can deliver news virtually in real time, but the quality of the “journalism” is sometimes questionable. It’s possible to microwave a meal in a matter of minutes, but few would argue that the resulting food is actually as good as the same dish prepared the old-fashioned way. When it comes to continuous performance testing, you can’t afford to sacrifice the quality of the testing in the name of expediency.

BlazeMeter thinks it has a way to address that challenge. It recently announced new functionality that allows developers to run tests using Gatling, The Grinder, Locust, Selenium, and JMeter—both locally and in the cloud—in parallel using a single, unified control language.

BlazeMeter referenced statements from Diego Lo Giudice, vice president & principal analyst for Forrester, who noted during a recent webinar the demand for software development speed is pushing teams to look at delivery cycles in terms of weeks, days, hours and even seconds.

The ability to run the tests in parallel shortens the build/test cycles. Developers can add more tests without extending the time involved in the testing cycle. Because all of the tests are run simultaneously, the total time will be only as long as the longest single test in the bunch, no matter how many tests are included in the mix. Running the tests simultaneously allows organizations to achieve greater speed without sacrificing the quality of the testing results.

“Developers that want to excel in the era of continuous delivery need tools that give them the most speed and flexibility possible,” said Andrey Pokhilko, chief scientist at BlazeMeter, in a press release. “We’ve been working with the open source community to develop ideal performance testing tools for any scenario. The new functionality combines the best of our community work with the infrastructure we’ve built up at BlazeMeter, marking a new set of capabilities to bring performance testing to new heights.”

The new BlazeMeter capability also streamlines testing by giving developers a unified control language for managing it all. BlazeMeter developed a “performance-tests-as-code” format that lets developers define tests using common YAML or JSON file formats. Testing teams can create load and performance tests as small fragments using any text editor. The BlazeMeter format enables developers to manage test logic and performance thresholds for various open source tools without needing to be an expert in the separate tools themselves.

A unified solution that allows for simultaneously running multiple test loads in parallel seems like a valuable tool for continuous performance testing. If it works as advertised, it seems BlazeMeter has combined simplicity, speed and quality in one.

April 4, 2016
Performance Testing: The Unexpected Missing Link in DevOps

It is no secret that DevOps and continuous delivery goes hand in hand in the enterprise, but how do processes such as performance testing fit in to the equation?

Many enterprises still harbor the traditional way of thinking—that the QA and performance testing group is completely separate from the development group. A widely held myth is that each has different roles, responsibilities, backgrounds and areas of expertise. However, with DevOps gaining popularity in the enterprise, the distinct separation between testing, development and operations is no longer. In fact, to successfully implement performance testing, the once-siloed departments must work together.

Creating a Culture for DevOps

The most common purpose of DevOps is to have an automated pipeline of software and application deployment updates. An enterprise cannot implement DevOps unless all are on board, from the top down. To do this, there must be buy-in from not only the software engineers and technology buyers, but also the C-suite. This ensures DevOps processes and strategies are being implemented appropriately.

However, it’s not just about getting the tools needed for DevOps; there must be a culture shift toward collaboration for it to be effective. That is when an enterprise gets the most from the shift to DevOps and becomes the most agile.

Once DevOps is embraced, experts and power users can help facilitate how it is being used to drive maximum benefit and help the organization get more from it. The key way to determine and evaluate this maximum benefit is through continuously testing throughout the development process. While the first thought may be for “functional” testing (to ensure code operates without errors and to provide fast feedback to developers when things go wrong), no organization can afford to release applications that deliver slow user experiences, and so “performance” testing with fast feedback is equally critical. Continuous testing is only scalable and sustainable if it is automated.

DevOps Makes Load Testing Easier

A key benefit of DevOps is that automation makes everything—from implementation to functional and performance testing—faster and more predictable. Tool chains ensure that the “right things” are done with every code push. A concern commonly cited by those who have not yet implemented performance testing is that it is “too time consuming” and they will be unable to facilitate different tests at the rate they need to be done to be effective. But surprise! Organizations that already practice DevOps are in a better position to add performance testing to the mix, in a fully automated fashion. A solution like BlazeMeter takes just minutes to get API tests up and running, by just adding the HTTP transaction for specific pages or endpoints to test. Once set up, these tests will continuously run automatically and will notify the team when there are any problems or potential problem areas. When there are known surges in website traffic, more Uniform Resource Identifiers (URIs) can be added to the test, but this will not impact the autonomy of it.

Another key function of DevOps is scalability through the enterprise, and scalability of user traffic is extremely important. While performance testing would still scale without DevOps, operating on the same principles makes each concept more relevant to the folks who are analyzing the outcome of both. The key is to integrate performance testing directly with an already existing continuous delivery or integration pipeline, such as Jenkins or CircleCI. Here, fast, small-scoped performance tests are able to run in parallel to ensure performance is as expected before promoting a build to the next stage. Tests run in staging or QA environments, such as IBM BlueMix or Chef, can also be automated and integrated.

Performance testing is also important not just because of its easy integration into DevOps practices and how it fits into the CD pipeline, but the overall customer experience as well. A properly functioning webpage or mobile application is the connection between your brand and your target customer, and if it doesn’t load properly or in a timely manner, you may lose a customer to a competitor. Performance testing ensures the user doesn’t experience slowdowns or the dreaded downtime, which is often the result of a poorly functioning back-end system. Incorporating performance testing into workflows creates a centralized environment where all team members have what they need to design, run or analyze tests, while communicating quickly and clearly with each other and everybody in the enterprise.

Performance testing and DevOps are not opposing forces; and performance tests run automatically within the tool chain save time, rather than consuming more. The reality is that during this age of rapid software delivery, testing is the glue that keeps the development process on track and high quality. Performance testing should be part of every software delivery workflow, and DevOps is allowing that to happen.

About the Author/Michael Sage, Chief Evangelist

Michael Sage brings a 15-year career of enterprise software pedigree and customer success to BlazeMeter. Prior to BlazeMeter, Michael (better known as “Sage”) was a solutions architect and consultant in software delivery and performance management. He has worked with industry-leading companies including Mercury Interactive, Hewlett-Packard and New Relic, helping teams implement integrated solutions that enhanced the performance and experience of their customers. A self-taught hacker and open source evangelist, Sage always has the pulse on the cultural zeitgeist inflected with a heathy dose of humor.

February 11, 2016
Performance Testing is a Team Sport

Performance testing is a vital component of any developer’s working process. The performance of your website or app can be judged by a lot of people in your organization, both technical and non-technical.

Developers, operators and performance engineers are on the front lines when it comes to the development of a functioning site/app, but there will always be people away from engineering with comments, too.

“Can we make this faster?”

“Is the UI making this a little too slow?”

C-level execs, board members and marketers can often make the journey into becoming amateur web performance testing experts and critics, but that’s okay!

Web performance is widely accepted as a crucial factor in user experience—for example, when it comes to the e-commerce buyer’s journey. Your application better be fast, or you’re going to end up costing yourself conversions and money.

Instead of having these conversations after the development and testing has been done, we should definitely include all the stakeholders in the process from the beginning. Sound familiar? (ahem, DevOps, ahem)

Assemble the Team

Here are a few simple ways you can get everyone involved in your performance testing cycle:

Test Early

In the business of collecting data, getting a baseline is key. Test your minimum viable product (MVP) as aggressively as you would test your “final” product. If the MVP can’t stand up to some pressure early on, it might be in deep trouble moving forward.

Test Often—and Automate

The idea here is to create a performance trend to show if you’re moving in the right direction. More data points is always better.

The best way to generate many data points is to programmatically run tests after each code commit, either using continuous integration software or by having a post-commit hook or something similar in the source code control system that makes an API call to fire off a load test.

This way, you’re arming yourself with the right information to know when new code is impacting application performance, and you’ll know when to jump in and investigate changes right away.

Give People Access

We’re living in the era of transparency in software as a service (SaaS), so it’s always a good idea to give your coworkers access to your testing dashboard data. While some people in the organization might not wholly understand the technical side, it won’t be difficult for them to grasp that an increasing load time can be a bad trend.

There are all kinds of dashboard SaaS companies out there who can help you do this—assuming you want to stay away from pounding numbers into a spreadsheet or table every day.

At Load Impact, we’ve received a lot of feedback from engineers who were looking for an easy way to share test information with team members, and from that, we created the “Team Members” functionality.

Testers simply invite team members to their organization, and the new invitee can see past results and even create scenarios to be tested. That way, everyone’s in the loop, and everyone can get involved to the extent of their expertise.

Play Nice and Share

Continuous testing is a mindset as much as it’s a methodology. It fits neatly into the ideals of DevOps, and a big part of that is an inclusive, collaborative culture with your team members.

So remember, play nice with your colleagues—technical and non-technical—and you’ll create a productive, creative environment for everyone to thrive in.

February 9, 2016
SaaS: Building Tools for the DevOps-Minded

DevOps, Continuous Delivery, Continuous Integration and Continuous Testing are no longer just methodologies and development strategies — they’re all a part of a growing economy within the tech industry.

Great Software-as-a-Service (SaaS) companies such as Chef, Docker and New Relic have done an amazing job of spotting the trends early and building their tools and services around a new wave of development processes. Chef and Docker alone have reportedly raised nearly $250 million, and New Relic went public in 2014.

In fact, this very website is another example of how impactful these new development practices are — as several specialized news outlets have been born out of these revolutionary development strategies. So, not only has the DevOps methodology helped developers create better products, faster; but the process has opened opportunities for technical experts to share their experiences on sites like this, e-books, webinars, conferences and any other way people communicate.

On devopsy.kinsta.cloud, you’ll see the best articles on the web discussing how DevOps is defined in different organizations, why Fortune-500 companies are shifting old practices toward CD and the best practices of CI from industry experts.

In the race to be the most DevOps-friendly solution — no matter the niche — hundreds of startups have been founded and funded, which has only enhanced the toolboxes of thousands of developers. You’d be hard-pressed to find a good SaaS company that doesn’t offer integration with Jenkins CI, TeamCity or Chef, and that’s because not only should a SaaS product solve an important problem, but it should have minimum disruption of your current development process.

With all that in mind, Continuous Testing is one of the most exciting spaces in technology today. Continuous Testing is a process that aims to synchronize testing and QA with dev and ops, which ultimately yields better products more quickly.

There are several factors that make Continuous Testing successful in a team’s software development lifecycle — such as open-source collaboration solutions, test automation APIs and tools to track the historic performance of applications .

With the release of Load Impact Version 3.0, the global performance and load testing SaaS company has announced its latest feature that fits right in with the DevOps mindset: Performance trending.

Performance trending allows users to plot runs of the same test over time. That gives companies a clear view into data that locates patterns of performance degradation or improvement in order to easily validate code and infrastructure changes.

In the spirit of Continuous Testing, DevOps, CD and CI — performance trending improves users’ ability to gain both a low-level and high-level view into the performance of their systems. And with the increased number of tests run by DevOps professionals, performance trending yields valuable information for companies concerned with their application performance — which should be everyone.

Between devopsy.kinsta.cloud and plenty of great conferences and meetups around the world, it’s safe to say this economy of DevOps-minded tools will only continue to grow in the coming years.

July 1, 2015