The logs go in and the visualizations and insights come out. You suddenly have access to data you never had before—data that helps improve your operations and your DevOps environment. But is that where the value of log analysis stops? No! Extending log analysis benefits your entire organization, not just the development and IT Operation teams.
Category: DevOps Toolbox
-
Continuous Testing – Accelerated !
Except for cases where test duration is the point of the test, as indicated in my prior blog “Isn’t Continuous Testing a Longevity Test Anti-pattern?”, faster continuous testing cycles are preferred because failing early and often is a DevOps goal. There are multiple reasons for this, but it should be evident that faster tests reduce the wait time to get results and since there are typically many development team members looking for results the developer time savings multiplier effect is huge for every minute of test time saved on each CI/CT cycle. When aggregated over multiple code branches a few minutes saved per test can be man-years of time saved over a year!
So what can be done to accelerate continuous testing? Frankly a thorough answer to this topic could fill a book. Below are some suggestions in a checklist format intended to be useful for DevOps .
- Organize the team for speed: When team members and the organization are explicitly selected for CT speed the entire DevOps process will be accelerated.
• A test architect familiar with the principles of best practices test environments and continuous test design is critical to design and tune continuous testing for speed. The architect establishes the CT strategy and speed goal relative to the coverage goal.
• A test analysis team reviews test results so no time is wasted on results analysis.
• Developers and test members are trained on best practices test design and their test scripts are peer-reviewed to ensure they comply with the strategy. -
Select powerful test resources: Powerful, dedicated, scalable resources will accelerate all test processes and shorten the time to results.
• Select the fastest processors, low latency storage and I/O to initialize and execute both the test systems and system-under-test entities quickly.
• Dedicate resources that have longer initialization times so they can be pre-initialized.
• Virtualize resources that can be scaled horizontally without sacrificing speed.
• If affordable, arrange enough resources to run ALL tests in parallel. -
Choose test tools that are designed for fast test execution: Look for the following features when you are evaluating test tools. Not only will this save you a lot of headaches in the long term, it will give you the flexibility to design tests that integrate with other DevOps tools.
• Restful APIs that do not require wait states between commands and responses
• Priority queuing of test cases
• Creation of test topologies that can be pre-loaded
• Ability to spawn out tests in parallel to run on any number of machines
• The ability to park a test that needs to wait for an event and return to it after proceeding with subsequent tests
• Caching features that enable initialization and results data to be processed for one test while the tool proceeds to execute other tests
• Multiple devices, session types and results data processing utilities implemented as an embedded part of the tool and not requiring slow speed scripting
• The ability to render commonly used test utility functions in a fast execution language. -
Organize test schedules and test suites to fail early: The sequence and approach for scheduling of test cases greatly affects the speed of testing. If possible the scheduler should launch all tests in parallel as soon as the build is ready. Otherwise priority schemes are needed.
• Organize the test scheduler and test suites to avoid any unnecessary tests.
• Cache initialization and topologies data and resources ahead of the next test execution.
• Check the test environment before running tests, abort if invalid conditions are detected.
• Organize tests so the most important ones are run first.
• Set a failure escape threshold and abort the test if exceeded.
• Eliminate dependencies between tests so that each one can run standalone or in parallel. Queue the next test while a test is running.
• Use smart gray box testing methods to limit test selection to the most important tests according to the software change and skip tests that are low risk.
• Run longer tests in parallel with shorter ones.
5.Design test cases to fail early: The test cases themselves need to be designed with the strategy to get test results as fast as possible.
• Design each test case with a single unique test purpose and make it self-contained so it may be scheduled to run standalone or in parallel with any other test case.
• If a test case has multiple verdicts arrange the highest priority verdicts first and close the test at the first failure.
• Prune or refactors tests that have a low failure history.
• Shorten dead-script timers, and make any wait timers conditional so they can expire as soon as the condition is met.
• Reduce the number of test iterations to fit a time budget.
• Reduce communication wait times by using the fastest protocols. Optimize the most frequently used utilities.- Arrange test results reduce to minimize analysis wait times: Arrange the test results to be consumable as soon as the tests are run so the system does not bottleneck.
• Report test results as the tests are run and accumulate the results across parallel test targets.
• Establish thresholds for aggregated test results and abort the tests and release test resources to be used by other tests if the thresholds are exceeded for any run.
• Keep track of test timings and adjust the test schedules according the average and peak run times for each test case.
• Separate test result report processes to persistent resources so no test process is waiting to finish a report and the test process can proceed to the next test in parallel with test results.
• Correlate the test results to the changes and use trends analysis to help speed up diagnostics and as an input to test selection such that the most failing test cases are given higher weight in future test runs.
The above is a partial list of suggestions for accelerating continuous testing that have been proven to yield good results for accelerating continuous testing and DevOps.
At Spirent we think testing has a bright future in DevOps. You can read more about our views at Spirent.com/solutions/devops
What do you think of these suggestions and do you have others that should be mentioned?
- Organize the team for speed: When team members and the organization are explicitly selected for CT speed the entire DevOps process will be accelerated.
-
Building software quality into your Continuous Delivery pipeline
The modern software development landscape places a premium on speed. It’s good to be first to market. If you’re going to fail then you should fail fast. A rapidly evolving product can be shaped according to customer feedback, and fast turnaround allows you to stamp out defects and hone the software for your audience.
A continuous delivery pipeline enables you to act and deliver more quickly. But you need to build in quality metrics to ensure it retains a laser focus, and doesn’t become an out-of-control fire hose. It may require some front-loaded effort and resources to set an accurate measure, but the resultant software quality turns short term cost into long term savings.
Plan for quality
You can build the expectation of quality into your software development process from the start. Design your tests before a line of code is written. Create a test architecture that can be woven into your CD pipeline. Build a self-adjusting system that applies the right tests at the appropriate time in development, covering unit tests through to performance testing. That way you always have the real-time insight you need into your software quality.
Why wait?
Test-driven development can save time and resources. By waiting for functionality to be developed before writing up your automated functional tests you’re creating more steps to complete than is necessary, and slowing down the process.
You need real examples of each feature and piece of functionality in order to design and implement it anyway, so why not create them from the beginning? They can serve as acceptance tests and keep developers focused on the core requirements for your software.
This can only work if you ensure that you have group buy-in from day one. All stakeholders must be involved in the acceptance test design. If you fail to get the input you need about what constitutes a holistic view of your software quality, then you can expect feedback that raises new concerns down the line. By that time it will be more difficult and expensive to retro-fit new quality metrics.
Aim for automation
The more of your acceptance criteria you can automate, the better. It provides you with a mechanism for taking the temperature of your software whenever you want to, for a clear picture of its health.
Naturally, there will be acceptance tests that you can’t automate. You shouldn’t try to automate everything. There’s no substitute for human insight and it’s notoriously difficult to predict your end user’s first impressions.
Functional tests are ideal candidates for automation, provided your test architecture considers optimization for regression and remains flexible enough to accommodate new examples when required. For the most efficient use of your resources you need to build in an understanding of dependencies. You need control over the decision on what to test and when, based on the quality attributes you’re examining.
Failures will be flagged at the earliest eventuality for all stakeholders, maximizing your opportunity to address them quickly. By building these quality metrics into your CD pipeline you can go live with confidence.
About the Author/Viktor Clerc
Viktor Clerc is product manager for testing applications at XebiaLabs, a provider of automation software for DevOps and Continuous Delivery helping companies to accelerate the delivery of new software. You may reach him at vclerc@xebia.com. -
Isn’t Continuous Testing a Longevity Test Anti-pattern?
Accordingly to Jim Coplien: “an anti-pattern is something that looks like a good idea, but which backfires badly when applied. “ In DevOps context, continuous testing is an attractive solution for the narrow context of testing specific software components quickly (in cycles measured in minutes or hours) but may be “bad” if the system that the software is a component of has failure modes (E.g., effects of memory leaks) that will only be observed during long term testing over several days and if the time between releases is greater than that. In other words, the pattern of quick test cycles prescribed by continuous testing appears to be a contrary Anti-pattern to the fundamental concept of long duration “longevity” testing. So what are the well formulated positive patterns that are applicable in its stead?
In my prior blog entitled “From QA to Continuous Testing” I indicated “QA test experts that are most skilled in the best practices of test design are valued because they will produce the best quality products.” QA test experts assert that it is unwise to release a software product that has not been tested for durations comparable to the lifecycle of a release. Only tests that are run for a significantly long time compared to the release cadence are deemed sufficient. Just how long is long enough? This is a subject for further discussion beyond the scope of this short article, but my own observation is that best practices test environments run accelerated level load and impulse load traffic patterns on a build candidate for at least 48 hours or more before a release is determined sufficiently tested for at least limited deployment. Even 48 hours is much longer than typical “continuous testing” cycles. So “what positive continuous longevity testing patterns are applicable in its stead?”
Continuous Longevity Testing Patterns may sound like an oxymoron but in fact longevity testing is compatible within the context of continuous testing principles. Below I describe three Continuous Longevity Testing Patterns that are sometimes used.
Pattern 1: Longevity testing time windows are reserved such as weekends. In this pattern the regular Continuous Integration/Test cycles are interrupted during the weekend and longevity tests are only run on the build that was last created prior to the weekend time window. This has the advantage of not requiring any special test resources other than the resources used for regular CT tests and does not require any sophisticated scheduling to orchestrate the test environment. The disadvantage is that the longevity test results are only available weekly, it is hard to diagnose which build caused a longevity failure and problems with release candidates will delay the release for a week until the next longevity test run validates the fixes.
Pattern 2: Longevity parallel testing is scheduled frequently, perhaps as frequently as every few builds, or perhaps daily. This pattern has the advantages that the regular Continuous Integration/Test cycles are not interrupted any time (they run 24/7), the longevity test results are available for more builds than Pattern 1, which reduces diagnostic complexity and may reduce release delays, but has a disadvantage that separate test resources must be reserved for each longevity test in parallel with the regular CT tests. For complex systems under test with many test targets and lots of builds this can be very expensive so there is a tradeoff between test resource costs, and the cost of delays if not done frequently enough.
Pattern 3: Intelligent longevity testing is scheduled only when certain conditions trigger the longevity tests to be started according to smart rules that correlate software changes to problems with longevity performance. This pattern has the same advantages as Pattern 2 and has the further advantage that the cost of test resources is usually much less than Pattern 2 if the rules for the triggers don’t fire too frequently. The disadvantage is the entire scheme depends on the creation of a rules engine and rules that understand the risk of changes for long duration testing. If the rules are too lenient then this pattern might allow too many longevity failures to escape but if the rules are too tight then this pattern is not any better than pattern 2 and yet more complex to implement.
So which Continuous Longevity Testing Pattern is best? The best answer is the one that best matches organizations’ goals for release cadence, cost of testing, and risk.
So indeed, Continuous Testing is a Longevity Test Anti-pattern but fortunately DevOps infrastructures implemented according to best practices have a variety of complementary Continuous Longevity Testing Patterns to choose from.
At Spirent we think testing has a bright future in DevOps. You can read more about our views at
Spirent.com/solutions/devopsWhat do you think of these patterns or do you have others that should be mentioned?
-
Amazon Simple Notification Service (SNS)
Amazon has packed a box full of services and covered nearly every aspect in making cloud a smooth experience for developers to put up exceedingly operational and structurally complex applications on AWS. Large numbers of cloud-based systems feel the dire need of or dependency on sending out emails on a subscription basis to end users. But it is unfortunate that the internet today is brimming with spammers who refuse to stop sending requests and retarded emails from servers with objectionable SPF configuration.
Taking care of the above mentioned issues, AWS publicized an exceptionally great service known as Simple Notification Service (SNS), designed to the needs of genuine notification delivery.
What is Amazon Simple Notification Service (SNS)?
The Amazon Simple Notification Service (SNS) has made it straightforward and trouble-free for you to construct and put together an application in its own way.
Amazon SNS is a speedy, adaptable, flexible, entirely commanded and managed push messaging service that allows you without any difficulty to set up, operate and send notifications to all your subscribers. It brings in the simple and cost-effective push notification service to Apple, Google, Fire OS, and Windows devices even, including all the Android devices in China with Baidu Cloud Push. It can also be used to push notifications to internet connected with all distributed services and smart devices.
It is capable of creating topics, subscribe to topics and even publish topics in an exceedingly scalable, flexible and beneficial manner. By means of Amazon SNS, you can direct event driven messages from your applications to your users. In order to bar and avoid any sort of message loss, all the messages broadcasted across the multiple availability zones to Amazon SNS are stored constantly.
Let’s deep dive into the block displayed in the figure:
- Topics: These are the access points or named groups of events for each identifying a specific subject, content or event type. Every single topic carries along a Unique Identifier (UI) that helps in identifying the SNS endpoint for publishing and subscribing.
- Owners: Owners are those who create topics and have a complete admittance and access to the topic.
- Subscribers: These are the clients usually the end-users, servers, applications or other devices that desire to receive notifications on particular topics of their interest.
- Publishers: They are responsible for sending messages to topics. Now SNS matches up the topic with the list of subscribers interested in the topic and distributes the message to each one of them.
Features and Benefits:
Amazon SNS has come up with sufficiently great benefits over the complications of developing custom messaging resolutions in support of a system that demands to be supervised, managed and maintained on site. Following are some of the key benefits and features of Amazon SNS.
- Reliable: Whenever Amazon SNS streams inside Amazon’s verified network infrastructure and data centers, all topics will be available each time applications calls for them. All messages broadcasted to Amazon SNS are stocked redundantly throughout numerous servers and data centers in order to prevent message loss.
- Scalable: Amazon SNS is modeled to encounter the demand of most needy and demanding applications which permit event driven limitless amount of messages at any time.
- Simple: For the majority number of cases the developer just by using only three API’S, can get started with the Amazon SNS which are: Create Topic, Subscriber and Publish. To provide you with intuitive functionality additional APIs are also made accessible.
- Flexible: With Amazon SNS, applications as well as end-users on different devices are able to receive notifications with the help of Mobile Push notification (Apple, Google and Kindle Fire Devices), Email/Email-JSON, HTTP/HTTPS, SMS or Amazon Simple Queue Service (SQS) queues.
- Secure: Amazon SNS is a secure service and makes sure that topics and messages are secure from any unauthorized access. All the Topic holders have a full authority of setting policies for each topic to put limit on who can publish or subscribe to a topic and can even make sure of the encryption via marking that the delivery procedure ought to be in HTTPS.
- Inexpensive: Pay-as-you-go method helps in saving a lot of money of Amazon SNS customers. There are no beforehand fees or commitments. It only charges you per-request for the messages, notification delivery and transfer fees that are sent through Amazon SNS.
Amazon Simple Notification Service also supports SMS (Simple Messaging Service)
Amazon SNS leaves no means of notification untouched and comes with SMS feature to subscribe a phone number via an SNS topic. Notifications can directly be addressed as text messages to cell phones, Smartphone as well as on any other SMS-enabled device. Amazon CloudWatch users who are actually responsible for monitoring and controlling their applications that are running on AWS are capable of receiving the timely updates for actions of interest or alarms as they set. Content delivery is also possible to the mobile device through email to the existing Amazon SNS users. Developers can combine mobile applications in conjunction with Amazon SNS to not only send messages but even take delivery of notifications over SMS.
Here’s a figure to better explain how Amazon SNS supports SMS
Pricing:
For Amazon SNS usage you can pay-as-you-go and have no upfront costs. You pay money according to the number of notifications that you are publishing and delivering. You even have to pay if you use additional APIs for managing topics and subscriptions. Price of delivering these notifications varies according to the endpoint type.
Below are the tables that will provide you with full details:
Publishers:
Currently, Amazon SNS only permits maximum limit of 256 KB for published messages. Each 64KB block of published data is billed as 1 request.
Fig: 3
Notification Deliveries:
Fig: 4
Data Transfer: Data transfer ‘IN’ is for $0.000 per GB.
Below is the pricing table for Data transfer ‘OUT’:
Fig: 5
Conclusion:
Amazon SNS is designed to use with other Amazon Web Services such as Amazon SQS and EC2. For example, applications which are running in EC2 have the capability of publishing event/information updates to Amazon SNS and transfer them immediately to all other end-users. In addition to this the subscriber can also choose and select Amazon SQS as a delivery protocol that delivers notifications to multiple SQS queues in parallel format keeping in mind the perseverance of messages and a formal assurance that the messages would be delivered. If you are an Amazon SNS user then use must enable this service to your mobile devices and experience its benefits.
-
Four Steps to Free Your Organization From Scheduled Maintenance
Whether you’re an operations engineer or a customer, scheduled maintenance is often viewed as a given when delivering software as a service. While far better than surprise outages, planned downtime is still tough for everybody involved– customers will always have off-hours access needs, and the low-traffic times you want to perform maintenance are often the same times your operations engineers and admins would like to be sleeping. On top of this, scheduled maintenance implies your system is less reliable than you think, because you’re afraid to change it during the workday.
How do we make scheduled maintenance better for everyone involved?
Planned maintenance is a non-starter at PagerDuty, since our customers don’t know when their monitoring tools will fire alerts. We’ve had to focus hard and innovate to deliver continuous uptime for our service, and Doug Barth, my colleague and a member of our operations team, recently did a talk at DevOps Days Chicago on the strategies we used to ditch scheduled maintenance altogether, and replace it with iterative maintenance that doesn’t compromise your entire system.
1. Deploy in stages
Your deployments need to be rock-solid. They should be scripted, fast and rolled back quickly, as well as tested periodically to ensure rollbacks don’t lag.
They also need to be forward and backward compatible by one version. It’s not an option to stop the presses when you push out a new version. Red-blue-green deployments are crucial here, as they ensure only a third of your infrastructure undergoes changes at any given time.
Lastly, stateless apps must be the norm. You should be able to reboot an app without any effect on the customer (like forced logouts or lost shopping carts).
2. Avoid knife-edge changes
Use canary deploys judiciously to test rollouts, judge their integrity and compare results. These test deployments affect only a small segment of your system, so bad code or an unexpected error doesn’t spell disaster for your entire service.
During his talk, Doug suggested a few practical ways to accomplish this:
- Gate features so you can put out code dark and slowly apply new features to a subset of customers.
- Make sure you’re testing the changes on a representative sampling of your customer base: different account sizes, usage patterns, and configurations.
- Find ways to slowly bleed traffic over from one system to another, to reduce risk from misconfiguration or cold infrastructure.
- Run critical path code on the side. Execute it and log errors, but don’t depend on it right away.
3. Make retries your new best friend
Your system should be loaded with retries. Build them into all service layer hops, and use exponential backoffs to avoid overwhelming the downstream system. The requests between service layers must be idempotent, so that you can reissue requests to new servers without double-applying changes.
Use queues where you don’t care about the response to decouple the client from the server. If you’re stuck with a request/response flow, use a circuit breaker approach, where your client library delivers back minimal results if a service is down—reducing front-end latency and errors.
4. Don’t put all of your eggs in one basket
Distribute your data to many servers, so that no one server is so important you can’t safely work on it.
At PagerDuty, we use multi-master MySQL clusters, which help with operations and vertical scaling. We also use decentralized, linearly scalable databases like Cassandra for our alerting pipeline: since transactions aren’t gated on the behavior of a single node, we can do operational work during the day.
Put together, these four strategies set an organization up to ditch annoying weekly downtime and help admins and operational engineers sleep more, worry less and maintain better—all ahead of schedule. Not only is this great for your team’s quality of life, but it also helps you deliver a great customer experience– no matter where your users are or what time it is.
-
Functional Testing for IoT
The Internet of Things (IoT) is becoming a bit more than a fun three letter acronym. And while internet connected devices have been around for a while, turning it into a practice is an important evolution. It allows the establishment of best practices, better integration into standard software development, and thus more robust applications, sometimes too robust. And because software drives everything, and software quality is key. How can you ensure quality in all those connected devices?
-
From QA to Continuous Testing
My prior blog “DevOps – It’s All About Continuous Testing” posits that QA testers don’t need to fear DevOps because continuous testing QA is a central part of DevOps infrastructures that are implemented according to best practices. That is not to say however that continuous testing is precisely the same as traditional QA testing or that QA test engineers can rest on their laurels in the new world of DevOps and continuous testing.
Traditional QA testing is a serial process that constitutes a distinct phase that follows the development phase and is conducted by an SQA test team “independent” of the development team. The idea was clearly described by Glenford Myers in his seminal books “Software Reliability – Principles and Practices” and “The Art of Software Testing” published in 1976 and 1979 respectively. According to Myers “4th Axiom [of testing] “It is impossible to test your own program”. Myers felt programmers shouldn’t do any testing, even unit testing. While the application of this axiom to unit testing was not followed by most practitioners in the software industry, the idea of independent software functional and system level testing has been a widely accepted practice for many years. The underlying rationale is that it is “impossible” for the same person (the developer) to objectively test his/her own code due to the inherent bias to want the code to work, while a good QA tester must have the opposite attitude to want to break it in order to uncover bugs in the code. If you follow this logic only “unusual” developers would be able to perform both roles successfully and therefore building a team based on this rare personal quality is essentially not practical.
Despite the obvious logic of “independent QA test” concept, many very complex software projects without an independent QA test team resulted in extremely high quality products. There are many high quality open source software products that had no independent test organization behind each release version. Meanwhile there are countless examples of software projects that had massive independent test resources that resulted in products that failed miserably. How can this be? Perhaps it is not “who” is doing the test that matters but rather it is the quality and frequency of the testing itself that is important.
Continuous testing, with its fundamental concept of “fail early and fail often”, is the new QA test method of choice. Sure, testing is still essential and QA test experts that are most skilled in the best practices of test design are valued because they will produce the best quality products. Years of software engineering experiences have demonstrated that developers can learn these skills if they are taught and even “normal” developers can do a great job of testing. Instead of QA testing “after development”, testing is conducted continuously as the software is developed, build-by-build, feature-by-feature. Obviously it is better to find bugs earlier rather than later. So why didn’t we always have continuous testing as part of development processes?
Several industry developments converged to make continuous testing the winning method for software engineering quality assurance. Numerous “modern” test methods such as agile testing, test driven development, behavior test methods, model based testing, and automated regression testing, to name a few, describe how quality software can be accomplished without the need for an independent QA test team. The advent of Agile development processes and DevOps infrastructures, in which development build-test cycles are accelerated to minutes or hours instead of days or weeks, has accelerated the development and adoption of automation and orchestration tools and methods. There is simply no other way to get a reasonable amount of testing done on each build-test cycle unless the tests themselves and the environment for testing are supported by orchestration tools and reliable regression tests are systematically automated as the product is developed.
Meanwhile virtualization technologies have enabled a wide variety of product test topologies to be stood up on demand at a speed compatible with the rapid build test cycles and yet not require expensive dedicated equipment configurations. It is this combination of improved test methods, Agile processes, continuous integration speed afforded by automation and orchestration tools, and low-cost test dynamically configurable infrastructures afforded by virtualization that has resulted in DevOps.
Continuous testing is “the new QA.”
At Spirent we think testing has a bright future in DevOps. You can read more about our views at spirent.com/solutions/devops
-
DevOps – It’s all about continuous testing !
I loved reading Steve Wilson’s blog post “Don’t Fear Testing Team. DevOps is Here!” https://devopsy.kinsta.cloud/blogs/dont-fear-testing-team-devops-is-here/ The idea that DevOps somehow reduces the need for testing certainly puts fear in the hearts of QA and test teams who may envision the demise of product quality, reduced test resources and even their very livelihood. I understand the sentiment of the idea about testing going away, but this is misleading.
Testing is important to DevOps. Prior to code integration to trunk, developers pre-flight test their code to make sure it doesn’t blow up the trunk. During the CI cycle and code commits from multiple developers integration testing is used to verify merges. During the continuous testing phase where nightly and weekend regression runs are automated for trunk and release branches it is testing that does the job of finding problems before the software release. During the development process the verdicts trends from testing are a primary measurement of progress. During the release deployment phase more testing verifies the release packages are ready for release. Rest assured test team, you are needed. What DevOps really does is bring testing into the mainstream of development processes, and avoids the problems created by having “big-bang” testing left to the end of the cycle, such as release delays, and quality issues.
Testing maturity is a key differentiator of best practices DevOps infrastructures. My experiences with DevOps infrastructures over the years indicate that testing maturity is a key differentiator of overall DevOps infrastructure maturity. Many organizations can automate their integrations, builds and delivery processes but still have trouble with the subtleness of test orchestration and automation. There is a vital role for testing architects and testing teams to offer their expertise for test design, test automation and test case development with DevOps. Whether the organization is using a test driven development methodology, behavior based test creation, model-based testing or whatever they use, testing is a vital part of the overall DevOps process — not only to ensure verify code changes work and integrate well — but to ensure the changes do not blow up the product. So clearly testing is still an integral part of product development and delivery.
At my company Spirent we think testing has a bright future in DevOps. You can read more about our views at http://www.spirent.com/solutions/devops
-
Crowdsourced Software Testing
As a developer or technical architect, there is no better feeling than taking pride in giving a quality product or service result to your customers. Now, with computer trends going rapidly towards cloud services and mobile devices, regular company IT development teams are finding their testing capabilities more and more limiting. They are faced with faster releases with more complex products. How do they keep up?
