devopsy.kinsta.cloud

Tag: testing

  • ‘DevOoops’ Moves: Poorly Defined KPIs

    ‘DevOoops’ Moves: Poorly Defined KPIs

    DevOps’ ultimate aim is to create efficiencies in the software delivery process. But, it doesn’t always work out that way. Sometimes DevOps initiatives lead to mistakes—something we lovingly call a “DevOoops.” CloudBees recently polled some colleagues and came up with five examples of a DevOoops in action. The following is a discussion submitted by product marketer Juni Mukherjee about poorly defined key performance indicators (KPIs).

    Scenario:

    How we define KPIs can change the game. For example, one quality engineering team defined the number of tests executed per sprint as a success metric. Humans are driven by incentives and the natural tendency of the team was to add more and more tests without even considering archival of outdated ones. Think of this: We could have a larger impact with fewer tests and, more importantly, fewer tests lower the test cycle time. So, instead of sheer quantity, we should focus on the coverage and effectiveness of tests. Similarly, a release engineering team defined the number of releases per sprint as a success metric. The number of releases reflects on velocity for sure. However, releases move bits from point A to point B without assessing the value added to business. It is critical to tie a business KPI (such as the number of new customers acquired, percentage of revenue increase, etc.) to velocity, such that we know we are investing in valuable speed and not suicidal speed.

    Management consultant Peter Drucker boiled his whole philosophy down to the following few words: “If you can’t measure it, you can’t improve it.”

    Successful DevOps leaders take this saying to heart. To understand where they’re going, how to get there and ultimately how far they’ve come, these leaders know how critical it is to establish KPIs that fit the overall objectives of their DevOps transformations. KPIs provide appropriate and continuous insight—and they plot a route to help you get to your destination.

    The big question is: Are you measuring the right things? If not, you could be spinning their wheels.

    To avoid this “DevOoops” scenario, you need to start by establishing KPIs that not only align to your objectives but can also be established so that they create a continuous feedback loop. As the project progresses, as you mature and learn more, you will likely need to maintain, monitor and possibly adjust and update these KPIs.

    What would be an appropriate test coverage goal? Teams all want to improve quality. But by how much? Seventy percent? Eighty percent? One hundred percent? This is an example of a KPI that needs to be considered in the context of your application as well as measured against your real objective.

    You can have 70 percent test or code coverage, but if that’s just treated as a measure to test as much code as possible, you may miss the mark. If you are covering 70 percent of the wrong code and the 30 percent that’s missing is critical code, you’ve put in significant effort with the risk of little or no return on quality level.

    In terms of your goals, increasing and monitoring test coverage may be an indicator of quality, but it may not measure your actual quality of work. In this case you want a KPI that’s measuring the outcome rather than the activity. Look at what is severity 1 and severity 2 defect rate in production. You can see code coverage get to 90 percent, but if you’re not seeing a decrease in severity 2 defects, you won’t really be seeing an improvement in our code coverage metric.

    Takeaways: 

    • You’re not implementing DevOps just to say you’re doing it; you need to develop goals and objectives for your initiative and align to them.
    • Once goals are defined, you need to align KPIs and success metrics to those objectives. Properly defining these success metrics and KPIs will dictate what gets prioritized when building pipelines.

    — Brian Dawson

  • Survey: DevOps Progress Slow, but Steady

    Survey: DevOps Progress Slow, but Steady

    Atlassian has published the results of a survey of 500 IT and software professionals that suggest now is the best and worst of times when it comes to DevOps.

    On the plus side, the survey found that more than half of respondents (57 percent) are encountering fewer bugs and outages after deploying a continuous integration/continuous deployment (CI/CD) platform. Nearly half (47 percent) of those who have adopted a CI/CD platform also report they make changes and receive customer feedback faster. A full 71 percent who have embraced microservices have also discovered it is much easier to test and deploy new software features, according to the survey results.

    However, the survey found three-quarters (75 percent) of development teams face issues with bugs, defects or delays when it’s time to release. A total of 73 percent of software development teams also reported spending 10 percent to 50 percent of their time on updates and upgrades to self-hosted software. On average, according to the survey results, teams rely on 4.3 tools to move code from development into production environments.

    More frustrating, 62 percent of respondents reported they resort to performing manual testing due to insufficient automated test coverage, additional manual processes or a lack of build/deployment pipeline automation. However, among those organizations that have adopted feature flagging to ease management of the application development process, 63 percent report better testing a feature and higher overall rates of software quality.

    The metric IT organizations are employing to measure DevOps success appears a little soft. A full 93 percent of respondents consider customer satisfaction to be the most important DevOps metric, but 60 percent admit it’s difficult to measure customer satisfaction for the features that have been developed.

    Sean Regan, head of growth for software teams at Atlassian, said that while there is clearly much work still to be done, the democratization of DevOps continues unabated. Rather than being limited to a few high IT practitioners, Regan said DevOps processes increasingly are being pushed into the realm of everyday enterprise IT operations. At the same time, many line of business executives are being included within those teams to ensure the software being developed achieves the desired business outcome, noted Regan.

    IT organizations are also starting to apply more engineering principles to the management of IT infrastructure, added Regan. In many cases, IT organizations are discovering a single engineer can be more efficient than 10 systems administrators. However, finding and retaining those engineers is a major challenge. The goal needs to be to create a set of DevOps processes defined by an engineer that are simple enough to be implemented by IT administrators, said Regan.

    Of course, many of those IT administrators today are schooled in less agile ITIL processes. But as ITIL continues to absorb some of the concepts and best practices pioneered by the DevOps community, the divide between engineers and traditional IT administrators should become that much more narrow to bridge.

    — Mike Vizard

  • IBM Adds Automated Testing Tool for Mainframe to DevOps Portfolio

    IBM Adds Automated Testing Tool for Mainframe to DevOps Portfolio

    IBM has added an automated testing tool to the portfolio of DevOps tools it has developed for mainframes.

    Sanjay Chandru, director of Z DevOps for IBM, said IBM Z Open Unit Test adds a continuous testing capability to the continuous integration/continuous development (CI/CD) platform based on an instance of an open source Git repository developed by Rocket Software that IBM launched last year. The automated unit testing tool automatically records and plays back test data for both batch-oriented applications written in COBOL or PL/I and CICS transaction processing applications.

    Chandru said IBM Z Open Unit Test is unique in that it eliminates the need to employ traditional debugging tools. Developers can generate test cases for COBOL CICS programs, as well as “stub out” CICS to achieving environment independence from CICS runtime during test case execution. A stub is an object that holds predetermined data, which is provided in response to calls made during tests. That approach eliminates the need to have objects that answer with real data. That means Z Open Unit Test operates without adding the overhead of a debugging tool, said Chandru.

    Any test case and the resulting program created then can be stored in any source code management (SCM) system.

    IBM is making a concerted effort to make mainframe a natural deployment target for any DevOps team, said Chandru. It increasingly has become apparent that millennials in particular aren’t biased toward one platform or another, as long as each platform is equally accessible, he noted, adding that as organizations embrace best DevOps practices to modernize their application portfolio, IBM wants to make sure the mainframe remains relevant.

    IBM launched the z14 mainframe in the fourth quarter of 2017. According to the company, 92 of the world’s 100 biggest banks use its mainframes, which handle most global credit card transactions alongside billions of ATM transactions and annual passenger flight bookings. While use of mainframes overall isn’t growing, the platform isn’t going away anytime soon.

    IBM has also enhanced the cryptographic capabilities of the z14 to make it one of the most secure platforms available and continues to invest in an instance of Linux for the mainframe. That latter capability makes the mainframe a more natural target for applications being developed and deployed using DevOps processes, said Chandru.

    Of course, the biggest challenge when it comes to DevOps and mainframes is not necessarily the technology. Most mainframe applications have been developed using waterfall processes for decades. Getting those teams to embrace DevOps processes represents a significant cultural hurdle for any IT organization.

    At the same time, however, DevOps teams who ignore mainframes are not taking advantage of an existing platform investment that has performance and security capabilities that are still unmatched after more than five decades. The real challenge is finding a way for old mainframe dogs to learn some new DevOps tricks while exposing proverbial whippersnappers to the capabilities of the most powerful mainstream platform ever built.

    — Mike Vizard

  • SauceCon 2019 Speaker Profile: Wim Selles

    SauceCon 2019 Speaker Profile: Wim Selles

    This is the fifth in a series of articles highlighting speakers at SauceCon 2019. SauceCon brings together the global community of Sauce Labs users and continuous testing experts to learn from each other and level up their automated testing and continuous delivery skills. This year’s event will be held April 23-25 in Austin, Texas.

    SauceCon 2019 speaker Wim Selles is passionate about front-end test automation and his enthusiasm is infectious.

    He started manual testing back in 2007, then began dabbling in web development to challenge himself. He soon fell in love with front-end web development, deciding that if he truly understood how code works, he would become a better tester.

    While employed at Tele2, a large telecom company in the Netherlands, Wim realized the need for an image compare tool for native apps, but he couldn’t find exactly what he needed. So he decided to roll up his sleeves and build an open source module himself.

    Wim’s SauceCon presentation, “Building Your Own Native App Visual Testing Module with Appium,” will focus on exactly how he did this. He will share the things he learned about Appium in the process, as well as how he managed to create a cross-platform solution to answer this need.

    Wim wants to make people as enthusiastic about test automation as he is. As a solutions architect at Sauce Labs, he helps customers successfully implement the Sauce Labs Continuous Testing Cloud. He conducts trainings, shares best practices and travels the world speaking (and learning!) about coding and testing.

    In addition to his passion for test automation, Wim enjoys contributing to the open source community and spending time with his family and two young daughters.

    You can hear from Wim and more than 30 other expert speakers at SauceCon 2019, the third annual Sauce Labs user conference. To see a full list of presentations or to register, visit the SauceCon website.

    — Rebecca Cramer

  • IBM Extends Cloud Reach to Include Integration

    IBM Extends Cloud Reach to Include Integration

    At the IBM Think 2019 conference this week, IBM unfurled an IBM Cloud Integration Platform that pulls together a range of tools in a single development platform that the company promises will reduce the amount of time it takes to deploy applications by making it easier to reuse integration code.

    Hillery Hunter, vice president and CTO for IBM Cloud Infrastructure, said by bringing all of its primary integration tools into a single cloud platform, IBM is effectively making an integration platform-as-a-service (iPaaS) environment available for the first time.

    IBM this week also formally rolled out IBM Services for Multicloud Management, a portal through which multiple clouds can be managed using software-as-a-service (SaaS) applications developed by ServiceNow, along with a set of IBM Services for Cloud Strategy and Design through which IBM will advise clients on how to architect the right holistic cloud strategy. The premise of both those services is that organizations soon will be rightsizing workloads to run on whatever cloud or on-premises IT platform that best suits them. The portal developed with ServiceNow provides access to dashboards and analytics designed to enable IT organizations to analyze cloud spend and manage workloads deployed on any platform, including Amazon Web Services (AWS) and Microsoft Azure.

    IBM is also extending the cloud services it provides to include the IBM Cloud Hyper Protect Crypto Service, which makes use of IBM LinuxONE mainframes to provide encryption key management built on the only FIPS 140-2 level 4-based technology.

    Hunter said IBM’s approach assumes organizations have already bought into best DevOps practices and API management to provide the means through which multiple clouds will be managed. The challenge many of those organizations are now wrestling with is implementing processes that determine where a workload not only should be developed and deployed but also moved as its characteristics evolve. To provide that analytics, IBM is employing machine learning algorithms running on distributed instances of Kubernetes to continuously track changes and updates to cloud platforms, said Hunter.

    IBM previously outlined a strategy under which it is trying to meld DevOps and ITIL-based processes to make it easier to manage workloads running on any cloud or on-premises IT environment. While the growth of cloud computing continues at an exponential rate, IBM executives this week took pains to remind conference attendees that 80 percent of enterprise-class workloads still run in on on-premises IT environment. In fact, IBM’s rationalization for spending $34 billion to acquire Red Hat is based on that fact. However, while many workloads will continue to run on-premises, IBM is clearly making a case for centralizing the management of all those platforms on IBM Cloud.

    It most likely will take the better part of the next decade to determine where different types of applications may wind up running. Predictions concerning the percentage of application workloads that will run in the cloud continues to be one of the most hotly debated topics in the IT industry. Whatever the outcome, however, it’s 100 percent certain more than one cloud platform will be involved.

    — Mike Vizard

  • Why 2019 Will Be the Year for Shift-Left Mainframe Testing

    Why 2019 Will Be the Year for Shift-Left Mainframe Testing

    While 2018 was the year for planning and implementing shift-left methodologies in testing, mainframe and server testers were, for the most part, left behind. These legacy infrastructure experts were tied down to old-school testing tools. Mainframes kept functioning, but tools and testing practices often became bottlenecks, preventing performance testing teams from testing more quickly mid-cycle and pre-release.

    Until now.

    The future of testing is in open source, and finally there are open source solutions being developed and tailored for mainframe testers. Now, mainframe testers can also shift left and become agile.

    Here are four reasons why mainframe developers and testers will benefit from shifting lift and embracing open source testing technologies:

    Frequent Releases

    Apache JMeter, BlazeMeter (open-source based), Taurus and Jenkins are ideal substitutes for legacy tools, for two reasons. These open source tools are very easy to use, so developers and testers can use them themselves. In addition, they can be automated easily—once scripts are set up, they just run, and developers can get the results and monitor the system and their code.

    These two reasons are the building blocks of what shifting left is all about: frequent releases. There is a much quicker option than developing, waiting for QA to run tests, arguing with QA about the results, fixing the code, waiting for test results again, realizing you forgot to commit an important part of the fix, sending the code again, waiting for tests and pulling an all-nighter to fix the code for the last time only to realize that your code clashes with a team member’s code and you can’t release.

    By integrating your code tests into the development pipeline, every piece of code is immediately and automatically tested, on its own and encapsulated with other parts of the code. Once the code is approved, a new version is immediately released, to the customer’s/user’s benefit. This also benefits the developers by giving them a clean slate to continue working on. If the code needs fixing, no problem—an alert is sent to the developer, who can easily identify the problem and fix it. The fixed code is immediately tested and released. Now that’s much easier and much more motivating for everyone involved.

    Fewer Bugs, Higher Code Quality

    Shifting left means more automated tests are run—and more frequently. By constantly checking the system for errors and by running tests every build, you and your team are constantly being alerted of errors, which you can then immediately fix. The result is higher quality code and much better system performance.

    Another positive outcome is less hassle for your team when developing, because you don’t have to keep dwelling on previous version errors. Those have already been fixed or dealt with, so you can focus on future plans and exciting developments, which will materialize in higher code quality.

    Cost Reduction

    Shifting left means faster and automated testing, resulting in a higher-quality product. By running more tests in parallel all the time, testers and developers can focus on developing new code, resulting in savings for the business. If you’re running tests in the cloud, you’re also saving infrastructure costs.

    A higher velocity of releases with quicker fixes and new features results in more satisfied customers, who stay with you instead of moving to your competitors. Customers stay with companies they feel is concentrating on the future, which is why Gartner’s Magic Quadrant “Visionary” recognition is considered to be a big deal for the testing industry.

    Better Team Collaboration

    The idea of automation and continuous testing might come across as being detrimental to teamwork, but the opposite is true. Even though developers become more self-sufficient with their own code, this complete ownership actually encourages teamwork. Team members will collaborate more on joint feature development through shared workspaces and dashboards, with each developer bringing the code they produced to integrate with others. They also will work together to discover the system’s shortcomings and learn how they can fix bugs and bottlenecks.

    Shift-left and automation for mainframe means more time for feature development, getting customer feedback and responding to it; quicker fixes; and faster releases. This is crucial for competing in the market, providing the best services, attracting the highest-quality workforce and maintaining product security.

    Start Your Mainframe Testing with Open Source Tools

    BlazeMeter recently contributed the JMeter RTE plugin to the open source community. This plugin enables load and functional testing on JMeter for IBM 3270- and 5250-based applications.

    Running mainframe tests enables mainframe developers and testers to finally introduce agility to their testing habits, bringing mainframe testers in line with their coworkers who already embraced shift left and open source testing toolkits.

    — Refael Botbol Weiss

  • DevSecOps Implementation Process and Road Map – Security at Every Step

    DevSecOps Implementation Process and Road Map – Security at Every Step

    DevSecOps is changing our perspective toward security and is equipping us to deal with its challenges in an intelligible manner. The systematic approach of DevSecOps helps us add value to our business processes while ensuring security.

    Here’s a summary of the key inputs and a recipe for a successful software development project. Implementing DevSecOps with the primary goal of enhancing security requires a step-by-step procedure.

    It requires the unification of culturally disparate teams of Security, Operations, Development and Test for the sole purpose of building a product marked by quality and backed by security.

    The DevSecOps process will require effective planning, design and release, which will be put together with a collaborative toolchain of technologies to bring about a common goal.

    Furthermore, as organizations continue to rely on automated delivery, avenues are created for testing opportunities beyond traditional bugs. In-house experts will be able to identify potential loopholes, design errors and even code weak spots.

    Being able to identify weak spots and flaws well in advance helps organizations maintain a competitive front in the market and trustworthiness before its target audience.

    DevSecOps Implementation Process and Road Map

    DevSecOps Implementation Process and Roadmap

    1) Planning: Planning is the first approach to any task at hand and the core focus of DevSecOps—security—begins from here. In the planning stage, DevSecOps professionals must go beyond creating feature-based descriptions. The focus should also be on security and performance, acceptance test criteria, application interface and functionality and threat-defense models.

    Tech terminology: Acceptance Test Criteria is used by DevSecOps professionals to understand user preferences and behavior to a software application behavior.

    2) Developing: Developers should approach DevSecOps with a “how to do it” approach, rather than a “what to do” approach. It is important for developers to bring together available resources for guidance, have reliable practices and a code review system in place for themselves and for others in the team to follow.

    3) Building: Automated build tools can uplift the whole DevSecOps implementation process tremendously. These tools ensure test-driven development, standards for release artifact generation and utilize tools to ensure the design aspect is in alignment with the team’s coding and security standards through statistic code analysis.

    They can also be used to identify vulnerable libraries being used in your application and replace them with new ones.

    4) Testing: Automated testing in DevSecOps should utilize strong testing practices including front-end, back-end, API, database and passive security testing.

    Tech Terminology: Passive Security Testing is a developmental stage testing where security testing is not conducted directly on the target.

    5) Securing: Traditional testing methods always remain in place in DevSecOps exercise. However, somewhere down the line, there is a tendency to identify issues toward the end of the development process.

    Through advanced practices such as security scanning, we tend to become more aware of the issues and can determine if the threat is a serious one or not.

    6) Deploying: Automated provisioning and deployment can fast-track the development process while making it a more consistent one. Infrastructure-as-code tools can perform the aforementioned audit properties and configurations and ensure secure configurations across the IT infrastructure.

    7) Operating: Regular monitoring and upgrades are the Operations team’s important tasks. DevSecOps teams ensure to deploy infrastructure-as-a-code tools to update and secure the entire organization’s infrastructure in a quick and efficient manner with no scope for human error. Operations personnel have to be especially watchful of zero-day vulnerabilities.

    Tech Terminology: Zero-Day vulnerabilities are a software breach that remains unknown to software productions personnel and can eat away at a computer program and other components such as a network.

    8) Monitoring: Constantly keeping a watch for irregularities in security can save an organization from a breach. Hence, it is essential to implement a strong continuous monitoring program with real-time to keep a track of system performance and identify any exploits in their early stages.

    9) Scaling: Gone are the days when organizations spent precious hours and money on the maintenance of large data centers. With the introduction of virtualization solutions and the cloud, organizations can scale their IT infrastructure or replace it in the event of a threat, which would be impossible to do with a traditional data center.

    10) Adapting: Continuous improvement is key to any organization’s growth. An organization will only be able to achieve the desired growth of it evolves in its practices including DevSecOps practices—security, functionality and performance. Therefore, an organization should adapt to continuous improvement and external changing trends.

    The DevSecOps is in itself a process of innovation that substantially adds value to an organization.

    Point to Remember: Security is everyone’s responsibility.

    — Veritis

  • TCoE as a Way to Level Up Test Automation

    TCoE as a Way to Level Up Test Automation

    A testing center of excellence (TCoE) can help you leverage automated testing efforts.

    The “World Quality Report 2018-19” claims that test automation is almost a heal-all concept in QA and one of the most booming trends for the near future. Still, the general percentage of automated testing in overall test efforts is quite modest. Apart from technical and financial impediments, test automation can be hampered by a number of organizational challenges. Here, testing centers of excellence (TCoEs) may be of practical value.

    Let’s find out what TCoE is, why test automation is a must and how TCoE may come in handy on your way to higher test automation levels.

    TCoE: Capsule Review

    A TCoE provides QA optimization and control due to consolidated testing best practices, tools, standardized methodologies and testing metrics. It aims to centralize QA resources dispersed around the whole enterprise, thus being more relevant for large-scale organizations with a number of departments or even subsidiaries. Capgemini research results prove that TCoE can bring such tangible benefits as optimized QA resources (people, budgets and tools), reduced testing time (by 30 percent) and costs (by 35 percent in a three-year time frame) and increased test automation rate (up to 50 percent).

    Why Level Up Test Automation

    Test automation is not just hype, but a proven way to get more from your testing. The ever-growing popularity of Agile and DevOps/continuous delivery projects necessitates that testing be performed within a shorter time frame. Test automation tools are highly efficient in performing the most time-consuming types of testing and the ones comprising large amounts of data, such as regression, performance, data warehouse or load testing. With its ability to execute thousands of test cases during every run, test automation provides coverage unachievable with manual testing resources.

    Test Automation Challenges That TCoE Can Solve

    The promotion of test automation is acute as it can result in tangible benefits. For now, the average level of automated test activities is only around 16 percent, as reported by the leading CIOs from around the world. However, it is pointless to automate all testing activities. Test automation is not a panacea for every testing project or testing type. Each project’s automated testing efforts require a careful consideration and ROI assessment, as test automation has a number of limitations. For example, it can save the testing budget only in the long run. So, there’s no need to introduce automation for small-scale testing projects. Besides, even in projects with a large testing scope, test cases that are unique for each release or source code modification, rarely repeated test cases and those with difficult-to-measure or not purely objective output shouldn’t be automated.

    However, the low rate of test automation comes not only from its natural technical impediments but also because of the organizational problems bound. Luckily, TCoE can handle some of test automation organizational challenges.

    Test Strategy Lacking Test Automation

    If the test strategy doesn’t fully correspond to software requirements and its peculiarities, it may lack testing types appropriate for automation. TCoE may be of help here as TCoE managers can assist test leads in building a more effective test strategy with respect to high-level QA priorities and help BAs to come up with multilevel software requirements, if a software requirements specification turns out to be incomplete.

    Besides, the initial part of test automation requires a significant share of testing time (choosing the repeatable test cases and scenarios, scripting and maintaining them) and budget. Without TCoE, some test leads may try to economize and strike automated testing off the test strategy, sure that the moves will help them meet the rigid deadline and budget. If it’s truly impossible to implement relevant test automation and ensure acceptable testing quality stays within time and budget, TCoE managers can suggest revising the testing time and financial limits for the project.. The alternative, of course, often results in even more expenditures because of the release of heavily bugged software.

    The Lack of Skilled Test Automation Engineers

    QA teams consist mostly of manual test engineers without code writing skills needed to script and maintain automated test scenarios, while developers don’t have the knowledge of relevant automation testing tools or the experience in creating appropriate test cases. So, neither manual test engineers nor developers can fill the gap without sufficient training.

    TCoE can help, with its managers knowing each test engineer and having an established knowledge-transfer process. TCoE managers can find test team members (preferably, manual test engineers) who are enthusiastic about test automation and provide training to them on a broadly applicable programming language (for example, Java or Python) and the most adjustable test automation tools (Selenium, Ranorex, TestComplete, etc.).

    Silo Testing Processes

    Uncoordinated and varied testing processes can hinder test automation. In different QA teams, test leads may choose disparate test management tools and test automation engineers are likely to use diverse test automation frameworks. Besides, with QA resources being dispersed around the company, it’s complicated to create a common base of automated scripts that can be reused in different testing projects.

    Efficient test automation requires unified testing processes around the enterprise, which can be attained with TCoE. From the earliest stages of its maturity, TCoE focuses on aggregation, analysis and standardization of testing processes and test automation tools and frameworks. In addition, the TCoE team captures test patterns, creates modular automated test scripts and gathers them in a library to be fully or partially used through the same testing types or similar testing projects.

    Excessive Test Automation Spend

    Test automation may be pricey due to automation software license costs. Software license purchase for almost every testing project anew can turn test automation into a nightmare for the finance department and lead to excessive testing costs. TCoE handles this challenge by providing conventional licensed test automation software tools that can be used for a number of projects.

    Also, test automation requires ongoing engineering resources for maintenance. Application under test (AUT) changes from iteration to iteration and all the automated scenarios should be checked for their validity. Only a few of them can remain unchanged while the majority needs restructuring and correction. A number of TCoE’s qualified test engineers ready to cater to continuous automated testing can be relocated to the project where they are most needed, not being bound to a distinct one. Besides, common test cases and scripts that were automated once can be reused through different projects.

    Wrong Time for Test Automation

    Another test automation pitfall is the difficulty of choosing the right time to introduce it during a development life cycle. If a test strategy implies a too-early start with scripting test cases, a test automation engineer risks finding the prepared test scripts and suites outdated in the next iteration due to requirements changes. At the same time, starting with test automation too late may result in excessive testing time and costs.

    TCoE consolidates test automation best practices and QA professionals with the deep knowledge of software and its requirements. Thus, it’s much easier for TCoE managers than for each test lead separately to evaluate a certain project and decide when it’s the safest time to develop and implement automated tests.

    On a Final Note

    The relevance of test automation is here and is predicted to gradually grow in the future. Still, automated testing has a number of constraints and trying to reach higher levels of its implementation is bound to organizational challenges. TCoE can assist in solving such organizational test automation problems as an unbalanced test strategy, the lack of test automation engineers, uncoordinated testing processes, excessive test automation costs and inappropriate time for test automation.

    — Andrei Mikhailau

  • DevTest Scales Out in the Cloud for 2019

    DevTest Scales Out in the Cloud for 2019

    It’s 2019, which means a whole new year of advancements in all areas of technology. One topic that is near and dear to my “cloud” heart is the acceleration and maturation of development and test (DevTest) scenarios and solutions in the public and hybrid cloud space.
     
    I’ve been talking with folks and presenting on the movement of Dev/Test to the public and hybrid cloud for a couple of years now, and I have progressively learned a couple of things from those with which I have had great and sometimes very frank discussions. First, I learned that the definition of DevTest has expanded exponentially, with the movement to the different clouds. Second, I found that the terminology of “DevTest” just doesn’t fit a single threaded way of thinking any longer, such as “It’s just for developers.” Let’s take a closer look at these.
     
    Back when I was a young technician just starting  in the new world of IT, I was working primarily with programmers (they weren’t called developers or devs back then) who would sit in front of their green screens, writing huge applications with COBOL and RPG, or maybe writing shell scripts on a Unix system. They would test their code in different environments that were always on the same infrastructure that the production code was running (which we all know these days is wrong, right?). They didn’t have secondary hardware or OS installations to test against because it was just too expensive to accommodate and most of the code could not support that separation. 
     
    Fast forward the clock almost 30 years, and now the developers need to have logically separated and secure environments to provide accurate and meaningful testing of the code that is “born in the cloud” as well as those created on-premises. The next few years will bring a big push for developers to have their own self-service environments, extensive code portability and more secure isolation for their code. By utilizing container technologies, automation and rapid deploy processes for their code, more development groups will eventually become their own application development as a service. 
     
    DevTest solutions are also moving far beyond just testing code. Quality Assurance (QA) is already a substantial consumer of the cloud for their load, smoke and regression testing. I predict that there will be a huge advent across all roles within IT—and not just the ones of development and QA—that will take the plunge into the DevTest solutions. They will be running release scenarios on their rapidly and continuously updated OS versions, on all of the software-defined environments, while also adding software patch testing and issue mitigations. 
     
    Knowledge transfer, training and E-Learning also will increase the use of these landscapes over the coming years, with everything from purpose-built labs, globally connected classrooms and even enterprises expanding their hands-on learning management system (LMS) offerings to their employees and customers. 
     
    Finally, I would suggest that there will be an inflection point in the coming years in which the use of private, hybrid and public clouds for DevTest—and, eventually, production—will become transparent to the user to the point that, within the overall vision of what you need to accomplish, what cloud or combinations of clouds you utilize won’t matter any longer. Eventually, for infrastructure as a service (IaaS) at least, the cloud will go the way of hardware, which has largely become a commodity in the new evolution of the modern software-defined data center.
     
    Technology is moving incredibly fast these days, especially when compared to the days of old when I was coming through the ranks in IT. The DevTest solutions will continue to evolve, progress much faster and touch more people in technology—and even other professions—than ever before. Start building a solid DevTest design and implementation plan for whatever your business and IT goals may be, and invest in the solution for all of your IT functions. By embracing and utilizing what is available in the hybrid and public cloud space for DevTest, IT will be empowering more productivity and faster execution moving into 2019 and well beyond.

    — Mike Nelson

  • Split Launches Free Managed Feature-Flag Service

    Split Launches Free Managed Feature-Flag Service

    As part of an effort to attract more DevOps teams, Split is making available a free managed feature-flag service for up to five users within an organization.

    Based on the same commercial managed service Split provides today, company president Trevor Stuart said the free service provides those five users with unlimited feature-flagging and unlimited usage in the hopes that the number of DevOps teams employing feature-flagging will rise to the point where the organization will opt to pay to consume the paid managed service. The free managed service also includes user segmentation capabilities to provide control over how many end users might be exposed to any new feature.

    The paid managed service provides access to additional capabilities such as integrations, access and permissions, team collaboration and full-stack experimentation.

    Stuart noted many DevOps teams are today attempting to incorporate feature-flagging into their processes by implementing one of more than 40 different open source projects. However, many of them quickly discover that building and maintaining a feature-flag platform is a difficult undertaking. Many developers decide that building a feature-flagging platform is a challenge they want to take on because it provides a critical set of DevOps capabilities. But when those developers leave the organization, the DevOps team is suddenly left with a platform no one else in the organization knows how to support, he said.

    Feature flags, also known as feature toggles, provide a means for DevOps teams to test and add new capabilities to their applications without having to take the entire application offline. Feature-flagging also enables phased application rollouts that make it easier for DevOps teams to assess new functionality as it is added. Any problematic update can be turned off with a single click. The downside is they generally result in more branches of applications that need to be managed carefully within the context of a DevOps process. Because of that complexity, vendors such as Split have been making the case for employing platforms that enable DevOps teams to more easily incorporate feature-flagging within a DevOps process.

    Stuart said the ultimate goal is to provide a more controlled approach to experimentation. Instead of randomly testing new features on end users, feature-flagging makes it possible to more easily test features on specific end users, said Stuart.

    It may be a while longer before feature-flagging is adopted more pervasively. But as DevOps teams become increasingly more sophisticated, there’s little doubt feature-flagging won’t become more commonly employed. The simple truth is that uncontrolled application experiments on end users usually are not appreciated. Savvy DevOps teams are finding ways to test application functions in real-world scenarios with minimal disruption to the majority of end users.

    The only real issue now is determining how best to go about implementing feature-flagging at a time when most DevOps teams already are struggling to keep pace with their existing application development and deployment projects.

    — Mike Vizard