devopsy.kinsta.cloud

Category: DevOps Practice

Plutora Brings Order to Application Testing Chaos
One of the reasons application testing doesn’t occur as often or extensively as most IT organizations desire is because managing multiple simultaneous testing projects is complex. To help simplify the management of applications testing at scale, Plutora has updated Plutora Environments, its continuous delivery management platform, which now includes an ability to manage the availability, usage and configuration management involved in testing applications at scale.

Jeff Keyes, director of marketing, says Plutora Environments now provides a single repository through which all application testing can be managed, including identifying and provisioning available IT infrastructure resources.

Made available as a software-as-a-service (SaaS) application, Plutora Environments provides needed governance and control capabilities. Without them, Keyes says, application testing tends to get de-prioritized when developers find it too challenging to quickly access the appropriate resources.

Capabilities added to the latest release of Plutora Environments include the ability to:
- Visually associate new code commits with the features or fixes via a dashboard,
- Ensure accuracy of test coverage across high-velocity tests,
- Trace updated code to specific defects or original requirements, and
- Automatically link change requests to associated test cases.
Keyes notes a large amount of IT infrastructure allocated to testing is being wasted due to a lack of oversight. In many ways, what’s been missing in many DevOps environments is some of the management functionality found it traditional IT service management (ITSM) software, he contends, and the latest release of Plutora bridges that gap within the context of an application testing environment.

Now that developers are being held more accountable for the quality of their applications, demand for access to testing resources will steadily increase, Keyes says. IT organizations then must balance a limited amount of resources against those demands—they may want to shift responsibility for application testing to the left as part of an integrated DevOps process, but to accomplish that goal, they will need to be able to provide developers with access to infrastructure resources for testing purposes on demand. Otherwise, developers will simply opt to continue writing code, even though many of the dependencies on which that code depends has not been tested.

Application testing, regardless of era, regularly gets the short shrift. Developers are almost always falling behind schedule. One of the easiest ways to make up for lost ground is to reduce the amount of time allocated to testing. Some developers even believe agile development methodologies enable them to respond fast enough to any flaw by delivering additional updates—many of which don’t get tested either, because of a lack of time. End users, however, are becoming less forgiving when it comes to application experiences. If an application doesn’t perform as expected the first time, customers are less apt to give that application a second chance, much less a third or a fourth.

Because of that reality, in many organizations there’s more focus than ever on application testing. The challenge is not necessarily creating or running the test, but rather, how best to manage the overall process end to end.

— Mike Vizard
January 15, 2018
Red Hat Survey Sees Broad Adoption of Microservices

A new survey from Red Hat shows the extent to which IT organizations are moving to modernize their IT environments: More than two-thirds of respondents (69 percent) reported they are using microservices for both new applications and re-architecting existing ones.

The survey, which includes responses from 168 IT professionals employing either Red Hat JBoss middleware or Red Hat OpenShift platform-as-a-service (PaaS) software, also confirms that IT organizations are employing a mix of legacy and emerging programming models to build and deploy microservices, says Rich Sharples, director of product management on the middleware team at Red Hat.

Related content: DevOps Chat: Microservices & Containers – Mastering the Replatforming of the IT Infrastructure

For example, Sharples notes, many of the microservices being deployed inside the same organization are being built with both traditional Java programming tools as well as so-called new stacks of software based on containers such as Docker. In fact, a full 87 percent of the respondents are either using or considering multiple technologies to build and deploy microservices, and nearly half the respondents (45 percent) said their organization is committed to “using the right tool for the right task” versus forcing the organization to employ a single technology, runtime or framework.

The top five benefits attributed to microservices are continuous integration/continuous deployment (CI/CD), agility, improved scalability, faster time-to-market, higher developer productivity and easier debugging and maintenance. One-third of respondents (33 percent) said they have achieved two of goals by implementing microservices in the last two to six months.

However, making the shift is by no means easy. The top four challenges associated with making the transition cited by respondents are developing/implementing in-house microservices tooling, reorganizing their organizations, working with vendor subject matter experts/using a vendor as a trusted advisor and purchasing or using a platform/solution.

Microservices clearly are not for the faint of IT heart. Developing and managing a handful is relatively trivial. But managing them at scale can prove daunting to even the most skilled IT organization. Sharples says the most important decision an IT organization can make is determining what application workloads to deploy as a microservice. In many cases, IT organizations will be better off rewriting an existing application that they know how to run than trying to create an application from scratch employing microservices—not only to gain experience, but also to be able to compare the results achieved against what was previously delivered as a monolithic application, he says.

In many ways microservices are the next logical extension of concepts that were pioneered with the advent of service-oriented architectures (SOA) more than two decades ago. The difference now is that microservices tend to be more granular and are often maintained by the developer rather than a centralized IT operations team. Regardless of what level of appreciation an IT organization may have for the history of computer science, the one thing that is clear is microservices are being more broadly adopted than advocates of SOA ever hoped to imagine.

— Mike Vizard

January 12, 2018
The State of Alerting in the IT Ops world

OnPage Corp. just finished a survey of more than 100 ITOps professionals from across the United States. Our goal was to acquire a greater understanding of how well engineers in the industry are performing when it comes to critical alerting and alert management of their IT teams.

We wanted to understand the antecedents of alert fatigue for ITOps that appear earlier in the food chain. We also wanted to see how many alerts teams receive per day as well as who gets alerted. We wanted to understand how alerts are managed. And, we wanted to see how well teams analyze their actions and take those lessons forward.

In many ways, the survey was successful. We received a large number of responses from a number of industries and acquired a strong sense of how ITOps is performing across the country. Unfortunately, we also saw that for all the Chaos Monkeys and strides toward improved response to alerting, there is still a significant lack of progress.

What’s All the Buzz About?

Automated alerting is an essential component of monitoring. Automated alerts are what allow teams to receive automatically generated alerts from multiple points along their IT stack and software. In theory, this multitude of alerts is what enables teams to more quickly identify the causes of a problem and minimize the severity. The hope is, that with early recognition of the issue, engineers will be able to minimize service degradation and disruption.

But alerts aren’t always as effective as they could be or need to be. Real problems often are lost in a sea of noisy alarms. As our survey showed, this is because teams are inundated with alerts coming in via multiple formats. Moreover, the barrage of alerts leaves teams inundated and practically unable to cope.

The Law of Above Average

Our survey showed that more than 80 percent of IT teams are alerted to critical incidents via email. General best practices would dictate that email is fine for daily communication inside a business. However, for critical incidents, email is less than ideal, as it allows critical incidents to get buried under a pile of other emails. There is no way for critical issues to rise to the top of the pile.

Since our questions were multiple choice, respondents could provide multiple answers with regard to how they received notice of critical incidents. So, while email was the most prevalent form of communication, individuals indicated that they are also simultaneously receiving alerts by SMS or phone call. Our survey showed that 58.9 percent and 51.4 percent of respondents received alerts via these methods.

Already from this nugget of information, we see the opportunity for both information overload and an opportunity for missed alerts. By simultaneously receiving alerts through multiple formats, the level of irritation and overload inevitably rises. At the same time, if emails are the only form in which IT professionals receive alerts then there is a high opportunity to miss alerts all together.

How Many Alerts Was That?

The survey results also indicated that just over 41 percent of ITOps receive 11 alerts or more per day. Additionally, just over 20 percent of this group received 40 alerts or more per day. While 40 alerts is clearly more than a team can reasonably manage or should manage, this figure also goes a long way toward explaining why some alerts just get missed. If more than 40 alerts are sent to you and your team every day, it becomes very difficult to prioritize alerts and determine which should be handled first.

Perhaps to better manage this large number of alerts, many teams use escalation procedures. Our survey showed that 76.6 percent of respondents have some sort of escalation procedure in place. At the same time, the most frequent ways to escalate critical responses was through email or SMS.

The conclusions one can draw from these numbers are that, despite the large number of papers written on improving alert management, many ITOps have not been able to achieve this end. While our survey did show that just shy of 59 percent receive a manageable number of alerts, 41 percent are inundated.

Not Just Intelligence, Business Intelligence

Perhaps analysts of the industry could be more optimistic if they saw that teams were using analytics to track how well they are performing. If teams employed analytics, they would be better able to review their progress, see where they are failing to meet the grade and then embark on routines to improve. Unfortunately, this is not the case.

When asked whether their team has employed any type of business intelligence to review and analyze their team’s performance, more than 70 percent reported that they did not subscribe to any BI platform. The problem with this result is more than just a missed opportunity; it is also the loss of opportunity to fundamentally improve the business at many levels.

One of the most important reasons why you need to invest in an effective BI system is because such a system can improve efficiency within your organization and, as a result, increase productivity. Effective business intelligence can also improve the decision-making processes at all levels of management and improve your tactical strategic management.

Yet by forgoing investments in these BI tools, teams are failing to investigate their processes and methods that would improve their team and minimize alert fatigue.

A Call for Smart Alerting

The lesson can be drawn from this is that companies don’t necessarily need more alerting. What they do need is to shift toward more smart alerting.

Smart alerting means that not every bump on the monitoring screen gets tied to an alert. Instead, monitoring output is calibrated so that possibilities are aligned with probabilities and impacts. Alerts also get sent to the teams or individuals that are best able to manage the issue. Additionally, alerts are actionable and come with instructions regarding what the problem might be.

Smart alerting also means that teams use business intelligence tools such as reports and graphs and charts to determine which of their practices have been effective or not effective. Without this insight, teams are often unaware of the subtle points that could really impact their team and provide them with a way to improve their output.

Conclusion

There are a number of insights that can be garnered from our survey. I encourage you to take a moment and download a copy of the study and see what you can learn that will help your team.

— Orlee Berlove

January 11, 2018
VC Predicts Major Shift to AI-Infused Applications in 2018

Thanks to the rise of artificial intelligence (AI), the nature of DevOps is about to change as developers spend less time writing code in favor of training models based on various types of algorithms to accomplish a task.

Dan Scholnick, general partner with Trinity Ventures, an early investor in companies such as New Relic and Docker Inc., says developers in 2018 will absolutely be required to gain mastery over how to implement machine and deep learning algorithms to stay relevant. Once a model based on those algorithms is incorporated into an application, the challenge for the DevOps team will be implementing the processes needed to make sure the AI created based on those models remains tuned and optimized to the business process it’s supposed to drive.

Now that the price of cloud services based on graphical processor units (GPUs) has dropped and open source tools for building AI models such as TensorFlow are readily available, Scholnick says the cost of infusing AI models into applications has dropped considerably. Before the end the end of the year, most users of business-to-business (B2B) applications will expect that application to exhibit many of same natural language and speech recognition capabilities that already are becoming commonplace in many consumer applications, he says.

Serverless computing frameworks also will make it easier to scale those applications up and down as needed using event-driven architectures. Most instances of serverless computing, such as AWS Lambda on the Amazon Web Services (AWS), are employed on a public cloud. But as serverless computing becomes more common, Scholnick says Trinity Ventures is expecting competition for workloads among cloud service providers to become fierce as IT organizations continue to shy away from proprietary technologies.

Scholnick says the rise of serverless computing doesn’t necessarily means batch-oriented applications will disappear. But it does mean that a much higher percentage of the applications being built will be either real time or near real-time.

The challenge most organizations will face building these applications is not necessarily developing them, as the tools needed to infuse AI into applications are themselves becoming more automated. The challenge will be in collecting enough data to inform the AI model. Most of the algorithms being employed to build AI applications are decades old; the only thing that has really changed is the cost of acquiring all the data required to drive those algorithms—it has dropped substantially, thanks primarily to the rise of cloud computing. Because of those lower costs, DevOps teams can expect that not only will new applications be infused with AI capabilities, but also there will be a significant effort to modernize or replace legacy applications.

At this juncture it’s not at all clear that DevOps teams are prepared to incorporate AI technologies into application development processes. But pressure to accomplish that goal is about to increase: A new market research report published by Tractica forecasts that annual worldwide AI software revenue will increase in 2016 to $89.8 billion by 2025 from $3.2 billion in 2016. Most of that growth will be driven by 266 AI use cases distributed across 29 industries, according to Tractica. Whatever the outcome, however, the DevOps landscape in 2018 is about to transformed utterly once again.

— Mike Vizard

January 10, 2018
BMC Software Advances Multi-Cloud Agenda

Long before anyone ever coined the term DevOps there were terms such as IT service management (ITSM) and IT infrastructure library (ITIL) that at one time epitomized the state of the art in help desk. Now, as IT organizations enter 2018, much of the focus in the new year will be integrating DevOps processes with legacy approaches to delivering IT as a service that still permeate the enterprise.

To achieve that goal, BMC Software has integrated BMC Remedy service desk software with the Jira project tracking software developed by Atlassian. In addition, the latest release of BMC Remedy provides support for Amazon Web Services (AWS) and Microsoft Azure public clouds.

Nayaki Nayyar, president of digital service management at BMC Software, says the rise of both DevOps and cloud computing has clearly changed the way organizations need to manage IT. At the same time, however, traditional service desks still play a critical role in managing IT. Most enterprise IT organizations are looking to extend their investments in service desks by providing tighter integration with DevOps tools in addition to managing applications and IT infrastructure running on-premises and in a public cloud, she says.

Nayyar also notes that modern IT management tools need to be accessible to a much broader constituency. Not only do IT operations teams want to be able to manage distributed IT resources, but also developers, business leaders and others need the ability to drill down into IT processes that affect specific applications.

While there is widespread recognition of that requirement, Nayyar notes progress has been at best uneven. A recent survey of IT professionals published by BMC Software finds 80 percent of the respondents said new management approaches and tools are required for effective multi-cloud management. At the same time, however, 40 percent of the respondents readily admit their organization does not have a firm grasp on how much they are spending on cloud services.

Nayyar says it’s clear IT organizations will need to rely more on various types of cognitive and artificial intelligence technologies to manage IT at scale. To provide those capabilities, BMC Software previously outlined a strategy for infusing cognitive and AI technologies across its entire portfolio. The latest release of BMC Remedy adds support for virtual agents and chatbots, in addition the ability to automatically classify and route IT events and incidents.

Less clear is how long it will take for traditional IT organizations to modernize their environments. Large web-scale companies have already taken advantage of modern DevOps processes to transform the way IT is managed. More traditional enterprise IT organizations are still evolving, clearly borrowing many of the concepts pioneered by DevOps practitioners. But most of them don’t have the software engineering skills required to build, deploy and maintain a modern DevOps environment built from scratch. Many of them are looking to providers of packaged IT management tools to provide them with a modern DevOps experience capable of spanning both modern and legacy application environments.

Most IT organizations will employ a raft of tools to accomplish that goal. But over time, it’s also clear most of them will eventually look to consolidate many of those tools to reduce as much of the friction inside their IT environments as possible.

— Mike Vizard

January 9, 2018
How DevOps Can Help You Thrive in Your DX Journey

Digital transformation (DX) continues to be top of mind for leaders who are working to find new ways to improve their business outcomes and take full advantage of the new technologies and techniques that are now available. One topic continues to be consistent with these thoughts: DevOps.

DevOps is enabling business leaders to look at new opportunities in their market with more confidence. This is in large part due to the noticeable improvements in their company’s ability to deliver to the growing and dynamic needs of the market. New software and features can be delivered more quickly, providing exciting new opportunities.

The skills and tools necessary for DevOps are rapidly being adopted. According to a recent survey by IDC and NetApp, 38 percent of organizations are adding new DevOps skills into existing organizations as part of their IT investment and integration strategy for DX. They are applying these techniques to fulfill some of the early results that DevOps promises as well as providing a foundation for continued improvement.

DevOps and DX

While not every organization will develop the capabilities to achieve a true DevOps implementation, the following can be applied universally to support a successful DX.

Don’t Isolate the Legacy

There is a natural desire to put heavy focus on the newer modes of working exclusively. IT leaders will become very engaged in the evaluation and implementation of new tools, changes in culture, increased rate of deployments and outages and all of the meetings and planning involved.

Successful organizations work to modernize and evolve to improve processes universally. They apply new learning and new models to the entire environment, not just the innovation areas. This is not to say that more traditional applications will require a value stream that represents the next-generation applications being developed. On the contrary, those applications tend to not change very often. The intent should be to apply technologies such as configuration management, cloud and code management to these segments of the environment to deliver a more stable, predictable and self-service environment. This helps bring the entire organization up and provides a more effective working environment.

Fortunately, the latest generation of data center technology available consistently delivers APIs and SDKs that enable new and diverse capabilities. Vendors also are delivering native integrations that support control through common tools. IT organizations now have more flexibility in both their tool choices and the outcomes those tools deliver. Infrastructure investments are supporting the next generation of demands while still delivering the resiliency and features that have become critical to managing more traditional application types.

Change the Way You Measure

It is common for organizations adopting DevOps to experience a noticeable uptick in the frequency and severity of incidents, as we have seen reported in the “State of DevOps” report. If measurement of success continues to be defined by mean time to resolution and mean time between failures, it is now more than MTTR and MTBF.

Leaders have a tendency to focus on the negative, because it needs to be “fixed.” What went wrong? How can we keep it from happening again? A shift needs to occur as DevOps is implemented to help facilitate DX.

If this shift does not occur, organizations will likely begin to feel resistance to the critical changes being applied. Team members may also be less likely to explore to new ideas or to “touch that fragile system” in the value stream.

A stronger understanding and focus on what is working needs to become front of mind. How quickly can we confidently deploy a new feature? How much growth have we been able to support? How much sooner are we identifying bugs and potential issues? By exploring areas that are working, it is possible to apply these gains in other areas of the business.

Move the Whole Organization Forward

Tools may be critically important in the adoption of DevOps, but there is no substitute for the application of good leadership. Companies on their DX journey will need to re-evaluate elements of their business that may have been originally considered separate to the transition. Skills applied to the innovation segment of the transformation should be considered for application in the more steady-state elements of the business. This effort will increase the effectiveness of the organization as a whole, while also making it better-equipped to address the next phase of transformation.

— Josh Atwell

January 8, 2018
How to Ensure DevOps Success in a Distributed Network Environment
Data-centric applications are not typical, and the environment where they’re deployed are also not typical. Most often, they’re deployed into large, distributed network of machines, which compels the question of whether DevOps practices and tools can be applied to distributed applications.

To be successful, DevOps practices for data-centric applications must work smoothly even when production environment is a distributed network. For that to happen, developers and testers must define best practices for DevOps as well as continuous deployment into these distributed network environments.

Can DevOps Work in Distributed Environments?

Let’s say you’ve adopted a distributed network environment for your data-centric application production environment. Unless you’re able to successfully and rapidly develop your application for such an environment, it’s a wasted effort. Although DevOps is the solution, how many DevOps tools allow you to develop applications in distributed network environments?

Here are three critical steps to ensure DevOps success in a distributed network environment:
1. Containerize applications
2. Use DevOps tools to enable a continuous DevOps life cycle
3. Use sandboxes throughout the DevOps life cycle
Why are they important? The above three steps help you address the what, how and where aspects of your data-centric DevOps problem:
1. The What (Containers) —A distributed network environment, most likely, is heterogenous and consists of different types of systems and applications. Putting your application within containers will make them look uniform and allow you to easily move them from non-production to production environments, as well as from on-premises systems to cloud.
2. The How (DevOps Tools) — To rapidly develop your application you’ll need to quickly iterate through the program-test-deploy cycle. DevOps tools help automate these steps.
3. The Where (Sandboxes) — Sandboxes help you decide where you develop your application so that its infrastructure and environment looks the same whether you access it from development lab, or testing lab, or production data center or anywhere else.
Of these, two steps are common to both data-centric and non-data-centric environments. Containerizing applications is definitely important to data-centric applications whose requirements don’t seem to affect DevOps tools much. Nevertheless, the DevOps practices still apply fully.

However, the data-centric environment is quite different from a typical application environment. This makes sandboxing for distributed network environment all the more essential for data-centric applications.

Playing in the Sandbox

Sandboxes (also called uber containers) are basically self-contained infrastructure environments that can be not only configured to look identical to your final target deployment but also created and run from anywhere.

Developers, for example, can create a sandbox that looks exactly like their production environment—hardware, network, OS versions, software and even cloud APIs. Sandboxes can be spun up so developers can do their development and then torn down when they are done.

Similarly, testers can create a sandbox that looks like their internal IT environment to run a bunch of tests and then re-configure it on the fly to look like their external cloud environment so they can run some more tests. This will enable them to test all possible environments that the application could run in, without actually disrupting the production infrastructure.

So, what, technically, is a sandbox?

A physical sandbox is a protected space where you have complete control and others are allowed only by your invitation. You can build whatever you want with the sand and even bring your own toys into it. If you don’t like it, you can always start over.

Similarly, a technological sandbox provides a controlled environment where you can build and test what you want without disrupting what’s outside the environment. You can even install your own software (bring your own toys) to create the desired environment for your application.

In fact, a number of vendors are now offering sandbox solutions (also called “environment as a service”) that provide a simple interface for creating any target infrastructure environment and configuring it with any control you want. You can bring applications, tests, tools and automated processes to your sandbox. User access to your sandbox is highly controlled so no one can mess up the infrastructure that you are currently using in your sandbox.

You can even reserve and schedule sandboxes so that whole teams of developers and testers can share the infrastructure for hours, days and even weeks at a time. These sandboxes can be triggered even from the outside, right from your DevOps tools.

If you’re developing a data-centric application such as an internet of things (IoT) application, it needs to be deployable on distributed network infrastructure. Sandboxes provide a great way to mimic these complex, large-scale environments and allow developers to build applications that can run in this type of environment.

Wrapping Up

Ideally, it is possible to combine containers, DevOps tools and sandboxes to enable continuous deployment in a distributed network environment. The key is to package your application in containers, use DevOps tools to automate the process of moving through the various stages of your development cycle and create sandboxes to mimic the target product environment required to run those applications.

About the Author / Sreeram Sreenivasan

For more than eight years, Sreeram Sreenivasan has worked with various Fortune 500 companies in areas of business intelligence and sales and marketing strategy. He’s the Founder of Ubiq BI, a cloud-based BI platform for SMBs and enterprises. He’s also the editor of Fedingo blog, which covers a wide range of business growth topics.
January 4, 2018
Dissecting the Role of QA Engineers and Developers in Functional Testing

Until a few years ago, quality assurance was generally considered low-level R&D. As a result, quality assurance (QA) teams were characterized by a low budget, lack of attention and, more often than not, poor attitudes. Moreover, QA engineers were considered the enemies of developers and the product.

However, today, this is no longer the case, and QA is part of the entire software life cycle. Today, organizations understand how a product’s quality affects the brand’s reputation and customer retention. In the Agile ecosystem, QA engineers are no longer solely focused on testing; they also fill an important role in the planning and delivery domains. The shift left approach—where testing is performed earlier in the life cycle—requires QA engineers to have higher technical abilities and involvement in the whole CI/CD process. Nevertheless, QA managers, who usually encourage assimilation in the scrum team, as well as rapid feedback for user stories and automation, must not forget the main objective of quality engineers: to ensure a high-quality product and a seamless user experience for customers.

High quality demands testing and verifications. There are different types of testing that can be conducted to meet quality goals. In one of its blog posts, Testing Excellence describes almost 60 types of testing that can be performed on a product. We’re going to look at one that is quite clearly one of the most important types of testing: functional testing.

Different than other forms of testing, functional testing is considered one of the most sophisticated test type, and the most comprehensive one.

Same Team, Different Responsibilities

One important fact every QA engineer must remember is that developers develop and QA engineers test. Even though QA and developers are part of the same scrum team, their work, methodologies and state of mind are completely different. In cases in which the significant contribution of QA isn’t visible to the whole team, testing might be considered an easy and clear task. Quite the contrary; testing an application is much more complicated and entangled than developing it. When planning tests and executing them, QA engineers must take into account several aspects developers likely never consider.

Understanding How the User Thinks

The primary distinction between dev and QA is state of mind. While developers write pieces of code that later become features in the application, QA is expected to understand how real users think, engage customers to learn business scenarios, and confirm that the application satisfies customer needs.

Let’s look at eBay, for example. Every veteran R&D member will agree that eBay isn’t just a simple shopping application, but a comprehensive system that includes a whole set of capabilities on both web and mobile that requires attention and deeper understanding to determine whether they really function correctly. In such a rich and mature online platform, we can identify several types of users and many cases in which the application may be utilized. A seller performs different steps than a buyer and requires different functionality, while a support agent needs an entirely different version of the application. When planning tests, all these personas must be taken into consideration with a particular test plan for each. In other words, testing functionality is not just simulating user operations.

In addition, eBay is not a standalone application—its web application lives inside the browser and subsequently depends on that browser’s rules and behaviors. This is true for eBay’s mobile apps, as well, which must meet the OS constraints to function correctly, whether it’s navigating to the seller’s store or making a purchase. This makes compatibility testing an integral part of functional testing. Most developers don’t think about the project availability matrix when they code, so QA is required to contemplate all the ways the customer will engage with the application and verify functionality for each.

Viewing the Application as a Whole

Dev and QA differ in their point of view—an obvious but important distinction. While developers focus on developing user stories, QA engineers test the user story, the feature it is part of and the application behavior with the new code. Integration testing is a crucial part of the application verification effort and requires a broader view of the application and how the new functionality is part of it. End-to-end testing is the fulfillment of the above two aspects, validating that real user scenarios in the context of multiple regions in the application behave as expected. End-to-end tests increase test coverage and reduce the risk of making the new code part of the application.

Paying Attention to User Experience

As noted, testing the application functionality is merely one step in functional testing. Testing the operations available in the application encompasses another type of testing—the user experience. If a new feature is developed, but users do not have easy access to it or cannot understand how to use it, it is as if the feature doesn’t exist. QA engineers must incorporate user experience as a main metric of features so users will not have room for error, which can lead to misuse and frustration.

For instance, eBay’s “all categories” page previously displayed all the categories by default, even though some of the categories had no items. While this was the correct behavior from a functionality standpoint, it caused user confusion and mistrust. Consequently, the “all categories” view was changed to show only categories with items.

Looking at the narrow current task, developers might implement features as they are told without questioning or contemplating the specifications. In the same way, testers who test features without realizing how customers perceive them are missing an important benefit of the test.

The above three aspects—thinking like customers, testing the application as a whole and paying attention to user experience—revolve around the way developers and QA engineers relate to new features. While developers tend to follow the specification guidelines, QA engineers must “think outside the box” when it comes to user consumption of the feature.

There are two other ways in which QA work is more complicated and yet invaluable—test maintenance and test automation.

Maintaining High Quality Throughout an Efficient Release

In general, developers split their time between developing new features and fixing defects. QA work is much more diverse. A substantial portion of the release is spent on testing functionality that already has been released and used. While the main effort of designing and writing test plans for this functionality was invested in past releases, every new feature developed requires a test plan and a maintenance effort on existing tests. One of the main goals for QA managers is reducing this effort to a minimum. Teams that succeed in creating clear and reusable tests simplify the maintenance struggle and optimize the test registration time.

Eliminating Risk Through Automation

However, for QA, the most cost-effective solution is test automation. At release end—the time when QA is at its most stressed and every commit might introduce a new risk—automation testing can eliminate risk and supplies early feedback on every change. Agile teams strive for the highest automation coverage percentage so the development team can have confidence in committing changes late in the release. That said, developers often aren’t interested in investing their time in automation. They support the QA automation effort, but they don’t want to “dirty their hands” by doing testing. Whether it’s UI automation—during which achieving element uniqueness is essential yet tricky and requires developers’ intervention—or API testing for APIs without proper documentation, QA faces intricate challenges that require sophisticated solutions, which developers are, in most teams, not part of. While the dev team focuses on the status of the new feature, QA attends to the application as a whole with extra attention to identify regressions created by a change.

Join Forces for Higher Rate of Success

Testing an application requires a completely different mindset than developing it. One of the main challenges in the Agile methodology is to create developer awareness of quality, involving other stakeholders in the quality assurance activities and making the QA process visible to everyone. QA managers must understand how day-to-day QA work is different than the work of other team members so they may truly represent the complexity, efforts and eventual success. R&D groups that join forces and understand that QA extends far beyond the actual tasks of the QA team have a much higher chance of improvement and set high standards for their product’s quality. Implementing the appropriate methodologies and utilizing the right tools lead to an increase in customer satisfaction and retention rate.

About the Author / Ronit Eliav

Ronit Eliav is the Director of Product Marketing at Panaya. With a deep interest in digital transformation and the latest IT trends, she enjoys creating thought leadership around such topics as agile and continuous delivery, DevOps, quality assurance and IT modernization projects. Connect with her on LinkedIn.

January 3, 2018
DevOps Primer: Using Vagrant with AWS
One of the most common problems in modern software development is the phrase, “It works on my machine,” when a particular piece of code runs on developer’s machine but produces error on other team member’s machine. Thankfully, now there is a plugin that can help determine why something works only in particular environments, turning the phrase into more an observation than an excuse.

Vagrant to the Rescue

Vagrant is a tool for building and maintaining portable software development environments. It isolates your project’s dependencies and configuration from your projects without disturbing the tools your application is consuming. You could easily reproduce similar environments on other machines by using the same configuration as your machine.

Basic Concepts in Vagrant

Before going further, it is important to understand the following basic concepts that are essential to the working of Vagrant.

Provisioners

Provisioners provide automation for installing different software and configurations. It is useful in cases when you don’t want to install software manually to your machine.

Providers

Providers provide basic services that are required to run your machine. Although Vagrant provides support for VirtualBox, Hyper-V and Docker, other providers can also be used through appropriate plugin.

Boxes

Boxes are Vagrant environment packages that can be copied to any other machine to replicate a similar environment. Vagrant provides a repository of these boxes that can be downloaded, and you can contribute to the list by adding your own customized box.

Install the Plugin

The first step in the process is to make sure that you have Vagrant installed on your machine. As I mentioned earlier, you could use providers by installing the respective plugins. For the purpose of this tutorial, I will download the vagrant-aws plugin through the following command:

vagrant plugin install vagrant-aws

Next, verify the plugin installation through the command:

vagrant plugin list

The vagrant-aws plugin should be present in the plugins list

Set Up the Vagrant Box

After installing plugin, the next step is to install a Vagrant box. The problem is that there is no box for AWS provider. To make things work, I recommend you use the dummy box provided by Mitchell Hashimoto.

Run the following command to add the dummy box:

vagrant box add aws https://github.com/mitchellh/vagrant-aws/raw/master/dummy.box

Although I have named the box aws, you can give it any name that works for you. Once the process finishes, I am ready to create AWS instances.

Create the Instance

Run the vagrant init command to create a vagrant file:

vagrant init

Next, edit the vagrant file as follows:

# -*- mode: ruby -*-
# vi: set ft=ruby :

# Require the AWS provider plugin
require ‘vagrant-aws’

# Creating and configuring the AWS instance
Vagrant.configure(‘2’) do |config|

# Use dummy AWS box
config.vm.box = ‘aws’

# Specify configuration of AWS provider
config.vm.provider ‘aws’ do |aws, override|
# Read AWS authentication information from environment variables
aws.access_key_id = ‘ABCR2NDJKZJIAJ5QVXGJK’
aws.secret_access_key = ‘WAuZE39lKHXAJnCfbZf2y6DzRfeCsPUU/yZMRDWo’

# Specify SSH keypair to use
aws.keypair_name = ‘vagrant’

# Specify region, AMI ID, Instance and security group
aws.region = ‘ap-southeast-2’
aws.ami = ‘ami-14b55f76’
aws.instance_type = ‘t2.micro’
aws.security_groups = [‘default’]

# Specify username and private key path
override.ssh.username = ‘ubuntu’
override.ssh.private_key_path = ‘~/vagrant/vagrant’
end
end

Vagrantfile Explained

The following list explains the important variables and aspects of the above file:
- config.vm.box : Specify the box. Using dummy aws box for AWS deployment
- aws.access_key_id : Access key ID of your Amazon account
- aws.secret_access_key : The secret access key for your Amazon account
- aws.keypair_name : Your SSH keypair in AWS
- aws.region : The AWS region in which you want to create your instance
- aws.ami : The AMI ID you want to use. AMI specifies the information needed to launch a virtual machine within EC2
- aws.instance_type : The AWS instance you want to create
- aws.security_groups : Specify the names of security groups that will be applied to your instance
- override.ssh.username : The username that will be used to access the instance
- override.ssh.private_key_path : The path of your SSH private key
Launch the Instance

After configuring the Vagrantfile, I will simply run the vagrant up command. Vagrant will launch an AWS instance through the AWS AMI provided in the configuration using the credentials provided in the Vagrantfile.

You can view the newly created instance in the AWS console panel.

You can now SSH into your AWS instance or perform other Vagrant operations.

Final Words

As you can see, using Vagrant to launch a AWS instance is simply a matter of installing the plugin and then launching the instances through easy steps. If you need help with the process, do leave a comment below.

About the Author / Ali Azmi

Ali Azmi is Associate DevOps Engineer at Cloudways – A Managed PHP Cloud Hosting Platform. He is skilled in developing PHP applications particularly in Laravel. He is passionate about contributing to open source projects.
January 2, 2018
All Day DevOps, Viewed from the Outside

This year was my first All Day DevOps event. I loved it, as much as I was able to attend. I was especially interested in how DevOps worked with government, as I work in state government. We don’t use DevOps. My guess is that if you work in state or local government, your group also may not use DevOps. This may also be true of some federal agencies.

I’ve not been fortunate enough to work in an organization that used DevOps. So, when I learned of the All Day DevOps event, I had to join.

I’m a DevOps outsider looking in. Where I work the waterfall project management paradigm is followed, mainly because waterfall includes the word, “requirements.” I’ve been told that scrum and, more generally, agile project management includes the concept of requirements, but they’re just not called that.

Since we use waterfall, our progress in delivering software solutions is lengthy. I appreciate the need to get a very good understanding of what the users want, but it can be difficult to meet so many needs in a timely manner. Consequently, many users will strike out on their own to come up with a software solution. In our state department, that means using Microsoft Access. Sometimes a division will get someone to write an Access app to quickly satisfy their needs, which they start to use. This works for a while until the demands on the app become too great and they come to IT begging for help.

I really should mention that not all of this is caused by IT being slower to respond than we would if we were using DevOps. A large part of this is a consequence of not having enough money to purchase a commercial, off-the-shelf software solution, if such exists. Fiscal limitations coupled with a long queue of projects to work on and a project management process that tends to be slower makes for aggravation on both sides.

Another thing I’ve noticed about my department is that both IT and development don’t work together as well as they could. I wouldn’t characterize it as adversarial—more like just not entirely trusting of each other. This IT/Dev department has more than 200 people throughout the state, so I find it interesting to see how it plays out. Before this job, I worked in a very small organization with just two IT people—and I was one of them. It was easy to coordinate and there was no divide between IT and development. In my current position, however, everyone has their assigned tasks: IT, development, business analyst or security. There is very little overlap. I very much would like to see more.

When I learned about the All Day DevOps event, I was thrilled with the prospect of being able to learn more about DevOps. Where I work, no one ever travels to any conference, so having All Day DevOps online was a huge bonus. Also, it was free to attend, which was essential. I caught a few of the sessions throughout the day. This may sound trite, but I was awestruck with the idea that All Day DevOps was going on worldwide! This demonstrates just how widespread the DevOps movement is. When I start promoting All Day DevOps to my colleagues, I will stress this fact.

I understand that All Day DevOps will be done again in 2018. I look forward to that because this year I learned of the event only the day before, which was hardly enough time to really prepare. If you’re like me, working in a government agency—or any organization, really—and you desire to do a better job of providing your users with software solutions they’ll enjoy and in a more timely manner, as well as work better with your colleagues, I recommend that you join in at the next All Day DevOps event. I’ll be there, so I hope to see you there as well.

About the Author / Rod Falanga

I’m a longtime software engineer working primarily in the Microsoft stack. Writing applications for both the desktop and web, with some experience in writing mobile apps. I’m also a longtime fan of the BBC’s Sci-fi show, “Doctor Who.” Follow me on Twitter @Doctor_Who.

December 29, 2017