— ROELBOB
Category: Blogs
-
14 Tools That Raise Web and App Performance
In the DevOps Era, teams increasingly are turning to a variety of tools to integrate, accelerate and automate. Speed and efficiency are critical. This is especially amplified in continuous delivery (CD), which enables IT teams to achieve fast and reliable releases. If successful, CD allows for a faster time to market, increased operational effectiveness and getting the right product and features to production as quickly as possible and at a high-performing level.
So, what are some of the tools that can help to raise software performance? Here is a list of 14 that we at log management platform Logz.io love, some of which will be entirely new to you.
Continuous Integration Tools
Jenkins
An extensible continuous integration (CI) engine, Jenkins is a top tool for DevOps engineers who want to monitor the executions of repeated jobs. With Jenkins, DevOps engineers have an easier time with integrating changes to projects as well as access to outputs to notice easily when something goes wrong. It’s not the fastest or the fanciest tool, but it’s really easy to use, and it has a great ecosystem of plugins and add-ons. It is also optimized for easy customization.
TeamCity
TeamCity is a major all-in-one and extensible, continuous integration server. Written in Java, the platform helps developers build and test software continuously as well as monitor externally run jobs such as cron jobs and procmail jobs. It increases the scale of automation and is gaining popularity quickly in DevOps circles. The platform is supported in other frameworks and languages by 100 ready-to-use plugins. TeamCity installation is really simple and has different installation packages for different operating systems.
Bamboo
Developed by Atlassian, Bamboo does more than just run builds and tests. It connects issues, commits, test results and deployments so the whole picture is available to your entire product team, from project managers to developers and testers to system administrators. Bamboo supports builds in any programming language using any build tool such as Ant, Maven and make, as well as any command-line tool. Build notifications can be customized based on the type of event and received over e-mail, instant message, RSS or pop-up windows in Eclipse-based IDEs and IntelliJ IDEA.
(For more, a colleague of mine has published a guide to continuous integration tools.)
Load and Performance Testing Tools
JMeter
Apache’s open-source, Java-based JMeter has a user-friendly GUI, making test development and debugging much easier. JMeter has been widely used for 15 years and is now a popular open-source alternative to proprietary solutions such as Silk Performer and HP LoadRunner. One of key benefits of JMeter is that it is scalable. When you need a higher load than a single machine can create, JMeter can execute in a distributed mode—meaning that one master JMeter machine controls a number of remote hosts. JMeter also has multi-protocol support and out-of-the-box support for HTTP, SMTP, POP3, LDAP, JDBC, FTP, JMS, SOAP and TCP.
BlazeMeter
For all of the benefits that it brings, JMeter is limited in its capabilities when used as a standalone tool. For an alternative, check out BlazeMeter, a good tool that allows you to run JMeter in the cloud, increasing scalability even further and providing comprehensive test results in easy-to-read graphs. With BlazeMeter, software delivery and DevOps teams can boost their productivity by seamlessly sharing test scripts with their peers for faster feedback, deeper learning and higher performance. BlazeMeter also has easy integrations with leading CI tools including Jenkins and TeamCity.
Taurus
Continuing on the previous point, JMeter is a great tool, but it’s not perfect. Automation and integration with other systems is difficult at times, and it’s another tool to learn and master. This where a tool like Taurus comes in. Beyond JMeter, the list of popular free and open-source tools include alternatives such as The Grinder, Gatling or Locust (also a great tool for load testing, Python-based), each having its own advantages and disadvantages. Taurus is an automation-friendly testing tool that extends and abstracts the above tools (as well as Selenium), and helps to overcome various challenges. Taurus provides a simple way to create, run and analyze performance tests. The idea of Taurus is to provide language creation/configuration scripts (in JSON or YAML) in a more user-friendly way. Not only can you create scripts with Taurus, one of its many additional benefits is that you can also use pre-existing scripts.
Application Performance Monitoring Tools
New Relic APM
Application performance monitoring (APM) tools allow you to target bottlenecks with your application’s framework. New Relic is the reigning market leader—and for good reason. It lets you pinpoint precisely where and when bottlenecks are occurring. DevOps engineers can use the tool to spend less time monitoring applications and more time building and deploying. New Relic offers web request monitoring for .NET, Java and more. It automatically shows a component-based breakdown of the most important requests. A popular, reliable tool, New Relic APM is a great choice for DevOps engineers.
AppDynamics
AppDynamics is also a great tool that enables you to monitor Java, .NET, PHP and Node.js applications. AppDynamics offers performance management solutions that let developers diagnose and fix application performance problems in real time. AppDynamics’ monitoring capabilities also help to simplify the management of complex, business-critical apps.
Dynatrace
Dynatrace helps users to monitor application servers, database servers and web services. Its feature set in terms of .NET offers code-level monitoring (including CPU and wait times), end-to-end tracing and user experience monitoring. It supports the entire application life cycle, from development environments to load testing to production.
Infrastructure Tools
AWS CloudWatch
Amazon offers AWS CloudWatch, which enables operations to see usage and performance information through their APIs. AWS CloudWatch offers metrics on data transfer, disk usage and CPU utilization that can also be configured to trigger notifications and alerts whenever thresholds are exceeded. AWS users can also inject their own metrics through CloudWatch APIs to generate a comprehensive cross-analysis.
Google Cloud Monitoring (StackDriver)
Google Cloud Monitoring provides dashboards and alerts for your cloud-powered applications. Teams can review performance metrics for cloud services, virtual machines and common open-source servers such as MongoDB, Apache, Nginx and Elasticsearch. You can also use the Cloud Monitoring API to retrieve monitoring data and create custom metrics.
Virtualization and Containers
Docker
An open platform for distributed applications, Docker is an application for DevOps engineers who want to “build, ship and run any app, anywhere.” Docker makes it easier for developers and SysAdmins to push code from development to production without the inevitable hiccups that come from using different, clashing environments during the entire application life cycle. Docker is driving the trend toward microservices, in which applications are broken into smaller, self-contained components that are packaged and deployed independently as containers.
(For those who like the platform, we have a guide to Docker monitoring with the ELK Stack.)
Configuration Management Tools
Ansible
Ansible, recently acquired by RedHat, is a the Python-powered IT automation and configuration framework that allows users to manage their systems without configuration or setup. DevOps engineers can leverage Ansible to save time in application deployment, configuration management and continuous delivery, as well as when managing systems. It recently released its latest version.
Chef
Chef is a systems and cloud infrastructure framework that automates the building, deployment and management of infrastructure via short, repeatable scripts called “recipes.” But the real power of Chef is in its use of pluggable configuration modules (a.k.a., cookbooks), nearly 2,000 of which are available within the Chef community. Chef can integrate with cloud-based platforms such as Rackspace, Internap, Amazon EC2, Google Cloud Platform, OpenStack, SoftLayer and Microsoft Azure to provision and configure new machines automatically.
So, that’s it! What do you think of these tools? Are there any others that you prefer as alternatives? I’d love to hear your thoughts in the comments below.
-
Changing Compensation Before it Kills Success
Have you ever seen the sign, “Will work for food”? If you deconstruct the premise a bit, we all work for the rewards systems that interests us. We “trade” our labor (ideally, focus our passion), for a set of acknowledgements that range from survival to pride. And ultimately it is our employer (whether the boss or the board of directors) who decides what our rewards systems will be. That is the common sense part. But when something such as DevOps enters the picture, it alters traditional thinking throughout an organization, and the ripple effects should (and eventually will) reach the HR department.
The most immediate effect of DevOps on HR occurs around talent acquisition. Finding a qualified DevOps engineer that it is possible to afford—and the company can retain—becomes the unicorn of recruiting departments from one coast to the other. The alternative is creating or adopting a training curriculum that can move a traditional programmer or engineer into a fully qualified DevOps engineer. Of course, advice varies on exactly what that is, and if it’s successful, your organization becomes a feeding ground for every other competitor to poach your folks right after training.
Because this HR acquisition problem is so great for most organizations, they can overlook some other underlying issues that continue to occur. These problems have always been there, but DevOps brings a spotlight to them that did not exist prior to its implementation. As stated earlier, we “do” what we are “paid” to do. In organizations that innovate in the software discipline, we have always “paid” that entire division to innovate. Prior to DevOps, the pace of that innovation, was slow and manageable. Once DevOps begins to take hold, the pace of innovation begins to enter the 400mph superhighway, where traditional control mechanisms no longer work.
In itself, the success of the controls over change—transitioning to a high rate of speed—will be determined by how we implement DevOps. It is the other half of DevOps that needs some focus. While our Dev organization is cranking out change exactly as we have always paid them to do, our Ops organization traditionally has been paid for exactly the opposite characteristic. We pay our Ops organization for stability. We want the trains to run, on time. We do not want a 2 a.m. phone call from an angry CEO—or, much worse, an angry customer—because our operations have “broken.” And, of course, the No. 1 culprit for causing breakage in operations is … [drum roll, please] … “change.”
In the yesteryear of Waterfall construction and manual delivery, our Ops teams had time to digest what change was coming, prepare for it, adjust to it and get it implemented smoothly (or at least pretend to). In the current day of Agile construction and DevOps delivery, I might have moved six new builds into production in the time it has taken you to read this article. Ops has no time anymore to digest and prepare for change, so if the quality of that change is not significantly better than it used to be, the angry-phone-call potential goes up exponentially. And yes, there are technical remedies to ensure quality (I will reserve discussing those for another time).
But the tradition of how we compensate remains in place. We comp Dev to innovate, and we comp Ops to remain stable. This makes not only silo thinking, but also mortal enemies during the adoption of DevOps into an organization. All of a sudden, each views the other as potentially causing harm to their ability to continue earning compensation. And they are right to see it that way, as HR has yet to alter the models to create partnerships instead of combatants.
To fix this, the compensation models must be adjusted to pay Ops teams to get involved in the software development lifecyle (SDLC) process from the upfront. The Ops teams should have extensive visibility into the testing cycle of new changes and how that works, and become the organization always asking Dev how it can “prove” its change works. The Ops teams must become the in-process auditors of the Dev teams. This keeps the Ops teams from being compensated for conditions that only exist at the end of the SDLC process, where they feel “dumped-on” by more rapid change delivery. The new comp model for Ops must focus on the number of changes successfully introduced into the production environment (i.e., the quantity of change that works), not just the stability they traditionally maintain.
The Dev team, on the other hand, must begin to be compensated only for after-action success. Dev teams must look at their code after delivery to production—and only if the code is validated (i.e., it works), is adopted by its intended customers (a completely different angle) and performs within expected timing parameters (i.e., it scales well). This becomes the criterion that is the foundation of the Dev team’s measurement of success. This puts an Ops set of glasses on the programmer instead of making him blind to those business issues. The Ops teams become the key advisors to the Dev teams on how to prove quality. And the joint discussions lead to a more sensible monitoring strategy, which is needed across the board.
There’s Compensation, and Then There’s the Bonus
Changing the point of view for the compensation is only a start. Next up: Begin to alter bonus compensation to a more lean model. Instead of large annual bonuses, where the basis for achievements often are colored by my boss’s mood over the last six weeks, organizations should increase the frequency of bonus payouts to match the duration of epic completions—or, if possible, of sprints. The idea is to tie meaningful bonus payouts to the length of time it takes to produce only successful change. If the change works, the company offers additional bonus comp; if not, then the team must improve to secure the next bonus opportunity.
Tie the bonus duration (whether to epics or sprints) to meaningful business impacts. In this way, each contributing employee understands “why” the work he or she offers has meaning to the company or its customers. This kind of bonus system allows for extreme flexibility in the priority of what teams work on throughout the year. Bosses can change priorities or shift them to match what executives see on the horizon. Employees are not tied to some list of annual work accomplishments that in a DevOps rapid world they had little shot of achieving unless they did them in a week. Instead, they have real-world goals, tied to real world time frames, where employees see the reward as they succeed at the work.
Organizations can add even more bonus comp this way if over-and-above work is needed. And “if” organizations compensate both Ops and Dev only for successful change (which implies quality in the SDLC and stability in Ops) they create a new rewards system that enables adoption rather than repel it. Beyond finding the talent they need, organizations need innovative methods of retaining them. Substantial rewards in real time might just be one way to do that.
Finally, being able to see the business impact of the change the joint effort of Dev and Ops teams produce allows for better ability to offer “appreciation” ideas as well. A simple “thank you” goes a long way in employee satisfaction, and when the executive and the team member understand “why” they are being appreciated, they go even longer. DevOps is not just a catalyst reserved to the “techies” who practice it; DevOps is instead a new way of thinking about how a company innovates. This kind of infectious thinking can easily spread to tertiary support organizations such as HR and Legal, if you let it.
To continue the conversation, feel free to contact me.
-
FLAP, Part 3: Full Data Center Provisioning
In the first two parts of this series (Part 1, Part 2), we covered server provisioning and application provisioning. Knowing a bit about each of these tools is useful (although one blog is hardly a thorough evaluation of strengths and weaknesses—I strongly suggest you do more research), knowing how they work together is the point of full layer automated provisioning (FLAP). Automating one or the other helps, but automating both—and integrating that automation—brings us to the level of “Oh, engineering needs a new test environment, and it has to be internal …” and, with a push of a button, it is delivered. That should be the long-term end goal for your organization, so staff time is free to attend to some of the many other issues that plague any given IT organization.
The thing is, these tools combine in both better and worse ways. Some are designed to work together as if one product; some take some work to get going but the combination adds value once connected; and some combinations are painful to put together and almost never are worth the effort.
One thing you’ll note is a repeated discussion of OS support for combined solutions. The reason we address it here, even though we have limited words and it’s already been mentioned in product write-ups, is simple: The limitations of these combinations are compounded. That is, if the server provisioning tool doesn’t support Windows, then the complete solution does not, no matter what the application provisioning tool does. This applies both ways, and is an important consideration for most shops, so we mention it here.
Tools Built to be Together
First off, we’ll take the easy ones. These are the combinations that were designed to work together, and do so pretty well. They vary in maturity level, but otherwise, they’re good options if they fit into your environment.
Razor and Puppet
This is the easy one. Razor was built to be the server provisioning part of Puppet. While the solution set is relatively new, and it shows in the installation phase (which has several manual configuration steps both before and after Razor install), it is maturing rapidly, and the two dev teams work pretty closely together. This is one of only two solution sets that uses a single UI for all provisioning—in this case the Puppet UI. Support for operating systems is broad, including all versions of Linux and Windows to boot. (get it? To boot?) If your data center is highly heterogeneous, or you are already a Puppet shop, it is worth looking closely at this combination. You just have to keep in mind that it is improving at a rapid rate, so those things you don’t like likely will change soon.
Satellite + Puppet + Foreman
As of version 6.0, Satellite is a collection of tools knitted together that include Puppet with Foreman. This means that Red Hat has a vested interest in making certain the two tools work well together. And they certainly seem to. While the operating systems support is limited to Red Hat OSes, the solution is pretty rock solid, and like Razor and Puppet, it is improving at a rapid rate since the switch to this new platform. The UI is independent of the tools underneath, and this is the other solution that offers a “single pane of glass” for all installation activities. The one catch with Red Hat, as mentioned before, is that the functionality of underlying systems is only supported when used with Satellite. How that fits into your organization’s plans is up to you.
Close Enough to be Second Cousins
There are some combinations that, while not officially interdependent, have a long history of mutual support, and thus make them strong combinations. These tools tend to come with straightforward directions to make them work together, and those instructions generally work well. Once paired, they can manage complete automation in a relatively integrated fashion.
Note that these combinations are not even a little exclusive. They are pairings the project teams saw made good sense, and have grown out of closeness. Both sides of these solution sets interoperate with other tools (see the next section).
Ansible + Cobbler
Ansible and Cobbler both trace their genesis to Red Hat, and they share some employees/team members—and have for a long time. That makes it natural that the two are closely integrated. Before Ansible started discussing all of the other server provisioning tools out there, it already had Cobbler integration.
As one would expect, there is Ansible setup, Cobbler setup, and then integration. But once combined, Cobbler can do for bare metal, Ansible can do for cloud, and together they provide a full-stack solution. Since both support most operating systems, the final solution does, too.
Juju + MaaS
Juju and MaaS have interoperated for a long time. Both are projects of Canonical, so it is no surprise that their interoperability continues. Years ago this author used them together and it was … difficult. But that was years ago, and just in the last year both projects have made strides in documentation and ease of use. Like most solutions in this category, there are separate installations and then integration configuration. Since both support a large number of operating systems, this solution can manage the full data center also.
Their traditional home, and the one you’re most likely to be comfortable with them in for the short term, is Ubuntu-heavy shops. But they do support most platforms out there, so it’s worth the time to look into them even for non-Ubuntu shops.
The Extended and Mixed Families
Then there are the ones that have not forged solid one-on-one relationships, some because they are forging a separate path and some because they want to interoperate with everyone. Note that some of the above systems meet this category—Puppet is integrated with most of the server provisioning tools out there—but due to space considerations, we’re not going to go over those products again here. This list contains both application provisioning and server provisioning tools not mentioned above.
Chef
Chef has put the bulk of its focus on cloud and virtualization, with the notable exception to this trend being in AIX, where it does support server provisioning. “The bulk of its focus” is not absolute, though, and that’s on purpose. Chef has had presenters at the annual conference talk about using it with server provisioning tools, so it isn’t unaware of the integrated solution space, just that’s not where their focus has been.
SaltStack
The Salt crew is happy to support everyone. While they don’t talk loudly about whom they are integrated with, the product is in use with most major server provisioning tools out there. As mentioned in the SaltStack write-up in part two of this series, cloud integration is pretty broad also, making Salt perhaps the most versatile of the application provisioning tools with regard to full data center automation.
Stacki
Originally built to interoperate with SaltStack, Stacki (disclaimer: this author’s employer) has moved on to support most of the availableapplication provisioning tools. The catch with Stacki is that some are supported with sample integrations and video tutorials at the open source layer, while others require a license and engagement of Stacki core engineers to obtain support. So there is a video showing Ansible and open-source Stacki working together and talking about the integration, but it would require a paid license and negotiation to get SaltStack or Chef with Stacki. Because of the way Stacki is designed, it is perhaps the easiest of all these products to integrate—simply add the appropriate “Pallet” and tell Stacki to include it in installations. It is getting the correct pallet that requires licensing for some platforms.
Conclusion
Since any server provisioning tool that allows users to customize the install can have the agents for a given application provisioning tool be the last step of a given install, these overviews are at least a little bit arbitrary. To be precise, they are easier to use in the given configurations, but all application provisioning tools can be forced to interoperate with any of the server provisioning tools listed.
The point of this series was really to help those looking into full data center automation quickly get up to speed on what’s available in the marketplace, and as much information as a few 1,200-word blogs will allow to help shorten the list to ones relevant to your organization. It is by no means complete, and there are issues and concerns with each of the products that only further research will bring to light, so don’t stop here.
-
Continuous Testing Live Network DevOps
My earlier blog, “From Laboratories to DevOps: Factories for Continuous Testing,” discussed the importance of and suggested practices for a fully automated, orchestrated software DevOps laboratory infrastructure, akin to a modern factory staffed by robots churning out products at a dizzy pace. The blog applies to development of software for networks, and the practices increasingly are important as the network becomes more software-based thanks to software-defined networking (SDN), network functions virtualization infrastructure (NFVI) and cloud-based networking. The edges of mobile networks, even the radio access networks, are becoming more software-dependent due to the need to position computing resources closer to mobile customers to resolve latency, bandwidth and spectrum concerns.
However, the need for highly automated software processes does not stop within the development lab.
At a recent DevOps Networking event in Santa Clara, California, Google Network Operations Manager Anees Shaikh indicated that Google’s live network has many thousands of nodes and management elements, supported by millions of lines of software-based configuration files that are updated tens of thousands of times per month. The sheer volume and speed of software updates for large networks is not feasible using manual methods. Automation tools for live network software updates are vital.
Typical live network DevOps environments use DevOps continuous tool chains such as Git for software version management, YAML for defining abstract configuration definitions, Jinja2 to translate the configurations into concrete templates that can be consumed by real devices and Ansible for automating the distribution and activation of config files to the network nodes. But what about testing? How can a network operator be confident that the many updates will perform as expected once deployed across the live network? This requires much more than simple syntax and semantics checking of the config file entries.
Recommended Practices for Continuous Testing of Live Network DevOps Environments
For continuous testing of live network DevOps environments to work, here are some best practices to consider:
- Lab-to-live transition: Prior to deployment of any updates to the live network software updates should be tested in a lab environment using production-equivalent equipment and topologies.
- What to test? The actual tests to be executed depend on the risk of changes that are being deployed. If a small, low-risk configuration change is all that is being deployed, then a simple configuration check may be sufficient. On the other hand, if a change has the potential to disrupt the network performance significantly, then a much more comprehensive performance test is needed.
- Test orchestration tools: The test environment prior to deployment will vary depending on the tests that are selected. Test environment orchestration tools are needed to set up network test topologies that match the tests rapidly—and release them just as rapidly so the test resources can be used for the next tests.
- Elastic on-demand topologies: The ability to scale test resources horizontally and vertically, on-demand, is essential to be able to match the speed of testing required for short duration deployment cycles.
- Automate live network tests during deployment: To verify software updates do not interrupt or degrade network performance, functional and performance tests between pairs of nodes across the network mesh are required as the updates are deployed in stages across the live network nodes, clusters, centers and regions.
- Automate live network testing after deployment: After verifying that the deployments are successful, network tests need to be scheduled periodically to ensure latent problems with the new software do not impact the network.
This is just a partial list of suggestions for continuous testing essentials for live network DevOps.
-
DevOps and Continuous Delivery: Not the Same
DevOps and continuous delivery are the same thing, right? Far from it. We peer through the fog to understand the difference between the two related but distinct methodologies.
We live in an era of apps. More and more of our personal, social and work life is governed by apps—communicating with one another through social apps, booking cinema tickets through your mobile device or scheduling a business trip through the airline app. This digitization of services has arrived. Organizations that understand this sea change have a unique opportunity to jump ahead of their competitors, acquire customers more quickly and grow revenues.
The battleground for this service development is DevOps. It’s not a product, but rather a methodology for helping organizations build teams and build software. DevOps is a culture, movement or practice that emphasizes the collaboration and communication of both software developers and other IT professionals while automating the process of software delivery and infrastructure changes. It creates a culture and environment where building, testing and releasing software can happen rapidly, frequently and more reliably.
Organizations are marching with their feet. According to a recent study, market demand for DevOps skills is growing, and DevOps engineers are among the highest paid IT practitioners today. And it looks set to stay that way. Research in the U.K. shows that 30 percent of 300 organizations surveyed in the run up to the 2015 DevOps Summit had either started to implement a DevOps strategy or had already merged Dev and Ops, with another 13 percent planning to go down this route.
Two Related Children: DevOps and Continuous Delivery
So now we arrive at continuous delivery. If DevOps is about accelerating the delivery of new, high-quality services, surely continuous delivery is the same thing, isn’t it? In other words, ensuring a steady stream of new services? You might look at it as a manufacturing production line: DevOps is the machine that builds the service, while continuous delivery is the conveyer belt that rolls the services off the production line—one big unified service development cycle.
That’s not the case. Although the distinction between DevOps and continuous delivery can be a little grey and cause confusion, there is in reality clear water between the two. Continuous delivery has become an essential ingredient for teams doing iterative and incremental software delivery. It is an approach whereby teams ensure that every change to the system can be released, and that any version can be released at the push of a button. Think of it this way: Continuous delivery aims to make releases dull and reliable, so organizations can deliver frequently, at less risk, and get feedback faster from the end users until deployment becomes an integral part of the business process and competitiveness of the enterprise.
Gartner summed up the distinction between DevOps and continuous delivery very neatly with a report that stated, “DevOps is not a market, but a tool-centric philosophy that supports a Continuous Delivery value chain.” The analyst argues that bimodal and digital business strategies stimulate demand for improved speed and effectiveness of software delivery.
Continuous delivery and DevOps do share common traits, though. Both methodologies are aimed at agile and lean thinking: each delivers small and quick changes; each relies on tight business an IT collaboration; and each share the common goal of faster time to market for new services.
However, the goal of DevOps is merge dev and ops roles together—and the processes they manage—to achieve business goals. It’s all about a common, shared culture and enhanced collaboration. About clearly defined business processes.
Automation has an essential role to play in both DevOps and continuous delivery. Automating DevOps helps your organization manage and secure continuous delivery to the business and IT users. The result is a faster, low-risk migration to a new world of composite applications, faster business innovation and better support for rapidly changing business process needs.
-
Rookie Mistake: Not Managing Mobile Customer Feedback
Managing mobile customer feedback effectively could be the single-most important success factor for your company and customers. Call it addiction or habit, but every passing day we encounter new data confirming the growth of mobile app usage and total hours spent interacting with them. In the “2015 U.S. Mobile App Report,” the CoreMetrics team has shared some useful data metrics regarding digital media consumption growth and app usage. A couple of notable themes or trends are:
- Americans across all age groups spend increasing time on Mobile Apps: By mid-2015, Americans on average had spent approximately 68 hours per month on their mobile devices. That number rises even further to more than 90 hours per month among millenials.
- A whopping 50 percent of time is spent within a single app and about 90 percent on the top five apps: Not only is the average time spent on mobile app usage increasing, it is concentrated in a few limited apps. A staggering 50 percent of their time is spent working within a single app, and that number tapers off rapidly as popularity, utility and quality decreases.
It is evident from this data that you want customers to include your app on their home screen. That means not only having a compelling utility, but also ensuring that the app is of the highest quality. In this article I will illustrate how you can accomplish that by incorporating mobile customer feedback loops in every stage of the development cycle and involving and engaging customers through your path to production. Along the way, we will look at various open source and commercial tools that enhance the ability to collect feedback at various stages of the continuous delivery pipeline.
Typical Continuous Delivery Pipeline
Most agile or lean software organizations have an automated continuous integration and delivery pipeline that helps manage the flow of release artifacts to production. While there might be nuanced variations across teams and enterprises, the typical path to production for apps or services is represented through the steps shown:
It will be useful to review this pipeline as we explore options to incorporate feedback throughout the path to production.
Continuous Delivery and Mobile Customer Feedback
Not too long ago, most negative user feedback consisted solely of poor ratings, comments and rants on social media platforms. Without having context, metrics and data, most development teams were left guessing on how best to reproduce and determine extent of the issue. This usually resulted in issues and features getting prioritized in an ad hoc manner.
Today, numerous open source and commercial tools are available that help solve the feedback problem. Most of them can be categorized in the following broad areas:
- Instrumentation
Despite recent reports that Facebook made its Android App crash to test user loyalty, most app crashes are not of the deliberate nature. There are a plethora of crash testing and reporting solutions, the most popular being Crashlytics (part of Twitter Fabric), Apteligent (formerly Crittercism), TestFlight (Apple) and HockeyApp (Microsoft). They each provide a client-side SDK to capture and report crash reports and a server-side service to to manage and analyze those reports. - Phased Releases – Alpha, Beta, Staged Rollout
Android application developers can take advantage of the alpha, beta and staged rollout capabilities available as part of Google Play. For alpha and beta tests, production APK is not required. Developers have a number of options to target specific groups for their alpha and beta tests. They also can utilize feedback channels available within the app to obtain feedback. Staged rollout allows a developer to progress from alpha and beta testing to production in a phased manner, constantly collecting feedback and metrics, comparing against baseline (if available) and progressing to 100 percent in production based on satisfactory results. Developers on the iOS platform can avail of beta testing capabilities provided by TestFlight and can invite up to 2,000 beta testers via email to test the App pre-production. - Direct In-App Feedback
Gone are the days when email, messenger or app store comments provided the only semblance of feedback. Customers increasingly expect to communicate via the app, and provide feedback without leaving it. Meanwhile, mobile app developers have better to tools to scale that feedback and manage it effectively to address bugs, prioritize features and get context—screen shots, system information, etc.—for specific feedback. HelpStack is an open source framework that makes in-app customer feedback relatively easy, so they can report issues, attach screen shots, embed device information and more. Customer feedback can be reported via email, or help desk integrations are available.
Apptentive, Crittercism, HelpShift, UserVoice and Parature are some of the notable commercial solutions that enable in-app feedback. - Metrics & Performance
When business analytics do not provide provide adequate answers, diving deeper into technical data might provide insights into what is going wrong. Two key mobile customer feedback metrics worth tracking is app uptime and performance. App uptime helps identify customers facing crashes while performance issues could be impacting engagement rates. Some of the tools mentioned in this article do provide an APM solution that enables monitoring of these key metrics. - Experimentation & Analytics
A/B testing or multivariate testing is a data-driven experimentation approach to promoting app feature changes. While some of the above tools enabled you as a developer to better listen to your app community, A/B split testing allows you to validate your hypothesis and invest further in features and capabilities that demonstrate lift.
Apptimize, Optimizely and MixPanel are few of the available commercial products that offer multivariate testing capabilities that can be incorporated in some situations without code modification or app updates. They also provide abilities to compare test campaigns against baseline and promote features that demonstrate positive feedback.
Incorporating Mobile Customer Feedback In Continuous Delivery
Leveraging data-driven tools throughout the continuous delivery pipeline increases the odds of a quality and high-utility mobile app. Even in situations where it is not possible to expose apps prematurely, it is possible to “dog food” your app—a term referencing a team or company that actively utilizes its own product with the intent of testing it. For example, frequent builds tied to developer commits enable alpha build testers and other team members to download and test the app. Meanwhile, beta builds can be downloaded by other members of the company. As release candidates emerge the beta community can be expanded to include members of the app community.
So take steps to incorporate mobile customer feedback loops at every step of the development cycle and garner a majority of the app user’s engagement time. Would love to hear how you and your teams are leveraging tools and techniques to build better apps and continuously listen to customer feedback.
