devopsy.kinsta.cloud

Tag: web apps

  • Architecting an Auth System for Applications

    Architecting an Auth System for Applications

    Applications today use many login and authentication methods and workflows. Here, I’ll share the most relevant and proven authentication workflows, which you can use as a basis for architecting and designing an authentication system for traditional web applications, single-page applications and native mobile applications.

    Authentication Workflows for Traditional Web Applications

    Traditional web applications load a web page and provide user functionality using a message-based model where a browser makes an HTTP request to a web server based on the URL in the address bar. The server responds to this request with HTML, CSS and JavaScript and then displays a resource in the browser. Along with traditional web apps, new web apps often still provide functionality in this manner.

    When a user submits a form or clicks a link or button, the browser sends a new HTTP request to the web server and changes the URL in the address bar. The server again responds by returning HTML, CSS and JavaScript and then displaying a resource in the browser.

    Browsers support only two HTTP methods for traditional web apps: GET and POST. GET is used to request data from a specified resource and POST is used to send data to a server to create or update a resource.

    Below are several workflows you can use for traditional web app login and authentication. Note that the application backend is the backend of the application (not the identity provider, or IdP) and native login form is a form built directly into the application instead of an external login form.

    Native login form to the application backend using …

    JWTs and refresh tokens stored in cookies

    Sessions

    Sessions plus refresh tokens stored in cookies

    OAuth 2 authorization code grant using …

    JWTs and refresh tokens stored in cookies (RECOMMENDED)

    Sessions (RECOMMENDED)

    Sessions plus refresh tokens stored in cookies

    OAuth 2 resource owner password credentials grant using …

    JWTs and refresh tokens stored in cookies

    Sessions

    Sessions plus refresh tokens stored in cookies

    Authentication Workflows for Single-Page Applications

    A single-page web app (SPA, pronounced as “spa” or “S-P-A”) runs entirely in the browser after loading the HTML, CSS and JavaScript source code from a web page. After retrieving the HTML, CSS and JavaScript that make up the application, the browser invokes a JavaScript framework that bootstraps and then renders the application.

    SPAs call APIs to handle user interactions and input. For example, if a user clicks a link or submits a form, the SPA might render a different page or form.

    APIs are invoked using XMLHttpRequest, a built-in browser object that can make HTTP requests in JavaScript. When the server responds, the JavaScript running in the browser updates the data dynamically without performing a  full page refresh.

    SPAs use a single request and response model. Since the browser uses XMLHttpRequest, it can support GET and POST, as well as other standard HTTP methods, such as PUT and DELETE. (PUT is used to replace all current representations of the specified resource and DELETE is used to, well, delete the specified resource.) The HTTP method chosen tells the server how to handle the request.

    Here are some workflows you can use for SPA login and authentication.

    Native login form to the application backend using …

    JWTs and refresh tokens stored in cookies

    Sessions

    Sessions plus refresh tokens stored in cookies

    Native login form to FusionAuth using …

    JWTs stored in local storage and refresh tokens stored in cookies

    JWTs and refresh tokens stored in cookies

    JWTs and refresh tokens stored in cookies

    OAuth 2 authorization code grant using …

    JWTs and refresh tokens stored in cookies (RECOMMENDED)

    Sessions (RECOMMENDED)

    Sessions plus refresh tokens stored in cookies

    OAuth 2 implicit grant using …

    JWTs stored in cookies

    JWTs stored in local storage

    Sessions

    OAuth 2 resource owner password credentials grant using …

    JWTs and refresh tokens stored in cookies

    Sessions

    Sessions plus refresh tokens stored in cookies

    When using OAuth for SPAs, note that the browser navigates away from the SPA to the OAuth provider. Once the OAuth workflow is complete, the browser is sent back to the SPA page where it quickly restarts the SPA with cached assets.

    Authentication Workflows for Native Mobile Applications

    Native mobile applications are usually installed on a mobile device or any smart device through an online marketplace like App Store and Google Play Store. Clicking the application icon on the device launches the device operating system, which starts the application and its user interface.

    Almost all native applications call APIs on the server to handle user interactions and input, such as button clicks and form submissions. Native applications often use libraries designed to directly access all of the classes, objects, functions and methods of the source code to communicate quickly with APIs and simplify API calls.

    auth mobile

    Here are a few workflows you can use for native mobile application login and authentication:

    Native login form to the application backend using JWTs and refresh tokens

    Native login form to FusionAuth using JWTs and refresh tokens (RECOMMENDED)

    OAuth 2 resource owner password credentials grant using JWTs and refresh tokens

    Note that native mobile applications can also use the OAuth authorization code grant to exchange an authorization code for an access token. This workflow works with many IdPs and is essentially the same as described in the preceding traditional web application and single-page application sections. The main difference is that the native application pulls the JWT and refresh tokens from the web view at the end of the OAuth workflow.

    Authentication System Best Practices

    The workflows listed above should give you a baseline for following authentication system best practices and architecting a sound login and authentication system for your applications. While these examples use FusionAuth as the IdP, any IdP can be used. Most IdPs are capable of a wide variety of login and authentication workflows, and the examples here are not meant to be exhaustive.

    As you’re designing an authentication system for traditional web applications, single-page applications or native mobile applications, keep in mind that some IdP login workflows can pose serious security risks. As with all things having to do with digital identities and user and data security, it’s important to choose both your IdP and your authentication workflow wisely.

  • How Are You Getting Your Cloud On?

    How Are You Getting Your Cloud On?

    With 71 percent of CIOs report having a cloud-first strategy, it’s clear enterprises are doing something. But it’s also clear that no two organizations are in the same place. While some organizations have gone “all in” by moving their production workloads to the cloud, most organizations are adopting a hybrid approach in which they are:

    • Building net-new next-generation applications that are built on cloud-native, non-relational database platforms, and
    • Selectively moving application workloads to one or multiple clouds and back to their on-premises data center.

    We at Datos IO thought we would share some of the ways in which we’ve seen organizations embrace a multicloud infrastructure strategy.

    Disaster Recovery

    One of the easiest ways to test the cloud and its promise of flexibility and agility is to set up your disaster recovery (DR) environment to act as a fail-safe mirror for the data in your on-premises environment. Cloud allows you to be more flexible in technology choices and offers elastic expansion of capacity. A cloud-based DR environment can be a conservative strategy to test the waters and prepare your organization. For more adventurous folks, you can think about building your DR strategy to focus on multiclouds.

    Test/Dev Environments

    In their journey to the cloud, many customers are trying to minimize their risk by moving test and development environments first. One way to do this is to move test/dev instant workloads for your batch analytical, integration and processing workloads. Some organizations have moved their production workloads off-premises, and could repurpose their on-premises infrastructure for test and development environments.

    Cloud Bursting

    As enterprises transition to the cloud, they are leveraging on-demand capabilities to transition analytical processing for peak processing hours, such as quarter-end financial reporting. In such cases, companies want to stand up an environment to offload processing and probably merge changes back to on-premises when operations are completed.

    Here are a few examples of how organizations are deploying new applications in the cloud.

    Data Lakes

    Some organizations use the cloud as a data lake, where data of all varieties—from different databases to different applications—go to a single environment. Big data file system technologies such as Hadoop are custom-built for data lake environments and offer a cost-effective way to store and process all the data. These new applications reside in the cloud and must be protected.

     Next-Generation Mobile and Web Applications

    Companies such as Netflix frequently use non-relational databases such as Cassandra and MongoDB to build new web 2.0 applications. A lot of new websites use MongoDB because of its support for objects, which works well in building a web catalog. Similarly, Cassandra is popular for applications such as web user profiles because of its ability to support distributed replication and fast write throughput.

    If one wishes to build a new application for the cloud, using a non-relational database is highly recommended.

    — Michael Colby

  • The Challenges of Digital Experience Monitoring

    The Challenges of Digital Experience Monitoring

    One of the great ironies of DevOps is that the better you are, the less likely it becomes that anyone will ever see your best work. If the code is supremely elegant, the app runs flawlessly, the servers do not lag and users and managers rarely say anything because that is exactly the experience they expect.

    People become aware of you only when something goes wrong—and at that point, they are rarely full of compliments. The experience means everything to them.

    Digital experience is the foundation of how online companies compete in the contemporary global market. That is why digital experience monitoring (DEM) has been recognized by Gartner as the most significant foundational process within application performance monitoring (APM). DEM is designed to improve not only your app but the entire experience that the user has in interacting with your brand online.

    If you’re unfamiliar with the DEM category, check out my primer here.

    DEM is the technological response to one of the oldest problems in business: not being able to be everywhere at once. On the digital side, the challenges of delivering the best possible performance across the widest scope fall into three spheres:

    • The problem of divergent user tech
    • The problem of network propagation
    • The problem of splintered responsibilities

    Let’s take a closer look at how DEM engages with each problem on its own turf.

    Divergent User Tech

    The mobile OS wars have settled on two champions for the time being: iOS and Android. As of February, those two account for 99.6 percent of mobile sales. Of course, from a DevOps perspective, nothing is that easy.

    Within iOS, around 79 percent of devices are on iOS 10, and another 16 percent are on iOS 9. The other 5 percent are scattered across various earlier versions. On the Android side, version 6 Marshmallow has the largest share at 31.3 percent of the market, and there are a plethora of other versions out there. Then you have to worry about the mind-boggling variety of hardware.

    For every combination of device and OS running your app, there are also users going to your mobile website on a handful of browsers. The numbers for February 2017 show 55.2 percent of the market is on Chrome, and 29.84 percent are looking at you through Safari. That leaves 14.96 percent of the market to be divided up among the default Android browser, Opera Mini, IE (at 1.02 percent, as hard as that is to believe) and many other contenders. It is significant to note here that the market leader did not even exist five years ago, so any apps you build now must be designed to work on browsers that haven’t been invented yet.

    The second subset of problems within the user tech consideration is geographic variation. How your app runs on 4G LTE in the city will be very different from 3G in rural areas. While Atlanta was judged to be the fastest city in the nation with downloads averaging 16.9 Mbps, users in the capital of Washington, D.C., can only work at 9.2 Mbps.

    The final subset covers issues related to the connection medium: Wi-Fi, cellular data, NFC, Bluetooth, ZigBee, WiMAX, HSPA, EV-DO, etc. It should be clear that there may be wide variations in digital experience based on the connectivity medium of various features within your app.

    Network Propagation

    It is not always the user’s fault. Old-school IT support tends to start with that assumption, but there are times when the problem originates within your own network or results from network latency generated by external errors.

    Everything that is happening on your servers, from DDoS attacks to viral videos to presentations in the conference room, could impact the performance of your app. As microservices becomes the dominant design model, it has become more common to see distributed applications get hung up in communications across the network. How will you track down which team is at fault or even assign fault when communications between microservices break down? The bigger question is, how will you correlate what happened on the front end with what happened on the back end?

    Splintered Responsibilities

    The second set of problems implies the third, which has to do with establishing a more unified view of your digital experience beyond just the IT world or whichever departments are involved. If your app involves contractors or third parties, the bigger view is larger than your company itself.

    It takes a paradigm shift to see your app as a unified tool. You have to assign a central command that takes a unified approach to keeping internal and external teams in agreement. This is the level where a team will determine the proper processes to launch at a moment’s notice and mitigate the damage from issues in real time.

    A central command outpost running DEM is needed to ensure a swift diagnosis of the performance anomalies. Then the DevOps team can resolve the issues and deploy fixes with minimal disruption to whatever the end user wants to do. Ideally, you want to do it with zero downtime or at least an SLA of nine 9’s (0.0864 milliseconds of downtime per day).

    About the Author / Kevin Goldberg

    Kevin Goldberg works in marketing at SolarWinds working on the Monitoring Cloud team. Prior to SolarWinds, he worked at AppDynamics and WalkMe. Follow his ramblings on Twitter at @Kevin_Goldberg.

     

  • 7 Reasons to Move to Parallel Testing

    7 Reasons to Move to Parallel Testing

    While development operations are more scalable now than ever, there is one area that continues to constrain the ability of organizations to scale: functional testing.

    Most organizations still rely on sequential tests to deliver quality assurance. No matter how rapidly a business can produce great products, its ability to bring them to consumers quickly is limited by its dependency on a slow, tedious testing process that does not adapt well to changing development needs or ever-faster business processes.

    That, at least, is the way things traditionally have been. But today there’s a replacement for sequential testing: parallel testing, which lets you leverage the cloud and virtualization to perform more tests at faster speeds than your physical infrastructure or homegrown testing grid could support on its own.

    What is Parallel Testing?

    First, a definition: Parallel testing is the process of running multiple test cases on multiple combinations of operating systems and browsers at the same time. The process is automated, and it often runs on virtual machines. For the QA professional, no additional effort is required once the appropriate test scripts have been developed. All the testing solution needs is the test suite and a pool of resources.

    Granted, new does not always equal better. But it is hard to make an argument against parallel testing, especially if you leverage a robust testing cloud. Here are seven key advantages of parallel testing to consider over sequential testing:

    You Can Test for Wider Compatibility

    Because parallel testing is faster, it allows you to test for wider compatibility in the same amount of time. If you’re releasing a new app and want to test the user interface (UI) functionality, you don’t have to limit yourself to testing against only the most popular OS and browser combinations or mobile devices—you can test all the combinations that are important to your users.

    You Can Radically Reduce Testing Time

    Good software testing must be thorough and comprehensive. In a sequential testing environment, it can be (and usually is) a very time-consuming process. Parallel testing can divide invested time (roughly) by the number of test machines being used in parallel, so that overall testing time is a fraction of that required for sequential testing.

    For example, if you are running 10 concurrent tests, you can reduce your testing time by a factor of about 10. This can cut days or even weeks from delivery time. This reduction in testing time also allows you to make more efficient use of your engineering team. They are freed from a set of necessary but time-eating and repetitive tasks, allowing them to concentrate on more forward-looking and innovative projects.

    Parallel Testing Optimizes Continuous Integration and Delivery

    Modern development best practices such as continuous integration and delivery require frequent and rapid functional tests. The execution of pre-scripted tests on a web or mobile app saves considerable time, plus having test data accessible in detailed reports is valuable to development teams who can use this information to quickly identify issues. Parallel testing in the cloud allows you to execute test suites continually as developers submit and integrate code changes throughout the day.

    The Cost per Each of Your Tests is Less

    Because parallel testing is automated and and can run at high concurrency in the cloud, the cost per test is considerably less. Leasing test time on a cloud testing service generally will be much less expensive than building and maintaining your own test infrastructure.

    Even the common practice of using development hand-me-down computers as test machines has its costs and practical limits. For example, as used systems age, they require more maintenance and they become less capable of duplicating real-world deployment conditions. Cloud-based testing grids mean environments have high availability, and they are up-to-date, making parallel tests the better option.

    It Supports Testing Science

    Parallel testing enables teams to focus on making tests repeatable, measurable and accurate. By simply testing faster, you can test more, and testing more produces more actionable data to find bugs and also improve test cases. Automated parallel testing delivers strong data on test results with key indicators of where tests can be improved. The net result is a test process that is more scientific.

    You Can Still Perform Sequential Tests, If and When Needed

    Parallel testing simply offers a different framework for testing, which can easily accommodate most or all of your existing test regime. This means that transitioning from sequential to parallel testing does not force you to sacrifice test targets, strategic testing goals or test quality.

    If anything, moving to parallel testing increases the range and depth of your test program, allowing you to better meet your test goals and it improves the quality of your testing. When you make the move to parallel testing, you are not abandoning traditional software testing. You are simply applying modern development practices and technology to it.

    You Do Not Need to Make the Switch to Parallel Testing All at Once

    The great thing about parallel testing is that you can transition to it in steps, starting with the tests that are most easily converted to parallel, then adapting some or all of the remaining tests after your team has had time to adjust to the parallel testing environment. Once you have started to use parallel testing and have integrated it into your development process, it is easy to expand the quantity and scope of parallel tests as the need arises.

    Conclusion: A Faster Test Run is a Penny Earned

    The faster you run a test, and the more tests you run, the more frequent the releases and the more defects you can find. The math is simple. Parallel testing in the cloud decreases the cost of maintaining test infrastructure, the cost of lengthy testing cycles, the cost of delayed releases and, finally, the branding cost of unhappy customers due to poor application quality.
    Even without changing the way you test, a shift to a parallel testing environment will provide an immediate speed enhancement—which means every test that still runs serially is a cost that can be avoided. And every test run in parallel is an opportunity to test your application better and faster at the same time.

    About the Author / Lubos Parobek

    Lubos ParobekAt Sauce Labs, Lubos Parobek leads strategy and development of Sauce Labs’ web and mobile application testing platform as VP of product. His previous experience includes leadership positions at organizations including Dell KACE, Sybase iAnywhere, AvantGo and 3Com. Parobek holds a Masters in Business Administration from the University of California, Berkeley.

     

  • Walmart OneOps Won’t Be One-Upped

    Walmart OneOps Won’t Be One-Upped

    One of the biggest names in retail also is kind of a big deal in DevOps. Walmart has developed its own platform, OneOps, designed to streamline operations and keep its application development schedule in check.

    “OneOps is an open source cloud and application lifecycle management technology platform with benefits including cloud portability so developers can seamlessly move applications, databases or entire cloud environments from one cloud provider to another,” says Brian Johnson, VP of Engineering, Platform, Walmart Labs. The latter feature allows developers to leverage the broad selection of cloud environments and pricing schemes to their advantage.

    The OneOps platform deploys Walmart website adds/changes, launches new mobile features such as Walmart Pay, and creates APIs so third parties such as Hiku can tap into Walmart’s online grocery offerings, at speed and with a lower investment in the cloud, says Johnson. The OneOps platform once only managed the Walmart.com e-commerce site, but now it also serves individual Walmart stores and supports other initiatives.

    According to Johnson, OneOps undergirds and empowers the work of developers as follows:

    • OneOps continuous lifecycle management continuously auto-pilots the process of automating application scaling and repair whenever any unforeseen changes take place.
    • OneOps promotes accelerated innovation so software engineers can spin up new environments for app hosting in minutes, since the DevOps platform skirts the need to specify the intricate details of any one cloud environment by modeling that for the developer.
    • This abstraction of the cloud environment gives developers control over the tools and technologies they select regardless of the cloud provider they use.

    Life Before OneOps

    “Before OneOps, the process of updating Walmart’s website was so tedious that we rolled out updates only every two months. The process used hundreds of manual steps and required significant resources,” Johnson says. “This was time-consuming and created room for manual errors.”

    Before OneOps, A/B testing was expensive and time-consuming, taking months to set up multiple cloud environments and conduct tests. By the time testing was done, the rationale for the new feature had waned and whatever opportunity it was created to take advantage of had passed, he says.

    At that time, creating enough virtual machines (VMs) in time to handle peak shopping events and traffic would have taken teams of people months to complete, at the risk of exorbitant operating costs, Johnson adds.

    The OneOps Afterlife

    OneOps now fully manages the Walmart e-commerce platform such that in a typical day the site sees more than 1,000 new software deployments, which developers execute on demand in a span of minutes on average, says Johnson.

    OneOps serves a variety of customers such as consumers of Walmart.com, who on black Friday found their shopping experience supported by the tens of thousands of VMs OneOps was able to spin up across multiple clouds to cradle the peak traffic, Johnson says. OneOps is the DevOps and cloud platform for SamsClub.com as well.

    Solving the Single Cloud Provider Dilemma

    Walmart targeted OneOps to use multiple cloud providers easily, because of the pitfalls of being locked in to a single provider. “The fear of vendor lock-in is the major challenge to cloud service adoption. Cloud vendors like AWS don’t make it easy for customers to move from one cloud to another. The complexities of cloud service migration mean that many customers stay with a provider that doesn’t meet their needs simply to avoid the cumbersome and expensive process of migration,” says Johnson.

    Further vendor lock-in concerns, he notes, include:

    • Contracts of lengthy duration geared toward ensuring higher server utilization, maximum ROI and profitability for the cloud provider but providing little benefit for the cloud customer.
    • Lack of freedom of choice where storage space, processing power, bandwidth and tools and technologies are concerned.
    • Because it is typically difficult to find any two cloud providers that use the same tools, compatibility is an issue. “You can’t simply port your site to a new provider without rebuilding it using the new provider’s toolsets,” says Johnson.
    • “While you build and test your site on the new cloud platform, you are still paying to maintain it on the old provider, doubling your costs,” says Johnson.

    This is why OneOps abstracts the use of differing cloud providers, making porting easy.

    The Big Question Is …

    OneOps solves a unique set of DevOps challenges related to the cloud. Can you name another platform that does something similar?

  • Case Study: Sauce Labs Covers Animoto’s Testing Needs

    Case Study: Sauce Labs Covers Animoto’s Testing Needs

    Animoto’s web and mobile platforms enable businesses and consumers to drag and drop stock and user-defined visual and audio elements to create videos via a tiered monthly subscription service. Animoto updates service features with increasing frequency—some of them several times a day.

    “We have a broad range of applications and features including an online video maker application, where we enable users to import videos and video elements from all the known, established sources such as Facebook and through Dropbox,” says Siddharth (“Sid”) Jain, VP of Engineering at Animoto. “They can then export videos out to a variety of venues such as YouTube.”

    Animoto must iterate new innovations continuously into its application/platform for high-quality yet simple video creation to compete with similar solutions in the marketplace such as WeVideo and Stupeflix. This fast-paced development process is challenging, since Animoto must update separate versions of its software for the web, Android OS and Apple iOS (both iPhone and iPad). “The mobile apps are largely consumer-focused for very easy, on-the-go, frictionless video creation and sharing, while our web apps feature server audiences of professional photographers, educators and small businesses,” says Jain.

    In addition to testing service capabilities of different versions of the video maker application, Animoto must test the on-boarding/registration flow and the free product experience, where users get their first taste of Animoto. “We do lots of interesting things such as testing support for multiple currencies around the globe and running continuous multiple AB tests to ensure that the free product experience is working,” Jain notes.

    Notable Animoto testing includes user funnel testing in a simulated user flow environment starting at sign-up and continuing through video creation, feature exploration and the upgrade experience. “We test this several times a day on the different platforms,” Jain says, noting Animoto tests for the seamless success of import and export of videos and video elements to and from Facebook, Dropbox and YouTube. “The emphasis is on ensuring the quality and consistency of the user experience, i.e., making sure that nothing goes out that breaks revenue in terms of service.”

    Porting to Sauce Labs Promotes Progress

    Sauce Labs‘ cloud service helps Animoto scale up and out instantly to test Apple/iOS versions of its application so it can meet the high standards set by Apple and ensure fast inclusion of new app versions in the Apple App Store. This helps Animoto to compete by getting new innovations and features to market more quickly. “The biggest advantage to using Sauce Labs is that we don’t have to create or maintain a test lab internally,” says Jain.

    This accelerated testing speed is especially important since the app must adapt to every new iOS version while maintaining backward compatibility. “It would be a very nightmarish thing to do this all manually,” he says.

    In addition to faster testing, Sauce Labs is helping Animoto ensure that testing was delivering on its promises of software improvement through accurate feedback. “We used to be religious about doing extensive unit tests. Now we emphasize integration tests over unit tests because we got bit by unit tests that were all green even though the software functionality was broken,” says Jain. “Now we have a balance between unit and integration tests while frontloading the latter with tests of customer-facing functionality and collecting fine-grain feature use data.”

    The comprehensive testing helps to ensure that users aren’t shying away from certain features and that features are not broken. Frontloading uncovers regressions very early so the Animoto developers dedicated to fixing them can catch them before they lead to breakage.

    Sauce Labs helps Animoto shrink technical debt. “Debt rears its head in awkward ways. We do get surprised, but having good test coverage allows us to know where the app hot spots are and to have feedback to fix these and verify changes swiftly,” says Jain.

    Results: More Stability, More Capacity

    “We have been able to reduce testing time from three hours to under 30 minutes for some tests,” says Jain. “Thanks to Sauce Labs’ cloud, we added VM capacity, moving from 20 VMs to 45 VMs for the testing environment.”