devopsy.kinsta.cloud

Tag: testing

  • Time, Training: How to Implement DevOps Effectively

    Time, Training: How to Implement DevOps Effectively

    Successful implementation of DevOps solutions requires two very important investments: time and training.

    New technologies have always transformed how the commercial world operates. The steam engine for example, fueled the industrial revolution of the 18th century and changed the face of business forever. This dramatic shift didn’t happen overnight, though; people had to learn new skills to manage steam power and operate trains, ships and equipment that previously did not exist.

    Adopting a new system within a modern day organization is no less challenging. It takes in-house expertise and top-down leadership to crest the learning curve as quickly as possible.

    Training Your People

    To realize the benefits of DevOps, businesses need people—particularly, system administrators and their managers, who know how to operate the systems. It really is that simple, yet most companies just don’t have the skills. Training, training and more training, coupled with lots of hard work and practice, is paramount.

    Middle managers in charge of integrating the new systems are first in line for training. They deal directly with upper management as well as with the new software’s end users. To communicate the importance of DevOps to both audiences successfully, they need to understand exactly what DevOps can achieve—and how to get it up and running in line with the company’s wider business goals. They need to know what skills are needed to implement DevOps companywide and identify the right people to learn the ropes quickly and efficiently.

    Push from the Top

    A successful switch to DevOps also is dependent on support and buy-in from upper management. They don’t necessarily need to have an in-depth knowledge of the systems, but they do need to push for fast and effective implementation—and allocate the resources to make it happen. With all key stakeholders working together to reach the same realistic timelines and deadlines, the new systems will be introduced far more seamlessly. If the process is at all disruptive, people will be reluctant to change their way of working and do things the same way they always have.

    Senior management must support the system change and encourage its use across the organization by openly embracing its potential.

    Test the System

    Traditionally, implementing a new production system was a long, onerous process with lots of restrictions and administrative hurdles to jump. In many instances, a committee of key stakeholders was needed to verify and approve each step. Forms had to be filled out, a thorough backup plan would be prepared and a bureaucratic chain of command would be followed. The further removed from the actual implementation project each signatory was, the less committed they were—which often led to repetitive meetings and costly delays.

    This labor-intensive change management approval process was in place to help prevent any failing in the production plan, which could have disastrous effects. Should for example, an online retailer’s check-out or internet banking infrastructure go down, business profits and reputation would suffer significantly.

    Implementing DevOps is different. It uses a test environment to speed up the onboarding process and minimize errors. IT engineers can do dry-runs before going live to check how the new systems are integrating and if anything is wrong before it actually does go wrong. Bugs and glitches are fixed without impacting the current systems and the approval process is far faster and much more confident. All of which can speed the entire implementation process and its success.

    The potential of DevOps is inextricably linked to the people who implement and operate the new systems as well as the old. To achieve the best possible outcome, your team needs to have the experience and skills to ensure a seamless integration that enhances core business functions without losing or disrupting the organization as a whole.

    About the Author / Francis Miers

    Francis Miers is a Director at Automation Consultants – Automation Consultants (AC) is an IT services and software company with a mission to improve the speed and reduce the cost of IT project deployment by implementing automation across the software life cycle. AC develops and deploys advanced software automation solutions. Customers include BskyB, HSBC, Invensys, Orange, EverythingEverwhere and Vodafone.

     

  • The Code Doesn’t Lie, and Other Operations Mantras

    The Code Doesn’t Lie, and Other Operations Mantras

    As engineers, we spend a lot of time talking about things such as release processes, QA environments and deployments. But at the end of the day, most software systems fail because the software itself is faulty. The code doesn’t lie—you can almost always find the solution to your problem within the code itself. Of course, looking to the code for answers is a time-consuming process if you’ve deployed too much in one batch. After all, sometimes the tiniest mistake can derail a whole string of code, and having to sort through a haystack of code looking for a tiny needle is not fun.

    There are a few corollaries to this mantra that “the code doesn’t lie”:

    • “Know your data like you know yourself.” Sometimes the quality of the data itself can cause a system to behave differently.
    • “A feature is a bug in a tuxedo,” which is largely for you product folks out there.
    • “Where there is one bug, there are many.” When you find a problem and it leads to a systemic set of problems, it may seem terrible at first, but the reality is, you have the opportunity to fix a lot in a short amount of time.

    I had a chemistry teacher who used to say, “A mole is a mole is a mole,” referring to Avogadro’s number. My version is: “A bug is a bug is a bug.” A feature that behaves incorrectly (even if built to spec) is a bug. A badly performing system is a bug. A transient failure is a bug. A race condition is a bug. No matter what they look like, it’s our job to squash the bugs.

    Branch Readiness is a Joke

    A very talented engineering director once made the comment, “Branch readiness is a joke,” about the branch/development model. As 23 different teams were preparing their branches for integration onto the main trunk, code changes were flying around with very little testing/certification. We even had code that would not compile.

    When all 23 branches hit the main release branch, all hell broke loose. And this is where the actual functional, integration and performance testing began. The key problem with this model is that integration and certification happened late in the game. The other problem was that we did not abide by our own criteria of what constituted branch readiness; instead, we were constantly softening the restrictions until bad code made its way into the system.

    We moved to a trunk-based model of development where check-ins are made directly to the trunk, and all code must compile and be pretested. Continuous integration and testing are run against the trunk all day and all night. We can ship from the trunk at any moment.

    Now you may be asking yourself, What does this have to do with operations? The answer is: everything. We can control what changes get to the site. We can certify what changes get to the site. We are assured at every moment of a minimum quality in the changes that do get to the site. These smaller changes give us the ability to move bits onto the site more frequently. We are not forced into large-scale, monolithic deployments, which inherently create more risk to the site itself, often with little ability to roll back.

    Learning from ‘Branch Readiness is a Joke’

    Just because the code doesn’t lie does not mean that you can always understand all of its implications. The key is limiting the amount of code being changed in any individual release to an amount an engineer or automated system can understand. By avoiding large, monolithic releases containing thousands of changes and instead testing, verifying and considering each change individually, we can understand what the code is telling us. We can release to the trunk with confidence that what we build and deploy is going to work.

    The One-Character Change

    We live in a world of bits and bytes (eight bits per byte). One byte is the equivalent of one character. In some cases, a one-character change is benign. For example, changing the number of members of a site on an informational page from 134M to 135M (4 to 5) will not cause any harm to the site.

    But sometimes, a one-character change can be disastrous. Consider a DNS change that has a one-character mistake for www.yourcompany.com. Get it wrong and you are off the air. It is very important to understand the impact of a change going awry. If it can cause a large impact, then we need to ensure we understand the change and have a clear plan to roll it back if we need to.

    Here’s a great example of a very small change going sideways very fast: We once scheduled a maintenance to test routing traffic through a European point-of-presence. Unfortunately, the time-to-live parameter was set to a large number (hours instead of minutes) on the DNS entry. The result was that the change, once implemented, couldn’t be fully undone for hours. It was a one-character change with a bad outcome.

    Learning from the One-Character Change

    The code doesn’t lie, but that only matters if we are paying attention. Put simply, not all changes are equal. By taking the time to think about what a change is intended to accomplish and the various ways it could go wrong, we will always be better off than if we didn’t do due diligence. The one-character change can be benign, or it can be catastrophic. It’s important to figure out the possible ramifications of any changes (no matter how small they may seem) before proceeding.

    Operations tends to be the first group to get called after hours when something goes wrong. As a result, we have a vested interest in quality of code shipping to our site. By working to improve the code quality before it hits production, you can significantly reduce the number of problems you encounter.

    Start with where the code gets committed into source control. For each change made ask the simple questions: Does the code still compile? Does your application still build correctly? Does it still work as intended? By automating the process of asking these questions for every change, we gain the ability to deploy new versions of our application based on any commit.

    Once we have good code quality making it into source control, we can begin making further improvements through the use of a canary process and start focusing on things such as performance. Remember, the code doesn’t lie. If we can understand each change, then we can start preventing problems before they begin.

    This post is part of the series “Every Day Is Monday in Operations.” Throughout this series we discuss our challenges, share our war stories and walk through the learning we’ve gained as Operations leaders. You can read the introduction and find links to the rest of the series here.

    About the Author / David Henke

    David Henke has more than 35 years of experience working in technology, including senior engineering leadership positions at LinkedIn, Yahoo!, and AltaVista Company. He’s also been a founder at two different software companies, both of which were acquired. Currently, David serves in a variety of board and advisory positions with organizations like NerdWallet and UC Santa Barbara.

  • A Holiday Tradition: 2017 Continuous Delivery Predictions

    A Holiday Tradition: 2017 Continuous Delivery Predictions

    It’s that time of year. As the calendar year comes to a close, many of us are following time-honored traditions: making preparations for the holidays, planning some much-deserved time off and enjoying time with friends and family.

    In the business world, the year-end also brings its own traditions, and first among those is making predictions for the coming year. I’m not one to forgo traditions, so in this article I’ll share my thoughts on the upcoming year within the context of the continuous delivery movement.

    To level set, continuous delivery is the most evolved and modern state of software development and testing—and is core to a successful DevOps strategy. Organizations that are able to achieve continuous delivery can rapidly develop, test and release applications—with higher quality and at lower costs.

    Simply put, the ability to more continuously deliver software is one of the biggest advantages to attain in the application economy, where the battleground for consumer loyalty is most often found in the ability to deliver superior applications.

    Now that we know what continuous delivery means, let’s get to my predictions for 2017:

    1. Continuous Delivery Means Continuous Everything

    Continuous delivery is still a relatively new concept, and one that many organizations strive to achieve. As a new concept, it suffers from some of the misconceptions that often plague emerging technologies or trends. Many technology providers latch on to the term and claim to provide “continuous delivery solutions” that in reality only address a small part of the continuous delivery challenge.

    Truly delivering software continuously means that all the tasks within the software development life cycle (SDLC) must be optimized to occur in parallel and continuously. This means development must be continuous, testing must be continuous, and likewise for release and operations.

    2017 will be the year that industry embraces the importance of an end-to-end continuous delivery value stream. In 2016, many organizations began their continuous delivery journey by addressing certain aspects of the SDLC; in 2017 many organizations will begin to realize a truly open and integrated continuous delivery workflow.

    1. Democratization of Continuous Delivery Tools (via the Cloud)

    Optimizing all aspects of the SDLC for continuous delivery, as mentioned above, means that related tools need to be easier to use and easier to access. Developers who have access to testing tools (left-shift) can improve the quality of code they produce. Similarly, QA engineers who have access to design requirements sooner can create more accurate test data and scenarios, earlier.

    However, for this to occur, the tools have to be both easy to access and simple to use. Developers have avoided complex testing frameworks, but with simple, easy-to-use tools available—often in SaaS forms—developers can certainly incorporate testing into their methodologies.

    In 2016, CA acquired BlazeMeter, which pioneered the notion of cloud-based performance testing employing a simple and easy-to-use (yet very powerful) product. We expect to see this trend expand into other tools within the continuous delivery value chain in 2017.

    1. The Industry Embraces Open Source (Really)

    In the past, we’ve noted that commercially motivated technology providers have tried to keep open-source technology “at arm’s length.” There may have been some lip service paid to openness, but in reality many commercial providers felt threatened by open-source solutions that were seen as competitive to their products.

    This perception has already started to change, but I believe 2017 will mark the tipping point where companies will truly embrace open-source solutions as part of their strategy; those companies that continuously resist (pun intended) will be at a significant disadvantage.

    Open source is important for many of the same reasons as the cloud: It is broadly adopted because it creates advantage in time to market, quality and cost effectiveness. Embracing open source—and by that I mean both integrating with open-source solutions and using them as part of a product strategy—only makes good business sense. It helps software development teams meet their objectives in delivering quality software faster, and at lower cost.

    Another year is rapidly coming to a close and we have a lot to look forward to in 2017.

    On the topic of continuous delivery, looking forward means spreading continuous delivery throughout the SDLC, using the cloud more and embracing open source.

    All of those things combined will help us advance the continuous delivery movement together as an industry. Here’s to another year of continuous delivery advancements!

    Don’t forget, you can also take a closer look at the ongoing evolution of continuous delivery and other DevOps-related trends by picking up a copy of my colleagues’ new book, “DevOps for Digital Leaders: Reignite Business with a Modern DevOps-Enabled Software Factory.” If you have some extra time over the holidays, you’ll find it to be an interesting deep-dive.

    — Jeff Scheaffer

  • How Continuous Can Integration Be?

    How Continuous Can Integration Be?

    My blog, “7 Pillars of DevOps,” identified seven essential best practices areas that make up successful DevOps. One of the seven pillars is continuous integration (CI). The blog explained that the pillars are not silos, but rather foundations that need to be kept in balance. A well-balanced CI implementation performs integrations that are fast enough to fit within the time budget for an end-to-end DevOps pipeline.

    Compared to older waterfall processes in which large software changes were integrated occasionally, the DevOps CI concept of frequent, small, incremental integrations has many merits. Quick feedback on frequent integration problems enables a release consisting of many individual changes to be built incrementally with confidence. The root cause of integration problems can be isolated much faster when the changes are integrated incrementally. Since faster integrations are better, just how continuous can integration be?

    Contrary to the name, integration is never truly continuous. Even if a continuous stream of changes to integrate existed, typical CI process steps (listed below) require a non-zero amount of time. These steps absolutely interrupt the continuous integration cycle to some extent.

    • Merge changes in the integration repository
    • Check integration repository entry criterion
    • Run static analysis scans of the integrated source code
    • Orchestrate the build resources
    • Create the build
    • Save build the artifacts to artifact repository
    • Run unit tests
    • Save unit test results to artifact repository
    • Release the build resources back to the pool
    • Orchestrate the functional build test resources
    • Run functional tests
    • Perform analysis and assessments
    • Save functional test results to the artifact repository
    • Remediate any problems discovered during the above steps
    • Promote the build to the delivery stage
    • Release functional build test resources back to the pool.

    For CI to perform as continuously as possible, there are three categories of CI best practices:

    1. Modularize the product design to support modular CI processing
    • Modular code designs, using 12-factor-app principles, allow incremental changes to be integrated.
    • Microservice architectures enable each portion of a product to be integrated as available rather than requiring the entire product to be integrated at once.
    • Using containers can help to optimize infrastructure orchestration.
    • Re-factoring build schedules to remove any unnecessary build dependencies between modules or microservices eliminates time delays.
    • All changes are checked into a common trunk branch. This requires the ability to toggle feature changes on and off quickly and easily to enable a release whenever it is needed.
    1. Optimize the CI Infrastructure
    • Vertical and horizontal scaling of both the CI build resources and test resources creates an infrastructure optimized for each CI stage run.
    • Predictive orchestration of CI infrastructures enables CI and test resources and processes to be set up in advance.
    1. Accelerate CI processes
    • Pretest and pre-check integration deliverables prior to integration to trunk, to minimize costly remediation delays.
    • Pipeline CI build and test processes to eliminate setup delays between processes.
    • Use Fail fast to prioritize the most important tests as early as possible.
    • Use accelerated testing techniques such as risk-based specific test selection to speed up CI testing.
    • Use in-process analytics that can detect threshold exceptions prior to the end of the CI build or test process.
    • Remediate problems fast using automated rules for roll-backs or targeted problem reporting.
    • Establish a policy for deciding when and how to revert changes that break CI processes so that the pipeline doesn’t get stuck when check-in breaks the trunk.

    The above CI practices commonly reduce the time for CI from many hours to minutes per build. While minutes is still not truly continuous, for most practical purposes CI processing in minutes is fast enough to suit even the most aggressive end-to-end DevOps delivery pipeline goals.

    Summary

    What do you think? Do you agree with the above CI best practices? Are there others you recommend?

    While DevOps implemented with all seven pillars provides a strong foundation for long-term enterprise business improvements, it is important to understand DevOps is not an island. Enterprises implementing DevOps should be aware that DevOps interoperates with other IT systems and practices. Enterprises are well-advised to put in place governance policies that encourage the selection of tool-agnostic IT partners with solutions that best suit the needs of each unique enterprise and can integrate and evolve DevOps together with all their IT systems.

    — Marc Hornbeek

  • Assembling the Key Components of Continuous Delivery

    Assembling the Key Components of Continuous Delivery

    As agile methodology was popularized at the start of last decade, engineering teams gained the opportunity to begin producing software in shorter cycles, while releasing it reliably at more frequent intervals.

    This concept garnered greater attention in 2009, when Tim Fitz published his thoughts on continuous deployment. By the time Jez Humble authored his seminal book, “Continuous Delivery: Reliable Software Releases through Build, Test, and Deployment Automation,” in 2010, the approach started to get significant mindshare. In the book, Humble advanced the three building blocks of continuous delivery: configuration management, continuous integration and continuous testing.

    Thereafter, Agile Operations—the practice of using end-to-end monitoring, fueled by analytics, to increase operational agility—appeared. This complemented continuous delivery, with the goal of increasing the quality and velocity of software delivery while minimizing overhead.

    Since then, and particularly over the last six years as DevOps adoption has exploded, there’s been significant discussion (and confusion) over the opportunity to embrace “continuous everything” across the software development life cycle (SDLC) to deliver functionality continuously and optimize end-user experience.

    Today, having spoken to hundreds of organizations employing agile and DevOps, it’s possible to expand on these concepts to better define the key components of continuous delivery.

    In this blog, I’ll detail the precepts of “CD” that CA’s customers recognize as critical in reducing the time, risk and expense of software delivery by frequently releasing smaller batches of code with extensive visibility and traceability.

    These building blocks of continuous delivery are:

    • Continuous development & integration,
    • Continuous testing. and
    • Continuous release.

    Continuous Development and Integration

    When it comes to delivering quality apps faster, every second counts.

    That’s why it’s important to automate the planning, development, unit testing and integration of code with different inter-dependent components, all while packaging it for quality assurance. We categorize these activities under continuous development and integration, as shown in Fig 1.

    This all starts with sprint planning and modeling of requirements in agile management software. Unit tests can be created automatically, based on requirements and use case stories. These unit tests then can be executed automatically based on pre-defined criterion, using automatically generated synthetic data.

    Fig 1: Continuous Development and Integration

    Next, virtual service interfaces can be generated automatically by simulating unavailable systems across the SDLC, allowing developers, testers, integration and performance monitoring teams to work in parallel—driving faster delivery, along with higher quality and reliability.

    The integrated code can be built and tested automatically against validation criterion, and upon passing, made available for continuous testing. This later phase, after code development and before QA, is also commonly referred to as continuous integration.

    Continuous Testing

    Continuous testing across app delivery not only enables enterprises to shift testing left, engaging it earlier in the SDLC, but it also helps them shift right, leveraging the same processes throughout staging and production.

    After the unit tests are passed and integrated builds are made available for QA, functional and performance tests can be completed—before the build is made available for staging and production. Various activities during continuous testing are shown in Fig 2.

    Before the functional and performance tests are performed, the build promotion criterion and configuration also need to be validated. Testers require provisioning of QA environments to deploy the build, along with virtual services and test data. Using test data management to create a subset of production data and mask sensitive information is another continuous testing requirement.

    Fig 2: Continuous Testing

    It’s also important to engage application performance monitoring in the QA environment before running automated tests, providing feedback about any problems back to developers.

    After the automated tests are executed and promotion criterion are met, the build is ready for deployment in staging or production environments.

    Continuous Release

    Continuous release matches release velocity to development velocity through controlled and scalable deployments.

    The goal of continuous release (sometimes referred to as continuous deployment) is to continually and automatically deploy the build into production; similar activities also can be performed in a staging environment.

    As shown in Fig 3, build configuration needs to be validated, with the production environment configuration updated before being sent to production. As with application code, configuration is also maintained in the version control system and can be automated for deployment.

    Fig 3: Continuous Release

    After the build is deployed into production, application monitoring should be initiated to track performance and customer experience. Feedback from the production environment can be passed back to developers to guide enhancements or sprint backlogs.

    This blog aims to outline core CD activities, and in subsequent articles I’ll provide additional details on entry and exit criterion, as well as metrics useful in tracking continuous development & integration, continuous testing and continuous release.

    In the meantime, check out, “DevOps for Digital Leaders: Reignite Business with a Modern DevOps-Enabled Software Factory,” a new book authored by my colleagues Aruna Ravichandran, Kieran Taylor and Pete Waterhouse.

    Based on learnings gained from numerous customers who have already embraced CD and DevOps, it highlights key practices for starting this journey, related ROI metrics and other useful tips.

    — Anand Akela

  • Keep Applications Running by Testing Early, Continuously

    Keep Applications Running by Testing Early, Continuously

    Almost every product we buy or use today goes through some form of extensive testing before it’s released on the market; typically, these tests are designed to replicate the worst-case scenarios that a product may encounter under real-world conditions.

    Automobiles, for example, go through extensive hot and cold weather testing in remote locations, and are driven over obstacle-laden tracks designed to push the limits of their durability. Even seemingly mundane objects such as children’s toys are exposed to abusive simulations to ensure that they don’t harbor any potentially hidden and harmful effects.

    On the current landscape, applications are the face of your business, often serving as the battleground where consumer loyalty is won or lost. Given this reality, it seems crazy to think that any applications are released having only been exposed to a minimum of tests. Yet, in the name of getting to market quickly, that’s far too often the case.

    As disciplines including agile and continuous delivery continue to increase the speed at which development teams can release software, it’s easy to put the crunch on testing in the rush to get an update released. But that’s a costly mistake in the long run, as research has shown that it is 50 percent less expensive to find an error in testing than to address the same error once it has reached production, and more than 90 percent cheaper  when addressed in the development/coding phase.

    Given the dramatic cost savings of addressing errors in development, clearly the goal of any organization focused on continuous delivery should be to shift testing left so a greater number of defects are found early in the development process. This is what we refer to as continuous testing, and it’s a huge part of the reason why I am so excited about CA’s acquisition of BlazeMeter.

    Performance load testing should be a core element of any continuous testing strategy because it is a critical factor in determining system reliability. Similar to the worst-case scenario testing that consumer products undergo, performance testing can quickly uncover applications defects that might otherwise go unnoticed. After all, the manner in which a system operates with a few hundred concurrent users may be very different than how it runs with thousands of users.

    BlazeMeter enables today’s development, operations and DevOps teams to conduct this critical performance testing earlier in the development process, saving countless hours and resources. The company is also competing—and winning—in a highly competitive marketplace where it has been pitted against much larger, more entrenched competitors.

    Part of the reason BlazeMeter is winning is its focus on offering a SaaS-based solution that is highly cost-effective and easy to engage. Using this model, customers can try  out the testing software with a few clicks, and then quickly move forward to broadly adopt the system. Another catalyst to its success: BlazeMeter was built using open source tools, such as Apache JMeter, used widely in the development community for performance and load testing. As a result, BlazeMeter can easily run and analyze JMeter and other open source-based load tests—such as Selenium, Gatling and Locust—within a commercial-grade performance testing system.

    This combination of a SaaS delivery model and incorporation of open-source tools, augmenting strong performance testing functionality, make BlazeMeter very complementary to CA’s existing continuous delivery solutions, including its CA Service Virtualization, CA Test Data Manager, CA Agile Requirements Designer and CA Release Automation products.

    At CA, we’re committed to providing the most comprehensive, integrated and open continuous delivery ecosystem, allowing our customers to release swiftly, test agilely and constantly increase reliably. The addition of BlazeMeter will further enable customers test continuously, thereby increasing agility and the overall quality of their software, to ensure that applications don’t break down on the side of the virtual road.

    — Aruna Ravichandran

  • What Every Front-End Developer Needs to Know

    What Every Front-End Developer Needs to Know

    One of the hardest things about being a front-end developer is the fact that everybody has very different expectations of what it means to be one. This, along with a technology stack that changes at the speed of light, makes interviewing for a front-end position tricky.

    To help you out, and to give you an idea of our views on the role, we have prepared some points on what we consider to be essential knowledge for a front-end developer.

    Be Able to Solve Problems

    This sounds quite abstract, so let’s talk about what it means. Unlike many other developer positions, a front-end developer wields the power of multiple languages and technologies. Being able to look at a task, split it up into necessary steps and choose the right technologies for the job is an essential skill—some things need just a line of CSS, but implementing them in JavaScript would require a 100K library or vice versa.

    Sometimes you won’t know the perfect tool for the situation at hand, and this is also fine. However, you are still expected to provide a solution. This solution may not be the nicest in terms of code quality or speed, but it needs to work—at least for the cases presented at hand.

    JavaScript

    React and Angular, along with other front-end frameworks, are quite impressive pieces of engineering and it is important to know them well if they are the primary frameworks you are using for the job. That said, knowledge of JavaScript, its core libraries and browser APIs will support you no matter which framework you use. Additionally, JavaScript is a general-purpose programming language, so doing some simple array or tree processing shouldn’t cause you any problems and shouldn’t immediately trigger the reflex to include that nifty 100k library that you would barely use.

    One more thing to pay attention to is the asynchronous but single-threaded nature of JavaScript. You should be comfortable with scheduling pieces of work into the future in various ways and know how to execute some code after those asynchronous calls.

    CSS

    In the world of component-based frameworks and living style guides, you can get away with minimal CSS knowledge most of the time. Yet, when something goes wrong with your layout, or when you need to implement something unique, you need to know how browser layouts work. In short, we want you to understand CSS, but not necessarily keep all the flexbox properties or browser-specific hacks in your head—although it is certainly a plus if you do.

    Know Thy Standards and Keep Up To Date

    Nothing gives you as much appreciation of the complexities involved in making the front end work as reading the standards does: HTML, CSS and ECMA. It’s easy to get overwhelmed with more than 2,000 pages of quite dense and very specific reading, so start small. Websites such as SmashingMagazine, PonyFoo, HTML5Rocks and others will keep you up to date on the latest developments and might trigger an urge to understand things deeper.

    Network

    Every awesome front-end application needs to get to the user’s browser somehow. It’s also likely that you will have to make some asynchronous fetch requests to back-end services. All of this makes use of network, which means you should be aware of what is involved in making an HTTP request, at least on the level of Wikipedia with a sprinkle of cache-related topics, as well as what request latency is composed of.

    While it is not a requirement for a front-end developer, knowing the connection on a TCP level is a nice thing to have, as is awareness of HTTP/2. Knowledge of REST services, at least from a consumer perspective, is certainly helpful as well.

    Debugging

    This goes back to the initial section on problem-solving, but also touches on every other front-end topic. Browsers are extremely complicated pieces of software with years of legacy, but still evolving at great speed, and this software is our domain as front-end developers. So whenever something goes wrong there, it’s our job to figure out why.

    Sometimes it means looking for JavaScript errors, other times you may need to profile your code to check for slow code paths, and occasionally you may need to use CURL to fire off some requests from your terminal.

    Security

    Web, being an open platform, is constantly under the threat of various attacks, targeting both users and companies. Every piece of code and every call the browser makes is a potential source of security vulnerabilities—initial load of the page, AJAX request, external scripts or iframe, and even a simple DOM manipulation can be vulnerable to XSS, CSRF and various other vectors of attacks. Being aware and vigilant is an essential requirement for the safety of the data of your customers and your company.

    Testing

    QA, as a department or a position, hasn’t existed at Zalando since we brought in Radical Agility in 2015. We believe that it’s each developer’s responsibility to ensure the quality of their produced code, and front end has some very specific strategies for testing and asserting non-functional properties of the code.

    We believe that unit testing is a must for any project, and we highly encourage TDD, especially for a dynamic language such as JavaScript. Generally, you also would want to supplement this with some end-to-end or functional tests to make sure users can do what we expect they should be able to do.

    Tooling/Automation

    All of the topics discussed here also require very specialized tooling. You definitely should be comfortable with the concept of resource bundling and the tools available for this purpose. As mentioned above, testing by hand is not our thing, so a setup with automated tests is a must for every project.

    Ideally, tooling and automation should be complemented by continuous integration/continuous delivery to make sure no code slips past without running those tests. Depending on the project, you also might want to use a transpiler such as PostCSS or Babel to help you write more modern code.

    Conclusion

    What we’ve collected above is obviously an enormous amount of knowledge and will rarely exist in a single mind; however, the strongest candidates have at least a general understanding and a keen insight with a select few to stand out as a front-end developer.

    About the Author / Dmitriy Kubyshkin

    dmitriykDmitriy Kubyshkin is a front-end architect at Zalando SE, helping delivery teams with the migration of their front-end features from a legacy monolith to their new microservices architecture. Connect with him on LinkedIn / Twitter.

  • Network Resilience and Security from A to Z

    Network Resilience and Security from A to Z

    An observer watching a bunker shot by legendary pro golfer Gary Player was heard to say: “I’ve never seen anyone so lucky in my life.” The player retorted: “Yes, and the more I practice, the luckier I get.” Yet, when it comes to cybersecurity, nearly half of organizations are solely relying on luck to get them through a cyberattack.

    There is not enough practice or training in terms of incident response, and testing happens haphazardly depending on the developer and organization—obfuscating baselines and the context necessary to ensure security and resilience. Particularly in terms of testing, we’ve noticed that despite awareness of its importance, bugs and vulnerabilities routinely slip through. In fact, a recent Ixia survey found that 34 percent of developers have deployed products that have had a few bugs. Worse, 31 percent said products harbored significant vulnerabilities that required patching later in the cycle when shipped.

    The problem has been exacerbated by the rapidly growing normalization of agile development processes. Groups of developers are tasked to build products piecemeal, leading to application development that is often incremental and happens in iterative cadences. Testing and oversight happen at each step of the process, but the segmented nature of the development cycle means that bugs and vulnerabilities often arise when the code is assembled, and are routinely missed. For instance, the average IT web application had 32 vulnerabilities, according to recent WhiteHat Security report. It’s clear that more than a local test—a comprehensive end-to-end test—and relevant training are critical.

    A Change in Culture

    Improvement begins by changing the culture that minimizes security testing for the sake of launch timelines. Too often, the product development team will come together just to be told they need to move up their release date. The habit results in products that walk a thin line of performance, as they may contain unknown security holes due to not being fully tested. Just consider the slew of IoT devices and services that have been proven to be vulnerable because of this habit—from cars to coffee machines to cameras. It’s a major reason for security incidents, even with multiple layers of security tools.

    This also requires the right training. Having seemingly secure code and the right security measures is not enough for a strong security stance if proper training is not implemented. Organizations need to learn how to respond. Yet, recent SANS Institute research into the incident response capabilities of companies worldwide found that 43 percent of respondents did not have a formalized incident response plan, and 55 percent didn’t even have an incident response team.

    This less-than-vigilant approach to system-level security can have worse implications once a product goes live in the network. For instance, updates, patches and configuration changes applied when a new feature is added can reset an organization’s security posture, and this sometimes goes unnoticed—leaving gaps for threat actors to exploit. Or sometimes a machine fails to update and becomes and entry point, as experienced by JPMorgan Chase. Continuous testing and training from the very onset of operation and throughout a tools lifespan are imperative for all employees, especially IT and security pros—even in its simplest iterations.

    What to Look For

    Successful and continuous testing and training expose security and IT pros to anomalous activity they’d not be able to recognize otherwise. It’s necessary to be able to answer questions such as, “What does suspicious activity look like on my network?” or “What security alerts require immediate action?” The more they see, the more refined their eye becomes—it’s all about a proactive approach paired with repetition.

    Which raises the question, what is considered a good test? It starts with accurately reflecting the widest range of attack types in live operation. It is also important to create an accurate sandbox to test in so that it can be as close to reality as possible. The more accurate the environment, the better prepared IT and security teams will be. Realistic depictions better prepare employees as new malware, phishing and DDoS attack types emerge every day—not to mention it helps keep up with evolving typical application behavior.

    More specifically, function and system testing should be at the heart of any program following the development cycle test process. Quality of service, performance and resilience all benefit, along with security, as a result.

    Some companies take shortcuts, such as using internally generated attacks or crowdsourced probes to attack their networks. Just as bad, some have the development team create and run their own test scenarios. While this can give the illusion that everything is good, it creates a false sense of security—a single scenario only protects you from one type of attack.

    The key is not to stop at the minimum when it comes to security testing and training. Developers spend lots of time building great features; why not spend time training teams on how to use them? Limited or biased tests can easily overlook glaring flaws.

    Ultimately, the right approach to testing can make networks robust and prepare IT teams for real-world attack situations, helping minimize response times and the negative impact on the business. As breaches now seem to be coming from every direction, testing needs to come to the foreground, from the development stages to when products sit directly in the line of fire.

    About the Author / Jeff Harris

    jeffharrisJeff Harris is VP, Solutions at Ixia, leading solutions marketing for Ixia’s security portfolio of products and capabilities. As a former product development leader of advanced networking, communications, and surveillance products for commercial and military applications, Jeff has a deep appreciation for security implications that occur in development and operation. Jeff has led first to market product teams in personal area networks, mobile ad hoc networks, and a wide range of surveillance equipment and platforms. Connect with him on LinkedIn and Twitter.

  • It’s All About the Tools: Lifecycle Security and Testing

    It’s All About the Tools: Lifecycle Security and Testing

    In the past, whenever a new application or a new version of an application was bought to market, users often were abused as beta testers for security and usability. This was largely because of a lack of opportunity to test a large codebase effectively and efficiently.

    Similarly, the subsequent patches and new applications versions were rolled out with minimal testing, often fixing the existing bugs but introducing just as many new ones. Because users didn’t know there was an alternative, they accepted this situation grudgingly, and the consensus formed that there was no such thing as bug-free software.

    Today’s users are much less tolerant of bugs, particularly when those bugs are security vulnerabilities in hosted applications. Nowadays, anyone offering an application online rarely has to wait long for hackers to examine the weaknesses, which, for suppliers, means the integrity of their own networks are at stake. Finding these security weaknesses in e-commerce applications or online booking systems, for example, is highly attractive to hackers because of opportunities for identity theft or to issue ransomware that may damage the company’s sustainability.

    Hackers don’t just target the obvious bugs in applications. Distributed Denial of Service (DDoS) attacks increasingly are aimed at an application rather than the network infrastructure, often making the application unavailable to legitimate users.

    Developers need to ensure they understand the importance of security and load testing over the application development life cycle. To deliver more secure products to market, it is essential to identify the aspects of development that can harness security faults and address these challenges with the time and financial support of C-suite and IT executives.

    Lack of Time for Security Testing

    Security defects must be avoided as much as possible. This requires intensive testing and the right tools. Time is precious, and time-to-market often is a critical success factor for new applications. For IT decision makers, it’s necessary to balance these security concerns and time pressures with an organization’s appetite for risk and its allocation of resources.

    In addition, IT leaders and developers need to look at the effectiveness and agility of their security testing. Agile software development decreases the period between software releases from months to days or even hours. It is widely acknowledged that the classical iterative, develop-test-develop process for troubleshooting takes too long, so it often is overlooked. As a result, security defects often still find their way into software versions and must be eliminated later, which drive costs higher. For example, according to the National Institute of Standards and Technology (NIST), the cost per defect in the development stage is US$80, but quickly jumps to US$960 per defect in the testing stage and US$7,600 in production. NIST estimates that these costs amount to almost US$60 billion annually in the United States alone.

    Developers are widely aware of the growing challenges with security testing. In a recent Ixia survey of 363 developers, a clear majority of respondents nominated security testing as the most critical component of the development cycle. In fact, 93 percent report they subject their applications to security testing during the development process early, and continuously. And yet, two-thirds of the respondents admitted to delivering products with bugs and/or security flaws. Whether this is because of a lack of time, appropriate tools or financial support, this can put application security and functionality and an organization’s network at risk. Subsequently, eliminating these defects with patches or updates is at least four times more expensive than addressing them during development.

    Test Early, Often and Under Real Conditions

    A range of comprehensive security testing solutions are emerging to integrate security testing more closely with the development cycle. These solutions give developers powerful and comprehensive test environments to monitor application behavior carefully in response to DDoS attacks, malicious traffic and automated attacks, which lets developers identify bugs or vulnerabilities at the encoding stage. These tools also generate realistic traffic for load and safety tests to simulate a wide range of protocol mixes and traffic patterns. In addition, these solutions include development tools such as integrated debuggers that import and replay recorded packet captures.

    By using REST APIs and a rich command line interface, these integrated security testing solutions work with a range of continuous integration/continuous deployment (CI/CD) frameworks. These agile CI/CD frameworks streamline discovery and resolution of issues found in different phases of the application development life cycle.

    These solutions also streamline communication and reporting within multifunction teams via a range of community features such as referrals, real-time feedback and task allocation. Using a completely virtualized solution also lets developers improve security testing across the application life cycle while minimizing any hardware or infrastructure investments. Plus, the agile CI/CD model helps disparate teams across the service provider—QA, production, development and vendor collaborate effectively and streamline discovery and resolution of issues found in different phases to accelerate the time to market.

    The ability to increase the development team’s agility and speed while improving application security is making many IT decision makers sit up and take notice. With company boards requiring IT departments to respond more quickly and effectively to technology opportunities, IT leaders are seeing the benefits of this new way of working that combines the twin priorities of agility and security.

    Developer teams should communicate clearly with IT executives to secure the required time, money and resources to ensure new applications and updates undergo rigorous and realistic tests before they are released to market to face the mercy of hackers. Using commercial tools and systems for security and load tests help to determine the behavior of applications under different stress loads. They also help developers deliver products to market in a more cost-effective, timely and agile manner.

    About the Author / Phil Trainor

    Phil Trainor headshotPhil Trainor is Head – Security Business, APAC at Ixia. Phil has worked in the network security industry for 14 years holding senior level engineering roles at California-based startups. He was recently a guest lecturer at the 2016 RSA Conference in Marina Bay Sands discussing ‘Advanced Persistent Threats’ and has also lectured at Blackhat, DefCon, ToorCon, and at many other prestigious security events.  Connect with him on LinkedIn.

     

  • Putting Ops Back in DevOps

    Putting Ops Back in DevOps

    What Agile means to your typical operations staff member is, “More junk coming faster that I will get blamed for when it breaks.” There always is tension between development and operations when something goes south. Developers are sure the code worked on their machine; therefore, if it does not work in some other environment, operations must have changed something that made it break. Operations sees the same code perform differently on the same machine with the same config, which means if something broke, the most recent change must have caused it … i.e. the code did it. The finger-pointing squabbles are epic (no pun intended). So how do we get Ops folks interested in DevOps without promising them only a quantum order of magnitude more problems—and delivered faster?

    The problem is exacerbated when the topic of DevOps comes up, as it is most often a developer doing the talking. And the topics under conversation are things every developer would have substantial interest in and pretty much nothing that would interest the operations part of the equation. Many Ops folks feel as though they have been dragged into this DevOps phenomenon by osmosis, without their consent, and now they are supposed to be reaping huge benefits from it (by magic, as near as anyone can tell). Developers working faster is traditionally not something Ops folks look forward to; rather, it may be their worst nightmare. So, if the advocates of DevOps all tend to be developers, how do we get operations folks to become our evangelists as well?

    QA Impacts on Evangelism

    It begins with quality—not the ethereal concept of quality, but rather the specific responsibility for who owns it. Finger-pointing be gone; developers now own QA. If the software development lifecycle (SDLC) environment is broken, developers own it. Ops can sit back, toast marshmallows and enjoy the show. Developers should be including whatever configuration file changes their application needs as part of the deployable package. So nothing is done manually anymore. This means no one could forget something, as machines do not forget; they do what we tell them. If by chance, some random operations staffer decides to doink with a config variable, the deployment software should simply overwrite the manual change with what is in the deployable package created by the developer. Even malicious changes can be overwritten, so now, all change is included in the development class environment in the one and only deployable package. That is the only package that moves forward.

    If developers own the QA, there is now a shift in the force: The operations team has a new role, one of coach and adviser, instead of doer. The software now is the doer, the developers are the packager and the operations staff are the witnesses of how it works. If it breaks, we all know who must fix it (and who is to blame). But once blame has been permanently affixed, something else changes, namely the culture. Now that the operations team does not take the blame, it remains interested in seeing its operations run without error. This allows operations to take an active “helping” interest in getting things fixed faster. It does not really matter who is to blame, once that argument has been settled; now it only matters if they can make something work. Harmony between factions begins to emerge.

    Scalability Impacts on Evangelism

    It continues with scale. Self-service is the nirvana of your average operations staffer. If the responsibility for provisioning can be moved into a push-button self-serve automation, nirvana will have been achieved. Imagine the offload of repetitious work tasks that consume the life of your average operations staffer. Instead of setting up (and taking down) environment after environment of one class or another for one application or another, requesters can all do it themselves with the push of a button. Even the accounting or financial impact is something the accounting team can guide the user on; operations does not have to worry about the cost of its operations anymore (or, at least, worry less). If too many environments are being provisioned, the accounting team has the power to change the rules and keep it from happening automatically without operations having to be the bad guy.

    At this point, Ops can resume a focus on something it tends to enjoy more: monitoring. Ops staffers enjoy being able to provide real-time reporting on where things stand at any point in time. It is even more rewarding to provide more meaningful information such as who is using this application, what users are doing with it and when are they performing that activity. To monitor that kind of information, operations must partner with developers at the outset of a project to determine what kind of application logs are needed later for monitoring. Without the daily barrage of mundane provisioning tasks, an operations engineer is free to focus on more rewarding (translate: higher skill-level) tasks. The days of changing tapes for the backup may have reached their end (or pick your similar set of mundane tasks which DevOps automation can fully eliminate).

    And Speaking of Monitoring

    It merges into the future. How do you create an operations staffer as a DevOps evangelist? Start by exploring potential that only DevOps can enable. All those nagging alerts that are created when an environment goes down while a deployment is made … all of that could be turned off by the DevOps automation, then turned back on when the deployment is completed. Again, it all would be machine-driven, with no human mistakes anymore, ever. This will begin to cut down “noise” in monitoring/alerting systems into only useful information. However, integrating monitoring systems into the normal DevOps continuum extends beyond just an on- and off-switch.

    Forward-thinking organizations begin to consider testing and monitoring as two parts of the same evaluation discipline. When a test is needed and when a monitoring characteristic is needed now can be determined in total rather than just in a development capacity or operations capacity. A holistic view of what can and what should be evaluated about an application, and how that will be achieved, brings the operations team to the table with the development and testing teams to plot holistic solutions that will live on beyond the development project timelines. Examining monitoring relative to version-controlled artifacts allows for organizational constructs within the monitoring discipline that never were possible before DevOps enabled them.

    The Masters of Hardware Abstraction

    Containers, or similar types of technologies, abstract an application from the hardware details it runs on top of. Technology such as this is a trend in DevOps that greatly simplifies the world of hardware from your average application developer. So who will retain the knowledge about the jungle that lives and breathes underneath? Ops has an extended role in understanding what lives underneath the abstraction layer that lives on top. Over time, only Ops will understand these particulars. They will become the only in-house experts about which cloud provider to use, which sets of physical hardware performs best and under what conditions.

    It is this kind on unique expertise that makes an operations engineer extremely valuable (translate: pricey). It is the kind of knowledge careers are made upon. It is ageism-neutral. Older employees do this today, and they will continue doing this in the future; the only difference will be fewer and fewer people will know how it is done. If the domain of application development is the province of our youth, then the domain of hardware abstraction is easily the province of our more experienced employees. These realities create personal value for our operations teams, and they are only possible in a post-DevOps embrace.

    Impacts to the Bottom Line

    The adoption of DevOps by the Ops portion of the equation is critical to overall success. Passive-aggressive resistance by operations staffers can delay or destroy altogether the value of DevOps. Overcoming this is a first step. But creating operations evangelists goes an order of magnitude further: With active operations staffers becoming internal DevOps evangelists, the entire operations team can begin to look forward at what DevOps will ultimately enable for them. The value they will carry back into development efforts becomes substantial and the entire company benefits. Careers can emerge that were not possible before it—and, just as importantly, so can profits.

    To continue the conversation, feel free to contact me.