devopsy.kinsta.cloud

Tag: devops

  • How to Succeed with DevOps by Outsourcing Software Development

    How to Succeed with DevOps by Outsourcing Software Development

    The DevOps journey can be successful when you use offshore outsourcing software partners, but it requires careful attention from your IT leaders.

    The old saying, “The only constant is change,” has never been more true for software development, whether your company is a SaaS provider or a manufacturer of discrete products that rely heavily on software delivered by your IT department. For the past several years we have seen four key movements:

    1. The continued adoption of outsourced software development. As far back as 2013, it was reported that 2 out of 5 CIOs surveyed planned some sort of outsourced software development for the coming year.
    2. Increased outsourcing options. While outsourcing to India is still a legitimate option today, it’s certainly not the only viable option available. Companies are now able to answer the question: “What country will provide my outsourced programming team?” in multiple ways.
    3. Agile as a mainstream methodology. Software development teams continue to embrace Agile as the preferred methodology for software development projects.   
    4. Interest in—or adoption of—DevOps. Since the term “DevOps” was introduced by Andrew Shafer and Patrick Debois at the 2008 Agile conference in Toronto, large and small companies alike have shown increased interest in embracing DevOps and the promise it holds for rapid software deployment.

    The current state of tools and service provider capabilities should entice IT leaders to adopt (or continue to adopt) DevOps methods for software development. More than ever before, there are capable teams of programmers across the globe that you can tap into for a successful software.

    There is No Substitute for Good Process

    The goals of higher output and greater efficiencies and even “continuous deployment” through DevOps have their roots in elite manufacturing methods such as kanban and lean manufacturing. Like its manufacturing method cousins, DevOps success hinges on good process. To see improvements through the use of DevOps, you must have clear understanding (and description of) your current software development process. You need to communicate your processes clearly in words and pictures with your outsourcing partner. Issues that are birthed out of shortcomings in your DevOps process will be exponential when you add the variable of outsourced development into your work steps.

    Key Takeaway: DevOps is only possible through good process. If you’re not committed to good analysis and continual improvement of your processes, then the use of an outsourced software developer will only allow you to create flawed software and generate chaos more rapidly.

    Tools Create Speed

    If good process is the engine of DevOps, then automation tools are the gasoline that makes the well-designed process run with speed. Automation of good process makes it possible to eliminate bottlenecks in the software development cycle—you can truly “sprint through your sprints.” Perform an inventory of your tools:

    • Are you able to seamlessly (and automatically) promote code?
    • Can you perform test-driven design?
    • Can you perform test-driven development?
    • Can you easily associate features and fixes with promoted code?
    • How do you perform regression testing?

    Key Takeaway: Automation tools must be used with consistency by you and your outsourcing partner. These tools will enable the process that you all follow daily in the development activities of your project.

    Choose Wisely

    Having the right outsourcing partner for software development is important. To outsource successfully in a DevOps approach, you must be very selective in your choice of an outsourced software development team.

    • The best partners employ a programming staff that have achieved the major certifications that are important for your DevOps project needs. Moreover, a good partner invests in its programming staff’s training on a continual basis. We like to see evidence that its programming staff regularly renews their certifications.
    • Experience. Equally important is having a software outsourcing team with demonstrated experience in your business vertical (example: discrete manufacturing, banking, etc.) and expertise in the system functional area (e.g. finance, e-business order processing, warehouse management, etc.). Of course, the software development team also should be versant and have demonstrable experience in the use of Agile and DevOps techniques.
    • The right partner should be neither too large nor too small. The outsourcing company needs a pool of programmers large enough to keep with the intended work pace of your project. Conversely, we caution IT managers to be wary of extremely large outsourcing companies. Your project and company must be “big enough to matter” to the partner you select. If you are seen as too small in terms of the revenue opportunity, the outsourcing company will defer attention and its top talent to larger customers, who are more able to influence decision-making.

    Key Takeaway: To succeed with DevOps outsourcing, you need the right partner, based on attributes of skills, demonstrable experience, and a size that’s proper for your company.

    Metrics Matter

    John Adams, the second U.S. president, once famously said, “Facts are stubborn things.” DevOps presumes a culture of continuous improvement—and our metrics help demonstrate that improvement. We collect metrics so we can accurately measure pace of work, quality of software produced, level of effort and so on. We measure events over time to see if we are improving, worsening or stagnant. Metrics are a critical means of communication and mutual accountability between your company staff (IT and business) and the outsourcing partner who is writing your software. In a DevOps-driven outsourcing relationship, you’ll regularly capture and report on metrics (estimated and actual) such as:

    • Lead Time: In software development, you need to measure the time elapsed from concept to the point delivery of the completed software. From project to project or even epic to epic, you want to see an increase in speed to deliver software.
    • Sprint Time: Teams using Agile will have sprints measured in a few days, instead of months, for the typical time for work grouped in “phases” or “activities,” using traditional development methods. Be sure to track metrics for your sprints.
    • Sprint Volume: How many sprints are needed to complete a story? How many sprints are needed for a version? For each of these metrics, look for consistent volume (i.e. number of sprints produced) over time.
    • Issues and Open/Close Rates: Measure how many production issues are reported AND closed within a specific time frame. Observe and take action on trends rather than establishing a hard and fast number; there’s no “right answer.”

    Key Takeaway: Metrics allow you and your outsourcing partner to objectively assess the pace and quality of rapid work being performed in a DevOps/Agile-driven development project.

    Collaboration is Key

    It almost goes without saying that you and your outsourcing partner will need to have good, regular rhythms of communication. This continuous collaboration is essential because you and your vendor-partner team are:

    • Operating in a common disciplined way using DevOps methods.
    • Performing work through a common set of tools.
    • Progressing through the project goals in a series of rapid sprints.
    • Capturing and analyzing team metrics to ensure project health.

    Honestly assess what barriers might exist to good collaboration. Do you need to ramp up your real-time collaboration tools? Do you have a master calendar of formal meetings where the agenda will drive good information-sharing, decision-making and issue resolution?

    Are time zone differences an issue? This may affect the geographic location from which you will select your outsourced development team. In a recent survey, one-third of U.S. companies that outsourced to India considered the 10-hour (or more) time difference to be a big challenge. Very often the “core team” in an outsourcing software provider will be working hours that are completely opposite of normal U.S. business hours. Some companies find that a “nearshore” outsourcing partner helps optimize collaboration. Because the outsourcing partner is in the same time zone as your IT leadership team, it’s easier to schedule meetings or even have impromptu discussions, which are so important in software-development methodologies such as Agile.

    Key Takeaway: DevOps and Agile require high collaboration. Set yourself up for success by ensuring there will be no reasonable impediments to good collaboration.

    A Perfect Storm – and Opportunity

    The recent evolution in best thinking and adoption of DevOps, and the parallel growth in options for software development outsourcing together create a kind of perfect storm in software development. The potential of DevOps is augmented by the inherent continuous-change nature of Agile. We see the ability to transfer work across the “virtual shop floor” of development without geographic constraints. This new programming paradigm of DevOps, partnered with Agile, is now matched by equally exciting (and relevant) growth in the options for delivering high-quality code from anywhere across the globe.

    About the Author / Steve Mezak

    Steve Mezak is founder and CEO of Accelerance, a leader in global software development outsourcing, as well as the author of “Software Without Borders,” co-author of, “Outsource or Else: How a VP of Software Saved His Company,” and a CIO columnist.

     

  • Puppet CEO Sanjay Mirchandani Reflects on a Year at Puppet

    Puppet CEO Sanjay Mirchandani Reflects on a Year at Puppet

    As Mike Vizard detailed, this year’s Puppet Conference featured an impressive lineup of new products and services brought to market. The company appears to be firing on all Puppet CEOcylinders. The event also marked the first anniversary of Sanjay Mirchandani as Puppet CEO. I had a chance to sit down with him to review the past year, as well as the announcements at the conference.

    I will leave the product stuff to Mike’s article, but I asked Sanjay what he thought was his greatest accomplishment over the last year. He said that hands down it was assembling the great team he has put together. To be frank, I agree with him. In a time when execs at tech companies seem to hop around like fleas, Sanjay has assembled a world-class, enterprise-proven leadership team.

    In my opinion, this builds on Puppet’s greatest strength, which is its community. Puppet, like many successful DevOps companies, builds on its free, open-source roots and is nourished by a passionate, involved worldwide community. This was once again in full view at Puppet Conference. Frankly, my favorite time of the day was lunch (no surprise there, but not for the obvious reasons), where I would get a chance to sit down with “strangers” and listen to their experiences, what challenges they face at their jobs and where they see things going. I love those opportunities for unfiltered dialogue with real practitioners. The feedback I heard was pretty consistent.

    They enjoy using Puppet and they appreciate what the company is doing. They noted was that even though the company is developing products that compete with other tools they are using currently, they like the fact that Puppet doesn’t force them to make an either/or decision. They can continue using the tools they are comfortable with or start adopting Puppet’s new tools, all the while using Puppet for what they have always used it for.

    I asked Sanjay another key question: “What is your biggest failure or disappointment this year?” The answer was surprising. He said he failed to understand that some people really were not clear whether they could use Puppet in a cloud environment. Mind you, this is during a conference in which several relationships with cloud providers were announced. But evidently, some people did not understand that it works in cloud environments as well as on-premises—and, for that matter, in a hybrid environment as well.

    Mirchandani is not underestimating this view any longer. Puppet is emphasizing its cloud chops at every turn. As I mentioned, I was not aware of this view either, so it is interesting.

    In any event, this year’s Puppet Conference was the biggest (about 1,200 people) and best yet. I thought the company did a great job trumpeting a number of new products, ideas, partnerships and a clear path forward. As one of the founding and leading DevOps tools providers, its future continues to look bright.

    — Alan Shimel

  • Node.js Foundation Embraces HTTP/2

    Node.js Foundation Embraces HTTP/2

    The latest version of Node.js will significantly increase the DevOps challenges surrounding the most widely used version of JavaScript in the enterprise. Node.js 8 adds the ability to support HTTP/2, a major revision of the core internet protocol that, in addition to being significantly faster, enables developers to make multiple requests over a single connection as well as set up pipelines for making a series of requests.

    Myles Borins, director of the Node.js Project and member of the technical steering committee at The Node.js Foundation, notes that while HTTP/2 eventually will be supported by multiple programming languages, DevOps teams most likely will encounter it first in web and mobile applications built using Node.js.

    Borins says HTTP/2 also should go a long way to accelerating the adoption of microservices applications—the majority of which are being built using Node.js.

    Node.js 8, available under a Long-Term Support (LTS) license, is 20 percent faster than its predecessors, Borins says. It is based on the JavaScript V8 engine that makes use of Turbofan and the Ignition compilers and interpreters, which equates to lower memory consumption and faster startup time across Node.js applications.

    Other new features include an Async/Await capability that allows developers to write more linear code versus relying on callbacks and a series of Experimental ES Modules that provide a standard pattern for writing modular JavaScript code that can run on both the web and in Node.js.

    Node.js 4 is now scheduled to be designated for end-of-life status in April, and Node.js 6 will face a similar fate soon after. The Node.js Foundation is encouraging organizations that have deployed Node.js applications to refresh them because security patches and updates soon no longer will be available for the core engines used in Node.js 4 and 6 applications. Node.js API (N-API), a project that enables modules to run against new versions of Node.js, is still considered experimental. That project is being lead jointly by Google, IBM, Intel, Microsoft, nearForm, NodeSource and individual contributors.

    The Node.js Foundation also announced that work has begun on Node,js 9, which is focused on modernizing legacy application programming interfaces (APIs) and implementing a new error system that associates a unique code with all errors in place of relying on traditional error messages.

    The Node.js Foundation estimates there more than 9 million instances of Node.js deployed across the web today, so managing Node.js applications across their life cycle has become a significant DevOps challenge. While Node.js is not the only programming language gaining momentum across the enterprise, the Node.js community is highly focused on increasing the robustness and manageability of Node.js applications in a way that should eventually please even some of the most hardcore fans on rival languages such as Java or even COBOL.

    — Mike Vizard

  • DevOps in the Age of Digital Transformation

    DevOps in the Age of Digital Transformation

    Digital transformation is one of the hottest buzzwords in the technology industry today. While it tends to be overused, the term does represent a widespread, ongoing movement that will set a standard for the next generation of enterprises.

    Digital transformation is all about keeping up with the pace of innovation and using technology to achieve business goals. A key differentiator in the age of digital transformation is customer experience. Consumers are increasingly engaging with brands on digital platforms, whether it be a website, social media profile or mobile app. The foundation of all these platforms is, of course, software.

    Agile software delivery is key because it helps organizations innovate faster and gets new software in the hands of consumers sooner. This requires new, more productive ways of working for enterprise IT teams. Development teams have already undergone several “transformations” in recent years, from the siloed waterfall approach to agile methodology and now DevOps. Software is the true differentiator in the age of digital transformation, and many companies are using DevOps to ensure their software meets the standard.

    Here are a few best practices for using DevOps to keep up with the pace of digital transformation:

    • Automate wherever possible. We no longer collect the work of multiple engineers over multiple sprints into a release. To out-innovate the competition in the digital transformation era, a team must release the work of a single engineer as soon as it is ready. This is usually accomplished through a CI/CD pipeline directly from the source repository through automated testing and then deployment into production, preferably without any human intervention. Choosing the right tools for your IT environment will cut costs, time and room for error.
    • Align the goals of development and security. DevOps isn’t limited to just the development and operations teams. Without security, all progress made in the digital transformation era is at risk to cybercriminals. The accelerating software development life cycle means little time for security analysis or penetration testing when the release candidate is ready to deploy. To ensure a piece of software is secure before pushing it to market, security standards must be included in the definition of “done.” Preventative scanning will not only train developers to write secure code, but will allow the software to pass the assurance tests included in any good CI/CD pipeline.
    • Measure ROI. Measuring the impact of DevOps can be extremely difficult and up to interpretation because of the focus on culture and processes. However, it’s important for DevOps teams to gain continued executive buy-in and dedicated resources by proving their value. The value of DevOps is determined in terms of achieving improved business outcomes. There are specific questions to ask to help determine ROI: How long does it take to release a code change? How long does it take for a product to reach a customer’s hands? The ability to continually deliver quality software faster results in a positive customer experience, ultimately improving brand reputation and revenue.

    While keeping up with the competition is necessary in the digital transformation era, putting customer experience first is how the true leaders succeed. By shortening the software development cycle, DevOps speeds the pace of innovation for an organization and helps earn customer trust and retention. Implementing these best practices can help a business use DevOps to place software and customer engagement at the heart of its digital transformation.

    — Pete Chestna

  • Building a Services and IoT Platform with DevOps, Part 3

    Building a Services and IoT Platform with DevOps, Part 3

    Business needs and IoT demands are taking new dimensions in connecting millions of devices, people, process, tools and systems. With endless possibilities of creating new IoT solutions, DevOps teams are the key players who handle the challenges in successfully rolling out a IoT solution while handling the business, development, operations and infrastructure demands in the background. This article focuses on some of the DevOps best practices to adopt during IoT development, enabling faster time to market.

    DevOps practices to adopt during IoT development

    1. Adopt Security by design – Teams need to be educated and made aware of security threats. Use threat models mitigation techniques such as using authentication; storing secrets separately and protecting those secrets; using tamper-resistant protocols, digital signatures, timestamps, audit trails, encryption, authorization policies, filtering; and running services with least privileges during security design and development.
    2. Shift left security analysis and security testing – Integrate security analysis tools, customize security compliance rules and run security testing as part of continuous integration pipeline, generate security threat reports on vulnerabilities and act on these threat reports to fix issues early in the development phase and minimize the security risks.
    3. Follow security standards during development such as security development life cycle (SDL), which has guidelines and measures for building the software for devices and application components of IoT platform.
    4. Perform code reviews for device software, application software, PaaS rules and configurations, as quality assurance can become a huge risk given the complexity of the infrastructure, millions of devices connecting and dynamic environment configurations. Code review tools need to be integrated to the CI pipeline with pull request code review.
    5. Use the right branching policies in a version control repository, including precommit/preflight builds, pull requests for merging the code changes and access control for maintaining branches and using the right branching strategies, as there will be multiple fragmented development and deployment pipelines for devices and application developments to reduce the overhead. Use distributed repository/central repository with traceability and auditability based on the requirements.
    6. Automate deployments and create deployment pipelines for device development, application development, infra-automation scripts development with build and deployment pipelines targeting for different target endpoints.
    7. Customize the builds and release pipelines to pull the packages from multiple build and deployment pipelines to integrate and deploy as units of deployment.
    8. Plan for frequent release cadences—say, monthly—for application rollouts. Release cadences for firmware update rollouts can be planned for three to six months, as device updates cannot be pushed so often.
    9. Continuously monitor deployments – Use a release management tool for orchestrating the releases and release coordination activities, with gated checks and approvals with a view of what is getting deployed to each environment, as well as traceability and audit.
    10. Enable feedback loops and automated alerts if firmware deployments or application updates fail.
    11. DevOps teams can define key tenets, criteria to be met and standards to be followed for the devices; infrastructure to be provisioned; device protocols usage and standards in connecting to existing/new devices and services. This can reduce the complexity in development and deploying to multiple varied devices by following the same protocols for connectivity and communication.
    12. Train the team of developers, QA and hardware developers on how to do continuous integration for IoT development, follow agile and lean practices and ensure security criteria for all layers of the application and in process.
    13. Adopt user-centric test automation and end-to-end testing approaches and build robust test cases in perspective of testing a complete flow, with devices/services that may not be in your scope but will be part of your overall IoT platform ecosystem.
    14. Create/utilize container based test environments and device simulators coupled with service virtualization for functional testing.
    15. Automated infrastructure provisioning – Use the right configuration management tools to automate the infrastructure along with dependencies to create environments through code instead of provisioning basic VMs.
    16. Use container and cluster management tools such as Docker Swarm or Kubernetes for service discovery, auto-scaling, self-healing and scheduling.
    17. Use RBAC, service accounts and security access controls in the infrastructure templates/scripts that are created instead of individual user credentials to limit the risk of unauthorized users provisioning the environments using scripts. Build security in devices, cloud security, application security and infrastructure automation templates. Use secrets, key vaults, encryption methods and certificates to secure the credentials and access control.
    18. Configure individual build and deployment pipeline jobs for application development and infra-automation development, as there will be a huge development effort involved in automating the infrastructure.
    19. Create infrastructure verification tests with scripting tools such as Python or use testing frameworks such as Pester or any other open-source tools and integrate the infrastructure tests with the CI/CD pipeline. Ensure to run these tests every time the infrastructure is provisioned.
    20. Version-control the infrastructure templates, scripts, user profiles, environment configurations and device configurations apart from source code.
    21. Set up a centralized dashboard of monitoring tools to monitor the clusters, nodes, firewall rules, VMs, applications, devices and services. Feedback should flow back to developers/QA/DevOps teams.
    22. Configure applications to provide logging about exceptions, memory leakages and response times and push the logs to the monitoring tools or the dashboards in monitoring tools.
    23. Use the telemetry from the applications and devices to derive the analytics graphs/trends and set up KPIs/metrics to measure application performance/availability, as well from derive business metrics to set SLAs for DevOps and operations, to improve productivity.
    24. Ensure DevOps teams have close coordination with dev/QA teams of both software and hardware teams, as managing IoT platforms need more skill and expertise in hardware and software in the IoT domain.
    25. Manage legacy and cloud infrastructure with multicloud management platforms, and monitor and optimize the infrastructure usage.
    26. Create a DevOps governance plan/guide, with clearly defined access control policies, access rights for devices, control limits on infrastructure provisioning in different clouds, limits on PaaS/IaaS services to be used, etc., and standardize the process to be followed in DevOps/development/testing/operations, tools, etc.

    Following DevOps practices for application development has become quite streamlined in recent years. However, in IoT, where traditional methodologies and practices are followed in industrial enterprises, adapting to agile methodologies and following DevOps practices coupled with right DevOps-skilled team, tools and expertise can streamline and simplify IoT developments.

    About the Author / Lavanya Subbarayalu

    Lavanya Subbarayalu is Senior Architect working with Technology Office in HCL Technologies. She has expertise in IOT, Azure, DevOps consulting & Microsoft technologies. She is associated with DevOps COE, working on design and Development of DevOps solutions and consulting tools. Connect with her on LinkedIn.

  • Case Study: DevOps Key to AWS Cloud Migration

    Case Study: DevOps Key to AWS Cloud Migration

    DevOps practices halved the time it took this SaaS company to migrate to AWS

    Australia-based Whispir provides communications applications as a service. One of its goals is to get products into users’ hands as soon as possible, and that requires development, operations and product staff working together, said Jonathan Swift, head of product and operations.

    Whispir formerly operated from a co-location data center, a model that continued as it expanded internationally. But the time and effort needed to run the systems was not cost-effective for an organization of its size, so the company switched largely to AWS. Governance issues faced by some clients force the company to still run its own systems in certain geographical locations.

    In the process, “[We] learned a lot about our app and how to make it cloud-ready,” said Swift. “Cloud isn’t free, cloud isn’t cheap” and is potentially more expensive than placing servers in co-lo data centers if the software isn’t built correctly.

    For example, it makes sense to make the most use of a physical server as it is largely a fixed cost. But in the cloud, the fewer resources you use, the less it costs.

    Crises Lead to Spikes

    On the other hand, a lot of Whispir’s business is crisis-focused (eg, Caltex uses Whispir at its Kurnell oil refinery in Sydney to send emergency messages), and it takes a certain amount of time to spin up additional instances to deal with spikes in message volumes. “That was a real challenge,” said Swift.

    Those crisis-communications clients expect Whispir to have sufficient resources to meet sudden demand. That’s an expensive proposition with co-located hardware, whereas Ccloud resources can be scaled up quickly, though not instantaneously.

    Seasonality has to be taken into account. One relatively predictable driver of crisis messaging is Australia’s bushfire season. Unfortunately, this overlaps Christmas, which is a peak time for marketing messages. So Whispir plans to run at around 200 percent of its normal baseline capacity during the fire season. This is easily done with cloud infrastructure, which Swift describes as “a beautiful thing” in this context.

    Some organizations are scared by multi-tenant environments. The ability to stand up dedicated instances on AWS is a great feature that Whispir can offer, especially to its clients in the financial sector.

    The company’s DevOps practices had been focused on the co-location environment, so even though kanban still “works very well for us” after moving to AWS, some previous choices had to be revisited. In particular, new monitoring tools were needed to cope with the elasticity inherent in the cloud and because the focus expanded from the hardware to the whole stack.

    Selected tools included the Zabbix open-source monitoring platform and Sumo Logic for trend analysis.

    Earlier this year, Whispir moved all of its Australian customers into a “pure AWS” environment. There are some great tools for migration, Swift said, “but it’s not as straightforward or seamless as you might think.” He recommends doubling any initial estimate of the time required for such projects.

    There were some nasty surprises: The first month’s bill from AWS was “huge” and it took a couple of months to determine the cause, even with the aid of AWS’s “really beneficial” premium support service.

    Security requirements in various jurisdictions had to be taken into consideration, with updates to various procedures to ensure the new cloud environment was compliant. For instance, the procedures required when a physical system was decommissioned were well-understood, but how could they be translated into a cloud environment? One specific example is that an in-house storage device can be destroyed when it reaches the end of its life, but in the cloud it’s largely a matter of relying on the provider’s controls.

    Avoid Lock-In

    But Swift warned that “the challenge is to not lock yourself in” to a particular cloud provider. For Whispir, which operates a mixed cloud/on-premises environment (no cloud provider offers the right onshore facilities in New Zealand and the hardware that Whispir has installed in Singapore is too new to be replaced by IaaS), that is a requirement rather than a strategic choice.

    Whispir would have avoided AWS-specific features that would make the platform easier to use and more difficult to get out of, despite the fact that using provider-specific toolsets can yield significant cost savings.

    Whispir is keeping an eye on Azure—”great tools, great partnerships, great ecosystems,” observed Swift—and may migrate to Microsoft’s cloud in around 18 months.

    A particular advantage is that Azure Stack can be run on private hardware, which could be useful if Whispir decided to enter a market where Microsoft Azure wasn’t present.

    Either way, the intention is to go 100 percent cloud to the extent that is possible.

    ‘Infrastructure as Code’ Means More Power for Ops

    The move to the cloud has made Whispir’s operations people more powerful, he observed. The infrastructure-as-code concept allows for rapid deployment, enables operations to contribute more to product development, reduces risk and enables the company to be more responsive to the market.

    Furthermore, dealing with on-premises hardware still very much a waterfall situation, thanks to the delays and dependencies involved. But moving to IaaS means infrastructure issues sit comfortably within the kanban model.

    IaaS also means that various network-related tasks simply disappear, but other issues arise in their place. For example, some external services used by Whispir (eg, SMS gateways) require the use of VPN connections from a known IP address, but with cloud infrastructure the best you may be able to do is specify a range of addresses. Whispir’s short-term fix was to route such connections through a specific server in the co-lo center, and in the longer term it found providers with less-restrictive arrangements.

    Moving from co-lo to cloud would have been “incredibly hard” without DevOps, Swift said. It would have taken at least twice as long, he noted, and the company wouldn’t be able to run the mixed environment as well as it does.

    “There’s no way we would have been so successful” without DevOps, he said.

    — Stephen Withers

  • Survey Finds DevOps Teams Under Increasing Pressure

    Survey Finds DevOps Teams Under Increasing Pressure

    The fact that a much larger percentage of companies now recognize they are software companies that happen to specialize in a specific vertical industry is now putting a lot of pressure on developers, according to a new survey of 750 development team leaders in the United States and the UK conducted by Testplant, a provider of tools that automate application testing.

    The survey finds that 42 percent of the respondents claim they are expected to design, develop and test apps in an unrealistic amount of time. More than one-third (36 percent) also report they are not given enough time to ensure apps are properly tested before deployment, while 56 percent note that use of outdated techniques and tools is holding them back from meeting the demands of the digital world.

    Because of this combination of factors, nearly half the respondents (49 percent) say they are deploying applications before they go through an ideal testing amount of testing, and 45 percent admit to putting applications into production that will perform below initial requirements.

    Dr. John Bates, CEO of Testplant, says it’s a matter of time before IT organizations will have to rely more on tools that automate much of the testing process, as pressure on DevOps teams to roll out new applications and update existing ones is only going to increase. The survey respondents (60 percent) say most of the pressure to accelerate software development is being driven from within the organization itself. But nearly half (49 percent) say pressure comes from competitors. As more organizations realize their competitive differentiation comes from the software they develop, there’s now a lot more focus on accelerating application development, says Bates.

    The survey also notes that the applications being built are more complex than ever. Over the next 12 to 18 months, half the respondents (50 percent) say their company will develop more apps using components for an internet of things (IoT) environment. Well over half (58 percent) also say their company will develop more apps with artificial intelligence (AI) components, and 62 percent say their company will increase the amount of applications that incorporate machine and deep learning algorithms.

    When it comes to deploying applications, automation is ranked as the most critical challenge by 67 percent of the respondents, and 70 percent claim they are now more focused on automated testing today than they were in the past.

    In total, 75 percent of the respondents say that with access to better tools, the applications deployed by their team would have a greater impact on the business. A full 81 percent also say that with more time, applications deployed by their team would have a greater impact on the business.

    Unfortunately, more time for application development is luxury most business don’t have today. Not only is a wave of digital business innovation is making it easier for startups to not only enter markets, but companies that traditionally have focused in one market segment are now extending their reach to adjacent areas. The difference between surviving and being made obsolete now comes down the level of customer experience an organization can deliver by the applications it makes available to customers.

    Unfortunately, when it comes to those applications, customers tend to be unforgiving. Once bad application experience and many of them will move on to try an application developed by a rival. Bates notes that too many organizations still seem to think they can test their applications on end users on the assumption they can fix any issue that comes up rapidly. In reality, rather than participate in what amounts to an extended beta testing process, most end users will simply look for a better alternative application experience.

    In short, the difference between success and complete failure now hinges on the level of quality application testing that can be implemented in the shortest amount of time possible.

    — Mike Vizard

  • 3 Practical Steps to Employing DevOps

    3 Practical Steps to Employing DevOps

    Everyone seems to be talking about DevOps but, if you are new to it, it might all seem a little overwhelming. One of the hardest things about DevOps is knowing where to begin. This is because to “do DevOps” its seems you have to suddenly start using a lot of new processes and tools at once.

    For an organization that doesn’t use DevOps today, the adoption of this three-step method will promise a generally clean approach.

    Source: https://ukcloud.pro/

    Step 1: Creating the Application Pipeline

    Regardless of the type of application, the pipeline looks—at a distance—very similar. The goal is to weave application releases into a new coordinated process that looks something like this:

    1. The developer makes changes locally to their laptop/PC/Mac, and upon completion (including testing, etc.) issues a Pull Request.
    2. Code review kicks in, the developer changes are reviewed and accepted and a new release can be created.
    3. Someone, or the system, runs a procedure to create a release artifact, which could be a Java JAR file, a Docker file or any other kind of unit of deployment.
    4. Someone, or the system, copies the artifact to the web or application servers and restarts the instances.
    5. Database migrations/updates may be additionally applied, although the database is often left outside of the application release process.

    This process evolves into an automated application-only continuous integration and continuous deployment (CI/CD) pipeline by using a toolset to automate the process as shown in the diagram below:

    Source: https://ukcloud.pro/

    A Typical CI/CD Pipeline

     

    However, be aware that this isn’t ideal (yet), because at this point the application alone is being released (injected) into existing environments.

    Why is this potentially an issue? If the developer has altered the local environment as well as the application but these local environment changes are not part of the release, then they’re not in version control. And, if they are not applied at the same time as the application, then the application will break and the common argument occurs: “Well, it works on my machine!” The solution is for application and infrastructure releases to be synced—and hence steps 2 and 3.

    Step 2: Creating the Infrastructure Pipeline

    Ideally, the infrastructure team has learned from the developer team’s DevOps and CI/CD pipeline journey and can expand and adapt it for the infrastructure (which increasingly is a public cloud).

    There are some differences with the infrastructure pipeline, specifically around units of deployment that are now the infrastructure layers in environments, the things that surround an application such as the DNS, load balancers, the virtual machines and/or containers, databases and a plethora of other complex and interconnected components.

    The big difference here is that the infrastructure is no longer described in a Visio diagram: it is brought to life in code, in a configuration file, in version control—this is infrastructure as code (IAC).

    Before this, a load balancer was described in Visio diagrams, Word documents and Excel spreadsheets of IPs and configurations. This is now swapped to describe everything about the load balancer in a configuration file.

    Here’s an example AWS CloudFormation Configuration for a Load Balancer:

    Whenever this file is changed in Version Control, such as changing the subnets a load balancer can point to, then the automation engine can update an existing infrastructure environment to reflect that one change, and any dependencies that change.

    It also means that you can apply this template to multiple environments just by changing the environment name from staging to production, for example, and be very confident that all environments are consistent, from the developer laptop to production.

    It is also possible to make the templates dynamic to change their behavior according to the environment, so in production the environment will scale out across three datacenters, but on a developer’s laptop it will use a local VirtualBox single-system.

    Step 3: Creating the Full-Stack Pipeline

    The goal of a full-stack pipeline is to ensure that the application and infrastructure changes over time are in sync, both in version control and the release deployments across each pipeline stage. And a developer should never say, “It worked on my laptop!” when a release fails in production.

    The popular CI/CD tools can now automate the full stack because everything is programmable. This means everything can be captured in version control, and the same configuration can use dynamic input parameters to build an environment on a developer’s laptop, or a QA system in the cloud or updates to production.

    Imagine a developer makes an application change that also requires a change to the database, the web instance scaling configuration and the DNS. All of these changes are captured in one version control branch and the developer builds a system on their laptop from this branch, and tests it.

    This is what platform-as-a-service systems can do. By adding an environment configuration file inside the same code base as your application, you can ensure that you have bound your application to the infrastructure. 

    AWS Elastic Beanstalk Example

    This is the target end goal, though it takes a lot of practice, and learning, to get there. It can be very satisfying to watch a build and a deployment, seeing the automation use tags and dynamic configurations to build, or update, the target environment with application and infrastructure changes.

    About the Author / Sarah Lahav

    Sarah Lahav is CEO of SysAid Technologies. As the company’s first employee, Sarah Lahav has remained the vital link between SysAid Technologies and its customers since 2003. She is the current CEO and former VP of Customer Relations at SysAid – two positions that have fueled her passion in customer service. Connect with her on LinkedIn and Twitter.

     

     

  • Akamai Gives DevOps Teams More Control

    Akamai Gives DevOps Teams More Control

    When most DevOps teams think about platforms, it’s usually a conversation about some form of software sitting on a server in a data center. But many web and mobile applications employ a content delivery network (CDN) as the platform on which they depend most to run. Akamai Technologies is moving to provide DevOps teams with much greater visibility into and control over what’s occurring across its namesake CDN.

    Ari Weil, senior director of industry marketing at Akamai, says CDNs now play a pivotal role in enabling IT organizations to deploy web and mobile applications in the cloud. To provide DevOps teams with more control over those deployments, Akamai is committing to treat the core CDN as code that can be programmatically invoked in addition to providing real-time visibility into applications running on Akamai and simplyfing the modification of applications in production, says Weil.

    As part of that effort, Weil says, Akamai is expending the range of application programming interfaces (APIs) that it exposes via toolsets spanning frameworks such as Terraform, Varnish and WordPress. The goal is to expand significantly the level of scripting and automation enabled by the Akamai CDN. Akamai is also making available a testing suite that DevOps teams can employ to test and fix applications in real time.

    Akamai is making it possible to deploy custom configurations and code across Akamai’s CDN in minutes, including the ability to remove content in seconds and revert to past configurations whenever necessary.

    Akamai has also released mPulse Lite, a free version of a digital performance management tool that identifies ways to improve application performance and the end user experience, and its IoS mobile application and companion IoS watch app to specifically optimize the performance of iOS applications accessing the Akamai CDN.

    Most DevOps teams are hyperfocused on what occurs inside the data center. But in the age of extended enterprise the level of dependency on caching and security services provided enabled by a CDN has never been greater. The fact that Akamai has also begun a more DevOps initiative is also testament to how far the DevOps movement has been able to extend its influence. Historically, providers of external services have been reluctant to provide any visibility into how their network services operate.

    Weil notes that it’s apparent the developers are also exercising more influence and control over IT. As such, providers of external services are now under more pressure to programmatically expose their services in much the same way as any other cloud service. In fact, many cloud service providers are leveraging their global footprint to make available CDN services that rival Akamai.

    Regardless of the CDN chosen, developers are under more pressure than ever from the business to optimize the end user application experience. Mobile and web applications are core to any digital business transformation initiative. In the face of that pressure, DevOps teams are finding that having more control over the CDN used to deliver those applications is now an absolute necessity.

    — Mike Vizard

  • Standardizing a Process that Drives Secure-By-Design

    Standardizing a Process that Drives Secure-By-Design

    In “Building Security into Code + Culture,” I described how our culture of coding is predicated on chaos, and the critical challenge for organizations is to establish the right mix of processes and tools that support a smarter, risk-based approach to development. A key aspect of this strategy is, of course, to implement a sense of order within the chaos that enhances security without hindering the speed of innovation.

    I overviewed a number of different recommendations to approach secure by design, such as making code more mature and baking in security measures and privacy/contractual compliance from the start. I also emphasized that organizations should tackle the rules, tools and fears pillars that help shape the culture, as each of these can be transformative in helping people use the right approach or method. Making the rules and defining fears is relatively straightforward, and with the right degree or training and awareness your organization can adapt.

    But what about tools? The right tools can do wonders to “force” people to use better, smarter, more secure methods—that is, if such tools exist. Though I have not done a deep dive into whether there are any robust development tools integrated with security software, I’m a realist and will go out on a limb to say none of them have reached maturity just yet. In light of this, it’s essential that organizations focus on standardizing the overarching process that will enforce privacy and security by design, development and deployment.

    The first thing to do is set the business drivers behind the process from a legal standpoint. Gather and research the privacy regulations and compliance requirements that affect your organization. Analyze and then translate these using a risk-based approach to security controls requirements, and then publish these as an awareness catalogue available to all developers across your organization. You should also ensure that training is available, on-demand and on a just-in-time basis, for any developer on any desired control.

    Next, set the technology drivers by prescribing to secure coding practices defined by the industry, such as the Web Application Security Consortium (WASC) – Threat Classification system (based on Open Web Application Security Project [OWASP] Top 10) and SysAdmin, Audit, Network and Security (SANS) Top 25 (which are based on the Mitre – Common Weakness Enumeration [CWE]).

    At this point, you must ensure that all your applications and systems comply with the security controls requirements catalogue you created. These can be batched into smaller parts that are handled by developers or teams of developers, all of which use your security controls requirements catalogue and industry-standard secure coding practices. Again, you’ll want to make sure that training sessions are available on a just-in-time basis for any developers who need them!

    Upon completion of each section or unit, code must be validated and verified for trust. Peer code review is the first step—have your developers present their work and provide the logic to a different team of developers. You can always employ static code analysis to scan code for any weaknesses, defects or violations of best practices, and subsequently enter the bug fix stage if any are detected.

    Then, you can merge the code branches into the main and compile for run time, so you can conduct live vulnerability scanning and penetration testing. Again, these should be based on the industry-standard secure coding practices adopted across your organization. At this point, you should be able to establish reasonable assurance for the release version of your application or system by having published artifacts for review by entities such as auditors or clients.

    Continuous improvement will require adding new features and subsequent emphasis on the validation and verification process. I would also recommend repeating the entire process from start to finish on a regular basis to reinforce the process, perhaps annually or semi-annually. And of course, if there are any major changes or shifts (as there are likely to be) in your privacy regulations, compliance requirements or secure coding practices, this will absolutely need to be addressed at the organizational level so your process addresses these head-on.

    Privacy and security are such important topics today because they are an artifact of the youth of the computer software industry. If this industry was more mature, then the outcome would be quality code. For some reason, it’s okay to create buggy, weak and inadequate software in a world in which we wouldn’t tolerate buggy, weak or inadequate cars, airplanes, bridges, buildings … and the list goes on. And the reason for this is pretty obvious: People are acutely aware that cars, airplanes, bridges and buildings put their very lives on the line, whereas software “just” makes them susceptible to things such as identity theft and ransomware.

    This is the message we’re sending: It’s not OK to kill people, but it’s perfectly acceptable to ruin their lives. This is not OK! It’s time for organizations to standardize the processes that drive secure-by-design, and embrace necessary and fundamental changes to how we approach development. As developers and IT professionals, the onus is on us to shape a better world driven by quality software.

    About the Author / Robert Hawk

    Robert Hawk is Resident Security Expert at xMatters. He has extensive experience in Information Systems Security, Computer Security, Cyber Security, Information Assurance, as well as Governance, Risk, and Compliance (GRC) Management. He specializes in frameworks and standards from: ISO/IEC, NIST, IEEE, IETF, ITU-T, Common Criteria, AMI-SEC, NERC, CIS, DoD, ANSI, PCI, and ISECOM. Robert is a lifelong researcher, innovator and instructor. Connect with him on LinkedIn.