devopsy.kinsta.cloud

Category: DevOps Practice

  • You Still Have to Make Time for Maintenance

    You Still Have to Make Time for Maintenance

    I’m going to take a side trek today into home computing with applicability to IT. Bear with me.

    We maintain a pretty complex home network, as home networks go. With three NAS servers, a small one-box SAN, some web servers and connections to several hosted environments. We both work from home, and my business is contained in one of the NAS servers and some of those hosted environments, so it’s all necessary. Scheduling downtime with several people using the network darned near 24/7 is difficult. Not medium enterprise level of difficult, but still an issue.

    Two of our NAS servers each have a disk out. It happens, that’s why they’re NAS servers. In general, the only day that replacing the disks makes sense is Sunday, when things are a little slower (both in schedules and network usage) than normal. One of our web servers is just plain dead–but there’s info on there we have to get off. The code is mostly being replaced, but the data is collected over years and we need to see how much we can salvage.

    None of these things are shiny DevOps things. Yet, they still have to get done. Importantly, they have to get done when automated tools aren’t running across the network or counting on the NAS to be responsive. The rush isn’t huge, because there are off-site backups, but restoring a remote backup is a long-term proposition–weeks, not days in our current environment–so it must happen as soon as practicable.

    Don’t forget that with the drive to automate everything. If you own the hardware, it’ll need maintenance. If you own the software, same thing. If you need data to move platforms, it will take time. All things we kind of hand-wave in the DevOps world. They’re seen as bottlenecks, if seen at all. Generally, we don’t even consider maintenance issues until they impact DevOps, but we know they are going to happen so we should have a plan for how to deal with them.

    Those few of you whose entire environment is hosted in a cloud provider have an easier job of it, but eventually you will be asked to move cloud providers. As sure as the sun will rise tomorrow, something will drive a move away from your chosen platform. Normally TCO, but other factors play in, too. When that happens, the work will be more than “Find time for the NAS to rebuild without overly impacting everything else.”

    So, spend some time thinking about it. Do you have plans to make room for maintenance? Are you scheduling maintenance into release cycles? Are you coordinating releases with updates that are not required by this specific release?

    This is where dev (typically Agile/Scrum) meets ops (typically system/network maintenance), and it is every bit as important as what is normally seen as DevOps. Get it right and have fun, keep cranking stuff out. Get it wrong and struggle with the problems infrastructure issues can inflict upon your cool new update. One of our new websites outright stopped development when the web server went down–for a variety of reasons, but the catalyst was the loss of the original. We’re working out what to do about that.

    Meanwhile, we didn’t make time yesterday, so I’ll be replacing disks and tearing apart a web server in a much smaller window that is available today. See you when I poke my head back up. Hopefully, you’ll have better planning than we did.

    — Don Macvittie

  • The Right Toolchain Is the One You Build

    The Right Toolchain Is the One You Build

    One of several iterations that IT likes to go through is corporate standards for technology versus best tool for the job. We like to have standardization because it allows staffing portability between projects, but we like right tool for the job because from the moment you choose a standard, competitive factors and changes in needs make the standard less of a big win.

    DevOps is still struggling through this process, particularly in enterprises where standardization bears a bit more weight on important projects.

    The fact is that DevOps is well suited to the idea of iterating standards–somewhere between a wild west of everyone using their “right tool” and an organization dictating the tool of choice.

    Most of us use Jenkins, because it does what we need it to do. What needs doing is different from organization to organization–some use it for continuous testing, while others go all the way through application release automation. But the fact is we could iterate and replace some, or all, of the functionality we’ve decided to use Jenkins for tomorrow. Kind of like a “We have a standard that is evaluated regularly” approach.

    For those new to DevOps, the array of choices is downright bewildering. Xebia Labs has a toolchain diagram page to help those already doing DevOps, and several sources have spreadsheets/PowerPoint templates to help you work through your existing environment. While all of the options these tools try to organize for you are great for the industry to find better solutions, it makes for a confusing mess. Your toolchain could be all AWS with some scripting, all Jenkins with plug-ins or 15 different apps working independently. A surprising number of organizations are even using Git for the bulk of their toolchain.

    What matters is that you know what you have, and know what your options are. Build the best toolchain for your environment today, and keep half an eye on it to be aware when something more suited to your environment comes along. We still maintain python scripts that do a fair amount of our automation, but several toolsets in place also.

    There are objectively better tools out there, but for our purposes they are almost universally overkill. So we go with what’s best for our environment. As time has gone on, we’ve integrated more and scripted less, but it will be a while before out toolchain is actually just tools–if ever.

    If your toolset is doing the build/test/deliver/deploy cycle well enough, you’re covered. If it’s not, you need to look for alternate solutions. Even if it is, make sure you’re aware of what’s on the market and not just what’s “hot” right now. Be aware of those companies that are up and coming, maybe in spaces that are already flooded. What differentiates them, and in our environment would that differentiation matter? That’s all you need to know–there is no need for a super deep technical evaluation, unless you’re actually considering them. Just enough to know if their differences add value.

    Keep cranking out the apps that make business run. Perhaps with an updated toolchain, perhaps not. In the end, what works for your environment is all that matters.

    — Don Macvittie

  • What Developers Should Take Away from the Iowa Caucus App Debacle

    What Developers Should Take Away from the Iowa Caucus App Debacle

    In the wake of the recent Iowa caucus, vote-counting software has fallen under suspicion after a smartphone app built by Shadow, Inc., brought chaos and delays to the first 2020 Democratic caucuses. Stopgap measures, such as Nevada officials canceling the application’s use in its upcoming caucus, are not enough to ensure vote tallies the electorate can be comfortable with. What’s needed is to correctly identify and fix the underlying problem. 

    The flaw does not lie with a lack of end-user sophistication, dataset complexity or inappropriate app security. Remember, government officials around the world are transforming urban areas into connected, responsive smart cities. Students are live-streaming interstellar space flights to Mars and Saturn. Uber drivers navigate past traffic jams, while seniors use apps to make time-sensitive healthcare decisions and adjust their retirement portfolios. People of all ages and experience levels do this and more, on the fly, with handheld mobile devices. 

    The real problem lies in the way leadership teams drive software delivery. The Iowa app failed for a number of reasons: It was released with no product testing after a short, two-month development runway conducted under a cloud of secrecy. It was not built to scale. Worse, there was little to no input, transparency or communication among the application’s most important stakeholders—the Shadow management team, the software developers and the end-users, in this case, precinct captains running the Iowa caucuses and party.

    The IT professionals in charge could have avoided these problems by following three simple principles of successful software development: 

    • Collaborate With All Stakeholders: Applications are no longer stand-alone widgets. They are digital experiences. With even the most sophisticated design, experiences that don’t serve their target audience are failures. A development process that prioritizes collaboration with end users can prevent such failures. Feedback from those with a vested interest in the outcome would have quickly raised red flags for developers. In Iowa, collaborating with intended users would have highlighted a need for accessibility training and connectivity issues, since the app was deployed in buildings with high-speed internet along with remote, off-grid locations. 
    • Build to Scale: Every developer knows to build for performance at an order of magnitude larger than you expect. This includes ensuring performance on the widest possible range of handheld devices and phones. From a developer’s perspective, the phone model will impact application performance due to different hardware specs. It’s imperative for designers to build apps that are native, so that all users, devices and performance profiles will function as intended. A useful software solution should reap success, not failure, when more people use an application than initially expected. 
    • Be Transparent: Valid concerns over security risks and election integrity unfortunately led Shadow’s team to prioritize secrecy over transparency. That didn’t work. Problems are not detected and solved when siloed or when small groups of people work on a solution in secret. Proper vetting and testing falls by the wayside. The risk that the app will not be suitable for its intended users increases. The app may have additional, exploitable security vulnerabilities that can only be found when teams ensure proper openness and transparency during the development process. Proper governance and guardrails that protect core data mitigates the security risk for election-based applications (and software in general). There’s little to be gained and much to risk by closing off input and vetting at any point during the full application development cycle. 

    Developers Move From Stressed to Empowered 

    One simple, effective way to solve or avoid these issues is by adopting an enterprise-grade, low-code software development platform. The “low-code” label refers to visual, model-driven software development, which employs an unambiguous visual language throughout the application development lifecycle, enabling nontechnical stakeholders, end users and professional developers to work together. 

    Such a collaborative platform can surface issues with senior leadership before it’s too late. Feedback from citizen developers—that is, not citizens in the voting sense, but rather, non-traditional developers—can capture leadership’s careful attention, precisely because the feedback is from nontechnical users. The right low-code platform would have enabled the Iowa caucus volunteers and stakeholders to collaborate on every step of the process without slowing it down.

    Another benefit of a robust low-code platform is having more people on deck to meet product goals with tight deadlines. Insights from citizen developers with unique backgrounds can also help when testing applications. With suitable governance controls, low-code functionality speeds up a developer’s ability to deliver software without taking away their ability to dive deeply and extend functionality with coding when needed. 

    Market analysts have evaluated the strengths and merits of different low-code platforms. In multi-experience use cases—such as Iowa’s, particularly because the eyes of the world are watching—it’s important to use a low-code platform that enables applications that are truly device native. This allows the software to take advantage of core device capabilities, typically delivering better performance and supporting the interactivity users are familiar with. Additionally, continuity online and offline would address connectivity issues in areas where signals are weak or nonexistent; the data can be properly synced when a signal is available. Lastly, a robust low-code platform provides tools to enable apps to scale automatically, avoiding the coding work required to allow an app originally built for 1,000 people to service 100,000 or a million. 

    An Urgent Role for Software Developers

    When roughly half of all eligible voters sit out general elections, talented developers should not lose sight of the many ways technology can aid and expand civic participation. Eventually, we may live in a world where election-based apps for registered voters are accessible and secure, regardless of the user’s device, age, location, physical ability or educational background. Building in redundant systems of paper-based vote confirmation and other offline systems will add integrity, inspire greater confidence and drive adoption of these innovations.

    What happened in Iowa is, unfortunately, not an isolated case. These complications are endemic to how most leadership teams drive software delivery. Development teams of all types will continue to run into similar issues until their leadership makes transparency, accessibility and scalability a priority by investing in the appropriate resources. Organizations can leverage the strength and impact of fully robust low-code software development platform to that end. 

    Since the introduction of low-code in 2004, platform capabilities have matured at an accelerated rate, moving to widespread adoption for enterprise use. During the same period, elections have increasingly relied on software-driven electronics, leading to consistent issues. Citizens casting ballots is one of the most sacred mechanisms of our republic. Any software used to tally votes must be bulletproof. Truly open and holistic low-code software development platforms can make this happen by proactively bringing people together to make sure software works for those who need it most.

    — Jon Scolamiero

  • Observability-Driven Development: From Software Development to DevOps and Beyond

    Observability-Driven Development: From Software Development to DevOps and Beyond

    In today’s world of cloud platforms and applications, typically delivered as SaaS (Software as a Service), it is critical for organizations to ensure efficient collaboration between software development and operations teams. The software development team codes the software application or platform, while the DevOps team streamlines and automates the entire production lifecycle, from development and testing to deployment.

    Today, applications are updated anywhere from weekly to daily, requiring fast turnaround. In the old days, every change request would go to the different stages–software design, development, testing and release. This can’t be sustained in an agile environment where updates must be as close to instantaneous as possible. 

    How can organizations achieve this, and where does observability fit into it all?

    What Is Observability and ODD?

    Observability-driven development (ODD) adds another layer to software development by encouraging the development team to think about the application availability and uptime throughout their development process and similar to unit-testing development, wrap their code with more verbose logging, metrics and KPIs. This gives the IT operations team more data on the application and improves overall observability, allowing an organization to detect predictable and unpredictable permutations of failures that may occur in the future. ODD observes the behavior of a system throughout the entire development cycle in order to learn what behavior is normal and abnormal, and to detect potential weaknesses.

    Observability provides everything with a real-time status of a product–this includes software developers, DevOps and any other department that requires it. Each role has different needs. Software developers need to know why a certain feature has to be added or developed in order to understand its business value and prioritize it over other features or updates. If a certain feature is likely to require many future updates, this also needs to be taken into consideration in the development phase so that the code can be adjusted easily without negatively affecting DevOps teams when they test and deploy the feature.

    These are just examples of things that must be taken into consideration during the development process. Organizations must be able to create transparency and effective communication between the different departments and roles.

    Inefficient Collaboration in the SaaS Market 

    DevOps need to ensure zero downtime, fast updates, constant monitoring and more. This can be done by maintaining an operational mindset during the development process. Traditionally, developers write the code, QA do the testing and operations take care of deployment, but there needs to be more collaboration between these three stages with each doing their part with an understanding of everyone else’s roles and needs pertaining to the product. When this isn’t done, numerous problems arise. Operational inefficiencies can ultimately lead to a delay in delivery time and even affect system availability. When there are multiple teams involved, especially in different locations and even time zones, there can be ineffective communication and hand-offs, particularly between software development and operations teams.

    Meanwhile, ad hoc changes must be implemented, negatively affecting product stability and release cycles. Miscommunication and inefficient operations lead to confusion over who is responsible for what task, feature and target. These are all issues that can result in significant downtime, losses, wasted resources and frustration within the organization. Effectively streamlining the software development cycle using full transparency and seamless collaboration between development and operations solves these problems.

    ODD Can Improve Your SaaS Product

    A strong proactive observability platform can predict and address issues before they occur, thus increasing effectiveness and speed when it comes to updating and tracking changes, as well as releasing new features. It simplifies complexities and management, and allows to automate operations so that organizations can maximize efficiency.

    Benefits of ODD include:

    • Common goals.
    • Shared metrics.
    • Execute on goals and recognize results.
    • Performance guidelines.
    • Application governance strategy.
    • Alignment between IT and business perspectives.

    Employing ODD when developing cloud applications is critical. This is due to the highly distributed nature of cloud architectures, increasing the chances of a failure exponentially compared to traditional systems. As a result, developers should be educated to think and implement observability to create highly available and resilient applications and systems.

    How to Integrate ODD in Your R&D Practices

    Today, development teams and practices are under constant pressure to meet scrum deadlines and release features on time. They are measured on their code effectiveness and meeting deadlines, while ODD adds another layer of measurement of serviceability and observability metrics to their considerations. 

    Implementing such an approach requires organizations to provide their development teams with the right toolset and knowledge on how their code affects the SRE and IT operations teams. From a technical standpoint, development teams should be familiar with all levels of monitoring for application infrastructure (for example Kafka, RabbitMQ, Kubernetes), application performance metrics (user experience, transactions per second, etc.) visualizations and their respective remediation processes. 

    ODD culture changes the way organizations think about their development process, and essentially injects an operational mindset to their daily practices and processes which should increase the availability and performance of their cloud application while streamlining collaboration and work processes, and improving teams’ satisfaction and productivity.

    — Avi Shalisman

  • Why BizOps Is Becoming Important and How It Can Help Your Teams

    Why BizOps Is Becoming Important and How It Can Help Your Teams

    Every 20 seconds, $1 million is wasted on IT investments that don’t align with business strategy. How much is your business wasting? To align business, development and IT operations, your teams need to adopt a new BizOps approach.

    In this blog, we’ll take a closer look at BizOps and why it’s becoming so important. 

    Why DevOps Falls Short

    The lack of alignment between business, development and IT is one of the main reasons why digital transformation initiatives are failing. Why is this such a widespread problem? While Agile and DevOps take steps toward continual improvement, they don’t fully connect your technology investments to business outcomes. 

    Therefore, barriers remain between business, development and IT, which fragments decision making and introduces risk. In a recent IDC BizOps report, analysts share that “Alone, Agile and DevOps provide great advantages at the project level but lack an enterprise and strategic context: your organization might be building things faster, but not building the right things.” 

    As your modern dynamic environments continue to grow more complex, the lack of alignment between technology investments and business objectives only continues to grow more critical and costly. 

    What is BizOps?

    To meet their agility imperatives, technology and business leaders are adopting an emerging methodology called BizOps. Like DevOps, BizOps is an approach, not a product or technology. BizOps is a framework for optimizing software development and delivery to foster the achievement of business objectives. 

    IDC analysts define BizOps as a data-driven decision-support mechanism that connects business and technology functions together to drive business outcomes. 

    The Obstacles and Requirements

    Teams that employ BizOps practices need to create a data-driven decision support mechanism that connects business and technology functions together to effectively achieve desired business outcomes. This isn’t easy: Enterprise leaders that look to pursue BizOps objectives are stymied by organizational silos. The result is that the data required is also locked in silos—stifling collaboration, insights and innovation. 

    In a blog post, Andreesen Horowitz shared that, to overcome these obstacles, teams need to take a new approach: “To support this new style of decision-making, organizations need to store, analyze and report on data in a fundamentally different way.” 

    Teams need to adopt new AI and machine learning technologies that augment and even automate decision making within a BizOps framework. Through this approach, enterprises can establish the continuous insights and collective intelligence that optimize decision making. In these emerging BizOps environments, decision making sees fundamental transformation, growing increasingly event-driven and automated.

    The Payoff

    BizOps joins technology and business functions to improve an organization’s ability to speed and optimize digital transformation. Through BizOps, your teams can ensure that digital infrastructures and software provide the operational characteristics—including speed, scale, efficiency and agility—that business services require. 

    By successfully employing BizOps, your teams can build a new level of transparency, traction and trust that drives collaboration across departments. Through BizOps, your teams can apply maximum resources and efforts to achieving vital business outcomes, such as boosting business growth, enhancing the customer experience and increasing profitability.

    — Kieran Taylor

  • The Definition of Faster in the Age of App Capital

    The Definition of Faster in the Age of App Capital

    Businesses want speed. One of the impacts of digital transformation—and the associated pressure to succeed in an app economy—is the desire to move at speed. Nearly half (48%) of organizations are driven by digital transformation to deploy faster according to our State of Application Services 2019.

    But it’s not all about deployment. It’s also about development and responsiveness to threats and changes in demand.

    Organizations want to develop and deliver apps faster. They want to adapt to unpredictable changes in business conditions faster. They want to respond to attacks faster.

    So, how are they going to get it?

    Faster Development

    Deploying more frequently implies you’re first developing faster. That often means adoption of speed-oriented Agile methodologies. GitLab’s 2019 Global Developer Report: DevSecOps found that the majority (54%) are choosing Scrum, while 37% go for Kanban.

    Componentization

    Methodologies aren’t enough if application architectures aren’t well-suited to development styles. Smaller, focused teams sprinting toward frequent releases isn’t really a good fit for developing new features or fixes for traditional and monolithic applications. Componentization through the adoption of microservices and more distributed forms of architecture fit more naturally with modern architectural and operational principles. It’s no surprise that, on average, more than 80% of a modern application is made up of third-party components, heavily weighted toward open source.

    APIs

    Adoption of APIs, as well, is no surprise amongst those seeking faster development through componentized application architectures. APIs decouple implementation from interface and allow teams to make modifications to processing without impacting the use of APIs from other components or applications. This is a common pattern, with an impressive 64% of organizations creating APIs today for use in either internal or external use cases. Nearly 50% of those organizations rely on APIs to get ideas to market faster according to Jitterbit’s State of API Integration 2018.

    Automation (CI/CD)

    The build pipeline, too, has to keep up if frequently deployment goals are to be met. That means CI/CD tools that seamlessly move code from commit to test to release. The most popular CI and build tools in GitLab’s survey were, unsurprisingly, GitLab (61%) followed by Jenkins (36%) and Travis CI (12%). Notably, our own research finds Jenkins in use by 16% for network automation as well—a promising result if organizations are seeking to extend DevOps beyond delivery into deployment.

    Faster Deployment

    Developing faster doesn’t get ideas to market faster. That requires deployment. While born-in-the-cloud technology companies have mastered the delivery-deployment divide with ease, many established enterprises find this transition challenging. Existing organization structures as well as the continued need to support traditional and monolithic applications introduces conflicts that can be challenging to integrate with modern operational demands. But there is no doubt that organizations must overcome these challenges for those applications requiring faster and more frequent deployments.

    Continuous Deployment

    Don’t be fooled; established organizations are embracing automation and applying it to the deployment pipeline. The issue is often that traditional IT structures introduce inconsistent automation and self-service efforts. We see this play out in our own research, where strongly siloed team structures continue to impact automation efforts across IT.

    Team structures matter and organizations whose compass is pointed at automated pipelines will need to address the cultural aspects of continuous deployment if they want to consistently deploy faster.

    Public Cloud

    Traditionally, the inability to achieve continuous deployment has pushed developers and app owners to turn to the public cloud, because it eliminates the obstacles in the slower, enterprise deployment path. The disparity in opinion on frequency of deployments is, in part, a contributor to the issue. Our 2018 NetOps/DevOps survey found that while 55% of DevOps and 52% of cloud architects felt their organization did not deploy frequently enough, only 30% of NetOps and the same percentage of operations agreed. 

    But it’s not the only driver. Digital transformation is certainly a factor; 33% of respondents in our State of Application Services indicated they are intentionally moving to deliver apps from the public cloud as a result of embracing digital transformation initiatives. The ability to easily integrate application services and subsequently automate operation is a huge boon to organizations seeking a faster route to frequent deployment.

    Containers

    In the cloud—but also on-premises—continuous deployment often requires the ability to adopt per-app pipelines and application services that support the model. Increasingly, organizations are turning to containers because of their ability to support rapid updates and operate seamlessly in highly volatile environments. We see containers in demand not only to support modern application architectures such as cloud native—the top container use-case at 33% of respondents to Diamanti’s 2019 Container Adoption Benchmark—but also for infrastructure. The demand for on-premises application services in containers has consistently grown year over year in our own research, rising from just 4% in 2017 to 15% in 2019.

    Faster Responses

    Not everything is about delivery and deployment. Organizations also need speed when it comes to security and operations. In today’s world, where more than half of all app interactions are conducted by bots, it’s important for organizations to respond quickly with a reject message to avoid falling prey to exploits or infection. 

    Real-Time Threat Analytics

    One of the ways organizations are seeking to speed up their ability to respond to attacks is through the adoption of real-time threat analytics. With security continuing to be a top priority and consistent challenge, it was no surprise to see this category ascend into the top five strategy trends and technology in 2019 with 41% of respondents tagging it.

    The availability of intelligent security application services will accelerate as machine learning and automation continue to apply their considerable capabilities to the problem of identifying troublesome traffic faster.

    Orchestration Included

    Machine learning isn’t the only way in which infrastructure and application services are getting smarter. As the need for faster responses to capacity and processing grow along with users, infrastructure and application services are evolving to include orchestration as a core capability rather than an add-on. Application service platforms with integrated orchestration layers will mean services that scale along with demand—on-demand. While such capabilities exist today—indeed, they are part and parcel of cloud and containers—the capabilities to automatically scale up and down based on application and user defined requirements is not native to most systems today. But they will be.

    Faster Identification of Bad Actors

    Finally, organizations need to combat bots for both business and security purposes. Whether organizations need to stop scraping—a real threat to business—or prevent bots from probing for vulnerabilities, identifying bad bot actors faster is critical today. Checking a box (I’m not a robot) isn’t enough today as bots are growing more intelligent and are able to foil such primitive techniques.

    Organizations are turning to Bot Defense services capable of applying more modern, successful techniques to quickly identify and block bad actor bots. We’ve seen the use of bot defense services grow quarter over quarter in response to this need, and expect this trend to continue.

    Faster Development, Faster Deployment, Faster Responses

    When it comes to business’ need for speed today, we wind up looking at processes. Development processes are automated to gain speed with CI/CD. Deployment processes are automated to gain speed with continuous deployment and cloud. Scaling and security processes are being automated by systems and the introduction of orchestration as native to application service platforms.

    Faster, it turns out, relies on an organization’s ability to automate processes across development, deployment and security. 

    — Lori MacVittie

  • Automation Hacks for DevOps Teams

    Automation Hacks for DevOps Teams

    DevOps teams are expected to do more work in less time. They need to plan, test, build, secure and release applications at a fast-paced rate. Of course, the DevOps team needs to achieve all of this while maintaining high software quality. This is why automation is a crucial part of the DevOps workflow. The goal is to delegate repetitive tasks to the machine, and free up dev time for more complex jobs. 

    Below are some helpful automation hacks you can apply to boost the effectiveness of your DevOps practices.

    Automating Regulatory Compliance

    Properly managing regulatory compliance is vital for many organizations to avoid fines and operate legally. Although compliance standards vary by industry, most focus on ensuring privacy and confidentiality. To meet these standards, companies must secure data and prove they are taking the appropriate measures. This proof is commonly accomplished with the maintenance of a clear digital trail.

    You can use DevOps automation to log and store digital access trails for your protected data. You can also use automated testing and deployment to ensure that only code that passes compliance tests is deployed. This prevents compliance issues from being pushed to production and helps keep your data secure.

    Automating Development Environments

    DevOps workflows require frequent creation and elimination of environments, for testing, deployment and development. These environments can be created manually but it is more efficient to create environments automatically. This can be done using a golden image with your required configurations.

    Using standardized base images ensures that you are working with consistent environments and standardizes configurations and privileges. These images can be used for local or remote machines and can be initialized automatically.

    There are a variety of tools you can use to create and automate environment creation. Chef and Vagrant are two of the most popular. Both of these tools offer an accessible API and enable you to script automation using common languages and syntax.

    Automating Code Review

    Code review is an essential part of DevOps processes but it is also one of the lengthier and more tedious processes. Automation can help you perform this process more efficiently. Automatic code review works similarly to manual review except it is done using tools that integrate with development environments. These tools can check for styling, security, duplication, complexity and code coverage, and can auto-suggest corrections. Some popular tools for this automation include Codebeat, Code climate and Codacy.

    Automating File Synchronization and File Uploads

    Many DevOps and development teams use the public cloud for quickly spinning up development and test environments. However, what happens when the production environment, or other critical resources used in the dev/test process, are deployed on-premises? Having a fast, automated and reliable way to synchronize files between the on-premise data center and the cloud can save time and headaches.

    There are a few approaches to synchronize files:

    • Scripting file uploads and downloads. The most basic method is to use shell scripts and FTP or SFTP to upload and download files between local and cloud environments. However, this can be fragile, unreliable and difficult to scale.
    • Using APIs of cloud storage services. You can move data to and from the cloud using cloud storage APIs, such as the Amazon S3 API, the Box API or specialized hosting services such as Cloudinary.
    • Hybrid cloud. The ultimate solution is to create a hybrid cloud architecture where applications, developers and users view the local environment and the public cloud as one unit, and data is synchronized transparently in the background.

    Collaboration Tools for DevOps Teams

    There are a number of collaborative tools that teams can use to further boost gains from automation. These tools can help bridge gaps in automation and processes and ensure smooth workflows.

    Slack: a chat tool that DevOps teams can integrate with automation tooling. It includes features for segregating discussions by topic, team or purpose; uploading shared files and documents; and initiating or receiving alerts from automated processes via integrations.

    Jira: a tool that enables you to create and assign tasks. It includes features for scrum and kanban methodologies, unified reporting and backlog tracking. You can integrate it with many common DevOps tools, such as Jenkins.

    Trello: a cloud-based Kanban board tool. It enables you to create project boards that can be shared across teams to track project progress. Through integrations, you can automate the updating of cards or initiate processes when cards are manually moved.

    Asana: another cloud-based Kanban tool. Like Trello, it enables you to track and collaborate on projects across a team. Typically, when teams use tools such as Asana, they create multiple boards for different project aspects. These boards can then be viewed individually or from a central dashboard.

    Conclusion

    Hopefully, this article helped you figure out how to leverage automation for improved efficiency. There are many automation options for DevOps, including automated regulatory compliance, automated infrastructure, automated code review and automated file synchronization.

    For DevOps project management, you can use collaboration tools such as Slack, Jira, Trello and Asana. Remember to make sure that any tool you introduce can be simply and easily integrated into your current environment. The tools of your choice also need to fit the skillset of all collaborators. For ease of use, keep your tooling ecosystem simple and minimalistic.

    — Gilad David Maayan

  • Six Routing Challenges When Managing Microservices in Production and How to Avoid Them

    Six Routing Challenges When Managing Microservices in Production and How to Avoid Them

    Even experienced DevOps teams can struggle with the nuances of managing microservices in production–where a greater spotlight on network communications can introduce new operational challenges beyond those common to traditional architectures. But, by being smart about your software-based routing components, it’s possible to avoid or reduce the impact of roadblocks in the following areas.

    Implementing Tracing and Monitoring

    Even applications that are rigorously tested during development will exhibit anomalies in production. To address these, DevOps teams require effective tracing and monitoring tools to gain visibility into what’s happening within clusters at runtime.

    As a strategy, developers should select routing solutions that integrate well with their existing monitoring and tracing backends. Remember that while a microservice may be developed by a certain internal team, once deployed it becomes available for others. If the application is instrumented to be traceable, compatible tracing tools can dynamically identify the source of service invocations and their call flows–enabling other teams to utilize the microservice. When the application is also designed to provide metrics, monitoring tools can usefully track the microservice’s key resource consumption metrics to assess the scalability of its subcomponents under load, and recognize any performance limitations that require attention.

    Because all inter-service communications travel through software routing elements, you can strategically design your application to enable microservices tracing and monitoring data to be collected. Teams can then put this data to work to more easily overcome challenges when building out new capabilities.

    Mitigating Communication Failures

    Considering the myriad communications taking place within production microservices-based environments, it only makes sense that this architecture will be prone to communication failures in which microservices simply cannot be reached. Whether these failures occur as the result of an error by the container, host, a network partition, or a short-term interruption in the availability of the service itself, resilient safeguards must be in place to mitigate any impact to an application’s user base. 

    For example: When an instance fails, the load balancer can be used to automatically reroute requests to healthy instances, and then return traffic to the failed instance upon detecting that it’s again available. In scenarios where an entire microservice becomes unavailable, client services should leverage circuit breaking to stop requests, return an error and initiate a fallback response. Doing so avoids dangerous repetition of retry requests, which only eat up resources and can lead to cascading failures.

    Each of these mitigation strategies could be accomplished by placing the necessary logic with each client. However, this technique would be both cumbersome to implement and prone to errors. The better approach to these challenges is to leverage routing solutions to perform instance health checks and circuit breaking, allowing DevOps teams to control how their applications respond to faults, rather than relying on logic at the network layer.

    Handling Unanticipated Load Spikes

    In production environments, loads can spike unexpectedly and for any number of reasons–ranging from simple sudden popularity with users to malicious distributed denial-of-service (DDoS) attacks. Whatever the root cause, applications must be well-prepared to withstand these events without failing.

    To ready applications to handle load spikes, rate limiting is a simple (but powerful) technique that allows operators to specify the request rates to front-end services. Implementing this safeguard through a routing solution answers the challenge of avoiding downtime due to surprise changes in load intensity.

    Securing Microservices Communications and Permissions

    Security is always a constant concern when it comes to operating production deployments. However,  microservices environments often have absolutely zero enforcement of internal network communications permissions. They also face the challenge of implementing proper encryption to secure sensitive data within both internal and external traffic. 

    While allowing any service to access any other offers flexibility, the risks are substantial. In order to safeguard microservices that access and expose sensitive data, it’s wise to limit access to only those client services with a vetted business justification. Routing technologies make it possible to enforce these secure policies using network segmentation.

    At the same time, routing solutions can help businesses encrypt the in-flight transport of critical data, which is not only a best practice, but in many industries a necessity from a regulatory compliance perspective. Service meshes can offer the benefit of providing secure TLS encryption for all internal east-west traffic, while edge routers offer to encrypt external north-south using a provided certificate. Also of note for north-south traffic: combining routing technology with services such as Let’s Encrypt can serve to fully automate lifecycle management of trusted certificates, eliminating the need for human intervention entirely.

    Achieving High Availability

    Leveraging software routing technology to achieve high availability is more cost-effective and efficient than traditional hardware failover solutions. It offers the opportunity to build production microservices environments that are quite resilient. For example: Routing implementation should utilize a highly-available architecture that includes a horizontally scalable data plane and a separate fault tolerant control plane. While the data plane allows for adding instances as necessary to meet capacity and resilience needs, the control plane is prepared to tolerate failures and ensure that users experience seamless and uninterrupted uptime.

    Deploying Across Multi-Cloud and Heterogeneous Environments

    Larger enterprises often utilize deployments across multiple cloud and/or on-prem environments, each of which may rely on disparate container orchestration technologies. Highly portable routing technologies capable of supporting a range of orchestration tools can vastly reduce the challenges developers face in supporting multiple environments. This allows them to deploy the same familiar solution and a common routing layer model across deployments.

    Routing technology is a key variable in determining the capabilities and ease with which DevOps teams are able to overcome challenges to the success of their production microservices deployments. By making a carefully considered choice in selecting a routing solution, teams can better prepare for these challenges–or even avoid them altogether.

    — Manuel Zapf

  • Survey: Internal IT Teams Embrace ‘DevOps Inspired’ Processes

    Survey: Internal IT Teams Embrace ‘DevOps Inspired’ Processes

    The “2020 State of Application Services” survey published today by F5 Networks suggests more IT organizations than ever are adopting some elements of DevOps principles to manage an increasingly diverse application portfolio.

    Based on a global survey of 2,600 IT professionals, the report finds 63% of organizations still place primary responsibility for application services within IT operations. However, more than half the organizations surveyed said they are moving toward “DevOps-inspired” teams. In addition, 73% of respondents said their organizations are automating network operations to boost efficiency.

    Lori MacVittie, principal technical evangelist for F5 Networks, said the report makes it clear many IT organizations are trying to adopt best DevOps practices gradually. They may in some cases set up dedicated DevOps teams, but the preference long-term is trending toward a more evolutionary transition, she said.

    At the same time, application environments are becoming more complex at a faster rate. The survey finds three-tier web and mobile application architectures come are most common (40%), followed by client-server applications (34%), microservices/cloud-native applications (15%) and mainframe monolithic applications (11%). The survey also finds 87% of organizations are using multiple clouds, but only 27% of respondents will have more than half of their applications in the cloud by the end of 2020. It’s also worth noting less than half the applications being managed (45%) are external-facing.

    In terms of application services deployed today, the most common are SSL VPN and security services such as firewall, IPS/IDS, antivirus and spam mitigation, followed by load balancing, global server load balancing and DNS and web application firewall (WAF) and distributed denial of service (DDoS) protection. In the next 12 months, however, software-defined wide area networks (SD-WANs) take the top slot at 41%, followed by DNSSEC (37%) and service mesh (37%). Botnet protection, anti-fraud and common performance optimization services fill out the top six, with 36% of organizations planning to deploy each.

    MacVittie said the survey results make it clear IT organizations will find themselves having to cope with more complex IT environments for years to come, with the most intense pressure being on cybersecurity. The survey finds 71% of organizations admit to having a skills gap in security.

    Overall, 62% of respondents said they are confident in their ability to protect applications in an on-premises data center. However, only 45% of respondents said the same for applications residing in public clouds.

    The study also suggests there is a strong link between organizations that have embraced digital business transformation and the need to build and deploy applications faster. A total of 80% of respondents said their organizations are executing on some form of digital transformation.

    However, as organizations make that transition, it is apparent IT teams are challenged by the need to be more responsive to the business, said MacVittie. The day is coming to an end when IT could operate as a utility that was managed by open and closing tickets as regimented tasks were completed. IT teams need to embrace DevOps to respond faster to rapidly changing business conditions, she said.

    It’s unclear how long it may take for best DevOps practices to be absorbed across mainstream IT teams. However, the one thing that is clear is that many of the DevOps concepts associated with ruthlessly automating IT in the name of agility are finally starting to hit home.

    — Mike Vizard

  • The Fallacy of Move Fast and Break Things

    The Fallacy of Move Fast and Break Things

    Ever since Mark Zuckerburg uttered the phrase “move fast and break things,” it has become the motto of many development teams. Companies wanting to be the next unicorn decided this must be the way to operate. The race was on to release more, ship faster, never stop. If moving slowly and methodically wasn’t working, doing more had to be the key to success.

    These companies would cherry-pick statistics from reports such as the Accelerate: State of DevOps Report. The 2019 report showed elite companies have 46 times more frequent code deployments than low-performing teams and a 2,555 times faster lead time to move from code commit to deploy to support their initiatives to move faster.

    The problem is moving fast doesn’t work across industries or for all teams. To effectively move fast, you need processes in place to support the velocity. The consequences of moving too fast and not being able to fix things when they break are high. 

    How We Got Here

    What has gotten us to the point where we are moving too fast? In short, we have. How do you feel when it takes months or a week for a PR to be resolved compared to hours or days?  As consumers and end-users of software, our expectations are continually rising. We have to ship quickly because that is what we as consumers expect. As customers, we are pushing companies, which in turn push their employees to meet the increasing expectations.

    What are the consequences when you try to move fast and not just break things, but fail? 

    • It takes longer to resolve incidents.
    • You lose customer confidence and sales.
    • Employees burnout and you have a high turnover rate.

    Let’s go back to the stats from the Accelerate: State of DevOps Report. Yes, elite teams ship faster. But, their changes are 1/7 as likely to fail and they recover from incidents 2,604 times faster than low-performing teams. It’s not just about moving fast and breaking things; it is about having the right systems and processes in place to support this way of working.

    Setting Yourself Up for Success

    You need two things to effectively move fast: a culture of psychological safety and smart investments in tooling. Employees need to feel empowered to speak up if things are moving too fast, if they are concerned about why a feature is being built and to identify gaps in the processes. They need to feel they won’t be blamed when something breaks. Building this requires empathy, open communication and teamwork.  This psychological safety is the foundation of being able to move quickly and quickly recover when things break. 

    Next up is selecting the right tooling and processes. Invest in tools that make things easier. Tools should be useful, usable and change the underlying problems, not create more. 

    Think about the tools in place to quickly resolve incidents when something fails:

    • Observability and monitoring tools to identify and notify when things go wrong.
    • Incident management tools to route, track and escalate issues.
    • Feature management tools to enable circuit breakers and load shedding to turn off features quickly. 

    The culture and tools are part of the equation; the final piece is having the right processes in place to effectively use the tools and support the people. 

    What are some processes you can implement to enable safety at speed?

    You should attempt to move fast, you should try to break things—but only when you have the right protections and processes in place. Using a combination of the right tools and processes, you can deliver more value faster without sacrificing quality or your employees’ health.

    — Dawn Parzych