Category: DevOps Onramp

  • 5 Tips for Securing DevOps: What You Wish You Knew Sooner

    5 Tips for Securing DevOps: What You Wish You Knew Sooner

    Foundations and frameworks, concrete and steel—not exciting. But that’s the foundation and framing of pretty much every modern building. Everything else that is part of a building–flooring, wiring, lighting, room placement and so on—is made possible by the foundation and the framework. If you want DevOps to succeed in your organization, you have to start by building a solid foundation and a framework, including security. What’s below is a suggested approach to the concrete and steel of DevOps.

    Three Foundational Needs

    1. Leadership and Governance
    Someone has to make the decisions, be responsible for leading the team(s), give direction and provide operational governance. Securing DevOps is not just about compliance with a set of security concepts; it’s also about setting the right tone for the workplace. Leadership and governance are key enablers of development success.

    The pressure to deploy software faster and more often is real—but so are the security, regulatory and many other concerns around automation that may uncover bugs or defects in software before it gets pushed to production. In an environment where there’s pressure to release code faster, how does one balance these competing priorities? What’s more important: Speed or quality? This tension between speed and quality remains one of the main challenges faced by organizations adopting DevOps practices today.

    Organizational leadership and governance regarding people and data must be in place to keep people working on the right things, keep up morale, provide direction about project prioritization and budget the right tools for getting the job done.

    An important subcategory of governance is data classification. Where does the data reside? Over which channels does it flow? What kind of data is it? Classifying the data has long-term implications for resource procurement such as what storage to use and if it needs to be encrypted, how stringent access control for the environments needs to be and interoperability of the various technologies.

    2. Regulatory Compliance
    It wasn’t too long ago that if a business remained only national, then there weren’t necessarily many regulatory requirements. Presently, if a business has a website (and who doesn’t?), then international and cross-border data transfer is a given unless one takes the course of geo-restriction, which in itself can be a major undertaking with repercussions.

    Determine regulatory needs. From where do people or other services access the data? What cross-border transfer is involved? To paraphrase the late 1960s-1980s public service announcement, “It’s 10:00 p.m. – do you know where your data is?” Design and implement accordingly.

    3. Risk Management
    One cornerstone of technology management is that it’s all about risk. If there’s a serious incident such as data intrusion or theft, what will the cost be to the organization and was the company part of the problem (e.g., negligence)?

    What are the technical vulnerabilities, threats and risks presented by the product or service? What kinds of testing (e.g., regression, penetration, vulnerability, CTI) would it take to assess and address the risks? What does the patch cadence look like?

    On the personnel side, what are the risks of key players leaving? What’s the morale of the team regarding aspects such as culture and pay? What’s being done to address those risks? Included are third-party, contractual, and privacy risks (remember data classification above?).

    NPM packages are a growing trend as an avenue of exploitation; uptime is a common requirement for meeting obligations, and privacy is considered by many to be more than a privilege but a human right, and not something to take lightly.

    Two Framing Activities

    The foundation activities undergird the entire structure of DevOps, but also consider what shapes the structure.

    1. Software Development Life Cycle (SDLC)
    The SDLC is vital. Without it, where do all personnel (team, contractors, consultants) know how to proceed with designing and implementing the product? What model is used, and who’s responsible for what? It’s all here in the SDLC. Of course, things like references for API architecture specifics and corollary application documentation would make one feel like wandering in the Library of Congress (and make developers want to burn it down like the ancient Egyptian Library of Alexandria!).

    Speaking of APIs: Due to the enormous rise of API use in all industries, including health care and retail, any use of APIs must be part of the SDLC.

    Not only are APIs on the rise for public use and interaction, but microservices use within an org has increased drastically. APIs are necessary internally and externally for businesses to reach, maintain and accelerate the speed of innovation, so including them as critical resources is crucial. The average API usage per company increased by 221%, so attending to that ecosystem by ensuring proper activities, especially security, are included is needed for developers.

    The SDLC is there to make workflow and production consistent (and concomitantly easier) by codifying what needs to be done for both company and customer, not to make work more difficult by presenting nauseating tedium. Like any corporate policy, it needs to remain useful, while not watered down.

    Here are a few specific aspects to include:

    ● Threat modeling in some form needs to be included. It doesn’t have to be a magnanimous effort, but the realities of threats will play a major part in development.
    ● Testing. Regression, stress, pentest, security – these and others will help guide what kind of testing environment one will need and lead to any need to increase items such as capacity and scalability.
    ● Secure coding. If it’s being done, take steps to improve. If it’s not being done, just get started. It’s always hard to go back and fix insecure code, but it’s a necessity due to the many risks and regulations involved in providing software to the public.

    2. Training
    Training is required because businesses have to adapt as technological changes create new opportunities that lead to continuously changing customer demands.

    Training is everywhere! There’s no one right place or right approach – just get training and change as needed (specifics should be handed down by leadership).

    On a strict and lowball budget? One option is OWASP membership (which costs $50 per person per year) and includes several types of AppSec and DevSecOps training. That small investment can help raise the level of clean and secure coding, threat and security consciousness and professional reputation.

    Setting the Course and Always in Motion

    An issue with setting any course in business is that it’s semi-permanent. A DevOps model put in place becomes part of the foundation, which can make one reticent to commit. The only way to prove the chosen methods is to set them in stone and do them consistently for a long time. Aspects of the model may change over time, but once that foundation is put in place, it takes an enormous amount of work, time and money to change it. The change wouldn’t be a remodel, but a reconstruction, so the model takes proper consideration at the outset to chart the right course.

    While DevOps needs a solid foundation, it’s an always-in-motion model. There are new technologies, new people, business changes, customer demands and societal and cultural shifts. These concepts can be included in calculations, but what those changes will be is incalculable. Hire and train the right people and the necessary steps and adjustments will be possible.

  • Rust Momentum Intensifies | Elon Says No WFH

    Rust Momentum Intensifies | Elon Says No WFH

    Welcome to The Long View—where we peruse the news of the week and strip it to the essentials. Let’s work out what really matters.

    This week: People won’t shut up about Rustlang, and Musk mandates Twitter teams return to the office. (more…)

  • Developer’s Guide to Fuzz Testing

    Developer’s Guide to Fuzz Testing

    Fuzz testing continues to grow in popularity among developers within the open source community. Google’s open source security (OSS) team recently reported finding more than 40,000 bugs in 650 open source projects through the use of fuzz testing. But if you have no clue what fuzz testing is (yet), don’t worry. You’ve come to the right place.

    What is Fuzz Testing?

    Fuzz testing is a dynamic testing method for finding bugs and security issues in software. During a fuzz test, a program or a function under test gets executed with invalid, unexpected or random inputs to uncover unlikely or unexpected edge cases. 

    Value of Fuzzing for DevSecOps 

    As we know, DevSecOps extends the approaches of DevOps and Agile by adding suitable security testing methods alongside every phase of the software development life cycle (SDLC), creating a dynamic and continuous testing process. 

    Part of that testing process can and should involve fuzzing to detect security and stability issues at all phases of the SDLC and empower developers to ship software quickly and securely. For example, to prevent regressions from going live by running automated fuzz tests on every code change.

    Especially in the automotive industry and complex embedded projects, where a lot of untrusted and complicated data gets processed, fuzzing brings excellent benefits to DevSecOps. Indeed, developers can apply fuzzing similar to unit testing to automatically test their security-critical modules for potential vulnerabilities. But with fuzzing on an integration level, they can also cover all the complicated and dangerous edge cases that can occur in the interaction between different modules. 

    Fuzz Testing Benefits

    Fuzzing code generates thousands of automated test cases in a second, with the fuzzer receiving in-depth feedback on code coverage and the program states reached to simulate the interaction of different modules during the execution of the source code. The fuzzer can adapt and mute its inputs based on runtime feedback in the following iterations to maximize code coverage. Some of the benefits are:

    1. (Almost) no false positives: If you find a crash, fuzzing provides you with the error source and the malicious input that caused the issue. So, you can be sure that each finding is a real vulnerability.
    2. Debug reproducible findings: Modern fuzz testing tools will provide you with the input causing the issue, including the source of the error. With this, you already have a lot of helpful information to reproduce the bug and load it directly into the debugger. 
    3. Find issues in forgotten edge cases: Because fuzzing is executing a program with invalid or random input, it also uncovers unlikely and unexpected edge case errors that would not otherwise have been revealed.

    Who Uses Fuzz Testing?

    As cybersecurity regulations and standards continue to expand globally, more and more organizations are extending cybersecurity programs and running automated security testing before they ship software. Whole industries, especially those that involve advanced quality and security regulations, are setting standards for security testing that recommend fuzz testing, including automotive, aviation, finance, healthcare, telco and energy.

    Digging deeper into the automotive industry, for example, a variety of International Organization for Standardization (ISO) and other organizational standards now recommend fuzzing:

    Best Practices

    The most effective way to do security testing, fuzzing included, is to do so continuously. This means integrating fuzzing right into your CI/CD pipeline. Doing so creates quick feedback cycles so developers can fix security vulnerabilities before the code is shipped. Integrating fuzzing tools into code hosting systems like GitHub is also valuable. Doing so allows communication and alerts to be shared across your DevOps team as soon as a problem is uncovered, which helps to facilitate fixing vulnerabilities. 

    What Bugs Can I Find?

    While the practice remains relatively new, fuzzing has already discovered thousands of bugs across different use cases. Bug and vulnerability types discovered include:

    • Remote code executions
    • Injections, which can be triggered by untrusted inputs 
    • Memory leaks: Incorrect memory allocation 
    • Exposure of sensitive data: Accidental exposure of personal data
    • Functional bugs: Inputs that are not being responded to 

    and many more bugs, such as undefined behavior, uncaught exceptions and buffer overflow. Learn more.

    Getting Started

    You can try out an open source fuzzer, like Jazzer (for Java testing) or the CI Fuzz CLI (for C/C++ testing). When you’re comfortable with fuzzing and want to use this approach in more complex environments, enterprise solutions offer added features like API fuzzing, reporting and CI/CD integration. 

    So give it a shot and join the community of open source developers using fuzzing to quickly and securely ship their software. 

  • How DevOps Helps With Secure Deployments

    How DevOps Helps With Secure Deployments

    DevOps is an approach to software development that emphasizes communication and collaboration between teams. It’s best known for bringing together people who previously worked in separate areas like engineering or testing; this allows them all to learn from each other as they work on projects together.

    DevSecOps helps organizations monitor and discover security risks throughout the development process rather than spending a lot on post-launch protective measures. This way, they are able to secure their product without overburdening developers with heavy-duty responsibility for what might turn out as an easy fix given time or knowledge differentials between team members—after all, they are responsible for building these things!

    The Goal of DevSecOps

    The goal of DevSecOps is to provide security best practices in a way that doesn’t disrupt team productivity.

    Secure development is the key to a smooth deployment process. It’s frustrating when you have security in your product but don’t see it being implemented or taken seriously. DevSecOps brings back that focus by reminding everyone just how important and necessary good hygiene practices are for both developers and operations.

    For security to be more effective and reliable, it should be incorporated from the very beginning. This means that instead of waiting until there is an issue or crisis before implementing measures for protection such as firewalls, encryption keys, etc.; you want your developers working on it up front so they can ensure everything will work well together later.

    It helps ensure security issues are found as early in the process as possible so it is close to decision-makers. It’s much easier (and less painful) when security issues can be fixed while you still remember what happened with your project—it’s like having an extra set of eyes on the project.

    Advantages of DevSecOps

    Security should be a high priority for any company looking to stay afloat in this digital age. From reducing risk to mitigating legal liability, identifying bugs earlier on during the project life cycle makes security management easier when things go wrong. Security becomes less painful as well since stakeholders will understand better why delays happen instead of just because “the team found issues.”

    If you shift your focus to include this earlier in the process, then costly issues can be discovered much faster and closer to where they happen, which will allow for more timely responses from developers or infrastructure engineers who must address those problems before moving on to another task.

    The goal of this system is to give Dev and Ops faster feedback on the work they are producing so they are notified right away.

    Achieving DevSevOps

    There’s no one right answer when it comes to implementing DevSecOps, as the needs of each organization will be unique. However, there are some common practices that can help to get you started.  

    Get Management Buy-In

    To make a fundamental shift in how your organization thinks about, builds and deploys software, you will need more than just peer-to-peer agreement. Management buy-in is necessary for this revolutionary change of mindset.

    The DevSecOps process is one that can help combat software vulnerabilities being exploited in production. It’s important for those who make decisions on risks versus revenue to see how this will benefit the company, or else it’ll be fruitless.

    Security is something that we should all be responsible for. It’s not just a concept or idea, but rather something tangible and real—something which can affect your company greatly if handled incorrectly by someone who doesn’t know what they’re doing. That’s why it always pays off in the end when employees take their own security concerns seriously; this will result in better decisions being made about how best to protect your assets from outside threats.

    You can’t just implement a new policy without management backing it up.

    Developer Responsibility

    DevSecOps takes security seriously by making it a part of the development process, not an afterthought. The entire team can work together to write more secure code and configurations that are robust against threats while also being easily reversible if something goes wrong—because they’re writing this stuff right from their own desks!

    It’s not just a job for security professionals anymore, either. It needs everyone from the CEO to contract workers to take responsibility for securing their own work instead of outsourcing the responsibility and expecting others to do it retroactively every time there is an attack or threat.

    Security and DevSecOps training

    There’s no quick fix for organizational culture, but it can be done. People are always resistant to change. There will initially be frustration as traditional behaviors are questioned and you create new ways of working that might not fit everyone right away.

    Managers who have a good understanding of these concepts will be able to alleviate some problems, and so will experienced engineers and other staff. It’s important for everyone involved in the process (from top executives down) to buy into this idea because it can’t succeed without their support.

    When everyone understands the DevSecOps approach, they’ll not only understand why it’s important but also see how to properly implement this new way of doing things.

    Implementing the Proper Tools and Processes

    While it is often easier said than done, getting buy-in and detecting security issues early in the pipeline sounds great. But achieving DevSecOps integration does not happen overnight, so you should start by looking at what you can do today to move this process forward more quickly.

    But there isn’t just one right way of achieving DevSecOps—every organization has its own unique methods for building software that needs to be considered when implementing these concepts across different organizations with varying cultures and workplaces.

    Key Points

    By introducing a new security component to the pipeline, DevSecOps is expanding our fundamental development and operations components with the DevOps best practices. DevSecOps is the future of software development. It ensures that security risks are monitored throughout all stages, not just after launch time when it’s too late to fix vulnerabilities and bugs before someone else exploits them for their own gain!

  • CNCF Survey Predicts Growing Wasm Momentum

    CNCF Survey Predicts Growing Wasm Momentum

    A survey of 93 developers conducted by the Cloud Native Computing Foundation (CNCF) found more than a quarter (28%) are using WebAssembly (Wasm) in their cloud-native development projects, with another 36% planning to do so during the next six to 12 months.

    The survey, published this week, is effectively a straw poll that found 42% of respondents are developing or plan to develop server-side applications using Wasm, while 48% are combining or planning to combine server-side workloads with client-side applications.

    Originally developed five years ago by the World Wide Web Consortium (W3C) to create a common format for browsers executing JavaScript code, Wasm is now starting to be used to rapidly build lighter-weight applications that can be deployed on any server platform. The promise of being able to write an application once and deploy it anywhere is finally being realized some 25 years after the initial introduction of the Java programming language.

    The survey found 70% of respondents viewed Wasm runtimes as a disruptive emerging technology. The primary programming languages used in association with Wasm are Rust (59%), Go (47%) and JavaScript (46%), the survey found.

    Nearly two-thirds (63%) said they are both porting new applications using Wasm and migrating existing workloads, compared to 37% who said they were creating new applications only. The primary reason cited for migrating workloads to Wasm was application portability (78%), better security (77%) and improved runtime performance (70%).

    Security ranks high because existing approaches to building applications rely on the aggregation of software components that tend to lack distinct boundaries between them. As a result, it becomes relatively simple for malware to infect all the components of an application. Wasm code runs in a sandboxed environment that isolates execution environments in a way that eliminates the ability of malware to laterally move across an application environment.

    Target platforms for Wasm applications include serverless computing frameworks (63%), edge platforms (54%) and the web (52%), the survey found.

    The biggest barriers to adoption were tools (46%) and lack of standards (45%). Just over half (56%) of respondents said Wasm developer tools are missing features and are difficult to use. A full 81% said they would like to see vendors provide full-featured and stable Wasm tools and projects.

    Matt Butcher, CEO of Fermyon, a provider of a platform for building Wasm applications, said that while survey respondents most likely represented more advanced developers, it’s clear Wasm momentum is starting to build. Much like JavaScript before it, Wasm is about to jump from the client to server-side application environments, he noted.

    Less clear is whether Wasm represents the next era of cloud-native computing or is simply another type of software artifact that will be employed to build applications alongside other artifacts such as containers.

    Wasm isn’t necessarily going to replace existing approaches for building applications any time soon, but it does provide an alternative to serverless computing frameworks for developing lighter-weight applications that run faster. Existing serverless computing frameworks typically use containers that are much slower to start up than Wasm code running natively on a platform, which accounts for why developers are looking to embrace the format to improve overall application performance.

    One way or another, more Wasm code will soon be finding its way into DevOps pipelines. The issue is determining how to incorporate it alongside all the other code moving through those pipelines.

  • Introduction to Cloud Custodian

    Introduction to Cloud Custodian

    Organizations continue to increase their footprint in the cloud. Yet, applying governance and enforcing policies across multiple cloud environments at scale is a challenging endeavor. Organizations typically have governance policies written in English, but translating them into enforceable code has historically required some custom workarounds. In the same vein as infrastructure-as-code (IaC), DevOps engineers now seek a similar GitOps approach to implement shared policies across cloud-based workloads.

    Cloud Custodian is one such toolset that can manage and enforce cloud policies in a standardized format. Using Cloud Custodian, a Cloud Center of Excellence has the building blocks to create policies for security governance, development guardrails and cloud cost optimizations. The free, open source project, which has been steadily evolving for years, recently gained its incubation status with the Cloud Native Computing Foundation (CNCF) and to date boasts over 350 contributors on GitHub.

    I met with Kapil Thangavelu, Cloud Custodian creator and maintainer and CTO at Stacklet, to learn more about Cloud Custodian and what features we should anticipate from the project in the near future. Below, we’ll provide a comprehensive introduction to Cloud Custodian, exploring its history, how to use it and its core benefits. We’ll also analyze a few sample policies to showcase how Cloud Custodian functions in practice.

    What is Cloud Custodian?

    Cloud Custodian got its start as an internal tool developed within Capitol One toward the end of 2015, explains Thangavelu. At the time, engineers were grappling with the newfound realities of managing the cloud at scale and realized that having one-off ad-hoc policies across teams was causing friction. “It’s really hard to manage and enforce policies at scale,” says Thangavelu. So, engineers toyed with creating a more GitOps approach to policies around governance and security that could be run as an internal SaaS solution.

    Thus, Cloud Custodian was born. The YAML DSL tool allows you to easily define rules to manage cloud infrastructure. It can offer real-time enforcement and produce event-based responses. For example, you can create policies that require tags on resources or one that turns off instances during off-hours. Or, a policy could prevent a developer without the proper privileges from accidentally making an open load balancer. Cloud Custodian supports AWS, Azure, and GCP and policies are specified to a resource type, whether it’s EC2, ASG, Redshift, CosmosDB or a PubSub Topic.

    Thangavelu describes Cloud Custodian as a bunch of Lego bricks that developers can pick up to build guardrails in arbitrary ways. Instead of shipping with canned rulesets, “Cloud Custodian makes customization itself a first-class citizen,” he says. Capital One open sourced the project and contributed Cloud Custodian to CNCF in August 2020.

    Example Policies

    Cloud Custodian comes with a library of actions and filters. Let’s take a look at a few example policies across the three main cloud providers. But first, to install Cloud Custodian from PyPI, you can initiate the following command:

    $ python3 -m venv custodian
    $ source custodian/bin/activate
    (custodian) $ pip install c7n
    

    Let’s consider an example policy that would improve security hygiene on AWS. Below is one such policy that creates a CloudWatch Event to be triggered anytime a user logs in from an invalid IP address. This could be used to ping a security administrator to investigate. With this rule, it’s also possible to turn on an auto-remediation function.

    policies:
    
      - name: invalid-ip-address-login-detected
        resource: account
        description: |
          Notifies on invalid external IP console logins
        mode:
           type: cloudtrail
           events:
              - ConsoleLogin
        filters:
          - not:
              - type: event
                key: 'detail.sourceIPAddress'
                value: |
                   '^((158\.103\.|142\.179\.|187\.39\.)([01]?[0-9]?[0-9]|2[0-4][0-9]|25[0-5])
                   \.([01]?[0-9]?[0-9]|2[0-4][0-9]|25[0-5]))|(12\.([01]?[0-9]?[0-9]|2[0-4][0-9]|25[0-5])
                   \.([01]?[0-9]?[0-9]|2[0-4][0-9]|25[0-5])\.([01]?[0-9]?[0-9]|2[0-4][0-9]|25[0-5]))$'
                op: regex
        actions:
          - type: notify
            template: default.html
            priority_header: 1
            subject: "Login From Invalid IP Detected - [custodian {{ account }} - {{ region }}]"
            violation_desc: "A User Has Logged In Externally From A Invalid IP Address Outside The Company's Range:"
            action_desc: |
                "Please investigate and revoke the invalid session along
                with any other restrictive actions if appropriate"
            to:
              - CloudAdmins@Company.com
              - SecurityTeam@Company.com
            transport:
              type: sqs
              queue: https://sqs.us-east-1.amazonaws.com/12345678900/cloud-custodian-mailer
              region: us-east-1
    

    Next, let’s consider a policy that would add some degree of governance for developers working on Azure. One simple yet popular policy involves tagging the ownership details to individual resources. This policy, therefore, will tag all resource groups with the creator’s email address:

    policies:
      - name: azure-auto-tag-creator-resource-groups
        resource: azure.resourcegroup
        description: |
          Tag all existing resource groups with the 'CreatorEmail' tag; looking up to 10 days prior.
        actions:
         - type: auto-tag-user
           tag: CreatorEmail
           days: 10
    

    Finally, let’s consider a policy that could aid in cost optimization on GCP. Below is a sample policy to enforce minimal CPU utilization for autoscalers on GCP. Implementing such a policy could help avoid underutilization to combat rising cloud costs.

    vars:
      min-utilization-target: &min-utilization-target 0.8
    policies:
      - name: gcp-autoscalers-enforced
        resource: gcp.autoscaler
        mode:
          type: gcp-audit
          methods:
            - v1.compute.autoscalers.insert
        filters:
          - type: value
            key: autoscalingPolicy.cpuUtilization.utilizationTarget
            op: less-than
            value: *min-utilization-target
        actions:
          - type: set
            cpuUtilization:
              utilizationTarget: 0.8
    

    Benefits of Cloud Custodian

    By automating a lot of the tedious policy management away, Cloud Custodian could reduce risk and accidents through more streamlined cloud governance. “It solves the natural problems when infrastructure is in everyone’s head,” says Thangavelu. By aggregating ad-hoc scripts and unifying policies across an organization, you could immediately instigate new rules without manually reminding all members of an organization, which could take years.

    For those familiar with Open Policy Agent (OPA), you may notice some overlap in the objectives, as both are engines for enacting cloud-native policies. Compared to OPA, Cloud Custodian has some developer experience perks. For one, you don’t have to use Rego, as the policies are written in YAML, which is a familiar configuration language for DevOps engineers. Cloud Custodian uses abstractions on event runtimes for each cloud provider. Furthermore, compared to OPA, you don’t need to bind the engine for a particular problem domain, says Thangavelu, as Cloud Custodian is specifically bounded to cloud governance and management.

    Cloud Custodian is also battle-tested. Many big names are using it in production, including Capital One, HBO Max, Intuit Inc, JP Morgan Chase & Co, Siemens and Zapier. Across its adoption, the tool tends to either be utilized by a Center of Cloud Excellence, security teams to conduct reporting or reveal real-time incidents or the CFO to direct cost optimizations.

    Future of Cloud Custodian

    Cloud Custodian maintainers describe the project as a consistent firehouse. Since the project tracks changes in all cloud provider resources in real-time, it’s quite a dynamic toolset. In addition to monthly updates, users can expect many additional updates on the horizon, including support for Tencent Cloud as well as additional Kubernetes support. Additional work is being done around shift-left capabilities to meet the developer where they are, says Thangavelu.

    For more information, check out the Quickstart Guide to Cloud Custodian, or view the project roadmap here.

  • DevOps Principles, Implementations and Culture

    DevOps Principles, Implementations and Culture

    When organizations begin using DevOps for application development, many only reference DevOps as part of their infrastructure and select a subset of DevOps before understanding the cultural and process changes needed for success. Instead, organizations must first determine which ‘Ops’ approach best promotes a DevOps mindset and integrate their workflows into their culture. In doing so, these organizations will better align their development and operations teams on common goals and protocols.

    There are many variations and subgenres of DevOps. However, it’s important organizations understand the ‘Ops’ choices available and choose the one that’s right for them.

    Principles Vs. Implementation Differences

    General DevOps and its subgenres can be split into two groups. The first and “purest” are the principles that these terms focus on unifying two or more industries with operations.

    The second type is those that focus on an implementation type.

    Principle Types

    DevOps: Improving Technical Processes and Culture

    DevOps is the overarching umbrella term for operations that combine Agile development concepts with IT operations and fast-tracks the development and deployment of applications in cloud environments. While there are different subgenres of DevOps, each subgenre is a different application of DevOps methodology, all geared toward making development more agile, stable and efficient. Because DevOps shortens the development life cycle by a significant margin, this streamlined approach to application development is quite popular.

    DevSecOps: Adding Security Checks

    DevSecOps bakes additional security measures into the workflow. Where DevOps typically has security checks at the end of a cycle, IT pros that leverage DevSecOps integrate security early and often in their DevOps journeys. With DevSecOps, collaboration between developers, security specialists and ops is crucial to maintaining agility and delivery speed, but the extra layers of security can slow processes down. DevSecOps leverages fast testing and robust pipelines to favor better code integrity. This method can be beneficial if an organization needs to elevate security during development.

    NoOps: Reducing In-House Teams for Operations

    NoOps is the lofty vision that DevOps is so automated it doesn’t require in-house expertise, meaning no manual intervention is required in processes. NoOps differs from DevOps because there is less focus on Agile and process management. This process assumes that developers do not need to know how to use the automated tools and uses continuous integration techniques to allow developers to focus only on application development.

    Though NoOps speeds up the deployment processes, if used by itself, NoOps lacks open communication and team management. Successful implementation of NoOps is only achievable if an organization does not have lofty technical goals or if its development team is small but strong and able to handle development and operations.

    GitOps: Using Git to Automate

    GitOps has been gaining traction in the development industry over the past two years because of its use of Git to automate continuous delivery pipelines. GitOps is a robust and manageable form of operations because it uses Git as the single source of truth for operations. In addition, due to developers’ familiarity with Git and pull requests, there is no complicated tool to learn, and changes to existing workflows are often unnecessary.

    The GitOps method can also boast that it is backed by some of the market’s most innovative cloud services, thus automating the process of building an update, testing for errors, reviewing code and pushing the update to production. Because Git is the only source of truth with GitOps, it is vital to effectively fortify the Git repository to avoid needless commits or pull requests.

    CloudOps: Implementing Cloud-Native Features

    CloudOps attempts to take more advantage of cloud-native features offered by modern service providers. Since CloudOps focuses on distributed development and deployment, there is no single point of failure, making the whole cloud environment more reliable. In addition, certain parts of the workflow in this type of operation can go stateless, making it cost-efficient as organizations only pay for the resources they actually use. Though CloudOps can be time and cost-efficient, organizations must get the configuration right for the approach to be beneficial, which could mean added complications to configuring CI/CD pipelines.

    CIOps: Requiring CI Operators for Configuration

    Continuous integration Ops (CIOps) is the latest iteration of DevOps, requiring CI operators to configure the infrastructure needed to support new code before deployment. The manual inputs necessary to ensure CI jobs’ correct configuration and deployment location bring advantages and disadvantages. Control over infrastructure allows two deployments to have different infrastructure configurations. The major disadvantages are the required manual intervention leaves room for error and the need to link an organization’s CI tool to its API can pose security risks.

    CIOps is better-suited for smaller development environments where automation is an inconvenience. While CIOps gives developers flexibility in configuring cloud infrastructures, the approach takes time and effort to run smoothly.

    ITOps: Automating IT Management Pipelines

    IT is loosely defined as the management of end user or employee systems. They are the systems that are required to run a core business. The fundamental difference is that there is little to no development. If your business doesn’t require a software engineering team and can function solely on prebuilt applications, then ITOps is for you.

    ITOps is in charge of delivering and maintaining services, applications and the underlying technologies necessary to run a business, covering jobs like system administrator and network administrator. ITOps focuses on updating laptops and office networks in addition to highly-tested commercial software and solutions for constructing the infrastructure, making updating infrastructure components more complicated. When using ITOps, an organization’s CIO will focus on managing the system via automation. 

    Choosing the Best Op for an Organization

    Choosing between the variations of DevOps is a matter of finding an approach that works best for the kind of apps the organization is developing and the cloud infrastructure they use. Though all of the above Ops strategies are based on unique ideas, they tend to focus on the technical side. Contrarily, DevOps stands out for its ability to comprehensively improve workflows and streamline technical processes while fostering a culture of innovation and collaboration in an organization.

  • Choosing the Right API Solution

    Choosing the Right API Solution

    There are more choices than ever when it comes to APIs–and that’s a good thing. But it also means you’ll need a strategic plan for choosing the right API solution.

    Two questions I’ve found can help guide that decision: 

    1. What problem are you trying to solve right now? 
    2. What functionality will you need in the next five to 10 years? 

    Once you’re clear on the answers, you can evaluate API solutions based on the following five criteria to choose the one best suited to your needs.

    Performance

    In a highly competitive global market, it’s not enough to have reliable uptime. Speed also matters–a lot. A site that loads in one second has a conversion rate three times higher than one that loads in five seconds.

    To accommodate current customer expectations for page speed, API response time should be no greater than 90 milliseconds. One way to blazing-fast API speeds is to take a client-side approach where data is delivered to an edge node of a CDN network instead of stored on a server somewhere in your data center.

    But let’s return for a minute to the first question I introduced above: What problem are you trying to solve? If you’re an online retailer, you want your site to load fast so your customers don’t click away and you want everything on it to be accurate–for example, if an item sells out while a customer is browsing, they shouldn’t be able to put it in their cart.

    Not every function requires the same level of speed. Product description content, for example, is mostly static, while inventory is not. So content delivered by your CMS’s API may not require the same speed as data from a transaction engine. 

    While high performance is important for every API you use, the actual performance benchmarks each one needs to meet vary based on function and on the end-user problem you’re solving.

    Documentation

    Thorough documentation is a must-have for any API solution you adopt. Without thorough documentation, your team will be forced to figure out how to use the API via trial and error supplemented with time-consuming support interactions. Strong documentation lets developers use APIs more effectively and, ultimately, ship products to market faster.

    But even among providers that offer thorough documentation, there are more and less helpful varieties. I’ve read plenty of documentation, for example, that is written with the assumption that the person reading it already understands how the API works. This makes it harder for newer developers to learn and understand the capabilities of an API.

    As you consider providers, look for those with documentation written for someone who is totally new to the API–which you and your team will almost certainly be. As a stylistic guide, look to documentation from Mozilla, MDM and W3 schools, which I think do an excellent job of communicating for non-experts.

    To gauge the helpfulness of a potential provider’s documentation, I recommend having your team read some of it and asking the partner team how often they update it. A best-case scenario is that they consider documentation an ongoing project that will never be complete.

    Security

    Security is pretty self-explanatory, so I won’t spend a ton of time on it here. Any API provider should offer robust security to ensure that the API cannot be used nefariously by bad actors or hacked to expose sensitive information. REST APIs almost always follow standards like 0Auth and TLS to maintain security. It’s worth looking at how the API you’re using describes its security practice.

    Flexibility and Extensibility

    Your team should have the freedom to create whatever they want with the APIs you use, meaning they shouldn’t be hampered by roadblocks in the API’s design. This freedom to create should exist in all aspects of the API: It should let your team create whatever customer experiences they can dream up, and it should let you manage back-end information in whatever way makes sense for your organization.

    In other words, an API solution should be well-defined but should not define everything you can do with it.

    Pricing Model

    The final piece to take into consideration is the API’s pricing model, by which I mean not only its actual cost but also the way the provider charges to use its service.

    For example, does it charge by order? By bandwidth? By number of records? Again, this comes back to the problem you’re trying to solve: If you have a lot of requests to make, for example, don’t choose a service that restricts requests per minute.

    And even if a pricing model works for your business today, will it work as you scale? This question brings me to my final thought.

    API Solutions Should Make it Easy to Grow and Evolve

    I started this piece with two key questions to guide the process of choosing API providers: What problem are you solving and where is your business going?

    If you’re exploring APIs as part of a replatforming project, there’s a good chance you chose a platform 10 years ago that could do everything–but that can’t keep up with evolving customer expectations or that restricts your team’s creativity today. The APIs you choose should let you meet current needs and make it easy to make changes–including swapping one API provider for another–as your needs evolve.

    In other words, when you choose the right API solutions, you should never need to do another replatform.

  • Microservices Explained: Not Your Father’s SOA

    Microservices Explained: Not Your Father’s SOA

    Microservices are frequently referred to as a variant or derivative of service-oriented architecture (SOA), if not essentially the same thing. While there are similarities and both are designed around the concept of services, that’s where the similarities end. Each was created around a different set of principles and intended to address different problems.

    Microservices architecture is built around key concepts that, when applied, not only differentiate it from other architectures but deliver advantages that accelerate software delivery and help to scale large, complex applications. To deliver on the promise of cloud-native, microservices must be loosely coupled so that, ultimately, they are independently deployable. Any microservice can be updated and deployed—just that containerized microservice—without requiring changes to other microservice(s). Loosely coupled microservices mean they are not sharing databases or maintaining state, for example.

    How big should a microservice be? Or, in other words, how much should it do? The answers range from ‘a single thing’ to ‘a small set of related code with high cohesion’. Another factor is that getting too granular can overrun a microservices-based application with large numbers of microservices, increasing complexity and making triage through observability and tracing very challenging.

    Scaling microservices-based apps is another important factor. Autonomous microservices can be replicated and scaled for performance and workload using container orchestration tools such as Kubernetes.

    Microservice communications are achieved through well-thought-out APIs created around the business or operational capabilities the microservice provides to anyone or anything engaging through the API. The API, and data provided through the API, are often referred to as ‘technology-independent.’ It’s probably better stated that microservice APIs are built using approaches like RESTful and GraphQL over networks that are not tied to any one specific operating environment, technology or programming language. JSON, YAML and XML similarly represent data that is transferable across technologies.

    The following diagram of a media application demonstrates the principles of microservices architecture. Microservices for scheduling media, digital rights management, tracking impressions and adding new ad insertions are loosely coupled, autonomous and have well-defined API interfaces. Each microservice can be changed, enhanced and deployed rapidly and independently of other microservices.
    Microservices SOA

    Service-oriented Architecture, or SOA, emerged to solve a very different problem; the overwhelming size and complexity of monolithic applications. Monolithic applications have very large codebases that often don’t do a good job of compartmentalizing functional or business logic. This makes changes to the codebase challenging even to developers who know much of the codebase well.

    Services in SOA were often used to gather a collection of similar business logic together and then share them as a service with other parts of the application, as needed. Technologies like message buses are used to make requests of a service and operate as traffic managers for requests across multiple services and other parts of the application. While reusability was a critical design consideration, SOA services were still quite large, and deploying services at high velocity was not an important design consideration. SOA services often take considerable time to code and test the service including interdependencies across other parts of the application before deploying as part of a larger software release.

    One important note: There are benefits and drawbacks to microservices and SOA architectures; neither is ‘better’ than the other. Which architecture to use when mainly depends on the purpose of the application you are building. For larger, more complex enterprise application environments that require integration with many other applications, you might choose SOA. That’s a better fit than for smaller applications that don’t require middleware elements for managing the requests within the app or between other applications. Microservices, on the other hand, are better-suited for smaller and well-partitioned web-based systems. They are becoming the standard for today’s cloud-native applications, and also are a great fit for developing a mobile or web application because they give developers more control.

  • The Whats and Hows of APIs: A New Developer’s Guide 

    The Whats and Hows of APIs: A New Developer’s Guide 

    For those who are new developers, you may not know a lot about application programming interfaces (APIs). However, it is likely that you’ve used them before as a consumer or an end user, whether purchasing online, using business communication software to reach colleagues or sharing images from a monogram logo maker on social media platforms, for example. It makes knowing about them helpful, regardless of if you plan to make and use one yourself or want to understand how the tools you use work. 

    Although coding and developer tools can be daunting for the outside world, we’ll put APIs into a language that new developers (and even non-developers) can understand. In this guide, we’ll cover what APIs are and the different types that exist. You’ll also learn how they work and how you can use them. By the end, hopefully, you’ll be more comfortable with APIs and know how they impact you and your business.

    What Is an API?

    An application programming interface is a piece of coding used by developers to communicate between different devices, systems, or applications. It allows information to be passed on—allowing procedures to be followed across multiple platforms. Depending on the API, this could be transferring data, images, videos, text, files and other content, or allowing developers to connect through the use of backend-as-a-service (BaaS) to link their apps to the cloud. APIs are often used in a business context and for personal projects or other purposes. 

    Generally, APIs are concerned with the raw information and passing this on to another program rather than the appearance or visual side of these applications. Another set of coding is used instead to build the aesthetic and usability of the interface, while the API tends to be less visible to users. Nonetheless, they form an essential part of our technology stack today, allowing systems to integrate and become more agile, making applications easier for users to navigate.

    Image sourced from developers.facebook.com

    For example, when you refresh your omnichannel communications application, a request is sent to the API to check other linked programs for new messages on your social platforms or calls to your 800 numbers for business. These programs inform the API of new communications and send the appropriate data across. The API returns this data to the application you are using, enabling you to read the messages from here or notify you of no new calls.

    Types of API

    1. Private

    A private API is used internally within a team, created by developers who work for the company or organization. It can help make highly personalized APIs for the needs of your business and adapt the API as and when needed. Private APIs can’t be used or worked on by external developers, as it is purely for your in-house team. These APIs tend to focus on aiding your company’s productivity and project management across multiple applications.

    2. Partner

    Alternatively, partner APIs are used externally with companies that your business collaborates with. Often, this transfers information between businesses, such as within an affiliate scheme or for external postage of online orders. However, APIs can also share and update files such as your release of liability contract or emergency procedure templates. Developers from both companies can improve the API system to effectively share data between partners. 

    3. Open/Public

    Because they are not restricted to usage by specific businesses, open APIs (also known as public APIs) can be used by anyone. You may need a subscription to give you access to the API depending on how frequently you use it, although there are other APIs that are completely free to use. Once you’ve sorted out your access, any software developer can use the API to make changes and transfer information more easily between various software for several purposes. 

    Image sourced from mailchimp.com

    4. Composite

    By taking the responses of multiple programs and streamlining these onto one platform, composite APIs are essential for business and personal use. Or, vice versa, a composite API allows you to make requests of multiple platforms simultaneously. It is helpful for omnichannel management, tracking customer interactions, online faxing and purchases across various platforms. These tend to require little input from developers and are set up for anyone to use.  

    How to Use an API

    • Choose Your API

    As you may have gathered, APIs can be used for several purposes. When choosing your API, look for something relevant to the business that helps you complete the tasks you already do. The software and tools you use may have recommended APIs to try, or there may be APIs that similar businesses use. Do your research to ensure your API performs the right functions and works with your rephrasing tool, calendar software or other add-ons you use. 

    If you’re getting used to APIs, you may want to start with a free option. It means that if it doesn’t work as you hoped, you’ve not lost anything by trying. Many APIs have a free trial period or a free subscription tier with the basic functions, so you can get a feel for the API and decide if it works for your business. When you’re confident in your choice, you can invest in the other subscription tiers that tend to allow for additional users and added functionality.

    • Find Your API Key

    An API key identifies you as the user, authenticating your actions using the API and tracking your usage. It is crucial to have, particularly with subscription-based APIs, as without it you may not be able to perform certain actions, or your users of the API may be limited. Even if your API of choice is free to use and publicly accessible, there still may be an API key to keep your information and preferences stored and linked between your platforms. 

    Image sourced from developers.google.com

    Your API key may be issued as you sign up for the service, so remember this and ensure those in your business who need it can access it. Otherwise, you may need to request or pay for an API key, although this should be clear when you sign up for your chosen API. To increase your API security and avoid your preferences being altered, keep your API key private and limit who has access to it. If your account is hacked, you may need to request a new API key. 

    • Read the Instructions

    It can be easy to overlook the documentation and instructions an API comes with, assuming you’ll get the gist through practicing and by actually using it. Although this may be true to an extent, you could be missing out on useful functions or not using the API as effectively as possible. Likewise, you could get yourself into a mess, not properly setting resource requests and limits, leading to more issues for your business when using the API in the future.

    Set aside time to read the instructions and documents for your API or divide the workload between your management team. You may discover it comes with setup instructions or useful tutorials you can use with the rest of your teams. As you introduce more people to the API, the instructions ensure everyone uses it in the same way. It prevents confusion for your teams and enables them to build their confidence with the API more quickly. 

    • Set Up Your Requests

    Depending on the API, there may be suggested requests for you to set up or you may have to create and code your own. Following the instructions from your API can help with this, allowing your in-house developers to reach their full potential within the API and informing you of the coding structure to use. Nonetheless, there are plenty of tutorials online covering how to set up your API requests without relying on coding expertise.   

    Image sourced from developer.paypal.com

    Just because a request is possible doesn’t mean it’s always relevant to your business. Consider why you’re using an API and what you want it to do once you’ve set up your requests. If you’re new to using APIs, start with a few basic requests and build on these as you gain confidence and experience, using CI/CD best practices to ensure it runs well. It allows you to become familiar with using each request and ensure it works in the way that you intended it.

    Are You Ready to Use APIs?

    As technology progresses and software is developed, having APIs in place is becoming more helpful. Being able to communicate between applications can streamline your virtual receptionist service, incoming video calls and outgoing messages, for example, as well as accelerate cloud-native application development. Having one place to make your requests and display information from multiple platforms gives you more control in managing your tasks.

    If you’re still new to APIs, you don’t have to start using them for everything immediately. Take your time learning to use them for specific functions, getting familiar with them, and enabling your teams to adjust to using them. From this foundation, add new APIs or requests as they’re needed, personalizing them to your business and the tasks you complete. Over time, you’ll gain confidence in using APIs and build a strong network of requests to run your business effectively.