devopsy.kinsta.cloud

Category: DevOps Practice

  • CPRA for Developers: Road to CCPA 2.0 Compliance

    CPRA for Developers: Road to CCPA 2.0 Compliance

    California voters passed the California Privacy Rights Act (CPRA) in November 2020. It replaces the 2018 California Consumer Privacy Act (CCPA) and is often described as a cousin to the EU’s GDPR law. The law applies to all for-profit organizations within the state. It also requires compliance from anyone doing business in California or collecting data from the state’s residents.

    Since the CPRA focuses on data protection, it applies to developers who deal with customer data or develop websites handling customer data. Building good-looking apps and websites that deliver a great user experience is no longer enough; data privacy must also be part of the equation.

    With that in mind, let’s look at everything developers need to know about achieving CPRA compliance.

    CPRA Compliance Checklist for Developers

    Here are the main compliance points organizations need to adhere to:

    • Have a process that allows customers to exercise their right to correct personal information.
    • Have a process that enables easy opt-out for customers regarding advertising and data sharing.

    Before or during consumer data collection, organizations must disclose:

    • The categories of data gathered and whether the data will be sold or shared.
    • How long the organization plans to keep the data gathered.
    • How the data will be used for targeted advertising and how users can opt out.

    Organizations mustn’t keep personal or sensitive information longer than necessary to achieve the disclosed purposes.

    These are just some of the compliance checkpoints. For more details on the CPRA responsibilities for businesses, visit the Californians for Consumer Privacy website.

    The Risks of Non-Compliance with CPRA

    CPRA compliance is costly, but non-compliance can lead to even heftier penalties. The California Privacy Protection Agency (CCPA) is a new agency responsible for enforcing the law. It will give out fines of $2,500 for inadvertent non-compliance and $7,500 for intentional non-compliance. The penalties will increase significantly for abusing data belonging to persons under the age of 16.

    Apart from financial losses, non-compliance is risky from a reputational and consumer trust standpoint. In the event of a data breach, these factors may even be more damaging than the fines.

    The Importance of the CPRA for Developers

    App and web developers have long prioritized deadlines, user experience and other factors over security. This is because they tend to focus on elements that improve engagement and keep users coming back. The CRPA puts a significant limit on data gathering, not only for customers but also for employees. That significantly changes a developer’s approach. 

    The new law requires developers to take a privacy-first approach throughout the software development life cycle (SDLC). This requires a complete mental shift for most developers, and it will take time to attain. Security is no longer a client request; it’s a mandatory requirement.

    Security, privacy and transparency must become focal points for software developers. These elements should influence every decision they make throughout the SDLC.

    For existing systems, the CPRA means operational and system updates relating to data gathering and storage, security and governance. Developers will have to adjust systems to gather the bare minimum of data, as the law encourages. 

    Data anonymity capabilities are also required. Data that isn’t crucial for app or website functions should be anonymized to reduce the risk of a data breach.

    Developers should focus on creating apps that:

    • Track data throughout its life cycle.
    • Anonymize data when possible.
    • Get rid of old, unused data.
    • Allow for easy data sharing across the organization.
    • Incorporate CPRA compliance best practices for developers.
    • Perform a gap analysis.

    Gap analysis is the process of determining whether the current system satisfies the desired or mandated requirements. A gap assessment will give developers a clear picture of any existing compliance gaps. For non-existing systems, a gap analysis will help determine how the new system can fulfill requirements. 

    A gap assessment will also determine whether the CPRA applies to your organization. For instance, if your organization doesn’t collect consumer data, the CPRA may not apply to you.

    Set Clear Timelines for Reaching CPRA Compliance

    Reaching CPRA compliance can be a long process, as developers have many requirements to meet. Doing everything at once with no clear plan is a recipe for failure. Instead, create a compliance roadmap with target dates for meeting each requirement. Follow the roadmap rigorously, but also leverage your company’s existing efforts to reach compliance. 

    Existing efforts may include significant changes like raising the cybersecurity budget. They may also include smaller implementations such as a VPN for remote workers or stricter access control.

    Identify and Update Privacy Notices by January 1, 2023

    Privacy notices to customers are an integral part of the CRPA. The law requires that privacy notices are updated at least once every twelve months. Businesses can no longer set-and-forget their notices.

    App developers must identify the existing notices in the system and develop new ones that are in line with the policy changes brought by the CRPA. Notices for employees, job applicants and others must not be forgotten during this process.

    Provide Additional Security to Sensitive Data

    Not all data has the same value. Consumer data and confidential company documents are examples of sensitive data. Developers must work with other IT employees to separate the sensitive data and provide additional security to ensure its confidentiality.

    Some additional security solutions are data encryption, restricted access, two-factor authentication and strong passwords.

    Quality Assurance and Testing

    When it comes to cyberthreats, you can never be 100% secure, even if you follow all the best practices and regulations. That’s why having additional help from automated logging and testing software is necessary to maximize security.

    Develop a process for continuous self-monitoring and quality assurance. Automated penetration testing is a great way to find holes in your system. It’s also very affordable.

    Conclusion

    The CPRA was instituted in November 2020. The law requires all for-profit businesses in California to comply with data protection standards. Other states throughout the US are also implementing the same or similar legislation to protect consumer data.

    Now is the time to reevaluate your data security practices. Developers must take a privacy-first mindset and start thinking about security implications throughout the entire software development life cycle.

  • Simplifying DevOps and Infrastructure-as-Code (IaC)

    Simplifying DevOps and Infrastructure-as-Code (IaC)

    Let’s face it, tools for infrastructure-as-code (IaC) configuration management may be vital, but they’re difficult to use. But what if it’s not so much the tools themselves as it is our understanding of them? What if we looked at provisioning and configuration systems differently and did so using vocabulary that was less confusing and more common? 

    Simply put, we’d gain a greater understanding of these tools—and, therefore, they would be simpler to use—if the terminology was simpler. Reframing these complex tools would not only allow us to make greater sense of this realm, but we’d be more effective in addressing the DevOps and software issues we all experience. 

    Coming to Terms With Infrastructure-as-Code (IaC)

    Software engineering encompasses all the actions developers take to create software and technology products. This typically entails writing code, testing it and making sure it satisfies the original requirements for users. Operations is the practice of ensuring those software systems are ready to go and will be able to serve users on a day-to-day basis. Operations engineers aren’t as focused on writing new code as developers are; instead, they are focused on how existing software (mis-)behaves in existing systems.

    Now let’s cover some other key phrases we use and hear all the time in DevOps.

    Forward engineering means creating a system based on requirements that exist for that system. It entails first building a model, implementing it in software, then configuring and deploying it into operation. While the goal is for the new system to closely match the original requirements, the modeling process can be ambiguous, causing a system to perform in unanticipated ways. Furthermore, over time, the system itself may differ from the original model due to interaction with its environment. Consider that some parts can change the configuration of other parts; procedures performed by engineers shift the system state or configuration, too.

    Unfortunately, forward engineering offers no feedback with regard to the eventual success or failure of the product during the model creation process. These models and the subsequent software are created first and then deployed to test for success. Such a system can only provide feedback later based on its behavior, which can then be used to improve and build new models.

    Reverse engineering calls for understanding existing systems, usually by first creating a partial mental model by inspecting such a system and then using it to plan any modifications. Often, changes are made directly to the system without always fully understanding how it works. Feedback for changes is fast because putting the system through its paces and inspecting the results is relatively simple, easy and cheap to do. A good example of reverse engineering is software security professionals who scan network services or memory to identify anomalies and exploitable gaps and then try to manipulate their content for instant feedback.

    Then there’s infrastructure-as-code (IaC). This modeling technique began with declarations about how server operating system content should appear and was later adapted for the purpose of laying out and configuring cloud resources using API calls. While many vendors and products have come and gone, most tools have been declarative—with good reason.

    Anything to Declare?

    With declarative tools, you specify how things should appear without explicitly showing how you’ll get there. Why? It’s the easiest way to create models, templates and maps because engineers only have to declare how things should look, not get them into the correct form or place to actually do so. The emphasis is on evolving from iterative improvements that take place over time.

    Another feature of declarative IaC tools is idempotence, which means repeating an action multiple times with consistent results; the system only changes when it detects deviations from the model. This can be compared with the imperative approach, which requires developers to first find all the differences between a model and system and then write each step that’s necessary for the system to match the model.

    Declarative IaC tools eliminate the need for developers to write an implementation for these changes manually. These tools save thousands of labor hours by publishing templates that can be used repeatedly to build identical systems with less time and effort. The problem is that declarative tools rely on a forward engineering mindset—meaning there’s no way to get feedback from a live system—leading to many complaints from users.

    The Rift With Drift

    Configuration drift is what happens when a declarative forward engineering model no longer matches the state of a system. It can happen when a developer changes a model’s code without updating the systems built using that model. It could also be the result of an engineer that does exploratory ad-hoc operations and changes a system but fails to go back into the template and update its code. 

    Is it realistic to ban operators from ad-hoc exploration? Actually, some companies have policies forbidding any operator or developer to touch a live production environment. Ironically, when a production system breaks, it’s the first rule overridden: Ad-hoc exploration is welcomed by anyone able to get to the bottom of the issue. 

    Engineers who adopt IaC usually don’t like the work that comes with remodeling an existing system. Still, because of high demand fueled by user frustration, there’s a tool for every configuration management language—they just fail to live up to engineer expectations. The best-case scenario is that an engineer can use a reverse-engineered template to copy and paste segments into one but will need to manually write it elsewhere.

    Bridging Gaps

    What is needed is a new type of IaC tool. These must integrate reverse engineering at their core and allow feedback from living systems to update the very models that created them. In doing so, operators can adopt an ad-hoc change or simply revert back to the system as it was declared in the original model.

    Yet, today’s standard tools fall short of this goal, and aren’t much good for helping engineers solve system operations issues. True, there are those capable of auditing templates and singling out misconfigurations and mistakes. Still, while helpful, the greater value would be the ability to review a living system, not just the templates that engineers initially used to create it. 

    We have monitoring and observability tools and IaC tools, but in between them is a wide chasm. For effective DevOps, what’s required is a new breed of IaC tool, one that integrates reverse engineering practices that supports operations engineers and enables companies to bridge any development gaps.  

  • Empathy for the API Developer

    Empathy for the API Developer

    Security teams have always been perceived as an impediment to delivery by software teams who feel that security imposes arbitrary and unreasonable policies and use poorly-integrated tools that are beset with high false-positive rates. With the advent of DevOps, security has been seen as an increasing obstacle to rapid deployment cycles. 

    Security teams believe developers do not care about security and will try their best to avoid security controls or policies. Security teams are often frustrated by developer pushback when remediating vulnerabilities, particularly when they fail to appreciate the cascading effects of making code and dependency changes late in the life cycle.

    From my perspective, with a foot in both camps and a decade of experience helping developers produce more secure code, I believe the key to addressing the disconnect between teams is to develop greater empathy for the developer.

    Why Insecure Software Exists

    Software development is a complex endeavor with many (often conflicting) requirements. Blaming software developers for insecure software is an overly simplistic and counterproductive approach, best summarized by a former colleague:

    “I can tell you that virtually all developers I talk to care about application security. They care about the quality of the products they create, the integrity of the data their software handles, and the reputation of the organizations they work for.” —  Jim Jastrzebski, Veracode

    From my experience, developers don’t deliberately set out to create insecure software; instead, there are a number of reasons it happens.

    Devs are overly optimistic: Developers are creative folks and natural problem-solvers. They can often visualize the perfect end result of their labor without necessarily considering the technical and security debt they may encounter along the way. Remember—things do go wrong.

    Overconfidence: I often refer to this as the Dunning-Kruger effect for developers. Faced with complex platforms, languages and frameworks, developers tend to overestimate their ability to comprehend and master them. Complex systems take time to fully understand and being overly confident can lead to unintended consequences and compromise security. Beware of insecure defaults and configurations.

    Bad things only happen to other developers: Think of this as a kind of schadenfreude for developers. During code reviews with developers, I have been surprised to hear comments like, “ … But that won’t happen to my code!” Despite vast evidence to the contrary, some developers think that bad things only happen to others and to others’ code. Assume a defensive mindset; expect the worst and hope for the best.

    Taking shortcuts: Developers are goal-oriented and want to produce working code on schedule. And schedules are tight. The inevitable consequence is that sometimes shortcuts are taken (missing error handler or input validation, for instance) leading to security vulnerabilities. Track and tackle your technical debt.

    Technical debt or legacy code: Speaking of technical debt, it’s probably one of the biggest issues facing developers: The burndown of technical debt and maintenance of legacy code. Software systems are complex and minor changes may wreak havoc, especially in microservices-based applications and architectures. Remember that developers may have legitimate concerns about making changes that are not absolutely necessary.

    This stuff is difficult: The biggest thing to remember to develop the empathy muscle is to appreciate the difficulty of the software development process. Developers are facing multiple conflicting requirements against tight deadlines and are required to master a variety of technologies. Mistakes will happen; the important thing is to learn lessons and improve the process.

    The last point is perfectly captured by this quote:

    “90% of security problems are just complexity problems. It’s technical complexity, it’s network complexity, it’s organizational complexity.” — Adrian Ludwig, Atlassian

    Care and Feeding of Developers

    Once security engineers have a better understanding of the challenges facing developers, they can focus on their messaging when communicating about security issues. Remember that the developer is unlikely to be personally responsible for every issue and should not be blamed or scolded. This will produce defensive behavior and is unlikely to achieve a positive outcome. Focus on the shared security challenge at hand and how a joint approach can solve the problem.

    A frequently-cited problem for developers is the relatively high false-positive rates security tools produce, particularly with code scanning tools. Security engineers should make sure that the tools are suited to your technical stacks and fine-tune the tool as much as possible. Ideally, the security team should triage the initial report and remove obvious false positives. 

    Additionally, certain legacy security tools do not integrate well into CI/CD pipelines and cannot be easily automated within the build process. If a developer has to use a manual or asynchronous process to perform a security scan, they will become frustrated and possibly ignore this requirement. Make sure security tools work well with the developer’s environments and workflows.

    Do not assume every developer has a deep understanding of the security issues identified—take the time to explain them thoroughly and/or document them internally. A great approach is to conduct shared learning sessions where developers and security teams work through a vulnerable application together to learn the concepts.

    Another frustration developers cite is a lack of clear understanding of the security requirements or requirements that are constantly changing. Ensure security policies are clearly stated and available to developers, and explain the reasons why the policies exist in the first place—for example, to meet PCI DSS accreditation. 

    Finally, make sure the combined teams are given time not just to learn but to absorb lessons and that success is recognized and rewarded. 

    “The sky is always falling—we never celebrate success.” – Chris Romeo, Security Journey

    Empowering Developers

    So, what does developer empathy look like for API developers?

    First, API developers and security engineers can both benefit from the positive security model that uses the OpenAPI definition as the contract for API behavior. Anything not specified by that contract is considered invalid—in other words, the contract specifies the allow list. This is in stark contrast to the traditional approach for web security which is reliant on blocking according to a block list—obviously, not all bad input can be known leading to false negatives with this approach. Overly aggressive blocking may block valid input and typically result in false positives; familiar to users operating web application firewalls (WAFs).

    The OpenAPI definition offers another advantage: The applicable policies can be expressed as code in the definition as opposed to via some verbose, abstract policy documentation. The benefit of policy-as-code is to uniformly enforce policy across the life cycle starting when the developer first starts coding the API. 

    Finally, developers need security tools that work in their preferred environments (for example, first-class IDE and CI/CD plugins) and provide value in the form of remediation guidance or actionable findings that help developers reduce technical debt. 

    Security and development don’t have to be at odds; help developers to help themselves by building empathy and working with them rather than against them.

  • Future of DevOps: Trends to Watch

    Future of DevOps: Trends to Watch

    Technology is transforming every aspect of industry, and digitalization and automation have flourished in the past few years. DevOps has established itself as an indispensable software development methodology for successful digital transformation.

    According to a recent market survey, the DevOps market will exceed $20 billion by 2026, growing at a CAGR of 24.7% from 2019 to 2026. DevOps has facilitated rapid and dependable software development, delivery, improved quality and higher customer satisfaction.

    DevOps, too, is continuously evolving to meet the changing demands of the corporate environment. Let’s look at the future of DevOps and the trends you should expect to see.

    Serverless Computing

    DevOps teams will see a rise in the adoption of serverless architecture, mainly because of the massive capital and operational costs of server infrastructure. Numerous DevOps groups are working with modular components offering an overview of the pipeline. Moving these modular components to serverless architecture will empower groups to mitigate a few pieces of errands identified with pipeline architecture and focus on deployment and development of the product.

    Serverless computing streamlines overall DevOps operations, including the complete software life cycle, from development to deployment, testing and maintenance in one place. Also, serverless computing reduces the developer’s workload by taking care of server maintenance, such as cloud monitoring and system updates. Other prime USPs include flexibility, reliability, speed and cost-efficiency.

    Microservices Architecture

    Microservices architecture has been gaining popularity as an alternative to traditional monolithic architecture in the IT industry, and this trend is expected to continue. Microservices architecture is a pattern of organizing computer systems into segments that can scale on demand. A microservices architecture pattern is the next evolution of service-oriented architecture (SOA) that structures an application as a collection of coupled services. If one piece of the application fails, microservices make it easy to identify and contain the issue without disrupting the whole application.

    DevOps and microservices architecture enable decentralized teams to innovate rapidly, maintain control over their tech stack and standards, track performance metrics, supervise the development and release cycles and reduce overall consumption time by achieving shorter software release intervals.

    Low-Code/No-Code Applications and Platforms

    To keep up with the demand for new applications and features, many organizations have turned to low-code/no-code development to deploy applications swiftly. Adopting a low-code approach with DevOps is a game-changer. It enables developers to experience higher agility and gives the organization a competitive advantage in the fast-paced, demanding software market.

    Low-code applications allow non-technical ‘citizen developers’ to contribute to software development with the help of visual interface integration. In addition to end users, developers and DevOps engineers are set to play a crucial role in this trend, using low-code platforms and tools that offer drag-and-drop elements and other extensions. These low-code application development tools support every aspect of software development, including app ideation, analysis, software design, code development, quality assurance, deployment and documentation.

    DevSecOps

    Many large enterprises are integrating and automating security into DevOps processes. The transformation from DevOps to DevSecOps is expected to gain even more momentum in the future. According to a forecast, the DevOps market size is expected to reach $5.9 billion by 2023 at a CAGR of 31.2% during the forecast period.

    More companies will embed security controls earlier in the software development cycle to ensure application security. Tools and frameworks will simply integrate with CI/CD pipelines and testing tools will see a rise in 2022. DevSecOps empowers developers to consistently track, monitor and remediate security defects during the various stages of development while improving the speed of delivery and quality.

    Kubernetes and GitOps

    Kubernetes is an open source container orchestration platform emerging as an infrastructure trend on the operations side. Kubernetes will expand further into the software development space in 2022 and beyond. It will ease the task of starting a local Kubernetes cluster on a developer’s machine. Kubernetes security is expected to take center stage as well, through infrastructure-as-code (IaC) scanning.

    GitOps is a Kubernetes-related infrastructure trend that allows developers and IT operators to use Git to manage clusters and deliver efficient, secure applications without misconfigurations. It automates releasing incremental updates, and consistent delivery will enable developers to build, test and deploy software rapidly.

    AI and ML

    Artificial intelligence and machine learning have already transformed the way businesses operate nowadays. In 2022, it is anticipated that AI/ML will further streamline and accelerate every phase of the SDLC. It enables DevOps teams to enhance their performance through intelligent automation, improving feedback and managing alerts.

    Advanced analytical methods like predictive analytics are becoming increasingly important as the market evolves. AI can enable proactive investigation of the DevOps pipeline to identify problems and offer possible solutions. AI and ML integration also enable deep learning, data mining, statistics and modeling, becoming an integral part of your business’ workflow.

    Conclusion

    DevOps continues to evolve and pivot to drive significant IT transformations that directly bolster business goals and objectives. It will assist in enhancing the company’s ability to design, produce, launch and maintain high-quality software solutions. The above trends will help organizations move rapidly beyond automation, focusing on continuously improving outcomes. These trends are catalysts for creating a regular and dependable release pipeline and building better communication between development, IT and business teams.

  • Dynatrace Embeds DX Monitoring in Observability Platform

    Dynatrace Embeds DX Monitoring in Observability Platform

    Dynatrace has enhanced its analytics capabilities to enable digital experience monitoring, including session replays, based on the logs, metrics and traces it collects via its observability platform.

    Steve Tack, senior vice president of product management at Dynatrace, said this update makes it simpler for DevOps teams to track user journeys across the multiple components and microservices that make up a modern application environment. Rather than trying to, for example, determine the impact of an issue by parsing through log data without any context, Tack said Dynatrace’s analytics capabilities will automatically fire up a session replay to enable DevOps teams to view the exact nature of the application issue being experienced.

    Previously, DevOps teams would have needed to deploy the Dynatrace session replay offering separately. The overall goal is to remove much of the friction DevOps teams encounter when trying to fix an application issue by providing context in a way that is more accessible, he added.

    In general, as observability continues to advance, it’s now become much less about the amount and types of data that can be collected, said Tack. The data is table stakes for achieving observability, he added. The focus now needs to be on the richness of the actionable analytics being surfaced once that data is collected, he said.

    In fact, as more organizations start to build and deploy applications based on highly interdependent microservices, the richness of the analytics is now more critical than ever, noted Tack. While microservices-based applications are more resilient than monolithic applications, troubleshooting them is more challenging whenever application performance degrades, he added.

    It’s not clear at what rate DevOps teams are embracing observability. The concept has always been a core DevOps principle, but, for the most part, more mature DevOps teams have only been able to achieve what amounts to continuous monitoring of pre-defined metrics. Observability platforms promise to surface anomalies indicative of IT issues before they escalate; DevOps teams can then launch queries to better determine their root cause and potential severity.

    Dynatrace is making a case for an observability platform, accessed as a cloud service, that employs the company’s Davis AI engine and machine learning algorithms to surface the root cause of application issues.

    As observability platforms continue to evolve, the days when IT teams determined the root cause of an IT issue by process of elimination are finally coming to an end. DevOps teams will be gaining more visibility into their application environments in ways that surface more meaningful actionable intelligence. That intelligence is especially critical for externally-facing applications driving digital business transformation initiatives.

    There is, of course, no shortage of options when it comes to observability platforms. In some cases, IT organizations are opting to rely on cloud services while others are extending DevOps platforms they built themselves and continue to maintain. Regardless of the approach, the issue will no longer be how to collect data as much as what to do with it once it’s collected.

  • 3 Ways API Analytics Can Help Application Owners

    3 Ways API Analytics Can Help Application Owners

    Software development is fraught with risk that can run projects aground like a ship against a rocky beach. Whether it is misunderstood requirements, rapidly evolving marketplaces or old-fashioned bugs and schedule slippage, all can derail otherwise well-resourced and managed software development projects.

    Even when software development teams eventually ship their product and collect payment from their customers, there is no guarantee that the product will be used. A 2022 study found that among those paying for SaaS software, 29% of their software is either underutilized or wasted. This study also noted that only 33% of IT professionals track SaaS software usage. The takeaway here is stark: As an industry, most of our labor is not going to productive ends. 

    This is not a new trend in software; an entire ecosystem of tooling exists to help software development companies collect user engagement metrics to drive development decisions. In fact, out of the top 10 million websites in the world, nearly two-thirds have an analytics solution in place. By contrast, however, API applications lag far behind in adoption of this practice. It is difficult to get market penetration data for API analytics, but a report from API management vendor Apigee revealed that even among its customers (that is, those who were already paying for a service that included API analytics), only 26% made use of the analytics functionality. This gives credence to the belief that most features go unused.

    API application owners who are not integrating real-world analytics usage data into their decision-making process are leaving time and money on the table. Despite not having immediate access to user behavior, API analytics can be quite valuable in helping API application owners make better decisions about application development and maintenance. 

    API application owners should look to the following three ways to make better product development decisions.

    Prioritize Feature Development in the Most Actively Used Areas of the Application

    One of the best lessons I learned from working at a bike shop is “stock what sells.” In other words, there’s no point in filling your shelves with products that your customers aren’t buying. The same goes for stocking your API product catalog with endpoints that your customer isn’t using. 

    API analytics allow API product owners to measure the use of features and evaluate their effectiveness. Product owners can use this valuable information to prioritize extending popular features or developing new features, with a clear yardstick for measuring usage. Further, this data enables development teams to “maximize the amount of work not done,” by shipping preliminary feature sets and evaluating usage metrics before a full implementation. Likewise, if the real-world feedback is not encouraging, it allows you to pivot to another opportunity within the product.

    Deprecate or Sunset Unused Application Features

    Unused code and features are a liability to an API product organization and can carry bugs, create performance problems and introduce security risks. I’ve seen situations as a software developer where these older features were written by developers who have long since left the organization. These unused sections often harbor serious security flaws and vulnerabilities (such as software dependencies that go unpatched for years) and dubious authorization logic that no one at the company fully understands.

    Unused, extraneous features also contribute to a bloated product that can distract end users from the core value of your API. It can be difficult to advocate for the removal of “working product features,” but API analytics data can help make a strong case that certain features are nothing but an expensive distraction. Removing old, unused parts of your application also frees developers to focus on the parts that users love. 

    Prioritize Performance Improvements

    Performance is often a fine balance between implementation time/costs and product requirements.

    It’s a well-understood maxim in software development that premature optimization is the root of all evil. Yet in the absence of real-world performance data, determining what represents premature optimization versus totally-necessary-and-prudent optimization often comes down to a subjective argument among engineers or worse, a risk-averse software development culture that contributes to slow time-to-market.

    Having an objective measure of performance impact, like the number of instances in which clients are impacted by suboptimal performance, can help application owners decide how to allocate their budget to improving performance versus developing new features or fixing bugs. Moreover, having this feedback loop in place can empower development teams to take a nimble “go fast, release often and evaluate” approach to development. Rather than optimizing performance up front out of fear that invisible problems will lead to bad performance and dissatisfied users, performance can be tracked and reviewed over time. 

    Selecting API Analytics Solutions

    API product owners operating a single application have a variety of choices when it comes to deciding which analytics tool to use for their product. Many specialized solutions exist for individual API development languages and frameworks. However, security teams have a unique problem: They need all the data, across all applications and across all the tech platforms within the organization. In that case, I recommend an API protection vendor with strong API analytics capabilities that not only protects against threats but also gives you the visibility you need to prioritize efforts. 

    API analytics should be a tool that powers innovation. Don’t leave it unused in your toolbox. 

  • How DevOps Teams Can Defend Against API Attacks

    How DevOps Teams Can Defend Against API Attacks

    Remember when ransomware was the main security threat that DevOps teams needed to worry about?

    Those days are over. Ransomware attacks are certainly still happening, but API security breaches—which increased by a whopping 600% in 2021—are now poised to become the top attack vector for threat actors, according to Gartner.

    That’s the bad news. The good news is that, in many respects, the security practices that DevOps teams already have in place to defend against ransomware can be repurposed to deliver API security, too—with a few tweaks.

    Keep reading for a look at the state of API security today, along with tips on expanding upon DevOps’ existing ransomware defense techniques to protect your APIs.

    The Preponderance of APIs

    Although it may seem surprising that APIs have suddenly become an attacker’s best friend, it’s actually not when you step back and think about how deeply dependent we’ve become on APIs over the past half-decade.

    APIs have been around for a long time. But until relatively recently, APIs were used mostly for specific types of applications, B2B or infrastructure integrations. It wasn’t until the pivot toward microservices and distributed architectures that internal (or east-west) APIs became the glue that holds application environments together and carries information (which is sometimes sensitive) between the components and micro-components of an application.

    As for external APIs, publishing public APIs has become a basic expectation for virtually any business with a software product—to the extent that there are now something like 22,000 public APIs out there as well as several orders of magnitude more internal APIs.

    The result is a world where APIs create potential attack vectors against virtually every application or service. It only makes sense, then, that the bad guys are increasingly focusing on exploiting APIs as a means of obtaining access to things you don’t want them to have.

    From Ransomware Protection to API Protection

    You might think that securing APIs requires fundamentally new security tools and practices. In reality, though, there are extensive parallels between mitigating ransomware risks and mitigating API security risks. And DevOps teams are a major line of defense against these.

    Here’s how to extend an anti-ransomware strategy to an anti-API-exploit strategy.

    Prevent Lateral Movement

    Like ransomware, which propagates from endpoint to endpoint laterally by exploiting flaws and vulnerabilities, API exploits also typically spread laterally across an environment.

    This means that even if you can’t prevent all API (or ransomware) attacks from breaching your perimeter, you can take steps to make it hard for breaches to expand. By detecting malicious activity inside your environment early on, you can stop lateral propagation of the threat before it leads to a largescale compromise.

    Focus on Data Security

    Ransomware attacks and API attacks are both focused on compromising the same crown jewel: Your data. Ransomware attackers want to hold that data for ransom. API attackers—such as those who stole sensitive information from compromised Peloton accounts, or others who breached LinkedIn’s API to scrape data about 700 million users—typically want to exfiltrate it, possibly to resell it or maybe just to harm your business’s reputation.

    Mitigating both ransomware risks and API security risks, therefore, boils down to securing your data. By enforcing strong access controls and segmentation over what both internal and public APIs can do, you mitigate the risk of data exfiltration due to API security breaches.

    Use Behavioral Security Models

    Investing all of your signature-based security controls in attack prevention doesn’t work well for either ransomware or API attacks, especially when they are zero-day or unknown attacks. While you certainly should do what you can to harden your environment, it’s impossible to guarantee that a breach won’t slip past your defenses.

    That’s why deploying behavior-based security models is key to protecting against both ransomware and API attacks. Behavioral security models detect anomalous activity inside an environment, such as unusual types of requests or strange request patterns. By modeling and baselining behaviors and detecting anomalies based on your model, you can prevent attacks from spreading once they are underway.

    Don’t Rely on Perimeter-Based Defenses

    Along similar lines, trying to protect the perimeter of your environment is not a surefire defense against either ransomware or API attacks. Instead, you need to distribute protections across all of your endpoints, applications, services and so on.

    Again, nothing will guarantee that attackers won’t get in. The success of your defense hinges, in large part, on your ability to make it hard for them to escalate their attack from a small-scale breach into one that impacts a wide array of resources.

    Look Beyond the Surface

    Ransomware and API attacks are similar in that both often involve attack methodologies designed to elude common security monitoring tools.

    For example, attackers might try to leverage ports 80 or 443 (the default HTTP/HTTPS ports), which will almost always be open on firewalls. It’s therefore imperative to avoid relying just on standard ports or on encryption to protect API traffic. Instead, you must look deep into the payload, then parse and understand the protocols. It is also important to monitor and collect data from multiple sources, then correlate and analyze them to a get a deeper understanding of what is actually going on within the environment.

    Conclusion

    To be sure, ransomware attacks and API security attacks are fundamentally different in some respects. They involve the exploitation of different protocols and the goals of attackers are usually a bit different.

    But in terms of how attackers operate, what they want to steal (your data) and the limitations of perimeter-based defenses, ransomware attacks and API attacks are actually pretty similar.

    That’s why developers and DevOps teams don’t need to rethink their entire security strategy to cope with the surge in API attacks. Instead, do what you’re already doing to protect against ransomware and use those techniques to help secure your APIs, too.

  • Filter the Firehose

    Filter the Firehose

    We are tired. Information overload is a problem in the modern world. We hear instantly about events we never would have known about otherwise, or that we would have learned about months after the fact. Today, moments after an event, we have thousands of “professionals” analyzing it for us, a millions-strong army of amateurs telling us they know everything about it and a legion of bots telling us what to think about the event. No matter what the event is. Often “the event” isn’t even newsworthy, and yet the process feeds on itself.

    We in IT have it worse. We have a flood of security and application events constantly flooding in and so much data that we call it a ‘data lake.’ More like a data ocean at some organizations. We’re tired. They call it by various names—alert fatigue, data overload, event flood, “the firehose.”

    No matter what it’s called, it’s a big problem. And if it is not effectively managed, we will make mistakes. Particularly in security—but also across IT—we cannot afford to allow the flood to distract us or burn us out.

    Look into AI. The amount of data, alerts, information, metrics, logs, traces, etc. that we are accumulating—and the rate at which we are doing so—is growing faster than humans can reasonably keep up with—even if there were no staffing constraints. Don’t hesitate any longer; don’t assume you or a vendor NOC are managing this well enough with only eyeballs. AI solutions have matured enough to reliably sift through at least the top layer(s) of noise, reducing the burden without increasing risk. In fact, given the increasing possibility of human error as volumes increase, probably with less risk.

    Ask vendors about it; talk to your security vendors about their offerings. A huge number of security vendors have NOCs now that they would love to have you subscribe to. Many of them are highly automated and use AI. Data vendors are behind a bit in this regard (for a lot of reasons) but they are catching up. Not too long ago, the vast majority of work done to normalize disparate datasets was done via trial and error. AI and scripts written from massive experience are lightening that load, at least.

    So check it out. Find ways to make things manageable again. We all know things are slipping through the cracks as the volume of items reported and the number of locations they are reported from increases. Don’t wait until you have another emergency; look into the options and get something running.

    For those of you already using AI/ML, unless you implemented it very recently, you’ll probably want to validate what you’re using, at what level and how much the AI/ML solution is helping, and tweak or replace. This space is moving too fast for “set it and forget it” today. Maybe in a few years, we can.

    With the rare exception of something I use every day and think you might benefit from, I adhere to a policy of not recommending products here, so I can’t point you in the right direction other than to say that your vendors (data, SIEM and security) can offer solutions and/or suggestions. That, and “You shouldn’t wait” are my suggestions.

    And keep rocking it. Part of that data/log/alerting growth is all the apps and infrastructure you have helped put in and that are critical to the organization. So protect it with every tool available—and use that free time to work more miracles.

  • Poor App Remediation Creates a Vicious Vulnerability Cycle

    Poor App Remediation Creates a Vicious Vulnerability Cycle

    A survey of 200 security professionals found nearly 83% of respondents reported that an increase in the rate at which applications are being deployed has led to an increase in the reintroduction of previously remediated vulnerabilities.

    The survey, conducted by Waratek, a provider of tools for managing security-as-code, found 93% of the security professionals polled currently work for organizations that have embraced DevOps or Agile development methodologies. Half of the respondents (50%) described maintaining security in those environments as challenging, with a slightly higher percentage (58%) noting that implementing security within DevOps process does slow down the rate at which applications are deployed. A full 85% also noted that manual security work slows the rate at which applications can be deployed as well.

    Surprisingly, more than two-thirds (67%) of security professionals also said they would be willing to sacrifice security to allow applications to be delivered on time.

    Nearly three-quarters (71%) also said they expected that keeping pace with DevOps teams would only become more difficult, with 61% also noting that keeping pace with DevOps teams resulted in delays to other critical security projects.

    Waratek CEO Doug Ennis said the root cause of the application security challenge can be traced back to continued reliance on manual processes. It’s too easy for developers, for example, to reuse code that contains vulnerabilities that were fixed in one version of an artifact but not another, he noted.

    That lack of automation also conspires to increase overall cybersecurity fatigue, added Ennis. More than 50% of survey respondents reported they spend days or weeks each year investigating false positives. There simply are not enough cybersecurity professionals available to make sure applications are secure without relying more on automation, said Ennis.

    In the wake of a series of high-profile security breaches, there is more focus on software supply chain security than ever. However, it’s not clear to what degree organizations are embracing automation and DevSecOps best practices to secure those software supply chains. The challenge is finding a way to build more secure applications without slowing down the rate at which those applications are built and deployed.

    Many organizations are clearly already shifting more responsibility for application security left toward developers and DevOps teams with hopes of achieving that goal. However, it’s still the cybersecurity team that get the call whenever there is a breach, noted Ennis. There must be workflows in place that enable automatic remediation in ways that don’t always require organizations to wait on a developer to build and deploy a manually-applied patch that is usually inconsistent, he added.

    The Log4Shell vulnerability is a perfect example of how flawed the current process for building and deploying patches is, Ennis explained.

    In fact, as more applications are deployed, developers often find themselves spending more of their time fixing vulnerabilities than they do writing new code. A new approach to building and deploying secure applications that leverages automation is clearly required. The challenge is creating automation that consistently achieves that goal each and every time required in a way that doesn’t break any of the applications that need to be fixed.

  • 3 New WFH and Hybrid Work Trends That YOU Need to Grok

    3 New WFH and Hybrid Work Trends That YOU Need to Grok

    Welcome to The Long View—where we peruse the news of the week and strip it to the essentials. Let’s work out what really matters.

    This week, a remote/hybrid work special: Bribes to relocate you, Fridays are forgotten, and big tech firms stop building offices. (more…)