Welcome to The Long View—where we peruse the news of the week and strip it to the essentials. Let’s work out what really matters.
This week: Wipro fires 300 for moonlighting at competitors, Python has a nasty 15-year-old bug, and companies are finding new ways to lay people off without calling it a “layoff.” (more…)
Welcome to The Long View—where we peruse the news of the week and strip it to the essentials. Let’s work out what really matters.
This week: Facebook engineers admit they’ve no idea what Meta stores (or where), California requires job ads quote salaries, and RISC-V will power future NASA spacecraft. (more…)
Welcome to The Long View—where we peruse the news of the week and strip it to the essentials. Let’s work out what really matters. (more…)
The threat of breaches is top-of-mind for many DevOps teams, and while security continues to shift left it appears to be a driving force for many decision-makers when it comes to choosing a DevOps platform or other technologies.
These were among the results of a GitLabs survey that also found getting security right is the number one challenge for DevOps teams.
Among the top reasons respondents said they chose DevOps were better code quality, developer productivity and operational efficiency—cited by 37% of survey takers, followed very closely by better security/more secure applications.
The survey also revealed that 53% of developers said they are fully responsible for security in their organizations, a 14% jump from 2021.
Michelle McLean, vice president at Salt Security, said it is risky for organizations to depend on development teams alone for security.
“Security and developer teams need to collaborate and work together–at every point in the application life cycle–not just development,” she explained.
For example, security teams must bring runtime insights back to development so they can be applied for education and training.
“It’s fundamentally important to choose a DevOps platform that either has security capabilities built-in or that can easily integrate with security platforms to facilitate collaboration by security and DevOps teams,” she said. “Otherwise, organizations run the risk of pushing out unsecured software or introducing other risks into the software supply chain.”
Tim Mackey, principal security strategist at the Synopsys Cybersecurity Research Center, added that an obsolete mantra in the cloud-native community held that security wasn’t a high priority as proper DevOps workflows meant that incremental changes were small and easily deployed.
He said the Log4j and Log4Shell experiences called that philosophy into question with a recognition that security issues can, and do, occur out-of-band from a release or feature and that continuous monitoring for external threats is part of a properly functioning security process.
“By implementing monitoring for new threats, regardless of source or nature, the ability for cloud-native development to quickly issue tested patches becomes itself a security feature,” he explained.
McLean pointed out that while there is a growing understanding of security in the pipeline with DevSecOps, it’s not enough.
“This is particularly true, for example, in the case of APIs. Because APIs aren’t just straight code, you must see them being exercised to spot logic flaws,” she said. “You need the ability to monitor them in runtime in order to spot anomalies and find areas where APIs might be exposing critical data. Although developers write APIs, their testing often falls short for security.”
To accurately see what’s going on with APIs, developers need to exercise the API–even in pre-production.
McLean said this is an area where security and development collaboration and integration is essential–APIs underpin a significant portion of the modern economy and the disconnect between security and development teams can have a direct economic impact.
John Bambenek, principal threat hunter at Netenrich, a security and operations analytics SaaS company, added there has always been a delay between the release of new platforms and the awareness that security capabilities for those platforms need to be implemented.
“With cloud-native and serverless technologies, that delay is as short as I remember in my career,” he said. “I greatly prefer the world where everyone at least understands the need for security instead of having to spend time getting buy-in that it matters in the first place.”
He said if you can manage and implement security in a seamless and efficient way early in the development process, it is far easier and cheaper to address issues than addressing them after code has already shipped—and that’s without adding in breach or liability costs.
“You can either fix it in dev or in prod, but you’re going to have to fix it sooner or later,” he said.
McLean noted many organizations are still maturing in their DevOps practices, not to mention their DevSecOps initiatives.
From her perspective, educating developers as to what is expected of them from a security perspective before a line of code is written is key.
“To help increase compliance, we’ve begun to see organizations taking steps to introduce security tooling into their pipeline process and add testing into the build process,” she said.
Then they’ll add a step that triggers an alert for non-compliant builds but won’t prevent the build. Then they’ll add a step that mandates sign-off by the developer that they’ve seen the security alert for a non-compliant build but still won’t interrupt the build.
“In rare organizations with highly mature DevSecOps processes, they’ll fail the build for non-compliance,” she added.
McLean said organizations need to understand what’s realistic given their maturity and risk tolerance and implement controls that match.
They also need to reevaluate those controls to look for opportunities to increase adherence to security policies and other compliance regulations.
Overall, continuing to educate through compliance automation in the software development life cycle (in development, testing and deployment phases) further helps reinforce best practices while ensuring issues are uncovered before they become a risk or technical debt.
Mackey said because DevOps platforms touch the software powering your business, when choosing any DevOps platform, the security of the platform itself and the competencies it enables should always be must-haves.
“In effect, any decision about new software should be based on how it improves the current security capabilities of the business,” he said.
Software development is fraught with risk that can run projects aground like a ship against a rocky beach. Whether it is misunderstood requirements, rapidly evolving marketplaces or old-fashioned bugs and schedule slippage, all can derail otherwise well-resourced and managed software development projects.
Even when software development teams eventually ship their product and collect payment from their customers, there is no guarantee that the product will be used. A 2022 study found that among those paying for SaaS software, 29% of their software is either underutilized or wasted. This study also noted that only 33% of IT professionals track SaaS software usage. The takeaway here is stark: As an industry, most of our labor is not going to productive ends.
This is not a new trend in software; an entire ecosystem of tooling exists to help software development companies collect user engagement metrics to drive development decisions. In fact, out of the top 10 million websites in the world, nearly two-thirds have an analytics solution in place. By contrast, however, API applications lag far behind in adoption of this practice. It is difficult to get market penetration data for API analytics, but a report from API management vendor Apigee revealed that even among its customers (that is, those who were already paying for a service that included API analytics), only 26% made use of the analytics functionality. This gives credence to the belief that most features go unused.
API application owners who are not integrating real-world analytics usage data into their decision-making process are leaving time and money on the table. Despite not having immediate access to user behavior, API analytics can be quite valuable in helping API application owners make better decisions about application development and maintenance.
API application owners should look to the following three ways to make better product development decisions.
One of the best lessons I learned from working at a bike shop is “stock what sells.” In other words, there’s no point in filling your shelves with products that your customers aren’t buying. The same goes for stocking your API product catalog with endpoints that your customer isn’t using.
API analytics allow API product owners to measure the use of features and evaluate their effectiveness. Product owners can use this valuable information to prioritize extending popular features or developing new features, with a clear yardstick for measuring usage. Further, this data enables development teams to “maximize the amount of work not done,” by shipping preliminary feature sets and evaluating usage metrics before a full implementation. Likewise, if the real-world feedback is not encouraging, it allows you to pivot to another opportunity within the product.
Unused code and features are a liability to an API product organization and can carry bugs, create performance problems and introduce security risks. I’ve seen situations as a software developer where these older features were written by developers who have long since left the organization. These unused sections often harbor serious security flaws and vulnerabilities (such as software dependencies that go unpatched for years) and dubious authorization logic that no one at the company fully understands.
Unused, extraneous features also contribute to a bloated product that can distract end users from the core value of your API. It can be difficult to advocate for the removal of “working product features,” but API analytics data can help make a strong case that certain features are nothing but an expensive distraction. Removing old, unused parts of your application also frees developers to focus on the parts that users love.
Performance is often a fine balance between implementation time/costs and product requirements.
It’s a well-understood maxim in software development that premature optimization is the root of all evil. Yet in the absence of real-world performance data, determining what represents premature optimization versus totally-necessary-and-prudent optimization often comes down to a subjective argument among engineers or worse, a risk-averse software development culture that contributes to slow time-to-market.
Having an objective measure of performance impact, like the number of instances in which clients are impacted by suboptimal performance, can help application owners decide how to allocate their budget to improving performance versus developing new features or fixing bugs. Moreover, having this feedback loop in place can empower development teams to take a nimble “go fast, release often and evaluate” approach to development. Rather than optimizing performance up front out of fear that invisible problems will lead to bad performance and dissatisfied users, performance can be tracked and reviewed over time.
API product owners operating a single application have a variety of choices when it comes to deciding which analytics tool to use for their product. Many specialized solutions exist for individual API development languages and frameworks. However, security teams have a unique problem: They need all the data, across all applications and across all the tech platforms within the organization. In that case, I recommend an API protection vendor with strong API analytics capabilities that not only protects against threats but also gives you the visibility you need to prioritize efforts.
API analytics should be a tool that powers innovation. Don’t leave it unused in your toolbox.
Welcome to The Long View—where we peruse the news of the week and strip it to the essentials. Let’s work out what really matters.
This week, a remote/hybrid work special: Bribes to relocate you, Fridays are forgotten, and big tech firms stop building offices. (more…)
Digital transformation is at the center of modern enterprise. Yet, as cloud initiatives take shape and the complexity of connecting systems and applications multiplies, the need to tie everything together and automate processes grows. APIs, otherwise known as application programming interfaces, fuel this automation. These small pieces of code—which connect and interconnect clouds, containers, microservices and applications—help organizations compress what used to be years of project development into months or weeks.
As businesses connect functionality—often across companies and service providers—APIs lead to an enhanced user experience. Simply put, they make it possible for much of the magic of the internet to take place, including seamless authentication, data sharing, payments and collaboration.
Today, it’s next to impossible to find an enterprise application that doesn’t rely on APIs. Yet, despite all the gains, APIs can cause plenty of pain. The slow performance or complete failure of an API can lead to an array of problems. These include customers who can’t complete transactions, employees who can’t collaborate through critical apps, and incomplete or incorrect supply chain data.
Today, a typical organization relies on more than 1,200 cloud-based applications and hundreds or even thousands of APIs. For many businesses, 50% to 90% of revenues are linked to APIs. If a system underperforms or breaks, there’s often no way to identify the root cause and fix it promptly. Reputational damage and lost sales often ensue.
It’s important to recognize that the monolithic applications of the past are a vanishing breed. Today’s highly-distributed software and services environment taps a universe of more than 24,000 APIs. It isn’t unusual for a transaction to pass through several cloud containers and across numerous companies en route to the desired outcome. Along the way, these APIs encounter different hardware, code and conditions.
When APIs work well, a process or transaction typically finalizes within milliseconds—and the digital experience usually excels. However, if a page loads slowly, a promotional code doesn’t work or payment doesn’t go through, the odds are high that somewhere along the line something went astray with an API.
For many organizations, there’s no simple and fast way to identify the source of the problem, understand the current impact, and take steps to fix the failure post haste. In fact, it isn’t unusual for several thousand or more failed transactions to pile up before the issue hits IT’s radar and anyone begins to notice a problem. By then, shoppers have likely abandoned shopping carts and sales have evaporated.
Adding to the challenge is the fact that APIs can fail in several ways. Not surprisingly, the number one problem is availability. If the API doesn’t function, the interaction or transaction cannot take place. In addition, an API might perform slowly or erratically, thus delivering highly inconsistent and frustrating experiences for users along with a problem that is difficult to diagnose and fix. Yet another problem is misconfigured or poorly functioning APIs that return incomplete data or the wrong data.
The real-world fallout can be significant. In practical terms, a customer shopping at a clothing retailer’s website may find it impossible to add socks to a shoe order or use one or more payment services. It may be impossible to cash in points from a loyalty program or view delivery information. Broken APIs can disrupt or break shopping carts, customer accounts, supply chain data, collaboration links, security tools, and much more.
Of course, when things go haywire, finger-pointing and conflicts with vendors, business partners and customers often ensue.
In today’s API-centric world, detecting problems early and often is critical. What makes the task so difficult is that API transactions do not unfold in a linear way. They bounce across servers and companies—often a couple dozen of them—scattered all over the world. Further complicating things: This routing can change at any given instant.
Identifying specific issues that trigger a problem—traffic bottlenecks, bandwidth issues, code problems and more—requires the right tools. It’s vital to understand how an API-based transaction behaves across a dozen or even two dozen hops and touchpoints that extend through clouds, containers and microservices.
Passive visibility tools, while useful for getting closer to the end-user, do not offer a comprehensive and granular view of events and processes. Today, businesses require broader and deeper observability capabilities that extend across an ecosystem of APIs and into the entire IT landscape—from network to device. It’s vital to understand how customers use websites, apps and other services.
As organizations delve deeper into clouds and digital interconnectivity, a more nuanced approach that incorporates both passive and active observability is in order. This approach makes it possible to emulate a user journey and quickly identify problem points. A business can detect problems as they pop up—after only a handful of failed transactions—rather than thousands or even tens of thousands of transactions later.
With a best-practice observability and remediation framework in place, an enterprise can establish baselines and triggers for alerts and notifications. It can assemble a complete picture within a single pane of glass and troubleshoot across the delivery chain. Whether the API involves a collaborative task within the organization or a financial transaction that stretches across several companies, it’s possible to view the entire transaction—and experience.
With both passive and active monitoring in place, a business can evolve beyond constant fire-fighting mode. Armed with the information and insight required to make vendors more accountable, businesses ensure they are getting the level of service (SLA) they are paying for. Make no mistake, as digital transformation initiatives accelerate and cloud complexity grows, it’s critical to ensure that APIs, microservices and other digital components are up to speed.
There is an ongoing trend in enterprise software: Using GraphQL as an aggregation layer to unify access to underlying data sources and REST APIs. Some call this the supergraph, and others refer to it as a meta layer. The idea is that by providing a more consumable layer to a wide range of internal sources, frontend developers are better equipped to access whatever data they need when constructing applications. But the quest to unify all data sources into a single schema doesn’t come without its share of trials.
GraphQL adoption is steadily increasing, with significant large enterprises making up some of the early adopters. For example, Netflix engineers recently shared how one approach gaining traction is “GraphQL microservices (GQLMS) as a backend platform facilitating rapid application development.” As the use of GraphQL increases across a large enterprise, though, scalability and security hurdles present themselves. These include retaining proper authorization throughout multiple layers and mitigating the risks of an inherently open schema.
I recently met with Hasura CEO Tanmai Gopal to discuss why GraphQL should be used as a middleware element to access and expose underlying datasets and APIs. According to Gopal, GraphQL is becoming the data API for modern applications—this could be a massive positive shift. However, the potential downside is around the time involved in integration. Thus, the key will be automating the integration of data sources so that developers and DevOps teams can focus on the apps and services they’re building instead of maintaining the data model itself.
“Data access is a bottleneck,” said Gopal. “It needs to be self-service.”
On one side, there are data producers, such as databases, internal services and external SaaS services. On the other side, there are the clients that consume this data, such as mobile apps, web apps, microservices and APIs externalized for third-party developers.
The problem, according to Gopal, is that the link between both sides is increasingly challenging to implement. It takes time to integrate with HTTP-based APIs and parse JSON payloads. The exchange protocols are dissimilar and you must solve for performance issues, caching and concurrency in data access. What’s more, the authorization context must be maintained for all data sources. These issues are exacerbated since not all developers are accustomed to the nuances of maintaining integrations.
Frontend developers want to communicate with a centralized data store, but also need to mix and match various sources. At this point, Gopal explained, a data access layer is necessary to handle the communication between both sides. Such a layer requires some integration automation to truly get APIs to developers at the edge. And it also requires a simple, unified schema—this is where GraphQL comes in.
“GraphQL is an absolutely delightful API to consume,” said Rajoshi Ghosh, co-founder of Hasura. “Using GraphQL as a data API can remove a lot of requirements of legacy knowledge of integration systems and how to securely fetch data.”
The concept involves using GraphQL as an intermediary data API—this would consume all sources, whether they are RESTful APIs, GraphQL services, an Amazon Aurora database or a PostgresSQL-style database—and unify them in a more accessible, pleasing construct. To create a unified API over these disparate sources, a middleware solution must map API resources to logical models and methods and add authorization rules to logical models, explained Gopal.
Although GraphQL enables developer agility, it’s the part in the middle that’s difficult to assemble. Building these connectors from scratch requires a lot of relearning and tedium to maintain, Ghosh said. And to truly cater to all data sources, organizations will likely need to craft custom plugins for legacy sources, he added.
Data-driven initiatives are a common technology goal, but the breadth of data sources is rising. GraphQL can play an important role in uniting these sources and providing flexibility for frontend developers to call precisely what they need. A unified graph is thus a boon for developer experience but will be challenging to set up from scratch.
Arguably, authorization is the biggest challenge when approaching GraphQL as an aggregation layer for multiple APIs. Any system that unifies multiple backend APIs must ensure that only the person with the correct permissions can access underlying data. A unified GraphQL layer must retain granular permissions that are endpoint-centric and unique to the specific caller.
“Most enterprises don’t have a good authorization story,” Gopal said. Implementing authorization for data APIs is hard, Gopal said, as it’s unlike REST where the user ID can literally be specified within the URL request. These kinds of fine-grained attribute level entitlements must be recreated as a policy engine and applied to the graph API as well. For example, Hasura implements authorization by using a token that scopes the entire experience to the particular user.
Looking to the future of this area, Ghosh noted that frontend stacks are becoming more robust and code in the backend is essentially becoming business logic deployed as serverless functions. And in this new reality of increasing abstractions, the role of modeling experts will likely become more important.
Furthermore, GraphQL must go beyond read-only capabilities to embrace another trend—real-time. Although most GraphQL APIs retrieve persistent data, GraphQL-based data brokers must also evolve to implement streaming, said Gopal, as real-time data and event-based subscriptions are becoming more common.
Today, you probably don’t need to unify all your sources into a unified data model all at once. This activity will likely be relegated to particular business domains or products to help them accelerate and demonstrate a proof-of-concept. But in the coming years, there will be mounting pressure to adopt a unified approach, Gopal said. “In a few years, if you don’t have a unified API strategy, you’ll have a massive problem.”
Welcome to The Long View—where we peruse the news of the week and strip it to the essentials. Let’s work out what really matters.
This week: Spear-phishing causes $540 million loss, a high severity bug in OpenSSL might be “worse than Heartbleed,” and Lennart Poettering is now working for Microsoft. (more…)
Welcome to The Long View—where we peruse the news of the week and strip it to the essentials. Let’s work out what really matters.
This week: Linus says next release will support Rust, FBI warns scammers are getting hired in deepfake interviews, and 80% of IBM staff stay at home. (more…)