Editorial Calendar for February 2016
Welcome to The Long View—where we peruse the news of the week and strip it to the essentials. Let’s work out what really matters.
This week, four things that caught my eye from Amazon Web Services’ re:Invent 2022 conference: SnapStart turbocharges Lambda, Graviton3E ARM HPC SoC, AWS continues hiring, and AWS’s origin story. (more…)
Publicly available vulnerability data can be a goldmine for insights into how DevOps and DevSecOps teams can prioritize threats and improve security across the pipeline. With this in mind, Inigo recently performed a deep-dive analysis of known vulnerabilities affecting GraphQL components—including GraphQL clients such as Relay and GraphQL servers such as Apollo, Graphene, Ariadne, GitLab Enterprise, Magento and others. Our main data sources for this investigation were the MITRE CVE database and the HackerOne Hacktivity portal.
First, a few caveats about the MITRE CVE database. While it’s considered the source of truth for CVEs and the best data source available for understanding these vulnerabilities, it has its limits. Not every vulnerability gets reported, and not every reported vulnerability gets an assigned CVE identifier. Vulnerabilities generally receive a CVE identifier when an impacted software vendor or researcher recognizes a vulnerability and files a CVE request with MITRE. Large software vendors including GitLab, GitHub, Adobe and many more also serve as CVE Numbering Authority (CNA) partners, authorizing them to issue CVE identifiers themselves (often only for the scope of their own software). All this is to say that the MITRE CVE database is a strong data source for our analysis, but it shouldn’t be considered comprehensive.
Inigo started its investigation by searching the MITRA CVE database for the term “GraphQL,” revealing 45 CVE records (at the time of the research) relevant to GraphQL clients, servers, and other components. Interestingly, the first tracked GraphQL vulnerability was CVE-2019-1000011 from 2019: Most likely not the first vulnerability ever discovered in GraphQL software, but the oldest available in this database.
Researchers were curious to know which classes of GraphQL vulnerabilities were most common, and which GraphQL software has the most known CVEs. The following chart displays the results of the analysis answering the first question.
Authorization bypass vulnerabilities are far and away the most common vulnerability class, and a majority representing 54.8% of all GraphQL vulnerabilities. At 16.7%, denial-of-service (DoS) vulnerabilities stand out as the second most common vulnerability class, followed by information disclosure (9.5%), code execution (7.1%), cross-site request forgery (CSRF) (7.1%) and injection (4.8%).
Analysis into which GraphQL software has the most CVEs revealed the following:
To be clear, these findings don’t necessarily speak to how hardened or how safe a particular GraphQL component is. Popular GraphQL servers simply draw more attention and have more eyeballs searching for vulnerabilities, whether they belong to attackers or security personnel. While this fact skews these results, viewing this data at a glance remains interesting.
The HackerOne Hacktivity portal offers a robust database resource that collects information on vulnerabilities as reported by researchers. The analysis looked at public reports of GraphQL vulnerabilities, finding 74 results since 2018.
As the graph shows, identified GraphQL vulnerabilities fell mostly into the authorization class (87%), with DoS (7%) a far second and race condition and session management vulnerabilities each coming in at 2.8%.
Bug bounties offer an interesting and likely influential factor here. Over the 2018-2022 timeframe, bug bounties granted for these vulnerabilities ranged from $250 to more than $20,000. At the same time, authorization vulnerabilities carry a high risk to sensitive data such as personally identifiable information (PII). This drives organizations to place premium bounties on these authorization flaws and motivates security researchers to focus on earning those prizes. Therefore, white-hat hackers will naturally test for authorization flaws first, perhaps leading to their overrepresentation in the data. It’s also important to note that bug bounty programs tend to prohibit testing for DoS vulnerabilities for obvious reasons, possibly reducing their representation here.
For developers, DevOps and security teams tasked with safeguarding GraphQL deployments, the first step is knowing what you’re up against. The findings above should serve as an eye-opener that helps teams to prioritize threats, whether they be authorization, DoS and other classes of GraphQL vulnerabilities. As more and more organizations rush into GraphQL (increasingly migrating from REST APIs), security cannot be put on the back burner. There’s too much at stake, and ensuring the security of GraphQL deployments isn’t nearly as difficult or time-consuming as many teams currently think.
GraphQL, the open source query language, has been steadily growing in popularity over the past few years. The language is excellent for frontend development, as it creates a usable interface to fetch precisely the data required for the specific client at hand. You can also use it to aggregate multiple backend services and data sources.
Studies show that the ecosystem around GraphQL is maturing—it has a very global user base and a growing number of tools exist to aid in its adoption. The 2022 State of GraphQL Report found that 61.8% of developers are happy with the overall state of the ecosystem. Yet many native GraphQL features go unused by developers, which suggests there is a wide range of experience levels, from novice to advanced users, working with the technology. This indicates there is plenty of room for growth and knowledge sharing.
Below, I’ll review the key takeaways from the 2022 State of GraphQL Report, considering overall usage data and focusing on the popularity of specific features. We’ll also highlight the state of standard tools used in conjunction with GraphQL to get the job done.
First off, there is a wide variance in the types of GraphQL APIs in production today, ranging from private to public endpoints. The report found 47.9% of developers use GraphQL for exposing an API intended for a personal website or app, 40.4% use it for private unexposed APIs for internal usage and 19.5% use GraphQL for publicly available APIs intended for third-party developer consumption. This demonstrates its flexibility to cater to multiple scenarios. And while the majority of clients that connect to GraphQL APIs include browsers (62.3%), it’s commonly consumed by other client types such as native mobile apps, other servers and desktop apps.
Previously, we discussed using GraphQL as a composition layer to combine underlying APIs and services. And the report backs up this idea that developers commonly use it to aggregate various data sources: 55.4% of GraphQL APIs hit databases, while 35% consume REST APIs and 14.6% eat other GraphQL endpoints.
In terms of benefits, one strong point is the ability to enforce and validate types for every object in the API. Users find other top benefits to be avoiding overfetching and aggregating requests. Yet, some hurdles still present themselves. These top drawbacks include error handling, performance issues, client-side caching and security woes.
Regarding user base statistics, most users have been working with GraphQL for less than five years. If we look at the type of sector utilizing GraphQL, we see it primarily leveraged in the programming and technology sector. This is followed by e-commerce and retail (16.4%), finance (8.9%) and news, media and blogging (7.2%).
Many of the more advanced GraphQL features still go unnoticed by the majority of users. For example, live queries is a feature that helps obtain live data from a GraphQL server, yet only 25% who know about it have used it. Over half of respondents had never heard of @skip, a directive that skips a field based on a value passed to it. Similarly, other directives like @specifiedBy, @defer and @stream also share little-to-no awareness or use.
Analyzing feature usage becomes a bit more serious if we consider the unadopted security and performance features. For example, only 43.2% of respondents have disabled query introspection. Disabling query introspection is a suggested security best practice to limit a hacker’s ability to perform surveillance. But as we’ve previously covered, security by obscurity isn’t enough to protect GraphQL, and an additional security emphasis is often needed.
As most GraphQL APIs are intended for personal or private scenarios, you would think that more security features would be leveraged to limit the threat of introspection and denial-of-service attacks. Yet, the report discovered low usage and awareness of security and performance features like query timeouts, query rate limiting and query cost analysis, all of which could be turned on to curb malicious behaviors.
Developers often seek to combine, aggregate or merge GraphQL schemas from different APIs. To do so, Apollo Federation is the most common solution, used by 22.5% of respondents. This is followed by Schema Stitching (16.9%), GraphQL Modules (4.4%), and GraphQL Mesh (2.9%).
Looking at other technologies used in connection, Next.js is the most popular framework and PostgreSQL is the most common database used. TypeScript and JavaScript comprise the most common languages used to write a GraphQL backend. Developers also use a balanced mix of IDEs to query and test endpoints, including GraphiQL, GraphQL Playground, Postman, Apollo Studio, and Insomnia.
The report also found relatively high usage and retention for tools like the GraphQL Code Generator schema builder, Apollo Client and Apollo Server. The data shows a crowded solution market with Apollo as an industry-leading outlier—many other clients, servers, API generators and schema builders have medium-to-high retention rates yet experience far less use.
The 2022 State of GraphQL Report sheds some interesting light on the state of GraphQL, validating a lot of the interest and trends we’ve covered lately on the blog. Of course, there is more to be said about many areas, such as smoothing the transition to GraphQL.
One key finding is the low number of security features being leveraged by developers. A greater emphasis on API security is required to avoid incidents and retain a zero-trust approach. This emphasis is part and parcel of responding to ongoing software supply chain threats and will require more awareness and thought leadership around GraphQL security best practices.
The 2022 State of GraphQL survey ran from June 15, 2022, to July 15, 2022, and collected 3,094 responses. The above analysis only scratched the surface and there are many other insights. For more findings, you can read the full results here.
VMware this week made a Spring Boot 3.0 update for building microservices-based Java applications that is based on the latest long-term support release for the Java Standard Edition (SE) platform.
Michael Minella, senior director of software engineering at VMware, said the latest edition of the framework supports Java 17. The hope is that adding this support will encourage organizations to move from Java 8 to the latest long-term release of the platform without requiring multiple upgrades to other previously released editions of Java SE.
In addition, Spring Boot 3.0 now also natively supports a high-performance Java development kit (JDK) known as GraalVM along with version 6.0 of the Spring framework for building Java applications using OpenJDK, an open source version of Java SE that VMware made available earlier this month.
VMware added support for Micrometer and Micrometer Tracing to improve the observability of Spring applications that will increasingly include multiple microservices dependencies.
Finally, VMware also added support for Jakarta Enterprise Edition (EE) 10, an open source extension of Java SE that is being advanced under the auspices of the Eclipse Foundation.
Minella said support for Java 17 coupled with GraalVM provides a stable foundation for building Java applications that are significantly faster than applications that rely on Java 8—still the most widely used edition of the platform. DevOps teams should begin to adopt a new standard for building and deploying the Java applications that still dominate enterprise IT environments, Minella said.
There is, of course, no shortage of programming languages being used to build enterprise applications. After three decades, however, Java continues to dominate, in large part because of the sheer number of developers that have Java programming skills.
It’s less clear how quickly legacy Java applications will be updated to run on Java 17 and the GraalVM. Now that Java 17 is supported in Spring Boot—the most widely used framework for building Java applications—the pace of adoption for the latest long-term release of the Java SE platform should accelerate in 2023.
In addition, Minella noted the latest edition of the Spring framework will put organizations in a better position to take advantage of forthcoming advances such as Project Loom, an OpenJDK effort to replace threads with a lighter-weight concurrency model based on fibers that are managed via the Java runtime rather than the underlying operating system.
Collectively, these and other technologies will make it less expensive to run Java applications that are significantly faster than previous generations of applications, he added.
Most DevOps teams will find themselves managing a mix of Java applications—constructed using multiple variants of the core Java SE platform—for many years to come. However, it’s probable most new Java applications will be based on Java 17. In addition, there will be efforts to modernize legacy applications based on previous iterations of Java SE. As a result, most DevOps teams working in enterprise IT environments should expect to see an increase in the overall number of Java-based projects using microservices moving through their pipelines in the months ahead.
Welcome to The Long View—where we peruse the news of the week and strip it to the essentials. Let’s work out what really matters.
This week: Data centers cause climate change, and Meta is rolling out Precision Time Protocol. (more…)
Welcome to The Long View—where we peruse the news of the week and strip it to the essentials. Let’s work out what really matters.
This week: The EU wants to put servers in orbit, a GDPR penalty for Discord, and Amazon has the hatchet out. (more…)
Oxeye today disclosed that it has discovered a critical vulnerability in the open source Backstage software used to build developer portals. Backstage was originally created by Spotify.
A 1.5.1 update to the Backstage platform remediated a sandbox escape vulnerability that can occur via a third-party Scaffolder plug-in that could be used to conduct unauthenticated remote code execution (RCE).
Oxeye CTO Ron Vider said Oxeye worked with Spotify to responsibly disclose the issue and enabled a patch to be created prior to the disclosure of the vulnerability, which Spotify gave a 9.8 out of 10 severity rating.
Backstage is now being advanced under the auspices of the Cloud Native Computing Foundation (CNCF), but is already widely used by organizations such as American Airlines, Netflix, Splunk, Fidelity Investments, Epic Games and Palo Alto Networks, as well as by Spotify.
The vulnerability is another example of a potential template-based attack through which run shell commands can be manipulated to inject malware into an application development environment.
In general, separating the logic from the presentation layer as much as possible can greatly reduce exposure to the most dangerous template-based attacks.
Vider said Oxyeye is focusing its vulnerability research on cloud-native platforms such as Backstage that are generally being used to build modern applications based on a microservices architecture. It’s not clear to what degree organizations have transitioned away from more monolithic approaches to building applications, but it’s clear that a much larger percentage of applications are now being constructed using microservices.
Cybercriminals, of course, are targeting these platforms as part of a sustained effort to compromise software supply chains in the hope that the malware they inject will find its way into multiple downstream applications. Those attacks, in turn, are convincing more organizations to embrace DevSecOps best practices to better ensure the integrity of their software supply chains. The challenge is many of those attacks are being aimed at open source software projects that don’t always have enough resources to quickly develop a patch in the event a zero-day vulnerability is disclosed.
While it’s impossible to know the extent to which cybercriminals are already exploiting vulnerabilities in DevOps platforms, the level of cybersecurity scrutiny being applied has dramatically increased in the wake of a series of high-profile breaches. The Biden administration has even gone so far as to issue an executive order requiring federal agencies to better secure their software supply chains. As a result, DevOps teams should expect there to be a lot more DevOps platform vulnerability disclosures in the months ahead, and that those vulnerabilities will require immediate patching.
Like it or not, software supply chains are going to become more secure. The only thing left to be determined is the level of pain that will be experienced as more vulnerabilities are discovered in the DevOps platforms that were used to construct them.
Welcome to The Long View—where we peruse the news of the week and strip it to the essentials. Let’s work out what really matters.
This week: People won’t shut up about Rustlang, and Musk mandates Twitter teams return to the office. (more…)
From Twitter and Netflix to Salesforce and Confluent, modern analytics applications are being built by developers across the world’s leading digital businesses to create digital tools that serve real-time analytics to hundreds or even thousands of concurrent users.
Looking more closely at what these apps do reveals some critical, strategic technology decision-making. In particular, why do developers increasingly turn to Apache Druid instead of the myriad of other databases out there?
In general, they look to Druid to build apps that deliver on a range of objectives, including creating operational visibility at scale, customer-facing analytics, rapid-drill down exploration and real-time ML pipelines. Let’s take a look at each of these in turn:
Building apps for operational visibility can include a wide range of use cases, such as product analytics, digital operations, observability, fraud detection and IoT. In each situation, developers need to minimize the gap between events created and time-to-insight.
From a technology standpoint, there are several databases that can analyze events in real-time, including stream processors like Apache Flink or ksqlDB, time-series databases like InfluxDB or TimeScaleDB and also key-value stores such as Redis. The problem is that all of these technologies share the same inherent limitation for analytics: They aren’t able to process OLAP-style queries effectively at any meaningful scale.
For instance, if the app in question calls for analyzing time and non-time-based dimensions over the past month, assuming ingestion was one million events per second, that would equate to more than 2.6 trillion events. Any database that supports stream ingestion can query the current status (setting aside the challenge to handle ingestion at this scale) and, to an extent, basic aggregation of numbers and counters. Difficulties can arise, however, if the app requires aggregation across all of these events, “group by” on non-time-based attributes or high concurrency. Using the wrong database leaves users looking at the ‘spinning wheel of death’ as they wait for results.
In these circumstances, devs turn to Apache Druid. A real-world example explains how: Salesforce engineers built an analytics app using Druid to monitor their product experience. The app enables engineers, product owners and customer service reps to query any combination of dimensions, filters and aggregations on real-time logs for performance analysis, trend analysis and troubleshooting. As a result, their query results are returned in seconds with billions or even trillions of streaming events ingested every day.
On the face of things, the technologies offered by Twitter, Forescout and Atlassian don’t have a huge amount in common. Look beneath the surface, however, and they all apply analytics to more than just internal decision-making. They have also created analytics apps and processes to provide their customers with powerful insights as part of a value-added service or a core product offering
What does that teach other organizations? When building analytics for external users, a whole different set of tech challenges must be addressed. While concurrency is clearly a key consideration, the database should also support current and future user growth with ease. The thing is, concurrency also has to be delivered economically, or it can have a huge impact on infrastructure costs.
Similarly, delivering a good interactive experience is dependent on sub-second query response performance—it has become a minimum user requirement almost everywhere. Instead of fighting PostgreSQL or Hive or any other analytics database to meet the performance requirements at scale, Druid offers the performance devs and users expect.
Take Confluent, for example, which is a popular event streaming platform. They built their cloud-based monitoring service, Confluent Health+, with Druid. Having initially used a different database, they discovered that, as their data volume grew, their legacy pipeline struggled to keep up with their data ingestion and query loads. By switching to Druid, they have been able to create the impressive user experience developers everywhere are focused on delivering.
Generally speaking, the primary role of reports is to fulfill a simple requirement: Deliver data in a consumable form. What organizations are increasingly looking for, however, is a deeper level of insight and understanding that shows why something happened. They can take this information and apply it to a wide range of possibilities, such as solving, anticipating or preventing future problems or opportunities.
Addressing these ‘why?’ questions means data has to be analyzed at tremendous speed to reveal root causes. While this is a simple proposition when the data set in question is small, when dealing with data drawn from today’s high-volume sources, such as cloud services, clicks and IoT/telemetry, the underlying database has to correlate a few to hundreds of dimensions across highly cardinal data with billions to trillions of rows. That presents a much more difficult challenge.
Indeed, when data is analyzed at a huge scale, full table scans are simply too slow. Exacerbating the problem is the typical query-shaping techniques used to speed up performance have expensive tradeoffs:
Looking at real-world use cases once again helps demonstrate the role of Apache Druid in situations such as these. For example, developers at cybersecurity company HUMAN built an analytics app for internet bot detection using Apache Druid. Previously, their cloud data warehouse was too slow to keep up with their need to drill into massive web traffic quickly. Today, their Apache-powered app gives their data scientists the ability to aggregate and filter across trillions of events instantly, meaning they can classify anomalies and train their machine learning models to detect real-time malicious activity.
For many people, analytics is synonymous with viewing a synthesized data set presented via a UI like a chart, graph, report or dashboard. Increasingly, however, new operational workflows are completely automated without any human intervention or data visualization.
Google Maps is an example of this use case, given it optimizes routing based on real-time and historical traffic flow and patterns. Another example is Reddit’s use of Apache Druid to deliver real-time ad placement within milliseconds of a new visitor arriving on the app. In this context, there is no data visualization to interpret; it has been replaced by automated decisions derived by inference. These performance requirements play particularly well to the capabilities offered by Druid.
Across a growing variety of demanding use cases, devs are adopting Druid to build inference-based capabilities into their apps. From diagnostics, recommendations and automated decisions, today’s organizations need instant query responses of high cardinality, alongside highly dimensional event data at scale and in real-time. Delivering performance levels like this is becoming increasingly important for businesses focused on innovation, quality of service and competitive advantage.
Business is accelerating, and experience– for customers, partners and employees–is everything. This means nearly all applications need to be enhanced with new features, security updates and bug fixes on ever-shorter cycles. But isn’t rushing new code into production, especially in cloud environments, a recipe for mistakes, security vulnerabilities and user frustration? Not if the right approach is taken.
Continuous delivery (CD) enables development teams to rapidly and regularly deploy software updates into production systems without jeopardizing user experience or creating security risks. Battle-tested open source CD orchestration solutions, including Spinnaker and Argo, provide a cost-effective way to deliver secure, high-quality software at massive scale to multi-cloud and Kubernetes environments.
Most software development teams suffer from quality issues related to outages, tracking down bugs and security vulnerabilities, complex upgrades, release rollbacks and more. According to a 2020 report by the Consortium for Information & Software Quality™ (CISQ™), software quality suffers as companies rush to add more value at greater speeds. In 2020, the cost to companies was $2.08 trillion in the U.S. alone.
CD relies on a framework of stakeholder collaboration, process automation and continual and rigorous risk assessment and feedback to ensure a smooth, high-volume flow of high-quality code is deployed to production systems. When done right, CD reduces software development risks by enabling development teams to deploy changes such as new features, configurations, bug fixes and even experiments into production safely, quickly and sustainably. It also reduces costs associated with software deployments, failed releases and customer frustration and attrition.
With effective CD workflows in place, development teams gain confidence and are more productive, delivering better products faster while eliminating the frustration of missed release dates, management wrath and the blame game associated with software bugs and vulnerabilities making it into production.
Prior to the rise of open source CD solutions, companies often relied on point automation using scripts. These could improve efficiency a bit, but when companies moved from the monolithic architecture of a mainframe or on-premises servers to a microservices-based production environment, the scripts could not be easily adapted or scaled to cope with the more complex environment. This led to the formulation of continuous delivery orchestration solutions that could ensure code updates would flow to their destination in a repeatable, orderly manner.
Two highly popular open source CD solutions have emerged, Spinnaker and Argo. Spinnaker was developed by Netflix and extended by Google, Microsoft and Pivotal. It was made available on GitHub in 2015. Spinnaker creates a “paved road” for application delivery, with guardrails to ensure only valid infrastructure and configurations reach production. It facilitates the creation of pipelines that represent a software delivery process. These pipelines can be triggered in a variety of ways, including manually, via a cron expression, at the completion of a Jenkins Job or other pipeline and other methods. Spinnaker also provides cluster management, as well as visibility into an application’s cloud footprint, letting users resize, delete, disable and even manually deploy new server groups using strategies like blue-green deployments.
Argo CD, a container-native deployment engine for Kubernetes, supports CD pipelines, enabling the automated deployment of applications within Kubernetes. Introduced in 2018, Argo CD utilizes the Argo GitOps Engine, an open source library that implements core GitOps features. It allows users to manage, automate and audit the software deployment with Git as the source of truth. Features of Argo CD include drift detection, the analysis of the health of application resources and a real-time view of application activity.
As open source software, both solutions offer all the benefits that the open source model offers, including low start-up costs, rapid and continual innovation and paid support when it becomes necessary.
Symphony provides a cloud-based collaboration platform for financial institutions that are highly regulated and render services to millions of customers, so the solution must meet rigorous standards for security and regulatory compliance. The company wanted to enable a multi-tenant system to improve performance and reduce the cost of operations for each client. However, the existing software delivery system created bottlenecks in integration testing, long triage times and manual handovers of releases to site reliability engineering (SRE). The ever-growing number of delivery pipelines and software delivery environments also increased infrastructure costs and the burden on human resources, making it difficult to deliver features faster and ensure compliance at scale.
Symphony determined it needed a completely new CD solution supporting infrastructure and software delivery workflows that they could retrofit into their existing platform. The solution needed to support infrastructure-as-code, pipeline-as-code and configuration-as-code, while providing visibility, auditability and faster approval processes. Symphony opted for a CD solution that harnessed the power of both Spinnaker and Argo to automate the entire end-to-end CD process, including pipelines for infrastructure, deployment, testing and promotion to production deployments. With the solution, Symphony was able to slash the time required to onboard new environments by 98%–just 10 minutes to get the infrastructure and platform ready for the client. This also allowed for fast and frequent changes.
With automated CD in place, the company enjoyed significant cost optimization through automation. It was also able to increase security and compliance through automatic policy checks that continually enforced compliance across the stages of each pipeline–without compromising delivery speed.
As you review potential CD solutions, ensure they can deliver the required level of:
While the benefits of a CD solution are enormous, deploying one is not trivial. It pays to make good choices to get it right the first time. A professional assessment of your existing environment and processes can help you in several ways, including:
If your company requires the rapid delivery of secure, high-quality software releases, the time to deploy a continuous delivery solution is now.