devopsy.kinsta.cloud

Tag: API

The Definition of Faster in the Age of App Capital

Businesses want speed. One of the impacts of digital transformation—and the associated pressure to succeed in an app economy—is the desire to move at speed. Nearly half (48%) of organizations are driven by digital transformation to deploy faster according to our State of Application Services 2019.

But it’s not all about deployment. It’s also about development and responsiveness to threats and changes in demand.

Organizations want to develop and deliver apps faster. They want to adapt to unpredictable changes in business conditions faster. They want to respond to attacks faster.

So, how are they going to get it?

Faster Development

Deploying more frequently implies you’re first developing faster. That often means adoption of speed-oriented Agile methodologies. GitLab’s 2019 Global Developer Report: DevSecOps found that the majority (54%) are choosing Scrum, while 37% go for Kanban.

Componentization

Methodologies aren’t enough if application architectures aren’t well-suited to development styles. Smaller, focused teams sprinting toward frequent releases isn’t really a good fit for developing new features or fixes for traditional and monolithic applications. Componentization through the adoption of microservices and more distributed forms of architecture fit more naturally with modern architectural and operational principles. It’s no surprise that, on average, more than 80% of a modern application is made up of third-party components, heavily weighted toward open source.

APIs

Adoption of APIs, as well, is no surprise amongst those seeking faster development through componentized application architectures. APIs decouple implementation from interface and allow teams to make modifications to processing without impacting the use of APIs from other components or applications. This is a common pattern, with an impressive 64% of organizations creating APIs today for use in either internal or external use cases. Nearly 50% of those organizations rely on APIs to get ideas to market faster according to Jitterbit’s State of API Integration 2018.

Automation (CI/CD)

The build pipeline, too, has to keep up if frequently deployment goals are to be met. That means CI/CD tools that seamlessly move code from commit to test to release. The most popular CI and build tools in GitLab’s survey were, unsurprisingly, GitLab (61%) followed by Jenkins (36%) and Travis CI (12%). Notably, our own research finds Jenkins in use by 16% for network automation as well—a promising result if organizations are seeking to extend DevOps beyond delivery into deployment.

Faster Deployment

Developing faster doesn’t get ideas to market faster. That requires deployment. While born-in-the-cloud technology companies have mastered the delivery-deployment divide with ease, many established enterprises find this transition challenging. Existing organization structures as well as the continued need to support traditional and monolithic applications introduces conflicts that can be challenging to integrate with modern operational demands. But there is no doubt that organizations must overcome these challenges for those applications requiring faster and more frequent deployments.

Continuous Deployment

Don’t be fooled; established organizations are embracing automation and applying it to the deployment pipeline. The issue is often that traditional IT structures introduce inconsistent automation and self-service efforts. We see this play out in our own research, where strongly siloed team structures continue to impact automation efforts across IT.

Team structures matter and organizations whose compass is pointed at automated pipelines will need to address the cultural aspects of continuous deployment if they want to consistently deploy faster.

Public Cloud

Traditionally, the inability to achieve continuous deployment has pushed developers and app owners to turn to the public cloud, because it eliminates the obstacles in the slower, enterprise deployment path. The disparity in opinion on frequency of deployments is, in part, a contributor to the issue. Our 2018 NetOps/DevOps survey found that while 55% of DevOps and 52% of cloud architects felt their organization did not deploy frequently enough, only 30% of NetOps and the same percentage of operations agreed.

But it’s not the only driver. Digital transformation is certainly a factor; 33% of respondents in our State of Application Services indicated they are intentionally moving to deliver apps from the public cloud as a result of embracing digital transformation initiatives. The ability to easily integrate application services and subsequently automate operation is a huge boon to organizations seeking a faster route to frequent deployment.

Containers

In the cloud—but also on-premises—continuous deployment often requires the ability to adopt per-app pipelines and application services that support the model. Increasingly, organizations are turning to containers because of their ability to support rapid updates and operate seamlessly in highly volatile environments. We see containers in demand not only to support modern application architectures such as cloud native—the top container use-case at 33% of respondents to Diamanti’s 2019 Container Adoption Benchmark—but also for infrastructure. The demand for on-premises application services in containers has consistently grown year over year in our own research, rising from just 4% in 2017 to 15% in 2019.

Faster Responses

Not everything is about delivery and deployment. Organizations also need speed when it comes to security and operations. In today’s world, where more than half of all app interactions are conducted by bots, it’s important for organizations to respond quickly with a reject message to avoid falling prey to exploits or infection.

Real-Time Threat Analytics

One of the ways organizations are seeking to speed up their ability to respond to attacks is through the adoption of real-time threat analytics. With security continuing to be a top priority and consistent challenge, it was no surprise to see this category ascend into the top five strategy trends and technology in 2019 with 41% of respondents tagging it.

The availability of intelligent security application services will accelerate as machine learning and automation continue to apply their considerable capabilities to the problem of identifying troublesome traffic faster.

Orchestration Included

Machine learning isn’t the only way in which infrastructure and application services are getting smarter. As the need for faster responses to capacity and processing grow along with users, infrastructure and application services are evolving to include orchestration as a core capability rather than an add-on. Application service platforms with integrated orchestration layers will mean services that scale along with demand—on-demand. While such capabilities exist today—indeed, they are part and parcel of cloud and containers—the capabilities to automatically scale up and down based on application and user defined requirements is not native to most systems today. But they will be.

Faster Identification of Bad Actors

Finally, organizations need to combat bots for both business and security purposes. Whether organizations need to stop scraping—a real threat to business—or prevent bots from probing for vulnerabilities, identifying bad bot actors faster is critical today. Checking a box (I’m not a robot) isn’t enough today as bots are growing more intelligent and are able to foil such primitive techniques.

Organizations are turning to Bot Defense services capable of applying more modern, successful techniques to quickly identify and block bad actor bots. We’ve seen the use of bot defense services grow quarter over quarter in response to this need, and expect this trend to continue.

Faster Development, Faster Deployment, Faster Responses

When it comes to business’ need for speed today, we wind up looking at processes. Development processes are automated to gain speed with CI/CD. Deployment processes are automated to gain speed with continuous deployment and cloud. Scaling and security processes are being automated by systems and the introduction of orchestration as native to application service platforms.

Faster, it turns out, relies on an organization’s ability to automate processes across development, deployment and security.

— Lori MacVittie

February 14, 2020
How Integration Can Keep Up with Digital Demands

Integration holds the key to accelerating innovation, but in many organizations, business systems and application integrations haven’t kept up with the demands of digital. Data still exists in a siloed blend of on-premises systems and unconnected cloud applications, and IoT has added another layer of complexity into the mix.

Understanding the three trends below, and how to leverage them, will help organizations continue to drive innovation for years to come.

The Rise of the Integration of Things

It’s no secret that the adoption of IoT is increasing, generating massive amounts of data from devices and sensors. As the number of devices increases in response to the explosive growth of IoT, the amount of data increases exponentially. This explosion of devices–and its associated data–is driving the need to leverage that data for the benefit of the business, such as improved customer experience, cost and downtime reductions or efficiency and productivity gains.

Integration is the key piece that will enable businesses to get control of this data and put it to work. Without the right integration tools, it’s only possible to leverage IoT at specific levels. Essentially, integration provides the ability to connect the data from IoT devices and enterprise applications, orchestrate workflow and enrich IoT data with contextual information from other systems for better decision-making.

Organizations can even use integration to link IoT platforms together for a comprehensive view of data, analytics and actions–making it easier and quicker to analyze, draw conclusions from and determine how to use the data. On top of this, they can also create complex and sophisticated responses when something goes wrong by leveraging automated integration workflows.

As we continue to connect our devices, the Integration of Things will be as important as the Internet of Things.

Blurred Lines Between Application Development and Integration

For cloud applications to survive and thrive, they require built-in integration with other apps and data sources. Otherwise, users are constantly switching contexts from one tool to another when trying to complete a task, and content is scattered across the organization in multiple applications and departments. Because of this, the lines between application development and integration will blur as apps are compelled to have integration embedded under the covers.

There will be many benefits to this. First, organizations will easily create and deploy integrations directly within their products, fully baked into the user experience. These in-app integrations to third-party sources will unlock additional capabilities for users, empowering them to do more.

Second, building integrations without the right tools and specialized integration know-how requires a lot of work. Embedded integration eases the pain of monitoring API changes and documentations so that maintenance is smoother and cheaper.

Finally, business services can involve complex interactions with cloud apps and data that require ordering, have dependencies or may even need to be reversible on demand. Organizations can handle these sophisticated jobs by leveraging advanced integration orchestration to manage app interaction–meaning that instead of using manual code for complex data mapping, they can create a visual drag-and-drop data flow.

AI Powers Smarter APIs

Businesses have already been leveraging intelligent automation to improve operations, drive innovation and personalize the customer experience. As we head into the new decade, API management will harness the power of AI with innovative ways of protecting and controlling APIs and the applications they serve.

Service mesh, an increasingly popular way of discovering and observing services in distributed networks, will evolve into app mesh, an application-aware model of monitoring, managing and securing your APIs. API policies will be able to reconfigure dynamically based on traffic, security threats and identified patterns. Additional layers of defense will come from bots deployed in an organization’s network to constantly monitor behavior and these bots will use machine learning to improve automated threat responses for APIs.

The integration landscape is changing, but understanding how to harness these trends will ensure organizations are well-equipped to take advantage of evolving technologies for innovation.

— Ann Marie Bond

February 7, 2020
API Security in DevOps: Are We Too Comfortable?

Postman performed its annual survey of developers, and as this survey and many other surveys have shown, developers are generally comfortable with the level of API security that their organization has implemented. This has been a consistent trend, with most developers being comfortable that they are protected.

The problem is most organizations also put API security and management in the hands of developers.

Knowing the rate of change going on in Agile/DevOps shops, and knowing the complexity of environment, I think this mentality is akin to the pre-WW2 French belief in the Maginot Line.

While API security is certainly a necessity, it is not managed in a vacuum. You really are only as secure as your weakest link, and there are a lot of links in an API-focused environment. User credentials, tokens, SSL, requesting apps and their authentication, (still) code injection concerns…The list goes on and on.

I’m never comfortable with the level of API security currently implemented in DevOps environments. And I think it’s dangerous for you to be. It only takes one bad code iteration for one of the client instances to be vulnerable. That vulnerability exposes every API the client in question has access to, no matter what is being used as access control.

In the end, APIs must trust someone, or they’re useless. And that someone is most often a server app running on an instance that your users connect to. That app most often has its own credentials. How are they stored? How are they protected? What code changes could impact that instance’s security?

API gateways and management tools, when properly configured, are a big help here, but they are not a panacea. Since incoming connections must be trusted at some point, and outgoing connections must be assumed clean at some point, when one of those two things is a wrong assumption, what do you have in place?

Here’s the scenario: User A has rights to call the ShowMeCriticalData API. This API is called from a server app running on instance B. The server app authenticates user A, then calls the API for him–either with his credentials (best case) or its own (bad for tracking problems). If instance B receives an update that makes it vulnerable to compromise, user credentials are still needed to call ShowMeCriticalData, which is relatively secure, as long as User A is protecting their credentials adequately. However, if the app running on instance B is proxying requests and using its own credentials, Instance B becomes a weak point and attackers could use the vulnerability to call APIs without authenticating users. Worse, if User A is careless with their authentication info, there is little an organization can do to stop at least some misuse.

API logging and review can be put in place to notice odd behavior by username, and offer early warning that a user has been compromised. But will only catch most issues.

Of course, there are smart people working on the whole array of API security threats from cross site scripting to authentication spoofing even into TLS weaknesses. But that work is only useful if we all remember that our security could be better. With knowledge that there are weaknesses, we can look for things to mitigate the risk. As you all know, if you have public APIs the risk will never be eliminated–the purpose of a public API is for people to use it, and that exposes it to hackers that seem to be more ingenious by the week–but minimizing the risk is worth the effort.

Being secure in your current situation means you’re not focused on making it better. With API security, particularly in the type of dynamic environment DevOps creates, this is a dangerous road to be on. Assume there is a fatal flaw, and work to reduce vulnerabilities at each access point into the overall system.

This also goes for APIs you don’t own. We rate limited an API that called out to Google Places to match up with our plan/budget for the project. Our app was never going to come anywhere near Google’s caps on usage, but should a portion of our infrastructure be compromised, we could potentially end up with a massive bill from Google because we called the API a zillion times in a month. Likely? Not really. Possible? Not after a few lines of rate-limiting code.

You all have made some astounding apps. Make sure they are astoundingly protected. Because both business and users are counting on it.

— Don Macvittie

January 8, 2020
Breaking Down the OWASP API Security Top 10, Part 2

Due to the widespread usage of APIs, and the fact that attackers realize APIs are a new attack frontier, the OWASP API Security Top 10 Project was launched. From the beginning, the project was designed to help organizations, developers and application security teams become increasingly aware of the risks associated with APIs. This past September, the OWASP API Security Top 10 release candidate (RC) was finalized and published on OWASP.

In my first article on this topic, I provided a high-level view of the interaction between API endpoints, modern apps and backend servers, in addition to how they’re different from their traditional browser-based counterparts. I also discussed why this project was so important to the contributors and industry overall.

In my second article, I focused on the first five risks and emphasized some of the respective possible attack scenarios. In this article, I will attempt to clarify the last five risks to help organizations understand the dangers associated with deficient API implementations. The following discussion follows the same order as found in the OWASP API Security Top 10.

API6:2019 – Mass Assignment

Modern frameworks encourage developers to use functions that automatically bind input from the client into code variables and internal objects. Which means users should have the ability to update their user name, contact details, etc. (within their profiles for example), but they should not be able to change their user-level permissions, adjust account balances and other administrative-like functions. An API endpoint is considered vulnerable if it automatically converts the client input into internal object properties, without considering the sensitivity and exposure level of these properties. This could allow an attacker to update things they should not have access to.

Example Attack Scenario:

To illustrate this further, imagine a ride sharing application provides the user the option to edit basic information about themselves in their user profile. For example, they can adjust their user name, age, etc. In this case, the API request would look like this: PUT /api/v1/users/me with the following legitimate information {“user_name”:”john”,”age”:24}

However, the attacker determines the request GET /api/v1/users/me includes an additional credit_balance property (field) as shown below.

{“user_name”:”john”,”age”:24,”credit_balance”:10}.

The attacker desires to increase their credit balance on their own and replays the first request with the following payload:

{“user_name”:”john”,”age”:24,”credit_balance”:99999}

Since the endpoint is vulnerable to mass assignment, the attacker can easily adjust their own credit_balance at will, for example changing it from 10 credits to 99999 as shown above.

API7:2019 – Security Misconfiguration

Attackers will often attempt to find unpatched flaws, common endpoints or unprotected files and directories to gain unauthorized access or knowledge of the system they want to attack. Security misconfigurations can not only expose sensitive user data, but also system details that may lead to full server compromise.

Example Attack Scenario:

Say for instance that an attacker uses a popular search engine such as Shodan to search for computers and devices directly accessible from the internet. The attacker found a server running a popular database management system, listening on the default TCP port. The database management system was using the default configuration, which has authentication disabled by default, and the attacker gained access to millions of records with PII, personal preferences and authentication data.

API8:2019 – Injection

Injection vulnerabilities cause computing systems to potentially process malicious data that attackers introduce. To put it in its simplest terms, attackers inject code into a vulnerable piece of software and change the way the software is intended to be executed. As a result, injection attacks can be somewhat disastrous, since they normally lead to data theft, data loss, data corruption, denial of service, etc.

Example Attack Scenario:

Suppose an attacker starts inspecting the network traffic of their web browser and identifies the following API request designed to help a user recover their password. The attacker identifies the request responsible to start the recovery-password process as follows:

POST /api/accounts/recovery {“username”: “john@somehost.com”}

Then the attacker replays the request with a different payload:

POST /api/account/recovery {“email”: “john@somehost.com’;WAITFOR DELAY ‘0:0:5’–“}

By adding the ;WAITFOR DELAY ‘0:0:5’–” the attacker observes that the response from the API took five seconds longer, which helps confirm the API is vulnerable to SQL injection. Exploiting the injection vulnerability, the attacker was able to gain unauthorized access to the system.

API9:2019 – Improper Assets Management

Old API versions are often unpatched and can become an easy way to compromise systems without having to fight state-of-the-art security systems, which might be in place to protect the most recent API versions. Attackers may gain access to sensitive data, or even takeover the server through old, unpatched API versions connected to the same database.

Example Attack Scenario:

Say for instance that an organization redesigning their applications forgot about an old API version (api.someservice.com/v1) and left it unprotected, and with access to the user database. While targeting one of the latest released applications, an attacker found the API address (api.someservice.com/v2). Replacing v2 with v1 in the URL gave the attacker access to the old, unprotected API, exposing the personal identifiable information (PII) of millions of users.

API10:2019 – Insufficient Logging and Monitoring

Without logging and monitoring, or with insufficient logging and monitoring, it is almost impossible to track suspicious activities targeting APIs and respond to them in a timely fashion. Without visibility over ongoing malicious activities, attackers have plenty of time to potentially compromise systems and steal data.

Example Attack Scenario:

Imagine that a video-sharing platform was hit by a large-scale credential stuffing attack. Despite failed logins being logged, no alerts were triggered during the timespan of the attack, and it proceeded without being noticed. As a reaction to user complaints about a possible breach, API logs were analyzed and the attack was detected, way after the fact. The company had to make a public announcement asking users to reset their passwords, and report the incident to their regulatory authorities.

Pertaining to the five risks above, one could easily imagine similar attack scenarios. Those provided were just examples of the nearly unlimited possibilities when attacking vulnerable API implementations. Hopefully, you can see the risks above are primarily caused by errors or oversights that could be easily remedied, especially when it comes to the way organizations utilize APIs today.

— Erez Yalon

January 3, 2020
Best of 2019: Breaking Down the OWASP API Security Top 10, Part 1

As we close out 2019, we at devopsy.kinsta.cloud wanted to highlight the five most popular articles of the year. Following is the fifth in our weeklong series of the Best of 2019.

As a result of a broadening threat landscape and the ever-increasing usage of APIs, the OWASP API Security Top 10 Project was launched. From the start, the project was designed to help organizations, developers and application security teams become more aware of the risks associated with APIs. This past September, the OWASP API Security Top 10 release candidate (RC) was finalized and published on OWASP.

In my previous article, I provided a high-level view of the interaction between API endpoints, modern apps and backend servers, in addition to how they’re different from their traditional browser-based counterparts. I also discussed why this project was so important to the contributors and the industry overall.

In this article, I aim to clarify the first five risks by highlighting some of the possible attack scenarios to help organizations and end-users understand the dangers associated with deficient API implementations. The following discussion follows the same order as found in the OWASP API Security Top 10.

API1:2019 – Broken Object Level Authorization

Attackers can exploit API endpoints vulnerable to broken object level authorization by manipulating the ID of an object sent within the client request. What this means is the client can request information from an API endpoint that they are not supposed to have access to. This attack normally leads to unauthorized information disclosure, modification or destruction of data.

Example attack scenario:

Say for instance there is an e-commerce platform that provides financial and hosted services to a group of different online stores (shops). The platform provides an API used to gain access to revenue charts for each of their hosted stores, and each store should only have access to their own revenue charts. However, while inspecting the client request from a single store who wants to gain access to their own revenue charts, an attacker can identify (find) the API endpoint for those revenue charts and identify the URL in use, for example /shops/{shop name}/revenue_data.json. Using the names of other stores being hosted on the e-commerce platform, an attacker can create a simple script to modify the {shop name} ID object in subsequent requests, and gain access to the revenue charts of every other store.

API2:2019 – Broken Authentication

Being different than authorization discussed above, authentication on the other hand is a complex and confusing mechanism concerning APIs. Since authentication endpoints are exposed to anyone by design, the endpoints responsible for user-authentication must be treated differently from regular API endpoints and implement extra layers of protection for credential stuffing attempts, in addition to brute force password and token guessing attacks.

Example attack scenario:

Suppose an attacker obtained a list of leaked username/password combinations as the result of a data breach from another organizations. If the API endpoint handling authentication does not implement brute force or credential stuffing protections such as CAPTCHA, rate-limiting, account lockout, etc., an attacker can repeatedly attempt to gain access using the list of username/password combinations to determine what combination(s) work.

API3:2019 – Excessive Data Exposure

By design, API endpoints often expose sensitive data since they frequently rely on the client app to perform data filtering. Attackers exploit this issue by sniffing the traffic to analyze the responses, looking for sensitive data that should not be exposed. This data is supposed to be filtered on the client app, before being presented to the user.

Example attack scenario:

Image an IoT-based camera surveillance system allows administrators to add a newly-hired security guard as a system user, and the administrator wants to ensure the new user should only have access to certain cameras. These cameras are accessible via a mobile app the security guard uses while at work. The newly hired security guard’s mobile app makes an API request to an endpoint in order to receive data about the cameras and relies on the mobile app to filter which cameras the guard has access to. Although the mobile app only shows the cameras the guard can access, the actual API response contains a full list of all the cameras. Using the sniffed traffic, an attacker can manipulate the API request to show all cameras, bypassing the filtering on the mobile app.

API4:2019 – Lack of Resources and Rate Limiting

It is common to find API endpoints that do not implement any sort of rate limiting on the number of API requests, or they do not limit the type of requests that can consume considerable network, CPU, memory and storage resources. The amount of resources required to satisfy a request greatly depends on the user input and endpoint business logic. Attackers exploit these issues causing denial-of-service attacks and associated endpoint outages.

Example attack scenario:

Let’s say an attacker wants to cause a denial-of-service outage to a certain API that contains a very large list of users. The users’ list can be queried, but the application limits the number of users that can be displayed to 100 users. A normal request to the application would look like this: /api/users?page=1&size=100. In this case, the request would return with the first page and the first 100 users. If the attacker changed the size parameter from 100 to 200,000, it could cause a performance issue on the backend database, since the size parameter in use is so large. As a result, the API becomes unresponsive and is unable to handle further requests from this or any other client.

API5:2019 – Broken Function Level Authorization

Although different than API1 above, exploitation of this issue requires the attacker to send API requests to endpoints that they should not have access to, yet are exposed to anonymous users or regular, non-privileged users. These types of flaws are often easy to find and can allow attackers to access unauthorized functionality. For example, administrative functions are prime targets for this type of attack.

Example attack scenario:

To illustrate this further, imagine during the registration process to a certain application that only allows invited users to join, the mobile app triggers an API request to GET /api/invites/{invite_guid}. GET is a standard HTTP method used to request information from a particular resource. In this case, the response to the GET contains details about the invite, including the user’s role and email address.

Now, say an attacker duplicated the request and manipulated the HTTP method by changing GET to POST. POST is an HTTP method used to send information to create or update a resource. The URL would look like this: POST /api/invites/new/{“email”:”hugo@malicious.com”,”role”:”admin”}. In this case, the attacker easily exploits this issue and sends himself an email invite to create an admin account.

In the context of the five risks above, one could easily image many similar attack scenarios. Those provided were just examples of the nearly unlimited possibilities when attacking vulnerable API implementations. As you can see, the risks above are primarily caused by errors or oversights. I believe these risks could easily be managed or nearly eliminated when organizations improve their secure coding practices, especially when it comes to the way they’re utilizing APIs.

— Erez Yalon

January 1, 2020
Don’t Expect Miracles
Let’s run some fun scenarios to start this one off, shall we?
- “We’ll decide what language to use at a meeting next month. Until then, get to writing code!”
- “We might deploy internally, though Azure is a strong option, and we’ve discussed GCP. Now get operations up and running, we’ll tell you when we’ve decided.”
- “The code is changing constantly, get moving and write the documentation.”
All three of these scenarios are equivalent. The difference is how we, as an industry, view them. The first two, we (collectively) would never tolerate. The third? We do that to the documentation team every day. In DevOps and agile, we do it every day in a never-ending iteration. And it has to stop.

Two Different Documentations

For those not aware, there is documentation for internal use and documentation for external use. While we’re certain you have internal covered with self-documenting code, this blog will focus on external use, because if you expose and API, documentation is mandatory. If you’re not an end-user product, any UI requires documentation, too. So to be clear, for this blog we are talking generally about external documentation for B2B systems, and specifically about APIs, though UIs can have the same issues, depending upon how complex your market is.

We Generate Docs

Ever tried to use auto-generated documentation? I’ve yet to try one generated set of API documentation that I liked (they say Google Places has it, but my understanding is devs actually write that documentation and it’s reviewed by tech writers, so maybe we count that as one I liked, but its process is not yours).

Generated docs are only as good as the developers working on the code–very much like self-documenting code. If someone puts “This parameter is a placeholder for the spline development iteration” into a generatable comment, users are going to be totally clueless what that means. Heck, I just dreamed that statement up, and while we can all see it turning up in generated API docs, even I don’t know what it might mean.

Documentation Slows Up Dev

Duh? If you are loathe to commit to completing a project until you know the parameters, why would you expect a documentation team to be any different? Of course they need the final product for review, just as the business wants to see the final app before release. The product your docs team produces is detailed documentation about what is. If you change what is underneath them, they look bad for having inaccurate docs. When really it should be IT that looks bad for not considering the entire value chain when delivering. If you deliver an API that is poorly documented and people don’t use it, you have delivered nothing of value to the organization.

What to Do?

Put documentation on your DevOps teams. I mean an actual person whose responsibility is documenting. If you’ve got technical writers, use one of them. If you don’t, find someone who likes helping people use your toys, and give them time to do it. For some organizations, technical evangelists can fulfill this role if you don’t have them too busy helping active users. Have them sit in on meetings, treat them as just another piece of both agile and DevOps. They need to know when functionality, URI or whatever else changes, so they can update the docs. The best place to get that info is at stand-ups or design meetings.

Just like always, they’ll want to be there for design, but they’ll also want to be there for implementation decisions/changes. Because current documentation is the only worthwhile documentation. Give them sign off on release–not release of code (that’s a done deal) but publication of APIs. Before you let users at the access points, make sure they have solid instructions in how to use them.

Risks

Everything is a trade-off, but this one has some pretty easy choices to make–fewer people use your API because your documentation isn’t great (outright sucks in many cases), or you run some risks of slower iterations and you dedicate resources to documentation. Given that if no one uses your API, it is all wasted time, a bit of slow-down and man-hour investment is an easy choice to make.

Tech writers are generally cautious people who want perfection–because this is their product, and they want it right. This can cause unnecessary slow-downs in an agile/DevOps world. The easy answer here is to look for the same traits in documentation DevOps team members that you look for in other DevOps team members–ability to adapt, understanding that everyone is just trying to make it better and willingness to work through changes quickly to deliver.

Just like you don’t want the DBA that says “no” to every request and you have to argue with just to move the project forward–I worked with this guy, genius, but really frustrating to have on an agile team–you don’t want the documentation person that adds weeks to timelines so they can get it right. This risk is easy to manage. You know how to choose people for a fast-moving environment, just follow your gut.

Profit

Well, okay, maybe not profit, but deliver a solid product quickly with sufficient documentation that users not steeped in your corporate shared knowledge will understand and be able to use. That’s the goal of public-facing APIs, to be used. And it’s the goal of docs people to make certain those calling the APIs understand what they’re getting into and how to make the most of it.

Keep on cranking out cool solutions and crank out better docs. Your customers are counting on it. I’ve had my team go find alternative solutions based on shoddy API documentation. If I can’t use it, why would I keep trying if there are alternatives?

— Don Macvittie
December 24, 2019
Predictions 2020: Application Trends, Opportunities and Challenges

Without wheeling out all the usual clichés, 2019 has been another whirlwind of disruptive innovation and opportunity–with plenty of challenges to tackle and circumvent along the way.

The threat landscape mutated with predictable unpredictability, multi-cloud app deployments are becoming mainstream fixtures and DevOps methodologies started exerting a newfound influence on business plans.

The big question is what happens next? What anticipated app-centric trends will change the game and tear up the rule book (again)?

Digital Transformation Takes Shape

In 2020, we will see more organizations shift away from aspirational sloganeering to substantively embrace what can, and should be, a seismic step-change.

Inevitably, business leaders will get more involved in application decisions designed to differentiate or provide unique customer experiences.

Expect a new generation of applications that support the scaling and expansion of business’ digital models to emerge. This will include taking advantage of cloud-native infrastructures and driving automation through software development.

Further down the line, digital transformation efforts will likely be AI-assisted, particularly as they leverage more advanced capabilities in application platforms, telemetry, data analytics and ML/AI technologies.

End-to-end instrumentation will enable application services to emit telemetry and act on insights produced through AI-driven analytics. We anticipate that these distributed application services will improve performance, security, operability and adaptability, without significant development effort.

The Rise of Application Capital

Applications are now firmly established as the main conduit for companies to develop and deliver goods and services. They have become modern enterprises’ important assets.

Even so, most still only have an approximate sense of how many applications they have, where they’re running or whether they’re under threat.

This will soon change.

To manage Application Capital effectively, it is essential to establish a company-wide strategy that sets policy and ensures compliance. This includes addressing how applications are built, acquired, deployed, managed, secured and retired. At a high level, there are six distinct and unavoidable steps that need to take place: build an inventory, assess the cyber risks, define application categories, identify the application services needed for specific activities, define deployment parameters and clarify roles and responsibilities.

The primary aim of an application strategy should always be to enhance and secure all digital capabilities–even as their reach and influence shift and expand.

DevOps’ Culture Club

The technical minutiae DevOps methodologies and associated tools got a lot of publicity this year.

2020 will be all about getting the culture right, marrying theory with best practice and unlocking new levels of productivity without upsetting the operational apple cart.

Culture is not optional. Team structure alone dramatically changes pipeline automation, with traditional single-function teams often falling behind their contemporary, DevOps-driven counterparts.

Consequently, we will see more collaborative team structures and alignment on key metrics that give NetOps additional means to focus on what the business requires: faster and more frequent deployments.

DevOps has a 10-year head start on NetOps in navigating and overcoming obstacles around certain types of integration, tools and skillsets. Collaborative teams can explode the status quo by promoting standardization on tools that span from delivery to deployment (such as Jenkins and GitHub/GitLab).

DevOps should not–and cannot–end with delivery. That means deployment functions–along with a complex pipeline of devices and application services–must be automated. This won’t happen without effective cultural realignment.

The Data Center Is Alive and Kicking

Conflating the adoption of SaaS with IaaS caused speculation that cloud was cannibalizing IT. Pundits warned that data centers would disappear.

The rumors were exaggerated. Data centers are still being built, expanded and run around the globe. The cloud hasn’t managed to–and likely never will–kill the data center.

Early in 2019, an IDC executive told channel partners at the IGEL Disrupt conference that over 80% of companies they surveyed anticipated repatriating public cloud workloads. Security, visibility and performance remain common concerns.

Repatriation-related opportunities include improving availability of multi-cloud operational tools and a push toward application architectures that rely on more portable technologies such as containers.

The data center is not dead. It’s just evolving.

Application Protection Challenges

According to F5 Labs, the server-side language PHP–used for at least 80% of websites since 2013–will continue to supply rich, soft targets for hackers. Situational awareness is critical to mitigate both vulnerabilities and threats.

Businesses are also realizing that applications encompass more than just the code they execute. Attention needs to be paid to everything that makes them tick, including architecture, configurations, other connectable assets and users. The prevalence of access attacks, such as phishing, are an obvious case in point.

F5 Labs analysis of 2019 breach data confirms the need for risk-based security programs instead of perfunctory best practice poses or checklists. Organizations need to tailor controls to reflect the threats they actually face. The first step in any risk assessment is a substantive (and ongoing) inventory process.

As ever, the industry will gradually incorporate emerging risks into business models. For example, cloud computing has gradually shifted from a bleeding-edge risk to a cornerstone of modern infrastructure. The risks associated with the cloud have either been mitigated or displaced to contractual risk in the form of service level agreements and audits.

API and You Know It

The word is out. Application programming interfaces (APIs) can transform business models and directly generate revenue. Cybercriminals know this.

More than ever, organizations need to focus on the API layer, particularly in terms of securing access to the business functions they represent.

One of the biggest issues is overly broad permissions, which means attacks through the API can give bad actors visibility into everything within the application infrastructure. API calls are also prone to the usual web request pitfalls such as injections, credential brute force, parameter tampering and session snooping.

Visibility is another major and pervasive problem. Organizations of every stripe–including IT vendors–have a notoriously poor track record of maintaining situational API awareness.

API security can be implemented directly in an application or, even better, in an API gateway. An API gateway can further protect APIs with capabilities such as rate limiting (to prevent denial of service attacks) and authorization. Authorization narrows access to APIs by allowing access to specific API calls to only specified clients, usually identified by tokens or API keys. An API gateway can also limit the HTTP methods used and log attempts to abuse other methods so you’re aware of attempted attacks.

Appy New Year!

All this is of course the tip of an increasingly interconnected iceberg. Any New Year’s resolution worth its salt should include a commitment to comprehensively master the development, deployment, operation and governance of application portfolio. The best way to do this, and to get visibility into the code-to-customer pathways for all applications, is to leverage a consistent set of multi-cloud application services. Here’s to a safe, innovative and transformational 2020!

Want to learn more about what to expect in 2020? Join us Jan. 23 for our Predict 2020 Virtual Summit featuring discussions from some of the industry’s best and brightest offering up their visions for the future. Sign up today for this free daylong virtual event.

— Vincent Lavergne

December 20, 2019
Oracle v. Google and Its Impact on APIs and Code
What, if any, impact will the Supreme Court’s rulings on Oracle v. Google have on the industry at large?

TL/DR: It is unlikely to impact the vast majority of us, vendor and enterprise developer alike.

Disclaimer: I am a technologist, not a lawyer. I have done some research, and talked to lawyers, but all that I say here is opinion—even my direct quotes of lawyers are their opinion. The Supreme Court can and has caught people off guard and taken cases in different directions than expected. Though I believe this path is the one it will follow, nothing is assured until the Court rules.

Throughout this blog, references to The Case are actually using Harvard’s case document.

Background

On Nov. 15, the U.S. Supreme Court agreed to hear a dispute between Oracle and Google. The Oracle v. Google case has been going on for nearly a decade. And its roots go further back than that.

The origins of the case go back to 2005, when Google and Sun Microsystems discussed the licensing of Java for Android. This will likely be important because it shows that Google knew it was someone else’s intellectual property (IP). More on this later.

The crux of the case falls into three parts:
1. Eight files that were decompiled and used. In one court case, these were found to never have made it into distribution and only used for testing (which limits liability).
2. Two source files from Java used directly. These were found to be infringing, but damages were set to zero with consent of both legal teams.
3. The part at issue: the copying of the entirety of Java’s interface from 37 modules. Most tech writers imply this as 37 APIs. It is not—it is hundreds of classes and thousands of methods. The 37 is the number of Java packages.
The case has been through four court hearings/rulings so far. To sum up the relevant parts of these cases:
1. The first case (split between a jury and district court, running at the same time) included patent violations. Those were defeated and Oracle has not appealed that defeat. On copyright infringement, the court found that Google did violate copyright on the 37 packages and the directly copied source code, but not on the eight decompiled files. The jury could not settle the issue of “fair use” of the 37 packages.
2. The district court decided that the eight decompiled files and the copied source code for one function were violations, but remanded the rest back to the jury trial to settle the issue of fair use.
3. In the end, the district court determined that APIs are not protected by copyright. Their decision involved two parts:
  1. That there is only one way to write a given API.
  2. The interface code and hierarchy can be replicated by third parties without violating copyright because it is an interface and not content.
4. On appeal, the finding was reversed, and APIs were deemed copyrightable, sending the case back again.
5. At this point, Google asked the U.S. Supreme Court to intervene. The court chose not to.
6. Back at jury trial, the court found that while APIs may be copyrightable, Google’s use was fair use in the legal sense. This is the one topic the jury could not originally agree upon.
7. Oracle appealed this decision, and the District Court ruled that it indeed was not fair use to copy the structure and exact interfaces.
  1. Where the jury and the district court appear to disagree is in the idea of APIs. Calling a publicly documented API is obviously fair use, and the jury latched onto that idea. The District Court appears to acknowledge the difference between calling an API and copying it.
8. And after some tumultuous hearings and years in the court, the Supreme Court has agreed to weigh in on the issue.
The real problem here is that copyright law is not a great fit for software. It never has been. If I rewrite your Java application in Python, using the same exact structure and naming, is that protected? All of the “words” have changed. It is akin to rewriting your Hindi book in English without a direct translation … except that in software there is the added issue of it behaving exactly the same.

But we can’t fix the misapplication of copyright law to software—only Congress can. So what to expect?

A fair number of people, from the open source community and from businesses that see opportunity in a ruling that favors their world view, are advocating that affording copyright protection to APIs is a horrible idea because it would hurt independent developers. This is not quite true. Independent developers are not generally copying Java. While this case does have bearing on coding practices and what developers can do, it does not have a huge impact on innovation, just on copying others’ work and how that is achieved—much like the “reverse engineering” court cases of the ’80s.

There are good arguments on both sides of this case, but it seems to me that fair use will lose. They are public APIs in the sense that anyone can call them; indeed, some of these packages are so core to the system that you could argue every app must call them. But calling a function is not the same as re-creating it. In that sense, they are not in the public domain. Anyone can read a book, but that does not make it public domain.

But no matter how this turns out, it is all good for the industry. This uncertainty has hung over the IT sector for nearly a decade. Just like when a new boss comes along and there is uncertainty in the department until everyone adjusts to the new demands/style of the boss, a solid ruling will allow settling to start. The industry can adapt once it knows what the rules are and we can move along.

The real long-term answer is to have an equivalent to copyright and patent that is designed around the unique parts of computer code. Clearly, software is not the same beast as a book or magazine. The considerations are half-patent and half-copyright, and trying to bend those two to meet the needs of software has created some serious issues. Oracle quoted a 1995 case to showcase this problem—the company is quoting it to win the appeal, but it should be quoted as the reason the protections need to be designed for this medium that is not a publication and yet consists of words whose sequence matters: “Applying copyright law to computer programs is like assembling a jigsaw puzzle whose pieces do not quite fit.” (attributed by Oracle to the concurring opinion of J. Boudin in Lotus Dev. Corp. v. Borland Int’l, Inc., 1995).

One interesting bit that the courts thus far have ignored, and I would love to see the Supreme Court take up, is this: If Google entered into licensing negotiations, the company knew it was Sun/Oracle’s IP. To then go and copy parts of it without licensing appears wrong on its face. It would be interesting to understand the process that ended in the copying and hear whether that bears upon the legal standing of the case. I can think of both malicious and innocent routes to get there; it would be interesting to hear testimony about motivations and thought processes.

But even without delving into the licensing negotiations, this case stands to redefine copyright with regards to software. J. Michael Keyes, a partner at the international law firm Dorsey & Whitney, had this (and more) to say:

“One of the issues that the court may decide is the scope of copyright protection for software. Google had early success below in convincing the trial judge that the snippets of copied code were not copyrightable. Google may urge to have the court take a look at this issue anew. Another huge implication is the scope of fair use. The jury found that Google’s use was “fair” and that it was not liable for infringement. The appellate court overturned that jury conclusion and found that Google’s use was not fair as a matter of law. The Supreme Court will likely wade into this thicket and provide guidance on the scope of fair use in the context of software code and what role an appellate court plays in reviewing jury decisions in that regard.”

What we (all of us) do not want to do is confuse “good for the industry” with “legal standard.” It is not the court’s job to do what we think is best for the industry; it is the court’s job to determine applicable law and what that implies needs to happen. Presumably, the Supreme Court will do just that. And it may not please any of us. But if we think the current iteration of the law is flawed, that is something to take up with elected representatives, not ask a court to “fix” on the fly.

For the vast majority of us, though, it impacts our development not at all. Using APIs or forking open source projects are all still perfectly acceptable, and no ruling by the Supreme Court is going to change that—because copying someone’s APIs is the issue here, not using them. And open source licenses allow for copying and improving projects. So carry on. Get the job done. And read about the Supreme Court’s final decision with a smile, because either way, the industry will adapt to the new normal and continue forward.

— Don Macvittie
November 25, 2019
Serverless and GraphQL: Build and Launch at the Speed of Thought
Websites and apps are now, often, one and the same. Traditional development methods for an online or downloadable app can take months, given the backend development aspects that need to be managed—infrastructure provisioning and management, security, scalability, consistency and more, particularly for companies that have users around the globe and potentially very high usage demands.

In addition, once the app is built, it can take several days to launch and ensure global availability. Once launched, the app can require ongoing maintenance and supervision to ensure availability and performance, as well as constant fine-tuning of the balance between over- and under-provisioning to manage costs.

Serverless computing, when combined with GraphQL APIs, is the Holy Grail of application development. These technologies make it possible for developers to build and launch apps lightning fast by removing server provisioning and management concerns. Developers need only worry about building a compelling app with usability features that optimize engagement. Additionally, with serverless and GraphQL, an app can essentially be launched to a global user base within hours or even minutes, rather than days, while avoiding the need to manage servers. Let’s take a look.

What is GraphQL?

GraphQL is an API standard for building modern, data-driven applications based on the widely known HTTP Protocol. First created by Facebook in 2012, GraphQL was key to the social media giant shifting its mobile app from HTML5 to a native mobile app. Based on its success, Facebook made GraphQL available to the public as an open-source project in 2015.

With GraphQL, you define an agnostic Domain Specific Language (DSL) around your data structure in terms of:
- The programming language you use. As an API standard, GraphQL is open for implementations in many languages, such as JavaScript, Java and C#. You can essentially use the standard with your language of choice.
- The database engine you use. GraphQL is only concerned with having the data ready, rather than the source of the data and where it is coming from, so you have the freedom to implement a GraphQL API with almost all database engines.
What’s All the Fuss About Serverless?

Serverless represents a major paradigm shift in the way we create and run web services and apps. Serverless computing operates in the cloud, freeing developers from the burden of thinking about back-end tasks, such as managing infrastructure, scaling servers and provisioning capacity and resources.

Below, I’ve listed some specific benefits that serverless provides, looking specifically at serverless databases—many of these advantages are part of other serverless offerings as well. The overall impact is that with serverless technologies, developers only pay for what they use, or what the app uses depending on user demand. Developers can scale up to virtually infinite disk space and compute loads in seconds when traffic spikes hit, and scale back down when users go offline. The NoOps aspect of serverless means maintenance costs and burdens are removed. Additionally, global availability zones allow for replication between different locations and allow you to keep data where it’s needed—close to the user to minimize latency. Built-in encryption means developers can secure their apps with only basic security know-how.
- Consumption-based billing.
- Auto-scaling.
- No operations overhead.
- Global low latency.
- Secure by default.
When these benefits are combined with the advantages of GraphQL, one will notice a perfect synergy between the two technologies. Not only does the process become more automatic, the standard facilitates communication between internal teams and makes it easier for clients to integrate with your software, through:
- API discoverability and automatic documentation.
- Consolidation of data.
- Language and data source-agnostic.
- Type safety.
Serverless Plus GraphQL Equals Developer Nirvana

There are numerous examples of companies that have managed to build faster and cut costs with serverless. One of the most impressive demonstrations is when students in Australia managed to rebuild the census application and handle loads that the official $9 million system could not handle, at a fraction of the cost using serverless technology. GraphQL takes these advantages a step further.

Airbnb, a company already known for its serverless adoption, claims to be moving 10x faster thanks to GraphQL, while Coursera noticed that their frontend developers found it easier to discover and query new data as their API evolves. These benefits are not just limited to a developer’s own company. For example, GitHub clients can now fetch data more efficiently and more easily thanks to the GraphQL-based public-facing API. GraphQL is even competing with SQL—when databases such as FaunaDB offer a native GraphQL API, users move to GraphQL at a rapid rate.

Given the advantages of serverless in removing server provisioning and maintenance, and the freedom GraphQL gives developers when it comes to using programming languages and database engines, developers are essentially able to build and launch apps on a global scale as quickly as they think of them. And in the end, don’t we all want our customers to have a better experience?

— Brecht De Rooms
November 13, 2019
OpenLegacy Partners With Compuware to Advance DevOps on Mainframes

OpenLegacy has allied with Compuware to make it easier to build and debug microservices-based applications employing REST application programming interfaces (APIs) to access data residing on mainframes.

Zeev Avidan, chief product officer for OpenLegacy, said the company’s namesake platform analyzes mainframe environments to create microservices-based APIs written in Java without requiring any changes to the legacy environment. That approach not only minimizes disruption to the mainframe environment, but it also eliminates the need to rely on enterprise service bus (ESB) platforms and legacy web services interfaces to access mainframe data, he said.

The alliance with Compuware now makes it easier access the OpenLegacy platform from within Topaz Workbench, a set of tools from Compuware that are used to apply DevOps practices to applications that either run on a mainframe or need to access data residing on a mainframe.

Avidan said it is becoming increasingly common for organizations to want to build and deploy applications that need to access mainframe data as part of various digital business transformation initiatives. The challenge they face is that most legacy monolithic applications running on a mainframe don’t have an API to invoke. OpenLegacy provides a framework for generating those APIs, which in turn makes it easier to incorporate mainframes within the context of a larger DevOps process. The alliance with Compuware furthers an ongoing effort by the two companies to foster the adoption of best DevOps processes in mainframe environments, he said.

Legacy waterfall-based application development processes are still relevant in mainframe environments, he added, but it’s also clear organizations want to apply DevOps processes to applications spanning hybrid environments that include mainframes. The majority of data in many enterprise IT environments is created on a mainframe. Rather than being forced to copy that data, API platforms such as OpenLegacy make that data more accessible where it resides. Once the APIs are in place, it becomes practical for DevOps teams to create software development kits (SDKs) to access various applications running on mainframes, Avidan said.

Organizations that have invested in mainframes generally are interested in agile development methodologies that enable them to derive more value from those investments. The challenge they face is finding a way to make that data available in a way that doesn’t disrupt the mission-critical applications running on the mainframe, he said. Over time, IT organizations running mainframes will adapt their processes to incorporate best DevOps practices, but the level of disruption they can embrace in pursuit of that goal has to be minimized, Avidan noted, adding it would be unreasonable to expect that level of cultural change to occur overnight.

Despite ongoing predictions about the eventual demise of the mainframe, the number of workloads running on the venerable platform continues to increase, thanks in part to the ability to run Linux on a mainframe. What many organizations that opt to run Linux on a mainframe don’t always initially appreciate is the degree to which that decision also opens the door to transforming their internal IT culture, regardless of whether they are ready.

— Mike Vizard

October 9, 2019