devopsy.kinsta.cloud

Tag: API management

  • Hasura Adds Join Capability to GraphQL Engine

    Hasura Adds Join Capability to GraphQL Engine

    Hasura today announced it has extended its GraphQL engine to make it possible to join data generated by multiple sources.

    Hasura CEO Tanmai Gopal said GraphQL Joins makes it possible to join data from across different GraphQL services that can then be accessed via a single GraphQL application programming interface (API). The extension of the federation capabilities enabled by the Hasura GraphQL engine eliminates the need for custom code that would otherwise be required to join that data, he added.

    The Hasura engine automatically generates a GraphQL schema from a data source that can then be used to accelerate the development of APIs. GraphQL Joins extends that federation capability to now include multiple GraphQL and, eventually, other data sources accessed via, for example, legacy REST APIs, said Gopal. That capability will make it easier for a wider range of developers to build applications that aggregate data across multiple sources, he added.

    The core Hasura Engine is based on open source software that the company claims has now been downloaded more than 500 million times. Hasura makes available a cloud service based on that core engine. That approach eliminates the backend complexity that IT teams encounter when adding GraphQL APIs to an application environment that already has a range of APIs that internal IT teams need to support, noted Gopal.

    GraphQL was originally created by Facebook, and it’s still not clear to what degree GraphQL APIs will supplant REST APIs. Developers tend to prefer GraphQL APIs because they provide more granular control over what data is accessed. The challenge is the number of backend services that expose GraphQL APIs is still comparatively limited. Most IT teams will not be replacing REST APIs with GraphQL-based APIs overnight, but the percentage of new applications that rely on GraphQL APIs will steadily increase. DevOps teams looking for easier ways to provide developers with access to backend services via a GraphQL API can employ the Hasura Engine to automate that process.

    Overall, the number of APIs employed within IT environments is rapidly expanding as more microservices-based applications are deployed. Each microservice generates its own API. Over time, IT teams will find themselves managing a range of types of APIs to support multiple digital business transformation initiatives that an organization may have simultaneously launched. In addition to building APIs, many of those organizations are now consuming data via APIs exposed by another organization. Over time, the level of interdependency between APIs and applications will serve to make application environments much more complex. The good news is that APIs make it easier to upgrade—and even replace—backend services without disrupting application services.

    It’s not always clear, though, who within IT organizations will manage those APIs. They are often created by developers that then look to IT operations teams to maintain them. Before too long, it’s not uncommon for IT teams to find themselves managing hundreds, perhaps thousands, of REST and web services APIs—and soon, that may include many built using GraphQL.

  • Kong Enterprise 2.7 Simplifies API Management

    Kong Enterprise 2.7 Simplifies API Management

    Kong Inc. this week advanced its service connectivity platform strategy with an update to Kong Enterprise that makes it simpler to collectively manage groups of application programming interfaces (APIs).

    Michael Heap, director of developer experience for Kong, Inc., said that capability will make it simpler to, for example, assign rate limits to different tiers of APIs.

    In addition, Kong Enterprise version 2.7 adds support for real-time and event-based use cases involving the open source Kafka streaming platform and webhooks. Usage of event-driven architectures is rising as more organizations embrace digital business transformation initiatives that require near-real-time processing versus relying on traditional batch-oriented applications, noted Heap.

    Finally, IT teams can now also securely store secrets such as user names/passwords, API tokens, database credentials and private keys that might be used to access Kong Gateway. An improved user interface for Kong Manager UI makes it simpler to configure the Kong OpenID Connect (OIDC) Plugin for the Kong Gateway.

    Overall, Heap said, the latest update also provides 25% increased throughput and improved latency. In tests run by GigaOM, a third-party research firm, Kong Enterprise 2.7 achieved 52,250 transactions per second (TPS) maximum throughput with a 100% success rate.

    Kong is making a case for a service connectivity platform that enables an IT team to manage APIs at a higher level of abstraction by integrating all Layer 4 through Layer 7 services for both monolithic and microservices-based applications. Rather than having to manage an array of networking services and associated services in isolated silos, that layer of abstraction makes it simpler to manage distributed applications that have dependencies on a wide range of APIs.

    In general, the number of APIs that organizations are using both internally and externally has expanded tremendously over the last several years. The challenge now is finding a way to not only manage and secure all those APIs but also retire them as new services come online. Many APIs are created and simply forgotten about because their developer neglected to inform IT operations of their existence. Those so-called “zombie APIs” then become a cybersecurity liability when cybercriminals discover they can surreptitiously exfiltrate data via those APIs.

    APIs are, of course, an integral element of any software supply chain. But as high-profile security breaches put software supply chain security in the spotlight, it’s only a matter of time before IT teams look for more efficient ways to manage and secure APIs at scale.

    Inevitably, that also means organizations will need to decide how much they want to rely on proxy software, API gateways and service meshes to achieve that goal within the context of a larger service connectivity platform.

    In the meantime, the days when APIs were managed in isolation from the rest of the IT organization are coming to an end. The only thing left to determine is how involved API developers need to be in their ongoing life cycle management.

  • When to Use API Management and Service Mesh Together

    When to Use API Management and Service Mesh Together

    Though sometimes described as competing architectures, API management and service mesh have slightly different use cases and can actually work well together. Whereas API management provides the business logic for outside-facing traffic, a service mesh excels at handling intercommunication between microservices. An organization could certainly adopt both simultaneously across their projects. So, when does it make sense to use both API management and service mesh?

    I recently chatted with Mark Cheshire, director of product, Red Hat, to discuss the nuances between API management and service mesh. According to Cheshire, API management and service mesh can work quite well side-by-side for particular use cases. For example, a large organization using service mesh could benefit from applying API management that wraps microservices in a usable contract for internal departments. Or, API management could help a company expose specific APIs from the mesh to outside partners.

    When to Combine API Management with Service Mesh

    Say you already have service mesh—do you need API management, as well? Cheshire offered four questions to keep in mind that may influence your decision.

    1. Consider the number of connections and the number of consumers: Is it many-to-one or one-to-many?

    Within a monolithic application, communication is more centralized—there is more of a one-to-one relationship. However, when you break apart services into microservices and reuse them across an organization, a service may now have many consumers. This is a good indication you need an API management approach for the interface.

    “When environments get larger and larger, you can’t manage that as a flat mesh,” explained Cheshire. “You need some kind of organizational grouping by line-of-business or by the functions that you may have.”

    API management can create more formal interfaces between internal groups in a large organization that likely have little to no contact with one another. For example, perhaps the customer data team needs a formal method to connect with the sales organization. Or, maybe a security team supporting a login microservice needs to access financial details from an accounting division.

    2. What sort of access control and limiting does the system require?

    Consider what access control is necessary to secure your environment and avoid service disruption. A service mesh will typically use a zero-trust approach, applying mutual TLS over every ingress to each microservice. API management can also bring authentication controls and more advanced authorization and traffic mechanics.

    API management can provide a streamlined way to apply fine-grained controls for inter-domain traffic that follows the business logic. This helps assign access to specific resources to different requesting parties, such as read-only access for basic users or update access for admins. Amid increasing API attacks, setting fine-grained policies and permissions around your endpoints is becoming increasingly important to avoid data overexposure. This imperative increases as the binary between external and internal traffic fades with more public cloud use, remote work and third-party contractors.

    API gateways also place greater emphasis on communication rules and patterns. Rate limiting, for example, is very important to avoid collapsing entire networks, explained Cheshire. In addition to rate limiting, filtering, pagination and content negotiation are also standard API management capabilities that further optimize the exchange between the client and server.

    3. Do you require documentation for the services?

    In a monolithic setup, you likely have all developers sharing the same repository, explained Cheshire. This may still be true within an internal network of distributed microservices wherein each microservice is independent, yet developers can still access each other’s repositories. When you’re connecting services inside a domain, it’s often just the code itself (and perhaps some in-line commenting) that represents the contract between services.

    However, north-south traffic requires much more formalization, he said. Formal contracts are often necessary when one domain is communicating with another within the same company. In this scenario, there’s less tribal knowledge around how things work. API management is useful, in this case, to produce a developer portal with human-readable documentation that explains the methods, request types, URL structures and sample calls. This makes it much easier for someone without preexisting knowledge of the system to integrate with it.

    This formal contract is often generated from a specific file, such as OpenAPI Specification, which has emerged as industry-standard metadata for describing REST APIs. Groups can use OpenAPI to automatically generate documentation and even drive specification-first development. All in all, standardizing the way you document APIs and microservices can significantly improve the knowledge-sharing around how services connect.

    4. What is the nature of interactions?

    Another trait to keep in mind is the nature of interactions between services, Cheshire said. He divided systems into two large types: First is a graph model, in which any service can connect to any service. The second is a hierarchical model in which north-south traffic enables one service to have multiple consumers. A hierarchical model is necessary for productization and SaaS since it creates a single intercept point to the underlying resources. Maintaining hierarchy here is vital to retain simplicity, as you wouldn’t want the consumer to be able to hit all private microservices in a connected graph (unless that’s the intent!).

    API management thus enables the sharing of APIs on the mesh. “While they both use communication proxies, and while there are parallels in function, the differences in domain set them apart, and most companies will see significant benefits from using them together,” wrote Dino Chiesa and Greg Kuelgen on the Google Cloud blog.

    Better Together

    In summary, API management and service mesh go well together. “The two are complementary and can work together,” added William Morgan, CEO, Buoyant. Joaquim Moreno wrote, “they are not mutually exclusive; we should think about them as different layers in your infrastructure.”

    Specifically, combining the two is beneficial when:

    • Exposing and productizing APIs on the mesh
    • Formalizing inter-service communication in large corporate networks
    • Applying fine-grained access control; rate limiting for API-based microservices
    • Standardizing services with more formal documentation
    • Interaction is a more hierarchical communication model
    • Providing access to partners
    • When you need to externalize data to third-party plugins or integrations

    “The worlds of APIs and services mesh are becoming ever more closely connected,” said Cheshire. And API management won’t stop at REST—as developers adopt new integration styles like Kafka, GraphQL and asynchronous formats, the solutions will evolve. “It’s essentially a governance layer for all your integration points.”

    Large companies that seek to standardize inter-domain traffic within organizations will likely benefit from API management. But API management also offers the potential to productize internal infrastructure. Another benefit is that an API management gateway may also monitor and log requests, helping spot actors at the perimeter and keeping a helpful tally of incidents.

    Of course, adopting both API management and service mesh may be overkill for some scenarios. And, it’s good to remember that introducing additional layers inherently increases overall latency. It’s up to developers and architects to weigh the options and determine what’s appropriate for their unique environment. To enable this, a healthy culture of open communication is important to ensure teams are on a similar page across divisions, to avoid mismatched adoptions that only increase technical debt.

  • How Are API Management and Service Mesh Different?

    How Are API Management and Service Mesh Different?

    The use of APIs and microservices is increasing—a recent study from F5 estimated that the industry is approaching 200 million total APIs. As organizations expand their use of APIs and microservices, they inevitably require some form of service management architecture. There are primarily two major design options for implementing this: An API management system or a service mesh.

    The key distinction between API management and service mesh can be framed using the four cardinal directions. API management is typically adopted for REST services that face external developers—referred to as north-south traffic. On the other hand, service mesh is more commonly used by internal suites of microservices to help control lateral, or east-west, communication. There are more nuances between each, as we’ll cover below, but this is essentially how they’re adopted in practice.

    However, this binary is becoming blurred as collections of enterprise digital applications expand and access to cross-company business domains begins to look and feel more like consuming a third-party SaaS. Furthermore, some suggest that with the right controls, service mesh can cater to external users in the same way API management does. What’s more, service mesh and API management are complementary—they can coexist within the same organization and even be deployed in the same projects.

    I recently chatted with Mark Cheshire, director of product at Red Hat, about the emergence of both API management and service mesh. According to Cheshire, both approaches accomplished similar goals but evolved very differently. Below, we’ll revisit why these two paradigms emerged and compare and contrast them. In a separate article, we’ll consider when it’s best to integrate them both.

    What is API Management?

    API management is most often associated with north-south traffic. By exposing your API to the outside world, you enable other developers to connect it to their systems and leverage your data and functionality like building blocks.

    API management arose in response to the rise of web APIs. API management is a broad concept encompassing many functions required to expose an API endpoint to the outside world. This can include authentication, authorization, request routing, documentation, rate limiting, monitoring, logging, monetization and life cycle management. A key goal is to make integration easier for the developer consumer. Architecturally, API management follows a gateway pattern. Consumers issue requests to the gateway which then performs routing to the underlying services.

    For APIs, the big catalyst was the emergence of Web 2.0, Cheshire explained. Numerous startups began consuming RESTful APIs to connect their web properties and partner ecosystems. Companies also found REST APIs to be a much more straightforward integration approach than SOAP and SOA. They also saw the ease-of-use benefits and adopted API-first designs within their internal infrastructure.

    As of 2021, over 70% of APIs are built for internal use. So, do all these services need API management? It depends. “If you communicate externally, you need API management,” explained Cheshire. But, it’s a grey area for services within an enterprise, he says, though inter-domain traffic often could benefit from API management.

    A diagram of API management capabilities, from 3scale
    A diagram of API management capabilities, from 3scale

    Strengths

    • Can enable north-south traffic
    • Protects the boundary of an application
    • Designed to ease external developer consumption
    • Potential for monetization, subscription plans
    • Typically plug-and-play; easier to support
    • Many solutions on the market
    • Developer experience and docs
    • Analytics and reporting

    Weaknesses

    • May be heavier than service mesh
    • Fewer open source options; vendor lock-in
    • Analysis paralysis in comparing solutions

    What is Service Mesh?

    Service mesh is most often associated with controlling east-west communication between internal services. Service meshes like Linkerd, Istio and Kuma make it easier to reimplement similar logic—such as networking, observability, traffic management, security policies and routing—across many microservices from a single control plane.

    Service mesh was born to address slightly different challenges than API management. “The transition from monolithic to microservices is all about east-west traffic,” Cheshire said. Companies found it slower to innovate monolithic applications and thus broke them up into microservices that communicate via APIs. But these microservices all needed similar operational capabilities, which would be cumbersome to repeat over and over for each service.

    “For one monolithic app blown into microservices, you may not even need a service mesh,” said Cheshire. “However, when you have hundreds or thousands, that’s when applying service mesh becomes critical.”

    Istio service mesh architecture
    Istio service mesh architecture

    Architecturally, service mesh is typically divided into a data plane and a control plane. The service mesh deploys a sidecar proxy that sits alongside each individual service. Istio, for example, utilizes the Envoy proxy. Other meshes like Linkerd use their own proxy.

    Strengths

    • Fits the microservices architecture paradigm well
    • Good for traffic routing; improving resiliency
    • NIST-recommended architecture for achieving zero-trust
    • Many open source options
    • Fits large microservices deployments

    Weaknesses

    • Requires a more hands-on approach
    • Potentially more maintenance to support
    • Unnecessary for a single service
    • Requires filter chain add-ons for north-south abilities

    Comparing API Management and Service Mesh

    API management is predominantly aligned with externalizing APIs to partners. When you do so, maintaining the contract with a consumer is more important to meet SLAs. Thus, API management commonly delivers life cycle management abilities, such as versioning. API management is necessary even if you are exposing a single API, as this service needs to be adequately documented and secured. That being said, these tenants can also be relevant to developers reusing software within an organization—there’s something to be said for adopting a product mindset when dealing with internal APIs.

    On the other hand, service mesh is more aligned with controlling internal networks. Microservices are rarely deployed alone; typically, multiple microservices make up an application. And, many more might exist throughout an organization for unique purposes. Service mesh thus helps maintain consistency throughout large networks of decoupled microservices. Service mesh is also a rapidly growing field, experiencing an explosion of new development and production use. A Cloud Native Computing Foundation (CNCF) report found that service mesh use rose in production by 50% in 2020 alone.

    To put the comparison another way, William Morgan, CEO, Buoyant described how API management is more about business logic, whereas service mesh handles intra-cluster operations:

    “API gateways typically handle ingress concerns, not intra-cluster concerns, and often deal with a certain amount of business logic, e.g. ‘User X is only allowed to make 30 requests a day’. By contrast, a service mesh like Linkerd is focused on operational concerns (e.g. instrument these calls and report their success rate; secure this communication between these pods) and purposefully avoids the mixing of business and operational logic; and focuses on communication between the services on the cluster.”

    Though service mesh and API management achieve slightly different aims, both layers make it easier to reuse web-based services. And, they certainly can be used in conjunction. For example, you could build API-first microservices with service mesh and simultaneously use API management to expose certain domains. In another article, we’ll consider when it’s best to integrate the two technologies.

  • Why API Quality Is Top Priority for Developers

    Why API Quality Is Top Priority for Developers

    It is no secret that web APIs have become increasingly important to the operation of modern businesses. According to RapidAPI’s Developer Survey and Insights report, 61% of developers used more APIs in 2020 than 2019, with 71% planning to use even more in 2021. We are no longer tied to a small number of monolithic service providers. Businesses tend to gravitate toward specialized managed services, such as billing and authentication, which are backed by APIs. Business models for new products and services are constructed based on APIs. Trust has become a necessity for APIs. APIs that perform consistently and with high quality earn trust, and those that do not are abandoned for a competitor.

    Why Does Quality Matter?

    If a market exists for an API, it’s a given that there will be competition. To remain competitive, APIs must build and maintain trust with their consumers; sales and marketing can only go so far. The highest quality APIs will inevitably capture the largest slice of market share. Quality isn’t a destination, either—it is a collection of goals that require continuous investment.

    What Does Quality Mean for an API?

    Just like the code that drives an API, the quality of an API itself is quantifiable. The following facets describe some broad categories used to measure API quality. The weight of each facet will vary depending on how business-critical the API will be when in production.

    Resilient. An API is resilient when it continues to operate in the face of adverse conditions. Can an API continue to operate (even at reduced capacity) if the database is down? How does it handle network traffic spikes?

    Robust. A robust API is liberal in what it accepts, but conservative in what it sends. Will it crash when presented with a poorly formatted request, or respond with instructions on how the caller can fix the problem themselves? Can the code be restructured without breaking the API contract?

    Secure. API security is vital in most deployments. Are efforts made to check for OWASP Top 10 Web App security risks? Are open source vulnerabilities known and addressed in a timely manner? Are secure endpoints gated with authentication?

    Discoverable. APIs that are well documented and well structured are much easier to consume and integrate with other services.

    Consistent. A consistent API is one that changes without negatively impacting existing consumers. Changes are backward compatible or versioned as new features are introduced. API consumers should know far in advance if a breaking change will be coming if it cannot be avoided.

    How Do I Get There?

    Processes, tools and, most importantly, culture, are the key to high-quality APIs. Let’s explore how we can approach each API quality facet.

    Resilient. Processes such as chaos engineering, load testing and manual quality assurance can uncover situations where an API is failing to handle unexpected situations. Deploying your API to a cloud provider with a compelling SLA instead of your own hardware and network shifts the burden of infrastructure resiliency to a service, freeing your time to build features for your customers.

    Robust. A comprehensive suite of automated tests isn’t always sufficient to provide a robust API. Edge cases, unexpected code branches and other unplanned behavior may be triggered by requests that were not considered when writing the test suite. Traditional automated tests should be complimented by fuzz testing to help uncover hidden execution paths. Canary and blue-green deployments can further expose your API to the real world in a limited manner and prepare for when even more unexpected requests will hit.

    Secure. API security is a moving target. It is expected that most APIs are built on layers of open source libraries and frameworks. Software composition analysis is a necessity to stay on top of zero-day vulnerabilities by identifying vulnerable dependencies as soon as they are discovered. OWASP guidance is a must-have—directing API developers to implement attack mitigation strategies such as CORS and CSRF protection. Application logic must be well tested for authorization and authentication.

    Discoverable. For REST APIs, the OpenAPI initiative provides a consistent language for describing the API. Many frameworks, such as Spring Boot, can even generate OpenAPI documentation directly from your code. gRPC-based APIs offer similar benefits by providing both instructions for accessing the API and building the clients to communicate with it. Adding support for GraphQL allows developers to connect multiple APIs together with consistent tooling. Whichever direction you choose, it is vital to keep API documentation up to date and accessible.

    Consistent. To be consistent, the API development process should make incompatible changes obvious. Code reviews can catch many breaking changes right at the source. API contract testing and integration testing can be used to discover API changes as part of the build. When working in a microservices environment, staging environments may alert engineers if an API change will cause problems before production.

    Quality Builds Trust

    According to the RapidAPI survey, 58% of executives said participating in the API economy was a top priority for their organization. The figure was even higher in certain industries such as telecommunications, where the number is 89%. Whether you are building APIs for internal use or for public consumption, your goal should be to earn the trust of API consumers. Investing in the culture, tools and processes to build quality APIs will pave the way to building and maintaining that trust.

  • A Funny Thing Happened On the Way to API Management

    A Funny Thing Happened On the Way to API Management

    As API use has exploded across IT departments large and small, the need to manage APIs as independent entities—both APIs themselves and calls to them—rapidly became apparent. We have been through this cycle several times before–ORBs, DCOM, object repositories … the list goes on.

    There are several competing goals when introducing some management structure into formerly as-needed remote calls occurs. We want to publish and advertise connections in the hopes we can avoid (or at least minimize) reinventing the wheel, while we also want to make access easier for app developers that are trying to use our APIs. Meanwhile, we want a centralized location where ops can monitor API usage, security can review APIs for compliance and “approved” APIs can be advertised or even hosted. Finally, we want to give internal developers access to approved external APIs that have been deemed appropriate and safe for the organization to use.

    As with previous iterations of this process, these goals, taken together, prove to be huge. There is a lot there to try and cover in an ever-changing API landscape. It is worse in this iteration, simply because of the proliferation of external APIs. Prior to REST, each iteration had more APIs, both internal and external, than the one before, and REST is keeping with that pattern, though now that we can look back, the external bit is a hyperbolic curve, not a line.

    So, it is interesting and cool to see the market fragmenting a bit. The entire package is a huge order that a few companies have taken on; I’ll leave it to you to decide if they’ve handled it well enough for your org. Others have taken one of two approaches—centralizing API access to external services that have been vetted and interfaced with by the vendor or enabling internal API developers and users.

    Now, we have a couple of manageable chunks to look into. I would even go so far as to say most vendors come from one or the other of these worldviews, and you should compare their “tilt” to your organization’s needs before purchasing an API management product. I would much prefer if we separated these markets into ‘API management’–those doing (or trying to do) it all, ‘API integration’–those simplifying and enabling access to approved APIs like Salesforce or ADP and ‘API enablement–those making writing, publishing, securing and advertising internal APIs easier. But I don’t get to make these calls, so we’ll have to watch it unfold.

    If your need is to streamline and protect your organization’s access to external platforms, and you use many popular services, start with API integration. If you need to get a handle on APIs that you are creating and using internally, start with API enablement. If you need both, and a centralized store of approved APIs regardless of source, look to API management. APIs have become central to our development efforts, and there will only be more of them in the future, so it is worth deciding where your org could use help and set out to research your options.

    Of course, you’re rocking it. DevOPs isn’t letting DEVops poke too many holes in the firewall; DEVops is cranking out solutions and devOPs is keeping them alive. This is just another tool to keep things on an even keel as more and more of the organization’s development is API-based.

  • Postman Employs Browsers to Simplify API Collaboration

    Postman Employs Browsers to Simplify API Collaboration

    Postman today announced it is making its namesake platform for managing application programming interfaces (APIs) available via a browser to make it easier to share and inspect APIs. Previously, Postman required DevOps teams to download a desktop application to access the Postman platform.

    Kin Lane, chief evangelist for Postman, said agent software developed by Postman makes it possible to create a proxy application to facilitate API request-sending at scale that overcomes the cross-origin resource sharing (CORS) limitations of browsers.

    DevOps teams can now simply attach a URL to an API that is shared with any other member of the team. Clicking on that URL exposes the API in a way that makes it possible to not only visually inspect it but also employ the Postman platform to make sure rules and policies have been applied appropriately.

    That capability also makes it possible to onboard new users to an API project in addition to eliminating the need to manage updates to the desktop application, added Lane.

    Now that more organizations have adopted an API-first approach to building and deploying applications in the age of microservices, those APIs need to be managed as discrete projects. Postman provides a means of achieving that goal now through a browser interface that can be accessed more easily by members of DevOps teams working remotely.

    Postman claims its platform has now enabled more than 1 billion API requests, with more than 48 million API collections now running on Postman. Overall, the company claims there are now more than 11 million users from roughly a half-million organizations on its platform. As the number of microservices-based applications being deployed increases, the number of internal-facing APIs that need to be managed increases exponentially. New functionality can be added to the application without changing the API. In fact, changes to the API are likely to be more disruptive than changes to the application.

    At the same time, cybercriminals are now targeting APIs that are often misconfigured in a way that exposes potentially sensitive data via an open port. As such, security teams are starting to focus more on ensuring processes are in place that ensure APIs have been reviewed before being deployed in a production environment.

    Fresh of raising another $150 million in funding to create a $2 billion valuation, Postman has emerged as a major API management platform in a crowded field that has experienced a wave of consolidation over the last few years involving acquisitions made by Red Hat, Oracle, Salesforce and Microsoft. Each of those acquisitions was made to enhance the API management capabilities of a specific platform. However, it would appear a large percentage of organizations prefer to manage APIs independently of any specific platform.

    Clearly, APIs are core to the implementation of DevOps processes. If those APIs are flawed, the impact on DevOps process can be far-reaching. Continuously validating APIs as they are deployed and updated is not just a best practice,  but it’s also a matter of survival at a time when complex distributed application environments have never been more loosely coupled.

  • Challenges of Designing API-Driven Experiences

    Challenges of Designing API-Driven Experiences

    As SaaS and mobile application development–and cross-software integration and communication–continue to become increasingly important for organizations across all industries to remain competitive today, APIs play a critical role. In fact, APIs dominate digital experiences today, with an average of 220 new APIs published every month, representing a 30% increase over the previous four years.

    As more organizations shift to an API-first development strategy to further drive innovation, partnerships and rich end user experiences, the proliferation of demand and consumption of APIs can bring its own challenges, if not properly executed from the get-go. Amidst the ever-increasing rush to develop and publish an API to remain competitive, we see several common issues arise–and often only after the API is rolled out–that negatively impact integrations, future partnership opportunities and end user experiences. Read on for a look at these common challenges and how to address them before you hit publish on your next API.

    Infrastructure Costs

    One of the biggest challenges when it comes to API design is the ability to manage costs, in particular infrastructure costs. From managing multiple gateway servers and instances, to building an entire API management program from the ground up, we see many common infrastructure approaches that organizations deploy today when designing APIs that lead to unnecessary and often unwieldy expenses.

    Critical to avoiding spiraling infrastructure costs is leveraging a single gateway to deploy, govern, secure and deliver global API traffic across various data centers. By using this approach, organizations can reduce man-hours spent managing and synchronizing servers; increase reliability by eliminating the need to manage multiple gateway instances; govern APIs without needing to build and maintain a separate API management platform; and eliminate regional gateway replication.

    By considering an API gateway at the onset of a design project to ensure API centric operations (such as authentication, authorization and throttling) are happening at the edge, organizations can proactively control costs and avoid financial surprises.

    Scale

    To a developer, there’s nothing worse than having a wildly successful API that can’t scale to meet the demand.

    Addressing scalability early on in the process can help define both early adoption, future success and the lifespan of the API. However, organizations commonly overlook the need to manage scalability in several ways–including planning for surges in API traffic and managing the volume of API requests consumers can make–which can lead to damaged user experience.

    Developers should look to a combination of load testing, authentication, throttling, quota management and API caching at the edge to improve and predict traffic, while preventing infrastructure from being overwhelmed by requests to ensure availability and reliability for consumers. Building in load testing early on can help a developer determine the traffic volume the application can withstand when surges in requests occur.

    Quota management can help enforce business service level agreements and limit the number of API requests that a partner is allowed to make. When it comes to API caching, some good places to start include: any resource accessible via HTTP GET, static data, immutable responses, infrequently altered or predictable responses and frequently requested data.

    Security

    While the commercial value of API development is clear, the reality is APIs can come with great cybersecurity risk. For one, APIs can provide a glimpse into the back-end of an application implementation and even the database it is connected to, providing hackers with new avenues for attacks. As the communication bridge between multiple applications, APIs left unprotected can also increase an organization’s attack surface and expose it to downtime and malicious attacks, including unintended misuse by legitimate users.

    Critical to protecting APIs is incorporating security at the infrastructure level with a multi-pronged approach that includes leveraging an API gateway to easily validate, authorize and control the access of legitimate API consumers (and block legitimate users), and endpoint protection against malicious traffic. Additionally, rate limiting–which puts caps on the number of requests per minute or second that API consumers can make–can prevent adversaries from overloading origin in an attempt to bring down API infrastructure in the form of a DDoS attack.

    Organizational Decentralization

    As speed and agility increase exponentially and API development evolves from microservices to nanoservices with individual team members owning what feels like ever-shrinking components, it’s easy to see how teams can organizationally splinter. While autonomy is important to making the development process flow quickly, decentralization is creating major governance and communication issues within many organizations today.

    This lack of governance creates its own challenges related to configuration visibility, changes and duplications across implementations which could lead to wasted resources, data leakage and more. Organizations should consider a single gateway that can expose APIs across multiple, distinct implementations but also provide suitable isolation so as to not affect the performance of other APIs when requests occur, while also providing separate control pages and the ability to set up distinct authorization and access rules for each API. The idea is the APIs are managed in the same decentralized way that is required to meet speed and agility requirements under a centralized infrastructure and governance policies.

    Strategic API design is not a simple task, but it is absolutely critical to organizations that desire long-term API success. By seeking cost-effective infrastructure, tapping tools to ensure scalability, building security protocols, practices and policies, and taking steps to more centrally manage the API design process early on will ensure longevity for the API, driving strong partnerships and, ultimately, providing better user experiences.

     

    This article was co-authored by Anthony Larkin, director of product marketing at Akamai. Anthony leads the go-to-market strategy for Akamai’s performance product portfolio. For more than 11 years at Akamai he has been dedicated to helping businesses remove barriers and unlock their potential to better engage users through web and mobile applications.

    — Sid Phadkar

  • Why Integration Must Be Hybrid

    Why Integration Must Be Hybrid

    Integration is the central piece that ties your enterprise together. The need to integrate has only increased with the proliferation of SaaS applications, IoT sensors and devices, B2B e-commerce supply chains and mobile app development in addition to the vast number of applications that still reside in on-premises data centers which also host mainframe, legacy and custom applications. One of the biggest drivers today is digital transformation—organizations are rapidly trying to become digital enterprises in order to differentiate themselves from the competition and develop new business models to create new revenue channels.

    In the meantime, APIs are becoming the standard way of connecting applications, data and devices, and providing services to partners and end users. The API economy is the term used for creating new products as part of an organization’s transformation journey. APIs are also rapidly becoming a new channel for B2B partner interaction. Since APIs and integration are two sides of the same coin, this allows organizations to quickly realize the value of integration to unlock data from different silos, drive consumption of this data and leverage business benefits for the application of this data in various end user and consumer apps.

    It’s a Hybrid World

    The most common understanding of hybrid is that organizations have applications both on-premises as well as the cloud. Pure cloud integration is a fantasy since most organizations will have applications and systems that have to stay on-premises for a variety of reasons: the cost and risk of redeveloping core capabilities (eg. An ERP application that has been heavily customized for the business or a core banking application that is mission critical), the lack of critical services in the cloud, concerns about security and sunk cost of on-premises applications and data sources (mainframe and legacy applications that just work and will be extremely expensive to rip and replace with a SaaS application).

    However, the definition of hybrid is much broader and according to Gartner, such a strategy must address a set of challenges that go across personas, domains, endpoints and deployment models. Lets explain each of these in more detail:

    • Personas: Integration development today is not just the purview of IT specialists but extends into ad-hoc developers and business users (often called citizen integrators) who despite not being technical cannot wait for IT to develop integrations for them.
    • Domains: Integration has always been a continuum supporting a variety of use cases that go across applications, B2B partners, data (for the purpose of doing analytics) and process (to support automation and orchestration).
    • Endpoints: The sources and targets for integration started originally with on-premises, then expanded to the cloud along with the need for mobile connectivity and more recently has exploded with IoT with the proliferation of sensors and devices.
    • Deployment Models: This covers the more traditional definition of hybrid and includes on-premises, cloud and hybrid (cloud and on-premises) but is now expanding into a new style called embedded, using a headless deployment model which is required for SaaS applications that need underlying integrations to satisfy their end user requirements.

    The Hybrid Integration Platform

    A true hybrid integration architecture makes integration available across all environments, for all the personas, overlapping all the domains and connecting all the endpoints. This will span on-premises ERM, CRM and mainframe, cloud and SaaS environments, edge devices and sensors and B2B partners–we call this integrating edge to app.

    A key architectural principle in hybrid integration is to allow integrations to be done close to where the applications are in order to reduce latency and increase performance. From an application connectivity perspective, hybrid requires applications hosted on-premises, public cloud and private cloud to all be connected seamlessly so an integration developer does not worry about where they are and only needs to easily wire together the mapping to move data from one application to another.

    From a B2B perspective, hybrid requires data from B2B partners (customer and suppliers) to easily be identified, exchanged and processed within the integration layer. From an IoT perspective, hybrid requires that data from sensors and devices either directly or aggregated after analysis is integrated with back-end applications. Finally, all the integrations need to be exposed via an API layer for internal business units and external customers to consume the data in a secure and privileged way through mobile or custom apps.

    These requirements have created the need for a hybrid integration platform (HIP) that supports hybrid deployment across on-premises and cloud, democratizes integration development across IT and business users, is versatile enough to support all integration patterns and is extensive enough to integrate everything.

    Components of a HIP

    A lot of organizations originally solved the integration problem by adopting enterprise service bus (ESBs) and integration platforms. These are still widely in use and in many cases powering mission critical applications for organizations. These integration products solved the application to application (A2A) problem and in conjunction with message oriented middleware (MOM) provided capabilities to decouple publishers and subscribers and allow new applications to be easily added.

    However, ESBs are undergoing an evolution since they are considered a monolith and organizations are seeing the need to deploy light weight integrations as microservices. This has led to the emergence of a microservices runtime (MSR) that can be scaled very quickly to meet the needs of the business. MSRs are typically deployed using Docker containers and scale using a container orchestration standard such as Kubernetes and are subsequently monitored using open source tools, such as Prometheus.

    A big requirement for deploying microservices is the need for DevOps which provides the discipline to manage the dev/build/test/deploy process using automated techniques. Many of these organizations also adopted B2B gateways to extend the reach of integration to their trading partners for the purpose of exchanging e-commerce documents such as purchase orders and invoices via standards, such as EDI and XML.

    A lot of transactions still occur using files leading to the need for a managed file transfer (MFT) offering to securely manage file interactions. These in conjunction with adapters and connectors provide reach to a wide variety of applications and data sources. A newer component is API management, which allows existing integrations to be exposed as APIs for access to a variety of consumers while providing policy based enforcement and security of the underlying services. APIs needs end-to-end lifecycle management and governance to control the creation to retirement process and also drives the need for asset management to track dependencies and relationships and provide impact analysis.

    IoT integration is becoming a common use case. An IoT platform along with edge capabilities will allow data from sensors and devices to be easily integrated and analyzed for the purpose of moving to other systems or to take action. In-memory is a key embedded capability for ensuring high performance and scale.

    Finally, iPaaS is a pure cloud offering which enables integration in the cloud but in conjunction with an on-premises integration platform, or agent, it can ensure data is seamlessly exchanged between cloud and on-premises with end-to-end visibility and monitoring across the different environments. Here is a quick summary of the components:

    • ESB or integration platform (with broad connectivity capabilities)
    • Microservices and DevOps
    • API management and governance
    • B2B and MFT
    • Messaging
    • In-memory
    • IoT platform
    • iPaaS

    A Practical Approach to Hybrid Integration

    A HIP can be extremely powerful but involves a number of components which can also make it overwhelming to tackle all at once. A practical approach is to add components (using a building blocks approach) as the need arises to address various integration use cases.

    Most customers already have an ESB or integration platform, so that is usually the starting point for integrating on-premises applications. Once an organization starts adopting SaaS apps then there is a decision to be made whether to simply do cloud integration using the on-premises ESB or to start adopting an iPaaS. A critical factor in this decision process is to follow the general guideline and allow integrations to be done close to the applications. The decision matrix for such a scenario can be described as follows:

    • If the use case is mostly cloud to on-premises and most of the applications are still on-premises (80/20 rule) then this can be done within the ESB by adding cloud connectivity options.
    • If the use case involves a number of cloud-to-cloud integration needs then it may be worthwhile to invest in an iPaaS for efficiency reasons (there is no need to go from cloud to ground and then back to cloud).
    • If the application mix gets switched to mostly being SaaS apps then an iPaaS should be adopted since most integrations will be done in the cloud anyway.

    Once you bring together an ESB and iPaaS you are in a great position to start leveraging API management and add B2B capabilities to round out most of your integration needs and start deriving value from these integrations.

    Business Benefits

    Hybrid integration provides a way for customers to move ahead with critical projects and strategic transformation by exploring more flexible solutions. Not long ago, integration had been painted as a back-office capability with little direct impact on the business. However, integration needs have moved to the front-office and addressing these requirements are key for successful digital transformation initiatives.

    The business benefits are as follows:

    • Organizations are seeing more demand from customers for a better customer experience. Customers have many choices, the best experience wins the business.
    • Businesses need to react faster to disruptive changes by increasing the delivery cadence and deploy integrations faster.
    • The technology world is experiencing many disruptions. Businesses need flexible architectures to disrupt instead of being disrupted. They need to be able to implement new projects, scale rapidly to take advantage of short-term opportunities and try new fast-fail ideas to leapfrog the competition.
    • Gain benefits of the cloud. There is constant pressure to reduce TCO, use an opex model instead of capex and be able to access and leverage all data available from new IoT and SaaS sources.
    • Democratize integrations. IT departments are at breaking point and can no longer meet the needs of the business which means business analysts need to develop integrations on their own in a self-service way, which allows more people to contribute to the company’s goals.

    Real World Customer Case Studies

    Retail Pharmacy: Our first case study is a large global enterprise with more than half a million employees and hundreds of distribution centers. One of the biggest changes in the company’s history was the merger of two iconic brands. The company needed a global platform strategy to stay competitive and remain ahead of Amazon and other digital disruptors in the retail space, and coined their version of a HIP: Independent Federation Layer.

    Software AG provided a solution to standardize global operations—a “cook once, serve many” solution. Rather than buy more tools, the company would invest in a unified set of software capabilities using webMethods HIP to integrate all systems. They set up a 200-person integration center of excellence. All retail stores are being migrated onto webMethods for a mission-critical retail transformation program to simplify and standardize sales and inventory processing. The next phase will be standardized processes for warehouse and order management, replenishment management, financial management and more. With webMethods HIP, this retail pharmacy has the agility to react to market challenges and opportunities.

    Transport Company: Our second case study is a multinational public transport company that operates bus, train and tram services in Europe, Canada and the U.S. They needed to lift and shift services to the cloud in order to create a new business offering that could scale easily. They refactored their webMethods integration code base and moved key services to the webMethods integration cloud to enable customers to purchase tickets using mobile and web platforms via third-party websites. They provided secure access to on-premises databases by exposing APIs to the cloud using webMethods API gateway. In the process, they created a reusable ticketing platform that they’re offering to other transportation providers. They now have a new line of business.

    — Subhash Ramachandran

  • Axway Extends API Management Reach

    Axway Extends API Management Reach

    With application programming interfaces playing a larger role in the DevOps environment, Axway has added to its API toolkit for developers. The company expanded its AMPLIFY API management platform to include a visual tool for composing and integrating APIs and enhanced its API Manager tool so that it now can be invoked to scale resources up or down as needed. In addition, Axway unveiled API Central Service that makes it easier to integrate APIs across an ecosystem of organizations that make their services available via the AMPLIFY marketplace.

    Other new capabilities include an open-source MQTT connector for building internet of things (IoT) applications, a connector to the Microsoft Azure Cloud and tighter integration between Axway’s MFT file transfer software and the Syncplicity file synchronization software it acquired earlier this year.

    Axway also merged the Titantium SDK for building native mobile applications using JavaScript into the Core SDK for developers and is previewing the Query Builder Analytics tool, which enables users to do ad hoc query of mobile statistics. The Titanium code is based on technology developed by Appcelerator, which Axway acquired early last year.

    The announcements were made at the recent Axway SPARK 2017 conference. Vince Padua, vice president of platform and products for Axway, says these and other enhancements to the Axway API portfolio are intended to make it easier for organizations to craft microservices spanning multiple platforms. In some instances, those microservices will be crafted by developers, but Axway believes more advanced users will be stitching together microservices on their own using visual tools, says Padua.

    While most early use of APIs was driven by web-based applications facing external users, using APIs to interconnect services inside the enterprise now dwarfs the number of instances of published external APIs, he notes. The proliferation of those APIs has created a significant DevOps challenge for organizations and requires an API management platform to streamline the management of various back-end service. At the same time, Padua notes that DevOps teams working in collaboration with line-of-business units have taken over much of the API management, which previously was left up to individual developers. Now, IT organizations are trying to strike a balance between maintaining control and developer productivity, he says.

    The shift in API management responsibilities has also driven a wave of mergers and acquisitions over the last two years as API management has become embedded in mainstream application development platforms. But there’s also a fierce debate over the degree to which legacy API management platforms can support microservices. As microservices based on emerging technologies such as containers or more mature Java programming tools continue to evolve, API management is becoming more critically important. Each microservice generates its own API that needs to be maintained as each microservice gets continually updated. That requires an API management platform that can support all levels of API call activity that once would have been assumed to be unimaginable.

    — Mike Vizard