Tag: modern applications

  • How to Prevent Zombie APIs

    How to Prevent Zombie APIs

    There is a growing reliance on web APIs. A full 92% of organizations plan to increase their investments into APIs over the next year, the 2023 Postman State of the API Report found. APIs are powering everything from internal microservices strategies to partner strategies and fully-fledged products.

    Yet, this newfound API sprawl has consequences; a looming threat that might rise from the grave to haunt you …

    I’m talking, of course, about zombie APIs. Zombie APIs are endpoints that are no longer maintained yet are still active. They may be unused endpoints, old features never officially deprecated or forgotten development or testing environments. Regardless, zombie APIs are a type of technical debt that could pose a legitimate threat if left to rot.

    The sudden growth of APIs, along with immature inventory management and a lack of internal communication, can contribute to creating countless shadowy endpoints. Below, I’ll review some of the issues that zombie APIs might cause and consider some actionable steps DevOps teams can take to prevent them.

    Understanding Zombie API Risks

    Forgotten APIs can cause a number of issues. For example, suppose an insecure testing environment was accidentally left exposed to the public. An attacker using automated endpoint scanning could find authentication and authorization holes to exploit, potentially exfiltrating sensitive data.

    It’s also a bad idea to let any piece of technology go unmaintained, given the daily influx of new exploits and vulnerabilities. This point is accentuated by the connected nature of APIs, which may plug into a vast network of databases and allow both read and write privileges to the backend infrastructure.

    Insecure APIs can cause a business both economic harm and damage reputation. Just consider the high number of API-related data breaches in recent years, which occurred at companies like Optus, Dropbox, Twitter and Zendesk. It’s even estimated that somewhere between $12 billion to $23 billion was lost in 2022 due to compromises in web APIs.

    Zombie APIs are an easy target for these sorts of cyberattacks because they tend to have insecure designs that are easier to crack. They also tend to use immature monitoring systems, which means hackers can act under the radar more easily than with actively used endpoints.

    Tips to Avoid Zombie APIs

    Keep an Active Inventory of APIs

    As they say, you can’t secure what you don’t know about. In fact, OWASP lists improper inventory management as one of the ten top API security risks. Therefore, proper inventory management is a good first step to avoiding zombie APIs. Scan your service portfolio for forgotten endpoints, catalog them and update the catalog regularly.

    It’s also a good practice to thoroughly document your services to aid in internal discoverability. If possible, adopting specification-first practices can also ensure the documentation and implementation are based on a single source of truth, thus avoiding mismatched versions and documentation drift.

    “The key to preventing zombie APIs is visibility,” said Jeremy Snyder, CEO, FireTail. “Many organizations lack ongoing, up-to-date inventories of all the APIs that are running in either production or staging environments.”

    Use Proper Versioning and Life Cycle Strategies

    Another method to reduce the chance of zombie APIs is to adopt mature API life cycle management practices. The chosen API versioning strategy should be clearly defined and codified. And versioning changes should be clearly communicated to all stakeholders.

    On that note, it’s just as important to plan out a sunsetting, retirement and deprecation strategy from the onset. Then, remove deprecated services when they have reached their end of life! This is not a glamorous task, but it should be baked into someone’s assignments.

    “Good cyber hygiene is an often overlooked strategy,” explained Snyder. “A second requirement is to have a policy around versioning and how many versions of an API can exist at any given moment. For instance, that policy may state that only the latest version can exist or the latest plus the previous. However, this relies on having visibility in order to maintain compliance.”

    Other areas of proper life cycle management include employing governance around development and deployment and utilizing automated testing in continuous deployment (CI/CD) pipelines. Both of these could help set policies and security requirements, which should also bolster API safety.

    Share Knowledge of Internal and Third-Party Services

    Tribal knowledge is the antithesis of DevOps culture. As such, it’s a good practice to de-silo and share internal knowledge of API-powered services to limit the propagation of rogue endpoints. This could be done through private network repositories or even an internal Postman Collection or Workspace.

    “Internal processes like organizational coordination and communication are critical,” added Snyder. “If teams don’t tell each other about new APIs being developed and launched into production, security teams can’t assess or audit them. Conversely, if app teams don’t know about network infrastructure resources designed to cover various API security aspects, developers don’t know what security controls need to be baked into the API code.”

    Visibility is Required to Avoid Zombie APIs

    A recent CISO survey revealed that API security was the number one concern for CISOs. For many reasons, web APIs are still low-hanging fruit for attackers. This is underscored by the 400% spike in malicious API attack traffic recorded by Salt Labs earlier this year.

    Many of these attacks, we’ve also learned, are occurring over authenticated APIs. This shows that greater accountability and visibility are required to begin to stem the tide of API-related security incidents.

    To reach an adequate level of visibility, leaders should consider adopting tooling that helps determine if there are outdated APIs or APIs with multiple live versions, said Snyder. “These tools should help display what security live APIs have and provide recommendations around how to fix them, if not providing the necessary code or modules to secure the APIs directly.”

  • The New Norm for Modern Apps: Security Observability

    The New Norm for Modern Apps: Security Observability

    Observability has burst onto the scene across all types of operational and security-focused activities. Its need is being driven by increased demands for businesses to be more responsive to changes and more proactive when dealing with potential problems. In particular, security observability holds the promise to help reduce the time to detect a cyberattack. And going a step further, it can help detect security vulnerabilities before an attack even occurs.

    What is security observability?

    In the past, most operations relied on visibility into their systems and applications to manage security. Visibility would be achieved via monitoring applications, systems, and their logs. Monitoring solutions would collect and analyze information to detect suspicious behavior or unauthorized system changes. And having defined which types of behavior should trigger alerts, solutions would take action as needed.

    Perhaps more to the point, monitoring lets you detect a known set of problems or conditions. Thus, it is essential for alerting and analyzing long-term trends. Simply put, monitoring lets users know how their systems and applications are functioning and being used.

    However, the problem with security monitoring of complex distributed applications is that it is a passive approach to highly dynamic operations. And it often does not provide any insights into root cause problems or anomalies that might be precursors to problems in the making.

    Security observability expands on monitoring by enabling correlation and inspection of the data to provide much deeper insights. Observability typically requires logs, metrics, and deep tracing. All data is used for modeling and analytics. Companies can mine the data, look for patterns, use artificial intelligence and machine learning to remediate problems and proactively defend against problems.

    Why security observability is needed now

    The need for security observability is growing due to the way applications are developed.

    The general industry trend is a move to cloud-native architectures based on microservices. Such an approach brings many benefits. Applications can scale quickly. Companies can easily change or update a small aspect of a larger application without impacting the rest of the app. And businesses can make use of new technologies. For example, a business can add a new front-end or implement more sophisticated AI or machine learning modeling.

    Such a development approach can introduce potential blind spots and complexity. In particular, modern applications built using cloud-native architectures and microservices introduce new challenges. The loose coupling of such applications, their distributed nature, and increased complexity makes it harder to understand vulnerabilities. As a result, traditional cybersecurity approaches break down. They miss the interactions and inter-dependencies of the many elements that comprise modern applications.

    Furthermore, traditional security monitoring solutions can be limited. For example, visibility often must be split across tools resulting in insecure platforms or situations where relevant data is not even used in monitoring.

    Adding to the security challenges is the widescale embracement of low-code/no-code development methodologies. Such technologies open development up to business units and citizen developers. These groups can build sophisticated applications, often with little oversight or guidance. As a result, apps may be created by assembling building blocks that may be outdated, unpatched, or simply have vulnerabilities.

    DevSecOps needs security observability

    With all these trends, it is not surprising that businesses are exploring new approaches to security. Just as quality assurance was melded into development processes using automated testing, security ops are being meshed with development. Hence, the rapid implementation of DevSecOps methodologies in organizations today.

    While security issues can grow when using cloud-native development methodologies, there is a lesser-known security benefit. Applications and services constructed using APIs and microservices have an advantage over other systems because developers and security staff can more easily observe what’s happening inside them. In addition, businesses can add links to the APIs and microservices and use tools to collect the details of such operations.

    Given this wealth of data, security observability expands on monitoring. Rather than just detecting vulnerabilities and incidents that raise compliance issues in the development process, DevSecOps using security observability, seeks to identify issues and automatically take corrective actions.

    Moving forward, businesses should expect more adoption of DevSecOps and more automation based on security observability. This will only serve to strengthen security and compliance across systems and applications.

  • Bringing z/OS Systems into Today’s Modern Application Development

    Bringing z/OS Systems into Today’s Modern Application Development

    Bringing z/OS Systems into Today’s Modern App Development

    Modern application development is all about automation, addressing the concerns of developers and the operational side of the house, and supporting the common CI/CD (continuous integration/continuous delivery) approach needed to keep apps up to date.

    Developers typically make use of a variety of open source and other tools to aid in their work. While most of these tools were created for traditional server or cloud-native environments. Some of the most powerful tools are available to run natively in z/OS environments.

    One example is Ansible, an open source community project for app deployment, configuration management and orchestration. It is available for z/OS.

    The IT automation engine works by connecting to your nodes and pushing out small programs called “Ansible modules” to them. These programs are written to be resource models of the desired state of the system. Ansible then executes these modules and removes them when finished.

    Simply put, Ansible allows developers to automate z/OS applications. With Ansible, developers can use existing JCL, REXX and z/OSMF to automate z/OS-based software. This lets the developer take a common approach to hybrid applications and integrate z/OS into an enterprise automation strategy consistently.

    A great advantage is that developers can work in Ansible and Python. These are readily available skills in the marketplace and they can be applied to z/OS applications.

    Another common tool is Jenkins, an open source, server-based tool that builds and tests software continuously, supporting CI/CD. Jenkins integrates with a broad range of source code management systems such as Git and Subversion. Jenkins’ capabilities can be extended to different platforms and systems using plugins. One such z/OS connector is the IBM z/OS Connector, a plugin for connection of Jenkins CI to IBM z/OS, including integration of IBM SCLM as SCM.

    With tools such as these, developers can build a CI/CD pipeline that, for example, calls open source scripting that you have running on Z. It might move some files from A to B, or it might call a compiler. It helps someone familiar with these tools carry out these tasks and incorporate them into their processes. If they are running on the z/OS platform, developers can interface and make calls right on the platform. For example, an app may take advantage of existing mainframe Rex scripting without having to rewrite it.

    Accounting for Cultural Differences

    In today’s always-on, instant-access-to-everything world, employees and customers demand frequent updates and new features to existing applications and services, as well as new and better offerings all the time. Meeting these demands is a competitive differentiator.

    Bringing commonly used open source development, workflow automation and CI/CD tools to the z/OS environment helps businesses speed the development of new applications to meet changing demands of their customers. It allows a business to leverage the data and applications on its z/OS systems and create innovative new applications.

    It’s also about providing the tooling to DevOps that they are familiar with and providing tools to build z/OS applications in the same way they build other applications. Many are building their applications off-platform with the same tools. Having tools that run on z/OS is helpful. It makes it easier. Businesses can make fewer application changes. So, there’s less risk.

    If a business has 20, 30 or even 40 years’ of legacy applications running on z/OS, those applications have proven to be secure, stable and reliable. An open source application development tool for z/OS offers a way to retain those qualities and modernize application development for building pipelines and instituting DevOps principles and practices that are well-known to the new generation of developers.

  • The Battle for the Frontend: Rethinking APM

    The Battle for the Frontend: Rethinking APM

    With the battle for the backend between data centers and cloud largely over, application development is the next frontier ripe for innovation. Studies project mobile app revenue at $935.2 billion by 2023 and Internet of Things (IoT) installations at $75.44 billion by 2025. It’s no surprise that software innovation is ramping up around application development in a big way. At the same time, the desktop is still a primary interface in the workplace, which has given rise to a wave of browser-based applications for enterprise collaboration and efficiency.

    Apps—both on devices and in-browser—are effectively what businesses run on. New JavaScript-based Single-Page Applications (SPAs) are replacing legacy cloud stacks, where the software runs primarily in a customer’s browser. However, this approach creates an observability gap for teams under pressure to deliver better software more quickly. Because no application is siloed, the distributed nature of modern applications adds further complexity—and increases the risk for defects—for developers, who must code faster, smarter and better to keep their businesses competitive in an application-centric world.

    The Battle for the Frontend

    IDC predicts that by 2022, more than 60% of global GDP will be digitized, with growth in every industry driven by digitally enhanced offerings, operations and relationships—and applications will be the primary method of delivery. Companies in all industries are battling for users, revenue and reputation only, but only those companies with the best software will remain contenders. To innovate continuously, ensure optimal user experiences, and ultimately, retain their competitive advantage, enterprises must ship code quickly, every day and with fewer errors.

    Application errors and crashes will dramatically spike churn rates, kill conversions and impact brand reputation and customer retention. As tolerance for faulty performance is rapidly diminishing, user experience becomes a business-critical concern that has a substantial impact on a company’s bottom line. While companies with faulty code are dealing with the fallout, those that rely on modern application performance monitoring (APM) are shipping better code more quickly and with less risk.

    These new business drivers, along with an explosion of devices—from smartphones to IoT devices—have led developers to move their code closer to customers to increase code speed and improve scale. But this shift means software is run on customer devices over which developers have no control, a reality that inevitably creates its own set of new problems to monitor. To win this battle for the frontend, development teams that create games, mobile applications, IoT and other high-performance applications need rich insights into application health in order to quickly and continuously fix software errors, all with minimal impact.

    Rethinking APM

    Unfortunately, many legacy application monitoring tools do not solve for the frontend. They are fragmented, or specialized, and missing support for JavaScript, mobile, desktop apps or embedded IoT devices—despite these platforms being a significant part of the application ecosystem. Legacy tools focus more on system behavior, offering performance metrics on availability, throughput and latency, but miss the mark when it comes to actionable insight that helps frontend developers make sense of complexity and quickly get to the root cause of errors.

    Modern applications require a holistic approach to monitoring that provides deep context and understanding about how to address issues when they arise, instead of simply displaying the symptoms of a problem. This approach should involve collecting and analyzing transactional traces, scaling search and storage to handle full traces, and building intelligent algorithms to detect and analyze issues. By moving monitoring from production environments to test environments, development teams can accelerate the software development lifecycle and reduce the impact on customers. APM must integrate with continuous delivery tools to monitor continuous health mechanisms.

    — David Cramer

  • Redis Labs Raises $60 Million Series E Financing to Bring Instant Experiences Everywhere

    Francisco Partners Leads Round; David Golob and Eran Gorev Join Redis Labs Board of Directors

    Mountain View, Calif., February 19, 2019—Redis Labs, the home of Redis and provider of Redis Enterprise, today announced it has raised $60 million in Series E financing led by a new investor, Francisco Partners, a leading global technology-focused private equity firm. The round included participation by the company’s existing investors, Goldman Sachs Private Capital Investing, Bain Capital Ventures, Viola Ventures and Dell Technologies Capital. With this investment, Francisco Partners’ Chief Investment Officer David Golob will join the Redis Labs Board of Directors and Operating Partner Eran Gorev will join as a board observer.

    Redis Labs was founded in 2011 by Ofer Bengal (CEO) and Yiftach Shoolman (CTO) around the promise of open source Redis, to deliver instant experience to modern applications at any scale, by building their datasets with native data structures and serving their applications requests directly from memory. In July 2015, Salvatore Sanfilippo, the creator of Redis, joined the company as leader of open source development. The company has now raised $146 million in total funding to-date. With these funds, the company plans to accelerate its global go-to- market execution, invest further in the enthusiastic Redis community, and continue its leadership in delivering the highest performing, most efficient database platform for modern applications.

    Redis Labs’ commercial product, Redis Enterprise, is the world’s fastest database, leveraging modern in-memory technologies such as NVMe and Persistent Memory to offer cost-effective deployment over multiple public clouds and on-premise data centers. Besides its native data structures, it features a variety of data modeling techniques, such as Streams, Graph, Document and Machine Learning, with a real-time search engine. The performance and flexibility of Redis has made it consistently one of the fastest-growing and most popular databases, including becoming the first to be launched more than one billion times on Docker Hub in 2018. Additionally, through strategic partnerships with Pivotal, Red Hat and others, the company has been able to drive further enterprise adoption and success with Redis Enterprise.

    “The impact of the Redis platform is being experienced everywhere as enterprises look to modernize or build new applications,” said Ofer Bengal, co-founder and CEO at Redis Labs. “This financing enables us to accelerate our strategy to deliver the fastest and most efficient database to the world and enable instant experiences for any modern application.”

    “Customers and partners alike tell us that Redis Labs has built the most robust and capable database solution for real-time computing across cloud, on-premise and hybrid environments,” said Matt Spetzler, partner and co-Head of Europe at Francisco Partners. “We are thrilled to be partnering with Redis Labs’ team as the company scales up globally to meet the needs of the Internet economy.”

    About Redis Labs

    Modern businesses depend on the power of real-time data. With Redis Labs, organizations deliver instant experiences in a highly reliable and scalable manner. Redis Labs is the home of
    Redis, the world’s most popular in-memory database, and commercial provider of Redis Enterprise that delivers superior performance, matchless reliability and unparalleled flexibility for high-speed transactions, recommendation engines, data ingest, fraud mitigation, real-time indexing, session management and caching.

    Redis Labs, ranked as a leader in top analyst reports on NoSQL, in-memory databases, operational databases, and database-as-a-service, is trusted by seven Fortune 10 companies, three of the four credit card issuers, three of the top five communication companies, three of the top five healthcare companies, six of the top eight technology companies, and four of the top seven retailers.
    Redis has been voted the most loved database, rated the most popular database container, and #1 cloud database.

  • PTC: Using DevOps to Transform Applications

    PTC: Using DevOps to Transform Applications

    Large companies and monolithic applications are not usually associated with DevOps. But I recently discovered that some of the best DevOps operations have come from the largest organizations and most complex applications, and PTC is the latest example.

    If you are not familiar with PTC, the company has a wide range of offerings for enterprises looking to improve their operations. Its solutions range from enterprise resource planning (ERP) to product lifecycle management (PLM) and Internet of Things (IoT) platforms as a service (PaaS). Its existing products started as on-premises solutions, and have expanded to modern solutions including IoT. But now there is a large demand to convert them to cloud-based offerings. And as PTC goes down this path, it needs to have a modern delivery chain to support it.

    Enter Cloud Services

    So PTC created its Cloud Services group to deliver cloud-based versions of its existing platforms to customers. Its cloud-based offerings are subject to all the same demands of any other modern software-as-a-service (SaaS) and PaaS offering—faster releases, more frequent updates and higher quality. After a few years of building the Cloud Services group, the company has proven it can automate and improve delivery of its traditionally on-prem platforms in the same fashion as poster-child DevOps environments.

    The journey was not easy, and it did not start with DevOps as a goal. The goal was simply to deliver solutions to its customers. DevOps was the natural outcome following the challenges it faced. Some of the challenges were common for all development operations, but others were very unique:

    1. Lots of variables: PTC has to support a wide range of customers across a wide range of deployments. PTC works with its customers on their varying needs of platform versions and also understands their varying requirements around software updates. And each potential variation has a unique snowflake configuration, which, unlike most cloud platforms that offer one production version to all users, its entire ecosystem of releases is actually a wide range of configurations and versions.
    2. Data integrity: Data is important no matter what. But PTC works with customers that have had data in their on-premises solution for a long time, and that same data needs to make it to the cloud, securely, without a single flaw. That data is part of business-critical applications and ongoing business processes.
    3. Educating users: As customers continue on their journey to the cloud, PTC is there to support the changes and enable them through customer success, continuous knowledge transfers and an endless online platform of PTC educational services.

    So not only does it face technical challenges, it also is facing huge change control and people challenges. Before any software delivery can begin, there is a specific part of the Cloud Services group that is responsible for the necessary development to SaaS-ify the existing applications and build a gold master for each release.

    The initial delivery chain was hand-to-hand combat: one solution, one customer, one release at a time. This required a lot of warm bodies to manage manual release processes and customers.

    “We could not keep on throwing people at the problem,” notes Tameem Hourani, senior director of Cloud Services. Even though PTC had established workflows via ServiceNow, each step was manual and simply not scalable. But the customers and releases were increasing at an ever-climbing rate. It simply would not be feasible to on-board new team members to handle the load.

    So it turned to good practices and tools to build a delivery chain that can take care of a brand-new deployment in less than two hours, instead of days, and upgrades in less than 10 minutes. Here is what the resulting setup looks like:

    ptc_flow

    PTC was not new to automation. The company had been leveraging ITIL practices for some time. But what ITIL lacked was exposure to the customer. So with the ITIL backbone and ServiceNow in place, PTC added pipeline-level orchestration called Foreman.

    In this configuration, both ServiceNow and Foreman essentially act as a system of record for the entire environment. ServiceNow is the system of record for all states and workflows. It also is the base entry point for any new update or deployment. And Foreman is the system of record for infrastructure and configurations across all customers and all deployments. But it also serves as the “checklist” of all the aspects of orchestration.

    To manage the infrastructure orchestration and create the templates for configurations, PTC turned to Saltstack. It originally used another tool, but the benefit of Salt is that it allows for more sequential orchestration, as opposed to another tool with a stateful-only option.

    PTC uses another unique tool called Intigua. Intigua peaked my interest because I have previously talked about the overhead in both resources and management of server agents. And because so much of what is done for headless monitoring and control of servers these days is done by agents, this actually becomes a serious management problem.

    “The only way to grow a modern delivery chain is tools. And most new tooling is driven by agents,” Hourani notes. “It is easy to forget that the new tooling and processes also come at a cost. Usually the cost is less than the benefit they provide, but it is there nonetheless. Management of agents seems simple enough, but [when] actually doing this at scale, things can easily get out of hand.”

    PTC had four-plus unique agents across a wide range of configurations. One of Intigua’s value points is the ability to “self-heal” and make sure the agents are always running correctly.

    Once it is handed an IP, Intigua installs and configures agents based on tags, policies and AWS regions. When Intigua is running, it will make sure agents are up to date, and that self-heal agents go to work should something go wrong. The ability to use Intigua as a single pane of glass in a highly variable grid of configurations is a huge benefit.

    Triggered by ServiceNow, configured by Foreman, Salt and Intigua, the infrastructure and code are deployed. But to learn and maintain all environments for the customer, PTC needed some strong production monitoring tools. It uses SumoLogic, deployed as an agent via Intigua, for log analysis. For most applications, using a single instance of Sumo is fine. But PTC has to segregate an instance per customer, and the process must be fully automated. There is some overlap with another monitoring tool, Zabbix, which has been set up to attempt self-healing upon a triggered alert.

    And then there’s Jenkins. While not used for the entire deployment process, it is used to drive testing and manage states during the testing process. Upgrades to production releases are done incrementally, and complete deployments are done full-stack.

    Services Layers Are All the Rage

    Cloud services and shared services are not necessarily new. I recently did a write-up on Wix, which has a similar approach. But the complexities associated with SaaS-ifying on-premises solutions, along with the number of variables, adds not only another layer of complexity, but also another layer of value to a repeatable delivery chain.

    I am humbled by case studies like these, because it proves that those who say their environments are too complex for DevOps have yet to see what’s possible.