devopsy.kinsta.cloud

Category: Application Performance Management/Monitoring

  • appCD Launches Platform to Securely Provision Cloud Infrastructure

    appCD Launches Platform to Securely Provision Cloud Infrastructure

    appCD this week launched a namesake platform that analyzes an application about to be deployed to automatically generate the code needed to provision the IT infrastructure required.

    Fresh from raising $6 million in seed funding, appCD CEO Sachin Aggarwal said that approach makes it possible to consistently create secure code for provisioning IT infrastructure versus requiring developers with little to no cybersecurity to manually write it.

    Instead, appCD reviews Python or Java code to understand intent, identifies dependencies and then infers application programming interface (API), service configuration, ingress/egress and other environment variables. Terraform code or Helm charts are then automatically generated based on the static analysis of the code that has been reinforced using artificial intelligence (AI) learning techniques.

    That approach also makes it simpler for platform engineering teams to centrally manage the provisioning of IT infrastructure using a set of pre-defined policies that are automatically implemented each time an application is deployed to ensure security by default, he added.

    DevOps teams can also use appCD to visualize the deployment environment before and after an application is deployed, said Aggarwal.

    While infrastructure-as-code (IaC) tools have accelerated the rate at which cloud infrastructure can be provisioned, they are also often the root cause of the security issues that plague cloud computing environments. Developers often misconfigure cloud infrastructure by, for example, inadvertently leaving ports open. The appCD platform reduces the potential such mistakes will be made, noted Aggarwal.

    That capability also eliminates the need to attach cybersecurity specialists to a DevOps team because issues are automatically resolved by the guardrails enforced by appCD, he added.

    In theory, at least, there may soon come a day when AI routinely improves the quality of the code being used to both build and deploy applications. The more the code used to train a large language model (LLM) is vetted, the more reliable the output becomes. In the meantime, platforms such as appCD are combining static analysis with AI reinforcement learning to achieve that same goal in a use case that is specific to generating code to provision IT infrastructure.

    Each DevOps team will naturally need to determine the best path forward, but as regulations pertaining to securing software supply chains become more stringent, it’s only a matter of time before organizations will be required to embrace DevSecOps workflows. In the meantime, DevOps teams should assume that given the probability cloud infrastructure has been misconfigured, it’s all but certain there will be multiple cloud security incidents. In fact, cybercriminals are actively scanning for misconfigurations that they have already created playbooks to specifically exploit in a matter of minutes.

    In the absence of being able to prevent all those issues from arising, the immediate priority becomes remediating as many issues as possible before cybercriminals exploit those vulnerabilities. However, it’s also important to draw a proverbial line in the sand to ensure that all cloud infrastructure being provisioned in the future is much more secure than it has tended to be thus far.

  • Copado Streamlines Custom Salesforce App Testing

    Copado Streamlines Custom Salesforce App Testing

    Copado today added a testing tool to its platform for managing DevOps workflows in Salesforce application environments that automatically creates scripts based on manual tests being conducted.

    Announced at the TrailblazerDX conference hosted by Salesforce, Copado Explorer enables DevOps teams to integrate those scripts into testing processes that can be repeated as required.

    David Brooks, senior vice president and lead evangelist for Copado, said the goal is to make it simpler for DevOps teams to add those tests to the regression tests that are incorporated into pipelines created using the Copado continuous integration/continuous delivery (CI/CD) platform.

    In addition, markup and annotation tools enhance collaboration by allowing non-technical subject matter experts to provide detailed feedback and insights to quality assurance and development teams as part of an effort to streamline communications between product owners, testers and developers. Testers without a technical background can use that capability to contribute to the creation and deployment of effective automated tests. That capability makes it possible to shift testing further left to incorporate the business teams that better understand how an application should function, said Brooks.

    The challenge is far too many organizations still overlook the importance of testing, noted Brooks. There is a tendency when using low-code tools to assume that not as much testing is required because the tools that generate code create a false sense of instant quality despite flaws that might not manifest themselves until an application is deployed, noted Brooks.

    Rather than constantly updating applications, more testing should improve the quality of the applications being built sooner. That’s critical because most organizations don’t have the time and resources needed to continuously iterate applications until they eventually arrive at what the business requires, said Brooks.

    It’s still early days so far as the level of DevOps maturity that organizations using the Salesforce platform to build and deploy custom applications have thus far attained, noted Brooks. However, just about every organization that licenses the software-as-a-service (SaaS) platform from Salesforce customizes those applications to better align them with their internal business processes, he added.

    Each organization will naturally need to determine to what degree they are prepared to deploy customer applications, but it’s clear it’s becoming simpler to build them. Late last year, Copado, for example, added generative artificial intelligence (AI) tools to make it easier to manage the DevOps workflows used to build these applications, which means the speed at which these apps are built and deployed is about to exponentially increase.

    Salesforce, of course, is only one of many SaaS platforms that organizations rely on to build and deploy custom applications. However, as one of the most dominant platforms, the pace at which custom applications are being built continues to increase, with a larger percentage of these applications being built by professional developers familiar with DevOps best practices.

    Eventually, those DevOps workflows will be extended to every kind of developer, but for now, most organizations building applications are looking for more consistent outcomes regardless of what type of tool is used to build them.

  • 7 API Observability Anti-Patterns to Avoid

    7 API Observability Anti-Patterns to Avoid

    Observability has become a buzzword, and the API world is slowly catching up. But be careful when looking for content about API observability- there are many outdated best practices and irrelevant pieces of content.

    I was recently invited as a guest to Stephen Townshend’s SlightReliability podcast, and his brilliant blog post on “Bad Observability” inspired me to start ranting about the topic.

    Here is a follow-up to our discussion: The observability anti-pattern to avoid when working with APIs. Thanks, Stephen, for the inspiration!

    Anti-Pattern: You Forgot Your Users

    It’s easy to lose sight of the fact that APIs are built for people to use. APIs are not just a technical interface; they are a bridge that connects digital experiences and enables people to accomplish tasks efficiently.

    There are two different kinds of users for your API:

    • The developers who are integrating your API into their system.
    • Your customers’ users, who will ultimately interact with the applications or services that rely on your API.

    If you can’t understand how they interact with your APIs, you’re missing half the picture.

    When developers integrate with your APIs, how long does it take them to make their first successful API call? What are typical errors slowing them down? Observability plays an essential role in surfacing insights to improve developer experience.

    Once your customers’ applications or services relying on your APIs are in production, can you track how your releases impacted their requests? Which version of your APIs are they using? What are the most popular endpoints? Are all customers getting the same performance? Same error rates?

    Anti-Pattern: Relying Only on API Monitoring

    API monitoring typically involves sending automated test requests to API endpoints and verifying the responses. It helps you keep track of the health of your APIs even when there is no customer activity at all, and can be used to report uptime.

    But API monitoring is nothing more than black box testing; you don’t see what is happening inside. You just know that your requests reach the API and receive responses, but you can only create and maintain tests for some possible use cases. You will miss something.

    Anti-Pattern: Using API Access Logs for Troubleshooting

    API access logs are records of requests and responses made to an API, capturing details such as the requestor’s IP address, HTTP method, requested endpoint, status codes and other relevant information for monitoring, debugging and security purposes.

    Using access logs to troubleshoot API issues is like searching for a needle in your microservices haystack. One of our users had a performance issue with their APIs; they spent days searching for the root cause using access logs. Is it the gateway? Does it need more CPU? Is it a configuration issue? They enabled distributed tracing and, within seconds, they could see what was happening: All the time was spent in the authentication server issuing JWT tokens.

    To really understand what is happening in your system, you need to be able to track a single user request from the API gateway to the upstream services and their dependencies for effective troubleshooting.

    As Martin Thwaites puts it: “Logs are a poor man’s observability tool.” You need distributed tracing.

    Anti-Pattern: Different Teams, Different Tools, Different Data

    Different teams have different needs. The tool that works for your DevOps team might not fit your product team best. By forcing everyone to use the same tool, you’re not promoting unity but inefficiency.

    However, while promoting tool diversity might be the right move for your organization, there should be some coordination and integration between different tools and data sources. Open standards like OpenTelemetry help ensure that information can be shared and leveraged across teams and tools.

    The DevOps and the product teams should rely on the same error rate when assessing the performance and reliability of the newest API product launched.

    Anti-Pattern: One Size Fits All

    Different API architecture styles have different needs when it comes to observability.

    REST APIs might be today’s default, but newer styles like GraphQL and gRPC are gaining popularity. Each of these styles has distinct characteristics and requirements regarding observability. Just think about GraphQL’s way of handling errors, which differs significantly from REST APIs. Web Socket and event-driven APIs have their own unique observability challenges, as well.

    Different API architectures have different needs, and your observability strategy needs to reflect that to improve the overall reliability of your application.

    Anti-Pattern: Observability for Production Only

    Why aren’t you using observability in pre-production?

    By using distributed tracing during the API development life cycles, developers can trace the flow of requests and responses across different services, providing a comprehensive view of how the API behaves under various scenarios.

    With observability, they can identify performance bottlenecks and even detect architectural problems like N+1 problems in GraphQL queries.

    For teams leveraging APIOps practices, observability data can be used as an additional validation before promoting an API to the production stage.

    Anti-Pattern: Starting the Trace at the API Gateway is Overrated

    Tutorials about observability usually start the instrumentation process at a microservice level. It is great to have detailed insights into your microservices. But a lot will happen at the API gateway level when dealing with APIs.

    You want to capture all your user transactions, including those that never reach your microservices because of rate-limiting rules, an authentication problem or a caching mechanism.

    Starting the trace at the API gateway gives you a clear entrance point and a complete picture of the journey of all your users. That’s why it’s important to use a modern API gateway with built-in support for OpenTelemetry.

    So there you have it: Seven API observability pitfalls to avoid. Keep your APIs observable and your users happy!

    To hear more about cloud-native topics, join the Cloud Native Computing Foundation, Techstrong Group and the entire cloud-native community in Paris, France at KubeCon+CloudNativeCon EU 2024 – March 19-22, 2024.

  • Low Code: The Key to Accelerating Custom App Dev for Moderate Developers

    Low Code: The Key to Accelerating Custom App Dev for Moderate Developers

    Custom app development has become a crucial skill in an increasingly digitized world. Many moderate developers face challenges while creating unique and functional applications. This is where low-code platforms come into play, offering a solution to accelerate app development and compensate for those shortcomings.

    Who are Moderate Developers?

    While every developer is constantly evolving and growing in experience, like any professional, they can be put in three baskets. The first is entry-level developers learning on the job who may assist with debugging, essential documentation, code repository maintenance and some simple feature requests. At the other end are very experienced developers who lead projects, architect solutions, oversee the development of applications and take full ownership of delivering killer features in a new application.

    Moderate developers are those in the middle, representing a large swath of professionals with the foundational expertise to implement most app designs and feature requests. With more experience than beginners but yet to be qualified to serve as seniors on a team, they have a shared set of challenges.

    Understanding the Challenges of Moderate Developers

    Moderate developers often need help with complex application development demands and skill limitations. They are faced with creating high-quality, functional and unique applications, which can be daunting in light of the rapid pace of technological change and user expectations.

    These moderately skilled developers often face common challenges such as:
    ● Limited Expertise: Cultivating an in-depth knowledge of programming languages and development environments is an ongoing process.
    ● Time Constraints: Developing an application from scratch requires significant time. Dedicating effort to a single project can constrain developers and their organizations.
    ● Scalability Issues: Growing and expanding applications increase the need for scaling and can create complexity. Making changes and adding new features can drag down an organization’s ability to focus on new projects.
    ● Security Concerns: Moderate developers may not be aware of an application’s security risks, jeopardizing the protection of a company.

    Low-Code Platforms Offer Hope

    Low-code platforms have been created to meet the needs of moderately skilled developers and competitively focused organizations. The use of tested drag-and-drop elements offers a secure and powerful way to overcome these challenges.

    Creating practical and powerful applications is available through low-code platforms by connecting ready-made templates. This process rapidly creates applications for use as mobile and web-based applications.

    Non-IT professionals can overcome their lack of time and knowledge through low-code platforms. Short-staffed organizations in their IT department can still provide a powerful connection for their services. The team can quickly get to work meeting its primary mission.

    Visual Modeling is the Secret Power of Low-Code Platforms

    Traditional application development involves writing lines of code, testing and managing by an experienced IT staff. Low-code platforms use pre-tested and connected visual components to build and implement successful and secure applications rapidly.

    Low-code developers can see the application they are building from the beginning of the process to its launch. The drag-and-drop aspect of these platforms is more intuitive and user-friendly. Using proven templates offered by these platforms can also help reduce errors.

    Duplicating Success

    Assembling tested and ready-made components boosts the chance of an application’s success. Pre-configured applications are a way to rapidly launch applications with the confidence of a proven track record.

    Low-code platforms offer users the confidence to trust the components while looking straight at their ongoing results from a clear visual perspective. More time and attention can be paid to an application’s unique functionalities rather than the complexity and drudgery of coding.

    Team Building and Collaboration Opportunities

    The visual nature of low-code platforms allows the entire team to play an active role in an application’s development and keeps everyone speaking the same development language.

    Built-in collaboration tools facilitate feedback loops, revision tracking, user stories and messaging, making it easier for teams to work together in the same office or distributed worldwide. Everyone can own the solution while having a positive impact upon its release.

    Ensuring Scalability

    Low-code platforms offer more than a “one and done” opportunity for growth and success. Robust and friendly customer-facing connections await users of low-code platforms who take advantage of their power. They can continue to generate further service opportunities and sales after the initial launch.

    Proven internal processing networks can be extended with the same low-code platforms that started this robust development. Low-code connections can be efficiently expanded using earlier low-code work.

    Ever-changing business needs can be readily addressed with existing low-code platforms. They can continue to offer ready-made templates for internal and external successes that fit an enterprise’s needs.

    The cloud-based nature of low-code platforms ensures a flexible and continuous service of the applications. This helps ensure reliability and responsiveness to a customer’s ongoing needs.

    Choosing the Right Low-Code Platform

    Identifying the best low-code software development platform is based on several key factors that developers must consider:
    ● Ease of use
    ● Scalability
    ● Integration capabilities
    ● Ability to customize applications

    Decision-makers must also consider their team’s expertise when choosing a low-code platform. Trial versions or free tiers can be beneficial for understanding the team’s expertise and a platform’s capabilities to meet an organization’s needs.

    The Future of Low-Code Platforms

    As businesses continue to navigate the complexities of digital transformation, analysts predict a bright future for low-code platforms. The trend will be continued enterprise adoption to ensure faster development and specific business needs. It’s also clear that low-code platforms will only partially replace traditional software development. But by democratizing app development and fostering a more robust culture of innovation, low-code platforms are positioned to significantly shape the future of business in the digital era.

  • Logz.io Leverages AI to Identify Anomalies in Real-Time

    Logz.io Leverages AI to Identify Anomalies in Real-Time

    Logz.io this week revealed it has added a real-time anomaly detection capability to its observability platform that makes it simpler to correlate the impact IT events will have on business processes.

    This capability is based on artificial intelligence (AI) models trained using historical data collected via the Logz.io Open 360 platform.

    Logz.io CTO Asaf Yigal said this addition to the Open 360 platform enables DevOps teams to employ machine learning algorithms to identify anomalies in mission-critical processes that warrant further investigation. Armed with insights that go beyond simple IT infrastructure consumption metrics, it becomes possible to significantly reduce mean time to resolution, he added.

    Logz.io has been embedding AI models within its platform since last year. Still, this latest capability will enable organizations to further bridge a divide between IT and the business that has long plagued many organizations, noted Yigal.

    The ultimate goal is to make it simpler for DevOps teams to instantly identify any anomalous application behavior that represents a risk to the business, including cybersecurity events, he added.

    The AI capabilities that Logz.io has been steadily adding to its observability platforms are largely transparent to DevOps teams, said Yigal. Instead, Logz.io is embedding multiple types of AI models into its platform that can be invoked in much the same way any other capability is provided, he added. DevOps teams are not necessarily going to need to build their own models to take advantage of AI to observe application behavior at scale, noted Yigal.

    The goal is to surface those capabilities consistently at a level of cost organizations can afford, added Yigal.

    It’s not likely AI will replace the need for DevOps teams any time soon, said Yigal. After all, there is a reason AI models are referred to as co-pilots versus an automatic pilot, he noted.

    It’s not clear to what degree organizations today can correlate the impact an IT event is going to have on the business. But as organizations continue to realize how dependent they are on applications to drive digital processes, the need to bridge the historic divide between IT and the rest of the business has become more urgent. In fact, many of the product and brand champions that organizations now rely on to manage a business unit often have a background in DevOps.

    Of course, business professionals that are steeped in IT are still relatively rare but as advances in AI continue to make it simpler to understand how complex application environments function the number of users of an observability platform will naturally expand beyond IT. The challenge is presenting each stakeholder with a view into the IT environment that meets their specific requirements.

    It may take a yet before the divide between IT and the rest of the business is permanently bridged but with each passing year it gets narrower. The challenge and the opportunity now is making sure everyone on both sides of that divide understands the art of the possible today that only a few short years ago would have seemed impossible to achieve.

  • New Relic Streamlines Observability Data Management

    New Relic Streamlines Observability Data Management

    New Relic this week made available a live archives capability that provides instant access to historical logs for up to seven years and eliminates the need to rehydrate, reload, re-index or move data.

    At the same time, the company also announced its observability platform is now integrated with infrastructure and observability tools provided by Sentry Software to help IT teams reduce costs and collect additional sustainability metrics alongside logs, metrics and traces.

    Jemiah Sius, senior director of developer relations for New Relic, said overall, New Relic is moving to make it simple to analyze and store telemetry data regardless of when it was collected. For example, rather than requiring historical data to be accessed via a cold storage service in the cloud, the New Relic platform now makes that data available directly within the database that is at the core of its observability platform.

    That approach also provides the added benefit of not incurring ingress and egress fees for moving data in and out of cloud storage services, noted Sius.

    IT teams can simply create a New Relic Query Language (NRQL) to identify which logs to route to a live archive in as little as 30 seconds without having to add an additional log collector.

    The overall goal is to make it simpler to investigate incidents faster, given the urgency that is created whenever there is an outage or audit request that needs to be immediately addressed, noted Sius. That’s critical because DevOps teams are always for time, he added.

    In general, New Relic is reducing the overall toil associated that many DevOps teams encounter today when managing telemetry data. Rather than being confronted with a complex set of data engineering tasks, telemetry data from multiple applications and infrastructure platforms can automatically be collected, analyzed and stored.

    It’s still early days as far as the adoption of observability platforms is concerned, but there is already no shortage of options. The challenge is that in addition to logs, metrics and traces, IT teams are now starting to also track cost and energy consumption metrics. Observability platforms provide the means to centralize telemetry data that is only going to increase in volume as more applications are deployed. The challenge has been framing the queries needed to surface meaningful insights. Fortunately, with the rise of generative artificial intelligence (AI) tools that make it simpler to create queries, the overall level of skill required to invoke an observability platform is dropping. In effect, generative AI is making it possible to democratize DevOps best practices.

    Ultimately, existing DevOps teams should be able to manage highly distributed application environments without having to hire and retain a small army of additional software engineers. The issue is that, in many cases, the size of the overall code base is increasing rapidly as developers take advantage of various generative AI tools to help them write code faster. Hopefully, advances in AI will arrive soon enough to enable DevOps teams to cope with all that code before DevOps teams are inevitably overwhelmed.

  • Observe Inc. Adds Tool for Exploring Distributed Traces

    Observe Inc. Adds Tool for Exploring Distributed Traces

    Observe Inc. this week added a Trace Explorer tool to its namesake observability platform that makes it simpler to search, analyze and visualize billions of traces.

    Traces are being used with greater frequency to track how resources are being invoked across increasingly distributed computing environments. That capability provides DevOps teams with a level of insight into workflows that goes well beyond the logs and metrics that almost every IT team routinely collects.

    Observe Inc. CEO Jeremy Burton said the challenge now is providing IT teams with tools that make it simpler to affordably manage traces at scale to reduce mean time to remediation (MTTR). The Observe platform enables IT teams to store the traces they run in a data lake from Snowflake for 13 months versus other platforms that only store a trace for a few days, which then requires IT teams to rerun the same trace multiple times each time they launch an investigation.

    The overall goal is to encourage DevOps teams to launch more investigations to improve application performance by making traces readily available using low cost S3 storage from Amazon Web Services (AWS) that is accessed via the Snowflake data lake that makes it simpler to apply artificial intelligence (AI) to telemetry data.

    That approach makes it economically feasible for DevOps teams to not just resolve an incident but also provide the level of investigation required to accurately identify its root cause, said Burton. Because the cost of storing telemetry data is so high, DevOps teams that employ other observability platforms are not able to thoroughly investigate incidents because they lack access to telemetry data that their organization decided not to store because of cost concerns, he added.

    Most organizations are still mastering the nuances of observability platforms that enable them to query telemetry data that, in addition to being easier to collect using open source OpenTelemetry agent software, is also increasing in volume thanks to the rise of microservices-based applications. Each microservice now generates its own telemetry data that needs to be analyzed and stored. As microservices make application environments more complex, the need for tools to analyze all that telemetry data becomes more pronounced.

    As businesses become more dependent on software to operate, it’s now more than apparent that legacy approaches to monitoring IT metrics are no longer sufficient, said Burton.

    It’s not clear how quickly organizations are adopting observability platforms, but there is already no shortage of options. Given the current state of funding for startup companies in the IT industry, IT teams would be well-advised to research the level of funding the providers of these platforms have before deciding which vendor to rely on, noted Burton. Last year, Observe Inc. raised an additional $50 million in funding.

    Regardless of the observability platform selected, the one thing that is certain is that observability platforms infused with AI are changing the way IT is managed. The issue IT teams often encounter now is convincing the organization to fund the acquisition of these platforms at a time when budget resources in many organizations are already being stretched thin.