devopsy.kinsta.cloud

Category: DevOps Toolbox

  • Couchbase Adds Columnar Database to DBaaS Environment

    Couchbase Adds Columnar Database to DBaaS Environment

    At the AWS re:Invent 2023 conference today, Couchbase, Inc. announced it added a columnar database based on the JSON file format to its database-as-a-service (DBaaS) portfolio to enable organizations to build real-time analytics applications.

    Scott Anderson, senior vice president for product management and business operations at Couchbase, said the Capella columnar database is designed to be integrated with the company’s JSON-based document database via a database change protocol (DCP) that Couchbase has embedded within its DBaaS environment.

    Deployed on cloud services from Amazon Web Services (AWS), the overall goal is to reduce the level of friction that IT teams would otherwise encounter when integrating a document database with a columnar database optimized to process analytics using columns rather than the rows typically associated with a relational database.

    That approach gives IT teams the ability to use a schemaless database architecture that provides an alternative to employing disparate databases that would need to be deployed and then integrated using extract, transform and load (ETL) processes to move data from one database to the other, noted Anderson.

    Capella columnar also uses the same SQL++ language as the company’s document database to streamline queries in addition to supporting Capella iQ, a copilot based on generative artificial intelligence (AI) that enables queries to be crafted in natural language.

    In addition, Capella columnar provides connectors to move data from Amazon DynamoDB, Amazon DocumentDB, Amazon Relational Database Service (Amazon RDS) and other data sources deployed on the AWS cloud.

    In the longer term, Couchbase is also planning to add vector capabilities to its DBaaS environment to make it simpler to securely extend generative AI models, said Anderson.

    In general, document databases have been widely adopted because they provide developers with an alternative to legacy database platforms that they can download and deploy themselves. The Capella DBaaS platform further simplifies that process via a managed service through which Couchbase assumes operational responsibility for managing databases. The columnar database extends that capability using a columnar database that can drive analytics capabilities to deliver a more personalized application experience, noted Anderson.

    It’s not clear to what degree organizations will ultimately wind up relying on as-a-service platforms, but since the start of the COVID-19 pandemic, the shift to this approach to consuming IT resources has increased significantly. As a result, the way internal IT teams are organized is shifting as lower-level tasks are either automated or handled by an external IT team that works for a vendor.

    Regardless of how databases are managed, however, there remains a need to integrate the data operations (DataOps) processes used to manage them with the DevOps workflows used to build modern applications. Each organization will need to determine how much economic sense it makes for them to manage that data versus relying on an external services provider. The challenge when relying on an external service provider is, as always, integrating that service into an existing IT environment.

  • Lineaje Allies With Persistent Systems on Open Source Software Remediation Service

    Lineaje Allies With Persistent Systems on Open Source Software Remediation Service

    Lineaje this week revealed it has partnered with Persistent Systems to provide a managed service for remediating open source vulnerabilities that leverages artificial intelligence (AI) technologies.

    Persistent is extending an existing open source software maintenance service by incorporating a Lineaje platform that makes use of machine learning algorithms to analyze software bill of materials (SBOMs) to provide recommendations for fixing code and surface remediations such as updates that should be applied.

    Lineage CEO Javed Hasan said Persistent Systems is augmenting that capability with a dedicated team of specialists that IT organizations can contract to maintain open source code along with previous investments the managed service provider has made in large language models (LLMs) that add generative AI capabilities.

    Options for the services range from simply identifying issues that an internal IT team should address to a white glove offering through which Persistent Systems ensures all open source components are continuously updated.

    In the wake of an executive order issued by the Biden administration that requires federal agencies to better secure their software supply chains, more attention is being focused on the inherent cybersecurity risks associated with employing open source software to build applications. Many open source projects are led by a handful of maintainers that often lack the time and skills required to ensure the components they employ have not, for example, been compromised by malware that might find its way into downstream applications.

    Those maintainers are not being paid, so cybersecurity patches to open source software might not be readily made available, and even when they are, many organizations lack the skills and resources required to apply them in a timely manner. The majority of known vulnerabilities within application environments are still unpatched. Many IT teams are also reluctant to upgrade components for fear of breaking applications that have dependencies that are often difficult to track.

    It’s not known how much of the open source software that many developers routinely incorporate into their applications might contain vulnerabilities that could be exploited, but the level of technical debt that has accrued over the last several decades is considerable. While only a small percentage of those vulnerabilities have been exploited, cybercriminals are becoming more adept at discovering them.

    Hopefully, more contributions to open source projects will improve the overall state of application security, but in the meantime, it’s clear organizations will need to rely more on AI to drive DevSecOps workflows as ever-increasing amount of open source software continues to be relied on to build applications, said Hasan.

    The challenge, of course, is identifying which vulnerabilities, if exploited, will have the biggest impact on the organization. After all, at this point, it’s not possible to defend every application so the focus needs to be on prioritizing remediation efforts based on their actual severity rather than a general rating.

    In the meantime, DevSecOps teams would be well-advised to have incident response plans in place for remediating breaches involving both known and zero-day vulnerabilities that, in addition to being all but inevitable, can now be exploited within minutes of being discovered.

  • Nobl9 Allies With Microsoft to Simplify SLO Creation

    Nobl9 Allies With Microsoft to Simplify SLO Creation

    Nobl9 this week revealed it has integrated its reliability center platform for managing service level objectives (SLOs) as code with the Microsoft Azure Monitor service to make it simpler to both create and track service levels.

    The Nobl9 Reliability Center provides access to dashboards for a single source of truth about the state of reliability in an application environment based on the metrics collected from the SLOs that have been defined.

    Nobl9 Chief Growth Officer Kit Merker said the integration with Microsoft Azure Monitor is the first in a series that will ultimately make it easier to define SLOs using historical data collected by third-party tools.

    The Nobl9 Reliability Center is already integrated with more than 50 DevOps, observability and incident management tools through which it collects metrics, events, logs, traces, alerts to track incidents, releases, rollbacks, runbooks and other documentation.

    Those integrations will ultimately make it simpler for IT teams to create SLOs using historical data that today are still challenging to define from scratch, noted Merker. In fact, the data tracked by Nobl9 can be used to automate remediations via those integrations, he noted.

    As more microservices-based applications are built and deployed, it’s becoming more challenging to maintain SLOs across applications that have many more dependencies than legacy monolithic applications. Each feature and application programming interface (API) added over time conspires to adversely impact performance. It‘s not so much that any given service will outright fail as much as it is determining where degradations are occurring that result in SLOs not being met.

    Ultimately, each IT team needs to provide some sort of objective benchmark that assesses their overall effectiveness at delivering application services by making it simpler to track whether SLOs are achieved and maintained. Not every service needs to necessarily meet a stringent requirement. Instead, IT teams need to have confidence in the fact that, on average, certain levels of performance are being consistently maintained.

    That’s crucial because as more organizations depend on software to deliver digital services, any disruption is likely to have a direct impact on profit and revenue. As such, the reliability of application environments has become a major concern in an era where IT teams are now trying to manage distinct services rather than individual applications.

    The challenge and the opportunity now is to find a way to proactively reduce that stress by programmatically managing SLOs as code rather than having to overprovision infrastructure resources to ensure maximum availability at the highest cost possible. At a time when more organizations are encountering economic headwinds, that’s no longer a luxury they can afford.

    Each organization will need to determine for themselves how aggressively they need to manage SLOs. Of course, SLOs as a concept have been around for decades, but with the rise of digital services that span multiple organizations, the level of reliability DevOps teams are expected to provide is now being written into business contracts. Not surprisingly, the overall level of associated stress DevOps teams are experiencing has correspondingly increased.

  • Immutable Infrastructure: The Next Step for DevOps

    Immutable Infrastructure: The Next Step for DevOps

    Immutable infrastructure, a concept that is rapidly gaining traction within the dynamic field of DevOps, offers a transformative approach to organizational security for companies of all sizes. 

    Instead of the traditional method where existing components are continuously updated and modified, immutable infrastructure advocates for an entirely different philosophy: Components are replaced wholesale with new versions and never altered after deployment. This type of paradigm shift promises a plethora of advantages, chief among them being enhanced consistency and reliability, which is a potential game-changer for DevOps practices.

    This article offers a thorough exploration of immutable infrastructure, taking a look at its core principles and examining why it’s becoming a focal point in the DevOps community. To help further bring these concepts to life, we’ll thoroughly examine the tools and techniques that are pivotal in its implementation.

    Understanding Immutable Infrastructure

    Immutable infrastructure is rapidly gaining traction in the world of cloud services and DevOps for its innovative approach to managing servers and components. Once deployed, these elements are not modified or updated; instead, new servers are created from a common image whenever changes are necessary. 

    This type of approach simplifies rollback procedures and minimizes troubleshooting, as the integrity of servers is maintained. Furthermore, this methodology, facilitated by OS-level virtualization, marks a significant departure from more traditional practices.

    And with the AI market already worth $313 billion, integrations of models and solutions will only become more popular. Integrating AI into immutable infrastructure can significantly enhance its efficiency and reliability. 

    In such systems, AI can be used to monitor and analyze system performance continuously. By applying machine learning algorithms, it can predict potential issues and automate the process of deploying new server instances before problems arise. 

    Comparing Immutable and Mutable Infrastructures

    The Immutable Approach

    Immutable infrastructure offers streamlined change tracking and rollback, with each new server or VM being easily identifiable. Its uniform configuration across servers simplifies testing and the unaltered state post-deployment reduces complexity. 

    This makes it ideal for environments with numerous interdependent elements, such as in DevOps and cloud computing, virtually eliminating configuration drift.

    Challenges With Mutable Infrastructure

    In contrast, mutable infrastructure necessitates more time for server provisioning due to manual configurations. It’s prone to various update failures and connectivity issues, with each server’s unique configuration complicating diagnostics. 

    This traditional approach struggles with unpredictability, making it less suitable for modern DevOps practices and resulting in frequent configuration drifts.

    Advantages of Immutable Infrastructure

    Opting for immutable infrastructure brings numerous benefits. Support calls are significantly reduced and security is enhanced, thanks in part to the use of machine learning to identify anomalies. 

    The implementation of safelisting rules strengthens security controls. Moreover, this approach mitigates configuration drift and negates the need for synchronization during updates, leaning on cloud platforms for swift VM management. It also ensures rapid recovery due to easy rollbacks and encourages experimentation, thus fostering innovation.

    Best Practices for Immutable Servers

    Adopting immutable servers involves several best practices all organizations would be wise to apply. Building images in a chroot environment helps isolate changes while keeping test suites lean and efficient while avoiding overburdening the change process as a whole. 

    Caching installation files near the build location optimizes update speeds, layering images on top of a base server image expedites updates, and minimizing the OS image reduces boot times and security risks. Additionally, using containers can dramatically speed up and simplify the process.

    Essential Tools for Immutable Infrastructure

    When it comes to immutable infrastructure, certain key tools have emerged as indispensable for efficiently managing and maintaining this modern approach. These tools help create, manage and deploy server images and infrastructure-as-code throughout organizations. 

    Here’s a closer look at some of these essential tools:

    Packer

    Packer is a crucial tool for creating identical machine images for various platforms from a single source configuration. This tool is instrumental in the immutable infrastructure because it allows for the creation of fixed server images that can be quickly deployed, ensuring consistency across environments.

    Terraform

    Terraform by HashiCorp is another key player, enabling users to define and provide data center infrastructure using a declarative configuration language. It helps maintain the infrastructure-as-code, which is an essential aspect of immutable infrastructure. 

    Terraform allows for safe and predictable changes to infrastructure, with efficient management of both cloud and on-premises resources.

    Docker

    Nowadays, Docker is synonymous with containerization, allowing applications to be packaged into containers—consistent, isolated environments that encapsulate all the necessary components of immutable infrastructure. 

    Docker helps to simplify the deployment process and ensures that applications run the same way everywhere, aligning perfectly with the core principles of immutable infrastructure.

    Kubernetes

    While not directly mentioned earlier, Kubernetes deserves recognition for being a powerful orchestration tool for containers, including those that are managed by Docker. Kubernetes excels in automating deployment, scaling and management of containerized applications, which aligns well with immutable infrastructure principles.

    SaltStack

    SaltStack offers configuration management and orchestration tools, standing out for its ability to manage and automate more complex and scalable infrastructures, making it suitable for environments where immutable infrastructure is implemented. SaltStack ensures that server configurations are enforced and maintained as defined, preventing drift.

    Jenkins

    An open source automation server, Jenkins plays a significant role in the continuous integration and continuous deployment, or CI/CD, pipeline. Jenkins helps to automate the various stages of the delivery pipeline, which is crucial in testing and deploying immutable infrastructures.

    Ansible

    While Ansible is generally associated with configuration management, it also excels in automated deployment. Because it doesn’t require agents on the servers it’s managing and uses a simple, human-readable YAML syntax, it’s useful for setting up and maintaining immutable servers.

    Git

    Git, a version control system, is essential for tracking changes in the configuration code, scripts and templates used to create and manage immutable infrastructure. It allows teams to manage their infrastructure-as-code, ensuring a detailed audit trail of what was changed, when and by whom.

    Ultimately, a slew of tools and platforms are available to make the transition to immutable infrastructure as seamless as possible. From container orchestration with Kubernetes to configuration management through Ansible, you have a robust array of options at your disposal. 

    Yet, when it comes to handling specification files or any form of documentation, you want an added layer of security. A robust custom document viewer with extra security features, for instance, could be an invaluable asset in this regard, offering the ideal blend of accessibility and protection while ensuring that your infrastructure is immutable and secure from prying eyes.

    Choosing the Right Tools For Your Organization

    When integrating immutable infrastructure into your operations, selecting the right mix of tools that align with your specific requirements and existing workflows is crucial. Essential factors such as compatibility, scalability and ease of use should be considered to create a robust, efficient and effective immutable environment. 

    Each of the tools highlighted above has its own strengths and is a part of a broader ecosystem designed to support the reliable, consistent and secure deployment of applications and services across organizations of all sizes. Companies can fully embrace the immutable infrastructure model by learning how to properly harness these tools, improving stability, scalability and efficiency across all of their IT operations.

  • Redgate Automates Pulling of Test Data From Production Databases

    Redgate Automates Pulling of Test Data From Production Databases

    Redgate today added a Redgate Test Data Management (TDM) tool that automates the pulling of masked test data from databases running in production environments.

    David Gummer, chief product officer for Redgate, said this capability eliminates manual tasks that previously inhibited testing or resulted in personally identifiable information (PPI) being inadvertently used to test applications.

    Redgate TDM automatically classifies and masks data residing in SQL Server, PostgreSQL, MySQL or Oracle database before it is used to test an application. It then produces a copy or clone of that data that is a fraction the size of the original, said Gummer.

    That approach creates a repeatable process for DevOps teams that can withstand a regulatory audit of application development processes, he added. In fact, with the increased focus on software supply chain issues, more auditors are specifically reviewing how data is secured during the application development process.

    Designed to be invoked via a command line interface (CLI), within an integrated development environment or via a graphical interface, Redgate TDM reduces the friction that currently occurs when DevOps teams request test data from database administrators (DBAs), said Gummer. Many DBAs are either too busy to fulfill those requests or reluctant to provide access to sensitive data that would result in the organization running afoul of a compliance mandate.

    Developers then often resort to using anonymous data that doesn’t actually reflect the environment in which their application will ultimately be deployed.

    The amount of data that developers need to build applications is only going to increase. In the absence of any way to effectively automate the process of accessing that data, it ultimately takes longer to build and deploy stable applications. In many cases, applications are simply not tested as thoroughly as they should be before being deployed in a production environment, resulting in rollbacks that might otherwise be avoidable.

    Advances in artificial intelligence (AI) are making it easier to create test scripts, but much of the testing setup process is still based on manual processes that increase the overall level of toil.

    At a time when more organizations than ever are trying to increase developer productivity, automating relatively mundane tasks such as pulling test data can have a big impact. Developers want to spend as much time as possible focused on writing code. Everything else is a task that gets in the way of achieving that goal. While there is a general desire to empower developers, that doesn’t necessarily mean they want the cognitive load associated with building applications to increase. In fact, one of the reasons platform engineering has emerged as a methodology for managing DevOps workflows at scale is to reduce the current level of cognitive load developers experience today.

    Of course, DevOps teams are, by their nature, ruthlessly committed to automating as many processes as possible. The challenge is bottlenecks in those processes conspire to slow down the overall rate at which applications can be built and deployed despite the best intentions of all involved.