devopsy.kinsta.cloud

Category: DevOps Practice

  • Why Observability is Important for Development Teams

    Why Observability is Important for Development Teams

    Observability, which allows development teams to understand better how their systems behave in production, is critical to successful software delivery. According to Gartner, by 2026, 70% of organizations that successfully applied this practice will achieve shorter latency for decision-making, enabling competitive advantage for target business or IT processes.

    Observability allows development teams to identify and diagnose issues quickly to make decisions that improve overall software delivery performance. Let’s take a closer look at why this practice is critical for development teams.

    Differentiating Observability and Visibility

    To better understand observability, it’s essential to clear up a common misconception – that observability and visibility are the same. Two critical differences exist between the two. Visibility allows development teams to look at the surface-level aspects of an application or system. In contrast, observability provides deeper insight into what an application is doing in production. Being equipped with a deep knowledge of observability allows developers to be more productive within their respective organizations.

    Impact on Developer Productivity

    With observability, developers can identify and diagnose issues within applications quickly, giving them more time to fix bugs and time for resolution, which are both critical success factors for software delivery. Observability allows organizations to quickly identify and respond to potential problems, such as operational issues or security breaches, before they become issues that impact customers or users. While there is no such thing as perfect software, observability helps developers quickly find and fix issues, bottlenecks, inefficiencies, etc., so they can get back to doing what they love: Creating new software.

    With the benefits of observability, developers naturally gravitate toward tooling and technology. There are many tools available for developers to take advantage of, and development teams must learn how to use these tools to integrate them into their applications and services. Let’s explore some of the more popular options.

    The Shift to Cloud-Native Tools

    The shift to cloud-native tools is driven by the increasing popularity of cloud computing and its benefits, such as scalability, cost-effectiveness and flexibility. The move to cloud-native has made an enormous impact on observability efforts. Having good tooling and processes available positively impacts observability efforts because it’s a critical aspect of the developer communities’ systems. Overall, the shift to cloud-native tools is driven by the need for organizations to take advantage of the benefits of cloud computing and the need for more efficient and flexible deployment and scaling of applications.

    Some of the most common tools in observability include:

    These tools offer developers greater insight and knowledge into how applications behave. The cloud-native approach necessitates a new way to aggregate all of these operations.

    Conclusion

    With observability, developers can understand how their systems work in production. These systems act in different ways depending on the environments they run in. Therefore, organizations must do more to provide developers with sufficient observability because of the limited visibility into the overall system performance.

    Observability allows developers to act quickly and remedy issues before they go into production and organizations can better measure success relative to the application’s performance. Hours of developers’ time are spent manually debugging and troubleshooting when organizations could have a system in place to do this automatically and allow developers to get back to what they love doing most: Creating software. Observability helps organizations make better-informed decisions, improve operations and performance and achieve their objectives more effectively.

  • 5 Unusual Ways to Improve Code Quality

    5 Unusual Ways to Improve Code Quality

    Code quality refers to how well-written and maintainable software code is. It encompasses factors such as readability, efficiency, scalability, reliability, maintainability and security. High code quality ensures that software is easy to understand, modify and extend, has fewer bugs and is less likely to break down. It also reduces development time and costs and improves the overall user experience. One common approach to determine code quality is to use automated tools that analyze the code based on predefined quality standards and generate a report with a numerical score.

    Why is Code Quality Important?

    Generating high-quality code ultimately enhances the end user’s experience. Focusing on creating quality code can also help consumers understand the safety and reliability of the software or program they’re using.

    Low-quality code can affect user experience and satisfaction, making it critical to write high-quality code that functions properly. When developers write low-quality code, it can create vulnerabilities that expose the software to cybersecurity threats, resulting in bugs and functionality issues and impacting the software’s reliability. 

    Here are some additional reasons high-quality code is important: 

    • Enhances code readability: Readability is important because it leads to higher rates of understanding among other developers. A code that’s easy to read is also easy to improve.
    • Improves the level of program sustainability: Program sustainability measures how long a code can survive with minimal changes. High-quality code can last for years, making it more sustainable.
    • Increases transferability: Transferring code from one platform to another is a complicated process. It’s much easier for developers to transfer high-quality code, as they only need to make minimal changes.
    • Lowers costs related to program or software: When software lasts for a long time with few errors, it can save a company money. Instead of paying for additional time to rewrite low-quality code, the company can rely on the original, high-quality source code.

    5 Unusual Ways to Improve Code Quality

    Here are a few ways you may not have thought of which can help your development team improve code quality and end-user satisfaction.

    Bug Bounty

    A bug bounty program is a type of crowdsourced security testing in which an organization offers rewards to independent security researchers or ethical hackers for identifying and reporting vulnerabilities in their software or systems. By offering rewards for vulnerabilities, organizations can incentivize security researchers to find and report security issues before they can be exploited by attackers.

    Bug bounty programs can improve code quality in several ways:

    • Incentivize security testing: Bug bounty programs incentivize security researchers to test an organization’s software or systems for vulnerabilities. This can increase the amount and quality of security testing that is done, leading to a more secure product.
    • Encourage responsible disclosure: Bug bounty programs encourage security researchers to report vulnerabilities directly to the organization rather than publishing them publicly or selling them on the black market. This allows organizations to fix vulnerabilities before they are exploited by attackers.
    • Provide feedback on code quality: Security researchers who participate in bug bounty programs can provide feedback on code quality which can help organizations to improve their development practices and write more secure code in the future.

    Auto-Generated Documentation

    Auto-generated documentation is a type of documentation that is generated from code comments and metadata as part of an automated development pipeline. It is typically generated using tools such as Javadoc, Doxygen, or Sphinx, which parse code comments and metadata to generate HTML, PDF, or other formats of documentation.

    Auto-generated documentation can improve code quality in several ways:

    • Improved code readability: Auto-generated documentation can provide clear and concise descriptions of code functionality, making it easier for developers to understand the code and its purpose.
    • Better code organization: Auto-generated documentation can help developers organize their code by providing a clear hierarchy of classes, methods, and functions, making it easier to navigate and maintain the codebase.
    • Consistency: Auto-generated documentation can ensure that documentation is consistent across the codebase, reducing the risk of errors and misunderstandings.
    • Quality assurance: Auto-generated documentation can be used as a quality assurance tool, helping developers to identify errors and omissions in their code comments and metadata.

    Coding Conventions

    Coding conventions are a set of guidelines and standards that outline how code should be written and formatted to promote consistency and readability. They typically cover aspects such as naming conventions, code structure, formatting, and documentation. 

    By following coding conventions, developers can improve code quality and make their code easier to read and understand, which makes it easier to maintain and modify. They also help ensure that code is consistent across a project or organization, which can make collaboration and integration of different code segments easier.

    Typical elements that coding conventions dictate include:

    • Naming conventions for variables, functions, and classes.
    • Indentation and formatting of code blocks.
    • Use of whitespace and comments to improve readability.
    • Guidelines for handling exceptions, errors, and input/output.

    Quality Gates

    Quality gates are a set of checkpoints or criteria that software must meet before it can progress to the next stage of development. They are typically used in software development processes to ensure that the software is of a certain quality and is ready for release.

    Quality gates can be automated or manual, and they are typically implemented at key points in the development process, such as after unit testing, integration testing, or acceptance testing. They can include metrics such as code coverage, code quality, functionality, performance and security.

    By enforcing quality gates, development teams can catch and address issues earlier in the development process, which can reduce rework, save time and ensure that the final software product meets the required quality standards.

    Smart Code Snippets

    Smart code snippets, also known as intelligent code snippets or live templates, are pre-written code fragments that can be inserted into a code file with placeholders for variables or user-defined inputs. They are designed to help developers write code more efficiently and accurately by reducing the amount of repetitive or boilerplate code they need to write and by providing contextual awareness and intelligence.

    Regular code snippets are static pieces of code that can be inserted into a file as needed. They typically contain pre-written code blocks or templates that can be customized to fit the specific use case.

    In contrast, smart code snippets are dynamic and can adapt to the context of the code being written. For example, a smart code snippet might include placeholders for variables that need to be defined or automatically fill in common code constructs based on the context of the code. 

    Conclusion

    The five lesser-known ways to improve code quality discussed in this article offer a starting point for teams looking to take a fresh perspective on their development process. Improving code quality is critical for the success of any software development project. While there are many established methods and best practices for improving code quality, there are also some less conventional approaches that can be surprisingly effective. By adopting a creative and open-minded approach to software development, teams can identify new ways to improve code quality and build more reliable, efficient and secure software.

    Image Source 

  • Bug Bounty Vs. Crowdtesting Programs

    Bug Bounty Vs. Crowdtesting Programs

    Digital quality and excellent user experiences are more important today than ever. When a digital experience doesn’t work as expected or is rife with friction points or glitches, customers are quick to give up and move on to a better, easier solution. Bugs and defects often come with high stakes and can result in loss of revenue, dissatisfied customers and potential damage to a brand’s reputation.

    With this in mind, software testing is tasked with the important role of catching and fixing bugs and defects before they hit production. There are several approaches to finding bugs in software, two of which are bug bounty programs and crowdtesting. In this article, we’ll examine the values of each.

    Bug Bounty Programs

    A bug bounty program typically involves people outside an organization with software testing experience, leveraging their expertise to hunt down product defects in exchange for money. Bug bounty programs can be a cost-effective way to find defects that otherwise go unnoticed. They provide an outside perspective and a sanity check for digital products to find vulnerabilities and defects.

    From a security perspective, bug bounty programs are also valuable. Today’s security teams have trouble managing the sheer volume of software vulnerabilities found in software. That’s part of the reason behind the growing number of cyberattacks and breaches that continue to be a topic of discussion. Bug bounty programs leverage white-hat hackers to find and report vulnerabilities in an effort to improve overall security posture and support the organization’s own cybersecurity team.

    Bug Bounty Benefits

    Bug bounty programs offer a variety of benefits to organizations, including:

    ● Helping with blind spots: It doesn’t matter how well your software is designed and coded. Defects and bugs, whether minor or major, will find their way into applications. The developers themselves and internal testing teams may only test certain paths based on organizational norms. Bug bounty testers offer a fresh perspective that can help reduce overall blind spots in application testing.
    ● Resource flexibility: Most organizations don’t have the flexibility to scale internal resources on short notice. Bug bounty programs offer outsourced help and are flexible. They can be scaled up if needed to help with finding defects before a major launch. Or, if a business needs to save money, it can be scaled down by either reducing payouts or limiting testers to high-value defects.
    ● Continuous evaluation: Whether an application is in use by customers or is just a prototype, a bug bounty program can go to work finding and revealing bugs and vulnerabilities. Bug bounty programs can continuously search for bugs that help improve the quality of an application.

    Bug Bounty Challenges

    While there are some great benefits to using bug bounty programs, there are challenges, as well, including:

    ● Money focus: As the name suggests, this is a bounty program. High-severity defects come with a higher payout. If a large percentage of bug bounty hunters are focused on high-severity bugs and vulnerabilities because of the higher earning potential, they may miss the smaller defects. This can add up and result in a less-than-ideal user experience, with smaller issues going unfixed.
    ● The perspective of the tester: No matter how experienced and skilled a tester is, everyone is limited by their own blind spots and perspective. A diverse group of testers can help offset this, but there is still no guarantee that they will test the product in the same way that customers will use them. This challenge is multiplied when we add in different devices, operating systems, languages, and locations. Bug bounty hunters may be more focused on finding high-value issues to earn more, while an internal tester may be more invested in a specific product or feature and making sure it works as it should.
    ● Communication: Many have gotten used to working from home or from anywhere and collaborating remotely. Bug hunters work like this too, but they are outsourced assistance from an organization, which requires a bit of extra thought for communication, project scope and qualifications to make sure misunderstandings or lags in progress are kept to a minimum.

    Crowdtesting

    A crowdtesting model leverages skilled testers and testing teams that can be sourced from a globally dispersed community of independent digital experts. Depending on the size and diversity of the community, testers can apply a nearly limitless variety of demographic characteristics, locations, skill levels, spoken languages, abilities, devices, browsers and operating systems into a testing scenario. This breadth of feedback provides valuable insights into how real users will interact with an app, device or digital experience—which may not necessarily be the “typical” way it is intended to be used. This real-world interaction can unearth bugs, friction points and localization issues that may otherwise go unnoticed before the experience is released to actual customers.

    The remote and on-demand nature of crowdtesting provides scalability and timely results in comparison to more traditional testing methods, like offshoring or lab-based testing, which are typically more limited in scope and flexibility. Plus, the crowdtesting model inherently provides the ability to scale testing services up or down as needed to meet the project-based requirements of a company’s internal teams.

    Crowdtesting Vs. Bug Bounty Programs

    Both bug bounty and crowdtesting programs involve digital experts looking for app defects and vulnerabilities with the goal of improving digital quality. These programs can be aligned in this goal and even work in tandem, but there are some notable differences between them worth mentioning.

    ● Scope: Bug bounty program scopes will always be somewhat limited. Defects can be found outside of the payment scope and go unflagged. Crowdtesting allows for either a broader or more specific scope, depending on needs. Organizations can give specific instructions, like making sure UX considerations are taken into account.
    ● Volume: Both crowdtesting and bug bounties rely on skilled professionals who know how to root out bugs and defects. Crowdtesting, however, allows for as many or as few testers as needed. Instead of gathering a moderate amount of defects from a more technical audience, crowdtesting enables testing for any amount of defects from a diverse and vast pool of testers. As a result, crowdtesting finds a larger variety of defects, and across more devices, locations, and OS versions. This can also be a challenge, as internal staff can become overwhelmed by so many bug reports to look through and address. Therefore, prioritization is key.
    ● Flexibility: Digital quality is a multi-faceted process. It’s not just about functional and security bugs but also accessibility assessment, making sure to test using specific devices and use cases, and more. Bug bounties fall short here, while crowdtesting delivers the flexibility to test and address situations your potential customers may face when an application is live.
    ● Specificity: Most bug bounty programs are not tailored. They can be application- or region-specific, but that doesn’t always reflect a target customer use case. Crowdtesting enables an organization to focus on specific demographics in terms of user experience. An organization can specify testers in certain locations with a certain device or devices and who speak specific languages. Crowdtesting isn’t always just about technical ability; it’s about finding the correct group of people needed for the task at hand.

    Both bug bounty and crowdtesting programs provide unique benefits that help organizations improve overall digital quality. Bug bounty programs offer help with organizational blind spots while leveraging testing professionals in a flexible way. Crowdtesting enables businesses to harness the power of real people using actual devices out in the world to test for use cases. It delivers the power to augment internal testing initiatives, helping to improve overall quality, usability, accessibility and more. When improving digital quality and user experience are the goal and end result, bug bounty and crowdtesting programs can each be impactful and helpful for today’s businesses and for improving the user experience.

  • Five Great DevOps Job Opportunities

    Five Great DevOps Job Opportunities

    DevOps.com is now providing a weekly DevOps jobs report through which opportunities for DevOps professionals will be highlighted to better serve our audience.

    Our goal in these challenging economic times is to make it easier for DevOps professionals to advance their careers.

    Of course, the pool of available DevOps talent is still relatively constrained, so when one DevOps professional takes on a new role it tends to create an opportunity for others.

    The five job postings shared this week are selected based on the company looking to hire, the vertical industry segment and, naturally, the pay scale being offered.

    We’re also committed to providing additional insights into the state of the DevOps job market. In the meantime, for your consideration:

    LinkedIn

    CVS Health
    Remote
    Lead Director – Enterprise Digital Security
    $140,000 to $280,000

    Indeed.com

    Splunk
    Remote
    DevOps and Observability Assigned Expert
    $143,200 to $196,900

    SimplyHired

    Book of the Month
    New York, New York
    Senior DevOps Engineer
    $140,000 to $180,000

    Monster.com

    Innova Solutions
    Plano, Texas
    DevOps Engineer
    $110,000 to $130,000

    Dice.com

    Solaris Finserv
    Kennesaw, Georgia
    DevOps Engineer
    $100,000 to $120,000

  • Grafana Labs Acquires Pyroscope to Add Code Profiling Capability

    Grafana Labs Acquires Pyroscope to Add Code Profiling Capability

    Grafana Labs this week announced it acquired Pyroscope and plans to merge Pyroscope’s open source continuous profiling project with the open source continuous profiling database that Grafana Labs launched last year.

    Richard Hartmann, director of community at Grafana Labs, said after logs, metrics and traces, continuous profiling is now the fourth pillar of observability that enables DevOps teams to identify how much infrastructure resources any given piece of code is consuming. That capability is critical for everything from controlling costs to improving performance to limiting the amount of carbon generated by an application both before and after an application is deployed, he added.

    The resulting Grafana Pyroscope project will be natively integrated with Grafana Labs’ open source dashboard project to make it simpler to visualize profiling data and correlate it with metrics, logs, and traces. Grafana Labs is also committing to adding profiling capabilities to the fully managed Grafana Cloud observability platform.

    Pyroscope currently makes a plug-in for Grafana available in addition to a cloud instance of Pyroscope. Integrating Pyroscope with the Grafana Labs portfolio will make it simpler for DevOps teams to visualize issues before there is a major disruption to an application environment, said Hartmann.

    It’s still early days as far as the adoption of observability platforms is concerned, but it’s apparent that as application environments become more complex, the plethora of monitoring tools that DevOps teams rely on will need to be streamlined. Observability platforms promise to unify logs, metrics and traces in a way that makes it simpler to launch queries to identify the root cause of an issue. The rate at which DevOps teams will embrace observability will naturally vary, but the biggest obstacle might not be the platforms themselves. Instead, the issue is understanding what queries can help DevOps teams better understand the root cause of an IT issue. Visualization tools, in combination with code profiling tools such as Pyroscope, should make it simpler to surface those issues.

    Grafana Labs is also enabling organizations to better address sustainability concerns that have emerged as more policies are enacted to encourage organizations to reduce the overall size of their carbon footprints. The European Union, especially, has been enacting policies that provide organizations with a range of incentives to encourage greater efficiency from IT environments. It won’t be long before more DevOps tools and platforms provide greater visibility and control over the amount of infrastructure application code is consuming to achieve sustainability goals.

    In the meantime, overall observability should continue to improve as organizations move beyond continuous monitoring to gain insights into how application environments are behaving. As IT environments become more complex, tools that monitor a set of pre-defined metrics are no longer enough to ensure application resiliency. The challenge—and the opportunity—is to determine how to take observability to the next level before the next inexplicable application performance degradation.

  • Four Technologies Transforming Data and Driving Change

    Four Technologies Transforming Data and Driving Change

    Data is getting a facelift in 2023 courtesy of event-driven architecture (EDA). These four technologies are set to transform DevOps and IT operations over the next year–whether it be how companies make use of data or the customer experience they offer.

    1. Applied Observability: A Powerful Source of Decision-Making

    Applied observability has been recognized by Gartner as a key 2023 strategic tech trend at its most recent Orlando Symposium. It ensures that the right data is delivered at the right time for rapid action based on confirmed stakeholder actions rather than intentions. “Observable” data include key digitized artifacts that appear when any stakeholder takes any kind of action. Applied observability feeds these observable artifacts back in a highly orchestrated and integrated approach to accelerate organizational decision-making and allows the business owner to track how long an action took to fully process at every point in the workflow.

    But unearthing these insights requires an event-driven approach to software architecture. Distributed tracing, for example, is a method of tracking application requests as they flow from front-end devices to back-end services and databases. By definition, and in contrast to traditional tracing, distributed tracing can be visualized to show a searchable, graphical picture of when, where and how a single event flowed through an enterprise, regardless of the number of hops the workflow took to fully process.

    Consider the potential traceability and end-to-end observability benefits across a payment ecosystem underpinned by EDA. Embedding distributed tracing into an event mesh emits trace events in OpenTelemetry format so banks can collect, visualize and analyze them in any compatible tool. This empowers them to not only confirm that a given message was published but easily understand exactly when and by whom, where it went, down to individual hops, who received it and when … or why not.

    2. Hyper and Intelligent Automation: Taking Automation to the Next Level

    Gartner and IDC identified both hyper and intelligent automation in their yearly technology trends for 2023. Hyperautomation moves automation up a level, adding more intelligence to automation and using a broader set of tools so that previously un-automatable tasks can be automated. Hyperautomation initiatives come in many different shapes and sizes and are being seen across a wide range of industries, from banking and insurance to manufacturing and health care.

    But hyperautomation requires an organization to be underpinned by an EDA and–more specifically, an event mesh–to maximize success. An event mesh is an interconnected network of event brokers that allows data to be pushed in real-time to parts of the organization where the data is needed. It does this dynamically, meaning that new event types can be added at any time and interest in events can be registered to allow a seamless interchange of data for the applications that are interested in using it.

    Take the aviation industry as an example. An event mesh streams information such as flight routes, delays, cancellations and mileage accruals between applications, connected devices and people anywhere in the world, instantly. With an event mesh, information about events can be continuously streamed to multiple systems; filtered so each system only receives the data it needs. This ultimately enhances the customer experience as passengers, pilots and crews are notified in real-time when something relevant occurs on each and every flight.

    3. The Acceleration of the Event-Driven API Economy

    The API economy has exploded. Research from MIT Initiative on the Digital Economy showed that over a four-year period, businesses using APIs saw 12.7% more growth in market capitalization compared to those that did not adopt APIs.

    But the API economy is changing as event-driven architecture and asynchronous event-driven APIs become increasingly important to companies striving to operate in real-time. Using both synchronous and asynchronous methods can result in an application environment in which system resources are most effectively used.

    AsyncAPI and RESTful APIs will be the next evolution of the API economy. API management platforms will need to adapt quickly to this new reality. As a prime example, Forrester recently spotlighted a global biotech company that is blazing a trail in the API-driven area. As the case study demonstrates, the organization understood that EDA is a complement to RESTful APIs–another tool in their digital business toolbox, filling in the gaps of a REST-only approach.

    As a result, they planned a unified event + REST approach from the outset. Both APIs and events are managed as enabling digital products, with the governance and life cycle of both unified as one process. A platform team builds a unified API + event platform for app dev teams rather than building each as a separate siloed platform. The results include faster speed to market and new business opportunities. The company also embraces FAIR data principles: Data that is findable, accessible, interoperable and reusable. A digital marketplace consisting of both events and APIs makes the data findable and machine-readable via Open API Spec (OAS) and AsyncAPI metadata. Data can then be accessed on demand via APIs and events push data in real-time.

    4. The Metaverse Continues at Pace

    Driven by technologies including digital twins, augmented reality and virtual reality, McKinsey estimates the metaverse will be worth $5 trillion by 2030. Data in real-time will be the vital common denominator to link the digital and physical divide and optimize the Metaverse. McKinsey lists real-time data as a vital element to facilitate adoption of the metaverse in several key industries including retail, banking/financial services and transportation.

    The metaverse cannot be static. It needs to be in real-time, in motion. It needs an EDA and a real-time event mesh to support it.

    For example, if an avatar visits a retailer in the metaverse, the retailer will need to keep track of countless user actions such as how long someone spent looking at a particular product, which products got the most interest, etc. The metaverse will produce a significant amount of data that companies will be interested in analyzing in real-time. This data can be used to make decisions from real-time adjustments to inventory to incentivizing commerce through real-time offers/discounts/coupons and even dynamically pricing products.

    Designing a Data-Driven Future With new Tech

    Together, these four predictions have the potential to revolutionize company operations, communications and decision-making–but to take advantage of their full potential and reap success, these four trends will need EDA by their side.

  • How Database DevOps Fuels Digital Transformation

    How Database DevOps Fuels Digital Transformation

    In today’s real-time, hyperconnected reality, data is a core driver behind most digital strategies. User data is constantly gathered and shared across various technology backends and platforms—as is system performance data to inform engineers. Data is pivotal for everything, from generating machine learning models, refining user experiences and even supporting new data-centric services. But although data is a clear backbone to many enterprises, there is no clear winner regarding which database to use.

    Large organizations often find themselves supporting different database styles, from relational databases to NoSQL and distributed SQL. And as organizations continue to adopt multiple databases, safely managing the processing and storage of data becomes more challenging to enact at scale.

    I recently met with Jakub Lamik, CEO, Redgate Software, to understand the role DevOps, specifically database DevOps, is playing in advancing data-driven digital transformations. According to Lamik, applying DevOps practices for database management can improve performance, increase productivity and help organizations stay compliant. Below, we’ll consider the state of today’s data-driven digital transformations and highlight ways to improve how DevOps is applied to database management.

    State of Data-Driven Digital Transformations

    The 2022 StackOverflow Developer Survey found the most popular databases to be MySQL, PostgreSQL, SQLite, MongoDB and Microsoft SQL Server. But most often than not, businesses deploy more than one of these in production. Redgate’s 2021 State of Database DevOps report found that 70% of businesses had more than one DBMS in use, and 48% had teams working on three or more different databases.

    Engineers operate in increasingly hybrid and distributed technology environments. This isn’t for no reason—in fact, 90% of organizations said multi-cloud is helping them achieve their business goals. Lamik also notes a diverse array of software technologies in practice today, from multiple clouds to decoupled microservices architectures and database types. But the desire to work with diverse toolsets causes fragmentation, which could hinder software delivery efforts. For example, organizations want to deploy the same framework and similar methodologies across different databases, said Lamik, and have it be database- and cloud-agnostic.

    Tips to Improve Database DevOps Operations

    To increase the pace of software delivery, infrastructure automation is often needed. And like other roles, database professionals are increasingly trying to automate operations and free themselves from mundane tasks. So, what are some ways to streamline database DevOps? First off, Lamik encourages adopting a common approach across multi-database environments, using standard toolsets that are flexible and portable. “The organizations that standardize deployments across different database environments can increase the frequency and reliability of deployments,” he said.

    Catching and resolving errors quickly is also becoming increasingly essential to meet objectives. And although DevOps brings many advantages, the CI/CD pipeline has gaps. “Observability supported by intelligent automation is key to closing those security gaps,” said Lamik.

    Thus, he encouraged additional oversight and tracking of changes to ensure stability throughout the database deployment pipeline. Also helpful is encouraging collaboration with security professionals (also known as DevSecOps) and integrating security principles into the DevOps workflow to ensure the process is secure by design.

    Another element of database security involves the proper observability and classification of sensitive data. As Lamik describes, organizations must be careful to distinguish between data used for production purposes and data used for development purposes. Ensuring sensitive data is cataloged correctly is imperative to avoid production data from being breached or accidentally publicly exposed.

    Final Thoughts

    In summary, applying DevOps ideals to database management could be a boon for development efforts. For those who haven’t yet considered it, a DevOps transformation will pay dividends, says Lamik. Furthermore, a common management layer could help wrangle multiple database types and help adopt new emerging database types, such as Snowflake and Postgres. Yet, when acquiring new database toolsets, Lamik admitted it may not be a quick transition. He recommends a land-and-expand strategy rather than a top-down approach (which typically isn’t as effective in empowering individual developers).

    Commenting on the industry’s recent move toward platform engineering, Lamik sees it as the next evolution of DevOps. Whereas DevOps brought helpful toolsets to streamline deployment and digitization, platform engineering is taking these practices and productizing them to be reusable at scale across teams and departments, he explained.

    Also, in the new cloud-native stack, containers are designed to be ephemeral, causing engineers to craft workaround solutions to enable stateful storage on platforms like Kubernetes. Regardless of deployment, data must have proper backup and recovery controls as well. In this new cloud-native paradigm, the role of database professionals is changing, explained Lamik, and “APIs are becoming the focal point of communication.”

  • Large Organizations Are Embracing AIOps

    Large Organizations Are Embracing AIOps

    A survey of 265 IT leaders found the primary reason organizations are applying artificial intelligence to IT operations (AIOps) is to improve service and application availability and performance (60%), followed closely by automation of operations (58%) and processes (54%).

    Conducted on behalf of OpsRamp, a provider of an AIOps platform, the survey polled 138 IT leaders working for enterprise IT organizations and 127 representing managed service providers (MSPs). The organizations surveyed generated at least $25 million in revenue and have more than 500 employees. The survey found monitoring/native instrumentation (55%) ranked as the most important feature of an AIOps platform.

    Bill Talbot, chief marketing officer for OpsRamp, said the survey suggested the initial reason organizations appear to be adopting AIOps is to improve monitoring and overall observability as IT teams continue to find simple tasks—such as discovering what is in their IT environments—challenging to achieve.

    Overall, the survey found the greatest IT operations challenge for enterprises in 2023 was automating as many operations as possible (66%), followed by driving greater efficiency and productivity of IT operations teams and processes (61%). Application-to-infrastructure dependency mapping is the top incident management challenge (64%), the survey found.

    The primary operational benefit cited was reducing open incident tickets (65%), followed by reducing mean-time-to-discovery (MTTD) and mean-time-to-remediation (MTTR) tied at 56% and automation of tedious tasks (52%).

    The survey also found 80% of respondents reported it took less than six months to implement an AIOps platform, with more than two-thirds (68%) noting it takes more than six months to hire engineers with the right skillsets for AIOps. Well over 90% of respondents said AIOps is delivering the level of value they expected.

    Despite advances in AIOps, however, two-thirds of respondents said they are concerned about the accuracy of the data being used to train AIOps platforms. A little more than a third (36%) are also concerned about AIOps deployment causing job loss.

    While there has been a lot of initial skepticism concerning AI applications in IT operations at scale, adoption has now crossed the proverbial chasm, said Talbot. IT teams are finding a wide range of static application environments that lend themselves to being automated using machine learning algorithms, he added.

    Less clear, of course, is the degree to which AIOps can be applied to highly dynamic IT environments where changes to code are frequently made. However, the more routine processes that are automated, the more time DevOps professionals will have to manage more challenging issues.

    At this juncture, it’s not so much a question of whether AIOps will be employed to manage DevOps processes as much as it is to what degree. It remains to be seen how generative AI platforms such as ChatGPT might be applied to IT operations, but it’s apparent that AI will automate a wide range of tasks. It’s not likely those capabilities will eliminate the need for humans any time soon, but it does mean that many of the rote tasks that make working in IT tedious may soon disappear.

  • DevOps Adoption in Salesforce Environments is Advancing

    DevOps Adoption in Salesforce Environments is Advancing

    A global survey of DevOps professionals building apps in Salesforce environments found 44% of respondents reported their organization has a continuous integration/continuous delivery (CI/CD) platform in place, with another 39% planning to adopt one.

    The survey, conducted by Gearset, a provider of a DevOps platform for building custom applications in Salesforce environments, polled 1,254 DevOps professionals that are building and deploying custom applications on top of the Salesforce software-as-a-service (SaaS) platform. The survey also found nearly half (49%) implemented change monitoring, with another 33% planning to follow suit. More than half (52%) implemented source-drive workflows, with another 33% similarly planning to follow suit. A total of 41% of respondents noted that, over the last 12 months, they have more custom code to manage.

    Overall, the survey found 54% of respondents reported increased productivity, followed by improved quality and faster releases at 41% each. Nearly three-quarters (72%) reported they are able to deploy changes from one environment to another within four hours. Well over a third (39%) of this group are able to complete a deployment in less than an hour. Just under a quarter (23%) are also able to release code multiple times a week.

    However, only 18% can release a new feature to production in less than a day. Nearly half can complete the release within one day to a week. A total of 43% claimed that less than 10% of releases include a bug or error, with 46% reporting they can roll back a release in a day or less, if needed. Just over a third can restore a process in a day or less.

    The biggest obstacle to DevOps adoption is lack of experience (41%), followed by a third (29%) of respondents that reported difficulties with collaboration and another 29% that experienced limited buy-in from the rest of the application development team. Only 30% received DevOps training once a month, with 50% citing additional CI/CD and automation training as their biggest need.

    Despite these issues, a full 98% are generating a return on their DevOps investments, with 59% of teams reporting returns of over $10,000 per month and 40% reporting returns in excess of $50,000 per month. The top two priorities for the next 12 months are to consolidate systems and processes and improve flexibility, tied at 47% each, followed closely by 43% that noted that digital business transformation is a top priority.

    David Runciman, technical content manager for Gearset, said that while Salesforce applications are based on unique object architecture, it’s apparent that organizations are extending those applications using, for example, low-code tools more quickly. The issue that creates is, as organizations move to deploy these applications, they are encountering bottlenecks that conspire to slow the pace of innovation, he noted.

    The maturity level of DevOps teams building these applications will naturally vary. However, as the number of applications being built—especially by professional developers—increases, it’s only a matter of time before more organizations embrace DevOps best practices to bring order to the chaos, noted Runciman. While there may be some citizen developers building applications in Salesforce environments, the bulk of enterprise-class applications are built by professional developers that typically appreciate DevOps best practices, he added.

    The issue remaining is just how quickly those DevOps teams will be able to implement DevOps best practices as demand for additional applications grows.

  • How Open Source Can Benefit AI Development

    How Open Source Can Benefit AI Development

    Enterprises are increasingly reliant upon open source software. A full 95% of IT leaders say open source tools are key to their enterprise infrastructure. Simultaneously, we’ve witnessed a sharp increase in the adoption of artificial intelligence (AI) and machine learning (ML). Most IT divisions want to take advantage of AI in some form, and open source technologies will likely be vital to these efforts.

    I recently met with Moses Guttmann, co-founder and CEO of ClearML, to understand the benefits of adopting open source software within the sphere of artificial intelligence. To Guttmann, open source has many benefits—the visibility it grants can equate to helpful feedback from the developer community, and transparency can inform new feature development and identify potential security gaps. There are also countless open source tools to help organizations develop and deploy ML models.

    Below, we’ll consider the benefits and potential drawbacks of incorporating OSS in AI development and explore ways to optimize the use of AI within an organization.

    Benefits of Using Open Source for AI

    AI has evolved in leaps and bounds. What was pure sci-fi a few years ago has become a reality through ChatGPT and other AI-powered platforms. The capabilities of AI are awe-inspiring, but Guttmann believes it still requires fine-tuning to move from entertainment purposes into a valid business driver.

    For example, custom ML models are difficult to construct from scratch, and generic APIs aren’t always that customized to the use case at hand to be valuable. Instead, Guttman foresees more and more organizations downloading open pre-trained models and retraining or customizing them to meet their specific needs.

    For software providers, there are many benefits to adopting an open source strategy. The visibility it creates helps evolve a project with feedback from the community, said Guttmann. This, in turn, helps organizations build better, more stable products. For example, ClearML adopts plenty of OSS to power their user interface, backend and other components, in addition to providing a popular open source CI/CD framework for ML workflows. “We’re true believers in the power of community and like to work with open source and contribute back to these projects,” said Guttman.

    Use Cases for Open Source in AI Development

    Open source software is powering innovation in many ways. And when it comes to AI, there are numerous open source projects to help accelerate AI development. Such projects encompass pre-trained models, data-structuring tools, frameworks to train models and AI deployment pipelines. For example, many open source machine learning frameworks can help engineers build and train their models from scratch. These free-to-use tools include TensorFlow, Keras, PyTorch, H2O, XGBoost and others.

    However, training models from scratch is time-intensive and complicated, said Guttmann. Instead, another option is to use pre-trained models accessible on GitHub or other repositories. Some popular examples are PaddleNLP and Natural Language Toolkit for natural language processing, OpenCV for machine vision and DeepFaceLab for deepfakes. There are also many open source Python packages that can structure and clean data for ML model training purposes.

    Open source software can also be applied to streamline the ML DevOps pipeline itself. AI requires intense processing and is typically deployed to its own computing environments. Infrastructure management around running and training AI is a promising area where open source could assist, said Guttmann. “Automating this process with an abstraction layer to allow data science and ML engineers to access product-grade hardware is a must,” said Guttmann.

    Potential Downsides of Open Source Usage

    Yet, of course, there are some potential downsides to using open source software within the field of AI. Supply chain attacks are still a persistent threat, and numerous open source software risks can present themselves. These include bad actors compromising a legitimate package, name confusion attacks, unmaintained software and untracked dependencies. License restrictions could also potentially block the inclusion of OSS in certain projects.

    Thankfully, the benefits of OSS appear to outweigh these concerns. Automated vulnerability scanning techniques can help detect known issues. It also helps that 89% of technology leaders see enterprise open source as more secure or as secure as proprietary software, according to a Red Hat report.

    Tips For Getting the Most From AI

    Open source is powering digital innovation on many fronts, including AI. But, simply looping in OSS isn’t the only aspect of fully leveraging AI—another will be the operationalization of ML management. The right automation infrastructure in place could enable the agility necessary to continuously retrain models. And since there are potential gaps in the CI/CD process, Guttmann recommended separating AI/ML from your normal CI/CD for security reasons.

    Guttmann also believes that, in general, companies should be more transparent around ML models, including sharing details behind their creation and visibility regarding the datasets fed into them. Overlooking this could lead to ethical concerns. Furthermore, limiting the “hallucinations” generative AI produces is another huge gap that must be bridged, said Guttmann. This could be solved by simply allowing the system to respond with “I don’t know,” instead of confidently spouting nonsense.

    In summary, open source could bring many benefits to bespoke AI/ML adoption. Many mature open source tools exist to enable engineers to create advanced AI and oversee the management and deployment of custom ML models. But the real test will be retaining this open creed by upholding visibility and transparency into the end creations.