Category: DevOps Practice

  • Leaders in Resilience See ROI From Observability

    Leaders in Resilience See ROI From Observability

    Observability is maturing throughout the software industry. Observability is often defined as merging metrics, logs and traces to get a fuller picture of an application’s state. This can help pinpoint disruptions or security incidents and offer a clearer path to remediation. Observability is touted to advance the application monitoring practices of yesteryear, but are investments paying off?

    According to recent findings, many organizations are using observability tools and seeing a positive outcome. A full 73% of organizations reported using observability tools for over a year. And impressively, 86% of observability leaders said the ROI on their observability tools has exceeded their expectations. The practice appears to be paying dividends for organizations that have gone “all-in” with mature observability processes.

    The State of Observability 2023 report released by Splunk demonstrated how observability leaders are reaping a positive return from observability tools. According to the report, observability is helping to meet performance expectations, avoid downtime and enhance resilience in general. However, not all organizations are on the observability maturity bandwagon. Below, we’ll examine some of the key trends from the report and consider the benefits of refining your observability efforts.

    Now that observability is more commonplace, dedicated roles are emerging to support these initiatives. In fact, 87% of respondents now employ specialists who work exclusively on observability projects. Site reliability engineers likely make up a portion of these experts, as observability data can be used to advance efforts around meeting service-level objectives (SLOs).

    But although observability might be more common than a few years ago, not every observability practice is mature or standardized. Just 40% have a formal approach to resilience that has been institutionalized organization-wide. Because of this, only 43% are confident in their team’s ability to meet its application reliability and performance objectives. It should be noted that this number increases significantly for those with more mature observability adoption, which we’ll expand on below.

    Furthermore, not all applications are cloud-native. Organizations reported operating and maintaining, on average, 165 internally developed business applications, half of which are hosted in the public cloud and half of which are on-premises. Due to the persistence of legacy modalities, observability tools must continue to support a hybrid architecture if they are to be comprehensive and useful.

    And although we’ve previously covered how many organizations are drowning in a sea of observability data, observability leaders now say their ROI on observability tools far exceeded their expectations. Perhaps this indicates that, with more open standards, analysis centralization and fewer false positives, teams are finding ways to avoid observabiliy tooling sprawl.

    Lastly, the report unveiled desires to loop AI and ML into the observability pipeline. In fact, 66% of observability teams already use AI/ML, while 26% are in the process of deploying the technology. And 91% said AIOps is an important enabler of their observability goals. Certainly, AI is becoming a pervasive instrument to accelerate many areas, including observability.

    Benefits of Observability

    An impressive 64% reported that their investment into observability has exceeded their expectations. So, what are some of the top benefits of observability? Well, the report found the topmost benefits to be areas like decreasing problem detection and resolution times, identifying “true positive” incidents, and enhancing application security.

    Observability is being deployed as a tool to evolve traditional security efforts, as it uncovers a deeper understanding of application behaviors. This can help security teams correlate errors to specific diagnoses and spot potential threats in a quick manner. As such, 59% said observability helps them uncover security issues thanks to the intelligence capabilities of observability solutions. And 55% say they can uncover more security vulnerabilities thanks to the visibility afforded by observability.

    Observability can also be used to improve visibility into hybrid architectures. A full 86% said it’s important to have flexible observability solutions that cover hybrid architectures, though half identify this as an area for improvement. Although we like to think that cloud-native development is the status quo, many enterprises still oversee a mixture of on-premises, private cloud and multi-cloud environments. A unified, agnostic observability layer is thus essential to peek into all these disparate environments to grapple with disparate data outputs.

    The Leaders of the Observability Pack Win

    Service interruptions can have negative consequences for organizations, including lower customer satisfaction, loss of revenue and loss of customers. In fact, 76% of respondents reported downtime can cost up to $500,000 per hour.

    Thankfully, observability has emerged to help improve overall resilience and avoid these negative repercussions. But, to truly reap the benefits, you must have a mature practice in place. As defined by the report, “observability leaders” have a more formal approach to resilience and have standardized this across their organization. And 89% of those leaders are completely confident in their ability to meet their application availability and performance requirements. The catch-22 with observability is that you must invest more energy into formalization and knowing when the effort is worthwhile is a moving target.

    A solid observability practice is supported by standardized metrics, logs and traces across the organization. Furthermore, to make observability behave correctly, you will need flexible tooling that can leverage data from multiple sources, such as the network, infrastructure, servers and databases.

    The State of Observability 2023 report surveyed 1,750 observability practitioners from around the globe, making it the most extensive study on observability to date. For more insights, you can pick up a copy of the report behind an email gate here.

  • Revolutionizing DevOps Culture Transformations With AI-Engineered Tools

    Revolutionizing DevOps Culture Transformations With AI-Engineered Tools

    In my recent article Revolutionizing the Nine Pillars of DevOps with AI-Engineered Tools, I explained that DevOps thrives in a culture of openness, transparency and shared responsibility. It’s about breaking down the silos that can exist between different teams (like development, operations, security and QA) and promoting effective communication and collaboration. AI-powered tools can improve collaboration through smart recommendations, fostering more effective communication and knowledge sharing.

    Collaboration and Communication: AI-powered chatbots can facilitate quick and easy communication across teams, enabling faster decision-making and problem-solving. AI-powered project management tools can improve visibility into tasks and dependencies, promoting better coordination among team members.

    Continuous Learning and Improvement: AI-based monitoring tools can help teams learn about system performance in real-time, identifying trends and patterns that can inform better decision-making. AI can also help in creating personalized learning experiences, identifying skills gaps and suggesting relevant training resources, promoting a culture of continuous learning.

    Shared Responsibility and Ownership: AI can improve visibility into the system’s state, helping all team members understand how their work contributes to overall system performance. This can promote a sense of shared ownership and responsibility. AI can also assist in automating the allocation of tasks based on an individual’s skills and availability, promoting a more equitable distribution of responsibilities.

    Automation and Efficiency: AI is pivotal in automating routine tasks, such as testing, deployment and monitoring. This can significantly improve efficiency and allow teams to focus on strategic, high-value tasks. AI can also help automate the detection and resolution of common issues, reducing downtime and improving system reliability.

    Feedback and Adaptability: AI-powered analytics tools can provide real-time feedback on system performance, enabling teams to quickly adapt their approach as needed. AI can also help in analyzing customer feedback and identifying key themes and sentiments that can inform product development.

    Risk-Taking and Innovation: AI can help simulate the potential impacts of different decisions, allowing teams to take calculated risks. AI can also drive innovation by helping teams identify new opportunities, predict trends and develop more creative solutions.

    Roadmap to AI-Assisted DevOps Culture

    A practical roadmap for transforming an organization’s DevOps culture using AI-engineered tools may look something like this:

    1. Define Goals and Evaluate the Current State: Identify what you want to achieve by integrating AI into your DevOps practices. This could be improving deployment times, reducing errors, increasing productivity and more. Evaluate your current DevOps practices and tools to understand where AI could bring the most value.

    2. Identify AI Opportunities and Select Tools: Look for areas where AI can enhance collaboration and communication. This could involve AI-powered chatbots for team communication or project management tools with AI capabilities. Identify opportunities for AI to support continuous learning and improvement. AI-powered analytics and monitoring tools can provide valuable insights for this. Determine where AI can help promote shared responsibility and ownership. AI-enabled visibility tools can assist here, too.

    Choose areas for AI-powered automation. This could include tasks like testing, deployment and issue resolution. Find avenues where AI can support feedback and adaptability. Tools that analyze system performance or customer feedback could be helpful here. Pinpoint where AI can assist in risk-taking and innovation. AI-powered simulation or predictive analytics tools could be useful.

    3. Implement AI Tools Gradually: Start by integrating AI tools into one or two areas of your DevOps process. Monitor the impact to ensure you’re driving the desired improvements. Gradually expand the use of AI tools to other areas of your DevOps practices.

    4. Train Teams on AI Tools: Provide adequate training to your teams so they can use these new AI tools effectively. Encourage a culture of learning and experimentation to help teams become comfortable with AI.

    5. Monitor and Adapt: Regularly monitor the impact of these AI tools on your DevOps practices. Are they improving speed, reducing errors, enhancing collaboration, etc.? Be prepared to adapt your approach based on the feedback and results you’re seeing. The integration of AI is a journey, not a one-time event.

    6. Continuously Improve: Continually look for new ways that AI can enhance your DevOps practices. AI is a rapidly evolving field, and new opportunities are emerging all the time. Remember that this roadmap is iterative and adaptive. AI integration should be seen as a part of the broader DevOps goal of continuous improvement.

    Benefits for Organizations, Teams and Individuals

    With the integration of AI-engineered tools into DevOps processes, organizations can expect significant improvements in efficiency, speed and quality. AI automation can drastically reduce manual, repetitive tasks, thereby lowering the chance of errors and freeing up resources for more strategic endeavors. This can lead to cost savings and a better allocation of resources. Additionally, AI can enable faster decision-making and problem-solving, accelerating the software development life cycle and making the organization more competitive. AI can also enhance product quality by providing valuable insights and analytics, enabling continual improvement.

    For Teams: AI can facilitate improved collaboration and communication, making workflows more seamless and efficient. AI-powered project management tools can provide better visibility into tasks, dependencies and progress, enabling smoother coordination. AI can also automate routine tasks, freeing up team members to focus on more challenging, high-value work. Additionally, AI can provide real-time feedback and insights, enabling teams to continuously learn and improve their processes. These improvements can lead to higher job satisfaction and productivity.

    For Individuals: Individuals stand to gain from AI integration in several ways. AI-powered tools can support skill development by helping individuals learn new technologies and best practices. Real-time feedback and insights can also foster personal improvement. Automation of routine tasks allows individuals to focus more on creative and strategic aspects of their work, leading to a more engaging and satisfying job experience. Furthermore, being part of an AI-driven DevOps environment can open opportunities for individuals to work on cutting-edge technologies and innovative projects, enhancing their career prospects.

    Measuring Success

    Choosing the right metrics is crucial for measuring the progress of your DevOps culture transformation with AI-engineered tools. Here are some important metrics to consider:

    Deployment frequency: This is a measure of how often new features, bug fixes or updates are deployed. An increase in deployment frequency can be a sign that AI tools are improving efficiency.
    Lead time for changes: This metric refers to the amount of time it takes from code being committed to it being deployed. AI tools can help to reduce this time by automating testing and deployment processes.
    Change failure rate: This is the percentage of changes that result in a failure or that require a rollback or hotfix. AI can help reduce this rate by improving testing accuracy and identifying potential issues earlier in the development process.
    Mean-time-to-recovery (MTTR): This is the average time it takes to recover from a failure. AI tools can help to reduce MTTR by quickly identifying the cause of failures and automating the recovery process.
    Customer satisfaction: AI can help improve the quality and reliability of the product, leading to increased customer satisfaction. Customer satisfaction can be measured using surveys or net promoter score (NPS).
    Efficiency metrics: Look at how much manual, repetitive work has been reduced by automation. This could be measured in terms of man-hours saved.
    Learning and development: Monitor the upskilling of your teams to using AI tools and technologies. This could be tracked through training completion rates, skill assessments, etc.
    Innovation rate: AI can free up teams to focus on innovation. Measure the number of new ideas or features being developed over time.

    Summary

    To transform DevOps culture with AI tools, organizations must set clear goals, identify AI opportunities, gradually implement those tools and monitor their impact. Training teams to use AI tools effectively and fostering a culture of continuous learning and improvement will enable this transformation and help DevOps teams thrive. This integration can significantly enhance efficiency, decision-making speed and product quality at an organizational level.

    On a team level, AI can streamline collaboration, automate routine tasks and provide real-time insights for continuous process improvement. For individuals, this transformation provides opportunities for skill development, personal growth and more satisfying job experiences. Monitoring progress with metrics like deployment frequency, lead time for changes, change failure rate, mean-time-to-recovery (MTTR), customer satisfaction, efficiency, learning and development and innovation rate ensures the transformation is delivering the expected benefits and contributing to continuous improvement.

  • Five Great DevOps Job Opportunities

    Five Great DevOps Job Opportunities

    DevOps.com is now providing a weekly DevOps jobs report through which opportunities for DevOps professionals will be highlighted to better serve our audience.

    Our goal in these challenging economic times is to make it easier for DevOps professionals to advance their careers.

    Of course, the pool of available DevOps talent is still relatively constrained, so when one DevOps professional takes on a new role it tends to create an opportunity for others.

    The five job postings shared this week are selected based on the company looking to hire, the vertical industry segment and, naturally, the pay scale being offered.

    We’re also committed to providing additional insights into the state of the DevOps job market. In the meantime, for your consideration:

    Indeed.com

    SAP
    San Francisco, California
    DevOps Engineer
    $141,378 to $162,000

    LinkedIn

    Colgate-Palmolive
    Piscataway, New Jersey
    DevOps Lead
    $112,000 to $164,800

    CareerBuilder.com

    Northrup Grumman
    Dulles, Virginia
    Principal DevOps Engineer
    $104,600 to $157,000

    SimplyHired

    Crate & Barrel
    Northbrook, Illinois
    DevOps Engineer
    $99,100 to $125,000

    Dice.com

    Tata Elxsi Ltd.
    Philadelphia, Pennsylvania
    DevOps Engineer
    $80,000 to $100,000

  • The Case For an API-First SCM

    The Case For an API-First SCM

    Somewhere around a decade ago, about the time DevOps really started to manifest, we began seeing technology companies—including some of the largest ones today—investing heavily in their APIs.  

    API-first companies managed to rapidly gain widespread adoption because of the benefits of being API-driven. Everything from automation and scalability to flexibility, performance, speed and monetization.  Once DevOps thinking made it increasingly clear that automation would solve problems like human error and increase performance and velocity, creating more machine-to-machine interaction became a critical factor in software delivery—because the problem in the technology world was not innovation, but rather integration.

    Twilio is a prime example. The company became famous for its API-driven approach to customer management and even had a famous Silicon Valley billboard trumpeting their success. They changed the game for customer communication for some of the largest companies in the industry, from Netflix to DoorDash to Coca-Cola, by making communication completely invisible to users. These APIs largely served developers by unifying communication on the backend and providing powerful customer-facing capabilities on the business side. But they’re not the only ones.

    API-first SCM

    API-driven companies managed to disrupt entire markets nobody imagined could be disrupted, such as Stripe for payment processing and Slack for immediate communication (their former head of developer relations, Amir Shevat, said that Slack’s developers were literally measured by their API’s active tokens). Companies that harnessed the power of their APIs saw unprecedented growth over the last decade due to the integration power they unleashed. 

    Today, there really is no self-respecting dev-first technology that doesn’t invest heavily in the robustness of its APIs. There are even entire companies today built around API integrations—from marketplaces like Rapid API to security companies like Akita Software to companies like Kong that focus solely on simplifying API gateways.

    The API-First Dev Tool Revolution

    The same transformation that disrupted various business industries is now beginning to take center stage in the technology and dev tools space. This is also part of the developer experience revolution that is the stimulus for all the nifty dev tools in the first place. As developers become the new decision-makers—masters of their own domain—and are no longer being forced to adopt and use technology someone else purchased for them, the experience of using these tools impacts their choice of whether or not to add these tools to their chain. This means that those creating the tools today need to deeply understand the developer’s workflow and optimize their products for this user persona.

    This very same evolution that typical tooling underwent is now impacting age-old tooling in the developer ecosystem—and developer pain and friction are starting to come to the forefront. This includes the need for better and more robust API capabilities, “as-code” (infrastructure-as-code, testing and security) tooling, low-code/no-code and code variations and much more.  

    It’s interesting to see that, even with the cloud-native and DevOps revolution, our most core developer tool—our source code management—hasn’t really changed much in the last 20 years. Git remains ubiquitous, but its design is optimized for developer workflows from two decades ago. It’s long past time for an API-first SCM, and there are a number of use cases that can benefit from better APIs.

    Evolution and Anatomy of APIs to be Git-Native

    Even with the most exciting innovations we’re seeing from Git SaaS tools like GitHub and GitLab—from Co-Pilot to Codespaces, pipelines and more—they’re only as strong as the technology they are built upon. As the world evolves, this fundamental weakness is proving to be a sort of kryptonite.  

    Despite Git being an old-school, monolithic tool with proprietary protocols, Git hosting providers like GitHub, GitLab and Bitbucket have invested a lot into making their services API-driven to cater to tool integration. True, each has its own API, which drove some players integrating with the service (especially in the early days) to integrate via an SDK rather than an API.

    GitHub itself provides a variety of APIs—from the simpler REST through the more elaborate GraphQL. These APIs provide you with the ability to do pretty much anything, but are somewhat cumbersome for some use cases. To deal with the core Git functionality, you are required to directly manipulate objects in the Git database.

    Code Deployment and GitOps

    As engineering organizations grow, the need for greater control of code and infrastructure resources has become a real pain—and this is where GitOps provided the methods and the tools to support automated policy management at scale. This automated policy and governance is largely enabled, machine to machine, via APIs, and many of its controls are built on code and configuration files that are hosted via Git (hence its name: Git Operations).

    When using this methodology, a configuration change is first committed to Git (as the single source of truth that will apply changes in all the relevant places). Only then does this trigger automatic processes to apply these changes universally. In some cases, these automatic processes have to commit files to Git via API.

    Another area where SCM APIs play a key role is versioning and release management.  DevOps teams need to manage the release of new versions of their applications, including creating release notes, tagging releases and creating release branches. These processes, which sometimes take place multiple times every day, include many steps and require accuracy—a comprehensive suite of efficient APIs is critical for such an operation.

    UX Customization

    Today, there’s a huge diversity of stakeholders in modern engineering processes, particularly in the context of game development, but this is also true in any kind of web-based or UI-based engineering, as well. Today, product and design are an integral part of engineering processes, but Git still remains the domain of developers.

    There are times when you’d like to have a unified interface for your asset management that pulls data from multiple APIs and sources—such as Git for source control and Google Drive for large files. The added benefit is to abstract and decouple the underlying technology from the user experience and interface so that users can be free to migrate and move between their technologies of choice. This is true whether at the SCM or the storage level without having to change the interface that the team is accustomed to.

    APIs that efficiently perform the full set of SCM functionality allow developers to use a single backend for managing code versions while still providing end users with the desired experience, regardless of what that might be.

    Versioning-as-a-Service

    SaaS products have made it significantly easier to manage and maintain our services through simple UIs and dashboards. However, this simplicity also has a tradeoff. Have you ever made a simple configuration or code change, quickly regretted doing so and then attempted to revert it via your SaaS console? In most SaaS apps that aren’t managed as code, this happens all the time—from Okta to Datadog and many others. Without the benefits of “as-code” and the version control that comes with it, reverting changes is nearly impossible.

    The most you can achieve is “undoing” a previous configuration—if you can still catch it with a CTRL+Z—simply because engineering didn’t bother (or did not have the resources) to implement versioning. Imagine if this could come as a service for all of your SaaS applications!

    For software developers, API-based version control will unlock the possibility of easily incorporating rich version control capabilities into their product—in the same way they can currently use other capabilities such as database, file storage, etc. 

    The World is Ready for an API-First SCM

    TL;DR: API-first models have delivered some of the greatest innovations that ‘dinosaur’ industries—from payments to communications—needed to move faster and do so securely. It is almost laughable that the technology industry has inherent limitations due to a core technology that has a legacy design. This should not be a technology limitation we should accept.  

    Our source code should be able to be controlled, manipulated and transformed via API to remove human toil from places it isn’t really needed and free people to focus on higher-order problems.  With the widespread adoption of generative AI and smarter machine overlords, we’ve discovered that there are many formerly manual tasks that we can offload to machines. Free SCM APIs to unlock the next generation of innovation—because soon there will be more machines than humans to manage them. APIs are the key to tapping into this power. 

  • Unleashing the Power of AI-Engineered DevSecOps

    Unleashing the Power of AI-Engineered DevSecOps

    In my recent article Revolutionizing the Nine Pillars of DevOps with AI-Engineered Tools, I explained that AI-engineered tools can help implement the portion of continuous security practices known as DevSecOps.

    DevSecOps involves integrating security practices into the DevOps workflow. AI can be used in anomaly detection to identify potential security threats and to automate security policy enforcement. AI-engineered tools have the potential to significantly enhance security across all phases of the software value stream in DevSecOps.

    Planning: The planning stage involves dealing with complexities related to risk analysis and threat modeling. AI can help automate risk analysis and threat modeling based on data from past projects and known security issues. This can help identify potential security threats early in the planning phase and develop appropriate mitigation strategies.

    Design and Development (prior to integration): Developers often face difficulties in writing secure code, identifying potential security vulnerabilities, and adhering to best security practices while coding. AI-powered static application security testing (SAST) tools can automatically scan the source code for known vulnerabilities and coding errors, providing immediate feedback to developers. In addition, AI could be used to generate recommendations for secure coding practices and strategies for mitigating identified vulnerabilities.

    Continuous Integration (CI): Continuously integrating new code can introduce new security vulnerabilities and regression bugs that are hard to catch and fix in a timely manner. AI can be utilized in dynamic and interactive application security testing tools (DAST and IAST) during the integration stage. They can intelligently automate tests and identify complex security issues by analyzing data flows and the behavior of code in an integrated environment.

    Continuous Delivery (CD): Ensuring that the code is not only functional but also secure before it is delivered is a major challenge. AI-powered tools can help manage security configurations, monitor changes and alert to any deviations from security policies during the delivery stage. AI can also be used to automate security testing in pre-production environments to ensure that security standards are met before delivery.

    Continuous Deployment (to production): Monitoring, detecting and responding to security threats in real-time in production environments is a critical and challenging aspect. AI can enhance security information and event management (SIEM) tools by providing intelligent real-time analysis of security alerts generated by applications and network hardware. AI-based runtime application self-protection (RASP) tools can also detect and block attacks in real-time, providing robust protection for applications in production.

    Roadmap to AI-Engineered DevSecOps

    It’s crucial to implement AI-engineered DevSecOps solutions in a structured and well-planned manner. Here is a logical roadmap for implementing such a solution:

    1. Assessment and Planning: Begin by assessing your organization’s current DevSecOps maturity. Evaluate the existing tools and practices for their efficiency and effectiveness. Identify areas that lack automation or present security concerns. Align the AI-engineered DevSecOps initiative with your business objectives. This includes defining key performance indicators (KPIs) to measure the success of the initiative. Outline the required resources and budget for the implementation. This includes potential AI-engineered tools, the team’s skills upgrading needs and additional infrastructural changes.

    2. Selection of AI-Engineered Tools: Based on your assessment, select the AI-engineered tools that best fit your organization’s needs. This selection should cover the entire software development life cycle: Planning, design and development, continuous integration, continuous delivery and continuous deployment. The tools should ideally be scalable, easy to integrate with your existing systems and capable of providing real-time, actionable insights to your team.

    3. Training and Upskilling: Train your team on the selected AI-engineered DevSecOps tools. This may require technical training on how to use the tools effectively and how to interpret the insights generated by the AI. Upskill your team to enable them to work effectively within the new AI-engineered DevSecOps environment. This could involve training in AI and machine learning, security best practices and new DevSecOps methodologies.

    4. Gradual Implementation and Integration: Start with pilot projects to validate and tweak your approach. The lessons learned from these initial projects will be valuable for the broader roll-out. Gradually implement and integrate the selected AI-engineered tools into your existing DevSecOps pipeline. This should be a phased approach that allows your team to adapt to the new tools and practices.

    5. Review and Refinement: Regularly review the effectiveness of your AI-engineered DevSecOps implementation against the defined KPIs. Continuously refine your approach based on these reviews. This could involve adjusting your security policies, refining how you use your AI tools or providing further training to your team.

    6. Continuous Improvement: Keep yourself updated with the latest developments in AI and DevSecOps to ensure your approach remains effective and up-to-date. Continually improve and evolve your AI-engineered DevSecOps solution based on emerging technologies, shifting business objectives and feedback from your team.

    Avoiding Pitfalls While Implementing AI-Engineered DevSecOps

    Implementing an AI-engineered DevSecOps solution comes with several potential pitfalls that can derail the process if not appropriately managed. Here are a few of them, along with suggestions for how to avoid them:

    1. Inadequate Planning and Alignment with Business Goals: Ignoring the strategic alignment between implementing AI-engineered DevSecOps and overall business goals can lead to undesirable outcomes. Clearly define the business objectives and how AI-engineered DevSecOps supports them. Outline expected outcomes and key performance indicators (KPIs) that align with business goals to guide the initiative.

    2. Neglecting Training and Upskilling: AI tools can be complex, and without proper understanding and training, their deployment may not yield desired results. Invest in training your teams on AI-engineered DevSecOps tools and techniques. Ensure they understand the functionalities of these tools and how to effectively use them. Upskilling your team will be crucial for leveraging AI capabilities.

    3. Ignoring Change Management: Introducing AI into DevSecOps is a significant change that can disrupt workflows and resistance from the team members. Adopt a systematic approach to change management. Engage stakeholders from the beginning, communicate the benefits of the transformation clearly, and encourage feedback. Also, plan for a gradual rollout to help teams adapt.

    4. Overdependence on AI: Relying too heavily on AI can be a risk if the tools are seen as a complete replacement for human judgment. Recognize that AI is a tool designed to assist human decision-making, not replace it. It’s important to maintain human oversight of AI outputs, as AI models might not fully account for all contextual and situational factors.

    5. Neglecting Data Privacy and Ethical Considerations: Using AI requires handling a lot of data, which can lead to privacy breaches and ethical dilemmas if not handled correctly. Develop a clear policy for data management, ensuring compliance with all relevant data privacy laws and regulations. Make ethical considerations a priority when implementing AI, ensuring transparency and fairness in the AI systems.

    6. Lack of Continual Improvement and Adaptability: AI-engineered DevSecOps requires a culture of continuous learning and adaptation. Failure to embrace this can lead to stagnation and ineffectiveness over time. Establish a framework for continual improvement. Regularly evaluate the effectiveness of the AI tools and be ready to adapt your processes based on new technological advancements, feedback, and changes in business needs.

    Summary

    Implementing an AI-engineered DevSecOps solution necessitates a thorough and strategic approach. This begins with an in-depth assessment of your organization’s current DevSecOps maturity and an understanding of how AI can enhance each phase of the software value stream. Essential steps in the roadmap include tool selection, team training, gradual implementation, regular review and continuous refinement and learning. While AI can significantly bolster security and efficiency, it’s important to avoid pitfalls like inadequate planning, neglecting training, resisting change, overdependence on AI, neglecting data privacy and ethical issues and lack of continuous improvement. By focusing on these areas, aligning the DevSecOps initiative with business objectives, investing in team upskilling, managing change effectively, maintaining human oversight, adhering to data privacy regulations and fostering a culture of continuous learning, organizations can successfully implement and maximize the benefits of AI-engineered DevSecOps.

    Remember, this transformation won’t happen overnight. It is a continuous journey that requires regular review and improvement. With the right strategy, the benefits of implementing an AI-engineered DevSecOps solution can far outweigh the challenges.

  • Pulumi Embeds Terraform Conversion Tool in Platform

    Pulumi Embeds Terraform Conversion Tool in Platform

    At its virtual PulumiUP event, Pulumi this week announced it embedded an ability to convert instances of Terraform into its infrastructure-as-code (IaC) platform via a simple command line interface (CLI) command. In addition, Pulumi added a Review Stacks capability that automatically creates a dedicated cloud environment to review code every time a pull request is opened.

    Pulumi is also making available a beta release of Pulumi Azure Native Provider 2.0, which decreases the size of the software development kit (SDK) by 60%. This is part of an effort to improve deployment times and reduce resource consumption by, for example, removing support for deprecated application programming interfaces (APIs).

    Finally, Pulumi has added support for property names and values to Resource Search for Pulumi Cloud, a search and analytics tool for cloud computing environments. DevOps teams can now additionally search for instances of, for example, versions of databases or specific availability zone as a result.

    Aaron Kao, vice president of marketing for Pulumi, said previously the company made available a separate tool for converting snippets of Terraform code. For the first time, Pulumi is also making available a converter plugin in the Pulumi engine which allows conversion tools from other IaC tools to be used.

    Review Stacks, meanwhile, makes it simpler for DevOps teams to pick up an unfamiliar codebase, make changes to both application and infrastructure code and share a live environment to review that code.

    Pulumi has been making a case for an alternative to Terraform that centralizes the management and provisioning of infrastructure using a set of embedded guardrails. One of the major issues enterprise IT organizations are encountering these days is the number of misconfigurations created when developers use tools like Terraform to provision infrastructure on their own. Those misconfigurations often lead to, for example, ports being left open and cybercriminals exfiltrating data.

    That approach is gaining traction among enterprise customers that are looking to centralize the management of DevOps processes, said Kao.

    In the wake of a series of high-profile breaches, more organizations are now reviewing the security of their software supply chains, including how infrastructure is provisioned. Many of those organizations will need to revisit those processes as pending legislation that requires more stringent requirements for building software eventually become law, noted Kao.

    At the same time, many organizations are also looking to improve the productivity of DevOps teams during an economic downturn that limits the number of full-time employees they might be able to hire. The pace at which software is being developed has not slowed, so there is a greater urgency to do more with less.

    It’s not clear how quickly organizations are centralizing the management of DevOps workflows, but with the rise of platform engineering, there is clearly more interest in defining consistent backend processes that multiple DevOps teams are currently managing individually. The challenge is finding a way to achieve that goal without impeding the pace of innovation enabled by embracing DevOps in the first place.

  • 92% of Devs Use AI, Survey Says ¦ Intel One Mono Font

    92% of Devs Use AI, Survey Says ¦ Intel One Mono Font

    Welcome to The Long View—where we peruse the news of the week and strip it to the essentials. Let’s work out what really matters.

    This week: GitHub touts unbelievable AI stats, and Intel’s beautiful open source typeface. (more…)

  • How Open Source Can Enhance Developer Experience

    How Open Source Can Enhance Developer Experience

    Developer experience (DX) is becoming more and more of a competitive advantage among developer-facing tools and platforms. A great developer experience that’s easy to follow and implement can truly help a software tool stand out from the crowd.

    Simultaneously, open source software is all the rage. In fact, a recent StackOverflow survey found open source software (OSS) to be, by far, the most revered and proven technology trend.

    With that in mind, what are the synergies between developer experience and open source? And how can building in an open source format aid the developer experience for your public-facing toolsets and internal development platforms?

    I recently met with Adam Frank, SVP of product and marketing at Armory, to explore the state of developer experience and open source adoption. According to Frank, open source can bring numerous DX benefits. Below, we’ll explore these areas and consider why organizations might benefit from adopting more open source within their developer-facing initiatives.

    The State of Developer Experience and Open Source

    So, what is the state of developer experience, and why is it important now? Well, DX is continually leveraged to help improve agility and competitiveness in the market. In addition to granting differentiation, DX also enables more flexibility, helping users switch from one tool to another if necessary.

    “Developer experience is at the forefront right now because technology and change are moving faster than ever before,” said Frank. “Companies need to focus on DX as a competitive advantage to survive and thrive.” At the same time, open source can help organizations move quickly and iterate more efficiently.

    Yet, historically, open source has not been associated with fantastic DX. Most of us have at least one horror story of an open source project simply going belly-up without warning. It’s common for an OSS tool to lack comprehensive documentationor suffer from a lack of funding for ongoing maintenance, zero customer support and deprecated modules. Some tools that developers directly download and contribute to may not possess a quality DX emphasis, says Frank.

    However, the status quo for OSS has undoubtedly evolved over the years, he adds. Nowadays, there are more established groups, like Linux Foundation’s Cloud Native Computing Foundation (CNCF), with specific Technical Oversight Committees (TOCs) to oversee individual projects. The maturity of OSS ownership now ensures core projects stay relevant and receive continual support—not to mention ongoing developer experience enhancements.

    On the other side, OSS is often absorbed by a platform team to enable their internal developers to move faster or ship more code, says Frank. From this perspective, open source communities give platform engineering the opportunity to contribute back to the project, helping them solve deeper-level problems that are more specific to their environments.

    How Open Source Can Enhance Developer Experience

    Not all IT leaders are sold on the idea of building in a transparent, open source model. Yet, this enhanced transparency can bring dividends in the long run, especially where developer experience is considered. Frank shared some of the benefits of adopting open source.

    Receiving Feedback From The Community

    “As a product manager, you want as much feedback from your users as you can gather,” said Frank. Receiving user feedback is a great way to fix DX issues, and OSS gives you that on a larger scale. By adopting open source, you benefit from collecting insights from your customer base and the developer community at large. This can translate into a broader range of use cases, which can significantly improve projects and overall experience.

    Increasing Ecosystem Transparency

    Building in a more transparent and open fashion so that everyone has the ability to submit pull requests can also help you work together with potential partners. This makes it easier to develop third-party integrations that ensure the broader ecosystem can work together. By decreasing the barrier to entry through OSS, you provide others with the ability to solve more problems around developer experience.

    Streamlining the Developer Toolchain

    “Developers want to innovate; they want to solve problems,” said Frank. Open source can also help expose inefficiencies in the developer toolchain and, notably, many open source CI/CD tools are often leveraged to automate deployment and increase efficiency. The end result is that developers can focus on innovating their own competitive advantage. “Making the deployment aspect of the developer lifecycle really seamless can significantly improve their experience,” he said. Therefore, any tools or extensions around CI/CD are an excellent area to consider open-sourcing.

    Increasing Flexibility

    Another benefit of the open source model is that you have far more flexibility than off-the-shelf proprietary software. As a customer, you don’t have to wait to submit enhancement requests. With OSS, you can fork a project or quickly collaborate with the maintainers. This equates to making fixes far quicker and providing the ability to customize developer experiences to the tasks at hand.

    Best Practices to Ensure Open Source Improves DX

    Open source provides many key benefits that can enhance software development and delivery. But, this should be taken with a grain of salt, as best practices still must be followed to ensure open source enhances, not hinders, developer experience.

    For example, Frank mentions that simplicity should be a guiding light. “Have simplicity for the user experience design in the forefront of everything you’re building and thinking about,” said Frank. Adopting simplicity should help ensure developer-facing materials are structured well and documented in an easy-to-comprehend manner.

    On that note, open source projects will require standards around areas like good documentation and well-written APIs that abide by contracts, said Frank. Anything less will hinder developers and confuse them with disparate data formats. On that note, he has witnessed OpenTelemetry emerge as a common way to unify data that systems produce with standard formatting, which increases the observability of these environments.

    Lastly, it’s critical to hire talent with experience in open source. Bringing in folks that have knowledge of how OSS communities operate will make onboarding easier and encourage safer open source security practices as well.

    What the Future of DX and OSS Holds

    Open source appears to aid developer experience in many ways, from driving more innovation, opening software up to feedback and developing a closer-nit developer community. But what does the future of developer experience and open source look like?

    According to Frank, we will likely see increased data standardization that makes it easy to work with various projects and data formats. OpenTelemetry, for example, makes aggregating logging easier. We’ve also seen standardization around OpenAPI Specification for describing web APIs. Another example is FinOps Foundation’s FOCUS specification to unify cost analysis.

    Open source tends to leverage open source. And as the industry continues to coalesce around open standards, we’ll likely see more robust, compatible layers of abstraction emerge throughout the market, Frank predicts.

  • BMC Adds Generative AI Capabilities to Helix Management Platform

    BMC Adds Generative AI Capabilities to Helix Management Platform

    BMC today revealed it is embedding generative artificial intelligence (AI) capabilities across its BMC Helix service and operations management platforms.

    BMC CTO Ram Chakravarti said the company is incorporating the open source GPT-J large language models (LLMS) to build generative AI models. Those models are optimized for specific domains using data collected by BMC and imported from other data sources, like the Jira project management software from Atlassian. That persistent layer of storage is designed to aggregate multiple types of data within what amounts to a Helix data lake.

    The generative AI capabilities added to BMC’s portfolio include natural language summaries of issues and events captured by BMC Helix Operations Management, along with an ability to create a historical analysis of a user’s specific environment to identify recurring issues more easily.

    BMC Service Management, meanwhile, can now correlate incidents and provide resolution summaries that can be stored. It’s also now possible to run clustering analysis on chat archives to find new or improved utterances that help virtual agents answer queries better and execute workflows.

    BMC initially opted to build its own LLMs because general-purpose LLMs—such as the one created by OpenAI—are subject to “hallucinations.” Hallucinations are created when incorrect data is included in a model. BMC is training its LLMs on a narrow range of trusted data to ensure the recommendations and subsequent automations it creates are valid.

    Over time, BMC will add additional LLMs; the company hasn’t yet decided if there are use cases where it makes sense to rely on an existing general-purpose LLM to add additional capabilities, noted Chakravarti.

    Organizations can also opt to use a containerized version of the BMC LLM if they prefer, but BMC will encourage most organizations to invoke generative AI via its Helix SaaS platform, he added.

    BMC Helix is an IT services management (ITSM) platform that integrates with a range of DevOps platforms. Helix is part of a larger existing BMC Connected Digital Ops initiative that seeks to unify ServiceOps and DevOps with AIOps, DataOps and AutonomousOps. Generative AI will automate a range of tasks from within Helix that can then be extended to DevOps platforms such as Jenkins.

    It’s still early days as far as applying generative AI to processes, so organizations should focus on use cases where it will add some immediate value, said Chakravarti. Once they develop a level of trust in the automation created and executed, they can then pragmatically expand the number of generative AI use cases where they make sense, he added.

    One way or another, generative AI is going to be applied to the management of IT. Many routine tasks that conspire to limit the productivity of IT teams today will be eliminated, but there will always be a need for a human to remain in the loop to ensure the right outcomes are achieved. However, it will also soon become possible for existing teams to manage IT at levels of scale that were previously unattainable.

    Image source: Photo_Credite_Indira-Tjokorda via Unsplash

  • It Is Time for DevOps to Focus on Ops

    It Is Time for DevOps to Focus on Ops

    The advent of both Agile development and DevOps were great steps in improving an IT process that had become ossified, with policies that alternated between resistance to change and trying to do things better/faster. Between them, Agile and DevOps broke the cycle of resistance … but only for development.

    The confusion around CI/CD is a great example. CD is both continuous delivery and continuous deployment. Do you know why? Because up to that point (and even recently), DevOps included Ops in the name but did little with or for operations. I have no doubt that it was originally intended to include everything from build testing to full-on deployment, and we are finally getting there.

    You have no doubt noticed that both operations and security are finally getting DevOps’ attention at critical points. I have been chatting with coworkers about value stream mapping and, if you map the value stream of a software delivery process, you will find that all of the weak points have shifted right. Thankfully, we are also paying attention to them these days.

    From GitOps to infrastructure-as-code (IaC); from cloud-first to policy-as-code (PaC), we are finally approaching the operations side with some level of seriousness. And it is generating results for those involved.

    My two areas of attention these days are DevOps and application security (AppSec). As I’ve written elsewhere, both are benefitting massively from things like policy-as-code. The ability to apply corporate deployment policies, security policies and compliance policies from a centralized location is massive. It is early days, but community contributions to PaC libraries are already massive, making it easier to deploy without recreating that big old list of generally useful policies.

    Given a project, a set of tools and a timeline, you could fully automate the process from the first line of code to running in production, with stops for approval and the specialized steps that still require human interaction (no matter how many public policies are out there, someone with knowledge of your network still has to set up IPs or ranges, for example. Or someone still needs to write the code). Someday, it may be possible to say, “We need a phone app that targets both major platforms and is hosted on Linux that does X, Y, Z,” and it just happens … but we are not there yet, and, IMO, the domain knowledge will be a roadblock for a while. But you can take advantage of what we have to work smarter, not harder. And the organization will be better off for your efforts.

    So keep at it, start to evaluate your processes and consider placing any of the great new DevOps technologies into your path. Let them do some of the grunt work so you can focus on adding value. And for all those coworkers that still don’t get it, thank you. You are keeping them in a job, whether they admit it or not.