devopsy.kinsta.cloud

Category: Blogs

OpenSSF Siren: Security for One, Security for All
It’s getting harder by the day to ensure your programs are safe. We used to count on the National Institute of Standards and Technology (NIST) Common Vulnerabilities and Exposures (CVEs), but NIST has almost completely stopped adding analysis to the CVEs. Plus, as CEO and co-founder of coding security company Chainguard Dan Lorenc pointed out, the CVEs themselves are drowning under a flood of bad CVEs, created by AI bots seeking security credibility for their authors. What’s a DevOps pro to do?

The Open Source Security Foundation (OpenSSF) has an answer: Siren, a threat intelligence sharing list.

Security mailing lists, such as oss-security, are nothing new. The OpenSSF is well aware of these. Indeed, it supports them.

Individual lists for particular problems are well and good. However, these days, as most of you know to your pain and sorrow, security issues seldom concern one specific program. They can spring from combinations of operating systems and programs.

The open source community has proven methods of communicating vulnerabilities to others within the community. However, developers don’t have a way to efficiently communicate information about exploits with the broader downstream audience. While consumers and enterprises may have intelligence-sharing structures in place, this does not always extend to the upstream open source community. OpenSSF Siren seeks to bridge these gaps.

As the OpenSSF states, Siren is meant to provide “a secure and transparent environment for sharing Tactics, Techniques, and Procedures (TTPs) and Indicators of Compromise (IOCs) associated with recent cyber attacks. Siren is intended to be a post-disclosure means of keeping the community informed of threats and activities after the initial sharing and coordination.”

So, Siren is the wrong place if you’re looking for information about zero-day security holes before they are disclosed. This isn’t for sharing security problems and answers once they are disclosed and, we hope, fixed.

Since open source powers up to 90% of modern software, the importance of this can’t be understated. Recent attacks on projects like XZ-Utils are stark reminders of the importance of proactive security measures.

Specifically, Siren brings you:
- Open source threat intelligence: Information is shared with the community about actively exploited public vulnerabilities and threats.
- Real-time updates: List members receive email notifications about emerging threats relevant to their projects. This gives you an opportunity to take action to mitigate risks swiftly.
- Adherence to CLEAR guidelines: To facilitate effective, unrestricted, transparent communication, the list follows the Traffic Light Protocol (TLP) CLEAR guidelines for the sharing and handling of intelligence. TLP is a simple color-coded system that designates how far the information should be shared. Red, for example, means the information should only be shared by people actively working on a security problem, while green means the security news should be restricted to the community.
- Community-driven resources: Contributors from diverse backgrounds collaborate to enrich the intelligence database, fostering a culture of shared responsibility and collective defense.
By leveraging the collective knowledge and expertise of the open-source community and other security experts, the OpenSSF Siren hopes to empower projects of all sizes to bolster their cybersecurity defenses and increase their overall awareness of malicious activities.

Ready to act? You can register for an OpenSSF Siren membership to receive real-time threat intelligence updates and talk about these issues. I hope to see you there. Together, we can make the world a little bit more secure for you, me, and everyone else.

Photo credit: Jan Antonin Kolar on Unsplash
May 21, 2024
A Gentle Introduction to Continuous Monitoring and Optimization

DevOps monitoring provides a thorough and up-to-date picture of the production environment’s condition and details on its services, infrastructure and applications. By gathering data from logs and metrics, you may monitor compliance and performance at every step of the software development lifecycle.

Monitoring isn’t just for production issues. It encompasses several procedures, such as planning, development, testing, deployment and operation.

A Roundup of Monitoring Types

The uses of continuous monitoring expand along with the size of businesses’ technological stacks. Many companies begin by tracking fundamental metrics, such as CPU utilization or customer behavior.

Infrastructure monitoring: You can gather and analyze data from your organization’s internal IT infrastructure and use that information to improve performance or other metrics. Some items to monitor are networks, servers, computer hardware and software, data centers, operating systems and storage. Common infrastructure monitoring tools include SolarWinds, ManageEngine, and Prometheus.

Network monitoring: Network monitoring focuses on finding errors, gauging components’ effectiveness and enhancing their usefulness. Everything on the network is monitored, including firewalls, servers, virtual machines and routers. A dynamic network monitoring system can help avert errors and outages that could harm performance. A few well-known utilities are Spiceworks, Cacti and Wireshark.

Application performance monitoring (APM): Is the software running fast enough? If not, where does the problem originate? You can obtain runtime metrics about an application’s performance, such as application uptime, security and log monitoring. APM solutions such as DataDog, Uptrends and Splunk are common tools for exploring many types of information, including API replies, backend application health, transaction volume and time metrics.

Cost monitoring: The DevOps pipeline entails many elements, each of which could be expensive. Any organization that wants to control costs – that’s everyone, right? – suggests that continuous monitoring of resource utilization is an essential need. These indicators help predict the overall cost and maximize resource use throughout DevOps stages.

Why DevOps Monitoring Matters

Since DevOps monitoring is proactive, it finds opportunities and gaps to enhance application performance before defects’ symptoms become apparent. By emphasizing potential areas for automation, monitoring further enhances the DevOps toolchain. With the aid of API-driven code, proper system monitoring offers insightful information that lets you see every element of your application stack. The monitoring process is enhanced when code hooks are incorporated into the application logic.

Automation: Enhancing communication between the development and operations teams is the central tenet of DevOps. However, coordination across teams may come to a standstill if there is a shortage of connectivity between the tools. Therefore, you can track commits and pull requests to keep track of Jira issues and alert the development team on its preferred communication channel, or you can use automation to enable a unified view of the complete development pipeline.

Visibility and transparency: A workflow has thousands of moving elements, each of which works at a different size and experiences varying latency and redundancy due to the introduction of micro-services and micro frontends. CI/CD allows developers to make frequent changes to their code, which adds to the complexity of a production system. Consequently, teams need ecosystem insight to repair any harm to the user experience as soon as it occurs.

Quality experience: A DevOps team that employs continuous monitoring technologies to manage its systems can minimize system downtime and business disruptions. This raises the bar for the quality of IT experience for internal and external stakeholders, such as staff members, business partners and clients. It optimizes business performance over time in every department of the company.

Doing DevOps Monitoring Right

Every organization has slightly different habits. Nonetheless, there are a few universally applicable principles.

Set clear goals: What is the overall goal of the DevOps monitoring implementation? Is it to improve system performance, address issues or enhance the user experience? Identifying these objectives helps you relate your monitoring goals to your targeted outcomes.

Select the correct monitoring tools: The DevOps team should consider some variables, including support for common monitoring tasks, scalability, user-friendliness and integration. Make sure they can adjust to the shifting needs of your DevOps workflows.

Gather suitable data: Concentrate on gathering and evaluating critical metrics. Steer clear of massive data volumes. Not all data is valuable, and having too much information might confuse. The DevOps team should rank these metrics according to how they directly affect the objectives. Doing so leads to better security, lower error rates, quicker response times and better resource usage.

Document everything: If you are serious about a culture of collaboration, properly document your monitoring procedures and encourage knowledge sharing among teams. This strategy promotes team members’ accountability and ownership of the monitoring procedures and activities.

Continuously review results: Ongoing analysis and improvement should be part of the process. Assess your monitoring plan to make sure it is adaptable and aligns with your goals. Use the data insights and user input to refine your monitoring methods.

A Few Sample Use Cases for DevOps Monitoring

Without a doubt, DevOps monitoring has several advantages for any business. But its appropriate use is necessary for its total success. Here are some of the use cases for DevOps monitoring.

Monitoring Git Workflows: Codebase conflicts can arise when numerous developers are working on the same project. Git can resolve these issues by utilizing rollbacks and commits. The Git process monitors these conflicts and maintains continuous development.
Code Instrumentation: The process of adding code to a program to track its performance and functionality is known as code instrumentation. Observing contextual values and monitoring stack calls are essential. DevOps processes can be evaluated for effectiveness and flaws. In this situation, testing and bug identification are critical factors.
Continuous Integration and Deployment Logs: Doing so facilitates troubleshooting failed deployments and allows for the resolution of difficulties. Consult the log files generated by continuous integration systems, especially their warnings and errors, and the continuous deployment logs to monitor the general health of the development pipeline.

Final Thoughts

As DevOps-managed services evolve quickly, so should monitoring solutions, given the increasing complexity and depth of software development. Making an informed decision regarding your development projects may require you to choose the best monitoring system that comprehends the numerous parts of an application and how they interact. Hiring competent service providers is therefore advised to guarantee you take the right action at the right time to gain the maximum return on investment.

May 21, 2024
Memcyco Report Reveals Only 6% Of Brands Can Protect Their Customers From Digital Impersonation Fraud

New York, NY, May 21st, 2024, CyberNewsWire

Memcyco Inc., provider of digital trust technology designed to protect companies and their customers from digital impersonation fraud, released its inaugural 2024 State of Website Impersonation Scams report. Notably, Memcyco’s research indicates that the majority of companies do not have adequate solutions to counter digital impersonation fraud, and that most only learn about attacks from their customers.

More than half of all respondents (53%) said their existing cybersecurity solutions do not effectively address website impersonation attacks, and 41% said their existing solutions only protect them and their customers “partially.” Just 6% of brands claimed to have a solution that effectively addresses these attacks despite 87% of companies recognizing website impersonation as a major issue and 69% admitting to having had these attacks carried out against their own website.

The creation of fake websites used for phishing-related attacks (which are a top cause of account takeover (ATO)) is a growing problem that has earned cybercriminals an astonishing $1 billion+ in 2023 alone, according to data from the U.S. Federal Trade Commission. That’s more than three times the amount reported stolen in 2020.

The report found that 72% of companies have a monitoring system in place to detect fake versions of their website, but still, 66% said that they primarily learn about digital impersonation attacks when they are flagged by customers.. More alarmingly, 37% of respondents learn about website impersonation attacks as a result of “brand shaming” by impacted customers on social media.

The inability to adequately protect against digital impersonation fraud raises a question about companies’ responsibility to reimburse their customers. 48% of survey respondents are already aware of upcoming regulations likely to enforce customer reimbursements, making effective protection against digital impersonation fraud a ‘must-have’ for avoiding revenue loss.

“One of the most alarming takeaways from the report is that website impersonation scams are growing because attackers rely on companies having limited visibility into these kinds of attacks,” said Israel Mazin, Chairman and CEO of Memcyco. “This creates a glaring blindspot in cybersecurity — the inability of companies to protect their customers online.”

The State of Website Impersonation Scams report was conducted together with Global Surveyz Research, based on the responses of 200 full-time employees ranging from Director to C-level executives at organizations in the security, fraud, digital, and web industries, operating transactional websites with traffic of more than 10,000 monthly visits.

Memcyco’s solution suite addresses the rising tide of website impersonation scams by using real-time alerts to secure end-users on every website visit and provides organizations with unparalleled insights into the scope and impact of all attacks on their sites.

The full report can be found here.

About Memcyo

Memcyco offers a suite of AI-based, real-time digital risk protection solutions for combating website impersonation scams, protecting companies and their customers from the moment a fake site goes live until it is taken down. Memcyco’s groundbreaking external threat intelligence platform provides companies with complete visibility into the attack, attacker, and each individual victim, helping to prevent ATO fraud, ransomware, and data breaches before they occur. Memcyco’s “nano defender” technology detects, protects, and responds to attacks as they unfold, securing tens of millions of customer accounts and reducing the negative impact of attacks on workload, compliance, customer churn, and reputation.

About Global Surveyz

Global Surveyz is a global research company providing survey report-as-a-service that covers the whole process of creating an insightful and impactful B2B or B2C report for any target market. Global Surveyz was established in 2020 and is the brain-child of Ramel Levin.

Contact

Sheena Kretzmer
sheena@memcyco.com

May 21, 2024
Defining DevSecOps for IoT

The Internet of Things (IoT) has brought plenty of benefits to companies and consumers, allowing businesses to track inventory in their supply chain, landlords to measure their tenants’ energy consumption, and pet owners to find a lost cat. It’s become a significant part of the Internet infrastructure, too. The number of global IoT connections grew by 18% in 2022 to 14.3 billion active IoT endpoints, with no end in sight.

But for all the technology’s advantages, it also presents new security challenges. IoT technology uses network connections to facilitate communication between devices and systems. When devices are IoT-enabled, they can collect and transmit data in real-time, enhancing efficiency, safety and convenience across various applications and industries. All that enables great technological feats… and gives bad actors new opportunities.

IoT device scale and diversity: Each device, from smart refrigerators to industrial sensors, operates with different technical architectures and security capabilities. This diversity makes it difficult to implement a one-size-fits-all security strategy. The sheer number of devices connected to the Internet makes it challenging to secure every endpoint against potential breaches.

Proliferation of security vulnerabilities: IoT security vulnerabilities typically arise from inadequate security design and practices. Many IoT devices lack robust security features due to cost constraints or oversight, leaving them susceptible to attacks. Devices could be shipped with default, easy-to-guess passwords, or without the capability to be updated or patched, making them easy targets for hackers.

IoT ecosystem complexity: IoT ecosystems typically involve multiple interconnected devices and systems, each adding layers of complexity in terms of management and security. Managing these systems requires careful coordination and a clear understanding of how information flows and is processed within the network. This complexity often results in security being an afterthought or being inconsistently applied.

Decentralized network devices: As security threats evolve, IoT software must adapt through regular updates to address new vulnerabilities. However, the decentralized and widespread nature of IoT devices complicates the updates’ delivery. Ensuring all devices are consistently updated is challenging when devices are geographically dispersed or lack remote update capabilities.

Enter DevSecOps

Most organizations that are cognizant of security are familiar with the concept of DevSecOps and likely employ people in that role. DevSecOps introduces a security-first mindset to the DevOps process, ensuring that security and compliance measures are integrated from the start. The main objective is to create a collaborative environment where the security, development, and operations teams work together to address security issues, reducing the potential risks and vulnerabilities.

As in other software development projects, it’s important for programmers and system designers to work with the security team and DevSecOps in particular.

Incorporate automated security testing tools into the CI/CD pipeline: This helps address security throughout the IoT development process. These tools automatically scan for security vulnerabilities in the code from the earliest stages of development through to deployment, making it easier to identify issues before production.

Ensure all IoT devices are securely provisioned: By securing these devices from the point of manufacture, it becomes harder for unauthorized users to gain access or compromise the device.

Implement MFA: Multi-factor authentication (MFA) requires users to provide two or more verification factors to gain access to IoT devices, adding an additional layer of security beyond just username and password. This authentication method is particularly important in IoT environments, where devices often collect or disseminate sensitive data.

Establish regular update processes: Establishing a process for regular firmware and software updates is critical in maintaining IoT security. These updates often contain patches for recently discovered vulnerabilities that could be exploited by attackers. Ensuring that IoT devices are always running the latest version of software helps protect them from such threats and maintains system stability and performance.

Plugging In

The integration of DevSecOps principles into IoT development is useful for addressing the myriad security challenges posed by the diverse landscape of IoT devices. By embedding security practices at every stage of the IoT development lifecycle, from design through deployment, organizations can significantly enhance the security and integrity of their IoT ecosystems.

This approach not only mitigates risks such as unauthorized access and data breaches but also supports the seamless and secure operation of IoT devices. As the IoT continues to evolve, embracing DevSecOps will be crucial for ensuring that innovations in this space are both robust and reliable, safeguarding the digital and physical assets they control.

Photo by Jorge Ramirez on Unsplash

May 20, 2024
Unlocking DevOps Mastery: A Comprehensive Guide to Success

In the dynamic realm of software development, achieving mastery in DevOps requires a nuanced understanding of its core principles and a strategic approach to implementation. DevOps is a collaborative approach that combines software development (Dev) and IT operations (Ops) to automate and streamline the software delivery process. It aims to shorten development cycles, increase deployment frequency and deliver high-quality software efficiently.

At the heart of a successful DevOps program lies a trifecta of essential elements: Automation, collaboration and governance. By mastering these pillars, organizations can unlock DevOps’ full potential, accelerating delivery, enhancing reliability and fortifying security.

Enhancing DevOps with Automation

Efficiency is a high priority for IT departments, according to a survey of IT leaders conducted by Rocket Software. Automation is the linchpin of DevOps because it enables organizations – and humans – to streamline workflows, minimize manual intervention, and accelerate software delivery. By automating mundane tasks such as code compilation, testing and deployment, teams can focus on innovation and problem-solving. Automation ensures consistency and facilitates rapid deployment. Ultimately it can enhance efficiency and enable continuous integration and delivery.

However, automation is not a one-size-fits-all solution. To realize its full potential, it requires careful planning, meticulous execution and ongoing refinement. To be effective, organizations must invest in robust infrastructure, tools and frameworks and encourage a culture of experimentation and continuous improvement.

Collaboration: Building Bridges Across Silos

DevOps is about breaking down silos, fostering collaboration, and promoting a culture of shared ownership and accountability. That isn’t about tools; it’s about people. DevOps is as much about culture as it is about technology. Open communication channels, facilitated by collaboration tools and platforms, can help cross-functional teams to work together toward common goals.

Effective collaboration extends beyond the boundaries of development and operations, encompassing stakeholders from across the organization. Every stakeholder has a role in the DevOps journey, from product management to quality assurance to customer support. Each person plays a vital role, contributing their expertise and perspective to the process. Each function is integral to the success of DevOps implementation, whether it’s someone from product management envisioning the roadmap, quality assurance ensuring product reliability or customer support addressing user needs.

Breaking down silos and fostering collaboration among these diverse teams is essential. You need a culture of transparency and inclusivity, where information flows freely and everyone feels empowered to contribute, create and offer opportunities for improvement. In a collaborative environment based on trust, barriers are removed, communication is streamlined and teams are aligned toward a shared goal of delivering high-quality, customer-centric solutions efficiently and effectively.

Governance and Security: Safeguarding the Software Supply Chain

In an era of increasing cybersecurity threats and regulatory scrutiny, governance and security are paramount concerns for organizations embracing DevOps. Integrating robust governance frameworks and security practices into the DevOps pipeline is essential to ensuring compliance, managing risk and safeguarding the integrity of the software supply chain.

From code analysis and vulnerability scanning to access control and identity management, organizations must implement comprehensive security controls to mitigate risks throughout the software development lifecycle. Furthermore, compliance with industry standards and regulatory requirements must be baked into the DevOps process from the outset rather than treated as an afterthought. Moreover, organizations must be vigilant about ethical considerations and algorithmic bias in environments leveraging AI and machine learning, where the stakes are heightened. By embedding security and compliance into every stage of the DevOps pipeline, organizations can build trust and confidence among stakeholders and mitigate potential risks to their reputation and bottom line.

DevSecOps, an extension of DevOps, emphasizes integrating security practices throughout the software development lifecycle (SDLC). Several key security practices and frameworks should be integrated into the DevOps program. These include implementing security automation tools for continuous testing, code analysis practices for identifying vulnerabilities in the source code and continuous monitoring solutions to detect and respond to security threats in real-time.

Additionally, organizations should provide ongoing security training and awareness programs to educate stakeholders about secure coding practices and emerging threats. Compliance with relevant security standards and regulations, container security practices, robust identity and access management policies, secure configuration management and adherence to governance frameworks are also essential. By incorporating these security practices into their DevOps program, organizations can establish a secure and resilient software development pipeline, effectively mitigating the risk of security breaches and protecting critical assets and data.

Mastering DevOps requires a holistic approach encompassing automation, collaboration and governance. By leveraging automation to drive efficiency, fostering collaboration to break down silos, and integrating governance and security to mitigate risks, organizations can unlock the full potential of DevOps and deliver value to customers with speed, reliability and security.

Cultivate a Culture of Learning

However, achieving DevOps mastery is not a destination but a journey — a continuous evolution fueled by a commitment to excellence and a willingness to embrace change. Organizations must cultivate a culture of learning and experimentation, empower their teams with the tools and resources they need to succeed and foster a spirit of collaboration and innovation that transcends organizational boundaries. By embracing these principles and practices, organizations can chart a course toward DevOps mastery and position themselves for success in an increasingly competitive and fast-paced digital landscape.

May 20, 2024
Five Great DevOps Job Opportunities

DevOps.com is now providing a weekly DevOps jobs report through which opportunities for DevOps professionals will be highlighted as part of an effort to better serve our audience.

Our goal in these challenging economic times is to make it just that much easier for DevOps professionals to advance their careers.

Of course, the pool of available DevOps talent is still relatively constrained so when one DevOps professional takes on a new role it tends to create an opportunity for others.

The five job postings shared this week are selected based on the company looking to hire, the vertical industry segment, and, naturally, the pay scale being offered.

We’re also committed to providing additional insights into the state of the DevOps job market. In the meantime, for your consideration.

SimplyHired

Houghton Mifflin Harcourt
Boston
Senior DevOps Engineer
$157,643 to $160,000

Dice.com

Mass Mutual Financial Group
Boston
DevOps Architect
$137,800 to 180,800

LinkedIn

CrowdStrike
Sunnyvale, CA
DevOps Engineer III
$115,000 to $180,000

Indeed.com

Salesforce
Herndon, VA
DevOps Engineer
$114,200 to $221,000

Discover
New York City
Principal Application Engineer – DevOps
$104,000 to $175,600

May 20, 2024
Security, Automation and Developer Experience: The Top DevOps Trends of 2024

If you ask most folks to describe the top DevOps trends in 2024, you’ll likely hear buzzwords like AI or DevSecOps.

Those are certainly trendy topics. But based on my everyday work helping businesses plan and execute DevOps strategies, I’m noticing a different set of salient trends in DevOps. Although much is being said about how technologies like AI might impact DevOps, the biggest changes I’m seeing right now involve other types of solutions and techniques.

Here are what I view as the three most important current DevOps trends, along with a breakdown of how they may change DevOps tools and processes.

Policy-Based Management and IaC Drive DevOps Security Innovation

Security has always been a priority for DevOps practitioners. But right now, I’m seeing DevOps teams adopt new strategies to improve the security of the applications and environments they support.

One large-scale change is the greater use of cloud-based policy management to enforce security best practices in cloud environments. Teams configure cloud services and resources using the code-based configuration frameworks that cloud providers support, then scan the configurations to detect risks.

This approach makes it possible to enforce cloud governance consistently, centrally and automatically. Instead of simply writing governance policies and hoping that engineers remember to follow them when they configure cloud resources, businesses are building automated governance guardrails via policy-based management.

In a similar vein, more and more of the DevOps teams I work with are embracing static code analysis of infrastructure-as-code (IaC) templates as a means of detecting risks. As with policy-based management of cloud resources, an IaC-centric approach to infrastructure provisioning makes it possible to automate infrastructure management and identify security risks earlier in the development lifecycle.

Some teams use cloud policy and IaC code scanning to warn one another about security policy changes that might cause an application or service to break. They do this by inserting configuration changes into their code using “audit” mode if their configuration framework supports it, or by configuring IaC scanners to flag changes if an audit feature is not available. This allows engineers to detect whether a change might cause a problem for an existing deployment.

This is important because security and application teams operate separately in many organizations. When the two groups lack an efficient way to communicate about changes, they can disrupt each other’s operations – the “right hand/left hand” problem. Configuration scanning provides a systematic way to ensure that each group is on the same page when it comes time to introduce changes – and the “audit” mode approach provides a grace period that allows time to react before a change takes effect.

Doubling Down on DevOps Automation with GitOps

Another overarching trend currently reshaping DevOps is using GitOps to make DevOps automation more efficient and consistent.

GitOps is the use of Git (or a similar source control system) to manage automated DevOps workflows. It involves defining configurations using code and applying them through features like GitHub actions.

When you opt for GitOps, you move DevOps automation controls from individual workstations to centralized source control repositories. The result is the ability to track and manage all automated workflows via a central hub. Doing so increases efficiency and mitigates issues like different engineers working with different versions of the same automation frameworks on their personal desktops.

In addition, GitOps automatically generates records of what changed and how it changed, because every action is logged through the source control system. This isn’t exactly documentation in the traditional sense and means that GitOps comprehensively documents every change – which is beneficial because human engineers tend not to be so thorough when documenting their actions.

To be sure, GitOps is not without its challenges. Implementing GitOps effectively requires additional skills – namely, expertise with both IaC frameworks and source control systems – that not all DevOps engineers possess. I also notice a tendency on the part of some teams to set up GitOps pipelines but rely on manual approvals instead of automated actions to trigger changes. That approach largely undercuts the value of automating pipelines in the first place.

However, these are challenges that teams can solve through education and by fully leaning into GitOps. In addition, techniques like automated testing of GitOps configuration code can help to build teams’ confidence in automations and reduce reliance on manual approvals.

Going forward, expect to see more adoption of GitOps techniques among teams seeking to level up their approach to DevOps automation. Automating individual DevOps processes (such as software testing and deployment) won’t be enough; truly efficient organizations will turn to GitOps to automate their entire DevOps workflows, from end to end.

Investing in Developer Experience

Making software delivery processes more predictable and efficient is merely a step toward DevOps’s ultimate goal, which is to help developers become productive and satisfied with their jobs.

To that end, I’m noticing interest and investment in developer experience right now. This is playing out through two interrelated initiatives.

One initiative is platform engineering. It involves creating DevOps teams that specialize in certain functions, such as network management or security. Those teams are designated to support those functions throughout the organization. This approach reduces cognitive overhead for developers by freeing them from the need to handle types of work outside their main focus. In other words, instead of forcing developers to be DevOps generalists, platform engineering lets different teams focus on doing what they know and enjoy best – leading to greater productivity and higher levels of job satisfaction.

The other major developer experience trend currently playing out is developer self-service. This means developers can obtain the technical solutions they need on demand without a complicated procurement process. In most cases, organizations enable self-service by implementing internal development platforms (sometimes called internal developer portals) (IDPs) which host ready-made infrastructure resources and software environments that developers can deploy on a self-service basis.

There are risks inherent in these trends. They require specialized types of skills. When poorly implemented, platform engineering and IDP solutions can create more problems than they solve. However, when you ensure that your teams have the requisite expertise, and deploy a carefully planned IDP that gives developers access to the resources they need, you’re likely to see a significant reduction in friction within your organization and a boost in developer productivity and happiness.

Real-World Changes, Beyond the Hype Cycle

Admittedly, discussing DevOps trends centered on security, automation and developer experience may not be as exciting as debating whether AI will take away DevOps engineers’ jobs. But if you want to know what’s truly changing in the world of DevOps – as opposed to which conversations are most hype-worthy – these are the places to look.

Security, automation and developer experience are also among the domains of DevOps where there is a great deal of opportunity to innovate – and, indeed, where adopting new tools and techniques will be critical for organizations that don’t want to be left behind as DevOps evolves.

May 17, 2024
Harness Survey Surfaces Raft of DevOps Challenges

A survey of 500 software engineering leaders (38%) and practitioners finds nearly half reporting they can’t release code to production without risking failures, with 39% noting their code fails to push to production at least half the time.

Conducted by Wakefield Research on behalf of Harness, a provider of a continuous integration/continuous delivery (CI/CD) platform, the survey also finds that when code does need to be rolled back more than two-thirds resort to manual processes.

Overall, 60% of respondents also noted they are still releasing code on a monthly or quarterly basis. A full 59% of developers said application security requirements limit their ability to release code frequently.

Nick Durkin, Field CTO for Harness, said one of the primary reasons that security remains a bottleneck is mainly because too much responsibility for it has been shifted left toward developers, versus relying on DevSecOps platforms infused with artificial intelligence (AI) to automatically identify and resolve issues in a way that empowers everyone involved in software development.

In general, AI should be used to eliminate toil such as creating and running tests as part of a larger effort to eliminate the need to babysit tools and platforms, added Durkin.

For example, 42% of the practitioners surveyed said deploying code isn’t fast or efficient, with 44% noting that testing code end-to-end isn’t efficient. More than two-thirds (67%) said they need to wait a week to completely test code and, even then, 32% said there isn’t high unit test coverage.

More than half (52%) of the developers surveyed attributed burnout as a primary reason their peers left their jobs, with just under a quarter (23%) working overtime for at least 10 days a month. A full 97% said they regularly switch contexts between on average 14 different tools.

Well over half (54%) reported it takes longer than a week to learn a new tool. A full 59% said it takes a week to build internal tooling. More than a quarter (28%) said it takes a day to build an artifact.

Nearly two-thirds (62%) of developers have also experienced scope creep as requirements expand, which makes them less confident to execute.

Report Finds Onboarding New Hires Takes Too Much Time

New hires don’t seem to provide any immediate relief either, because onboarding them takes an average of 100 days, according to 71% of respondents.

Additionally, 40% of developers said their organization doesn’t enforce good security and governance policies. A total of 41% don’t have automated security and governance policies, while 42% said they don’t have robust identity and access management policies.

A full 62% of the executives surveyed said they would prefer to address these issues by adopting an integrated DevOps platform but the rate at which that transition is occurring is unclear. Many organizations have invested heavily in legacy DevOps platforms that they are often reluctant to rip and replace. In many cases, organizations are transitioning to a modern DevOps platform for new application development projects while continuing to employ legacy platforms to update existing applications.

Very few would argue that software development workflows are optimal. The challenge is determining how best to move forward from where most organizations are today, in a way that minimizes as much disruption as possible at a time when organizations have never been more dependent on software.

May 17, 2024
GitLab Readies Enterprise Edition of AI Tools for DevOps

GitLab this week revealed that, in the coming months, it will add an enterprise edition of its artificial intelligence (AI) add-on for its namesake continuous integration/continuous delivery (CI/CD) platform.

Duo Enterprise combines the existing ability to make code suggestions and maintain privacy controls using large language models (LLMs) with additional capabilities that enable DevOps teams to proactively detect and fix security vulnerabilities, summarize issue discussions and merge requests, eliminate bottlenecks and enhance team collaboration.

A dashboard also provides a value stream forecasting capability to provide insights into its usage of AI features and their impact on the metrics being tracked by a DevOps team.

Additionally, GitLab Duo Enterprise provides the option for self-hosted model deployments to support organizations that cannot connect their secure, air-gapped environments to internet-enabled services.

Paul Nashawaty, practice lead for application development and modernization at The Futurum Group, said capabilities such as vulnerability explanation and automated remediation will further the usage of AI across the software development lifecycle (SDLC). At present, only 18% of organizations use AI in production applications, he noted.

Separately, GitLab this week also made generally available a CI/CD catalog to provide a centralized portal through which DevOps teams can discover, reuse, and contribute pre-built CI/CD components. In addition to the public catalog, organizations can create a private catalog to distribute customized pipelines that automate specific workflows.

In addition, GitLab is readying a GitLab 17 update that will include observability tools, project planning capabilities, a secrets manager that runs natively and integrations with static application security testing (SAST) tools.

That update will also add a registry to enable data scientists to develop AI models on the same platform where engineers build, test, secure and deploy code.

Finally, GitLab is making an edition of its CI/CD platform for the Google Cloud Platform that runs in a private cloud to address compliance requirements.

Apply AI to Manage DevOps Workflows

As more developers rely on AI tools to write code the overall size of the codebase that DevOps teams will be required to manage is only going to increase. As that occurs, DevOps teams will need to apply AI to help manage the DevOps workflows through which that code eventually winds up being deployed in production environments. Most organizations won’t be able to hire a small army of software engineers so the only way to rise to the challenge will be to rely more on AI. Less clear is to what degree that requirement will drive organizations to replace their existing CI/CD platforms in favor of more modern alternatives that have AI capabilities.

In the meantime, however, DevOps teams should be preparing now for an AI era where many of the routine tasks that today conspire to slow down the pace at which applications are built and deployed are eliminated. The challenge then becomes determining how DevOps teams augmented by AI tools will revamp their existing workflows to enable more applications to be simultaneously deployed and updated at a level of scale that a few years ago would have seemed unattainable.

May 17, 2024
Building Toward Unified Observability
Gartner coined the term AIOps in 2016 to describe the use of machine learning (ML) analytics technology by IT operations to improve availability, performance and efficiency. AIOps has made significant strides since then and has achieved a new level: Unified observability, which shifts IT operations from reactive monitoring to proactive IT management.

IT operations are quickly evolving. According to a recent Digitate survey, 90% of IT decision-makers across all industries plan to implement more AI and automation, and 90% of planned initiatives in IT-related AI and automation are expected to occur this year.

Getting the Technology Part Right

AI is poised to revolutionize IT Ops, ushering in an automation and intelligent management era. However, this path to the future isn’t without its challenges.

On the one hand, businesses need to get the technology part right. Observability goes beyond the scope of monitoring and requires a deeper understanding of how different components and systems interact.

Challenges for organizations include:
- System complexity: Customers can struggle, especially when navigating how to capture, document and maintain relationships in a system of record – often a configuration management database.
- Diverse data sets: Customers also encounter issues when defining the scope of observability, integrating data from diverse sources, and ensuring compatibility with existing tools that support varied data formats.
- High data volume: Some applications and platforms generate gigabytes of information daily, in the form of logs, metrics and traces, while in others there are massive blind spots that require more critical attention.
- Expectation misalignments: Some organizational stakeholders may expect instantaneous setup of a fully intelligent and autonomous system when they engage with AI-driven IT Ops. This is not how it works in practice. No tool in the market provides instantaneous results without some level of customization.
It is important to start such projects with the right set of expectations, in phases, and with different teams involved to collaborate instead of continuing to deploy their solutions in silos.

The Role of AI in Observability

AI takes observability to the next level. It can analyze vast datasets to discern patterns and anomalies, correlate the results, and then forecast and even predict potential issues.

While AI doesn’t provide an instantaneous setup of a fully intelligent system, it certainly aids in making sense of data by identifying hidden dependencies, capturing normal behavior and doing impact analysis. In case of a system failure or anomaly, AI helps IT teams automate the response, which has a major impact on system availability and performance.

The integration of AI enhances observability and improves the overall customer experience, minimizing downtime, ensuring optimal performance and preventing issues before they impact end-users.

Steps to Implement AI Successfully

Achieving unified observability and proactive IT management requires a strategic approach. Begin by analyzing existing processes and tools.
- Conduct a thorough audit: Examine current processes, tools and workflows to identify gaps, bottlenecks and inefficiencies across environments. Involve all relevant stakeholders in this process to ensure a holistic understanding of IT.
- Deploy your observability solution in stages: Once you have a clear overview of the processes and tools, launch a gradual deployment of a unified observability solution, allowing for adjustments and finetuning. Start with a vertical view, connecting applications with their underlying infrastructure, and later move into the horizontal axis, focusing more on a business process view. To select the data you want to ingest, focus on what provides value.
- Upgrade with AI: Once your observability is in place, enable AI and generate insights. Allow time for the platform to collect the data necessary to create a baseline.
- Finally, bring in automation: The objective is to give the platform the right to decide and execute the first response, which can be to send a simple notification, run a complex root cause analysis or execute self-healing actions.
  This process requires a collaborative effort, involving teams responsible for IT management, platform management, tooling and security.
What Comes After Unified Observability?

Unified observability won’t be the last stop for IT operations, but rather a stopover towards an autonomous enterprise. The idea is to reach a stage where IT systems detect and respond to issues autonomously and continuously learn and adapt to the situation or context.

The synergy of AI and observability will lead to self-healing systems. As predictive analytics become more sophisticated, there will be a complete shift from reactive to proactive IT management. Organizations can anticipate a more resilient and agile IT infrastructure that aligns with business objectives.

Before we get there, it’s key to truly embrace the process of unifying observability, AI and automation that correspond with companies’ unique IT landscapes for enhanced efficiency, improved customer experiences and futureproof IT operations.
May 16, 2024