devopsy.kinsta.cloud

Category: DevOps and Open Technologies

Editorial Calendar for February 2016

Best of 2023: Will ChatGPT Replace Developers?

As we close out 2023, we at devopsy.kinsta.cloud wanted to highlight the most popular articles of the year. Following is the latest in our series of the Best of 2023.

AI is buzzing again thanks to the recent release of ChatGPT, a natural language chatbot that people are using to write emails, poems, song lyrics and college essays. Early adopters have even used it to write Python code, as well as to reverse engineer shellcode and rewrite it in C. ChatGPT has sparked hope among people eager for the arrival of practical applications of AI, but it also begs the question of whether it will displace writers and developers in the same way robots and computers have replaced some cashiers, assembly-line workers and, perhaps in the future, taxi drivers.

It’s hard to say how sophisticated the AI text-creation capabilities will be in the future as the technology ingests more and more examples of our online writing. But I see it having very limited capabilities for programming. If anything, it could end up being just another tool in the developer’s kit to handle tasks that don’t take the critical thinking skills software engineers bring to the table.

ChatGPT has impressed a lot of people because it does a good job of simulating human conversation and sounding knowledgeable. Developed by OpenAI, the creator of the popular text-to-image AI engine DALL-E, it is powered by a large language model trained on voluminous amounts of text scraped from the internet, including code repositories. It uses algorithms to analyze the text and humans fine-tune the training of the system to respond to user questions with full sentences that sound like they were written by a human.

But ChatGPT has flaws—and the same limitations that hamper its use for writing content also render it unreliable for creating code. Because it’s based on data, not human intelligence, its sentences can sound coherent but fail to provide critically informed responses. It also repurposes offensive content like hate speech. Answers may sound reasonable but can be highly inaccurate. For example, when asked which of two numbers, 1,000 and 1,062, was larger, ChatGPT will confidently respond with a fully reasoned response that 1,000 is larger.

OpenAI’s website provides an example of using ChatGPT to help debug code. The responses are generated from prior code and lack the capability to replicate human-based QA, which means it can generate code that has errors and bugs. OpenAI acknowledged that ChatGPT “sometimes writes plausible-sounding but incorrect or nonsensical answers.” This is why it should not be used directly in the production of any programs.

The lack of reliability is already creating problems for the developer community. Stack Overflow, a question-and-answer website coders use to write and troubleshoot code, temporarily banned its use, saying there was such a huge volume of responses generated by ChatGPT that it couldn’t keep up with quality control, which is done by humans. “Overall, because the average rate of getting correct answers from ChatGPT is too low, the posting of answers created by ChatGPT is substantially harmful to the site and to users who are asking or looking for correct answers.”

Coding errors aside, because ChatGPT—like all machine learning tools—is trained on data that suits its outcome (in this case, a textual nature), it lacks the ability to understand the human context of computing to do programming well. Software engineers need to understand the intended purpose of the software they’re creating and the people who will be using it. Good software can’t be built by cobbling together programs with regurgitated code.

For example, ChatGPT cannot understand the ambiguity in simple requirements. Although it is clear that if one ball just bounces and comes back but another bounces and then bounces again, the second ball has traveled further, ChatGPT struggles with this nuance; that nuance will be needed if these systems are ever to take over from developers.

It also has trouble with basic math, such as when it’s asked to determine which is greater and offered a choice between a negative and positive number. ChatGPT confidently tells us a correct summation of the space, but cannot understand that -5 is less than 4. Imagine your thermostats going haywire because the heating kicks on at 40 degrees Celsius instead of at -5 degrees Celsius because the AI program coded it that way!

Pre-trained AI code generation also raises some legal questions with regard to intellectual property rights; it cannot currently distinguish between code that is licensed in a restrictive or open fashion. This could expose people to licensing compliance risk if the AI borrows a prewritten line of code from a copyrighted repository. The problem has already prompted a class action lawsuit against a different OpenAI-based product called GitHub Copilot.

We need humans to create the software people rely on, but that’s not to say there couldn’t be a place for AI in software development. Just like automation is being used by security operations centers for scanning, monitoring and basic incident response, AI could serve as a programming tool for handling lower-level tasks.

This is already happening, to an extent. GitHub Copilot allows developers to use ChatGPT to improve their code, add tests and find bugs. Amazon offers CodeWhisperer, a machine language-powered tool designed to help increase developer productivity using code recommendations generated by natural language comments and code in the integrated environment. And someone has created a Visual Studio code extension that works with ChatGPT.

And one company is testing AI for developers. DeepMind, which shares a parent company with Google, released its own code generation tool, dubbed AlphaCode, earlier this year. DeepMind published the results from simulated evaluations in competitions on the Codeforces platform in Science magazine earlier this month under the headline Machine Learning systems can program too. Headline grammar aside, AlphaCode achieved an estimated rank within the top 54% of participants by solving problems “that require a combination of critical thinking, logic, algorithms, coding, and natural language understanding.” The abstract for the paper says: “The development of such coding platforms could have a huge impact on programmers’ productivity. It may even change the culture of programming by shifting human work to formulating problems, with machine learning being … responsible for generating and executing codes.”

Machine learning systems are becoming increasingly advanced each day; however, they cannot think like the human brain does. This has been the case for the past 40+ years of study into artificial intelligence. While these systems can recognize patterns and increase productivity for simple tasks, they may not always produce code as well as humans. Before we let computers do code generation en masse, we should probably see systems like AlphaCode rank in the top 75% of participants on a platform like Codeforces, though I fear this may be too much for such a system. In the meantime, machine learning can help with simple programming problems in the future, allowing developers of tomorrow to think of more complex issues.

At this point, ChatGPT won’t be disrupting any field of technology, especially not software engineering. Concern about robots displacing programmers is vastly overstated. There will always be tasks that developers with human cognition can do that machines will never be capable of.

December 27, 2023
DevOps 2024: DevOps Isn’t Dead; Long Live DevOps

Well, it has been quite the year in general and quite the year as it relates to DevOps. As I sit here on the cusp of the 10th year of publishing devopsy.kinsta.cloud, I look back and recognize how much DevOps has grown, evolved, matured and shifted over time. As was the case 10 years ago, there are those who say DevOps does not work or insist that DevOps is dead. My response:

“They stab it with their steely knives
But they just can’t kill the beast.”
~ Eagles, Hotel California

No, it seems you can’t kill DevOps. Much like Obi-Wan Kenobi being struck down by Darth Vader, DevOps, it seems, is more powerful than ever. It has seeped into the very fabric of IT.

While cloud native has become the dominant form of computing, the fact is you really would not have cloud-native stacks without DevOps fundamentals enabling them. As cloud-native continues to grow, so does DevOps.

One of the interesting things about DevOps is the fact that it really has become an enterprise toolset. While initially scaling DevOps across an organization was challenging (and still is for many), it has become much more common to see large enterprises—or ‘horses,’ as Gene Kim would call them—adopting DevOps methodologies across significant parts of their IT teams.

Speaking of Gene Kim, some have pointed to his recent announcement that he’s changing the name of his DevOps Enterprise Summit (DOES) to Enterprise Technology Leadership Summit as evidence of DevOps’ demise. We will be interviewing Gene on Techstrong.TV the first week in January 2024 and will address this with him, so stay tuned. But without giving out too many spoilers, Gene felt that DOES had expanded beyond just DevOps to encompass a general technology discussion (hey, AI will be in there for sure) aimed at technology leaders (C-suite). But rest assured that Gene Kim does not think DevOps is dead.

Looking ahead to 2024, I see DevOps making broad inroads into the SMB and mid-enterprise market. The DevOps toolset and methodology have matured to the point where it can be adopted and used successfully in IT environments not as resource-rich or process-driven as large enterprises generally are.

Another continuing story in DevOps is DevSecOps. Security and/or quality continues to be a key driver for DevOps pipelines, with the software supply chain and SBOM security still driving lots of adoption and adaptation.

Did I mention AI? You can’t discuss much today without mentioning it, or so it seems. AI is a catalyst for automation. Generally, in software development and deployment, where there is automation, you guessed it—there is DevOps.

But here is one out of left field for you: ITSM. It seems like IT service management is making a big comeback (some say it never left). Today’s ITSM incorporates many DevOps fundamentals to increase scale and agility. On this front, stay tuned; in partnership with PeopleCert, the folks behind ITIL, Prince 2 and the DevOps Institute, we will roll out a series of virtual events in 2024, tying all three of these key technologies together.

So, there’s lots going on in the DevOps world for 2024. You can count on us here at Techstrong Group to keep you in the know.
On that note, this will close out our coverage for 2023. To all of our readers, watchers, listeners, friends, sponsors and contributors, a Merry Christmas, Happy New Year, and here is to a healthy and prosperous 2024.

December 26, 2023
Best of 2023: Are We Witnessing the End of Open Source?

As we close out 2023, we at devopsy.kinsta.cloud wanted to highlight the most popular articles of the year. Following is the latest in our series of the Best of 2023.

Hailed as a groundbreaking movement in software creation, open source software has become an indispensable pillar of modern software development. Often described as democratized technology, it has empowered countless individuals and organizations to collaborate and innovate. However, the community now stands at a critical juncture, and there’s some speculation that we’re seeing a decline in the open source space.

Why is Open Source so Popular in Modern Software Development?

At the heart of open source lies the power of collaboration. Unlike proprietary (closed source) software, open source invites participation from a diverse range of contributors, allowing them to work collectively to create and improve software. This inclusivity fosters shared knowledge and cooperation, leading to the creation of groundbreaking software.

In addition to collaboration, the open source development approach eliminates financial barriers and significantly enhances accessibility in software development. With open source software typically distributed under licenses that allow users to access, use, modify and distribute the software freely, this eliminates the need for costly licenses usually associated with proprietary software. This affordability allows organizations of all sizes to leverage innovative solutions without accruing high upfront costs.

Within the framework of open source software, transparency is ingrained. The accessibility of the source code makes it possible for users and contributors to scrutinize the source code, promoting trust and accountability. By promoting scrutiny and peer review, transparency ensures that flaws are quickly identified and addressed. It also empowers users to have greater control over their choices in technology.

With these benefits coupled with success stories from enormous projects like Linux, Apache and MySQL, open source positioned itself as having the potential to compete with proprietary alternatives, leading to widespread adoption across the technology industry.

More Projects are Jumping Ship

Given these numerous benefits, one might wonder: What challenges could arise with this approach? In recent years, there have been sentiments amongst contributors that the space might be losing steam. Their concerns are fueled by a few key factors that may erode or undermine the collaborative essence of the movement.

A huge factor is the commercialization of open source projects. A popular example of this phenomenon is the GitHub acquisition – GitHub, the home for open source projects for years, was acquired by the tech giant Microsoft in 2018. This news ruffled some feathers in the community, as Microsoft was seen as a proponent of proprietary software. It also raised doubts about whether GitHub would maintain autonomy over its decision-making process.

Regardless of these doubts, the community overcame the bump in the road rather quickly and continued contributing to the platform for years. However, in 2022, GitHub announced that a sub-program called Copilot, which open source developers had been working on, would emerge as a commercial product, with the profits going to GitHub itself. This development infuriated community developers, and they faced a rude awakening that any open source platform could decide to commercialize their projects without contributors sharing in the financial rewards or even gaining recognition for their work. Another open source project that has fallen victim to commercialization is OpenAI.

There have also been increasing concerns about whether open-sourcing core business models will cause more harm than good. Unlike in the past, it’s a lot easier for projects to become popular, creating a high possibility that someone else can steal the idea or offering and quickly create a commercial product based on the project.

These considerations are part of a broader discussion regarding the shifting dynamics and potential departure from the core principles within the community, causing participants to be uneasy.

The Dust Settles: Open Source Remains Strong

Despite the concerns and challenges facing the community, open source has undeniably transformed how we build and deploy software. We must approach the future of the movement with cautious optimism.

Drawing from past experiences like GitHub and OpenAI, it is clear that the community is resilient and adaptive. Contributors who do not agree with the acquisition of GitHub or the commercialization of OpenAI will take their gifts and contribute to projects whose values they agree with.

However, open source is at a crossroads and is grappling with challenges and changes that demand introspection. This introspection, hopefully, will lead to adjustments in the open source space that reaffirm its core principles and address concerns. Open source is not fizzling out, at least not yet; we can think of it as undergoing a transformative phase that will shape its future and ensure that it remains an important pillar in the software development landscape.

To hear more about cloud-native topics, join the Cloud Native Computing Foundation and the cloud-native community at KubeCon+CloudNativeCon North America 2023 – November 6-9, 2023.

December 22, 2023
Senser Extends AIOps Reach to Manage SLOs and SLAs

Senser is extending the reach of its artificial intelligence for IT operations (AIOps) platform to now include an ability to define and maintain service level agreements (SLAs) and service level objectives (SLOs).

SLOs are a set of internal performance goals that require access to telemetry data from service level indicators (SLIs), while an SLA is a formal commitment to maintaining specific levels of service.

Senser CEO Amir Krayden said the company’s AIOps platform collects data from SLIs and then applies predictive AI models to enable IT teams to achieve SLOs and SLAs. The Senser AIOps platform leverages extended Berkeley Packet Filter (eBPF) and graph technology to gain visibility into the entire IT environment versus requiring IT teams to deploy agent software. Machine learning algorithms are then used to aggregate and analyze that data to define thresholds for predicting performance in addition to recommending benchmarks for tracking SLOs and SLAs.

That approach provides a single source of truth for identifying the actual level of service being provided based on a topology of the infrastructure, network, applications and application programming interfaces (APIs) that makes it possible to identify the root cause of issues and the potential impact of an outage for degradation of performance.

IT teams have been attempting to achieve and maintain SLAs and SLOs for decades, but given all the dependencies that exist in a distributed computing environment, it’s difficult to achieve that goal. Senser is making a case for applying AI within the context of a platform for automating the management of IT to define and maintain SLOs and SLAs to make it possible to consistently manage SLAs and SLOs in a way that reduces the level of cognitive load that would otherwise be required. Senser is also working toward adding generative AI capabilities to provide summaries that explain what IT events have occurred.

Collectively, the goal is to provide IT teams with a more efficient holistic approach to monitoring and observability that legacy platforms are not going to be able to achieve and maintain, said Krayden.

At the core of that capability is eBPF, a technology that allows software to run within a sandbox in the Linux microkernel. That capability enables networking, storage and observability software to scale at much higher levels of throughput because they are no longer running in user space. That’s especially critical for any application that needs to dynamically process massive amounts of data in near-real-time.

As the number of organizations running the latest versions of Linux continues to increase, more hands-on experience with eBPF will be gained. IT teams may not need to concern themselves with what is occurring in the microkernel of the operating systems, but they do need to understand how eBPF ultimately reduces the total cost of running IT at scale.

Ultimately, the goal is to reduce the current level of complexity that today makes effectively managing highly distributed computing environments all but impossible for IT teams to manually maintain in an era where the pace at which applications are being built and deployed only continues to accelerate.

December 21, 2023
From Machine Learning to DevSecOps: Six DevOps Trends for 2024

The DevOps market is set to grow by 25% annually between 2024 and 2032, fueled by the growing demand for faster software delivery and higher-quality output. Emerging technologies like AI-driven development and serverless architectures have introduced new products in the space, while traditional processes like version control and CI/CD will be further refined. But amongst this innovation, security and quality should remain as organizations’ highest concerns. Looking into the coming year, it’s hard not to see where the DevOps industry is going based on the past year. As such, we have endeavored to provide six distinct DevOps trends we think will be particularly influential in the new year.

The Integration of AI and Machine Learning

One of the most notable trends in the future of DevOps is the increasing incorporation of artificial intelligence (AI) and machine learning (ML) into the software development life cycle. The explosion of AI technology in 2023 has set the stage for more predictive analytics in code development, automated testing and intelligent monitoring in the operations phase.

As Google’s 2023 State of DevOps Report highlighted, DevOps teams feel that AI will contribute most to analyzing data and security, alongside identifying bugs. However, a solid use case has yet to materialize for most teams, and the report shows that early adopters of AI have experienced a minor decrease in their software delivery performance and a significant decrease in their operational performance. So, it’s possible that at this stage teams are implementing AI without considering what problems they’re trying to solve, or without paying full attention to the security and quality implications.

Emphasis on Increased Productivity

While AI adoption will be driven by the underlying push for increased productivity in DevOps tools and practices, DevOps teams should focus on drawing maximum value from the existing tools that automate mundane and repetitive tasks, allowing them to focus on more critical aspects of development and operations. This will be made possible by DevOps vendors that provide enhancements to collaborative tools, simplified interfaces and more intuitive workflows, reducing the cognitive load on developers and operations professionals.

Culturally, the continuing trend towards remote and hybrid work environments is influencing productivity strategies in DevOps, with tools and practices that support remote collaboration and communication becoming vital. Cloud-based DevOps tools are facilitating seamless collaboration, allowing teams to work efficiently irrespective of their physical locations.

Alignment to Existing Cloud and Microservices Architecture

We can expect the new alignment of DevOps with cloud and microservices architectures to continue into 2024. This integration is about more than just adopting new technologies — it’s about creating efficiencies that enhance agility and revenue growth.

The combination of DevOps practices with cloud and microservices enables organizations to leverage the inherent benefits of these platforms. Cloud infrastructure is scalable and flexible, reducing overhead costs associated with physical servers. Microservices allow for rapid development, testing and deployment of independent components without disrupting the whole system. Together with DevOps automation and continuous practices, these architectures accelerate innovation and time-to-market.

This alignment of DevOps with cloud and microservices will lead to enhanced scalability, resilience, collaboration and customer experiences. By being able to quickly scale, update, and respond to demands, organizations can better satisfy customers and meet market needs, and this, in turn, will drive revenue growth and productivity.

The Drive for Higher Quality

While increased productivity and revenue generation will be a key priority in 2024, it’ll be matched by an unwavering commitment to quality. With the rising complexity of software and the increasing dependency of businesses on digital platforms, the margin for error is slimming. Quality assurance practices will become more integrated into the DevOps workflow, ensuring that each release meets the highest standards of performance, security, and user experience.

Continuous integration/continuous deployment (CI/CD) practices are evolving to include more rigorous and automated testing protocols. These protocols are designed not just to catch bugs but to assess performance under diverse and real-world scenarios. Real-time monitoring tools are being integrated more deeply into DevOps workflows, providing ongoing insights into application performance and user experience. This continuous feedback loop ensures that any issues are identified and addressed promptly, thereby maintaining high standards of quality post-deployment.

DevSecOps Maturity

In the same vein, cybersecurity will continue to be a primary concern in 2024, with core DevOps tooling and processes stepping up by incorporating advanced security features. Version control systems will take the lead here by introducing robust access controls, encryption of code repositories and integration with security testing tools to ensure that the code is not only efficient but also secure from vulnerabilities.

Security integrations in version control will become more advanced, possibly incorporating real-time scanning for security vulnerabilities, tighter integrations with identity management platforms and pattern analysis to predict and prevent potential security breaches in code.

And as AI is pioneered throughout DevOps tooling, organizations will need to consider the security implications. The Open Worldwide Application Security Project (OWASP) has predicted that data leaks, permission issues and insecure plugins could pose serious security vulnerabilities for software. Teams that can harness growth with AI in 2024 while building sufficient security through DevSecOps practices like backup and monitoring will future-proof themselves.

Speed and Security

The speed of DevOps growth, innovation and adoption is only set to increase in 2024. We’ll see teams pioneer new technologies to meet business priorities and revenue targets. In doing so, it’s very easy to jump on the AI or serverless bandwagon and assume that sprinkling it on every technical discipline will make for a bright and shiny future. While some key opportunities in the DevOps space will benefit from the acceleration and tighter integration that new technology brings, organizations looking for lasting growth will need to prioritize quality and security practices.

The key trend we’ll see in 2024 is teams getting smarter and more focused on how they apply new technology to their DevOps trajectory, delivering faster, more accurate releases that free up time to deliver ever greater value to their customers and business. So, the DevOps winners of 2024 will be the organizations that manage to balance the speed of new technology adoption with robust security.

December 19, 2023
What Will DevOps Look Like in 2024?

The last two years have been marked by significant downsizing in technology companies. As large layoffs at companies like Amazon, Meta and Google make major headlines, there is a general shift away from the “spending cash and growing fast” mindset that free capital and strong growth during the pandemic launched in the tech industry. The name of the game is to be more conservative with spending, and that mindset will drive a lot of investments in the next year. While these cost pressures may feel challenging to technology organizations, there are a few key opportunities as well. Many trends are starting, sustaining and ending, so let’s focus on the top six likely to drive changes in the industry in 2024 and beyond.

1. AI—Specifically Generative AI—Move Right on the Hype Cycle

While AI is not new, OpenAI and ChatGPT have created an amazingly accessible and powerful tool that is positioned to drive a lot of value. The hype cycle is still in full swing as dozens of new uses of the technology hit the market daily, and if you’re not trying to build a generative AI-enabled product, you’re getting asked when you will start. In 2024, we will likely see LLMs move into the trough of disillusionment as inflated expectations and projects don’t all play out. However, those organizations with generative AI products that truly deliver value to users should expect to see massive growth. Technology organizations will need to make hard decisions about AI products. You will either find traction in the next year and see huge growth or quickly find it doesn’t work and shift focus back to non-AI products.

2. Developer Experience Gains Momentum

While DX has been talked about for years, we’re seeing new investments in tools that empower developers. This trend is partially born out of a need to reduce tool and system sprawl for engineering teams, to empower teams and allow them to move faster and to improve efficiency. While you may not build or buy a DX platform, you will likely start a DX team or committee that better represents the folks building your software. This will lead to some great internal changes and key efficiencies getting attention that may have been previously overlooked. The wonderful thing about this trend is that it both brings efficiencies to the organization and makes developers’ lives easier. It’s a win-win!

3. Increased Pressure to Reduce Costs

Even for organizations that have avoided layoffs, most are seeing pressure to shift into a leaner existence. This often means critically looking at all sources of spending and taking on new projects to optimize them. Maybe a cloud re-platform can reduce hosting spend by 20%, or maybe you start looking to remove or combine SaaS tools to reduce seats. You may even see hiring freezes for critical teams. Organizations will have to embrace other trends to increase the efficiencies of their teams to meet business demands here. Don’t be surprised to hear “do more with less” become a mantra around the office.

4. Increased Accountability Through Data

While data platforms for technology organizations continue to grow, teams are finding that it’s not enough to just measure a few metrics. Organizations will be required increasingly to use the data they have to show they are delivering value. Partially due to increased pressures mentioned above, teams will need to become more predictable, and better at identifying and handling risk at every stage of development. It’s no longer enough to just build a great development engine, teams must ensure that the engine can power them to their destination.

5. ESG in Technology

Focus on DEI in technology continues to require significant investment, however, organizations are starting to consider a broader umbrella of social responsibility. This can range from considering the environmental impacts or carbon production of your servers to changing your software supply chain to avoid supporting particular organizations. These types of requirements are appearing sooner and more overtly in the software development process, and may significantly impact how teams choose to build or implement software.

6. DevSecOps Continues to Grow

There has been a renewed focus on security in recent years after some critical attacks exposed vulnerabilities in the software supply chain. It’s not surprising that security is now written into the fabric of how we build and deliver software. Teams are now expected to consider security at all stages of development from product planning to operation in production, and require additional experts, training and resources to both monitor and ensure compliance. Requirements for SaaS products continue to evolve as the last remnants of on-premises solutions evaporate, and teams should be prepared to move quickly as new threats are discovered and need to be patched.

As the technology industry prepares for 2024, it’s clear that innovation and adaptation will be the keys to success. From the continued evolution of AI and the growing focus on the developer experience to the imperative to reduce costs, increase accountability through data, embrace ESG practices and fortify security through DevSecOps, technology organizations face an exciting future.

December 19, 2023
The Future of Jenkins in 2024

As software development practices evolve, the tools that support them must also adapt to stay relevant. Jenkins, the venerable automation server used by many DevOps teams, is at such a crossroads as it faces the changing landscape of cloud-native applications and Kubernetes adoption.

This article examines Jenkins’ current utility, particularly its features that have made it a staple in CI/CD practices, and addresses the question of whether it will maintain its relevance in 2024 and beyond.

How Jenkins is Used by DevOps Teams

CI and CD

Jenkins has become an integral part of many DevOps teams due to its CI/CD capabilities. Continuous integration (CI) is a software development practice where developers regularly merge their code changes into a central repository. Continuous delivery (CD) is the practice of delivering the software in short cycles to ensure that it can be reliably released at any time.

Jenkins supports the CI/CD approach in a number of ways. First, it allows developers to automate code integration. This means that whenever code changes are pushed to the repository, Jenkins can automatically build and test the project. Second, Jenkins allows for the continuous delivery of software by automating simple deployment processes (although it lacks full deployment orchestration capabilities).

Build Pipeline Visualization

Another key feature of Jenkins is its build pipeline visualization. This feature allows DevOps teams to visually represent the process of software delivery from code integration to deployment. The pipeline visualization provides a clear and intuitive representation of the software delivery process, making it easier for teams to understand and manage the process.

Build pipeline visualization in Jenkins is customizable, allowing teams to tailor it to their specific needs. For example, it can be customized to show the status of each stage of the delivery process, from code integration to testing to deployment. This allows teams to quickly identify any issues or bottlenecks in the process and address them.

Automated Testing

Automated testing is another area where Jenkins shines. With Jenkins, DevOps teams can automate the process of testing their code by running scripts and integrating with automated testing tools as part of the build process.

Jenkins supports a wide range of testing frameworks and tools, allowing teams to choose the ones that best fit their needs. Moreover, Jenkins can be configured to automatically run tests whenever changes are made to the code. This ensures that any issues or defects in the code are detected early and can be addressed promptly.

Workflow Scripting

Workflow scripting is another key feature of Jenkins in DevOps teams. With Jenkins, teams can script their workflows, automating the process of software delivery. This allows teams to standardize their workflows, ensuring that each delivery process is consistent and predictable.

Jenkins supports the Groovy scripting language for workflow scripting. This means that teams can use the power and flexibility of Groovy to script their workflows. Moreover, Jenkins provides a user-friendly interface for scripting workflows, making the process easy even for those who are not familiar with Groovy.

Current Usage and Adoption Statistics

As of 2023, Jenkins remains a leader in the CI/CD space. With its extensive plugin architecture and large user base, it is used by thousands of organizations worldwide. According to the 2022 DevOps Survey by the Cloud Native Computing Foundation (CNCF), Jenkins was used by over 50% of respondents for automation and CI/CD, maintaining a significant presence in the market.

According to the Jenkins website, there are over 1 million active Jenkins users and more than 200,000 active installations worldwide, with the platform handling 1 million+ jobs per day. It had about 44% of the global market share for CI/CD in 2023.

The growth trajectory of Jenkins, however, has shown signs of plateauing when compared to newer, more cloud-native-focused CI/CD tools. This observation is backed by Google Trends data, reflecting a steady but not sharply increasing interest in Jenkins over recent years.

Despite facing competition from emerging tools designed for cloud-native development, such as GitHub Actions, GitLab CI and CircleCI, Jenkins has responded by evolving its ecosystem. The creation of Jenkins X is a strategic move to cater to Kubernetes users and address some of the criticisms related to cloud-native development. However, as we’ll discuss, Jenkins X has not been readily adopted by the industry as of yet.

Limitations of Jenkins in Cloud-Native Environments

While Jenkins is a powerful tool for DevOps teams, it has serious limitations in cloud-native environments. These limitations stem from the fact that Jenkins was originally designed for traditional, monolithic applications, not for the dynamic and distributed nature of cloud-native applications.

Statefulness

One of the critical limitations of Jenkins in cloud-native environments is its inherent statefulness. Cloud-native architectures typically favor stateless applications, which are easier to scale and manage in dynamic environments like Kubernetes.

Jenkins, however, relies on maintaining state—it stores configuration, build history and user data, making it challenging to scale and manage efficiently in a cloud-native setup. This statefulness requires persistent storage, which complicates deployments in distributed systems where resiliency and horizontal scaling are key. Moreover, the risk of data loss or inconsistency increases in cloud-native environments, particularly during scaling operations or when recovering from failures.

Configuration Complexity

Jenkins is often criticized for its complex configuration requirements. Setting up Jenkins in a cloud environment involves intricate configurations that are not only time-consuming but also prone to errors. This complexity is exacerbated when integrating with modern cloud-native tools and services, requiring extensive customization and maintenance of plugins.

In addition, Jenkins’ traditional master-slave architecture was not designed for dynamic cloud environments, where resources need to be provisioned and deprovisioned rapidly. This leads to a significant operational overhead for DevOps teams.

Immutable Infrastructure

Adopting an immutable infrastructure approach is a cornerstone of cloud-native environments, promoting the idea of replacing or rebuilding servers rather than modifying them.

However, Jenkins’ traditional model does not align well with this principle. It typically requires modifications to its runtime environment for updates or changes, contrasting with the cloud-native approach of immutable deployments. This can make it challenging to maintain consistency and reliability in cloud-native systems where automation and minimal manual intervention are paramount.

Microservices Limitations

Jenkins was originally designed for monolithic architectures and doesn’t naturally fit into microservice-based applications, which are prevalent in cloud-native environments. In microservices architectures, applications are broken down into smaller, independently deployable services.

Jenkins, however, tends to treat applications as monolithic units, making it less efficient for handling multiple, independent microservices. This limitation affects its ability to provide fine-grained control over individual services’ build, test and deployment processes. Consequently, Jenkins may struggle with the high degree of automation, flexibility and scalability required in microservices environments.

What is Jenkins X?

Jenkins X is an open source project that provides pipeline automation, built-in GitOps, and preview environments to help teams collaborate and accelerate their software delivery at any scale. It is an improved version of the original project, designed specifically for Kubernetes, a popular container orchestration platform.

Jenkins X streamlines the technical aspects of building, testing, and deploying software. It automates the installation of popular tools like Helm and Draft, making it easier for developers to get started. It also provides pre-configured pipelines so teams can start delivering software right out of the box.

Jenkins X also introduces a new Jenkinsfile format, Jenkinsfile.yaml, which provides a more straightforward and human-readable way to describe pipelines. This new format, combined with the automation provided by Jenkins X, helps reduce the complexity of pipeline configuration and management in Kubernetes.

Limitations and Criticisms

Jenkins X has not been widely adopted by the software industry and has not achieved the respect and prominence of its predecessor, Jenkins. Let’s review the main limitations and criticisms of the tool.

Lack of Maturity

Jenkins X’s relatively recent entry into the market means it has not had the time to mature like its predecessor. This lack of maturity can manifest in several ways, including fewer features compared to established tools, fewer community-tested plugins, and potentially more bugs or unresolved issues. Due to the fast-paced evolution of cloud-native technologies, tools need time to adapt and stabilize; Jenkins X is still in this phase of its lifecycle.

Moreover, a mature tool typically has extensive documentation and a large community of users who can offer support and share best practices. While Jenkins X has seen growth in its user base, it still lags behind more mature CI/CD solutions in these areas. This can lead to challenges when teams are looking to solve complex problems or when they require advanced support.

Limited to Git Projects

Another criticism of Jenkins X is that it’s limited to Git projects. While Git is the most popular version control system, it’s not the only one. Some teams may use other systems like Mercurial or SVN, and these teams would be unable to use Jenkins X.

Even within the world of Git, Jenkins X has limitations. For example, it doesn’t support all Git hosting platforms. While it works with popular platforms like GitHub and Bitbucket, it doesn’t support others like GitLab. This can be a significant limitation for teams that use unsupported platforms.

Community Support

Finally, there are concerns about community support for Jenkins X. While the project is open source and theoretically has the potential for wide community involvement, some critics argue that it’s primary source of support is CloudBees, the company behind Jenkins.

These critics worry that if CloudBees were to withdraw its support, the project could falter. They also argue that the project’s focus on certain tools and platforms (like Kubernetes and Git) reflects CloudBees’ business interests more than the needs of the broader DevOps community.

Conclusion

In conclusion, Jenkins’ role in the CI/CD landscape has been pivotal, and its continued evolution with projects like Jenkins X demonstrates an effort to adapt to the needs of modern, cloud-native environments. While Jenkins X has yet to reach the widespread adoption and maturity of its predecessor, its specialized focus on Kubernetes and GitOps reflects the shifting priorities in software development.

Jenkins, despite showing signs of plateauing in growth, still possesses a strong user base and an established presence in the market, supported by a vast plugin ecosystem and a robust community. Its limitations in cloud-native contexts—such as managing statefulness, configuration complexity, and supporting immutable infrastructure—pose challenges that Jenkins X seeks to address.

As we approach 2024, the relevance of Jenkins will likely hinge on its ability to integrate more seamlessly with cloud-native technologies and the willingness of the community to evolve its ecosystem. The breadth of its adoption and the depth of its integration in existing development pipelines mean Jenkins will continue to be a significant player in the DevOps toolchain. However, its long-term relevance will depend on the project’s ability to innovate and remain adaptable in a landscape that is rapidly embracing cloud-native solutions.

December 15, 2023
Sourcegraph Launches Cody Generative AI Tool to Write Code

Sourcegraph today made generally available an open source Cody tool that leverages generative artificial intelligence (AI) tools to write and fix code.

Version 1.0 of Cody invokes StarCoder, an open source large language model (LLM) for creating code made available by Hugging Face and a chat tool that makes use of dense-sparse vector retrieval system for code and documentation to invoke GPT-4 Turbo, Claude 2, GPT-3.5 Turbo, Claude Instant and Mixtral-8x7B LLMs.

Designed to be deployed as an extension of VS Code, Cody is available in a free edition as well as a professional edition, dubbed Cody Pro, that Sourcegraph supports. Cody Pro will be free until February 2024, after which it will cost $9 per month. There is also a version for the JetBrains editor available in beta, along with an experimental Neovim offering.

Cody is also integrated with the code search tools provided by Sourcegraph to provide development teams with additional context that, depending on the use case, delivers a completion acceptance rate of 30% or higher for code.

In addition to enabling developers to write code faster, Coy also simultaneously invokes the graph technology Sourcegraph created to provide additional context to a prompt that makes it simpler to surface relevant code by parsing code and producing a graph that captures the semantic understanding of code. Cody, for example, can search the codebase for files and documents that can be used to suggest a rough plan of attack for implementing a new feature or resolving issues that surfaced in a bug report.

Cody can also create, save and share custom commands to make it simpler to reuse the prompts created to write code.
Future versions of Cody will be more tightly integrated with the Sourcegraph universal code graph to provide additional context. In addition, Cody will soon leverage deeper graph context to make autocomplete suggestions based on symbols defined elsewhere.

Sourcegraph CEO Quinn Slack said the combination of graph technologies and LLMs reduces latency while helping to ensure the code generated by the LLM doesn’t produce hallucinations such as type errors and imaginary function names.

In addition, Sourcegraph is committed to applying AI and graph technologies beyond the codebase itself to provide deeper insights into the environment code will eventually be deployed in, added Slack.

It’s not clear how much developers are relying on generative AI tools such as Cody and GitHub Copilot to write code faster, but it’s clear the pace at which software can be written is accelerating. That increased pace of application development will eventually require DevOps to revisit workflows that were not designed to manage codebases that, in the months ahead, will significantly increase in size.

Each developer will most likely have a personal preference for one generative AI tool versus another, depending on the quality of the code generated. The issue is that no matter how good any of those tools are, however, the code generated will still need to be reviewed by a human before it can be deployed in any production environment.

December 14, 2023
Hitting the Accelerator for Change: How Open Source Drives Software-Defined Cars

Every new car on the road today contains hundreds of millions of lines of code, and that’s a good thing because the technological functions they enable add greatly to the car’s ultimate value. For over 100 years, automotive consumers coveted cars because of their paintwork, their leather interiors and the speeds they can travel. But now, their real and perceived value is influenced by technology features. Technologies such as electric vehicles, self-driving and other software-defined services are what drive the value of the car—and it’s no longer too nerdy to describe cars as ‘software-defined.’

Navigating the Road to the Future

For car manufacturers and their suppliers, the challenges of the software-defined car are already very much in focus. Original equipment manufacturers (OEMs) have been hiring new workers and re-training existing ones to navigate these challenges, such as cybersecurity and connected technologies. Ten years ago, OEMs such as Renault and Volkswagen had few in-house software engineers. Fast forward to today, and both companies have numerous resources focusing on software.

Software will soon become ever more central to the vehicles we drive, and that will soon drive us, as vehicles become more connected, autonomous, shared and electric (CASE). In the next few years, autonomous vehicles will soon be handling tens of terabytes of data per day, gathered by sensors such as LIDAR and cameras. Customers are increasingly expecting smartphone-esque features, which involves keeping pace with the very latest apps and technologies.

Combined with features such as passenger entertainment systems and remote diagnostics, the requirement for software and processing power in vehicles is going to grow at an ever-increasing pace. Open-source software can provide automakers with the power to keep up.

You Must Shift Gears

Inevitably, these processing power requirements will require a seismic shift in the way OEMs work. Currently, OEMs are struggling to maintain existing systems‒while at the same time attempting to develop new platforms. It’s a hangover from years in which vehicles were not connected or had extremely basic connected services. This has led to a situation where it’s difficult to upgrade the software at pace due to the requirements and constraints of the hardware.

There are lessons for car-makers in the smartphone world. Think about how a software update can be applied across multiple different smartphones. An iOS update works just as well across a new device or a device from four years ago. One iPhone will see multiple updates, adding new features and boosting security.

For car manufacturers, this won’t be an easy shift: The industry needs more defined standards to allow an easier interface between hardware and software. The industry also needs to reconsider how it develops hardware and how this relates to software. These challenges will require a new way of thinking for car manufacturers as they transition to a world where time cycles are critical. I believe that cross-industry collaboration is needed to accelerate the development of autonomous vehicles.

Keep an Eye on Moving Parts

Complexity is building at an increasingly rapid pace in the car industry, and this shows no sign of slowing down. In a report on the industry, McKinsey noted that the average complexity of individual software projects in the automotive sector has grown by 300% over the last decade. By 2025, the number of lines of code in the average luxury car will have doubled from today’s figure of 100 million lines of code, according to a prediction by Volkswagen‒which is already more than a military aircraft like the F-35. In full-self-driving cars, the figure could potentially reach a billion lines of code, analysts believe.

It’s not just the software that is becoming more complex, the hardware is too. Modern vehicles already have up to 150 Electronic Control Units (ECUs), often scattered through the vehicle, close to sensors. Increasingly, manufacturers are moving towards a model where these ECUs are combined and consolidated, connected by an ethernet network. As the amount of data vehicles deal with increases, it’s unlikely that all of this will be directly uploaded to cloud servers; instead, much of it will be processed inside the vehicle before ‘edited highlights’ are uploaded to the cloud for processing.

This has important implications for vehicle maintenance and also poses cost issues. Car manufacturers face challenges when it comes to managing the ever-growing complexity of their systems while trying to keep annual recurring costs down.

Watch the Hacker Threat

The need for cybersecure vehicles is also growing, particularly as we see self-driving vehicles enter the fray. In 2015, two security experts showed off a hack where they were able to ‘switch off’ a Jeep driving at 70mph down an American highway. It marked the dawn of the ‘hackable’ car—although the seeds of this were sown decades earlier when the first ‘connected’ components were unveiled in the 80s and were not built to be resilient to hackers. A 2021 report on global automotive security was damning, saying, “There have been 110 Common Vulnerabilities and Exposures (CVEs) that year related to the automotive industry, 33 in 2020 alone compared to 24 in 2019.”

Ultimately, the automotive industry faces increasing regulation in the face of these threats. As governments begin to regulate software within the automotive industry, those designing the products and software will become more accountable.

Open Source and Safety … Always

For car manufacturers, there are also important issues related to functional safety—unsurprising when you are dealing with a device that weighs a ton or more. This directly impacts software development, with additional costs for any system that has an impact on driver or passenger safety.

Car manufacturers need to learn from the aviation industry, where systems related to safety are rigidly separated from those that are not, reducing costs due to compliance.

It’s often difficult to reuse existing software in cars today, as it can require porting to a different configuration. Costs can rise, not in a linear fashion, but exponentially. How much money will it cost to write, implement and test 100 million lines of code ‒ or a billion? This is the challenge manufacturers are facing. When it comes to managing these costs, open source will become ever more important and not only will allow the reuse of existing software, but will enable other manufacturers to contribute to said software.

Software-Defined Cars

The future is rushing towards us at speed—and in it, cars are increasingly software-defined. In this future, open source offers OEMs a highly attractive solution to the considerable hurdles ahead, from safety issues to cybersecurity. Open source will unleash the power of the ambitions of the car industry—not just to compete with the tireless innovation of other industries, such as mobile, but to exceed it. For consumers, cars are already the most complex devices they own: and they could be the most innovative. The flexibility and scalability of open source will allow car-makers to free their imagination and create the innovative, software-driven car of the future.

December 8, 2023
8 Essential Generative AI Tools for Building Stand-Out Applications

Building advanced AI apps demands the right tools. Generative AI is a game-changer with the potential to transform how we craft AI-powered solutions. Whether you’re a pro or new to AI, the right generative AI tools can elevate your projects significantly. In this article, we’ll explore the essential generative AI tools that can help you build remarkable AI applications.

These tools cover a wide range of functions, from content creation to design and data analysis. By the end of this blog, you’ll have a better understanding of how generative AI tools can elevate your AI projects to new heights.

GPT-3 by OpenAI

GPT-3, or Generative Pre-trained Transformer 3, is a remarkable language model designed to create text that closely resembles human writing. Its applications span content generation, language translation and chatbot development.
● Pros: GPT-3 is highly adaptable and capable of tackling a wide array of tasks. Its primary strengths lie in its ability to handle complex language-related challenges, making it a versatile tool for developers.
● Cons: While GPT-3’s capabilities are impressive, its availability may be limited and using it extensively could entail significant costs. These factors are worth considering for projects with high resource demands.

Runway ML

Runway ML is tailored to cater to the creative aspects of AI, offering a suite of tools tailored for artists, designers and developers. It encompasses a wide array of generative models that empower users to generate art, design prototypes and craft interactive experiences.
● Pros: Runway ML stands out for its user-friendly interface and real-time creative capabilities. This makes it an excellent choice for individuals venturing into the realm of creative AI projects, providing a smooth learning curve.
● Cons: It’s important to note that certain advanced features within Runway ML may be accessible only through a subscription, limiting full access in the free version. This subscription-based model should be considered when assessing its cost-effectiveness for your specific project needs.

Hugging Face Transformers

Hugging Face Transformers serves as a repository of pre-trained models specifically designed for natural language processing (NLP) tasks. Its applications are extensive, encompassing tasks such as sentiment analysis, text classification and more.
● Pros: Hugging Face Transformers stands out as an open-source platform with a vibrant and engaged community. This ecosystem offers a diverse array of models and tools, making it a valuable resource for developers seeking to harness the power of NLP.
● Cons: It’s important to acknowledge that newcomers to Hugging Face Transformers may face challenges when customizing models and engaging in model training. While this platform offers an array of resources, it may require a certain level of familiarity with NLP and machine learning concepts for optimal utilization.

Midjourney

Midjourney is a versatile tool designed to tackle creative tasks spanning text, music and image generation.
● Pros: Midjourney presents a robust offering of pre-trained datasets and generative models, catering particularly well to creative projects. Its user-friendly interface ensures accessibility for individuals, even those who are new to AI, making it an excellent starting point.
● Cons: However, it’s crucial to recognize that while Midjourney excels in the realm of creativity, it may not be the top choice for AI applications that require a strong focus on data-driven tasks. Depending on your project’s needs, this limitation might affect its suitability.

Synthesia

Synthesia is a specialized AI tool designed for video generation and customization. Its core function revolves around the conversion of text into AI-generated videos.
● Pros: Synthesia streamlines the video production process, delivering substantial advantages for content creators and marketers. Its user-friendly interface simplifies the video creation journey, ensuring that a wide audience, regardless of technical expertise, can harness its potential.
● Cons: However, it’s essential to recognize that Synthesia’s primary expertise centers on video generation. This specialization can occasionally limit its adaptability when dealing with a broader spectrum of creative tasks that extend beyond the realm of video production. Depending on your project requirements, this specialization may influence its suitability.

DeepCode

DeepCode is a specialized tool dedicated to code analysis and improvement. Its primary mission is to assist developers in identifying and rectifying issues within their codebase.
● Pros: DeepCode serves as a highly valuable resource for developers, offering concrete benefits such as the enhancement of code quality and the reduction of errors. It’s thoughtfully designed to streamline the coding process, making it more efficient and effective.
● Cons: It’s imperative to recognize that DeepCode’s focus is specifically centered around code analysis. This level of specialization may limit its applicability when addressing AI projects or tasks that fall outside the realm of software development. Depending on your project’s nature, this focus may impact its relevance to non-developer AI initiatives.

Soundraw

Soundraw is an AI-powered music generator designed to empower users to craft unique, royalty-free music, perfectly suited to enhance a wide range of projects and content.
● Pros: Soundraw’s most significant advantage lies in its ability to generate original compositions, thereby mitigating the risk of copyright issues. It also offers secure and enduring licenses for all creative uses, ensuring that your content remains free from copyright concerns.
● Cons: While Soundraw’s AI capabilities are impressive, it may occasionally lack the nuanced depth of creativity found in music created by human composers. The variety of available themes and moods may be somewhat limited, which could affect its suitability for highly specific or unique creative projects.

VQ-VAE-2

VQ-VAE-2 is a specialized tool known for its exceptional performance in image compression and generation tasks. It’s the preferred choice when the goal is to create high-quality images while simultaneously reducing data size.
● Pros: VQ-VAE-2 stands out as a robust solution for image-related AI applications, delivering substantial benefits in terms of enhanced compression and improved image quality.
● Cons: It’s essential to acknowledge that leveraging VQ-VAE-2 effectively may necessitate a solid understanding of image processing and AI concepts. This tool is best suited for users with advanced knowledge in these domains.

Wrapping Up

By using generative AI tools in your AI projects, you can open up exciting opportunities and make your work more efficient. But, it’s crucial to think about your project’s unique requirements, check if the tools are accessible, and understand their costs.

Explore what each tool can and can’t do to decide which one suits your AI project best. With the right generative AI tools, you can bring your projects to life and shine in the world of artificial intelligence-powered applications.

December 5, 2023