devopsy.kinsta.cloud

Tag: software

Survey Surfaces Raft of Ongoing Agile Development Challenges

A survey of 788 software development professionals finds 71% are using agile development methodologies within their software development life cycle (SDLC) but only 44% said it is working very well (11%) or somewhat well (33%) in the enterprise.

Conducted by Digital.ai, the survey identifies end-to-end visibility and traceability from business initiative through deployment as the top opportunity for improvement (44%), followed by an ability to measure cycle time, wait time and bottlenecks (34%), continuous testing (29%) and identification and measurement of technological risk (28%).

Nearly half (46%) identified too many systems as a primary reason Agile was not scaling, followed by siloed teams (37%), culture clash (34%), inconsistent adoption (30%) and the inability to measure business value (28%).

Despite these issues, nearly two-thirds (63%) said they have visibility into their Agile pipeline, with 55% claiming they have complete visibility. However, only 36% said DevOps teams currently work together as one team to accelerate innovation and deployment of high-quality and more reliable software. At the same time, only two-thirds (66%) claimed at least 50% of their applications were delivered on time and “with quality.”

Joyce Tompsett, general manager of the State of Agile project at Digital.ai, said the survey makes it clear many organizations are still trying to strike the right balance when it comes to developing software. For example, the survey finds the top two reasons organizations adopt Agile (tied at 41%) are to accelerate time to market and prioritize delivery and measure customer/business value. Most software development teams are being evaluated both based on velocity (33%) and the value delivered (29%), the survey finds.

The survey also identifies customer satisfaction (43%), time to delivery (39%) and competitive advantage (34%) as their top priorities for 2024.

In general, the survey finds that 33% of respondents reported business leaders and executives are actively leading and participating in company-wide agile transformation. However, more (41%) said there was not enough leadership participation. Well over a third (37%) said business teams simply don’t understand what Agile is or what it can do. More than a quarter (27%) said there is not enough training.

Not surprisingly, the most widely adopted methodology is Scrum (63%) with the Scaled Agile Framework (SAFe) being the most widely used (26%). Another 22% don’t have a mandated framework, while 12% said they have created their own.

The debate over what precisely constitutes agile development is, of course, ongoing. Each organization tends to adopt the elements of the framework that make the most sense for them, so the actual state of adoption from one organization to the next will vary widely. The one thing that is certain is the more complex the IT environment, the more challenging it can be to embrace agile development.

And there continues to be lots of room for improvement. If only two-thirds of organizations are delivering half (50%) of their applications on time with an acceptable level of quality, it means there are more projects that are failing to meet that standard than many software development teams might care to admit.

As more organizations become more dependent on software, however, tolerance for that level of failure will only continue to decline as businesses and leaders lose patience with application development practices that do not live up to expectations.

February 7, 2024
Why Startups are Betting Big on Low-Code/No-Code

It is exhaustive — from infrastructure to app delivery, from data to applications — to modernize your practices, processes and providers to ensure you have the underlying foundation to take advantage of whatever comes next.

Two or three years ago, apps created through low-code/no-code platforms were not usually as detailed under the surface as software developed from scratch, yet they sufficed for certain purposes. There was even a clear distinction between software developers and everyone else out of necessity because software development was incredibly difficult to master.

But now, as we head towards more advanced AI, the SaaS-based low-code/no-code (LCNC) platforms empower businesses to create software exponentially faster and cheaper than a code-based approach.

The future, therefore, of startups in this changing business landscape becomes even more competitive, taking into account the increasing importance of customer-centric UI modeling, data product development and cutting-edge technologies like machine learning and deep learning.

High-Value Acceleration

Applications built through traditional software development can take months or even years to launch. If you create a code-based solution, it’s also easy to blame others and there’s a lack of understanding if it’s not well-documented. “Low-code no-code functionalities can change this dynamic,” says Dr. Ingo Mierswa, an industry-veteran computer scientist and founder of Altair RapidMiner. “The visual workflow describes what it’s doing, so people can understand it, buy into it and take ownership of the results.”

Visual workflows can be understood and created by more people. “They make it easier to maintain and explain why anything, such as a model’s decision or a bug, has happened because the workflows are documenting themselves,” Mierswa articulates. “It would be hard to create such an explanation if you had to sift through 100,000 lines of code, especially if the code had been changed 17 times in the past two weeks.”

You need to iterate, you will make mistakes, and you don’t want to waste time with coding non-critical things. “My big advice for all starters is to always think about what’s critical for your product. What’s your IP? What do you need to own and customize? And if coding is not that important for your product, I think going low-code/no-code is always the better way,” Mierswa shares. “It’s more important for startups because you have less resources, less time, more competitive pressure and markets are moving very quickly.”

Everyone is a Citizen Developer

In the early 1960s and 1970s, there was a clear distinction between software developers and everyone else out of necessity. Software development was incredibly difficult and time-consuming to learn and master. But now with the onset of low-code/no-code solutions, everyone from custodial teams to C-level executives can create and customize applications that make their jobs more efficient.

Because no-code/low-code platforms are built on the fundamental premise. “People closer to the problem are better equipped to find simpler and effective solutions,” says Kissflow CEO Suresh Sambandam. “It’s a team sport, and accountability, the most important benefit of LCNC solutions, accelerates development and empowers even more people to collaborate the right way.”A big advantage,” Sambandam says, “is that CIOs of large enterprises can solve the IT talent shortage plaguing the industry today by leveraging expert business users who are ‘citizen developers.’ Many companies can’t avoid struggling to recruit and retain tech talent in a highly competitive market, but low-code/no-code solutions exceedingly help bridge that talent gap by freeing up the bandwidth of existing resources.”

At the highest level, you always need people who understand the business problems that need to be solved. LCNC functionalities also empower adjacent team members — designers, product managers, data scientists and others — to take part in the different stages of development as the product takes shape.

Reducing Technical Debt to Overcome Friction

In an enterprise scenario, most of the core systems like ERP, CRM, SCM in the front end, as well as in the back end, are bought from standard vendors. There is NO differentiation, as all the top enterprises buy from the same vendor. On the other hand, the bulk of differentiation comes from middle-office processes that are unique to an enterprise. Sambandam points out, “Often these are business processes that require heavy domain knowledge that frequently change to adapt to the evolving business conditions and hence business users are better equipped to build these workflow automations.” He highlights that conventional software development models don’t do justice to the agility required by business users and are too complex for them to use. While business users have the domain knowledge, the system would also require integration touch points with other enterprise systems. These integrations often require deeper technical expertise.

It is essential to create a cultural change within your organization, where it is not just about having the right tools, but also about investing in your employees and skilling them up, commented DevRev Head of Engineering Ahmed Bashir.
It still is hard work, and there will be friction. “Overcoming friction requires creating a tighter loop across departments and managers, closing the gap between developers and customers,” he underscores. “And LCNC platforms are designed to act as a system of action, surfacing real-time information that connects work with developers, customers and product.”

This is exactly where platforms like Kissflow, Blaze and Bubble, which support both no-code and low-code use cases come in and let business users handle the domain portion of the application (90%) and IT teams configure the integration points (10%). This greatly reduces friction between IT and business users, especially when it comes to solutions for long-tail internal workflows.

Long-tail applications include things like approval management, creative requests, incident management, customer onboarding and IT helpdesk. “We looked at the space and decided to tackle this,” Sambandam shares.

Long-tail apps and processes are often viewed by IT as a drain on time/resources they don’t have, yet it’s a lack of internal workflow processes automation that gum up digital transformation. “We found as we scaled the company that when you empower business users to quickly build and augment workflows that directly impact their day-to-day by giving them the tools to do so without advanced technical knowledge, they are motivated to make it happen,” he affirmed.

Security is a Business Enabler

The main objective of security is to protect organizations and people. Organizations prepare for potential security threats by implementing policies and ensuring they have the tools to mitigate and quickly respond to outside threats.

However many low-code/no-code adopters do not have security expertise on their team so they rely on the LCNC platform to offer security features readily available for the apps they build. Because creating applications with custom code can lead to a variety of security issues. And such applications are difficult to secure, with the majority of tested applications containing one or more session management vulnerabilities, and others with input validation errors.

Getting a firm handle on security means considering what to do. That sounds simple enough, but it’s surprising how many startups make blunders because of poor planning and shoddy strategies. “We’ve been looking at a trend. Companies want enterprise-level security like audit logs, SSO and other data security requirements for their apps,” commented Nanxi Liu, Co-CEO of Blaze. “Seeing this strong trend, we worked to design our platform with enterprise security and scalability in mind, since most early-stage companies do not have security expertise on their team and rely on platforms to offer advanced security features for the applications they build.”

Make, however, no mistake — there is no best security but the best practices. “That’s part of why,” Liu says, “compliance-driven businesses like financial services and healthcare companies that require the highest level of security to build their customer portals, consider enterprise-grade no-code and low-code platforms that provide the required level of security.”

With low-code/no-code platforms, your applications are secure with industry-standard protocols, data protection practices, and features to block unauthorized access.

To underscore: LCNC allows non-technical people to build complex apps without needing to be familiar with traditional software logic. You don’t need computational thinking because visual components in low-code/no-code platforms automatically have the logic built-in. “And now with the mind-bending AI breakthroughs,” Liu remarks, “users can tell AI in natural language the workflow or feature they need and see it instantly created.”

February 2, 2024
From Reaction to Robots: Riding the AI Wave in 2024

As we navigate another year of consistent zero-day breaches, legislative pivots, the explosion of AI tooling and threat actors growing bolder and more desperate, it’s safe to say that getting comfortable with change is a requirement for thriving in the technology industry.

We occupy a notoriously unpredictable space, but that’s half the fun. Compared to many other verticals, technology—especially cybersecurity—is relatively youthful, and the future should be something we can all look forward to blossoming in sophistication alongside the technology we swear to protect.

So, what can we expect in the industry in 2024? We put our heads together, looked into our crystal ball, and these were the results:

Government Regulations Around AI Will Turn the Industry Upside Down

It was the talk of the conference circuit in 2023, with several high-profile presentations at Black Hat, DEF CON, Infosecurity Europe and many more warning of the explosive changes we can expect from AI implementation across every industry, especially cybersecurity. As tends to happen with low barriers to entry for such transformative technology, adoption has outpaced any official regulation or mandates at the government level.

With significant movements in general cybersecurity guidelines and benchmarks around the world, including CISA’s Secure-by-Design and -Default principles in the U.S. and similar initiatives from the UK and Australian governments, it is essentially a foregone conclusion that regulations around AI use will be announced sooner rather than later.

While much of the debate surrounding the mainstream use of AI tooling and LLMs has centered around copyright issues with training data, another perspective delves into how AI is best used in cybersecurity practices. When it comes to coding, perhaps its most human quality is its similar hardship in displaying contextual security awareness, and this factor is deeply concerning as more developers are adopting AI coding assistants in the construction of software. This has not gone unnoticed, and in a time of increased scrutiny for software vendors adopting security best practices, government-level intervention certainly would not surprise.

… And Demand for AI/ML Coding Tools Will Create a Need for More Developers, not Less!

Much has been written about the AI takeover, and for the better part of a year, we have been subject to a plethora of clickbait headlines that spell doom and destruction for just about every white-collar profession out there, and developers were not spared.

After months of speculation and experimentation with LLMs in a coding context, we remain entirely unconvinced that development jobs are at collective risk. There is no doubt that AI/ML coding tools represent a new era of powerful assistive technology for developers, but they are trained on human-created input and data, and that has rendered the results far from perfect. Perhaps if every developer on the planet was a top-tier, security-minded engineer, we might see genuine cause for concern.

However, just as the average adult driver vastly overshoots their ability (notice how everyone says they’re a great driver, and it’s always other people who lack skill? That’s a classic example of the Dunning-Kruger effect!), so too does the development community, especially when it comes to security best practices. According to one Stanford study into developer use of AI tooling, it is likely that unskilled developers using this technology will become dangerous. The study claimed that participants who had access to AI assistants were more likely to introduce security vulnerabilities for the majority of programming tasks yet also more likely to rate their insecure answers as secure. This poses a significant issue; poor developers will be enabled to introduce security issues faster, and if anything, this will only increase the need for security-skilled developers with the knowledge and expertise to code securely and use AI technology safely.

We Will see Consequences for Software Vendors who Don’t Ship Secure Code

CISA director Jen Easterly has made it abundantly clear that software vendors should not be permitted to “pass the buck” when it comes to security within their products, highlighting that the current responsibility for software safety is largely passed to the consumer.

We have agreed with this view, and we believe it will take a shakeup of this magnitude to move the needle toward code-level security—not to mention appropriately educating developers—to be taken seriously.

While CISA’s powers only extend so far—essentially, they can only enforce secure-by-design practices to vendors that sell to federal agencies—this still presents a new security benchmark to hit for many large software vendors. Colonial Pipeline, SUNBURST and, more recently, the MOVEit data breach are all large-scale cyberattacks that affected government-level systems at some point, and with these new guidelines in play, there is a real possibility that future highly visible breaches will receive greater scrutiny and reprimand.

Reactive Security Will Start to be Seen as Old-School

As the goal of increased cyber resilience continues to dominate cyber strategies across multiple verticals, those who rely on reaction and incident response as the only core tenets of their plan will find themselves in a place of unacceptable exposure and risk.

Security professionals must act swiftly in the face of adversity and outright attack, but modern times call for modern solutions, and we simply cannot afford to take a less-than-holistic approach. “Shift left” needs to be more than a rapidly aging buzzword; code-level security should be prioritized, alongside upskilling and verifying the competence of the developers working on the software and critical digital infrastructure we take for granted. After all, who wants the security program equivalent of an Etch-a-Sketch when they could have an iPad Pro?

Now, more than ever, governments and enterprises alike must commit themselves to a preventative, high-awareness security program in which every member of staff is enabled to share responsibility. It’s not enough to cite the cybersecurity skills gap as a reason for falling behind; investment in security-aware developers and fostering collaboration between them and their AppSec counterparts should be a driving force in remaining as secure as possible, both as an organization and within the production of software.

Matias Madou, co-founder and CTO at Secure Code Warrior, contributed to this article.

January 10, 2024
Best of 2023: Are We Witnessing the End of Open Source?

As we close out 2023, we at devopsy.kinsta.cloud wanted to highlight the most popular articles of the year. Following is the latest in our series of the Best of 2023.

Hailed as a groundbreaking movement in software creation, open source software has become an indispensable pillar of modern software development. Often described as democratized technology, it has empowered countless individuals and organizations to collaborate and innovate. However, the community now stands at a critical juncture, and there’s some speculation that we’re seeing a decline in the open source space.

Why is Open Source so Popular in Modern Software Development?

At the heart of open source lies the power of collaboration. Unlike proprietary (closed source) software, open source invites participation from a diverse range of contributors, allowing them to work collectively to create and improve software. This inclusivity fosters shared knowledge and cooperation, leading to the creation of groundbreaking software.

In addition to collaboration, the open source development approach eliminates financial barriers and significantly enhances accessibility in software development. With open source software typically distributed under licenses that allow users to access, use, modify and distribute the software freely, this eliminates the need for costly licenses usually associated with proprietary software. This affordability allows organizations of all sizes to leverage innovative solutions without accruing high upfront costs.

Within the framework of open source software, transparency is ingrained. The accessibility of the source code makes it possible for users and contributors to scrutinize the source code, promoting trust and accountability. By promoting scrutiny and peer review, transparency ensures that flaws are quickly identified and addressed. It also empowers users to have greater control over their choices in technology.

With these benefits coupled with success stories from enormous projects like Linux, Apache and MySQL, open source positioned itself as having the potential to compete with proprietary alternatives, leading to widespread adoption across the technology industry.

More Projects are Jumping Ship

Given these numerous benefits, one might wonder: What challenges could arise with this approach? In recent years, there have been sentiments amongst contributors that the space might be losing steam. Their concerns are fueled by a few key factors that may erode or undermine the collaborative essence of the movement.

A huge factor is the commercialization of open source projects. A popular example of this phenomenon is the GitHub acquisition – GitHub, the home for open source projects for years, was acquired by the tech giant Microsoft in 2018. This news ruffled some feathers in the community, as Microsoft was seen as a proponent of proprietary software. It also raised doubts about whether GitHub would maintain autonomy over its decision-making process.

Regardless of these doubts, the community overcame the bump in the road rather quickly and continued contributing to the platform for years. However, in 2022, GitHub announced that a sub-program called Copilot, which open source developers had been working on, would emerge as a commercial product, with the profits going to GitHub itself. This development infuriated community developers, and they faced a rude awakening that any open source platform could decide to commercialize their projects without contributors sharing in the financial rewards or even gaining recognition for their work. Another open source project that has fallen victim to commercialization is OpenAI.

There have also been increasing concerns about whether open-sourcing core business models will cause more harm than good. Unlike in the past, it’s a lot easier for projects to become popular, creating a high possibility that someone else can steal the idea or offering and quickly create a commercial product based on the project.

These considerations are part of a broader discussion regarding the shifting dynamics and potential departure from the core principles within the community, causing participants to be uneasy.

The Dust Settles: Open Source Remains Strong

Despite the concerns and challenges facing the community, open source has undeniably transformed how we build and deploy software. We must approach the future of the movement with cautious optimism.

Drawing from past experiences like GitHub and OpenAI, it is clear that the community is resilient and adaptive. Contributors who do not agree with the acquisition of GitHub or the commercialization of OpenAI will take their gifts and contribute to projects whose values they agree with.

However, open source is at a crossroads and is grappling with challenges and changes that demand introspection. This introspection, hopefully, will lead to adjustments in the open source space that reaffirm its core principles and address concerns. Open source is not fizzling out, at least not yet; we can think of it as undergoing a transformative phase that will shape its future and ensure that it remains an important pillar in the software development landscape.

To hear more about cloud-native topics, join the Cloud Native Computing Foundation and the cloud-native community at KubeCon+CloudNativeCon North America 2023 – November 6-9, 2023.

December 22, 2023
Where to Start With Secure Design – Tips for Developers

Designing secure software is an increasingly hot topic, and for good reason. In recent weeks, we’ve seen a global effort from governments and cybersecurity agencies to encourage security by design through new guidance and changes to procurement rules. During the G7 in May, the QUAD nations (Australia, India, Japan and the United States) released a set of Joint Principles for Secure Software. In the U.S., there are even plans for legislation to make software makers liable for the security of their products. The need for software to be made secure at the design phase has been spotlighted further by recent cybersecurity incidents such as the hack by ransomware group CLOP that exploited flaws in MOVEit software.

But secure design is not a part of every developer’s skill set, and it’s not on the syllabus of every coding course. If you’re brilliant at writing code and keen to know more about how you make that code secure, where do you start?

Here are five tips from former DevSecOps and threat modeling engineer Fraser Scott, now IriusRisk’s VP of product:

1. Get to Know the Threats

To be able to design secure software effectively, you need to understand the threats. A great starting point is the OWASP Top 10. OWASP aims to identify the most critical security risks to web applications. Top of that list is broken access control, and insecure design itself sits at number four. Understanding the biggest risks is the first step towards mitigating them in software development.

2. Try Threat Modeling

The foundation of secure design is threat modeling, which is the process of identifying and mitigating plausible security threats. In its simplest form, this is about looking at your software design and asking Adam Shostack’s four questions:

1. What are we working on?
2. What can go wrong?
3. What are we going to do about it?
4. Did we do a good enough job?

The OWASP Top 10 gives you a sense of what the risks might be, and threat modeling is the process of identifying them in your design. See if any of those top ten risks might impact your design, consider what could go wrong and work out what you can do to mitigate it. Before you start writing any code, consider whether you’ve done a good enough job to make the software secure. The sooner you find design issues, the cheaper and easier they are to fix.

3. Connect With a Community

There are plenty of communities of people out there grappling with the same challenges that you are. Experts in security and threat modeling and developers are coming together to discuss and try to solve some of the key challenges in secure design. One such community is Threat Modeling Connect, where people use the forum to share ideas and discuss issues. Experts also host webinars and write blogs. The community also held its first-ever conference in October. There is also a community dedicated to threat modeling on Reddit, and the OWASP Slack has a #threat-modeling channel.

4. Experiment With Automation

Companies who are developing software on a large scale use tools to automate the threat modeling process. IriusRisk makes a community edition of its automation tool available for free, for life, to people who want to give threat modeling a try. As someone who is starting out in secure design, using a tool like this, which generates many of the relevant threats and countermeasures for you, is a great way to experiment with your first threat models. And we’re not the only ones; you can also use the OWASP Threat Dragon and Microsoft’s Threat Modeling for free.

5. Build a Library of Secure Code

One shortcut to secure design is using existing secure design patterns and road-tested components. As you develop software, keep a library of secure code that has been threat-modeled and scanned using application security testing tools that you can return to in the future. The Threat Model Cookbook on Git Hub has some sample threat models you can use as a starting point.

November 17, 2023
How Open Source Software Powers High-Impact Climate Science

According to the NOAA National Centers for Environmental Information (NCEI), the past nine years have been the nine warmest since record-keeping began in 1880, and 2023 is on track to set another record. Making the hard decisions involved in addressing this global crisis requires analyzing and understanding enormous quantities of data using sophisticated computational methods. Luckily, innovations from the open source software community are helping revolutionize climate science, letting researchers quickly derive insights from data.

Delivering Critical Collaboration

Modern science relies heavily on complex computational and numerical analysis techniques. However, the massive amount of measurement and simulation data available in fields like climate science makes traditional approaches impractical. With petabyte-scale data, researchers can no longer expect to download or obtain physical copies of entire datasets, nor can their local workstations or laptops handle datasets of that size once they arrive. Moving the scientists to the data is also impractical since researchers all over the world need to make use of the same data.

Instead, scientists and engineers from Pangeo.io and other global collaborations have developed a robust set of open-source tools for working efficiently with large, remote datasets, bypassing restrictions from proprietary software that’s limited to small datasets installed locally. Instead of monolithic commercial applications or inflexible Fortran codebases, modern researchers use flexible and lightweight approaches for distributing computation across cloud computing resources shared between many collaborators. These innovations are critical for climate science since further delay could lead to catastrophic challenges in combating global warming.

Deriving Faster, Flexible Data for Greater Output at Scale

Python’s scientific ecosystem, in particular, has gained prominence and is top of mind when considering open-source platforms best suited to execute climate solutions. Incoming graduate students more often know Python than any other programming language now that Python has taken over as the world’s most popular programming language. Python’s open-source ecosystem includes a wide variety of packages, each focusing on a specific area of functionality. These components evolve quickly and regularly introduce new features while maintaining compatibility through standard protocols. Unlike a monolithic, centrally designed approach, Python’s open ecosystem provides more flexibility when scaling data and solutions.

In high-impact fields like climate science, the goal is to enable scientists and researchers to concentrate on their domain-specific questions rather than becoming entangled in the intricacies of data manipulation and resource management. The numerical Python stack of open-source tools, combined with the ability to access available data in the cloud on a massive scale, facilitates this division of labor. Scientists can focus on writing software for the analysis parts specific to their expertise, meaning faster progress toward climate solutions and products.

With modern cloud-computing approaches, it makes more sense to move computation to the data rather than vice versa (since code is much smaller and more portable than data). With a platform like Jupyter running on a remote cluster and remote Python kernels as the execution environment, scientists can leverage the same infrastructure that hosts massive datasets in cloud object stores. Improvements in software technology and data formats also make it simple to access specific chunks of very large files, such as the temperature records associated with one particular city out of the entire world’s data. Accessing data becomes fast, cost-effective and scalable, bypassing traditional workflows involving repeatedly downloading large datasets consisting primarily of data not needed for a particular analysis.

For climate science specifically, this flexible remote-computing approach also means easier collaboration between developers and scientists to create innovations that could move climate studies and solutions forward. Distributed computing technologies such as Dask, Ray and Spark enable parallel and out-of-core computing, allowing for processing larger datasets at higher speeds. Several remote Jupyter-based services now exist, enabling even the main Python process (the “kernel”) to run remotely while utilizing the browser as an interactive development platform. Because all these tools are open, researchers can apply emerging technologies such as machine learning (ML) and artificial intelligence (AI) techniques as soon as they are developed without waiting for software and hardware manufacturers to catch up.

Creating Data Visualizations and Dashboarding

For humans to understand and validate the data and the steps involved in analysis requires visualization at every stage, which is difficult when the data is too large to transfer locally for display. New open-source tools like Datashader and VegaFusion offer a server-side rendering of even the largest datasets, efficiently rendering distributed datasets into a viewable image and then transferring only that image to a scientist’s local browser for display. Matplotlib, HoloViews, Altair, Plotly and hvPlot in Python, along with Makie.jl for Julia, now allow server-side rendering to be embedded into fully interactive plots in a web browser, letting researchers explore datasets and identify interesting features or detect issues with data quality or analysis steps. The Panel and Dash dashboarding tools in Python allow these plots to be combined into complete web applications for researchers to share their interactive analyses, all without any proprietary tools or restrictions (see this USGS wind turbine application as an example).

In a field with such a dire need for fast innovation, open-source technology is crucial to providing the creativity and speed required for climate scientists to fight global warming. The collaborative and transparent delivery of innovation is what the scientific community is increasingly counting on.

Dr. Martin Durant, staff software engineer at Anaconda, contributed to this article.

November 9, 2023
A Practical Guide to Mitigating DevOps Backlog

Every company aspires to get the first-mover advantage in its respective industry. This strategy allows companies to establish a robust brand presence and offer strong product offerings in the market. However, at times, this may also lead early-stage startups to prioritize pushing product features to market without adequate consideration toward stabilizing and maintaining the product. The above-mentioned approach may help win business and keep customers satisfied; achieving excellence in DevOps still remains an elusive goal. From operational issues to the lack of focus on streamlining DevOps, several factors add up to the already existing pile of DevOps backlog.

According to the Puppet DevOps report 2023, nearly 80% of organizations remain in the middle of their DevOps journey, experiencing varying levels of success within the team level but not across the entire organization. And, one of the areas that could hinder an organization’s success is the accumulating pile of DevOps backlog.

What is DevOps Backlog?

In typical software development teams, the primary focus is often on crafting functional and groundbreaking solutions, which is understandable given the constant drive to meet strict deadlines and deliver exceptional products. However, this relentless cycle of prioritizing outputs can inadvertently relegate the development process to a lower priority, resulting in the accumulation of tasks and issues contributing to the DevOps backlog.

DevOps Backlog consists of user stories, bug fixes, technical tasks, or any other work required in supporting and sustaining the development, deployment and maintenance of a product. It is dynamic and changes with time as and when new tickets get added to the list.

Key Factors Fuelling DevOps Backlog

Managing a substantial DevOps backlog can be one of the most complex challenges for companies to overcome. Within the CI/CD pipeline and the collaboration of multiple teams, intricate system architectures and more, several potential drivers contribute to the growth of the DevOps backlog. Let’s explore some of them:

Relying on a Small DevOps Team: In many cases, DevOps teams consist of a limited number of specialists who act as the lynchpin, holding various processes together. Larger development teams often rely on these smaller DevOps teams, resulting in dependencies that lead to blocked pipelines and delayed handovers, further adding to the DevOps backlog.

Architecture Changes: As development progresses and new services are introduced, architectures naturally evolve and change. This constant evolution necessitates new tooling and infrastructure requirements. Adapting to these changes and aligning new tools and frameworks with the evolving architecture contribute to the pile of the DevOps backlog.

Adapting to Changing Business Requirements: Business requirements are dynamic and evolve over time, adding complexity to the DevOps backlog. For instance, if your organization operates as a software-as-a-service (SaaS) provider or in the healthcare industry, you may need to establish new environments tailored to specific countries, regions, or regulatory frameworks. Managing these new environments and ensuring compliance with relevant regulations adds to the workload of the DevOps team and adds to the DevOps backlog stockpile.

Ticket Overflow: In many scenarios, developers do not have direct access to DevOps toolchains, resulting in an abundance of developer tickets that need to be processed by the DevOps team. This flood of tickets significantly reduces the bandwidth and efficiency of the DevOps team, impeding their progress in addressing the backlog.

Mitigating DevOps Backlog: Prioritizing Essential Strategies

While collaborating with companies to establish DevOps standardization, we initially assumed that the DevOps backlog challenge was limited to a select few. However, we were surprised to discover that even well-established companies in the later stages of product development struggle with significant DevOps backlogs without strategic interventions. Our conversations with over 200 organizations convinced us that the DevOps backlog is a prevalent issue faced by most companies today.

A Lack of Focus on Proactive Approach

The symbiotic relationship between business growth and technological advancements is undeniable. As time progresses, the architecture inevitably becomes more intricate, resulting in a growing number of tasks and challenges within the DevOps backlog. The complexities surrounding DevOps resonate with numerous companies, yet the proactive approach to addressing them remains an issue.

At our organization, we firmly believe that discussions revolving around DevOps stories are vital for organizations aiming to tackle this critical issue head-on. Now, let’s delve into the top 10 stories that software development teams can incorporate into their backlog to strike a balance between prioritizing software development and implementing essential process improvements.

1. Automating Provisioning and Configuration of Resources and Infrastructure
Proper provisioning and configuration of resources and infrastructure are vital for software development and deployment teams. By automating these processes and leveraging the appropriate toolchains and metrics, teams can streamline resource management, fostering consistent and efficient deployment practices. This, in turn, leads to enhanced productivity and improved overall development efficiency.

2. Application Life Cycle Management
Application life cycle management encompasses the comprehensive management of an application throughout its entire life cycle, from development to deployment and beyond. Teams that engage in discussions surrounding these stories take the necessary steps to streamline the entire life cycle. By leveraging toolchains and implementing proficient workflows, these teams achieve heightened productivity and maintain consistency across the application’s lifespan.

3. Release Management and Strategies
Stories related to release management and strategies go beyond mere release scheduling. They encompass crucial aspects such as zero downtime deployments and the implementation of effective rollback strategies. By prioritizing these stories, development teams can ensure smooth and seamless deployments, minimizing disruptions for end-users and optimizing the user experience.

4. Observability
Observability is the key to implementing effective monitoring practices for applications. This involves setting up alerts, capturing relevant metrics and configuring dashboards to track application performance and resource utilization. Prioritizing observability empowers teams to proactively identify and resolve any issues within the DevOps pipeline, ensuring a stable and reliable environment for continuous improvement.

5. Database Management
Database management entails activities such as upgrading and migrating databases, implementing scheduled backups and defining restoration policies. A robust approach to database management ensures the stability and reliability of the data infrastructure, mitigating the risks of data loss or downtime. By prioritizing this story, teams can safeguard their valuable data assets and maintain operational continuity.

6. Access and Permissions
Secure access and permissions are essential for developers across different environments, including testing, staging and production. This story emphasizes the importance of maintaining a transparent audit trail, identifying and rectifying permission leaks and ensuring compliance with security standards. Prioritizing access and permissions not only promotes a secure development environment but also minimizes the risk of data breaches or unauthorized access.

7. Environment Creation and Management
Establishing and managing environments is a complex and time-consuming process that requires collaboration among various stakeholders. From development to quality assurance (QA), pre-production and production environments, meticulous configuration and provisioning are paramount. By addressing this crucial need, teams can ensure well-prepared environments, minimizing discrepancies that may arise during the development and testing stages.

8. Security and Compliance
Security and compliance are fundamental elements that should be considered when designing and implementing an architecture. Adhering to the standards and regulations defined by the geographical countries of deployment is crucial. Neglecting this aspect could result in substantial fines, vulnerabilities and the need for extensive damage control in the future, wasting valuable resources. Therefore, prioritizing security and compliance is imperative for long-term success.

9. Cloud Cost Visibility and Optimization
The dynamic nature of cloud computing necessitates efficient monitoring and optimization of costs, striking a delicate balance between frugality and system performance. Teams must possess comprehensive visibility into cloud cost usage to identify areas of potential savings. By scrutinizing resource allocation, assessing utilization patterns and implementing cost management strategies, organizations can minimize expenditure without compromising the functionality and stability of their applications.

10. Exploring New Tools and Frameworks
It is imperative for organizations to continually explore new tools and frameworks that help elevate software development practices. By implementing zero-trust networks and network segmentation, teams can fortify their security measures, bolster network resilience, and proactively mitigate potential risks. By remaining at the forefront of technological advancements, teams harness the power of these transformative tools and frameworks to optimize overall efficiency and propel innovation to new heights.

Intervene Now to Tackle DevOps Backlog

Prioritizing product features over process can lead to complex issues in DevOps backlog. Solving this problem requires a major shift in mindset and a proactive approach toward mitigating DevOps backlog. Whether your DevOps backlog is in the initial stage or has reached a later stage where it is getting more complex, the time to intervene is now.

November 6, 2023
Five Great DevOps Job Opportunities

devopsy.kinsta.cloud is now providing a weekly DevOps jobs report through which opportunities for DevOps professionals will be highlighted to better serve our audience.

Our goal in these challenging economic times is to make it easier for DevOps professionals to advance their careers.

Of course, the pool of available DevOps talent is still relatively constrained, so when one DevOps professional takes on a new role it tends to create an opportunity for others.

The five job postings shared this week are selected based on the company looking to hire, the vertical industry segment and, naturally, the pay scale being offered.

We’re also committed to providing additional insights into the state of the DevOps job market. In the meantime, for your consideration:

Indeed.com

Intel
Chandler, Arizona
Cloud Software DevOps Engineer
$156,410 to $250,410

LinkedIn

General Dynamics Information Technology
Remote
Cloud FinOps Analyst
$122,336 to $204,360

CareerBuilder.com

Infoway Software
Doral, Florida
DevOps Lead
$120,000 to $190,000

SimplyHired

Peraton
Herndon, Virginia
DevOps Systems Engineer
$112,000 to $179,000

Dice.com

University of Virginia
Charlottesville, Virginia
DevOps Developer
$95,000 to $125,000

October 9, 2023
Will ChatGPT Replace Human Software Developers? Probably Not

Since the release of ChatGPT, there has been a great deal of hype around generative AI and how companies can leverage it to cut costs and democratize software and application development. Naturally, with discussions of cost-cutting and democratization come whispers about what will happen to software developers.

This is a real and valid concern, but software developers’ skills, expertise and creativity are still very much needed.

While generative AI and AI code generation tools like ChatGPT have shown some promise and potential benefits, they are still in their infancy—like many other innovative technological advancements. We also don’t know what scenarios they may present down the road or their true abilities when the technology matures. For instance, how will it integrate with other technologies? We don’t know what will happen when a ChatGPT-generated line of code breaks or needs to be changed. We don’t know if it can provide a novel solution to a unique problem or what security threats it will present.

Given these unknowns, technology executives should think twice about replacing experienced and creative technology talent, such as software developers, with AI code generators.

Will ChatGPT create novel code to solve a unique problem never encountered before? Probably not.

A Tale as Old as Time (Or at Least a Decade)

The technology industry has been searching for and developing new ways to make certain software development tasks much easier and more streamlined for years. One example of this is low-code/no-code.

The notion of simplifying application development and replacing software developers with laypeople (citizen developers) has been around for more than a decade now, as low-code and no-code solutions have grown more popular. These solutions have promised that companies don’t need technical talent to get their software and app projects off the ground.

However, if you look at the impact of these solutions today, their use can result in large amounts of technical debt and almost always require the skill of experienced software developers.

The reason? Building complex software and applications is extremely difficult; it’s an art.

Low-code and no-code solutions have their rightful place and can make things easier if a company is looking to launch a simple app or static web page. These solutions can increase the pace of development and time-to-market and enable everyday people without any development skills to facilitate them. However, they are not actually a complete solution and often overlook aspects of development that a human software developer would typically address. Without a skilled expert involved, low-code/no-code platforms often can’t solve a unique problem a company has.

So, how does this relate to AI code generators like ChatGPT? Here’s how.

A Similar Situation—With One Key Difference

When thinking about their place in the development process, AI code generators are not that different from low-code or no-code solutions. The thinking is that they will also enable non-technical individuals to create software and applications with ease.

Yet, there is one key difference—they promise expertise, too. But is the expertise coming from the AI code generator or the person piloting it? The answer is simple; it is not from the code generator.

There have been examples of companies and individuals that have tried using ChatGPT to build code, and they have appeared to be successful. However, without the input of the individuals using it, it never would have been able to create the code on its own unprompted.

This means that the expertise of skilled, human software developers is still very much needed, especially when building intricate software and apps that require complex functionality such as optical character recognition (OCR), internet-of-things (IoT) connectivity and more.

A Future With Software Developers

Because of the expertise still required to create elaborate and innovative software and applications, AI code generators will not replace software developers anytime soon, if ever.

The technology used in these tools is still in its infancy, presenting many lingering questions and potential challenges. We will see this confirmed by the trials and tribulations experienced by the few companies and executives attempting to replace software development teams with tools such as ChatGPT.

Knowing AI code generators follow a similar path as low-code and now-code solutions, companies and executives should use them as developer enablement tools. These tools should complement the work of software developers by empowering them to focus on the complex tasks software and apps today require.

September 14, 2023
4 Salesforce DevOps Adoption Challenges
In today’s ultra-competitive digital landscape, adopting DevOps practices has become crucial for organizations looking to accelerate their development processes and enhance software delivery. Salesforce is no exception to this trend.

Salesforce DevOps essentially refers to applying DevOps principles and practices to the development and deployment of applications and customizations on the Salesforce platform. When done right, it integrates development and operations teams and offers numerous benefits, including better collaboration, faster release cycles and improved code quality.

Furthermore, Salesforce DevOps empowers organizations to adapt rapidly to changing business requirements, respond to complex customer needs faster and improve their experience via process automation (for instance, streamlining the product configuration and price quote generation process with CPQ automation) and drive innovation more efficiently on the Salesforce platform.

However, while the advantages of Salesforce DevOps are evident, its adoption comes with its fair share of challenges. In this article, we will dive into the four core hurdles that organizations face when embracing Salesforce DevOps. Moreover, we will provide actionable tips to tackle these challenges head-on, helping you ensure a successful Salesforce DevOps adoption journey.

1. Siloed Development and Operations Teams

DevOps isn’t just about technology and automation; it is also about people. In the realm of Salesforce DevOps, one of the most significant challenges that organizations encounter is the existence of siloed development and operations teams. Traditionally, these two teams have operated independently, often with minimal interaction, leading to fragmented workflows and hindered collaboration.

This disconnect between dev and ops can impede the delivery of high-quality Salesforce applications and features. The lack of timely communication and coordination can result in misaligned priorities, delays in deployment, and an increased risk of errors during the release process.

Moreover, siloed teams can stifle innovation and limit the potential for continuous improvement. The development team may create features that do not align with the operational requirements or may not be feasible to deploy efficiently. On the other hand, the operations team might struggle to understand the intricacies of the development process, hindering their ability to provide valuable insights or feedback.

To overcome the challenge of siloed development and operations teams in Salesforce DevOps, organizations must foster a culture of collaboration and shared responsibility. Here’s how:
- Promote cross-functional collaboration: Encourage regular meetings and knowledge-sharing sessions between development and operations teams. Foster an environment where team members feel comfortable discussing challenges, sharing insights, and jointly working towards solutions.
- Break down communication barriers: Utilize collaboration tools such as Slack and Asana that facilitate seamless communication and information sharing between teams. This can help reduce misunderstandings and keep everyone informed.
- Align incentives and goals: Create shared objectives and metrics that encourage both development and operations teams to work together towards common goals such as improving product-market fit or feature adoption. Recognize and reward collaborative efforts to reinforce the importance of cross-functional teamwork.
2. Version Control and Source Code Management

Effective version control and source code management are vital pillars of successful Salesforce DevOps practices. However, many organizations with layered dev teams find it tricky, leading to complexities, inefficiencies, and potential risks in their development processes.

Version control is essential for tracking changes to Salesforce customizations and configurations over time. Without proper versioning, development teams may find it difficult to collaborate, understand changes made by others, or revert to previous states in case of issues. Additionally, source code management allows for better organization and control of codebases, ensuring consistent and reliable deployments.

To tackle the challenges related to version control and source code management in Salesforce DevOps, teams should implement best practices and use appropriate tools. Here’s how:
- Adopt a version control system: Implement a robust version control system, such as Git, specifically tailored for Salesforce projects. Git allows teams to track changes, manage branches, and collaborate efficiently.
- Establish clear branching strategies: Define clear and consistent branching strategies for different development scenarios, such as feature development, bug fixes, and releases. This helps manage parallel development efforts and ensures smooth collaboration.
- Encourage code reviews and collaboration: Conduct regular code reviews to ensure code quality, identify potential issues, and ensure compliance with best practices. Encouraging collaboration during code reviews fosters knowledge sharing and enhances the overall development process.
- Automate deployments and rollbacks: Integrate version control with your Continuous Integration and Continuous Deployment (CI/CD) pipeline to automate deployments. This reduces the risk of manual errors and facilitates easy rollbacks in case of issues.
- Educate and train development teams: Provide training on version control best practices and tools to ensure all team members are familiar with the processes and benefits of using version control effectively.
3. Continuous Integration and Deployment

Continuous Integration and Deployment (CI/CD) are core principles in the DevOps methodology that aim to automate and streamline the software delivery process. However, implementing effective CI/CD practices for Salesforce development can be a daunting task for large-scale enterprise companies, presenting them with various challenges along the way.

One of the primary challenges in CI/CD for Salesforce is the complexity of customizations and configurations. Salesforce projects often involve multiple components, such as Apex classes, triggers, Visualforce pages, and metadata configurations. Coordinating the integration and deployment of these components in a seamless and automated manner can be intricate, leading to potential bottlenecks and delays.

Moreover, organizations may encounter difficulties in ensuring the stability and reliability of their CI/CD pipelines. Incomplete or inaccurate tests, improper versioning, and inadequate rollback mechanisms can result in the introduction of bugs or errors into the production environment.

Here are a few ways to combat this challenge:
- Automate the build and deployment process: Use Salesforce-specific CI/CD tools that automate the build and deployment process, reducing manual intervention and ensuring consistency in deployments.
- Implement comprehensive testing: Incorporate automated testing into the CI/CD pipeline to validate changes and ensure code quality. This includes unit tests, integration tests and regression tests to detect issues early in the development life cycle.
- Use sandboxes for testing and staging: Utilize Salesforce sandboxes to test changes in environments that mimic the production setup. This allows teams to identify and address issues before deploying to the production org.
- Establish version control for metadata: Use version control for Salesforce metadata to keep track of changes and enable seamless collaboration among developers working on different features.
- Monitor and optimize pipeline performance: Continuously monitor the CI/CD pipeline to identify areas of improvement and optimize the deployment process for speed and efficiency.
4. Test Automation and Code Quality

Test automation and code quality are critical aspects of Salesforce DevOps that directly impact the reliability and performance of applications. However, organizations often have a tough time when it comes to implementing comprehensive test automation and ensuring consistently high code quality in their Salesforce projects.

Manual testing can be time-consuming, error-prone and insufficient to keep up with the speed of Agile and DevOps development practices. The lack of automated testing can lead to delays in identifying bugs, lower confidence in code changes and a higher risk of regression issues when deploying new features.

Furthermore, maintaining code quality becomes challenging as Salesforce projects grow in size and complexity. Without adequate quality control measures, development teams may introduce code that does not adhere to best practices, leading to suboptimal performance, security vulnerabilities and increased technical debt.

Here are a few tips to tackle this challenge:
- Implement automated testing frameworks: Utilize Salesforce’s built-in testing capabilities, such as Apex unit tests and Lightning Web Component (LWC) tests, to automate the validation of code changes and functionalities.
- Adopt test automation tools: Consider leveraging third-party test automation tools that can extend and enhance your testing capabilities, such as Selenium for end-to-end testing of Salesforce applications.
- Create a comprehensive test suite: Develop a comprehensive suite of test cases that cover different aspects of your Salesforce application, including positive and negative scenarios, boundary testing, and integration testing.
- Integrate testing into CI/CD pipelines: Incorporate automated testing into your CI/CD pipeline to ensure that code changes are automatically tested before being deployed to production.
- Conduct regular code reviews: As mentioned earlier, establish a culture of code reviews to assess code quality, identify potential issues and ensure adherence to coding standards and best practices.
- Invest in code analysis tools: Use code analysis tools such as SonarQube and CodeScan to identify potential bugs, security vulnerabilities and maintainability issues in your Salesforce codebase.
Wrapping up

Embracing Salesforce DevOps offers numerous benefits for organizations, including increased collaboration, faster delivery cycles, and improved software quality.

From breaking down silos between development and operations teams to establishing robust version control and source code management practices, addressing CI/CD complexities, implementing test automation, and prioritizing code quality, a lot goes into driving successful Salesforce DevOps adoption. Armed with the solutions, it’s time to implement them, adapt them to your organization’s unique needs, and embark on a journey of unmatched quality and collaboration in your Salesforce projects.
August 8, 2023