devopsy.kinsta.cloud

Tag: software development

Improving the DevOps Process for Mobile App Developers
Development teams—perhaps especially mobile development teams—have heavily invested in systems to automate their processes and accelerate the delivery of mobile apps. From build, test and release to tracking and monitoring, the mobile DevOps team depends on systems like Fastlane, Bitrise, Jenkins, Azure Pipelines and GitLab—and really, that list just scratches the surface.

The mobile app market moves so fast that automation is the only way mobile DevOps teams can keep up with their competitors and rapidly changing customer expectations. They have to iterate and release rapidly—the more frequently a publisher puts out new features, the more highly customers rate their app. In the GitLab 2022 Global DevSecOps Survey, 70% of respondents said their teams release code every day or every few days.

Unfortunately, there’s one major component of mobile app development that sits outside of these automated processes in most mobile DevOps teams: Security. For the most part, developers still implement security manually, and the process for ensuring apps are secure mostly comes down to code scanning and penetration tests. Again, from the GitLab survey, 53% of developers are running static application security testing (SAST), but unfortunately, the data from those scans is often not feeding back into developer workflows. Less than three out of 10 teams (29%) pull scan results into a report for developers.

Additionally, there’s still a large disconnect between the security and development teams. Nearly half (47%) of security pros said that developers miss more than three-quarters of the bugs in the code, leaving them for the security teams to find, and more than half (56%) said it was hard to get developers to prioritize fixing code vulnerabilities. In fact, prioritizing vulnerability remediation was security professionals’ biggest challenge. Additionally, it’s interesting that when it comes to shift left security, the emphasis appears to be on early code scanning, not actually building security into apps earlier in the process.

Data-Driven Decisions and Security Automation

First, DevOps teams need to integrate data about the security of their mobile apps early in the process so they can make informed decisions about what protections to incorporate into the next build. Scanning information and results from penetration tests clearly need to feed back to the development team as quickly as possible—there’s no point in doing these tests if the information remains unavailable and not acted on.

But mobile devices have the capability to collect and send much richer data about the security threats apps are actually facing in the field. By collecting this data, DevOps teams can make data-driven decisions about which threats are the highest priority to combat.

However, as noted earlier, it does little good to collect data if it’s never used. And the slow pace of manual security implementation does not allow DevOps teams to implement protections quickly or nimbly enough to keep pace with the rapidly changing threat environment. DevOps teams need to automate the build, testing, release, tracking and monitoring of security to the same degree as every other aspect of an app’s development. Specifically, they need:
- A system that can store, provide version control and audit security in every release
- An automated system that can build the desired protections into the app within the organization’s existing CI/CD processes
- Automated verification that the protections slated for implementation are actually included in the release
- A feedback system from data collected in the field, including data about the effectiveness of protections already implemented. This proves security measures are working and reinforces the value of the DevSecOps process.
With this combination of data and automation, mobile app developers can transition from DevOps attempting to shift left security towards a DevSecOps process that is completely data-driven. Instead of reacting to the latest threat that’s making headlines in the news or working off gut feelings about the direction the threat environment is moving, DevOps teams can look at trending threat data from their very own apps, slicing it according to growth, geography, device, OS version and a multitude of other filters. As a result, the organization can pinpoint exactly which threats are emerging as the next big thing so they can defend against them early.

And with the assistance of automation, the DevOps team can keep up with the trending data, building security protections into the app within days or even hours of making a decision on what to include.

To improve the DevOps process, security has to shift left, but moving code scanning and penetration testing earlier in the process won’t do much good to harden protection without real-time data from the field and automation for security implementation. What DevOps needs is a transition to data-driven DevSecOps.
March 13, 2023
Balance Software Quality, Dev Speed and Cost With CI/CD

In many cases, success in the business world relies on software, be it a compelling mobile app, desktop application suite, interactive website or even an embedded system for the IoT. Because of this need, companies with mature software development processes enjoy a significant competitive advantage over those without. Part of fostering this maturity involves crafting quality software with high velocity but low costs.

Modern software development methodologies, like Agile and its variants, play a key role in crafting applications in an effective fashion. As more companies adopted an Agile approach, the movement also fostered DevOps, a management structure combining software engineering and network administration teams while leveraging automation to build apps more quickly. In many cases, the ultimate goal for these adopters involved achieving the programming Holy Grail of continuous integration and continuous delivery (CI/CD).

So, let’s take a closer look at continuous integration and delivery and how it helps companies successfully balance quality, speed, and cost in their software development projects. In a competitive global economy, adopting this approach remains a crucial differentiator, whether or not a business also fully adopts DevOps. In either case, understanding the benefits of CI/CD helps your software company achieve its business dreams.

A Closer Look at DevOps

As noted earlier, DevOps somewhat grew out of the Agile movement, but it isn’t a software development methodology in and of itself. Instead, it involves a company merging its software development team and its network administration staff largely responsible for helping the process of building applications. It increasingly leverages modern technologies like containers and virtualization for development, QA and production servers and automation for builds and testing processes.

This organizational structure and mixture of tools combine to help companies build high-quality software with high velocity and at a lower cost. A mature community grew around DevOps, including tools, best practices, process experts and more. While DevOps originally served as the sole domain of large enterprises, now businesses of all sizes involved in software development benefit from adopting this approach.

Defining Continuous Integration and Continuous Delivery (CI/CD)

Continuous integration effectively automates the critical software development processes of builds and testing. It especially makes sense for shops using multiple programmers, as it works with the team’s source code version control system to ensure all programming changes are merged and remain in sync whenever a developer checks in their code.

Any check-in typically kicks off a process where unit tests and a build are automatically run to ensure those changes don’t “break the build.” In pre-DevOps days, this tended to be a highly manual effort, leading to inefficiencies in any software development project. In short, the team only discovered any integration issues after a manual test. Automation now plays a crucial role in making this aspect of application building significantly more efficient. CI remains a critical best practice in any DevOps shop.

Continuous delivery shares many similarities with continuous integration, especially its use of automation and the latest software development tools. However, instead of performing builds for internal developers and the QA team, CD focuses more on delivering a software product for release to a user community, customer or the public at large. Again, this is another part of the software development process that led to inefficiencies in the days before DevOps and automation.

In terms of software development for mobile apps, CD ends up being the last part of the development process before the app gets delivered to the App Store or Google Play. It also comes into play with any fixes or feature improvements after the app is on the market. A key aspect of engendering goodwill among your customers directly relates to the speed of app improvements. Adopting a CI/CD approach plays an important role in making app changes with high velocity.

CI/CD is the Heart of Successful Modern Software Development

Simply stated, if your company wants to be a successful software developer, adopting both continuous integration and continuous delivery remains crucial. Following an older, manual approach causes too many bottlenecks and inefficiencies to thrive in a highly competitive industry. No matter how great your startup’s idea or app design is, failure to deliver it with high velocity ultimately results in failure. Once again, this same concept applies to fixes and feature enhancements after the app’s launch.

At the same time, you don’t need to fully embrace the DevOps organizational structure by creating a separate team made up of software engineers and network administrators. At many startups with a small headcount, your developers might handle many of those network tasks, including managing virtual servers and containers. Hiring a network admin or two might be an unneeded luxury.

In the end, if you want to build your startup’s app in the most efficient manner, adopting the CI/CD approach makes perfect sense. It provides speed that allows your team to effectively balance quality, speed and cost, resulting in a great app without overspending. It’s the right approach for any successful software project.

March 10, 2023
The Impact of Developer Happiness on Productivity

Even in an era of unprecedented tech layoffs, most companies are still eager to find and retain top software developer talent. What’s more, many firms are discovering that they lack direct insight into their development teams’ needs and priorities. As a result, some of them are struggling to keep their developers happy and thus productive. A recent Zenhub report found, for the first time, a quantitative link between developer happiness and productivity, including what keeps developers in the same position instead of moving on to another job at the first sign of perceived trouble.

Software Development Happiness

The 2022 Software Developer Happiness report is based on a survey of 380 software professionals from a host of different organization types. In brief, the survey found that two of the biggest contributors to devs being unhappy in their roles were a lack of work/life balance and not working on what’s perceived as “quality work.”

Even with a looming recession and thousands of layoffs, there remains a dearth of senior software development talent. For more junior talent, there will be more available than in previous, pre-pandemic years. As companies strive to do more with less, they still require development teams to build their products and/or the applications that drive their business. Setting up more junior teams for success via collaboration and productivity tools will be key to navigating this new environment.

Given these findings, if organizations want to retain developer talent, they still need to make sure developers are working on meaningful projects and that they’re able to achieve that work in a way that balances the needs of the business with the needs of the individual.

An ideal example of this phenomenon is the developer tool ecosystem companies make available to their teams. While the growth of this ecosystem has been a net positive for developers, it has also created a lot of tool sprawl and significant integration challenges.

Less Time on Application Development

Stephen O’Grady, an analyst at RedMonk, an analyst firm focused explicitly on software developers, talked a lot about this through a term he coined called the “developer experience gap.” He is referring to developers spending more time integrating tools than working on application development or application code. That integration work comes at a real cost. It means developers are spending less time on application development, are having to work longer hours or do the integration work off the side of the desk.

The real bottom line to these survey results is that developers are happiest and stay at their jobs longer when empowered to focus on building code. This means spending less time interacting with management, attending meetings or resolving and integrating multiple solutions. The report’s results also emphasized the importance of work-life balance in encouraging developers to stay at their current job longer, while poor work-life balance accelerated the hunt for another job. These insights into developer retention are especially critical as, according to the report, 71% of developers haven’t decided how long they’ll stay with their current employer. The report also suggested that 33% of developers have stayed in a job longer than usual due to the pandemic, which could mean an increase in developer turnover when the pandemic comes to an end.

Another key finding is that collaboration challenges are viewed as the biggest impediment to developer productivity. Delays in feedback result in lost productivity; nearly 70% lose a minimum of three hours per week per developer and 75% spend at least an hour per week on reporting instead of writing code.

Finally, developers said their biggest priority is ensuring their software meets business demands, and most developers want high-performance productivity tools. In fact, 93% of developers believed these tools are critical to their success.

Increase Developer Happiness

So what can team leads to alleviate these issues? For both team leads and more senior management, the biggest way to increase developer happiness is simply to let them focus on coding. Developers like to showcase their skills, so encouraging them to work on interesting and impactful projects that challenge their skill sets is a great way to boost morale. Ensuring an appropriate work-life balance and paying the developers competitively are the other two recommendations for improving developer retention.

Additionally, senior management should take another look at how they are compensating their teams. Financial compensation will always be a critical element, but we should expect to see companies shift their focus to programs and benefits that their employees truly need. Particularly given the somewhat chaotic year in 2022 and continuing in 2023, those in the technology industry will shift to valuing organizational stability over just a paycheck. An improved focus on mental health, conflict resolution and work/life balance will also be a big part of attracting the appropriate talent to an organization.

The full report provides further detailed analyst recommendations for project managers, product owners, team leads and executives. The entire report can be downloaded for free here.

March 7, 2023
Taking the Modular Route

In today’s world of continuous software development and integration, removing bottlenecks and increasing efficiency is of utmost importance. One surefire way to improve this efficiency is leaning into modularity–or ‘taking the modular route.’ In software engineering, modularity refers to building complex products or processes using individually distinct functional modules.

An analogous way to describe this is thinking of modularity like you would Lego. Every Lego can connect to every other Lego using the same approach. But the way you put them together differs, from simple constructions to magnificent architectural creations, even though they’re made using components that connect in the same exact way.

Put simply, modularity is a deployment strategy that allows for quick configuration changes through standardized methods. In that sense, software development is much more manageable when it is modular.

A Way to Address Growing Complexity

Modularity has been key in helping to build complex business solutions. Before developers started using this approach, the risk of a single update or software release taking down the program was very high–and common. Fortunately, modules became popular and combatted that issue with their parallelity, allowing multiple teams to work independently without affecting each other’s work.

This cut down on the inevitable complexities that would emerge from the common issue of too many cooks (developers) in the kitchen (working on an application). And this applies all the way up the ladder to business leaders as well, who are able to now choose from various modules and plug the module into their ecosystem according to their specific requirements.

The feature-independent modules can be regularly updated and tweaked without affecting other parts of the ecosystem. This way of continuous development looks seamless to the end user without any downtime. And developers often employ prewritten modules, which conserves resources and makes modularity successful.

Promoting Innovation

Developing modules individually greatly increases the rate of invention. Each department or business could delve further into its operations by focusing on a specific module. Having many teams focus on the design of a given module fostered numerous parallel solutions. Modularity, in this way, helps innovation happen more quickly within organizations–teams are given the ability to expand to a marketplace model where third-party developers can create their custom modules that are easily integrated.

If you want to see a perfect example of modularity’s positive impact on innovation, take a look at how quickly the mobile app marketplace has grown. Thanks to modularity, third-party developers created their own modules based on set standards, which gave way to innovation and the mobile app marketplace. External developers can develop custom modules and plug them into a separate system.

Take a look at your phone and it’s evident in Apple’s App Store. It has thousands of apps–the majority of which are from external developers. If an ecosystem supports modularity, custom module development is made available to partners and clients, which not only expands the product scope but also helps to fast-track the launch of new products.

Standardization and Scaling Up

Modularity’s standardized function is a crucial advantage for developers. The same modules can be used across different businesses, which reduces the need to design custom solutions. During the development phase, which is fast-paced and demanding, this standardization improves product quality while reducing errors and oversights.

The business flows are designed with the same plug-and-play approach–giving the flexibility to change them with almost zero to minimal code changes. The plug-in nature of modules also allows complex systems to be scaled up with new modules.

During downtime, they can reduce the number of modules, which reduces costs and gives teams immense flexibility to manage that cost and resources over time. This is invaluable for changes like government compliance requirements that continually go into effect. Plug-and-play modules enable IT teams to maintain the continuous business processes.

Ease of Maintenance

Because of the standardization of the modular route, maintainability is considered an inherent part of system design. The ease with which a system is kept up efficiently and easily is crucial to IT teams, and modularity gives way to the maintainability of a product. New fixes or updates can be easily integrated in the independent modules and diagnosing quickly helps in reducing the turnaround time.

The end business user also does not need to worry about the internal details of the modules. They should have a high-level knowledge of what the module does and can integrate it into their system without being bogged down with knowledge of the inner workings.

Reuse, Reuse, Reuse

Reusability is one of the biggest advantages of taking the modular route because the same functional unit can be used in various instances, making for rapid development and, ultimately, faster time-to-market.

One real-life example of this is a company that needed a new connection to onboard new customers, which required custom configuration on the client’s end. Every time a new integration needed to be created, the process repeated. As any developer will tell you, the method was not scalable in the long run–and had massive maintainability issues. This is where the modular approach saved the day. The company designed a system where a new client connection could be configured using pre-existing modules and reused for multiple clients.

Collaboration and Interchangeability

Developers are not allowed to be one-person bands–the work requires constant teamwork across departments and teams. Modules are perfect for this–multiple teams can work simultaneously on different modules, which cuts down the laborious development cycle. Teams can work in different design patterns and develop unique solutions to improve product quality.

Interchangeability goes hand in hand with this–the business user can change an existing module and use a different module with minimal effort to the business flow. Options are then increased for the end user, and they can choose from many different solutions.

The modular pattern is not entirely new, but its success relies on the collective effort of IT teams and business leaders to do their part in designing and following a standard software framework. Only then will developers be able to take the modular route and deliver innovative, quality software for organizations across industries around the world.

February 13, 2023
Lineaje Unfurls Platform for Creating and Managing SBOMs

Lineaje this week unfurled a platform for creating and managing the software bills of materials (SBOMs) that are increasingly mandated by enterprise IT organizations and government agencies.

Fresh from raising $7 million in seed funding, Lineaje CEO Javed Hasan said the SBOM360 platform provides a simpler way to manage all the SBOMs that will soon inundate organizations.

The SBOM360 platform makes it possible to use a search engine to discover all the components of an application and all the dependencies that exist within it, including vulnerabilities, and indicators of compromise (IOCs) in seconds, said Hasan. The company then adds fingerprinting technology that enables the platform to attest to the authenticity of the entire software supply chain, he added.

While there is no shortage of tools for creating SBOMs, there are not as many platforms that enable DevOps teams to both create and manage them. In the wake of an executive order issued by the Biden administration requiring federal agencies to have an SBOM for all the software they use, interest in SBOMs to better secure software supply chains has significantly increased. However, very few organizations have a process in place that enables them to operationalize all the SBOMs that might be created.

More challenging still, the pace at which software is being updated has accelerated to the point where many SBOMs are likely to be outdated. The only way to keep pace with that rate of change is to rely on a platform designed from the ground up to dynamically create SBOMs in seconds, noted Hasan.

Finally, IT organizations are now being held more accountable for legacy software often polluted with vulnerabilities that need to be remediated, he added. As such, organizations need a tool that makes it simpler to identify vulnerabilities within the context of an SBOM.

It’s not clear whether greater awareness of the need for SBOMs is translating into concrete action, but the expectation is that more organizations will soon be following the lead of the federal government by requiring them. That shift will naturally bring a lot of additional pressure to bear on DevOps teams. Historically, organizations tended to allow developers to aggregate software components without always checking to see if they are compromised. Cybercriminals are becoming increasingly more adept at injecting malware into software components in the hope that malware will find its way into any number of downstream applications.

Regardless of the approach an organization takes to better secure software supply chains, more rigor is being applied to how software is built and deployed. The days when developers were free to aggregate software components as they saw fit by, for example, relying on open source software downloaded from a repository, are now at an end. The expectation now is the developer will be held accountable for the integrity of all components used within an application as organizations move to implement additional processes to more rigorously review code before it is deployed in a production environment.

February 9, 2023
5 Unique Challenges of Mobile App Testing

At first glance, testing a mobile app may not seem to be very different from testing a conventional desktop app. Mobile and desktop apps are often written in the same languages and hosted on the same servers. They must also meet the same basic user expectations in areas like loading speed and accessibility.

But when you dive into the details, you realize that mobile apps are fundamentally different beasts from desktop apps–and that, by extension, mobile testing requires a unique approach. You can’t simply take a software testing strategy that works for desktop apps and graft it onto your mobile apps.

This article breaks down five of the main reasons why mobile testing necessitates a different strategy from desktop application testing and which unique needs QA engineers should consider when testing apps.

1. Mobile Configuration Variability

Perhaps the biggest difference between mobile and desktop testing is that with mobile apps, engineers have a much wider set of possible configurations to contend with.

In the desktop world, there are just two main operating systems – Windows and macOS – and a relatively small number of OS versions for each family. And although there are many types of PCs and laptops, they all conform to the same basic hardware standards.

In contrast, there are over 24,000 distinct types of Android mobile devices out there – to say nothing of iOS devices, which add to the diversity of hardware. There is also a wider selection of mobile operating systems and versions.

What this means for QA teams is that there are many more variables to test for with mobile devices. It also means testing must be more efficient so that engineers can test for as many potential configurations as possible without delaying software release cycles.

2. Lack of Mobile Testing Standards

With a traditional, web-based desktop app, there is a consistent set of standards that the app is supposed to comply with when rendering content – specifically, the standards set by the W3C, a consortium that advocates for a standards-based, interoperable World Wide Web.

In the mobile realm, however, there is nothing equivalent to the W3C standards. Apps can render content in widely varying ways, many of which are device-specific.

Here again, this increases the need for specific testing teams to account for more variations and edge cases. With desktop apps, ensuring that the app complies with the W3C standards is often sufficient, but mobile testing is not so simple.

3. Unique Mobile Accessibility Requirements

Accessibility testing, which ensures that accessibility features like the ability to increase text size function properly, is important for delivering a great experience to all users, whether they access apps using desktops or mobile devices.

But with mobile devices, accessibility testing is harder, because there is more room for error when implementing accessibility features. For instance, the smaller screen sizes of devices – and the greater variation in average screen size – could mean that an increase in text size will cause the app to render some text off the screen. Or, a “nighttime mode” feature on a screen could result in lower-than-expected contrast between text and backgrounds, causing accessibility challenges for some users.

A mobile testing strategy needs to be able to accommodate risks like these, which aren’t as pronounced for desktop apps.

4. Mobile Environment Differences

By definition, mobile devices may be used in a wide range of physical settings. Depending on the location from which a user accesses a mobile app, application performance could be impacted by a variety of environmental factors that don’t typically apply for desktop apps.

For instance, limited network connectivity might undercut application performance when users who travel too far from cell towers. Or, energy-saving features on mobile devices that are running on low battery could decrease the speed at which apps render content.

Once again, these factors create additional risks that QA teams need to address when planning testing routines.

5. Higher Stakes for Mobile Testing

It’s a best practice to strive to deliver the best possible experience to all users, whether they are on desktop or not. But the reality is that poor user experience in mobile apps tends to have a more negative impact on a company’s brand.

The reason why is that it’s easy for users to highlight poor app performance by giving apps low ratings or leaving negative comments in marketplaces. They can’t do these things for most desktop apps, because, unlike mobile apps, most desktop apps are not downloaded through centralized marketplaces with user-rating features.

This difference doesn’t make mobile testing more technically challenging, and to be clear, I’m not saying you can skimp on desktop testing because users will abandon your desktop apps if they don’t work as they should, but it does increase the stakes of perfecting a testing strategy. Businesses simply have more to lose, reputationally speaking, from delivering buggy apps than they do from low-quality desktop apps.

Conclusion: The Need for Purpose-Built Testing Solutions

All of the above is why we need more testing strategies and solutions that are designed specifically for mobile.

Historically, QA teams have frequently tried to extend desktop testing strategies to address mobile as they’ve added apps to their catalogs, but that approach simply doesn’t work. Mobile apps and devices are too different in fundamental respects to be shoehorned into a desktop-centric testing routine. The sooner QA engineers realize this, the sooner they’ll optimize their user experiences.

February 1, 2023
Atlassian Extends Automation Framework’s Reach

Atlassian this week extended its namesake automation framework to include its Confluence collaboration software. In addition, it’s now possible to automatically create bi-directional branches in Bitbucket, GitHub or GitLab platforms from within the company’s Jira project management application.

At the same time, it’s now possible to automatically generate feature flags from within Jira through integration with the feature management platform created by LaunchDarkly and automate alert and incident responses stemming from the Amazon Web Services (AWS) cloud.

Dilani Kahawala, head of product for the automation platform at Atlassian, said these extensions to the Atlassian Automation framework are part of a larger ongoing effort to remove the silos within software engineering teams and the rest of the business.

DevOps teams, for example, will now be able to expand the number of those automated actions being taken because change requests can be automatically created. Atlassian estimated that IT teams will spend 25% less time managing change requests and save 115 hours per month.

To save more time, templates that automatically create Wiki pages in Confluence can be spun up at the completion of every sprint.

Since it first launched for Jira in 2020, Atlassian claims more than 10 billion automated actions have been executed across the Atlassian platform.

Kahawala said as more organizations realize how dependent they are on software to drive revenue, the need to close the divide between software engineering teams becomes much more pressing. She noted that changes to the timetable for one project can have a cascading effect across the entire business. At a time when organizations are counting on digital business transformation initiatives to remain competitive, the speed at which organizations share updates across silos is critical, said Kahawala.

At the same time, organizations are also striving to become more productive in an uncertain global economy. Automating rote tasks makes it possible for development teams to spend more time writing code, added Kahawala.

Less clear right now is whether businesses can absorb the pace of change that is enabled by modern software development. Historically, business leaders complained that their dependency on IT was holding the business back because it took too long for IT teams to implement changes. Today, however, many organizations routinely deliver code updates to mission-critical applications daily. While not every delivery may involve a major new update, it’s apparent that the pace of innovation is now occurring at speeds many organizations find challenging to absorb.

As organizations continue to adopt best DevOps practices, the pace of innovation is only going to accelerate further. The challenge—and the opportunity—is to ruthlessly eliminate as many manual tasks as possible to accelerate the pace of innovation. After all, software engineering pros have better ways to spend their time than manually creating yet another branch in a continuous integration/continuous delivery (CI/CD) platform or, worse yet, reenter data into a project management application.

Join the makers of Jira for a brand-new (free) Agile & DevOps event. Atlassian Presents: Unleash is designed to help your team collaborate from discovery to delivery to running great software. Get first access and “cheat codes” to the newest Jira Software features and products from Atlassian; network with peers, industry experts, and Atlassian product leaders; and solve real-life challenges around your Jira and toolchain configuration and workflow. Live in Berlin or digitally on February 9. Register for free.

January 26, 2023
The Database of the Future: Seven Key Principles

The database of the future will be more than just a database. Instead, it will be a platform that focuses on the user experience and abstracts the technology. Looking at the evolution of application development, an enormous amount of effort has made it easier to build and deploy applications. Similarly, databases need to evolve for ease of use with a delightful developer experience and an exemplary user interface that surfaces features and functionalities that engineers will actually use.

Here are seven principal features needed to chart the development of the future database.

1. Serverless by Default

Serverless is unquestionably the most significant paradigm shift in software engineering in the last five years and the one that’s been realized the least. There aren’t that many people at this point that are all in on serverless. It’s the shift away from buying a server to a service that does what you need and scales with your growth.

Serverless by default is the only way to ensure the database of the future will be easy to operate while being consistently reliable. Serverless is evolving from running functions in the cloud to becoming the standard for fast-paced development teams that don’t want to worry about vCPUs or node pools but focus on outcomes.

The companies that are doing it the best still deploy servers, but they are leveraging products that scale independently, regardless of how many servers they have. Their component vendors can increase the number of resources allotted to them, and they only pay for what they use, and the architecture of the solution scales in that way. They’re not buying a single server and pushing it as hard as they can. They’re telling one platform, ‘here’s my code,’ they’re telling another, ‘here’s my data,’ and then that scales up based on usage much, much farther than buying a single server.

When people think serverless, they think JavaScript and Lambda, but there’s serverless PHP out there, too. It’s not just the JavaScript community. Other software paradigms are going for this. We will see more and more people pulling this out when they need to do some quick code and they want to ship to production as quickly as possible and scale as quickly as possible.

2. Easy to Operate

The most significant advancements in DevOps have been freeing developers from complex operations that overwhelm their day-to-day operations. Engineering teams today are organized around product and business goals, not maintenance and servicing. But databases have yet to catch up. Product teams still rely on experts to perform overly complicated database operations tasks.

The database of the future must fit into everyone’s DevOps workflows. It must be usable for any developer and leveraged by every team within a technical organization. The database needs to be operable by everyone and no longer require database admins to manage backups and capacity planning or worry about disaster recovery.

Developer-centric features like database branching, database developer environments, autoscaling and usage-based billing will be commonplace.

3. Reliable

Reliable simply means it’s harder to hit the scaling limits. Not only does it mean that the database is there, but that you get the right information when something is going wrong, and you can collaboratively help scale.

Nothing scales infinitely. There are not enough computers in the world to scale something to infinity. You’ll always be able to query a database and pull too much information out of it too fast. No matter how many cars you get or how much horsepower you have, you can’t go a million miles an hour. A good database must let you know when you’re hitting those limitations and help you move past them, whether at a small scale, medium scale or large scale.

4. Interoperable

Future databases will be accessible to every imaginable client–from smart refrigerators to fleets of autonomous delivery vehicles. But this client-agnostic approach requires planning for many more transactions and connections than most current databases could ever manage simultaneously.

In the future, databases can handle unlimited connections through smart connection pooling, query replication and intelligent query routing. Databases will also need to serve a wide set of connections and clients. Although we are already seeing many databases offering web APIs and new connection types, as these get tested at scale, we will begin to see them adopted and used to power the next generation of applications.

5. Globally Distributed, Locally Available

Today, most businesses and applications don’t need a global database, which often introduces unnecessary complexity and operating costs. However, there are many reasons the future database will need to be globally distributed, as they must be located where you need them the most.

The key to this is knowing what your requirements are. Many people look at multi-region and think it’s faster reads, failover, scaling and breaking the speed of light. Ultimately, it’s all a series of tradeoffs. For example, when you read from a local replica, you trade off consistency because you have asynchronous replication. But this is really leveraging the global internet to deliver a better outcome where it’s necessary.

Being compliance-ready with regionally stored data enables you to scale rapidly to a new region and provides a great user experience by quickly serving locally available data. High-transaction, low-latency applications such as mobile games already need this capability.

But there’s one caveat: Global data distribution cannot break principle number two—being easy to operate. Tools will need to be elegant and have straightforward interfaces as their main goal is to make operations easier, not harder.

6. Intelligent

The future database is intended to be used by people and will be able to tell them how it needs to be used and how to make it better. All the tools you buy today can only give you the data you need to make some educated guesses. The future database will replace that with concrete recommendations, then self-healing actions.

This starts with basic data collection and observability and moves to, “let’s record every query that someone makes and use some decision engines and some AI and recommend how they can shard automatically.” It will be able to say, “I’ve recorded your database use for the last 30 days; this is how you should shard for optimal performance. Click the button, and we’ll do it.”

The future database will say, “You’re scaling too far for the single machine at a per-user level. You must break it down into two or break it down into four, and we figured out how to do it for you, so you don’t have to figure that out. We’ve sharded your data on your behalf.”

7. Linearly Horizontally Scalable

Future databases will also need to be designed for future scale. Only a handful of companies on the planet have hit a scale that tests the limits of current technologies, but as information grows, more companies will get there. The next databases will do that by automatically scaling both vertically and horizontally within seconds. Many existing databases require a maintenance window, replication and a failover before they can be scaled up or out. But downtime is unacceptable, and scale should not come at the cost of performance. And it doesn’t have to. The database of the future will scale intelligently and without a blip on the status page.

In short, the consumer should be able to get what they need when they need it, with no risks. It should be easy to say, “I’ve doubled the number of servers, I’ve doubled the amount of throughput. I’ve quadrupled the number of servers and the amount of throughput.” This is what sharding provides. Good sharding happens automatically at a super high scale, and you’ll only pay for what you use as you go along. The future database will do all of this without users having to turn a knob.

In conclusion

It will take the entire database industry to agree on these principal features, adopt a user-focused approach and build future database platforms designed to make engineers more successful by delivering superior products and services to their customers. I envision a day where every company no longer worries about its infrastructure and can simply focus on its business. I want development teams to take back control over their development workflow. Using a database should be as easy as writing code.

January 25, 2023
Why Can’t We Be Friends? Improving Developer-Tester Alignment

Sometimes coding can be such an art form that it deserves to be a masterpiece in a museum. Developers are proud of their code—at least until the so-called art critics (aka testers) step in with their critiques. And then the rapport can swiftly become as messy as a painter’s palette. You have probably seen this scenario play out time and again at your organization. In the mad dash of a software release, the relationship between developers and testers can quickly become stressful and strained. And while your teams know about helpful tools that can streamline collaboration, you may not be aware of other steps you can take that go beyond the screen to ensure an even better developer-tester relationship.

Full disclosure: At my organization, we’re in the software testing business and we’re not playing favorites when it comes to testers versus developers. We’ve just learned a thing or two about what makes a happy collaboration for all the parties involved. And we’ve seen that organizations that improved their developer-tester alignment not only accelerate a smoother release cycle but also leave no hard feelings on the table for the next time.

Leave the Keyboard, Take the Cannoli

Developers and testers will find that if they move their interactions away from the keyboard, they’ll discover that sweet spot of improved communication and transparency. Whether at your daily standup, over a cup of coffee or by simply picking up the phone, spending a few minutes chatting can address miscommunications before they become major problems that can delay your release.

If something seems off, bring it up as early as possible. Take, for example, the classic case of missed requirements. One of our customers recently shared a story with us that really hit home. At the top of the sprint, a particular user story included only seven acceptance criteria–which caught the attention of one of the developers. He pointed it out in a conversation with the testers and business analysts on the project. Together, the group identified four additional acceptance criteria missed during the initial scoping.

Without that conversation, the team would have assumed those seven acceptance criteria were the only ones that tests needed to be written against. A number of defects would have slipped into production, and it wouldn’t have been the software developers’ or the testers’ fault. Instead, the testing team realized significant gaps in test coverage and their test plan grew from seven to 40 test cases.

The takeaway? Make sure that developers, testers and business owners are clear on how business requirements align with what software developers plan to build, as well as how testers plan to test against them. Have this conversation before the sprint begins—and don’t forget the cannoli.

You Complete Me: In-App Test Management

“I love documenting test plans in spreadsheets!” said no one, ever. All that manual work just takes time, introduces errors and leads to frustrations among developers and testers who have to go digging for the information they need. Adding a Jira app, for example, for test management can be the key to completing the alignment between teams.

In-app test management gives both teams real-time data for decision-making and collaboration with built-in reporting on testing. That means that software testers can fully plan, design, track, and analyze testing activity within the same workflows that developers and testers are already using. For developers, there’s no need to ask testers for status updates or results. And testers can quickly see how tests align to requirements, which helps them brainstorm additional tests and provides developers better context and information to develop around.

The cherry on top? The entire team can maintain true traceability between requirements and test coverage, which both minimizes finger-pointing and ensures that everyone has a clear picture of what’s required before the release is ready to ship.

There’s an Awful Lot you can Tell About a Person by Their Software

As Mama always said, shoes tell where someone is going and where they’ve been (mic drop, Forrest). By talking the talk and walking the walk in a counterpart’s shoes, developers and testers can better understand each other’s worlds. In-app test management can give developers a window into the tester’s thought process, because it shows them not only test status but also how a tester arrived at a particular defect. After gaining this familiarity, developers may be more open to inviting testers to sit in on traditionally dev-focused activities.

Take unit testing or code reviews, for example. Most QA teams tend to steer clear. But by joining in, testers can ask questions and grasp which pieces will be validated at the unit, integration, and functional levels. A bonus is that testers get an inside look at the codebase, helping them develop more technical knowledge, as well as collaborate with developers on how the unit tests they plan to run–gasp!–fit into the overall picture of quality and release readiness.

Remember, the true purpose of QA is to provide information that verifies that the application works well enough to be put in front of customers. By understanding the testing that’s going on at the unit level, QA can be more efficient by focusing on what’s not being covered in unit tests.

The result? A more cohesive, effective test plan and a release that the whole team–developers, testers and business owners–can be proud of. In other words, a release masterpiece.

January 18, 2023
A DevOps Thanksgiving: What Are You Thankful For?

As crazy as it seems, here we are on the cusp of another Thanksgiving. This year has gone by so quickly, and as we enter the holiday season and the end of 2022, those of us in the DevOps community have much to be thankful for. First and foremost, I’m so happy to once again see my colleagues and friends in-person at conferences and events around the world. I’m looking forward to resuming travel not just in the U.S. but the rest of the world and seeing DevOps enthusiasts share their knowledge, their experiences and further the state of DevOps.

While we are thankful for the resumption of a more normal lifestyle, we also need to pause to acknowledge and mourn for those we’ve lost during this terrible pandemic. Their lives and the time taken from all of us during these past years can never be replaced.

They say every dark cloud has a silver lining; if there was a silver lining for DevOps during the COVID-19 pandemic, it was the unprecedented pace of digital transformation. Make no mistake—digital transformation is powered by software development and DevOps powers software development. It wasn’t just the sheer number of applications or lines of code that were developed and deployed during these past few years, but the accelerated maturity of software supply chain software pipelines that became the de facto standard for how software is developed and deployed; that was greatly accelerated as a result of the push to digital transformation.

It was not just software development that was accelerated, software supply chain attacks gave rise to an entirely new subset of security. Software supply chain security and SBOMs are now everyday words in the security lexicon. While I’m not happy about an entirely new class of attacks, I’m grateful for the response the White House and the U.S. government gave to our industry as a whole to address software supply chain attacks. I’m also grateful for the continued evolution of tools and techniques, people and processes around DevOps.

While some look to make marketing hay by proclaiming that DevOps is dead and other such nonsense, the truth is that DevOps is thriving. It is the way software is developed today. In fact, in conjunction with cloud-native, it would seem that we have a whole new stack and an entirely new way that software is created.

Looking forward, on the horizon we continue to see the development of AI that is making a real impact on observability, testing and application management. I think we’re also on the verge of AI having a profound impact on software development, as well. Low-code/no-code development tools paired with AI promise to take some of the most mundane software development tasks and automate them. This will free up overworked software developers to focus on higher-value software development tasks.

Despite macroeconomic conditions putting a damper on the entire tech space (if not the entire world economy), I remain enthusiastically bullish about the continued growth and maturation of DevOps and I am excited to see what the weeks, months and years ahead bring. I am grateful to be involved in our industry during these heady times, and I’m grateful to you, our readers, supporters and the community for your continued support of our company.

To that end, I wanted to announce that we’re trying something new this year at Techstrong Group, and we hope you’ll join us. We want to know what you believe makes tech strong—and we’re asking people to record a quick TikTok video or LinkedIn video with the hashtag #Techstrong. We’ll select the winners and are giving away two (2) Sony PlayStation5 game consoles to the best two entries as selected by our judges. You can get all the details on our social media channels on Twitter, Facebook and LinkedIn.

So, now you know what I’m grateful for—we really want to hear what you’re grateful for and what you think makes tech strong! Throughout the holidays and the new year, we wish you health and happiness—and know that tech will remain strong!

November 23, 2022