devopsy.kinsta.cloud

Tag: developers

How To Keep Engineers Happy and Reduce Burnout

Software development can be a mentally taxing profession. It requires intense mental focus, often under heavy workloads, with unclear job expectations and a lack of control. And with recent labor shortages due to layoffs and resignations, teams may also be understaffed, making matters more complicated. These issues can compound and lead to prolonged excessive stress—that’s when burnout hits.

In fact, one report found that 82% of developers surveyed indicated that they had experienced burnout in the last six to eight months. Some top causes of burnout include increased workload, a poor work culture and personal reasons. But whatever the cause, for 73% of these developers, burnout is negatively impacting their productivity or personal life. So, what can leaders do about it?

I recently talked with Randall Degges, head of developer relations at developer security company Snyk, to learn how to address ever-present burnout issues in software development teams at scale. Snyk, which provides popular automated vulnerability detection tools, recently closed a staggering $196.5 million series G funding at a $7.4 billion valuation. With that kind of momentum, they must be doing something right. According to Degges, there are a handful of tactics to consider to prevent burnout, like ensuring developers are heard, not assigning tasks without context and making improvements to the DevOps workflow.

Understanding Burnout

Burnout is an ever-present issue, explained Degges. It’s common within any high-stress job where you’re on the hook for constantly delivering new features to support the business. Specifically, in software development, burnout can quickly arise when developers don’t feel they have a voice. Or worse, when they’re tirelessly producing software they know won’t ever see the light of day.

A paper entitled Burnout in software engineering: A systematic mapping study took an interesting systematic approach and analyzed decades of research into burnout in software engineering. It found the following qualitative work-related factors to be most correlated with burnout. (Interestingly, the paper suggested machine learning might soon help produce more quantitative indicators of burnout.)

Causal-relationship of burnout, taken from ‘Burnout in software engineering: A systematic mapping study.’ (Deciphering this diagram gives me burnout!)

7 Ways to Fight Burnout

Since we know burnout is a pervasive problem, how can engineering leaders minimize the issue? Below, we’ll outline some specific strategies to consider and how they might help.

1. Give Developers a Voice

First, Degges encouraged leaders to ensure developers have a say in what they’re building. He recalled working within a large enterprise, sitting in a cubicle and having absolutely no say in the features his team was working on. This often included code they all knew would never make it into production.

Slaving away on irrelevant work is never motivating. Instead, a good internal culture provides a comfortable atmosphere where developers can raise issues early on, said Degges, hopefully before leadership puts targets destined to fail on the roadmap.

2. Provide More Context

On that note, when decisions are made, don’t throw orders over the wall in a vacuum. Instead, Degges recommended providing more context into what you’re all creating. Giving visibility into the reason behind decisions will align everyone on project roadmaps and create a shared purpose. This helps foster a culture of empathy and listening to others, said Degges. (David Shackelford affectionally calls this attitude HugOps.)

3. Optimize Communication Patterns

Optimizing communication patterns for your working environment is also vital to encourage a healthy culture, said Degges. For example, this could equate to optimizing for the physical reality of distributed teams—no one should have to clock into daily standup at 4:00 a.m. If you are working under timezone constraints, Degges recommended adopting more asynchronous communication.

For instance, his team relies heavily on Slack and provides dedicated channels for sharing ideas. They’ve also incorporated automation to summarize verbose Slack communications with a recap of conversations. In general, removing unnecessary meetings and optimizing communication patterns could greatly reduce the baggage an engineer might carry in their day-to-day work.

4. Practice Internal Developer Relations

As they say, practice what you preach. Many tech-driven companies have adopted a developer relations (DevRel) division to build a community with external developer consumers. But are you practicing DevRel internally? According to Degges, DevRel teams should act as a bridge between external and internal teams.

“Our goal is to not only to think and empathize with customers but also to encourage internal people to be authentic community members themselves,” said Degges. This can equate to empowering internal employees with resources like mentoring, blog writing or speaking opportunities, he said.

Additionally, other groups are emerging to help increase internal developer satisfaction. For example, platform engineering and developer productivity engineering (DPE) are being applied by many organizations to improve developer experience around internal tools and processes. Naturally, this will require working closely with internal developers to source feedback and make quality improvements.

5. Automate Away Security-Related Toil

Another method to upkeep engineering morale is through automating away toil. And security vulnerability discovery is one area full of painstaking manual tasks. Half of applications have high-risk vulnerabilities, and new exploits in open source libraries are being discovered almost daily.

As more software supply chain risks emerge, it’s challenging to keep up with the pace of vulnerability discovery and necessary software patching. Therefore, implementing real-time automated vulnerability detection, said Degges, can help remove many manual processes and free up energy to devote to other areas.

6. Hire Based on the Work, Not the Interview

Before developers even get the job, they can be put through high-stress on-the-spot tests during the interview process. Experiences here can set the tone for the entire engagement with the company. But activities like white-boarding algorithms are pretty disconnected from day-to-day work, said Degges. In his opinion, engineers should instead be hired based on the quality of their past performances and the work they’ve produced. This could be artifacts like their personal blog or contributions to open source projects.

7. Apply Artificial Intelligence (AI)

Lastly, what’s a tech blog post in 2023 without a mention of AI? Generative AI like ChatGPT has rocked the programming world and is being integrated into more development environments and engineering workflows. The future of AI looks very bright and is poised to help developers produce more with less time and resources.

Of course, automated code generation will require sanity checks, but these tools can certainly help many ancillary functions. For example, Degges’ team is using ChatGPT to spin up content outlines and automate keyword research, helping support their educational content production.

Soothing the Burnout

Even with new tools emerging and a newfound emphasis on developer experience, burnout is still prevalent. A full 58% of developers and security professionals said they are currently experiencing burnout, according to a 2022 Cobalt study. This is, perhaps, due to the rapid release cadences of today’s software delivery.

“In the past, it took months or years for new software updates to roll out. Now, developers only have a matter of weeks,” wrote Bill Mann, CEO of Styra, on Forbes. “Suffice it to say the stakes are high for today’s software teams.”

Burnout is psychologically taxing and can negatively affect employee happiness and retention. Therefore, it’s an excellent idea for engineering leaders to take it seriously and consider what positive cultural changes they can make.

April 6, 2023
Managing Engineering Teams During Economic Downturns

The tech industry has faced strong economic headwinds over the past year or so. We’ve seen thousands of layoffs at public tech companies, along with weak stock performances and a decline in venture capital funding. Although some analysts see light at the end of the tunnel, the current slump doesn’t seem like it’s quite over yet. As such, some IT leaders are learning to cope with a slashed budget and a leaner engineering team.

Simultaneously, engineers commonly desire more flexibility from their employers, both in terms of remote working opportunities and flexible hours. In fact, only 15% of developers are working 100% on-site, according to The State of Tech Hiring 2023 survey, which surveyed 14,000 professionals. In addition to being open to flexible working conditions, some organizations are examining how their internal tools could be improved to increase developer experience and productivity. Amid a downturn, leaders might need to consider these new strategies and many others to maintain steady progress.

I recently met with Amanda Richardson, CEO of CoderPad, to discuss tips to lead your engineering team during this downturn. According to Richardson, finding the right blend of contractors, remote work opportunities, and helpful tooling is necessary amid the current climate. Leaders can also adopt many creative methods to encourage productivity and motivate developers. Below, we’ll overview four ways technical leaders can build and sustain their teams.

1. Consider Hiring Contractors Vs. Full-Time Developers

Recruiters often struggle to fill highly technical positions like full-stack developers, backend developers or DevOps professionals. If a company can’t fill full-time roles, contractors are another valid option. According to Richardson, contractors provide companies with more flexibility because they are easier to hire and cut. These mercenaries could swoop in to pick up the slack by solving common tedious problems, like Salesforce integrations or less-sexy administrative work, she said.

Of course, contractors don’t solve every need, and they’re not bought into the company’s vision. That being said, more and more companies are leaning toward contractors to help fill technical gaps. In fact, 63% of recruiters now say their company hires contingent workers for tech needs, up from 42% last year, the report found.

2. Target Remote Work

Work-life balance has become a top requirement among today’s engineering workforce. For example, remote working options rank as the third-highest priority when looking for work, right behind salary and flexible working hours. Many engineers work better without the interruptions and noise of a traditional office space, explained Richardson.

Yet, the desire to work remotely is at odds with the policies of some large tech companies, who are attempting to push developers back into the office. Interestingly, this may present an opportunity for small-to-medium enterprises to embrace flexibility and get a leg up on the competition. “Give the people what they want,” said Richardson. “If you want to retain talent, remote work is what you must do.”

3. Use Tools to Fill in the Gaps

Another method for filling technical gaps amid a downturn is improving efficiency with new tools. For example, platform engineering is a strategy that aims to develop reusable, self-service platforms to manage internal infrastructure. Centralizing common automation and builds could bring more consistency to deployments and reduce grunt work, freeing up engineers and making them more productive.

Of course, we’ve also seen a lot of interest in new generative AI technologies, as well as development automation using low-code/no-code platforms. According to Richardson, leveraging AI and new automations presents a newfound opportunity to empower the workforce. For example, ChatGPT can solve interesting challenges in the prototyping phase, like generating a mock database or co-authoring support materials. Technical experimentation is a critical aspect of any team, and new AI capabilities could help bolster and support the operational sides.

4. Retain a Strong Culture

Lastly, maintaining a vibrant culture is another strategy to help retain talent. Richardson suggested internal hackathons and coding challenges to bring teammates together in an organic way. An internal hackathon could also be organized to rethink internal processes, providing an outlet for engineers to suggest innovative solutions. There’s also the basic stuff, like daily standups or brown bag lunches, that can create a space to share ideas. “It takes a good leader to figure out what’s going to engage their team,” said Richardson. (One business owner I interviewed actually condoned cooperative gaming during work hours to build community and has seen moderate success with this approach).

Final Thoughts

Of course, the above suggestions are easier said than done, and it will take a leader with a keen understanding of their unique engineering team’s attitudes and requirements to find their own version of success. On that note, talent comes in many shapes and sizes and traditional education isn’t everything when assembling a development team. In fact, a full 59% of developers do not have a university degree in computer science, the CoderPad report found, indicating that leaders may have to be more flexible when it comes to recruiting.

In general, following the above tips could be helpful when managing technical talent. Enabling the company to hire contractors and remote workers increases your potential talent pool, and investing in new tools and company culture could help retain workers. Considering half of today’s developers are considering a job change in the next year, leaders should act now to avoid high turnover before it negatively impacts their team’s output and morale.

April 3, 2023
Improving the DevOps Process for Mobile App Developers
Development teams—perhaps especially mobile development teams—have heavily invested in systems to automate their processes and accelerate the delivery of mobile apps. From build, test and release to tracking and monitoring, the mobile DevOps team depends on systems like Fastlane, Bitrise, Jenkins, Azure Pipelines and GitLab—and really, that list just scratches the surface.

The mobile app market moves so fast that automation is the only way mobile DevOps teams can keep up with their competitors and rapidly changing customer expectations. They have to iterate and release rapidly—the more frequently a publisher puts out new features, the more highly customers rate their app. In the GitLab 2022 Global DevSecOps Survey, 70% of respondents said their teams release code every day or every few days.

Unfortunately, there’s one major component of mobile app development that sits outside of these automated processes in most mobile DevOps teams: Security. For the most part, developers still implement security manually, and the process for ensuring apps are secure mostly comes down to code scanning and penetration tests. Again, from the GitLab survey, 53% of developers are running static application security testing (SAST), but unfortunately, the data from those scans is often not feeding back into developer workflows. Less than three out of 10 teams (29%) pull scan results into a report for developers.

Additionally, there’s still a large disconnect between the security and development teams. Nearly half (47%) of security pros said that developers miss more than three-quarters of the bugs in the code, leaving them for the security teams to find, and more than half (56%) said it was hard to get developers to prioritize fixing code vulnerabilities. In fact, prioritizing vulnerability remediation was security professionals’ biggest challenge. Additionally, it’s interesting that when it comes to shift left security, the emphasis appears to be on early code scanning, not actually building security into apps earlier in the process.

Data-Driven Decisions and Security Automation

First, DevOps teams need to integrate data about the security of their mobile apps early in the process so they can make informed decisions about what protections to incorporate into the next build. Scanning information and results from penetration tests clearly need to feed back to the development team as quickly as possible—there’s no point in doing these tests if the information remains unavailable and not acted on.

But mobile devices have the capability to collect and send much richer data about the security threats apps are actually facing in the field. By collecting this data, DevOps teams can make data-driven decisions about which threats are the highest priority to combat.

However, as noted earlier, it does little good to collect data if it’s never used. And the slow pace of manual security implementation does not allow DevOps teams to implement protections quickly or nimbly enough to keep pace with the rapidly changing threat environment. DevOps teams need to automate the build, testing, release, tracking and monitoring of security to the same degree as every other aspect of an app’s development. Specifically, they need:
- A system that can store, provide version control and audit security in every release
- An automated system that can build the desired protections into the app within the organization’s existing CI/CD processes
- Automated verification that the protections slated for implementation are actually included in the release
- A feedback system from data collected in the field, including data about the effectiveness of protections already implemented. This proves security measures are working and reinforces the value of the DevSecOps process.
With this combination of data and automation, mobile app developers can transition from DevOps attempting to shift left security towards a DevSecOps process that is completely data-driven. Instead of reacting to the latest threat that’s making headlines in the news or working off gut feelings about the direction the threat environment is moving, DevOps teams can look at trending threat data from their very own apps, slicing it according to growth, geography, device, OS version and a multitude of other filters. As a result, the organization can pinpoint exactly which threats are emerging as the next big thing so they can defend against them early.

And with the assistance of automation, the DevOps team can keep up with the trending data, building security protections into the app within days or even hours of making a decision on what to include.

To improve the DevOps process, security has to shift left, but moving code scanning and penetration testing earlier in the process won’t do much good to harden protection without real-time data from the field and automation for security implementation. What DevOps needs is a transition to data-driven DevSecOps.
March 13, 2023
Report Identifies Top 10 Open Source Software Risks

Endor Labs, a provider of a platform for managing open source software, published a report that classifies the top 10 open source software risks of 2023. The company published the list as part of an effort to better educate application development teams about issues that can lead to software supply chain compromises. That list includes:

Known Vulnerabilities: A version of a software component may contain vulnerable code that is accidentally introduced by its developers. When vulnerability details are publicly disclosed, there may not be a patch readily available.

Unmaintained Software: A software component may not be actively developed or maintained anymore, resulting in patches for functional and non-functional bugs not being provided in a timely manner—if they’re provided at all.

Name Confusion Attacks: Attackers may create components whose name resembles the names of legitimate open source or system components. This is also known as typosquatting. Bad actors might also attempt to mimic trustworthy authors (brandjacking) or play with common naming patterns in different languages or ecosystems (combosquatting).

Compromise of Legitimate Package: Attackers may compromise resources that are part of an existing, legitimate project or associated distribution infrastructure to inject malicious code into software components.

Outdated Software: A project may use an old, outdated version of a software component even though a newer, more secure version exists.

Untracked Dependencies: Developers may not be aware of a dependency on a component because it is part of another upstream module they employed.

License Risks: A software component or project may not have a license at all, may have one that is incompatible with the component’s or project’s intended use or whose requirements are not or cannot be met.

Immature Software: An open source project may not apply development best practices such as having a standard versioning scheme or lacking a regression test suite, review guidelines or documentation.

Unapproved Changes: A software component may change without giving developers the chance to notice, review or approve such changes because the download link points to an unversioned resource, a versioned resource has been modified or tampered with or due to an insecure data transfer.

Unknown Origin: Details about the source code, build process or the distribution process of a software component may be unknown or non-verifiable.

Endor Labs CEO Varun Badhwar said organizations need more visibility in the potential operational risks that come with increased reliance on open source software. That doesn’t mean organizations should use less open source software, but there are issues that many development teams may not always appreciate. A recent analysis of nearly 2,000 software packages published by Endor Labs, for example, found 95% of all application vulnerabilities can be traced back to a transitive dependency created when a developer employed an open source component.

Regardless of the root cause, responsibility for application security continues to be pushed left toward application developers. The challenge is it’s not possible to achieve that goal without first knowing what issues developers and the DevSecOps teams that support them need to be focused on. Perfect application security is, of course, unattainable, but any reduction in the number of vulnerabilities littering the software landscape will go a long way toward reducing the current level of stress developers and DevOps teams experience.

March 6, 2023
Taking the Modular Route

In today’s world of continuous software development and integration, removing bottlenecks and increasing efficiency is of utmost importance. One surefire way to improve this efficiency is leaning into modularity–or ‘taking the modular route.’ In software engineering, modularity refers to building complex products or processes using individually distinct functional modules.

An analogous way to describe this is thinking of modularity like you would Lego. Every Lego can connect to every other Lego using the same approach. But the way you put them together differs, from simple constructions to magnificent architectural creations, even though they’re made using components that connect in the same exact way.

Put simply, modularity is a deployment strategy that allows for quick configuration changes through standardized methods. In that sense, software development is much more manageable when it is modular.

A Way to Address Growing Complexity

Modularity has been key in helping to build complex business solutions. Before developers started using this approach, the risk of a single update or software release taking down the program was very high–and common. Fortunately, modules became popular and combatted that issue with their parallelity, allowing multiple teams to work independently without affecting each other’s work.

This cut down on the inevitable complexities that would emerge from the common issue of too many cooks (developers) in the kitchen (working on an application). And this applies all the way up the ladder to business leaders as well, who are able to now choose from various modules and plug the module into their ecosystem according to their specific requirements.

The feature-independent modules can be regularly updated and tweaked without affecting other parts of the ecosystem. This way of continuous development looks seamless to the end user without any downtime. And developers often employ prewritten modules, which conserves resources and makes modularity successful.

Promoting Innovation

Developing modules individually greatly increases the rate of invention. Each department or business could delve further into its operations by focusing on a specific module. Having many teams focus on the design of a given module fostered numerous parallel solutions. Modularity, in this way, helps innovation happen more quickly within organizations–teams are given the ability to expand to a marketplace model where third-party developers can create their custom modules that are easily integrated.

If you want to see a perfect example of modularity’s positive impact on innovation, take a look at how quickly the mobile app marketplace has grown. Thanks to modularity, third-party developers created their own modules based on set standards, which gave way to innovation and the mobile app marketplace. External developers can develop custom modules and plug them into a separate system.

Take a look at your phone and it’s evident in Apple’s App Store. It has thousands of apps–the majority of which are from external developers. If an ecosystem supports modularity, custom module development is made available to partners and clients, which not only expands the product scope but also helps to fast-track the launch of new products.

Standardization and Scaling Up

Modularity’s standardized function is a crucial advantage for developers. The same modules can be used across different businesses, which reduces the need to design custom solutions. During the development phase, which is fast-paced and demanding, this standardization improves product quality while reducing errors and oversights.

The business flows are designed with the same plug-and-play approach–giving the flexibility to change them with almost zero to minimal code changes. The plug-in nature of modules also allows complex systems to be scaled up with new modules.

During downtime, they can reduce the number of modules, which reduces costs and gives teams immense flexibility to manage that cost and resources over time. This is invaluable for changes like government compliance requirements that continually go into effect. Plug-and-play modules enable IT teams to maintain the continuous business processes.

Ease of Maintenance

Because of the standardization of the modular route, maintainability is considered an inherent part of system design. The ease with which a system is kept up efficiently and easily is crucial to IT teams, and modularity gives way to the maintainability of a product. New fixes or updates can be easily integrated in the independent modules and diagnosing quickly helps in reducing the turnaround time.

The end business user also does not need to worry about the internal details of the modules. They should have a high-level knowledge of what the module does and can integrate it into their system without being bogged down with knowledge of the inner workings.

Reuse, Reuse, Reuse

Reusability is one of the biggest advantages of taking the modular route because the same functional unit can be used in various instances, making for rapid development and, ultimately, faster time-to-market.

One real-life example of this is a company that needed a new connection to onboard new customers, which required custom configuration on the client’s end. Every time a new integration needed to be created, the process repeated. As any developer will tell you, the method was not scalable in the long run–and had massive maintainability issues. This is where the modular approach saved the day. The company designed a system where a new client connection could be configured using pre-existing modules and reused for multiple clients.

Collaboration and Interchangeability

Developers are not allowed to be one-person bands–the work requires constant teamwork across departments and teams. Modules are perfect for this–multiple teams can work simultaneously on different modules, which cuts down the laborious development cycle. Teams can work in different design patterns and develop unique solutions to improve product quality.

Interchangeability goes hand in hand with this–the business user can change an existing module and use a different module with minimal effort to the business flow. Options are then increased for the end user, and they can choose from many different solutions.

The modular pattern is not entirely new, but its success relies on the collective effort of IT teams and business leaders to do their part in designing and following a standard software framework. Only then will developers be able to take the modular route and deliver innovative, quality software for organizations across industries around the world.

February 13, 2023
Platform Engineering: Creating a Paved Path to Reduce Developer Toil

This article is part one of a series of three on the boom in platform engineering and its role in reducing cognitive load and creating a better developer experience.

Developer toil and cognitive load in the cloud-native space is real. The complexity posed by microservices, Kubernetes, and “software-defined everything” almost necessitated that ops needed to solve many of these issues to support cloud-native development.

Developers taking on full dev life cycle ownership and the requirement to shift left have resulted in cross-functional and operational concerns that have added even more complexity to already challenging workloads and unclear workflows.

To fuel productivity and consistency and enable developers to build and ship apps faster and more securely, one must understand what a ‘paved path’ might look like and where responsibility for different aspects of the development life cycle resides.

Supporting the Majority of Developers Without Alienating the Rest

Most developers want and need paved paths, particularly as paradigms shift; the adoption of cloud-native development being one such shift. These paths must help them reach the “safety” and productivity on which they are measured in their daily output and provide focus for completing their tasks. Freedom to make choices about processes and tools is not a priority because it ends up being noise that adds to the cognitive load and, in most cases, does not serve the company’s software shipping goals. This majority needs tighter guardrails and guidelines to do their work and get their code into the world without having to figure out too many operational parameters for themselves.

The cloud-native tooling landscape is vast, duplicative and overwhelming. Just trying to select and test tools could be a developer’s full-time job. Likewise, many organizations and developers have reported frustration—they are wasting a lot of time on repetitive work and routine tasks that deliver no business value, such as setting up environments and troubleshooting CI pipelines along with slow feedback loops in the development process.

As organizations increase pressure to deliver more with less and do so more frequently, these responsibilities make for a poor and unproductive developer experience and a far-from-ideal use of organizational resources. Stack Overflow’s 2022 Developer Survey reported that 62% of respondents spend more than 30 minutes a day fixing problems and looking for answers, leading to somewhere between 333-651 hours of lost time per week across a team of 50 developers. This reinforces a need for upfront investment in the paved-path developer platform, which eliminates much of this frustration.

It’s worth noting there is also a so-called 1% developer–one who wants to experiment and discover new tools on their own. Defining the developer experience within narrow guardrails ends up stifling this minority, which not only hampers their productivity but could stand in the way of innovation for organizations as a whole.

Building quality software requires accommodating both kinds of developers. For most development teams, the goal is to give them a clear route to building apps faster and shipping safely by prescriptively paving the path. The path also must provide enough freedom for adventure, allowing developers to stray from the path (within reason and where it makes sense).

Internal Developer Platforms: Paving Just Enough Path With Platform Engineering

Platform engineering and internal developer platforms (IDPs) are much-touted approaches to taming the toil. It is a part of the solution, and as it gains traction, it is becoming more mature. That is, instead of developers and development teams taking on full responsibility for every phase of the software life cycle and potentially cobbling together something resembling a developer platform, the industry trend is moving toward more comprehensive, commercial out-of-the-box platforms. Whether or not this addresses the actual needs of the development organization, this shift works because developer experience is now a business-critical requirement, and a centralized developer platform facilitates that experience (with flexibility enough to house all types of developer experience and developers working across industries).

The developer platform concept, supported by architecture, DevOps and SRE teams, contains the ingredients of transformation both for business and for developer experience. As the concept reaches critical mass and the “productifying” of developer platforms matures, one thing seems likely: By shifting to a platform model, development organizations are moving from ops-first to developer-first Kubernetes. The developer experience becomes defined by its optimized tools and workflows (and enables wider adoption of cloud-native development across larger organizations and broader industries).

Refocusing Developer Efforts: Abstractions, not Distractions

Cognitive toil will exist for developers regardless of the tools they’re offered. Technologies grow more complex and change frequently, and there are performance metrics that must be met. Taking the focus away from developers’ primary concerns—coding and shipping software and features—contributes to this toil without generating much return. As much as developers should be able to gain enough understanding of the way their code behaves and the consequences it can have downstream when shipping and running it, they should not need to become DevOps experts as a part of their work. This is why DevOps, despite predictions to the contrary, isn’t dead.

The paved-path developer platform gives developers exactly that: The ability to shift left without drifting right. The platform provides much-needed abstractions and tools that enable developers to refocus on priority work without distracting or overwhelming them with too many choices.

Becoming dev-first doesn’t kill off ops, either. Dev-first Kubernetes and a centralized developer platform elevate the importance of the developer experience while continuing to highlight the importance of DevOps in supporting and optimizing the developer experience. This is an entire topic in and of itself, which will be explored in part two of this article series.

February 7, 2023
Why GraphQL Will Reign King

After years of development, GraphQL is set to reign as the go-to API for data fetching. But what is GraphQL? Simply put, it’s a query language that allows you to fetch data from any server, and it’s a noteworthy alternative to the likes of other web service development styles like representational state transfer (REST) and simple object access protocol (SOAP). There are a few reasons why GraphQL will dominate the DevOps world, including its overall efficiency, ability to work with any server and overall ease of use.

Understanding GraphQL

GraphQL is a query language for APIs and a runtime for fulfilling queries with existing data. GraphQL provides a complete and digestible description of the data in an API, gives clients the power to ask for exactly what they need, makes it easier to evolve APIs over time and enables powerful developer tools.

Top Benefits of GraphQL

GraphQL addresses many pain points that often slow down developers, who prioritize adaptability and working quickly and efficiently when building applications.

For one, it is much more flexible than other data query languages, allowing users to specify exactly what data they want and in which format. Other data query languages require users to retrieve all data in a specific format, regardless of whether they actually need the data. That excess, unnecessary intake can cause delays and confusion.

GraphQL’s ease of use is also invaluable, thanks to its simple and intuitive syntax. The language is strongly typed, which means developers can catch errors early on in the development process before code goes into production. This can save much-needed time and money in the long run.

Another benefit is that it can be used to fetch data from multiple sources because it uses a schema that defines the different types of data available. This means if a developer has a field in a schema that could potentially return data from multiple sources, they can specify which source to fetch the data from. Developers don’t get this benefit from other data query languages, which often allow data fetching from only a single source.

Advantages for DevOps Teams Over Other Data Query Languages

As the world of software development continues maturing, so do the tools and technologies DevOps teams use to build and operate their applications.

REST and SOAP are popular web service technologies, but they come with some major drawbacks. Notably, both REST and SOAP rely heavily on URL structures and verbose XML payloads, which makes it difficult to evolve the API without breaking existing clients. And because each endpoint is a unique URL, providing a consistent experience across different parts of the API can be challenging. REST and SOAP are also quite chatty and can make it hard to optimize performance.

In contrast to REST, GraphQL provides a single endpoint for all data requests and is much faster than making many HTTP requests. GraphQL’s declarative query language specifies precisely the data needed – no more, no less – increasing bandwidth and improving performance. Applications can run faster and use fewer resources, which is especially important for mobile apps that need to be lightweight and fast.

Understanding the data is crucial for DevOps teams, and GraphQL allows developers to visualize data in a graph. Teams can see the relationships between different points more easily, leading to better-informed decisions about the best ways to manipulate the data.

The increased efficiency when conducting API calls is also a plus with GraphQL, which helps teams retrieve all information they need with a single API call. Traditional REST APIs often require multiple API calls, mainly because each resource in a REST API is represented by its own unique URL, along with its own set of permissions and data that needs to be retrieved. GraphQL can eliminate this bottleneck.

Overall, dealing with complexity is significantly reduced when developers use GraphQL. When using traditional REST APIs, writing a lot of code to handle different edge cases and error conditions is the norm. But GraphQL streamlines this – much of the complexity can be handled by the server itself, making way for simpler and more maintainable codebases. DevOps teams are also able to decouple their front-end and back-end development efforts by using a schema-based approach. GraphQL enables frontend and backend components to be developed separately without worrying about breaking changes, which means faster development cycles and fewer errors.

The amount of data that organizations need to understand, organize and query will only increase in the coming years. DevOps teams can get ahead of the ongoing data and information onslaught by taking advantage of GraphQL and its unmatched ability to provide an efficient, powerful and flexible API for data fetching and manipulation.

January 10, 2023
The DevOps Workflow is Broken. Here’s How to Fix It
While great progress has been made in bridging the gap between developers and operations engineers, fundamental differences in roles and responsibilities, alignment, and access to information hinder further progress.

Everyone is familiar with the DevOps infinity loop: This is an ideal workflow where every stage of the software development life cycle feeds into the next. While the philosophy is well understood, organizations have worked on bringing developers and SREs closer together under the DevOps banner. The reality is most of the time, the DevOps infinity loop is broken. Developers continue to focus on building features without the benefit of data-driven insights that production data can provide. Developers often “push and pray” without fully understanding how their changes will affect performance and the user experience. As a result, companies miss the opportunity to improve products faster and delight the customer sooner.

According to Gartner, 90% of DevOps initiatives will fail to fully meet expectations. And yet, the global DevOps market, which accounted for $7.3 billion in 2021, is expected to reach $37 billion by 2030. So to summarize, we’re all spending billions of dollars on the implementation of initiatives that are expected to fail in some meaningful way 90% of the time. The question is: What is failing and what can be done about it?

As the DevOps toolchain continues to expand, collaboration across tools is difficult and success depends on your organization’s ability to simplify and automate while achieving alignment. Developers use a variety of tools from various vendors in their daily work. The most widely used are VS Code from Microsoft, Jira from Atlassian and Slack from Salesforce. The most popular code hosts/repositories include GitHub, GitLab and Bitbucket. Bridging the gap are CI/CD vendors such as Jenkins and CircleCI. Ops engineers typically rely on other vendors for their daily work, such as New Relic, Datadog and Dynatrace. So, how do we ensure that all stakeholders in the SDLC have access to all the relevant information simultaneously, understand the objectives and get an integrated perspective of the different inputs and workstreams independently of the tools used?

There are three essential areas that DevOps initiatives must tackle to create alignment, collaborate successfully and streamline workflows. Let’s tackle each of these areas individually.
1. Collaboration. The first step is good collaboration. Most organizations rely on Slack or Microsoft Teams for this purpose, but they are both suboptimal when it comes to collaborating on code and incidents that point to code. Having code and data at the center of DevOps collaboration ensures that everyone has access to the information needed to tackle issues, reducing and even eliminating differences of opinion in the process.
2. Visibility. Second, ops engineers and developers must have visibility into the performance of applications across the entire tech stack. These three barriers can occur in different aspects of an organization’s operations and their impact can limit the team’s ability to deliver value to the customer just as much as workflow inefficiencies. Visibility helps the organization reach alignment on common objectives.
3. Context Switching. Third, given the number of processes and tools involved in a successful DevOps implementation, it’s important to work toward reducing context switching and integrating tools within established workflows. Integrating collaboration, tracking, history and observability into a single pane of glass can greatly improve clarity and time to resolution.
For DevOps teams, increased velocity and transparency are always worthwhile pursuits. However, it’s important to avoid myopic focus on workflows when other nagging problems could be hiding in plain sight.

By committing production telemetry, developer teams can adopt an objective, data-driven approach to software engineering—saving time and avoiding potentially misguided approaches based on opinions rather than facts. Visibility, transparency and velocity can help engineering teams meet ever-increasing requirements and provide their organization with the best possible business outcomes.
December 20, 2022
From Frontend to Full Stack: How I Leveraged Modern Dev Tools to Make the Leap

A few years ago, I had an idea about how to make publishing recipes on the internet more accessible and profitable. As a front-end engineer-turned-product manager, I could write code as long as it involved JavaScript. But I needed someone who could handle the whole stack, especially the back end—you can’t build a platform with frontend tools only. So I searched for a technical co-founder, poked around online forums and talked to friends. Eventually, I found a candidate (let’s call him Tom) who seemed like a good fit. He’d built a recipe app in the past and was comfortable with both the front and the back end. It seemed great. We struck a deal and dove in.

Tom started building the API and database while I made prototypes with a no-code builder called Bubble. I loved using Bubble. Without writing any code, I could build things fast and see what I was creating. Eventually, though, there were too many limitations with the no-code approach, including the lack of a Git repository for version control and history. So we decided to switch to full code with React Native mobile apps.

A few years prior, I’d switched career paths and became a product manager, but it had been a while since I had dug deep into writing code. And although I had a background in React, I felt insecure about building the frontend of a whole web platform on my own. But alas, you do all sorts of scary and undesirable things when you’re a startup founder!

After jumping into it, though, I quickly remembered that I really enjoyed coding. I picked up React Native pretty quickly and started building screens. Tom took care of the data while I took care of the UI. All was good.

Well, all was good until it wasn’t. We were a few weeks away from launch day and were finishing up. We even had a few early-access bloggers lined up to launch with us. And then, surprise! Tom decided to take a job as the CTO of another startup. His parting words were, “You’re technical enough; seems like you can handle it.”

I was floored. Aside from the front end, I had no idea how anything worked. We had cloud functions running on Pulumi hosted in Google Cloud, and I had no idea how they worked. I didn’t know how to train the machine learning AI models we used for ingredient recognition. The most I knew about SQL was that you used it to get stuff from a database. And to top it all off, our recipe scraping API was written in Python, a language I had no experience with, running a version that did not work on my M1 Mac. What in the heck was I supposed to do?

Well, I guess I did what any engineer would do. I figured it out step by step. When the Postgres database ran out of rows on the free Heroku plan, I figured out how to migrate it. When the recipe scraper API needed updating, I learned how to code in Python and deployed it with Docker. It was an incredibly stressful time. I was up against a looming launch deadline with a codebase teetering on collapse at any moment—mostly because I didn’t know how to hold it up.

The silver lining was the GraphQL API platform, Hasura. It was by far the easiest part of the stack for me to grasp. Understanding how our data was laid out was easy enough—I could simply explore it like a spreadsheet. I wasn’t sure how to build GraphQL queries, but the tutorials on the Hasura website helped me learn the basics and the query builder in the console did the rest. It’s no-code, but better—a no-code console makes it super easy to add new tables and modify permissions, and a Git repository keeps everything in check and easy to reference.

Once I understood how Hasura and the rest of the stack worked, I was able to finish the screens that I needed for launch.

Today, the company has grown significantly. We have a new co-founder, but she’s nontechnical. After figuring out how to run the entire stack, I decided I was technical enough to be the primary engineer. I’ve learned how to use all sorts of Hasura functionality that makes our lives so much easier—listening for events, firing off actions in our API to run business logic, and much more. Through all this, I now consider myself a full-stack engineer.

December 14, 2022
Index Tracks Impressive Growth of API-First Startups

APIs are the building blocks of modern software, and their use continues to grow around the globe. A full 89% of investment into APIs will increase or stay the same over the next 12 months, Postman’s 2022 State of the API report found. And the evolving API-first approach is ushering in many innovations and startups into the field. This trend is epitomized by the latest research from GGV Capital, which maintains an API-First Index of leading developer-first software companies.

The API-First Index tracks API-first companies that have raised $50 million in funding or more. And as of Q3 2022, the number of businesses that met this threshold increased to over 60. Just in Q3 alone, $350 million in funding was raised by companies on the index, and, in total, the index represents over $14 billion in funding. Undoubtedly, interest and investment in this space are expanding and will continue to increase in the year ahead. It’s a good area to keep tabs on, as the emphasis on modularity impacts how software is pieced together.

I recently met with Tiffany Luck, partner at GGV Capitol, who oversees the API-First Index. Below, we’ll explore the index a bit more and get Luck’s viewpoint on why these findings represent a unique moment for API-first technology and business models as a whole.

The API-First Index

The API-First Index focuses on companies that serve niche functionally through APIs; also known as API-as-a-product. The list includes an assortment of API-based tools from FinTech, communications, health care, commerce and other sectors.

For example, Stripe is likely the most well-known API-as-a-product, as it is commonly integrated as a way for businesses to accept payments. There’s also MessageBird, an API for sending notifications across any channel. Other examples include Mapbox, an API for geolocation data, Mux, an API for creating live and on-demand video experiences, Nylas, which offers a popular email API and Alpaca, an API for stock and cryptocurrency trading and many others.

Currently, the index doesn’t include the entire API platform landscape of tools, which ranges from areas like API management to testing, documentation, monitoring, design, security and observability. There are also middleware integration platforms to consider as well. With these included, the list of well-funded API-related toolsets would surely be even longer.

Why the Excitement About API-First?

The number of people deciding to build API-first companies is rapidly increasing. These API-based web components are interchangeable and represent the future of software development. “The world is going toward web and cloud components,” said Luck. The trend represents a global movement with much effort reflected in Europe, Asia and the U.S. Furthermore, API models seemed resilient throughout ongoing economic uncertainty and the patchy VC funding of 2022. “APIs are holding up very well compared to the broader VC landscape,” said Luck.

So, why is there so much excitement around APIs? Well, people are looking for best-in-class structural blocks that can be used to build common features like geolocation, AI, weather, payments or specific CRM functions. Niche APIs can swiftly deliver particular features without the bulk of a large platform. Another reason is that they’re relatively sticky—API-based delivery models are efficient at building a self-service platform that expands over time, which is a very keen prospect to investors, said Luck.

Unicorn APIs Solve Hard Problems

In evaluating API-first companies, Luck looks for those offering B2B solutions to super-hard, shared problems. This equates to reusable functionality that is difficult to build, maintain and scale individually. Furthermore, the solution must solve a need that’s common across companies but that is not core to their products.

For example, Luck has her eye on a few impressive API-first use cases. Stream, for example, powers in-app user-facing features. Knock offers solutions for handling notifications. But many of the top most-challenging areas include financial services, as this functionality is difficult to build from a regulatory and connectivity perspective. Examples include Pinwheel, which aggregates real-time income and employment data. Method API does something similar but for aggregating consumer debts.

Luck described APIs as a “modern way of accessing what would otherwise be technically inaccessible.” We’re seeing this action unfold in open banking, as new standards around the globe now mandate that consumer financial data be programmatically accessible, often in the form of an API connection.

Developer Experience: The New Competitive Advantage

In a competitive market surging with powerful developer-facing tools, non-functional aspects such as developer experience (DX), security, performance and reliability are becoming more of a focus. According to Luck, developer experience is probably at the top of the pyramid as it represents a catch-all for many of the aforementioned traits. Specifically, developer tools with high-quality DX have excellent documentation and must maintain backward compatibility to avoid breaking changes on the consumer side. “Ultimately, what you’re implementing impacts that customer’s end customer. Customers want zero negative impact and zero disruption,” she said.

The API-First Index is good for the industry as it helps educate the public on private investing rounds. And as discussion around it increases, it brings awareness and visibility to the benefits of building API-first.

December 13, 2022