devopsy.kinsta.cloud

Category: Continuous Delivery

Continuous delivery is a software engineering discipline that allows organizations to consistently develop and release quality software at a rapid pace. It is achieved by automating the software development lifecycle, enabling tasks such as development, testing and release to occur concurrently and continuously.

Digital Experience and the Future of Observability

Over the past two years, there’s been significant hype around observability in the IT operations and DevOps world. As shorthand for providing better insights, observability has seen a rise in lockstep with distributed architectures, cloud-native applications and the increasingly complex IT environments associated with them.

As we round out 2022 and move into the new year, there’s evidence observability is gaining mainstream adoption by IT operations and has begun to take off, with the market reaching a better understanding of what observability is and why it matters. This broader adoption offers a moment to reflect on the impact of observability so far and how it will continue to evolve in the year ahead.

What is Observability?

Though there’s ongoing debate in the industry about what observability actually means and what an observability platform should include, most people in the industry will argue observability is the evolution of monitoring and promises to offer more complete answers than standalone monitoring. Gartner, for example, first asserted in 2020 observability is “the evolution of monitoring.” But the two aren’t mutually exclusive, and observability and monitoring will continue to coexist and fill distinct needs for different teams based on what information they need about their environments and how they need to act on it.

So far, observability has had the most considerable impact on application performance and digital experience monitoring.

The days of monolithic applications are over. Instead, modern applications are highly distributed, are built natively in the cloud using container technology, are organized around microservices and leverage databases in multiple cloud environments. These increasingly distributed software stacks have introduced enormous flexibility for developers but huge complexity for operations teams, making it harder to deliver an optimal experience to the end user. The DevOps teams tasked with building and managing custom web applications are under pressure, and the result is, as one recent article so concisely put it, “Devs don’t want to do ops.”

Observability has provided a welcome respite for DevOps teams who need intelligent visibility into their environments to determine how applications are performing. This has made application performance a key focus for many observability providers.

An Intolerance of Bad Digital Experience

Ensuring external customer-facing applications are performing optimally is the key driver behind this. Digital businesses with external applications core to the company’s success need to ensure those applications perform optimally—customers are fickle. They’re intolerant of bad digital experiences, and alternatives are plentiful. Any e-commerce vendor knows no website equals no sales.

Traditional application performance monitoring (APM) tools look at telemetry to flag application issues or outages needing to be addressed. But understanding the health of your applications from this narrow standpoint isn’t the same as understanding how the applications perform from a user perspective. In other words, if you aren’t considering how applications are performing (or not!) for customers, employees and other users, you’re not seeing the forest for the trees. Observability helps solve this by elevating APM into a more holistic digital experience approach by adding visibility into how the end user engages with your application.

By providing a view into how an application is performing with the context of what’s happening throughout the entire stack—including networks, databases, systems, and more—and how issues are impacting the user experience, DevOps teams can shed light on previous blind spots.

Elevating the importance of digital experience through observability enables teams to understand and respond to issues more efficiently and provides critical data to help DevOps teams further improve their apps beyond just debugging and issue remediation. By understanding how users engage with and experience the application, you can gather critical data to improve or simplify the digital user experience. In our e-commerce example above, this could mean fewer clicks or an easier checkout experience (better digital experience = more sales).

Cloud migration efforts, coupled with ongoing hybrid and remote work and understaffed tech teams, make prioritizing what matters most even more important. Focusing on the experience of the end user—but with end-to-end visibility across the environment—allows stretched DevOps teams “do more with less.”

2023 Will be the Year of Digital Experience Observability

An observability approach prioritizing digital experience is the first step toward building a complete customer-centric environment. This eliminates the potential risks of missing issues directly impacting application performance and creates an enhanced, complete experience for every user. This will make 2023 the year of digital experience observability.

January 20, 2023
6 Ways To Empower Developers and Increase Productivity

A big goal of DevOps is to increase productivity. Yet many organizations still struggle with the efficiency of their software release processes. In fact, 54% of engineers stated they have too many manual steps within their deployment process, according to an Armory survey. And with the pervasiveness of multi-cloud, developers now face an escalating number of deployment environments, each carrying their own nuances.

Development teams want to move as quickly as possible without barriers. Yet, there are often hurdles to seamless deployments. For example, the code might be 100% ready to deploy, yet someone still must validate it before it reaches production. Manual bottlenecks like this in the CI/CD process can make developers feel like their code isn’t worthy of deploying and may even amplify feelings of burnout.

I recently met with Anna Daugherty, senior product marketing manager at Armory, to see what technology leaders can do to remove barriers within their software release cycles and increase productivity. According to Daugherty, many of these strategies boil down to having empathy for developers and their needs. “Developers aren’t machines; they’re people, and people deserve to have a quality of life when they’re doing their jobs.” Below, we’ll highlight tips and tricks to keep in mind to improve release fluidity.

1. Identify Where Your Manual Checks Are

The first step to speeding up CI/CD is identifying where manual steps exist within the development workflow. The DevOps life cycle, often visually depicted as an infinity loop, includes many states, such as design, build, test, deploy, assess and monitor. But if each step has one or more manual steps, you accrue a lot of manual debt, said Daugherty.

Therefore, engineering leads should seek to remove manual checks whenever possible. Of course, this may not be realistic within certain situations, such as financial institutions that require manual approvals at the last mile. However, everything should be smoothed out and automated up to that point, said Daughtery.

2. Shift From Continuous Delivery to Deployment

Depending on who you ask, the ‘CD’ in ‘CI/CD’ may stand for continuous delivery or continuous deployment. As part of reducing friction in the software release process, Daughtery encourages teams to begin to embrace the latter whenever possible. If deployment remains inhibited by unnecessary manual steps, it can decrease scalability and interoperability.

3. Increase Confidence With Testing

Of course, if we introduce more automation into the CI/CD pipeline, we must ensure the software is rigorously tested for errors and vulnerabilities. Daugherty encourages development to embrace automated, continuous testing approaches at every juncture, whether pre-production, inter-production, or throughout the application lifecycle. There are also advanced deployment strategies to consider, such as canary testing, which rolls out new changes only to a small subset of users or blue-green deployment can deploy to different places at different times.

These deployment strategies can help build confidence while reducing the risk of disrupting a majority of your user base with untested production updates. “The more you can integrate testing into your process, the more confident your process will become,” said Daughtery.

4. Chose The Right Toolsets

So, how do we implement more automation into the DevOps cycle? And what sort of tools should we look for? According to Daugherty, when scouting for DevOps tools, technology leaders should prioritize open solutions that aren’t closed-off environments. Instead, they should offer flexibility and the ability to integrate with the tools you already use every day. “You can have all the tools in the world, but if they don’t fit together, that can drag down the process.”

5. Consider Open Source Software (OSS)

On that note, consider open source software (OSS) where possible. OSS like Argo or Spinnaker has the potential to empower developers and bridge gaps in the CI/CD process. Other projects from the Cloud-Native Cloud Native Computing Foundation (CNCF) and Continuous Delivery Foundation (CDF) are mature with proven stable adoptions.

And although there is some weariness around OSS, particularly around unknown vulnerabilities and project maintenance concerns, “you can still be successful, secure, and scalable with open source,” said Daugherty. Open source also delivers community benefits and helps engineers feel like they are part of a bigger picture. “[OSS] is the opposite of burnout. It brings people together. It’s a community builder.”

6. Become Developer-First

Transitioning into a developer-first organization is another way to increase developer satisfaction and efficiency. But to become a developer-first organization, developers will need more say in the processes—this will involve balancing top-down requirements with total developer control. For example, leaders should suggest tools to enhance productivity but accept feedback on an ongoing basis to see what works best.

To support these efforts, the concept of developer experience (DX) has entered the lexicon in recent years. Building with developer empathy can result in more usable, frictionless software that increases productivity. DX is crucial within the burgeoning area of platform engineering, as the goal is to create reusable, internal self-service platforms for developers to leverage.

Interestingly, Netflix extolls what it calls full-cycle development, in which a developer oversees the building and operational aspects of a single component throughout its entire life cycle. “It’s an interesting way of addressing burnout because you feel more connected to what you create,” said Daugherty.

Final Thoughts: DevOps Isn’t Dead

Above, we’ve highlighted some potential strategies to increase developer productivity. By replacing manual bottlenecks with more automation, you can increase deployment fluidity. Considering how tools are acquired and iterating upon feedback from engineers can also help improve consistency while retaining quality talent.

Looking to the future, Daughtery hopes that similar roles such as DevOps, site reliability engineering and platform engineering can come together and join forces. Because, at the end of the day, they share many goals. And contrary to some grabby headlines, DevOps is far from becoming obsolete. DevOps is not dead. It’s very much alive and increasingly important.

January 20, 2023
Optimizing the Developer Experience in 2023

2022 was a banner year for innovation, but not only in the traditional “churning out new features” way. Many organizations started thinking about their processes and how to amplify them to maximize efficiency and employee retention. And when they started digging, companies realized that the developer experience was a major pain point that could be rectified–and a ripple effect of positive gains would result.

Companies started looking more closely at how the tools and processes developers use support and enhance their work. Great tools and thoughtful processes boost developer happiness and satisfaction. And when those levels rise, guess what else increased? Productivity, efficiency and employee retention rates.

We saw this trend of developer support take hold last year, but with more work to be done across many companies, we’ll see it continue in 2023 and beyond, impacting just about every facet of tech in new ways. Many companies have realized what does work by trying what doesn’t work and are making big changes from there.

Shifting Results

A major player in the changing developer experience has been the shift left approach that’s taken hold in recent years. About 80% of organizations have said they are adopting this idea of moving more of the responsibility for security and compliance to the front of the pipeline to avoid a compliance scramble at the point of delivery. The result, however, is a compliance process that ends up being more complex and less efficient, putting additional burdens on developers.

Involving more people creates more opportunities for security breaches and slows down the entire software delivery life cycle. It also shifts security responsibilities to developers and requires them to do significantly more than is listed in their job description. In fact, 60% of those shifting left said their development team is bearing the burden of the change. Rather than focusing on writing code, developers have been called on to understand regulatory policies, build regulatory controls into their applications or pipelines, inspect massive log files generated by more and more diagnostic tools for security info and spend more time in meetings to discuss audits. That’s not a recipe for fulfilled developers; it’s the path to a loss of innovation.

Stop Paying the Compliance Tax

When developers–or anyone on your team–are tied up in compliance tasks, they aren’t spending that time in profit-earning efforts. This “compliance tax” costs companies an estimated 15% each year–that’s 15% of their time spent managing compliance instead of innovating or providing services that add value. That’s not to say that compliance isn’t important–it’s essential, in fact–but limiting the burden of compliance improves the overall experience of developers.

When developers aren’t innovating, they’re not happy. They want to be writing great code, not translating regulatory governance policies. In today’s culture of quiet quitting, companies simply can’t afford to turn over compliance tasks to developers. It’s a waste of their talent and a surefire way to lose great developers and the chance to attract new ones.

Shift Left, but Do it Right

The good news is the compliance tax is largely avoidable. Instead of shifting responsibilities left, shift compliance everywhere. No one person or group, including developers, should be in charge of these tasks–that approach just isn’t comprehensive enough to ensure security and compliance throughout the software delivery life cycle.

Instead, security and compliance should be moved away from a point-in-time part of the cycle to a continuous process built into every step of the life cycle. CI/CD reduces the burden on any one person or group and is proven to be more efficient at catching anything that is non-compliant. When the tools are doing the heavy lifting, the people aren’t making as many mistakes and non-compliant code isn’t slipping through.

Baking in compliance allows developers to spend more time doing what they do best: Creating amazing features. This, more than anything, will boost the developer experience. Developers want to create and, when they experience the right environment and the right resources, innovation tends to happen.

Beyond the Developer Experience

Focusing on the developer experience pays off in more ways than just creating happy employees (though that alone is reason enough to make a change). Innovative developers are the backbone of leading tech companies. When they’re churning out new features that are getting public recognition and are touting a culture with a positive developer experience, companies attract more talented developers and the innovation loop continues.

So, if you’re looking for more innovation in 2023, start where you’ve always started: Your developers. Set them up for success by automating what can be automated and baking in the mundane – yet essential – tasks to truly optimize the developer experience. Then sit back and watch what incredible capabilities your developers can generate.

January 19, 2023
Why Can’t We Be Friends? Improving Developer-Tester Alignment

Sometimes coding can be such an art form that it deserves to be a masterpiece in a museum. Developers are proud of their code—at least until the so-called art critics (aka testers) step in with their critiques. And then the rapport can swiftly become as messy as a painter’s palette. You have probably seen this scenario play out time and again at your organization. In the mad dash of a software release, the relationship between developers and testers can quickly become stressful and strained. And while your teams know about helpful tools that can streamline collaboration, you may not be aware of other steps you can take that go beyond the screen to ensure an even better developer-tester relationship.

Full disclosure: At my organization, we’re in the software testing business and we’re not playing favorites when it comes to testers versus developers. We’ve just learned a thing or two about what makes a happy collaboration for all the parties involved. And we’ve seen that organizations that improved their developer-tester alignment not only accelerate a smoother release cycle but also leave no hard feelings on the table for the next time.

Leave the Keyboard, Take the Cannoli

Developers and testers will find that if they move their interactions away from the keyboard, they’ll discover that sweet spot of improved communication and transparency. Whether at your daily standup, over a cup of coffee or by simply picking up the phone, spending a few minutes chatting can address miscommunications before they become major problems that can delay your release.

If something seems off, bring it up as early as possible. Take, for example, the classic case of missed requirements. One of our customers recently shared a story with us that really hit home. At the top of the sprint, a particular user story included only seven acceptance criteria–which caught the attention of one of the developers. He pointed it out in a conversation with the testers and business analysts on the project. Together, the group identified four additional acceptance criteria missed during the initial scoping.

Without that conversation, the team would have assumed those seven acceptance criteria were the only ones that tests needed to be written against. A number of defects would have slipped into production, and it wouldn’t have been the software developers’ or the testers’ fault. Instead, the testing team realized significant gaps in test coverage and their test plan grew from seven to 40 test cases.

The takeaway? Make sure that developers, testers and business owners are clear on how business requirements align with what software developers plan to build, as well as how testers plan to test against them. Have this conversation before the sprint begins—and don’t forget the cannoli.

You Complete Me: In-App Test Management

“I love documenting test plans in spreadsheets!” said no one, ever. All that manual work just takes time, introduces errors and leads to frustrations among developers and testers who have to go digging for the information they need. Adding a Jira app, for example, for test management can be the key to completing the alignment between teams.

In-app test management gives both teams real-time data for decision-making and collaboration with built-in reporting on testing. That means that software testers can fully plan, design, track, and analyze testing activity within the same workflows that developers and testers are already using. For developers, there’s no need to ask testers for status updates or results. And testers can quickly see how tests align to requirements, which helps them brainstorm additional tests and provides developers better context and information to develop around.

The cherry on top? The entire team can maintain true traceability between requirements and test coverage, which both minimizes finger-pointing and ensures that everyone has a clear picture of what’s required before the release is ready to ship.

There’s an Awful Lot you can Tell About a Person by Their Software

As Mama always said, shoes tell where someone is going and where they’ve been (mic drop, Forrest). By talking the talk and walking the walk in a counterpart’s shoes, developers and testers can better understand each other’s worlds. In-app test management can give developers a window into the tester’s thought process, because it shows them not only test status but also how a tester arrived at a particular defect. After gaining this familiarity, developers may be more open to inviting testers to sit in on traditionally dev-focused activities.

Take unit testing or code reviews, for example. Most QA teams tend to steer clear. But by joining in, testers can ask questions and grasp which pieces will be validated at the unit, integration, and functional levels. A bonus is that testers get an inside look at the codebase, helping them develop more technical knowledge, as well as collaborate with developers on how the unit tests they plan to run–gasp!–fit into the overall picture of quality and release readiness.

Remember, the true purpose of QA is to provide information that verifies that the application works well enough to be put in front of customers. By understanding the testing that’s going on at the unit level, QA can be more efficient by focusing on what’s not being covered in unit tests.

The result? A more cohesive, effective test plan and a release that the whole team–developers, testers and business owners–can be proud of. In other words, a release masterpiece.

January 18, 2023
Devs Increasingly Rely on IaC to Manage Multiple Clouds

A survey of more than 300 DevOps professionals conducted by Firefly, a provider of a platform for automating cloud provisioning, found that more than a third of organizations (36%) have allocated the equivalent of two or more full-time engineers per month to writing infrastructure-as-code (IaC) templates.

A full 82% are also deploying the templates within the context of a DevOps pipeline, with 68% reporting more than half of their cloud instances are now managed using IaC tools, according to the survey. A total of 27% reported they are now managing every aspect of IT as code.

The challenge is that 61% of respondents also admitted they can’t automatically detect configuration drift over time, with more than a third (35%) spending days or weeks remediating IaC templates. Well over half of respondents (56%) said they are also retrofitting existing cloud instances using IaC tools.

Cindy Blake, vice president of marketing for Firefly, said this issue is likely to become more challenging to address as the number of clouds in use continues to increase. The survey found 44% of respondents already use multiple cloud providers. Overall, 42% reported having more than 10 cloud accounts.

The top IaC benefits identified by survey respondents are improved efficiency (75%), increased reliability (55%) and easier compliance (52%) and simpler troubleshooting (49%).

Not surprisingly, the survey identified knowledge (46%) followed closely by complexity (43%) as the top challenges organizations face when using code to manage cloud infrastructure. Factors increasing complexity included cost and complexity (70%), followed by security (61%).

In addition, only 14% of respondents reported that all their engineers are IaC literate. More than a third (36%) said they only have one IaC-literate engineer. As a result, many organizations are now attempting to navigate IaC bottleneck issues that could be alleviated if they relied more on automation that can be integrated within a continuous integration/continuous delivery (CI/CD) platform, noted Blake. She added that many organizations are already allocating too much engineering time to manually writing IaC templates.

Those manual efforts often result in misconfigurations that are easily exploited by cybercriminals, added Blake. Many developers using IaC tools such as Terraform have had little to no cybersecurity training, so the odds they will make a mistake are high.

At a time when more organizations are focused on the productivity of DevOps teams and ensuring the integrity of software supply chains, there is a clear need to automate processes that increasingly span multiple clouds. More organizations than ever are employing multiple clouds to ensure application availability in the wake of a series of outages over the past few years, Blake said.

Regardless of the motivation, multi-cloud computing is here to stay. Developers opt for different clouds based on the performance capabilities they can provide for various classes of workloads. There may be one cloud platform that runs the bulk of the workloads, but the number of organizations that have standardized on a single cloud are comparatively few. The issue is finding the best way to manage multiple clouds that doesn’t require as much manual effort so organizations can reduce the total cost of managing IT environments.

January 11, 2023
Broadcom Survey Reveals Spike in VSM Adoption

Value stream management (VSM) is gaining traction as organizations look to find ways to engage customers more efficiently via digital business transformation initiatives.

A survey of 500 executives, IT and business leaders conducted by Dimensional Research on behalf of Broadcom published this week found 39% of respondents implemented a VSM platform with another 16% having some equivalent capability. Another 31% said their organizations is moving in the direction of adopting a VSM platform. A total of 60% of respondents indicated that they will be using VSM to deliver products in 2023, with 95% of organizations that have adopted VSM also engaged in digital transformation.

Laureen Knudsen, chief transformation officer for Broadcom, said the primary goal is to close the divide that has emerged over the years between software development teams and the rest of the business. More than two-thirds of respondents (68%) reported there are disconnects between the business and software development.

Overall, the Broadcom survey identified increasing customer value (58%), improving product quality (55%), reducing costs (51%), adding more customers (50%, increasing profits (49%) and increasing speed to market (45%) as the top strategic issues organizations face in 2023.

The top benefits derived from investing in VSM were increased transparency (42%), improved organizational alignment (39%), faster solutions to customers (38%), data-driven decision-making (37%) and improved access to data (36%).

VSM traces its lineage back to lean manufacturing methods, which called for each step of a manufacturing process to be continuously measured. As software development has evolved and organizations realize how dependent they are on software, there is a growing appreciation for the value of monitoring things such as the impact of missed software development deadlines on the business. At their core, VSM platforms aggregate DevOps metrics and then match them to a set of key performance indicators (KPIs) defined by business leaders. Once those metrics are gathered and correlated, organizations can apply advanced analytics. The goal is to apply algorithms and other forms of predictive analytics that will enable organizations to more easily identify the precise impact software development processes have on the business.

Knudsen said the survey makes it clear a lack of discipline when it comes to software engineering now needs to be addressed as organizations realize how dependent they are on applications to engage customers. Many organizations have, for example, adopted flawed approaches to agile development that wind up creating chaos across the software development life cycle, she noted.

Many organizations, for example, are unsure how much time development teams are spending on retiring the technical debt accrued within legacy applications versus actually working on developing new applications, Knudsen added.

Business conditions tend to change rapidly, so aligning software development with corporate goals is difficult. Nearly three-quarters (72%) of respondents said they are frustrated by the state of the business and constant changes to strategy. Digital business transformation initiatives, however, will require organizations to adopt a more structured approach to software engineering, said Knudsen.

The issue, of course, is the pace at which businesses are already being digitally transformed. The accelerated pace requires almost immediately re-engineering software development processes for them to succeed.

January 10, 2023
Why GraphQL Will Reign King

After years of development, GraphQL is set to reign as the go-to API for data fetching. But what is GraphQL? Simply put, it’s a query language that allows you to fetch data from any server, and it’s a noteworthy alternative to the likes of other web service development styles like representational state transfer (REST) and simple object access protocol (SOAP). There are a few reasons why GraphQL will dominate the DevOps world, including its overall efficiency, ability to work with any server and overall ease of use.

Understanding GraphQL

GraphQL is a query language for APIs and a runtime for fulfilling queries with existing data. GraphQL provides a complete and digestible description of the data in an API, gives clients the power to ask for exactly what they need, makes it easier to evolve APIs over time and enables powerful developer tools.

Top Benefits of GraphQL

GraphQL addresses many pain points that often slow down developers, who prioritize adaptability and working quickly and efficiently when building applications.

For one, it is much more flexible than other data query languages, allowing users to specify exactly what data they want and in which format. Other data query languages require users to retrieve all data in a specific format, regardless of whether they actually need the data. That excess, unnecessary intake can cause delays and confusion.

GraphQL’s ease of use is also invaluable, thanks to its simple and intuitive syntax. The language is strongly typed, which means developers can catch errors early on in the development process before code goes into production. This can save much-needed time and money in the long run.

Another benefit is that it can be used to fetch data from multiple sources because it uses a schema that defines the different types of data available. This means if a developer has a field in a schema that could potentially return data from multiple sources, they can specify which source to fetch the data from. Developers don’t get this benefit from other data query languages, which often allow data fetching from only a single source.

Advantages for DevOps Teams Over Other Data Query Languages

As the world of software development continues maturing, so do the tools and technologies DevOps teams use to build and operate their applications.

REST and SOAP are popular web service technologies, but they come with some major drawbacks. Notably, both REST and SOAP rely heavily on URL structures and verbose XML payloads, which makes it difficult to evolve the API without breaking existing clients. And because each endpoint is a unique URL, providing a consistent experience across different parts of the API can be challenging. REST and SOAP are also quite chatty and can make it hard to optimize performance.

In contrast to REST, GraphQL provides a single endpoint for all data requests and is much faster than making many HTTP requests. GraphQL’s declarative query language specifies precisely the data needed – no more, no less – increasing bandwidth and improving performance. Applications can run faster and use fewer resources, which is especially important for mobile apps that need to be lightweight and fast.

Understanding the data is crucial for DevOps teams, and GraphQL allows developers to visualize data in a graph. Teams can see the relationships between different points more easily, leading to better-informed decisions about the best ways to manipulate the data.

The increased efficiency when conducting API calls is also a plus with GraphQL, which helps teams retrieve all information they need with a single API call. Traditional REST APIs often require multiple API calls, mainly because each resource in a REST API is represented by its own unique URL, along with its own set of permissions and data that needs to be retrieved. GraphQL can eliminate this bottleneck.

Overall, dealing with complexity is significantly reduced when developers use GraphQL. When using traditional REST APIs, writing a lot of code to handle different edge cases and error conditions is the norm. But GraphQL streamlines this – much of the complexity can be handled by the server itself, making way for simpler and more maintainable codebases. DevOps teams are also able to decouple their front-end and back-end development efforts by using a schema-based approach. GraphQL enables frontend and backend components to be developed separately without worrying about breaking changes, which means faster development cycles and fewer errors.

The amount of data that organizations need to understand, organize and query will only increase in the coming years. DevOps teams can get ahead of the ongoing data and information onslaught by taking advantage of GraphQL and its unmatched ability to provide an efficient, powerful and flexible API for data fetching and manipulation.

January 10, 2023
Southwest Airlines: ‘Shameful’ Technical Debt Bites Back

Welcome to The Long View—where we peruse the news of the week and strip it to the essentials. Let’s work out what really matters.

20 Years of Neglect Led to ‘Meltdown’

Last month’s débâcle of canceled flights was caused by decades of technical debt. That’s the analysis of Columbia University professor Zeynep Tufekci. (more…)

January 5, 2023
How Embedded Analytics Can Transform Future Businesses

It’s no secret that big tech companies like Google, Amazon and Netflix are not only leveraging data but also delivering embedded analytics and insights as a standard functionality of their software, services, applications and toolkits. The analytics appear to consumers as recommendations, usage habits and other helpful information. Such insights not only help consumers extract the most value out of the product they are using but also help providers understand how they can provide better services. Businesses that embed analytics to provide a differentiated user experience also have a direct impact on engagement and revenue.

Have you considered how embedded analytics could differentiate your company’s own software and hardware products? Let’s understand how embedded analytics can help transform technology businesses:

1. Analytics are expected and welcomed in your products

By embedding analytics into existing enterprise software and applications, users can drill down into real-time and context-aware data (for example: How busy a location will be at various times of the day) and can benefit from faster data-driven decision-making without worrying about application compatibility. Commonly, software applications provide in-app analytics to delight and serve their customers, while hardware companies can provide helpful stats on a web portal or app.

2. Embedded analytics is easier than ever today

Cloud technology has completely disrupted tech-enabled industries. It’s easier than ever for software, apps and hardware to connect to a data repository to save logs that are a rich source of analytics. The invention of the S3 bucket and object store has made it fairly cheap to keep log data. Finally, the transition to platform-as-a-service (PaaS) and software-as-a-service (SaaS) has fundamentally added an “easy button” to tracking consumption patterns and resource utilization.

3. Embedded analytics helps design better products and services

Embedded analytics can give businesses more insight into application telemetry (how customers use the data, how they engage with the product) and provide insights into the challenges they face from a usability standpoint. Providers can use advanced analytics to quickly iterate, draw patterns and make predictions, troubleshoot customer challenges, discover new avenues to monetize, detect and mitigate security threats or create entirely new features and applications that put customers on a faster path to improved application experience and better decision-making.

4. Embedded analytics increases operational efficiencies

The tech industry is transitioning from traditional monolithic architectures where applications are self-contained to microservices architectures where different components are spun up as needed via API calls. By embedding analytics directly into developer tools, businesses can manage resources without sacrificing availability or performance. Companies can allocate and optimize resources dynamically and understand peaks and valleys in consumption. Businesses must be able to leverage historical and real-time data to build predictive models to proactively identify and mitigate issues with individual components of the application before the customer experiences a problem.

Embedding Analytics in Your Application

Before you start embedding analytics in your application, it’s important to choose the right database foundation, as this will support current needs and future goals. Below are the top capabilities that should be standard features in analytics databases:

1. Handles massive data volumes: The database must scale easily and efficiently with minimal cost using massive parallel processing. Today’s scale might be terabytes, tomorrow it might be petabytes and so on.

2. Enables speedy data analytics: Speed is a key factor in a positive user experience. Users want instant insights because they don’t like waiting for results. The database should be capable of optimizing load and query performance so that analytics is both timely and relevant.

3. Embeds machine learning: Machine learning algorithms are critical for predictive analytics. A platform that can embed ML and iteratively learn from new data can make a big difference and add a whole new level of intelligence.

4. Features native integrations: The more time developers spend figuring out tools and integrations, the less time is spent innovating. The platform must ideally have out-of-the-box integrations with popular applications, data sources (on-premises, cloud, multi-cloud or hybrid), ETL, and visualization tools (e.g., Microstrategy, Looker, Power BI, Qlik, Tableau). It should be flexible enough to support user-defined functions for things that don’t have native integration.

5. Encourages self-service: The database should make it easy for data scientists to leverage tools and programming languages they already know such as R, Java, C/C++, Python and SQL. Data scientists will also appreciate a platform that has built-in functions for a wide range of analytics (such as geospatial, time-series, event-series, real-time, machine learning, text analytics, pattern matching and regression) and delivers the tools to spot trends, uncover anomalies and anticipate the unexpected.

6. Supports highly advanced analytics: Many platforms can analyze single tables or do a simple look-up but few have the power to analyze dozens of tables, hundreds of data types, dimensions and attributes from numerous sources over several years of history.

7. Unified: The database should be able to streamline everything right from data ingestion to exploration to production until delivery. Users should be able to analyze data in its place to always have a single source of truth.

It’s not hard to visualize that the most disruptive companies of the future will be the ones that harness data as a core asset and embed analytics in their products and services. Embedding analytics requires the right database foundation; to get to the right foundation, IT teams need to cut through the noise and study the fine print. With the proper database, the future is yours to create.

January 4, 2023
Best of 2022: Agile/Scrum is a Failure – Here’s Why

As we close out 2022, we at DevOps.com wanted to highlight the most popular articles of the year. Following is the latest in our series of the Best of 2022.

Welcome to The Long View—where we peruse the news and strip it to the essentials. Let’s work out what really matters.

This week, something a bit different: I’m devoting my entire column to Agile and Scrum. They’re increasingly getting a bad reputation, being associated with the worst aspects of toxic workplace culture: Sexism, racism, burnout, micromanagement. (more…)

January 3, 2023