Welcome to The Long View—where we peruse the news of the week and strip it to the essentials. Let’s work out what really matters.
This week: Facebook engineers admit they’ve no idea what Meta stores (or where), California requires job ads quote salaries, and RISC-V will power future NASA spacecraft. (more…)
GraphQL, a query language for APIs and a runtime for fulfilling those queries with your existing data, is rapidly growing in popularity. According to the latest State of JavaScript survey report, 44.3% of respondents said they had used GraphQL APIs in 2020 compared to only 5% in 2016. Additionally, many Fortune 500 companies now use GraphQL APIs for their backends in production, while top tech companies like Facebook and GitHub use the API technology to improve their product offerings and boost system performance.
With the ever-changing technology landscape, you might have doubts about adopting GraphQL API for your backend and be unsure how it can improve your current data needs. However, regardless of what language or architecture you are using, GraphQL APIs can help simplify data access, supercharge the product development process, improve application performance and handle complex back-end data scenarios.
One advantage of GraphQL APIs is that they provide a single unified endpoint for your backend to meet both client and general API use cases. A single GraphQL API can handle all the client’s requests and forward them to the correct backend service.
As companies and their products scale, REST endpoints can become large and incredibly hard to manage. One of the main problems in REST is the N+1 problem, where the client calls the server N+1 times to get N number of resources, mainly because a single REST resource does not provide enough information for the client to operate with.
Brennan Saeta, an infrastructure engineer at Coursera, cited centralizing REST APIs for simple consumption by the client as one of the reasons why GraphQL API appealed to them.
GraphQL API hides the complexity of the underlying backend system by aggregating the required data from different sources and providing a simple interface that clients can communicate with through a single resource.
GraphQL API improves application performance by reducing JSON payload size and network overhead from multiple API calls. By allowing the client to specify its data requirements, GraphQL APIs eliminate the need for unnecessary generic data being passed from the back end and minimize the number of roundtrips required to retrieve back-end data.
As companies scale and iterate their products, the data required by different clients and pages changes. In REST API design, when new requirements arise, developers must decide whether to create a new endpoint and have clients make a new request for that data or add more data to an existing endpoint. The first option leads to more server roundtrips, and the second option increases payload size, which can hinder app performance.
With more applications being accessed over flaky mobile networks and app speed being an important metric for retaining users, minimizing server roundtrips and data transferred over the wire is paramount. Companies like Netflix and Pulley have reported faster page load times and overall app performance after switching to GraphQL APIs.
With GraphQL APIs, clients receive only the data they specify from the backend server in a single trip, saving network bandwidth and resources.
GraphQL API improves system reliability by ensuring that client queries are valid and in sync with the GraphQL schema specification before query execution. GraphQL uses a type system that describes what type of data can be queried from the backend server and in what form it exists. A strongly-typed schema offers enhanced security over REST APIs by preventing breaches like SQL injection and unintentional data leaks, resulting in a more secure and reliable application.
Type-safety ensures that any breaking changes by the backend server would be caught by clients consuming the data, resulting in less downtime. With GraphQL APIs, you can modify and deprecate API fields while still maintaining backward compatibility. It is easier to know which fields in the API are being used by what clients and which ones are not. With GraphQL, API developers can confidently make changes to the existing APIs without breaking the existing functionality.
Swiggy, India’s largest food delivery platform, discussed how adopting GraphQL API for their back end enabled them to make changes to their existing APIs without breaking the consumer application.
GraphQL ensures API consistency, reliability and security, resulting in better system performance.
Adopting GraphQL APIs improves the developer experience for both frontend and backend developers. Backend developers want to build APIs that are intuitive and easy to explore and navigate. GraphQL helps with that by reducing the time spent documenting APIs.
Through a concept known as introspection, GraphQL APIs allow clients to query the schema and discover the resources available on it. With excellent tools and libraries like the GraphiQL IDE, the interactivity for GraphQL API endpoints is much more intuitive for the developer.
API consumers want APIs that are flexible, easy to integrate and support a variety of client use cases (web, iOS and Android). Because of REST’s inflexibility, it is challenging to create a single endpoint that can meet all of the client’s data requirements. Often the same client has to call multiple APIs to get the data it needs from the backend. Additionally, with REST APIs, frontend developers have to rely on the backend for API specifications before building product features, leading to slower development time.
Peter Downs, director of engineering at fintech startup Pipe, said choosing GraphQL APIs early on cut their development time by 90%. With GraphQL APIs, frontend and UI developers can quickly and easily iterate on features by simply specifying the data requirements in the GraphQL API, thereby reducing the dependence on backend developers to make changes.
The best way to learn GraphQL is by getting your hands dirty and building applications with it. Depending on your language of choice, there are GraphQL libraries in many different languages to help you get started.
As with any new technology, adopting GraphQL APIs for your back end and building high-performant GraphQL backends requires experience, which is not peculiar to GraphQL alone.
If your application relies heavily on interconnected data, you aim to develop an API to be used across several clients or are looking for a high-performant API solution for your backend that minimizes bandwidth usage, then GraphQL API is a perfect data layer for your backend.
Enterprise networks are vast, intricate systems with multiple methods for storing configuration and state information. The configurations of those networks are also constantly changing as new applications, services or other updates are added. When you consider the complexity and dynamic nature of an enterprise network, it may be alarming to learn how much data–including most network configurations–is still updated manually, often in the middle of the night. To make that process even less efficient, the people doing the updating are required to manually log those updates on spreadsheets or into a custom-built database.
Relying on manual processes to maintain and manage the state of an enterprise network not only leaves the door open for errors but can also lead to the network spreadsheets or databases being out of sync with the actual current state of the network. Is the network correct? Or is the spreadsheet correct? Should we make the network look like the spreadsheet? Or should we update the spreadsheet to reflect the network? What is the actual source of truth?
The good news is that the reality of modern networking has changed the paradigm. As programmable networks are deployed more broadly, manual CLI interfaces are no longer necessary. Reading and writing network configuration and reading network state can be accomplished quickly and easily with machines and automation.
More enterprises are embracing network automation because they recognize the efficiencies it can deliver when managing complex networks and the benefits those efficiencies can add to the bottom line. But if you’re going to automate your infrastructure, your automation solution will need to gather authoritative information about it from somewhere. This source of truth holds the knowledge needed to enable automation to make changes to the network.
Many enterprise networking teams feel they need to rely on a single source of truth to know the true state of the network. They believe that building a single system/server/database that can synchronize with the configuration and state information of all devices across the whole network is a viable solution. But such a solution doesn’t currently exist. Given the multifarious nature of today’s networks, relying on a single source is both impractical and potentially expensive.
Rather than asking “What is the single source of truth?” perhaps the better question is “What will provide better data: Creating a single source of truth that tries to synchronize to the entire network? Or accessing distributed sources of truth needed to accomplish the automation in real-time?”
Instead of relying on a single source of truth, enterprises should focus on accessing multiple sources of truth to conduct proper and meaningful automation.
If you’re running a service provider network, for instance, your business is your network. If you’ve spent millions, or even a billion dollars on your network assets, it’s worth your time to have a system in place that accurately manages and maintains those assets. A single source of truth won’t provide that.
Also, as the network scales in size and complexity, the time it takes to synchronize a source of truth to the network also grows, limiting how often you can synchronize and leading to a greater time delta between synchronization states.
Finally, the process of trying to create a single source also doesn’t address the real issue, which is that enterprises are still relying on manual processes that will almost always result in the source of truth databases and the actual network status being out of sync at some point. And that can have serious negative effects on a business as well.
The case for accessing multiple sources of truth is a relatively simple one: it allows for greater flexibility and enables enterprises to more easily leverage data to benefit their business.
With the advent of programmable networks–machines talking to machines–the most current automation solutions can access multiple distributed sources of truth–different APIs and databases that are responsible for being the source of truth for different parts of the network – and federate and sync data from these systems in real-time, providing a truer window into the status of the network and more accurate, actionable data that delivers business value back to the enterprise.
So why are some enterprises still hesitant to commit to a full network automation project? There are various reasons for this, but clearly, there’s concern that not having a single source of truth for network data – or data that may not be 100% accurate – will compromise the automation process. It’s a classic “fear, uncertainty and doubt” thought process: “What if the data is only 70% accurate? If we try to automate with bad data, we’re going to break the network 30% of the time.”
Some network teams may believe they can’t automate anything until they can combine their five or six sources of truth into one huge database. At the same time, leadership may be pushing those network teams to move forward with automation to get more value from the network. By choosing to wait until they finally have a “pristine” database, they could lose months or even years of time, to the benefit of no one.
The solution is to take an incremental approach. Look at all the various network assets–programmable, cloud, legacy, etc.–and start the automation process in the areas where good data exists and where a source of truth doesn’t need to be built. In parallel, you can be cleaning up the other data and add automation when that data is ready.
Being able to utilize multiple sources of truth is better than worrying about having a “perfect” source of truth and holding back on an automation initiative. In the long run, it will be beneficial to both the network team and the business.
Just as important as committing to an automation solution that can integrate with multiple sources of truth to make decisions, is deploying the right automation solution. The most effective automation solution will embrace:
● API-first–The only way to integrate multiple sources of truth into a unified one is through an API-first approach, allowing systems to talk to each other.
● Data federation and transformation–Having unified sources of truth is only impactful when teams can fully integrate the data together while ensuring it speaks the same language.
● Unified view–An abstracted and federated view of the data, processes and logic from integrated systems enables a single pane of glass across an entire network, as well as management tools for simplifying automation across the organization.
With today’s modern, programmable networks–and the absolute necessity for enterprises to have access to accurate, real-time network data–relying on a single source of truth is based on a flawed assumption that we can always have a synchronized database, and therefore is not a viable strategy.
By adopting network automation, organizations can adopt a distributed source of truth solution by enabling the multiple systems of record, and their collective data, to act as the source of truth, mitigating data quality issues and manual errors.
As an enterprise implements its automation strategy, having multiple sources of truth within the network will result in more accurate, timely and actionable data for automation to succeed, which will increase efficiency and drive the business forward.
Matt Caster, chief solution architect for Neo4j, explains how open source Apache Hop projects advances integration of DevOps and DataOps. The video is below followed by a transcript of the conversation.
Mike Vizard: Hey, guys, thanks for the throw. We’re here with Matt Casters, who is Chief Solution Architect at Neo4j and we’re talking about a new open source project called Apache Hop, that helps you move data round more easily. Matt, welcome to the show.
Matt Casters: Thank you very much for having me, Michael.
Vizard: So walk us through what exactly is Apache Hop, what’s the problem we’re trying to solve, and it seems like we’re trying to move more data than ever, and when I was a kid people told me moving data was not necessarily a great idea, but we’re doing it anyway. So what’s the challenge and what are we trying to solve?
Casters: Oh yeah, yeah. So that’s a great observation. I remember when everything was going to be the new mainframe, do everything in one box, right? And the opposite happened; we saw more and more services on prem, in the cloud, and all mixed up. See, I think, you know, more specialized services, in the cloud, on prem, graph databases, you know, advent of containerizations, virtual machines, and containers. So it’s become like a really complicated ecosystem out there and I think a lot of organizations are struggling to find expertise to service and cater to all those needs.
I think what we’re trying to do is make the life of the developer easy by wrapping up user interfaces and easy-to-use tools around all these technologies so that you get like UMLI diagrams that you can work with and that can be easily understood by not only the experts but medium-level educated people on subjects, right? For example, if you’re working with a Kafka queue you might need to know how it works in general, but you don’t need to dive into the Java or Python APIs to get the best out of it. I mean that’s kind of it.
So lower the maintenance costs. And this comes down to the DevOps side of things. And while we try to make the life of the developer easy we position ourselves exclusively on the side of the organization. We’re not made to just make life fun for the developer, but we really try to make the life fun for the developer, but we really try to make life for the company or the organization that uses the software better.
So with that in mind you come to the conclusion that, okay, we need to also focus on configuration management, life cycle management, version control, unit testing, security investment, and that companies put into often many years of work of data orchestration, data integration, ETL work, right? So that is what I think Hop has been trying to focus on the last couple years by making sure that all these technologies are standard in the tool and easy to use.
Vizard: Do you think that we’re reducing the need for specialists, ’cause back in the day you had to have somebody who knew how to program with ETL and that capability and now with this we’ve got more of a visual tool, and maybe some people still prefer CLIs and APIs, but this is designed for developers to just easily handle this task themselves without any outside intervention, right?
Casters: Yeah, that’s the lore. We fully acknowledge that it’s like a 95/5-percent kind of deal, right; you’re never going to do the last 5-percent without some scripting or coding. But solving the 95-percent is a big deal, it means that all of a sudden you open up possibilities to people that we would never have expected of using Apache Hop. Right? I’ve seen salespeople use it to extract the sales numbers from Salesforce. I would’ve never expected that in the past. It’s like, “Oh, they can figure it out.” “Oh, I can just enter the details: username, password.” “Oh, I can read this” and before you know it people start to use that.
Vizard: Citizen developers are here with us today. One of the things about Hop is that it’s lightweight and I don’t think people appreciate just how much heavy lifting went into ETL historically. So why does it matter that this thing is lightweight and metadata-driven and what’s the impact?
Casters: Well, at some point we saw the advent of LANDesk, you know, microservices, but also in its previous life, Kettle, before that was used was for plotting trains in more like an IOT settings. Call these Edge devices, maybe Raspberry PIs or very tiny devices. And there’s a real use case out there for a device that doesn’t have to do it or anything; it’s just trimmed down to the bare essentials for what it needs to do. Maybe it would be like measure temperature and air humidity and pressure and then send it off every 30 seconds or something, over some – using some Rescall.
And then this whole Hop library trims down to something like 30-40 megabytes and starts up fast and uses next to no memory. That is not in contrast to saying like, oh, maybe I run it on an AWS LAN in a small services. And it also gives us the ability to create new packets, right? You can only remove them, remove functionality, but you can add functionality quite easily as well. And that is really important. We’ve also already seen people implement machine learning or libraries or Python iteration and plug-ins for Hop. So this will be our next goal, is to create a marketplace around that plug-in kernel kind of architecture.
Vizard: Do you think that the functions of DevOps and DataOps are starting to converge more? Is this becoming more of a team sport or is DataOps just going to be folded into DevOps? What’s the relationship?
Casters: Well, what I’ve seen personally is now that we have the ability to lock everything that a workflow or pipeline does into a graph, a Neo4j graph, it makes it easy to see where the arrow was, right? Can use graph algorithms to figure out not just where an arrow was, but figure out what the execution pat was in milliseconds. So the convergence of, “Oh, the developer has an easy time figuring out what went wrong, but also on Monday morning when you come in the office and see that the Saturday night run failed for some reason, or the monthly load of the data warehouse or whatever people are doing, they don’t have to troll through millions of lines of loading texts for a large workflow, but just see straightaway what went wrong, where it went wrong, saves a lot of time.
So is that DevOps? For sure. For sure DevOps. It’s also ease of development. There’s really like an overlap and we need to do better in providing a lot of these tools. They’re still too primitive in a lot of our software, As if locking text is enough, right? We can do so much more. [Laughs] We’re executing stuff; we know where it went wrong. Why guess? Why go through the effort? [Laughs]
Vizard: There’s a lot of focus these days on the volume of data that people are trying to work with, but it seems to me it’s also the speed in which that data is starting to emerge more, and people are trying to work in near-real time, and the data is going to be updated. We’re moving beyond batch. So are we really prepared for not just the volume of data, but the speed at which that data is starting to emerge. And for that matter, there’s a lot more types of that data, right?
Casters: Yeah. So being from the Hop from the beginning opted to choose Beam to work as an abstraction layer so that we could execute on Spark, Flank, and Dataflow for the really fast use cases, but also, yeah, like real-time streaming, and just like playing, just streaming, Spark streaming and cloud Dataflow are really cool. And this comes along with the advent of all the streaming back-answer messaging services: Kafka, Pulsar, jewel clouds pops up. I think repertoire, right? I probably forgot like Azure. [Laughs]
But yeah, so these services have become really popular in making it possible to create cool Hop and Spoke architectures, where you can send stuff in all directions. Yeah, so it falls under the sort of like streaming category, and you can do all sorts of cool load-it-and-see exercises there. And opening that up is quite easy for us in Hop; all the pipelines are streaming, whether it’s in batch or not. So yeah.
Vizard: So what’s your best advice to folks as they kind of look at these DevOps/DataOps challenges? What do you see as the best practice that people do the right thing? And for that matter, what kind of mistakes are people making that maybe they should avoid?
Casters: So I think when it comes to these best practices, and then we’re talking about the classical list, put everything in version control. You know, keep your passwords and usernames and everything separate. Don’t hardcode anything. Like these are not specific to ETL data orchestration, but they’re genera programming best practices – development best practices. So even if it is like graphical programming, it still applies. Same with unit testing and integration testing.
What we’ve seen in the past is that if you want to do, for example, unit testing, there’s a lot of plumbing that you need to put in place. So specialized Docker containers and so on and so on and so on. What we’ve tried to do is always look at the return investment of metadata. So what is the return investment for setting up like a lot of infrastructure and a lot of manual plumbing? It’s very low, right? So you see some benefit from a unit test like that, but it’s not great. So by building the integration testing and unit testing into Hop, into GUI, making it very easy to create, we’re saying like, “Okay, now you really don’t have any reason not to do it anymore. Right? [Laughs] Well, apart from laziness or not knowing what the value is. But at least those best practices are not different from any other development job that you might have.
Vizard: All right, so we finally reached a point where we’re going to automate the orchestration of the data, just like the rest of any other DevOps process for that matter. So it’s onwards and upwards. Matt, thanks for being on the show.
Casters: Hey, you’re welcome. It’s good to talk to you, Michael.
Vizard: All right. Back to you guys in the studio.
Welcome to The Long View—where we peruse the news of the week and strip it to the essentials. Let’s work out what really matters.
This week: I’m devoting my entire column to new ways of working. Whether it’s remote work, enforced hybrid working or a four-day working week, let’s triage the trends and futz with the future.
Too many managers are stuck in their old ways. Despite the success of pandemic-era WFH, they yearn for how it used to be. But that ship has sailed.
Mark Gurman: Apple Sets Return-to-Office Deadline
“Pushing employees back into the office”
The company … set a Sept. 5 deadline [and] will require employees to work from the office on Tuesdays, Thursdays and a regular third day that will be determined by individual teams. [It] has been working toward getting its employees back in the office since at least June 2021, when it first announced the three-day policy.
…
The iPhone maker has been one of the most stringent technology companies when it comes to pushing employees back into the office, irking some staffers. … An Apple spokesman declined to comment.
It “Shows the Company Hasn’t Learned Anything,” cries Jason Aten:
“Not the same thing as actual collaboration”
Right now, every company is trying to figure out how to get its employees back to the office. Despite the fact that the world did not, in fact, come to a stop over the past two years. … Many companies—especially Apple—had their best two years ever when most of their employees were working from home.
…
Apple SVP of software Craig Federighi … told employees in an email … he “can’t wait to experience the special energy of having all of us back in the office together again!” … I imagine a lot of the people who work in the software organization are wondering whether that “special energy” actually makes them more productive—or if it’s just a thing managers feel as they watch employees.
…
If we’ve learned nothing else from the pandemic … coming to the office doesn’t make them more productive. It might make managers feel better … but that’s not the same thing as actual collaboration.
But WFH doesn’t suit everyone. Here’s Aunty’s Kate Morgan: Why workers just won’t stop quitting
Anthony C Klotz, a professor of management at University College London’s School of Management who coined the term “Great Resignation” [says] in addition to the original push factors … the reasons people are resigning have diversified. … Some workers are now swapping jobs that require them to be present in the workplace for remote positions. Other workers … are leaving remote jobs for ones that have a larger in-person component.
WWSJD? quonset knows:
Regardless of the jerk he was, Steve Jobs had grand visions which he brought to fruition. … One thing he stressed was the personal connection and serendipitous encounters.
…
It’s why, before he died, he was arms deep in the design of Apple’s headquarters. He wanted people to bump into each other, he wanted interaction between different groups. It’s why it’s designed the way it is. People are funneled to certain areas and have no choice but to interact with others.
Is there an unintended brand consequence? u/Latinhypercube123 demonstrates there might be:
Hahahaha. Apple is such a dead end company now. They’ve literally become what they ridiculed IBM of being: Corporate monolith of sameness.
But is there a better way? You should read Evelyn Yu: Bank Says Four-Day Week Is The ‘Future of Working Life’
“Flexible work is here to stay”
Six months after switching to a four-day working week, the UK’s Atom Bank said it’s seen benefits from talent retention to improved productivity. … Days lost to sickness fell over the period and customer service ratings also improved.
…
Even as the pandemic recedes and some bosses push for a return to the office, there’s increasing evidence flexible work is here to stay. Office vacancies in central London are at the highest level in more than 15 years.
Not just in the UK—it’s in Canada too. Here’s Sean Davidson:
“Huge impact on employee retention”
Ross Wainwright, CEO of Alida, said the software company implemented four-day work weeks for its 500 employees at the start of June and the two-month trial wraps up this week: … “The early results are fantastic. … The employees are clearly happier. … This is about getting better balance for our employees and helping to invest in their mental health.”
…
He said the trial is also having a “huge impact” on employee retention. … The most “unanticipated” result of the trial was how empowered his employees told him they felt, knowing the company trusted them to get the job done with a reduced schedule: “We empowered the employees to get the work done within their schedule.”
Also in Toronto is u/SuperChloe:
We had similar results at my company when we initially trialed 4-days (also not longer days, still 9-5) — overall increased output/productivity and employee satisfaction. We’ve moved to permanently having a 4 day work week.
Meanwhile, @AnnaLovesVoting is just fine with a hybrid schedule:
I don’t need 100% WFH, I’m willing to go to an office 1 maybe 2 days a year.
You have been reading The Long View by Richi Jennings. You can contact him at @RiCHi or tlv@richi.uk.
Image: Carles Rabada (via Unsplash; leveled and cropped)
The last few decades have seen multiple major shifts in how applications are built and deployed, and now we’re seeing the start of a new shift that opens up enormous opportunities for all kinds of developers. Instead of bringing data to applications, developers are bringing their applications to the data.
Let’s back up. In the 1990s, as data grew too large for local machines, applications moved with the data to data centers—a model more like the mainframes that preceded them. Then, in the 2010s, those same applications migrated from data centers to the cloud, bringing the data with them again. Now we’re seeing a third major shift, where customers take back control of their data and centralize it in the cloud. And developers are picking up on this trend, building their applications to not control their own data, but instead to connect to the customers’ data, letting them have and control a single copy.
In a traditional SaaS application, the SaaS vendor stores and processes data in their own platform, even if that platform is running on one of the major public clouds. A traditional CRM (customer relationship management) tool, for example, will use its own data platform to store all of the data that all of its customers are managing with the tool. When a customer wants access to their data, often to join with other data sets, they either have to export the data (via APIs or CSVs) or upload the other data set into the tool and use whatever analytics capabilities the tool has built. This is painful for customers and also painful for the developer who has to support use cases that are further afield from traditional relationship management.
In this new world, the customer runs and the applications all connect to the same data platform, storing and processing their data there. This means that joining two data sets is a single query away, with no movement of the data required. It also means that the customer owns and controls the data, making security and governance hurdles much easier for developers to overcome. And, because the customer pays for the data platform, the developer doesn’t have to mark up the costs and is able to command significantly higher margins.
While it’s early, this shift is already underway. Fast-growing startups like Supergrain, Anecdotes.ai and Rudderstack, to name a few examples, are taking full advantage of this shift and leading investors have picked up on it as well.
There are two key trends that are leading to the emergence of these connected applications: The wide adoption of cloud data platforms and the prioritization of control over data by customers.
In the last ten years, a new class of highly scalable and cost-efficient data platforms has emerged, allowing enterprises of all sizes to adopt technology that was previously only available to the most sophisticated vendors for use inside their own products. Previously, SaaS vendors had to roll their own complex data platforms that were very specific to their use case and charge high prices to cover the cost of running those platforms. Today, we’re not only seeing SaaS vendors migrate to cloud data platforms but also enterprises adopt the platforms themselves. This means that, for the first time, every enterprise has access to the best data platforms available.
We’re also seeing a huge change in how enterprises think about and secure their data. Partly because of regulations, and partly because of publicity around data breaches, large and small companies are paying significantly more attention to how secure, how controlled, and how well governed their data is. When your data is spread across dozens or more different SaaS applications, it’s nearly impossible to guarantee its security and access controls. Centralizing that data into a single platform, and connecting your applications to it, allows for a single point of control and security for every piece of data.
It’s clear that connected applications are a big win for customers: A centralized source of truth with strong security and control. But this model has enormous benefits for application vendors as well.
First, it changes the go-to-market dynamics by letting you tap into work that customers have already done to centralize their data. According to George Xing, co-founder and CEO of customer engagement platform Supergrain, CEPs built with traditional SaaS models suffer from several drawbacks including long implementation times, poor reporting, and higher costs. The connected app model makes Supergrain more competitive, he said.
“Instead of moving your customer data to a third-party data platform, we bring the marketing software to your data warehouse, unifying source-of-truth customer data and marketing product experiences into one cohesive system,” Xing said.
It also accelerates time to market. Developers no longer need to build and maintain a data platform for their customers and can focus instead on their core product differentiators. They also get to leave responsibility for data security and compliance to the customer and their data platform.
With the customer managing and paying for storage and compute directly, vendors can significantly increase their margins, charging for the value they provide, rather than trying to mark up the compute they’re consuming.
The incredible growth in the amount and types of data available to companies is only accelerating this shift in how applications store and operate on their customers’ data. We’re going to see more and more SaaS vendors support connected application deployment models, but it won’t replace vendor-managed models. Instead, there will be opportunities for both models to flourish depending on the customer and specific application. We expect that many vendors will offer both models, just as we’ll see others go even further, bringing their entire application—including its UI, logic, authentication, etc.—fully into the customers’ data platform.
The opportunity for companies that take advantage of these new models is enormous, and we’re in the very early days. But the applications that take full advantage of this shift will be the winners of the next generation of data apps.
Welcome to The Long View—where we peruse the news of the week and strip it to the essentials. Let’s work out what really matters.
This week: Agile is bad, but “Wagile” is worse.
Plus: This prod is even worse than yours. (more…)
Sustainable development is a blueprint for societal growth that is applied around the world to balance the needs of the present while optimizing for the future. Sustainability consists of three pillars — economic, environmental and social — that must work together to meet this goal. The complexities of digital transformation and application modernization can benefit from applying a sustainability lens to this challenge that not only includes the technical benefits of cloud computing but, more broadly, the three P’s of sustainability: Profit, people and planet. Application modernization will benefit the business in measurable ways, but can also affect unexpected positive cultural and green initiative benefits.
Sustainability initiatives have always had to take a holistic approach that combines human, ecological and economic health and vitality. Sustainable application modernization is no different in that business leaders are dealing with finite resources (budget, staff and time) that must be applied now to align with longer-term priorities and their associated consequences. The main goal of application modernization is to transform an enterprise’s monolithic applications into modern architectures that can more fully take advantage of cloud-native services such as containers, microservices, serverless and a range of DevOps methodologies. These technical capabilities lead to a number of high-value cloud benefits including scalability, elasticity and engineering agility. Too many modernization strategies and business plans stop there and this is a mistake. By undervaluing the sustainability benefits that include economic, environmental and people upsides, technology business leaders are not accounting for some of the most important benefits of modernization.
Breaking down the modernization problem along the three pillars of sustainability helps define sustainable application modernization:
• Economic sustainability: The business benefits of modernization must account for releasing more features and fixes more frequently, less downtime, lower technical debt and higher rates of innovation.
• Environmental sustainability: A modern architecture is more resource efficient, scales at a much finer level than a monolith, and is able to take advantage of cloud data centers that offer the “greater efficiency of aggregated compute resources.” These cloud-native architectures enable more intelligent application orchestration and resource allocation usage based on demand, need, cost, seasonality and performance.
• Social sustainability: Besides the ongoing challenge of recruiting and retaining talent for maintaining a monolith, the resulting development team structure is a direct reflection of the architecture itself and this is most often monolithic, too. A more domain-centric approach using modern technologies creates leaner, more focused team topologies that can also operate effectively in remote work structures.
The broader benefits and impacts of effective application modernization align directly to metrics associated with elite performers in DevOps research reports that track four key performance categories: Frequency of application deployments, application feature lead time, change failure rates and time to recovery. This long-standing and annual research report from DevOps research and assessment (DORA) metrics also now reaches beyond the business and technical dimensions into the performance, culture, people, burnout and team impacts of these modern best practices.
Most application modernization projects never fully get off the ground because technology leaders look at modernization as just an IT problem. The business cases for these projects focus on technical benefits or simple cloud migration and lift-and-shift infrastructure cost improvements. A sustainable application development methodology opens up this calculus to the three dimensions of sustainability: Economic, environmental and social.
Much as everyone from developing countries to the richest nations in the world view sustainability initiatives along these lines—balancing economic growth, environmental health and the vitality of their citizenship—application modernization projects can be a catalyst to major improvements that drive profit, maximize efficiency while improving company culture, retention and team growth.
The enthusiasm for application modernization is high, but organizational precendent and a history of failed efforts stop most projects before they get started. These three steps can set you on a sustainable path going forward.
Sustainable application modernization methodologies can change the course of application development inside an organization and provide a catalyst toward a thriving future that pushes the technology forward and can maximize the value to the business, environment and company culture.
Mobile apps have become the primary point of innovation for many companies, and skilled mobile developers are in high demand. The business asks developers to innovate fast using the latest smartphone features and capabilities to drive high download rates, wow their users, capture customers and grow their business. Dev teams facing constant pressure to move faster may push security to the back burner, focusing on required features and delivery dates the business demands. However, many dev teams have figured out how to innovate faster with security built in by ensuring that devs understand key security requirements and coding best practices. Working with hundreds of development teams on securing thousands of mobile apps, we found four common areas of security failure that can be addressed easily.
Mobile app devs may instinctually use Hypertext Transfer Protocol (HTTP) for network communications. However, HTTP puts private user information out in the open for threat actors to intercept. Developers can fix this common mistake by using HTTPS instead, which encrypts data sent to and from servers via industry-standard SSL. Android developers can use the available NetworkSecurityConfig file to set up a predetermined configuration for all network connections made within the app or manually incorporate HTTPS. iOS developers can use App Transport Security (ATS) as the default feature that enforces secure communications in iOS apps and prevents any insecure connections between the mobile app and the server from being completed.
Certificates add an additional layer of security to HTTPS connections by enforcing additional validations when performing a connection. Certificates include the certificate authority (CA) that signed it and the list of hostnames known or accepted by the application. Apps that validate these components when performing connections with the server significantly reduce the risk of a man-in-the-middle (MITM) attack that can steal credentials and sensitive data.
To verify a certificate has been issued by a valid CA, Android developers can reference the preconfigured list of CAs included on devices running the mobile operating system. Android developers can implement native classes such as HostnameVerifier to achieve proper hostname verification within their app.
iOS developers can leverage ATS that provides built-in validation, or use methods in the NSURLSession class to manually code these instead.
Attackers often use hardcoded info within the source code of a mobile app to take advantage of users. For instance, attackers can use credentials stored within app files to gain access to a user account. Attackers can also find hardcoded API keys or URLs to gather private data or take over an app entirely. Mobile app developers can prevent these areas from being compromised by not hardcoding keys, passwords and URLs into the source code. Encrypting transport and decrypting data on the backend also adds an extra layer of security.
Outdated cryptography algorithms like SHA-2, RC4 and DES allow attackers to easily break a seemingly secure mobile app. Using older algorithms may also make a mobile app non-compliant with industry regulations leaving an organization at risk of fines or legal jeopardy. Devs can avoid this problem for Android and iOS mobile apps by using the latest cryptography algorithms available, selecting those that are suitable for a specific app scenario. When signing your binary before publishing it to the store, use keys with a length of at least 2048 bits (preferably 4096 bits), and on Android use SecureRandom or SecRandomCopyBytes on iOS when generating random values for cryptographic implementations. Android devs can leverage the Keystore class and iOS devs can leverage Keychain services to store highly sensitive data. Devs should avoid insecure modes of operation, improperly generated cryptographic keys and initialization vectors (IVs) to guarantee that the information that is encrypted cannot be decrypted by a malicious actor.
Today’s mobile app users need confidence that mobile apps are built with security in mind. While building innovative mobile apps, developers have a responsibility to learn a practical set of secure coding best practices to protect their users (and their businesses).
Confidence levels among IT workers have plummeted as tech teams’ jobs have become more complex. The reasons behind IT worker angst are unprecedented, signaling a shift paralleling other post-pandemic industries. Running applications across cloud and on-premises infrastructure, hybrid office environments with distributed workforces and the accelerated adoption of new and automated technologies have all contributed to increased IT complexity.
In our recently-released SolarWinds IT Trends Report 2022—Getting IT Right: Managing Hybrid IT Complexity, we examined the current state of IT along with areas of opportunity for technology professionals who must manage this increased complexity as hybrid IT accelerates. “The continued shift to hybrid IT drives increased levels of IT management complexity, but tech pros feel a lack of confidence in how to best manage it,” according to the report.
Solutions are necessary, as the research shows only 8% of tech pros surveyed for the IT trends report feel “extremely confident” in their ability to manage increasingly complex networks. More than half of the tech pros interviewed said their organization manages hybrid IT complexity by training staff and adopting IT monitoring/management tools. However, budgeting, time constraints and hybrid work environments exacerbate this challenge. These responses point to a need for intervention and assessment, as IT professionals’ work is more business-critical now than ever. Strong IT infrastructure is necessary for fundamental processes across all industries and business environments.
According to a report by market intelligence firm IDC, as more mission-critical workloads move to connected cloud architectures spanning public, private, hybrid and multi-cloud environments, enterprises recognize the need to invest in the tools which will help them ensure consistent policies and performance, despite budget and observability implementation issues. The SolarWinds IT Trends report revealed nearly half of respondents lack visibility into the majority of their organization’s apps and infrastructure.
Indeed, visibility is increasingly becoming the clear answer for tech pros charged with managing greater levels of complexity in these diverse and distributed environments. To develop their teams and business for long-term success, organizations must set aside time and resources to upskill and train tech pros to help them properly implement observability strategies and manage hybrid IT realities more effectively.
But some aspects of the remote revolution, including training IT workers for increasingly complex network management, brainstorming sessions, providing sensitive feedback and onboarding new employees, lose their effectiveness when done off-site. For those who’ve made the switch to teleworking, most have found more work-life balance but less connection with co-workers or consistent support from leadership, according to a Pew Research Center study.
Software company Culdesac announced it’s moving workers online and closing its San Francisco offices. Over 14,000 Google workers worldwide moved to a new office or went fully remote during the pandemic. Other companies, like Microsoft, have said employees who want to work more than 50% of the time remotely must have their schedules approved by their managers.
Things are quickly shifting as we enter the era of post-pandemic IT infrastructure existence.
Solutions are growing in availability. The Internet Engineering Task Force (IETF), an international community of network designers, operators, vendors and researchers, is working on standards for automated network management, which aim to improve and make the management of networks more efficient as they continue to increase in size and complexity.
While organizations also lack insight into their network’s impact on ROI due to increased hybrid IT complexity, looking forward, they must re-examine their investments from the past few years. Part of this requires companies to have visibility into their IT environments to understand what’s working and not working.
Technological change could bring about economic growth and opportunity, but only for those willing to adapt. IT infrastructure is so much deeper than most businesses understand, and as the existing data confirms (and most experts in the field agree), investing in IT infrastructure is necessary. A business’ network—and its speed and security—can be the difference between satisfied or frustrated employees and customers.
As hybrid IT acceleration has increased network complexity and lowered tech pros’ network management confidence, these post-pandemic problems are a chance for organizations to evolve. “The healthy thing to do is to look ahead, accept the reality, and start getting ready for the transition,” said Chirag Shah, an associate professor at the University of Washington Information School. “So instead of feeling the anxiety and being defensive, think of this as an opportunity for advancement.”