Welcome to The Long View—where we peruse the news of the week and strip it to the essentials. Let’s work out what really matters.
This week: Herbert Diess is out at VW because software is hard (yo), fiber optic cable is hard to find, and the guy who said LaMDA was sentient is dumped by Google. (more…)
The intersection of CI/CD and network automation has grown substantially over the past few years. Network teams have seemingly reached the point in their automation and orchestration journey where it has become critical to implement a strategy to manage the testing, version control and deployment of both network changes and automation assets.
While CI/CD has grown in popularity, there are—at least on the surface—many differences in the types of pipeline implementations to which network teams want to apply the technology: Different tooling, different scenarios and different problems being solved. Despite the seeming disparities, there are some key themes that are consistent throughout most pipeline implementations. For instance:
Network teams either want to develop pipelines to better manage specific changes to the network or they want to use pipelines to better manage the testing and deployment of automation and orchestration assets like workflows or scripts. Where initially those teams may have turned to network infrastructure-as-code (IaC), a significant (and growing) number are investing in automation asset pipelines.
The trust and resources being dedicated to these pipelines is just one of a growing number of indicators that automation and orchestration solutions are maturing and moving beyond initial implementations to large-scale, business-as-usual technologies.
A close examination of recent pipeline developments provided more clarity about the respective strengths of CI/CD tooling and network automation tools for network change management and life cycle—GitLab, GitHub, Jenkins and CircleCI to name just a few. Most network teams would prefer to stick with a “right tool for the right job” philosophy, wherein the best tool for any given task can easily be integrated into an overall solution.
In this context, the “right tool” philosophy looks at pipelines and at network automation/orchestration tools to identify what they do best. Then, a strategy is developed that maximizes the benefit of both technologies. Using this approach, teams can avoid the pitfalls of trying to solve everything with one or the other, leading to a suboptimal process.
Some good current rules of thumb are based on two simple points:
By integrating both technologies into a common pipeline, network teams can get the best of both toolsets.
The example below illustrates an integrated model where network automation tools provide the engineering logic for the generation, testing and deployment of configuration changes, while the CI/CD tools (represented by the orange arrows) carry the activities through a repeated, automated process, ensuring that the changes go through a consistent path with proper version management, approvals and testing. Multiple teams have successfully implemented this model and they are beginning to iterate their solution to incorporate their unique experiences and improvements.
The widespread expectation from the industry is that, in addition to the integration of CI/CD and automation into a common pipeline, we will begin to see other indicators of maturation of orchestration and automation. Some of these are areas that are already taking shape, such as the shift in skillsets and competencies outside of traditional CLI-centric engineering toward DevOps and CI/CD skills, or the growing importance of quantifying the business impact of automation and orchestration.
Welcome to The Long View—where we peruse the news of the week and strip it to the essentials. Let’s work out what really matters.
This week: Linus says next release will support Rust, FBI warns scammers are getting hired in deepfake interviews, and 80% of IBM staff stay at home. (more…)
Salesforce today announced that it is now using robotic process automation (RPA) to connect disparate data via a no-code tool developed by its MuleSoft unit.
Shaun Clowes, chief product officer for MuleSoft, said bots created using an RPA-infused integration platform can now automate much of the effort previously required to integrate data residing in different systems. Salesforce acquired MuleSoft in 2018 and, since then, has been working to make the platform for integrating application programming interfaces (APIs) a core element of its business process workflow tools.
The news was announced at the MuleSoft CONNECT 2022 conference. The overall goal is to employ RPA and no-code tools to accelerate digital business transformation initiatives without writing code, noted Clowes. Overall, Salesforce claimed its customers are running 4.8 billion MuleSoft transactions daily, resulting in a 74% decrease in operational costs and a reduction of more than 100 billion hours of work every month, using Salesforce Flow tools that are integrated with the MuleSoft integration platform.
In theory, no-code tools that integrate disparate data sources can be used by professional and so-called citizen developers alike. That capability should reduce the time and effort required to create those integrations. A recent survey conducted by Salesforce found nearly 96% of respondents said that modifying and rebuilding automation is a challenge as systems and business requirements change. In addition, four out of five respondents noted that restructuring existing application and data landscapes to support automation would likely compound their organization’s technical debt.
More than half of respondents (53%) said they are using integration and API management capabilities to some extent, with 44% employing those capabilities to drive business process automation initiatives. A full 80% said they are working toward putting hyperautomation on their technology roadmap within the next 24 months.
Of course, the MuleSoft platform can be employed beyond the portfolio of software-as-a-service (SaaS) applications provided by Salesforce. However, it’s also become apparent that more organizations are building and deploying custom applications on top of a range of SaaS applications in addition to building and deploying applications written in procedural code that might be running in the cloud or in an on-premises IT environment.
It will be up to each IT organization to determine how much they need no-code integrations as part of a larger DevOps workflow, but as it becomes easier to create integrations using RPA tools their numbers should increase. The impact all those integrations will have on DevOps pipelines remains to be seen, as well. Many organizations will need to revisit their DevOps workflows as the number of application development projects being simultaneously launched continues to increase.
In the meantime, it’s clear that no-code/low-code tools infused with RPA capabilities will empower more end users. That shift will have a profound impact on DevOps teams, as the sheer volume of code and associated integrations increases. The challenge, of course, will be finding a way to make sure that the integrations created are secure and that they can actually scale as more data sources are added.
Welcome to The Long View—where we peruse the news of the week and strip it to the essentials. Let’s work out what really matters.
This week: Warren Buffett asks too much for wind energy, it’s “completely ridiculous” to say Google’s chatbot LaMDA is sentient, and Microsoft finally kills Internet Explorer. (more…)
Welcome to The Long View—where we peruse the news of the week and strip it to the essentials. Let’s work out what really matters.
This week: Verizon fined for allegedly spreading disease, Bolt Financial repeats the mistakes of the past, and the founder of Yandex resigns. (more…)
Welcome to The Long View—where we peruse the news of the week and strip it to the essentials. Let’s work out what really matters.
This week: Apple lets devs increase subscriptions “without consent,” Twitter has more than 19% spam accounts, and we dig into the claim that Microsoft is about to double salaries. (more…)
As the machine learning market matures, new tools are evolving that better match data science and machine learning teams’ needs. Vendors, both open source and private, have been quick to introduce new products that better meet requirements making it easier and faster to develop models and enable collaboration.
These new offerings range from cloud-based platforms that make it easy to build and deploy models to specialized software that can speed up the training process. As an example, the existing field of ML experiment tracking is a multi-billion dollar market with a crowded amount of vendors: Comet ML, Weights & Biases, Neptune AI, Databricks’ MLflow and more. These tools will need to evolve to move beyond meeting only the basic requirements of ML teams like experiment tracking, visualizations and model training to address collaboration and technical requirements so these teams can get models to market faster.
The types of teams that are driving modern requirements for data science are the bet-the-farm types, where it’s all about finding new technology and solutions to drive efficiencies and get models to market faster for their organizations to maintain a competitive edge. Going forward, we’re seeing the following as requirements for ML teams as they evaluate tools to add to their MLOps stack.
Data scientists and ML engineers use multiple tools to do their jobs, from Jupyter notebooks for model coding to DVC for experiment and data versioning, to automation tools like AirFlow to model visualization tools. Solutions will need to be open to integration with other tools instead of a walled garden that many ML tools, like those offered by cloud vendors (e.g. Amazon SageMaker, etc.) are now.
ML developers and engineers are familiar with standard DevOps tools and workflows like GitOps (think GitHub or GitLab) and live in CLI-oriented solutions. ML tools should integrate tightly with these existing tools instead of complicating the developer experience with a separate ecosystem. And as a part of the developer experience, workflow is also important. Data scientists can run hundreds of experiments using their preferred CLI-based solutions but, often, all those experiments are streamed and posted into existing ML tools. A more optimal approach would be to offer data scientists the experience of running experiments locally and letting them commit only the most important experiments for further discussion and collaboration so the team isn’t overwhelmed.
This approach makes a stronger foundation for something like tracking ML experiments. As ML teams race to get models to market and continually improve them, being in lockstep with DevOps teams to deploy those ML models into production is critical. An example would be a developer tracking five experiments with changing hyperparameters. For these experiments, they may have a link to a Git repo for a particular version of source code. But what if the source code changes or data set changes are not committed or reported in their existing ML solution? Then, the connection is lost and existing metrics would be based on a previous version of source code or data that’s no longer accurate. That lost connection slows down time-to-market and makes it harder to optimize models.
As seen with the above examples, ML teams juggle myriad tools and workflows as they develop models for various applications and services. The requirements discussed will help these teams to collaborate better and build models faster in an increasingly heterogeneous MLOps landscape.
Welcome to The Long View—where we peruse the news of the week and strip it to the essentials. Let’s work out what really matters.
This week: The first silicon for Wi-Fi 7, Google’s search quality “is” failing, and what Severance tells us about The Great Resignation. (more…)
Welcome to The Long View—where we peruse the news of the week and strip it to the essentials. Let’s work out what really matters.
This week: No remedy for banned Facebook users, new ARM-based VMs on Microsoft Azure, and The Great Resignation will still be a Thing for the foreseeable. (more…)