devopsy.kinsta.cloud

Category: Blogs

So Many Options, So Little Time

So many language/framework options, so little testing time, that is.

Recently, two projects I was working on drove me to check out a breadth of languages and/or frameworks to handle cross-platform development. While Unity and Unreal have traditionally been my game platforms of choice and I typically use Xamarin for cross-platform (though native development is more common for us on both Apple and Android), I decided I needed to do the occasional sweep to find out what is new out there.

The two projects are a game on which I’m helping a new game developer learn and a graphics-heavy application that is unsuited to web development. Both would work best in a cross-platform environment; both would benefit from a game engine’s graphics manipulations. But both could be done in any language that supports graphics, as there isn’t a heavy physics requirement to the game.

So I went looking at what’s available now and how it fits. From React Native to Godot to Flutter and even Kivy, I kicked the tires on each, got them running, ran and traced samples, wrote some code of my own, even talked with friends that were using them to get their opinion.

The one thing that came through loud and clear? There is no way that security and testing are keeping up with this language/framework proliferation. Kivy has a language (more of a specification than an actual language, but still—source code) that can run on top of Python (it’s not required to use the framework, but it’s kind of assumed you will if the app gets beyond a certain size); Godot has a language that is integral to the framework and that you use pretty much everywhere. React has a ton of add-on code and both Flutter and Dart are new languages.

I like stuff from each of them. I don’t like that we’re going to use them in ever-increasing amounts and we’re going to expect testing to stay on top of those implementations. Each of these platforms/languages/environments has a large following that is doing some of the testing, but since they’re all relatively new, “How much testing?” is a valid question. And tests have to be done in those languages, implying that those writing need to know them and so do testing teams.

Where did we end up? We haven’t made a final decision yet, but taking input from the person I’m mentoring, coworkers and those who are using each of the above, we’ll probably end up using Godot. For 2D apps, it is the bomb. Our team is small, so I can offer my knowledge of Godot to shore up testing and everyone agrees that its Python-like language is acceptable, while the graphic design element is nicer than the competition. Both applications will benefit from Godot’s graphics processing capabilities and the game will benefit from game elements built into the Godot Engine.

Here is an interesting side note: All of the products we looked at are open source. Both of the default products that we (and just about everyone else) would choose for the game are commercial. That’s interesting, but we don’t need the weight of those commercial solutions for a simple 2D interactive game/app.

For you all, the questions might be harder to answer. With larger projects come larger requirements for source size, test, you name it. Choose carefully, with an eye on the future. Personally, I think React Native, Godot and Flutter would all be good choices for cross-platform applications, with Godot the best solution for cross-platform 2D games. And all three are likely to be around with a good size following for a long time, making them safer choices. That is less clear with Kivy, though the popularity of Python appears to be driving growth in Kivy use.

Whatever you choose, plan for all phases of development, not just writing code. That’s true of everything, but new cross-platform tools are complex beasts, so it is worth reminding you all.

And hey, if you have a cool app you developed with one of these tools, point me to it. Always good to see what you all are doing, and nice to see the breadth of solutions these languages can create.

August 25, 2021
How to Successfully Scale Agile

Agile has proved its value to businesses across industries and geographies. In fact, the pandemic elevated it to an absolute necessity for organizations’ success and survival. Business leaders around the world invested aggressively in Agile transformation as they scrambled to meet changing customer demands through newly hybrid and remote workforces. Not surprisingly, the faster they adopted Agile methodologies at scale, the higher the value it delivered for them. Research indicates that the value Agile delivers can be a chance to grow ahead of peers as high as 63%.

What enterprises in the U.S. now need is to move the right Agile levers to maximize business outcomes. The same research prescribes an agenda to make that happen:

Mapping Customer Journeys

This involves assessing what purpose the company’s offering serves and how value flows through the systems that surround it to the customer. By identifying the critical path of value flow and prioritizing Agile investments at the nodes of value, both volume and velocity of value to customers can be impacted. For example, at Marcus by Goldman Sachs, every team, no matter which department they belong to, responds to the question about what they do with an answer that has clear customer focus: Enabling customer onboarding, improving the customer service experience or helping with the customer password journey.

Using Customer Insights

This can be achieved through A-B testing techniques and analytics platforms to unearth the attitudes and motivations that cause customer churn, build loyalty and convert emotions to action. This involves gathering customer feedback through surveys, discussions, and, in the case of digital channels, insight from the channel itself. Additionally, this lever enables diverse perspectives as customers are onboarded, with inputs all the way from design to operations. Kraft Heinz, as an example, leverages Snowflake’s platform to leverage data to drive new digital experiences for consumers. Snowflake gives them real-time ability to sense, respond and evolve, with insights resulting in more meaningful interactions with customers.

Organizing Teams Around Offerings Rather Than Functions

This involves bringing together skills and capabilities to launch disruptive products in the market and win back market share. Product-centric agile value delivery is especially helpful in scenarios that involve multiple handovers between internal teams that could lead to delays and siloed developments. For instance, Daimler draws on the principles of swarm to pull in expertise from various departments and create cross-functional teams to collaborate when developing new products.

Upskilling the Workforce

It’s imperative for organizations to upskill their workforce for continuous delivery, strategic vision and skill alignment within autonomous, cross-functional teams. The rapid pace of development required to meet changing customer needs often creates a skills gap that needs to be bridged. Siemens is a great example. They accelerated their digital transformation journey in 2020 and realized how vital it is for every employee to easily access upskilling and reskilling opportunities. To stay ahead of the curve, they deployed an agile digital learning platform for their employees while leveraging and simplifying their existing learning ecosystem.

Taking the Minimum Viable Product (MVP) Approach

This lever assesses real value delivery, validates hypotheses and enables quick course correction as required. Budgets are always limited, and an agile MVP approach also maximizes investment value with faster delivery, optimized processes and increased flexibility. A recent case in point is the approach bp took when started work on an integrated energy-as-a-service (EaaS) offering that will provide end-to-end management of a customers’ energy assets and services. Their first step was to pilot the plan in an innovation center/work campus before exploring opportunities to manage energy more broadly at industrial and business parks.

Expanding the Agile Workplace

Companies need to extend and adapt the agile workplace concept for a hybrid reality by creating remote analogs for meetings, sales execution and IT support along with advanced collaboration tools. A great example is GE Appliances; at the very start of the pandemic they accelerated their workplace with automation-driven agile services support across their global command centers and service desks.

Amplify Agile with Cloud

Scaling Agile without sound cloud infrastructure is a good job half done, especially when firms want to buttress speed and utility with business resilience. In fact, the very best firms stand to take a good share of the $414 billion in profits cloud generates across industries and regions every year.

August 24, 2021
Majority of Software Engineers Want Remote Work Options

The pandemic brought unprecedented change to the digital ecosystem, impacting users, companies and software engineers alike. COVID-19 accelerated ongoing digital transformation and forced many organizations to innovate rapidly. Newfound reliance on digital experiences increased demand for software programmers, exacerbating demand in a market already short on talent. In this era, work-from-home (WFH) became commonplace and most engineers moved toward full-time remote work positions.

Having had a taste of freedom from corporate settings, most developers now want remote work options in a post-pandemic future. A significant amount also desires to work 100% remotely. As a result, many companies have instituted permanent work-from-home policies. Yet, they are still grappling with the repercussions of a fully remote workforce. For example, there are differences of opinion about whether to adopt location-based salary adjustments and whether or not those remote workers should be granted unique employee benefits.

The 2021 State of Remote Engineering Report, conducted by Terminal, surveyed 1,100 software engineers in North America and explores how the pandemic has affected the remote developer workforce; showcasing both the positive and negative effects of remote work on engineers. Interestingly, the report exposes international talent’s declining interest in physically moving to the U.S. as domestic conditions worsen and remote work becomes more globally accessible. Below, we’ll explore the key takeaways from the study to posit how this new normal is influencing the productivity, morale and recruitment of software engineers in North America.

The New Normal: Remote Work

The new normal has dramatically shifted the status quo. Impressively, findings show that 86% of engineers are currently working 100% remotely. This is in stark contrast to the pre-pandemic numbers, which showed 19% working remotely 100% of the time. Many of these employees prefer to keep these new working arrangements and standards in place going forward. The study found that 80% of engineers surveyed want options for working from both home and the office, and one out of three want to work remotely 100% of the time.

There are many reasons why engineers prefer the flexibility of remote work to the traditional workplace. For one, there is no need to commute, thus saving time and energy. Engineers also cite an improved work-life balance and appreciate the increased autonomy. It also appears that many organizations handled the shift to remote work effectively—74% of engineers say their companies adapted well to remote work. Due to these positive results, 76% of engineers would be happy working fully remote 60% of the time or more.

Remote work not only benefits software engineers but could also bring productivity enhancements to help achieve business goals. Without workplace distractions, 70% of engineers surveyed reported they are more productive when working from home. Yet, this new normal isn’t without its drawbacks. As the traditional corporate firewall diminishes, CISOs struggle with a dilemma: striking an ideal balance between granting new IT freedom and locking down systems and restricting access to adopt a zero-trust mindset.

Issues With the Pandemic and Remote Work

Of course, there are some other drawbacks associated with full-time remote work. A lack of day-to-day interaction with coworkers is a challenge cited by 58% of participants in this study. For 39% of engineers, it’s harder to collaborate or feel part of a team in this new dynamic. Burnout or lack of motivation and home-based distractions can also be barriers for remote-working engineers. The loneliness of at-home work can take its toll; 38% say their mental health has gotten worse amid the pandemic.

Companies also face challenges recruiting within this new paradigm. The talent shortage remains a big issue for many companies trying to rapidly deliver new software; 64% of engineers say there is a software engineer shortage and 65% say this gap holds back technical development. Though there is a clear skills gap, most companies are not offering adequate benefits coverage to entice full-time workers. Of these benefits, 23% of engineers would like a home office stipend for utilities, internet and other expenses; 22% also want more flexible work hours and 11% desire new technology and productivity tools. And 26% of engineers also ranked support for mental health as one of their top three sought-after benefits.

The report also addresses some inefficiencies associated with hiring engineers for remote positions. For 59% of engineers surveyed, recruitment involves far too many interviews. Many of us working in IT can likely attest to being approached by LinkedIn scouts with irrelevant offers. As such, 40% of engineers say they are often interviewed by people who don’t understand the technology. Other recruitment issues include generic interviews and long delays, both of which could frustrate candidates. It appears companies must fix their hiring practices or risk losing potential talent.

The Era of Employee Choice

If anything, the State of Remote Engineering report drives home the imperative to consider full-time remote work as a viable option for prospective hirees. To fill engineering labor shortages, some U.S.-based companies have historically flown in international talent. Yet, the desire to move to work in the U.S. is declining. In fact, 39% of engineers have no interest in moving to the U.S. for a job. Engineers cite crazy politics (56%), gun violence and safety concerns (55%), racism (47%) and mishandling of the coronavirus (51%) as reasons not to move to the U.S. The idea that employees must remain in physical proximity with a company’s headquarters is changing; thus, companies will have to go where the existing talent already lives.

In addition, today, engineers have more bargaining rights than they did in the past. “The era of employee choice is here,” said Clay Kellogg, CEO, Terminal. If we’re in a seller’s market, engineers are the ones to profit. Increased bargaining rights could mean remote engineers may demand equal pay, no matter their location. The study found that, on the whole, engineers are not in favor of location-based salaries; 49% say they would only accept a location-based salary if their cost of living dropped by 20%.

In addition to offering competitive wages, teams are finding creative ways to maintain culture and avoid burnout. For example, 55% of engineers think virtual get-togethers like happy hours and social activities help stave off burnout. With all these findings in mind, companies should consider instituting a work-from-anywhere policy wherever possible and provide additional benefits and support for remote workers to attract more engineering talent.

The State of Remote Engineering surveyed over one thousand software engineers across the U.S., Canada and Latin America. For more granular details, consider viewing the full report here.

August 24, 2021
F5 Networks Planning Open Source Projects Beyond NGINX

F5 Networks today revealed plans to launch multiple projects in the months ahead that will extend its commitment to open source beyond the NGINX proxy software the company acquired in 2019.

The announcement was made during an online NGINX Sprint event; two of the projects are extensions of NGINX that will address networking requirements between microservices-based applications in addition to the management of fleets of NGINX instances. The third project is a platform-agnostic open source developer tool for building modern applications that does not necessarily require NGINX proxy software.

At the same time, F5 Networks also unveiled an open source NGINX Modern Apps Reference Architecture that gives IT teams access to a complete, fully operational microservices-based application that can be deployed in minutes on a Kubernetes cluster. Based on a modular approach made up of open source components that come pre-wired and pre-integrated, F5 Networks is committing to extending this framework in collaboration with multiple third-party vendors and organizations.

Rob Whiteley, general manager for the NGINX product group at F5 Networks, said the company is also committing to modernizing its NGINX proxy software to leapfrog rival open source Envoy proxy software being advanced under the auspices of the Cloud Native Computing Foundation (CNCF). NGINX is still more widely employed than Envoy largely because it is lighter-weight and easier to deploy, but Whiteley conceded that Envoy has more advanced capabilities.

Envoy, of course, is at the core of the Istio service mesh. F5 Networks supports both Istio as well as a service mesh it has developed on top of NGINX. Depending on the number of application programming interfaces (APIs) that an organization needs to support, an IT team might make use of proxy software, ingress controller or an API gateway for a service mesh to manage them.

Finally, F5 Networks said it will participate in the Kubernetes ingress project and that it is joining the gateway API community as its implements its own NGINX-based gateway controller to ensure its proxy software and Kubernetes remain tightly aligned in the future.

In general, Whiteley said F5 Networks also is making a commitment to be clearer about where the line between the open source and commercial editions of its offerings is drawn. As a rule, the open source editions are designed for developers and small teams while the commercial version is designed to scale within enterprise IT environments. The overall goal is to promote innovation without enabling cloud service providers to reap all the revenue benefits of F5’s research and development by creating a service based on that open source software, said Whiteley.

As part of that effort, F5 Networks is expanding the number of open source projects it supports to make it easier for developers to contribute code beyond the core NGNIX proxy software, said Whiteley.

It’s not clear to what degree F5 Networks will be able to rally the open source community to advance additional projects beyond NGINX. The company is promising to provide additional details at an unspecified future date. However, Whiteley said there is still plenty of room for innovations not only in emerging technologies such as service meshes but also proxy software that is already widely employed within many enterprise IT organizations. Especially since most of them are embracing microservices-based applications that will ultimately change every aspect of IT.

August 23, 2021
Edge Computing: Tech’s Next Trillion-Dollar Opportunity

Even as technology continues to move into the cloud, a simultaneous shift is already underway in the opposite direction as computing increasingly moves to the edge. One key reason: by 2025, the number of connected devices in use is expected to exceed 56 billion units, according to IDC. Edge computing is not a successor to the cloud, but another expansion of technology’s reach that represents an exceptional opportunity for investment and growth.

As edge-device and computing adoption continues to skyrocket, organizations need to process the proliferating data from these devices locally so they can act on it in real-time. This will require investments in edge-adapted infrastructure and related tools and platforms. Enterprise technology is about to be transformed yet again — the need to process and analyze some data at the point where it is both created and consumed is becoming a business requirement as mandatory as moving other parts of the technology stack into central locations in the cloud.

Organizations already must deliver compelling customer experiences with edge devices based on data that must be continuously refreshed. The alternative—uploading and downloading data to and from a cloud service or local data center—often takes too long. Whether the data is on a consumer’s smartphone or a manufacturing floor, organizations need to process and analyze data as near as possible to the point where a real-time data-driven response is required.

The competition to provide these capabilities is already intense. Many vendors—ranging from cloud service providers to telecommunications carriers to longtime providers of server and storage infrastructure—view edge computing as their next trillion-dollar opportunity.

Why We Compute at the Edge

Edge computing has been around for decades in various forms. There are countless examples of standalone software being deployed on, for example, devices used on factory floors that automate various aspects of a manufacturing process. In recent years, many of those systems have been connected to the Internet to enable organizations to collect and share data more easily. The challenge now is that the amount of data being generated at the edge often exceeds the point where it is practical to transfer that volume of raw data across a wide area network (WAN). Platforms that can process and analyze large amounts of data at the network edge are required. The aggregated results of data processed on an edge platform can then be shared more efficiently with other applications running in the cloud, in an on-premises IT environment or even on the network itself.

Low-latency requirements are today’s biggest challenge. In many scenarios, an application running on a gateway at the edge of a telecommunications network needs to be able to respond in a fraction of a second to a mobile computing application’s request for data. In other circumstances, an edge application is driving an automated factory operation that needs to dynamically adjust to analytics processed at the edge of an extended enterprise.

There are even more advanced use cases, some of which are described below. These include in-car computers that process computer-vision data in real-time for driverless vehicles and robotics applications that promise to transform manufacturing.

Hardware Advances Enable Edge Computing

The often competing demands of low-latency network connections, high-power computing and large storage requirements are driving innovations in edge hardware. Electric carmaker Tesla, for example, has added custom-made devices to its processors capable of pre-processing massive amounts of data, performing machine learning inference to make quick driving decisions and predictions and even do some deep learning model training on the vehicle itself–before sending a subset of the data to central systems to do even more training.

Earlier this year, Tesla unveiled a processor with six billion transistors. According to the company, its device boosts performance by 21 times compared to the Nvidia GPU it replaced in its Tesla Model S, Model 3 and Model X models. These powerful devices are used in addition to CPUs from traditional providers such as AMD and Intel for workhorse applications like infotainment.
With this computing power at the edge, a Tesla car can process sensor data that allow it to recognize and self-pilot around pedestrians, other cars on the road, emergency roadside signage and potentially hazardous moving objects. This sensor data is also processed with pre-loaded mapping data and GPS connectivity. Additionally, Tesla’s in-car computers relay information over Tesla’s network to massive storage facilities where the data is analyzed to improve Tesla vehicles’ self-driving and other functions. Improvements to the vehicle’s software stack are then downloaded to it over the network.

Healthcare is an example of a sector with similar needs for advanced machine learning-taught devices and software. Images from MRI and other scanning devices with inference-computing capabilities deployed at numerous locations would also require computing power and network infrastructure similar to that which Tesla’s cars need. Most hospitals, however, don’t have the interest or wherewithal to build their own powerful edge computing hardware and will need to consume this capability from technology vendors.

As advances in semiconductor circuits approach seven nanometers and smaller, the ability to run applications at the edge in an energy-efficient manner becomes more realistic for a wider range of organizations. With each new iteration of processors, the total cost of edge platforms will continue to decline while the amount of horsepower available to run applications steadily increases.

Networking at the Edge

Regardless of how much data is processed and analyzed at the edge, the sheer volume of data creates more demand for network bandwidth and, consequently, the potential for bottlenecks. One main bottleneck is at the backhaul—the stretch between the edge towers and central servers and network. Whether tomorrow’s edge network infrastructure will consist of 5G, more ultra-powerful WiFi connections and devices, low-latency daisy chain systems, fiber systems such as those that Google is developing, satellite connectivity, yet-to-be-created technology or a combination of these is still unclear–but one thing is certain: Data volumes are increasing faster than network bandwidth.

Today, organizations have a choice of connecting edge platforms among several available wired and wireless network alternatives. The number of use cases involving edge computing is expanding in part because wireless 5G networks make it possible to share data across multiple interconnected edge platforms. For example, manufacturing employees wearing augmented reality headsets that can share data and analytics in near-real-time not only with their coworkers on the floor but with colleagues hundreds of miles away.

As telecom carriers continue to virtualize the network infrastructure they employ to deliver 5G services, the cost of delivering them should steadily decline across a widening range of geographic areas. Today, a 4G network can support roughly 4,000 devices per square mile. A 5G network could pack up to three million devices in the same area.

It’s still early days as far as 5G is concerned. A Capgemini survey of 1,000 industrial organizations that plan to incorporate 5G into their operations finds less than a third (30%) have progressed to trials and real-world implementations. Yet, carriers are already at work defining the next generation of 6G wireless standards that promise to deliver another giant leap for wireless networks in another decade.

Regardless of the network employed, organizations will still need to strike a balance between how much data needs to be processed, stored and eventually disposed of at the edge versus the amount of aggregated data that needs to be transferred over a WAN and incorporated within various other platforms that process data centrally or function as systems of record.

Factoring in Security

Arguably, the biggest obstacle to edge computing is security. Each time an organization deploys an edge platform, the attack surface they need to defend grows. Many of these edge platforms are also managed by operational technology (OT) teams that don’t have as much cybersecurity expertise as does, say, the data center security team at Google. Most organizations are trying to meld their OT and traditional information technology (IT) teams in an effort to deploy, manage and secure edge platforms. However, the cultures of these two teams are significantly different. It takes time for organizations to harmonize the efforts of these two groups.

More challenging still, cybersecurity expertise is hard to find and retain. Cybercriminals have discovered how vulnerable OT environments are. Attackers affiliated with various nation-states have already demonstrated that industrial control systems controlling electricity grids and other critical infrastructure systems are primary targets should war erupt. From their perspective, each edge computing platform connected to a network is yet another vehicle through which malware can be introduced into an IT environment. It only takes a few minutes for malware to propagate laterally across an entire platform.

Ultimately, organizations will soon spend billions of dollars securing edge platforms. Once the edge systems are locked down and secure, edge processing can improve overall cyberdefense—processing more data at the edge without moving it around as much can dramatically reduce the attack surface.

How the Edge Computing Ecosystem is Evolving

Once the right level of infrastructure is in place, the next challenge is building applications. The widespread adoption of container technologies has allowed developers to build applications using artifacts that require much less memory and storage. Most container environments today, like those based on Docker and Kubernetes, are decentralized, in a sense, but are run in centralized environments like data centers. But work is progressing on even smaller container formats that make it even easier to deploy application software on, for example, an oil rig hundreds of miles from any server or gateway.

Lighter-weight instances of the open source Kubernetes container orchestration engine are starting to gain traction. Kubernetes, by surfacing a consistent set of application programming interfaces (APIs) regardless of what platform it is deployed on, provides an opportunity to centralize the deployment and management of modern microservices-based applications across highly distributed computing environments.

Industry API specifications and frameworks are also beginning to emerge for machine learning, data science and other technologies that empower edge computing. An example is oneAPI, supported by Intel and others, which provides a framework for developing edge applications.

Providers of development platforms are also racing to create frameworks that make it simpler to build applications within edge architectures that seamlessly invoke backend application services running in the cloud. Distributed event-streaming platforms like Apache Airflow and Kafka can play a role in allowing edge computing platforms to share data with those backend platforms at scale. Some organizations will also invoke the services of content delivery networks (CDNs) to improve the performance of applications deployed across a global network of points-of-presence (PoPs) that have already been constructed.

Finally, stateful applications deployed at the edge can require local databases that provide access to persistent forms of data. The bulk of container applications deployed today are stateless—they store data on an external storage system. Building stateful applications that access data in containers on a local Kubernetes cluster is more challenging.

There are already millions of developers familiar with container platforms who are standing by until improved hardware and software simplifies and secures deployment of highly portable applications as far out on the edge as possible.

The Coming AI and Data Management Challenge

The most exciting edge innovations ahead will likely be driven by the rise of machine learning and AI. Today, AI models are typically trained within centrally located cloud computing platforms. An AI and machine learning inference engine can then be created and deployed in a production environment, where a trained AI engine deduces what actions to take as new data is discovered.

Over time, those inference engines are subject to drift as the amount of new data collected exceeds the parameters on which the original AI model was based. Data science teams working in collaboration with IT and OT teams then need to train a new AI model to replace the inference engines at the edge. Advances in new AI and machine learning capabilities, in conjunction with lower-power and higher-performance processors and other components to help lower costs, can improve the accuracy and performance of prediction systems, but typically doesn’t reduce the degree to which this drift happens.

If AI models can be at least partially retrained at the edge, this can help to reduce prediction accuracy drift. However the software and hardware infrastructure required to retrain an AI model efficiently within an edge platform is still very early-stage. That said, as the cost of increasingly high-powered hardware declines and edge machine learning software systems improve, the ability to train at the edge will become more feasible.

The Economic Impact and Investment Opportunity

The global edge computing market was valued at approximately $3.6 billion in 2020 by Markets and Markets. The research firm forecasts this market will reach $15.7 billion by 2025, representing an astounding 34.1% compound annual growth rate (CAGR).
McKinsey forecasts that the economic value generated by use cases involving internet of things (IoT) applications alone will be somewhere between $3.9 trillion and $11.1 trillion per year by 2025. Add in all the other use cases for edge platforms and the potential for positive economic impact becomes enormous.

Increased computing at the edge will be a huge technology market in its own right and will also drive increased demand for centralized cloud computing. Gartner notes that by year-end 2023, 20% of installed edge computing platforms will be delivered and managed by hyperscale cloud providers. The other 80% of this massive emerging market opportunity is up for grabs.

Lonne Jaffe co-authored this article.

August 23, 2021
Why You Should Use GitOps to Experiment With AI

As the pace at which artificial intelligence (AI) models are being constructed and inevitably updated starts to increase, it’s becoming more apparent that the pace at which data science teams are currently experimenting also needs to dramatically quicken.

Many data science teams today are fortunate if they manage to successfully deploy two AI models in a production environment over the course of 12 months. It’s already apparent, however, thanks in part to the rise of digital business transformation, that there are potentially hundreds of use cases for AI models within each enterprise IT environment. Truth be told, it’s hard to envision an application being developed today that would not incorporate one or more AI models. The challenge is, the demand for AI capabilities is now far outpacing the ability of data science teams to deliver.

The best way to solve this issue is to recognize that AI models and the components employed to construct them are, in reality, just another kind of software artifact. In fact, the need to accelerate the development of applications using artifacts is largely a problem already solved. Rather than reinventing the wheel, the time has come to apply what we already know to the development of AI models.

The Rise of GitOps-Based Processes

Modern software development teams today routinely make use of Git-based repositories to share code and collaborate. These repositories are jam-packed with vetted software artifacts that development teams continuously reuse via an opinionated instance of a DevOps workflow based on a set of GitOps best practices.

As a subset of DevOps, a GitOps workflow enables data science teams to employ a Git repository that already exists within their software development environment to manage artifacts. Git enables versions of files to be created using what is known as a Git branch. They also employ concepts known as tags (Git Tag) and remotes (where the server is located). Those three different concepts, or references, exist under the hood in Git. In fact, the entire flow is based on those concepts. When a new feature is implemented, a new branch is created and then merged into the production branch with a tag that says something like “This is a new version that needs to be deployed.”

An AI model, like any other software artifact, needs to be updated. As new data sources become available, an AI model is likely to drift beyond the parameters originally intended. A rapid change in business conditions may also require changes to an AI model. The same processes that DevOps teams apply to update any software artifact can also be applied to rapidly update an AI model. Employing a Git repository to also manage AI artifacts extends the benefits that software development teams enjoy to data science teams.

Often, organizations will have an MLOps solution to streamline AI model development that is completely separate from their DevOps tools. These solutions will have dedicated features like version tracking for data and experiments specifically for AI models. But many MLOps solutions exist in a closed ecosystem, meaning they don’t work off of existing tools like GitHub used on the DevOps side.

Also, data science teams have to separately buy, deploy, and manage these closed MLOps solutions. Going with a GitOps-oriented solution streamlines management of added MLOps tools and makes it easier for both application developers and data science teams to gain visibility into each other’s respective efforts. The end result is not just faster AI experimentation at lower cost; the likelihood that more AI models will successfully be deployed in a production environment increases dramatically.

Software and AI Teams Working Together

The convergence of DevOps and MLOps is already inevitable. The longer organizations continue to employ isolated MLOps platforms to build and deploy AI models, the more expensive it will become to meld MLOps and DevOps processes.

Data science teams are conservative by nature. The difference between success and failure often comes down to how thoroughly an AI model is validated. However, time to market doesn’t have to be sacrificed to ensure an AI model works as promised. There is no such thing as a great AI model that arrives late when rivals are racing to develop similar AI capabilities—there are only missed opportunities. Speed enabled by the reusability of artifacts that drives faster experimentation using GitOps workflows is of the essence now more than ever.

August 23, 2021
Infrastructure-as-Code Security will Improve Your Cloud Security

Moving to the cloud creates a tremendous opportunity to get security right and reduce the risk of a data breach. Most organizations start their cloud security efforts by ‘shifting security left’ thereby addressing issues in the CI/CD pipeline. Fixing known vulnerabilities and risky configurations before pushing images to production makes sense. But there is an opportunity to shift security even further left—as you set up your cloud infrastructure—to reduce risk. Infrastructure-as-code (IaC) security allows you to define security controls as you configure your cloud infrastructure.

Getting Started on Your Cloud Journey

Before we dive into IaC security, let’s talk about why platform teams want to deploy infrastructure-as-code as they move through their cloud journey. Most organizations have not yet tackled IaC. They start their journey by experimenting with cloud configurations until they have defined a stack that works for their teams. In this early stage, application teams may be defining their own cloud stacks. They are manually configuring infrastructure or using scripts. Configurations may not be standardized across teams.

Standardization of Infrastructure Saves Time

Although experimentation is reasonable early on in the journey, standardization of infrastructure saves time. Cloud services are no different than applications that run on premises. When your teams are using different versions of tools in their stack, getting an application up and running can be a nightmare. Interoperability between services is challenging, and it can feel like you are fixing a different flavor of the same problem over and over.

Mature cloud platform teams typically prefer to create a ‘golden image’ with a baseline set of services for development teams. They may create a menu of a few standard options to meet different teams’ needs. Because they are standardized, the platform team can take time to validate that they interoperate and are secure. For example, these golden images will be regularly scanned for vulnerabilities and updated to ensure they meet internal security standards and any relevant regulatory requirements.

Automating Infrastructure Deployment Reduces Errors

Manually configuring the environment in the cloud for each team is tedious. Once these configurations have been standardized, the cloud infrastructure can be provisioned as code using templates. A declarative approach defines the desired end state, whereas an imperative approach defines the detailed steps to achieve the end state. The templates are normally stored in a Git repository and version controlled. Automating the configuration of infrastructure not only saves time but also reduces errors that can disrupt operations.

IaC Security Shifts Security Further Left

Once you have templates that define your infrastructure, standardizing permissions and access rights for services-as-code is a logical next step. The security team typically defines the guardrails or policies for accessing cloud services. These policies can then be defined as code along with the infrastructure configurations. This IaC security is a further ‘shift left’, allowing security to be embedded into infrastructure, even before an application can be deployed. Implementing policy-as-code minimizes the opportunity for human error in configuring security services. By defining these policies and enforcement guidelines together, the DevOps and security teams can establish a trusted partnership.

Although some organizations may start their journey by implementing IaC security, for most organizations it happens later in the cloud journey. The order is less important than ensuring you take a DevSecOps approach and embed security at each stage to manage risk. Be sure to address the following areas as part of your cloud and container security program:

Continuously Check Cloud Permissions and Configurations

Even if you have IaC templates, you need a program to regularly take inventory of all cloud assets and confirm permissions and configurations. The Center for Internet Security (CIS) benchmarks are commonly used to check passwords and access rights. Beyond enforcing these best practices for security on cloud services, continuously checking for changes to permissions and suspicious activity is critical. The cloud vendors capture logs across all of their services, and your security tool should continually monitor and alert using those logs.

Scan Images for Vulnerabilities and Risky Configurations

Scanning for vulnerabilities and ensuring configurations use best practices allows you to shift security left into the development pipeline. Scan everywhere you have images—including within registries, pipelines and also check for new vulnerabilities identified for images running in production. Be sure to scan hosts as well as containers. Defining scanning policies and enforcing through the development process creates guardrails for the development team and clarifies expectations. Most regulatory compliance standards have image scanning requirements as well.

Detect Threats and Respond to Alerts

Prevention is fundamental but must be supplemented with threat detection and response. Alerting on unexpected behavior can identify the presence of a threat. Initially, teams may use out-of-the-box detection rules as a way to get started and then customize rules over time. Checking configs against dockerfile best practices is a good way to get started. Overly permissive access rights for containers is common in Kubernetes environments and should be allowed only when required.

As your team develops more expertise with containers and Kubernetes, you can also implement zero-trust network security policies to block unexpected communication between pods, services and applications.

Some DevOps teams handle triage of alerts, as they can quickly determine if the issue is a configuration issue or an indicator of compromise. Teams often map detection rules to the MITRE ATT&CK framework, allowing their security teams to manage alerts consistently across their environment. For any team to effectively respond to a security alert, it is important to capture a detailed record of exactly what happened before, during and after that event. This visibility can be particularly challenging with containers but it is critical because containers have short lives. The 2021 Sysdig Container Security and Usage Report revealed that about half of containers live less than five minutes.

Complying with Regulatory Standards

Complying with regulatory standards can be painful. Some standards were written with cloud computing in mind, like NIST 800-53 and SOC2, and others were not. With those broader standards, it is important to understand the controls that are relevant to containers and the cloud.

As you consider your compliance program, be proactive—don’t wait until you are ready to roll into production. Start early by engaging your compliance team and helping them get up to speed. Your role is to provide evidence of compliance to pass an audit at any time using the documentation from your security platform.

Getting Cloud and Container Security Right

The move to the cloud creates a tremendous opportunity to get security right and reduce risk. The fact that physical security and a chunk of operations are delegated to cloud vendors frees up security and development teams to focus on their more narrow scope within the shared security model. By taking a secure DevOps approach and inserting security into every stage of the application life cycle, your team can confidently and securely run cloud and containers.

August 23, 2021
The Limits of Expression

August 23, 2021
The Pace of Change Requires Better Testing
The COVID-19 pandemic in 2020 created a pressing need to give people additional ways to do their jobs and handle the tasks of day-to-day life via mobile devices and forced the growth of app innovation at a never-before-seen rate. Thanks to new application development technologies and agile processes along with open source and embedded libraries, organizations are delivering greater functionality, faster, and for multiple devices (think web, mobile, tablet, etc.). And while a never-ending stream of new apps and updates has been great for consumers, it’s proven to be a pain point for many enterprises whose testing processes aren’t up to speed, literally and figuratively.

While test automation is an essential component of agile development, it’s not addressed in published agile methodologies such as Scrum and Kanban. Of course, testing tools and methodologies have changed along with modern development practices, but they haven’t always kept up with today’s rapid-fire pace of AppDev, and the past 18 months have only served to elevate the need for improved testing. For companies to keep up with and move beyond their competition, testing must be conducted in parallel with development.

Automated and Continuous Testing: Analogous, not Synonymous

There are many ways to test software, including manual, user interface (UI), automated and continuous testing (CT), but most enterprises today employ some form of automated testing and, increasingly, continuous testing. All too often, people throw the terms automated and continuous around as if they are synonymous, but they aren’t. So, before we delve into the merits of one over the other, it’s important to have a clear understanding of what each of the terms means.

Automated testing refers to conducting software tests programmatically with little to no human interaction. It measures actual outcomes in relation to predicted outcomes, and, as it’s less time-consuming to execute than manual testing, has become the norm for modern enterprises. Continuous testing, meanwhile, uses automated tests to provide uninterrupted feedback at every stage of the software development life cycle and, therefore, plays a key role in DevOps.

Automated and continuous testing work hand-in-hand to ensure software quality, cost-savings and speed time to market and should be conducted in lockstep with development. Whereas it’s possible to have one without the other, it’s far from advisable, however.

The Upside of Continuous Testing

Continuous testing takes automation to the next level and, as such, holds several advantages over automated testing alone.
- Early and actionable feedback at each development stage means issues can be addressed sooner—before they evolve into larger, more complex issues.
- Agile teams’ velocity is measured by how much they can complete in a sprint. Because of this, a team’s testing focus is narrowed to the sprint they are immersed in, meaning that problems affecting functionality previously deployed often aren’t discovered until later in the development life cycle, or they are discovered by users after deployment.
- An automated and continuous approach means more problems are identified and fixed before deployment.
- Release cycles are accelerated as organizations can respond to market changes faster and in real-time.
- Correcting problems early reduces rework, which translates into cost and time savings.
- Continuous integration requires continuous testing.
- CT tests not only the parts of the application that have changed but also how those updates work within the app’s entire functionality.
- Unlike automated testing, CT requires no human intervention.
What’s the Holdup?

Despite numerous benefits, CT isn’t as widespread as one would hope. One of the reasons for the disconnect comes down to investment, whether it’s hiring the right people, training or tools.

Perhaps the greatest stumbling block comes at the hiring stage. CT requires a specific set of technical skills coupled with a natural aptitude for test development and a mindset focused on resiliency. Beyond investing in staff, there are upfront (and ongoing) automation costs to consider. Unfortunately, it’s not as if you can implement CT and never touch it again—CT becomes its own software delivery pipeline that must be developed, maintained and enhanced to ensure it doesn’t become obsolete.

Continuous Testing: What Could go Wrong?

As the pace of AppDev quickens, so does the imperative to integrate it into DevOps. Failure to do so has serious implications, the most obvious of which is that without CT, companies fail to test as much and as frequently as they should. Regression tests aren’t done as deeply, for instance, so when a new app or update is released, it’s more likely that something will break. Further, without the right skill sets, teams have a tendency to examine whatever is new and fail to see how it fits into the whole. Moreover, without proper testing, you may inadvertently introduce bugs or discover too late that functionality doesn’t work as planned. Relying on manual and slow, fragile UI testing will only hinder progress in organizations that don’t adapt.

Negative consequences go beyond the confines of an agile team or enterprise as poor software design and implementation feed into the all-important issue of customer (dis)satisfaction. As everyone knows, unhappy customers aren’t loyal customers, and that translates to a lower bottom line. Another equally serious (perhaps more serious, depending on your perspective) ramification is the potential impact on data security, exposing organizations to a very different but significant risk posture.

Thanks to DevOps, ideas are moving from deployment to business value at lightning speed. It stands to reason, then, that testing also must be faster than ever.

When you consider that what you see today is the least amount of change you’re going to experience and that tomorrow is guaranteed to bring more, it should be abundantly clear why continuous testing is so important: The less testing you do, the more problems will occur downstream.

Businesses develop software for a reason—to get business value. If your app isn’t working, you aren’t reaping its full value. So, instead of asking yourself if you should implement continuous testing, you need to be asking yourself why you haven’t done it already. By failing to invest in CT now, you will undoubtedly pay for it later.
August 20, 2021
Kentik Open Sources Five Projects to Advance Network Observability

Kentik today unfurled a developer hub, dubbed Kentik Labs, through which it is making five open source projects available as part of an effort to advance observability across network operations (NetOps).

The five projects are:

kTranslate: A system for pulling and pushing network data that is the basis of the Kentik Firehose integration the company created for the New Relic observability platform

NetDiag: Scalable, asynchronous implementations of low-level network diagnostics

Convis: Example code showing how to attribute process and container information to network traffic via the Linux kernel using eBPF

kProbe: A high-performance host and sensor network probe (PCAP)

Grafana App: A real-time tool for ingesting and querying network data

Nick Stinemates, vice president of business development at Kentik and co-chair of Kentik Labs, said the goal is to enable DevOps teams to incorporate NetOps within the workflows already being employed to programmatically manage the rest of IT.

Stinemates noted that, as IT environments become more distributed, managing NetOps has become more challenging. If there is a failure, it can be difficult for DevOps teams to determine if the issue is due to application code, the underlying infrastructure or some other root cause. Kentik Labs is committed to providing tools that will help DevOps teams reduce their mean time to recovery (MTTR), he added.

Most NetOps teams increasingly rely on software to manage distributed networks either in the form of graphical tools provided by router and switch vendors or by using the Python programming language and the open source Ansible automation framework that enables IT teams to declaratively manage IT infrastructure.

As the teams that manage network operations gain more proficiency with those tools, it becomes simpler to incorporate them within the construct of a larger DevOps team. While DevOps teams have been programmatically managing IT infrastructure-as-code (IaC) for years, network management has largely remained in a separate silo. However, as more applications are deployed in the cloud, it’s clear NetOps needs to become more integrated with the rest of the workflows employed to deploy and manage a highly distributed IT environment, said Stinemates.

Less clear is to what degree the networks that make up a hybrid cloud computing environment will continue to be managed by network administrators versus site reliability engineers (SREs) that are starting to assume responsibility for managing compute and storage.

Regardless of who manages the network, the next generation of microservices-based applications that are dependent on application programming interfaces (APIs) to call various services are much more latency-sensitive than previous generations of applications. Many of those applications will regularly experience performance issues due to networking issues that are often not readily apparent to DevOps teams. The goal should be to, at the very least, arm those DevOps teams with the tools required to confirm the source of those issues before a “war room” is convened to determine how to address the issue.

August 19, 2021