Welcome to The Long View—where we peruse the news of the week and strip it to the essentials. Let’s work out what really matters.
This week: Alleged theft of GPL code, and Amazon will run its data centers on gas. (more…)
Category: Features
-
Voice.ai ‘Stole’ Code ¦ AWS Gets Filthier
-
Cisco Bets on OpenTelemetry to Advance Observability
Cisco today announced a technology preview of a Cisco Full-Stack Observability Platform based on instances of open source OpenTelemetry agent software being advanced by the Cloud Native Computing Foundation (CNCF) to collect metrics, event, logs and distributed traces. The Cisco Full-Stack Observability platform will be available in June 2023.
The platform, announced at a Cisco Live EMEA event in Amsterdam, Netherlands, will combine the capabilities of multiple Cisco platforms and technologies into a single offering.
At the same time, the company also announced the bidirectional integration between its existing Cisco AppDynamics application life cycle management (ALM) tool and the Cisco ThousandEyes network monitoring platform beginning in April.
Finally, Cisco is making available a Business Risk Observability offering to provide business risk scoring. Business Risk Observability combines capabilities of its Kenna Risk Meter tool—gained when Cisco acquired Kenna Security in 2021—with the ability to monitor business transactions through Cisco AppDynamics to provide better visibility into application security. In addition, Cisco has integrated its Panoptica application programming interface (API) management security tool and the Talos threat intelligence platform within the Business Risk Observability Platform.
Carlos Pereira, a Cisco fellow and chief architect for strategy, incubation and applications at Cisco, said Cisco is moving to narrow the historic divide between application development and cybersecurity teams to enable organizations to adopt DevSecOps best practices and make it simpler to prioritize the remediation of vulnerabilities that are likely to have the most impact on the business.
Like most providers of observability platforms, Cisco is betting that more applications will be instrumented as OpenTelemetry software continues to mature. Cisco is now the single largest contributor to this project, noted Pereira.
OpenTelemetry will also play a critical role in improving application security as more applications become instrumented, he noted. In fact, a recent Cisco AppDynamics survey found more than three-quarters of respondents (76%) believed that DevSecOps is essential for organizations to effectively protect against a multi-stage security attack. However, only 43% reported their organization had started implementing DevSecOps workflows.
Observability, of course, has always been a core DevOps tenet, but achieving and maintaining it is challenging. Most DevOps teams today aspire to maintain some level of continuous monitoring. However, as it becomes easier and less costly to instrument applications, interest is rising in observability platforms that make it simple to investigate anomalies indicative of an issue that could disrupt an application environment. Most of those observability platforms are infusing those platforms with machine learning algorithms to surface those anomalies as part of an effort to augment the capabilities of a DevOps team.
Regardless of the approach to observability, many organizations will soon be trying to determine whether to rely on existing monitoring tools that make it possible to track pre-defined metrics. In some cases, there may still be a need for that legacy approach to managing IT. Still, in other instances, it may be possible to rationalize some of those platforms in favor of a more advanced approach that relies on a more advanced approach to achieving observability.
-
Five Great DevOps Job Opportunities
DevOps.com is now providing a weekly DevOps jobs report through which opportunities for DevOps professionals will be highlighted as part of an effort to better serve our audience.
Our goal in these challenging economic times is to make it easier for DevOps professionals to advance their careers. Of course, the pool of available DevOps talent is still relatively constrained, so when one DevOps professional takes on a new role it tends to create an opportunity for others.
The five job postings shared this week are selected based on the company looking to hire, the vertical industry segment and, naturally, the pay scale being offered.
We’re also committed to providing additional insights into the state of the DevOps job market. In the meantime, for your consideration:
LinkedIn
Apple
San Diego, California
Site Reliability Engineer (SRE) – Video
$115,000 to $217,500SimplyHired
ScientiaMobile
Reston, Virginia
Senior DevOps Engineer
$110,000 to $150,000Monster.com
Larsen & Toubro Infotech Ltd.
Farmington, Connecticut
DevOps Engineer
$110,000 to $130,000Indeed.com
TherapyNotes.com
Remote
Senior DevOps Engineer
$105,000 to $155,000Dice.com
MRoads
Dallas, Texas
Platform Engineer – SRE
$100,000 to $120,000 -
Blameless Integrates Incident Management Platform With Opsgenie
Blameless this week announced it has integrated its namesake incident management solution with the Opsgenie alerting platform from Atlassian.
Aaron Lober, director of product marketing for Blameless, said the Blameless platform is now integrated with both Opsgenie and PagerDuty, the two most widely used platforms for managing alerts.
Blameless provides an incident management platform that normalizes data collected from a wide range of DevOps tools and platforms to determine how best to respond to different incident classes. Armed with those insights, it then becomes possible to quickly assemble the best team to respond to an event, said Lober.
That capability reduces stress because Blameless also surfaces recommendations for resolving incidents based on the data it has collected, he added.
The Blameless platform is tailored to organizations that have hired site reliability engineers (SREs) to manage IT operations using DevOps best practices, noted Lober. The overarching goal is to eliminate rote processes that, over time, conspire to burn out incident management teams that often experience high turnover rates, he added.
Opsgenie has been connected to the Blameless platform via the service catalog and alerting functions. Blameless users responding to or managing an incident can escalate issues to any teams defined within Opsgenie. The service catalog and alerting data generated throughout the life cycle of an incident is automatically captured and stored on the Blameless Incident Timeline. That capability makes it possible to identify appropriate service owners and the escalation protocol that was followed by that team. Blameless can also display the actual user name of each responder that has been notified.
When accessing Blameless via Slack or Microsoft Teams, Opsgenie can automate both when an incident channel is created and pre-select relevant services that should be made available. Alerts can also be manually triggered using commands from the Blameless chatbot for those who prefer to trigger alerts from within Slack or Teams themselves.
As IT environments become increasingly more complex, more organizations are making a concerted effort to apply DevOps best practices to incident management. The goal is to reduce downtime and increase overall resiliency at a time when organizations are more dependent on software than ever. As part of that effort, IT teams should identify the repetitive processes that span multiple incidents and then move to automate as many as possible.
In the longer term, it’s not clear what impact artificial intelligence (AI) will have on incident management. At some point, though, IT teams should expect their efforts to be augmented by machine learning algorithms capable of identifying issues before they balloon into a major crisis. However, there will always be some human intervention required to manage incidents for the foreseeable future, noted Lober.
In the meantime, most IT teams should continue to expect the unexpected at a time when IT environments have never been more dynamic and change multiple times a day.
-
OpenAI Hires 1,000 Low Wage Coders to Retrain Copilot ¦ Netflix Blocks Password Sharing
Welcome to
The Long View—where we peruse the news of the week and strip it to the essentials. Let’s work out what really matters.
This week: ChatGPT darling OpenAI wants people to write code in English, and the unintended consequences of blocking shared accounts. (more…)
-
Red Hat Brings Ansible Automation to Google Cloud
Red Hat today announced that the Red Hat Ansible Automation Platform is now available on Google Cloud to simplify the management of yet another cloud platform.
Using the Ansible Automation Platform, it’s now possible for IT teams that invoke Google Cloud to access pre-integrated services such as Google Virtual Private Cloud, security groups, load balancers and compute and instance groups.
Pete Cruz, head of product and technical marketing for Red Hat Advanced Cluster Management for Red Hat, said Google Cloud—alongside existing support for rival cloud services—will make it simpler to consistently apply automation across multiple cloud and on-premises IT environments.
In addition, IT teams can also count the consumption of the Ansible Automation Platform toward any existing enterprise spend agreements with Google.
Ansible has emerged as a de facto standard for IT automation and is widely embraced by IT vendors and cloud service providers. That level of support has fostered the creation of a level of automation content that is now unrivaled by any other automation framework, noted Cruz.
In the meantime, the pressure to automate IT processes is building during uncertain economic times. As IT environments continue to become more complex—thanks, in part, to the rise of cloud-native applications—many IT organizations are finding it more challenging to convince business executives to hire additional full-time employees. As a result, many of those IT organizations are now looking to automate as many processes as possible as more workloads are distributed from the network edge to the cloud.
Rather than create a single playbook to automate every IT process on an end-to-end basis, organizations generally mix-and-match content created by others to automate workflows across highly distributed computing environments, noted Cruz.
The biggest challenge when automating IT processes is simply overcoming inertia. Many processes are still manual simply because they always have been. In other cases, IT teams are still relying on undocumented scripts created by IT administrators who have left the organization. As the IT environment continues to evolve, it’s not likely those scripts will continue to scale.
One way or another, IT is becoming more automated. The irony is that after using IT to automate almost every business process imaginable, it’s only recently that IT teams are getting around to automating their own workflows. Many of the rote tasks that used to be performed by entry-level personnel have been steadily disappearing thanks to automation.
DevOps teams, of course, have always been ruthlessly committed to automating as many IT processes as possible in the name of efficiency. The rise of Ansible, however, makes it easier for every IT organization to achieve that same goal because they don’t have to wait for a specialist to create a playbook to automate a process. An Ansible playbook that can be easily customized or extended often already exists. The real issue may simply be knowing where to look for it.
-
Automation Challenges Holding DevOps Back
A survey of 500 IT and engineering leaders in the U.S. published today finds organizations build an average of 6.7 internal applications per year themselves in addition to investing in 7.7 separate platforms to build applications.
Commissioned by Internal, a provider of platforms for building application development environments, the survey also found organizations are spending 40% of their time building and maintaining these internal tools and workflows. While more than three-quarters (76%) of respondents said that this work is necessary, it distracts from building applications that drive business revenue. A total of 82% said a lack of true flexibility and customization is the top reason so much time is spent on building and managing internal tools and workflows.
A full 94% of respondents said they want a more collaborative platform that enables technical and non-technical users to build internal tools/workflows faster together. The survey also found 91% of respondents believe a platform that encompasses application development and workflow automation would save development time.
Internal CEO Arisa Amano said one thing that holds IT teams back is that managing the platforms used to build applications requires too much programming expertise. Internal is making a case for a graphical approach to building application development environments so IT administrators can easily construct them using a drag-and-drop interface on behalf of an application development team. That approach makes it possible to allocate more programming resources to building applications, she said.
More organizations than ever are revisiting how applications are built as the number of projects increases despite an uncertain global economy, noted Amano. Organizations of all sizes are reluctant to hire additional full-time engineers even as digital transformation initiatives demand a growing number of custom applications, she added. The only way to effectively address those conflicting goals is to adopt a platform that automates much of the workflow associated with building an application development environment, said Amano.
It’s not clear how willing DevOps teams are to replace the platforms they use today to build those environments, but many organizations are still relying on bespoke platforms they built and continue to maintain. Many of those platforms are made up of a collection of tools that DevOps teams cobbled together over the years. However, many of the software engineers that initially built those integrations may have since moved on and the documentation left behind is likely to be scant, noted Amano.
Most DevOps teams in 2023 will continue investing in higher automation levels to accelerate application development. The quandary they all face is whether to continue to invest in the platforms they have already or migrate to a platform that makes it simpler to build and maintain an application environment using automated workflows built and maintained by a vendor. Regardless of what they choose, it’s certain the days when organizations could afford to throw engineers at every integration challenge have come to an end.
-
Cisco AppDynamics Survey Surfaces DevSecOps Challenges
A survey of 1,150 large enterprise IT professionals from the AppDynamics unit of Cisco suggested application security progress is being made slowly and steadily as organizations continue to embrace DevSecOps best practices.
The survey finds that while more than three-quarters of respondents (76%) believe that DevSecOps is essential for organizations to effectively protect against a multi-staged security attack on the full application stack only 43% reported their organization has already started to implement DevSecOps workflows while another 46% said they are currently considering making the shift.
More than three-quarters of respondents (78%) also said a lack of a shared vision between application development and security teams presents a challenge to application security over the next 12 months. More than half (55%) said they currently considered security to be more of an inhibitor of innovation than an enabler. Nevertheless, 81% admitted a lack of application security skills and resources is now an issue for their organization, with 79% acknowledging that securing the full application stack is now a priority for their organization.
A full 92% admitted that the rush to rapidly innovate and respond to the changing needs of customers and users during the COVID-19 pandemic has come at the expense of robust application security. Well over half of respondents (58%) also noted their organization often ended up in ‘security limbo’ because they didn’t know what security issues to focus on and prioritize. A total of 88% acknowledged their organization could be doing more to secure the full stack of modern applications across the entire application life cycle.
Not surprisingly, 93% said it’s important to be able to contextualize security so that they can correlate risk in relation to, for example, application performance, end user experience and business metrics. However, more than two-thirds (68%) reported that their current security solutions work well in silos but not together. A total of 85% stated that cultural changes to support the shift to a DevSecOps approach are important to improve application security.
AppDynamics Executive CTO Gregg Ostrowski said it is apparent organizations are still wrestling with not just application security technologies but also people and process issues. The most critical issue is to knock down the silos that exist in organizations that often result in one team demonizing another, he added.
Overall, application security is becoming more challenging as the attack surface expands. A full 89% of technologists reported that their organization experienced an expansion in its attack surfaces over the last two years, with almost half (46%) noting that this expansion is already increasing cybersecurity challenges. One of the primary drives of that expansion as been the rise of the internet-of-things (IoT) applications (59%), rapid adoption of cloud computing platforms (56%) and digital business transformation initiatives (51%).
On the plus side, more than three-quarters of respondents (76%) said they believe artificial intelligence (AI) will play an increasingly important role in addressing application security challenges. The issue, of course, is how soon those advances might be made at a time when attacks against software supply chains are clearly on the rise.
-
Jellyfish Adds Tool to Visualize Software Development Workflows
Jellyfish today unveiled a tool that identifies bottlenecks in software engineering processes using data the company collects via Git repositories and project management software from Atlassian.
Krishna Kannan, head of product for Jellyfish, said Life Cycle Explorer provides managers with a tool that makes it simpler to identify the root cause of a process breakdown.
Life Cycle Explorer goes beyond providing simple top-level metrics to calculate the time consumed at each stage of a workflow for any given process and then visually map it to surface trends, noted Kannan.
That makes it possible to identify gaps, see where work areas overlap—indicating parallel processes—and drill into how outlier events may be impacting the overall life cycle of a development project.
The launch of Life Cycle Explorer comes at a time when there is more focus on the efficiency of DevOps teams than at any time in recent memory. During uncertain economic times, organizations of all sizes are looking to optimize DevOps workflows to ensure maximum efficiency because many organizations are hesitant to hire additional full-time engineers.
The important thing to remember when analyzing DevOps workflows is there is no such thing as the optimal “Goldilocks” workflow that can be universally applied, noted Kannan. Instead, most DevOps leaders will discover one team excels at one segment of a DevOps workflow while others are more efficient elsewhere. Life Cycle Explorer makes it easier to identify those best practices in a way that makes them simpler to replicate when appropriate, said Kannan.
Life Cycle Explorer follows the addition of a benchmarking tool and DevFinOps offering that addresses cost capitalization reporting for research and development. The core Jellyfish engineering management platform is designed to ingest data from the tools an organization already uses to build applications and then applies proprietary algorithms to create a comprehensive view of workflows across an organization. The challenge many organizations encounter today is they often lack visibility into workflows that have each been individually constructed by a diverse range of DevOps teams working on very different types of software projects.
At the very least, DevOps teams need to establish a baseline that surfaces the current rate at which software development occurs. Once that baseline is established it then becomes possible to measure the impact changes to software engineering practices are having. Today many organizations make educated guesses about what changes are needed without really understanding the impact they might be having across the entire software development life cycle.
Of course, it will be up to each organization to determine the degree to which they want to measure workflows. The Jellyfish platform makes it simpler to identify workflow issues, but there are also value stream management (VSM) platforms that track the impact software engineering workflows are having on specific business goals. Those VSM platforms are typically much more challenging to implement and maintain. The Jellyfish approach focuses more narrowly on improving the efficiency of DevOps teams.
Regardless of approach, things that are not measured are not being managed. At a time when organizations are more dependent on software than ever, the need to manage software development life cycle as efficiently as possible has become a lot more important.
-
How To Build Anti-Fragile Software Ecosystems
Just as organisms are susceptible to diseases and viruses, software systems are susceptible to hacks and errors. And within many complex interconnected systems, a minor bug could have a cascading effect across an entire organism, causing it to break. As such, increasing the resiliency of a software ecosystem is something engineers continually strive for. But how do we get there?
One concept to consider is anti-fragile. Being anti-fragile refers to the ability of a system to thrive in adversity and adapt to change. Specifically, with regard to software, anti-fragile can equate to responsive coding practices, such as auto-scaling or default backups, to mitigate unforeseen events. Chaos engineering, for example, is one method to expose edge cases and build with adaptability in mind. And with the introduction of intelligent automation, often fueled by machine learning (ML) and artificial intelligence (AI), it’s becoming more and more feasible to build anti-fragile software systems.
I recently met with Asim Razzaq, co-founder and CEO of Yotascale and former head of platform engineering at PayPal, to discover what it takes to build anti-fragile software. According to Razzaq, anti-fragile represents the next evolution of reactive, resilient software systems. And introducing anti-fragile concepts into your application networks can be done with just a few (relatively) simple steps. Below, we’ll define what anti-fragile is and see how this concept can be applied to software development and DevOps.
Understanding Anti-Fragile
As the adage goes, “What doesn’t kill you makes you stronger.” And it’s true—many natural systems tend to thrive under inhospitable conditions. As described by Nassim Nicholas Taleb in the book ‘Antifragile: Things That Gain from Disorder,’ anti-fragile systems thrive in adversity. For example, consider how human muscle grows under stress and load. Immune systems similarly adapt and respond to new viruses and improve the body’s resilience over time. So, how does this relate to software development?
Well, organizations need to develop highly-tolerant software systems to meet rising digital consumer expectations. Especially when under extreme loads, systems must be elastic to adapt and survive. To meet high user demands, as well as specific service-level objectives (SLOs) and agreements (SLAs), most software companies are already investing in cloud-native tools that improve reliability. For example, a hallmark of site reliability engineering (SRE) is introducing automation to enhance the stability of applications.
One example of gauging anti-fragile in software environments is chaos engineering, says Razzaq. Chaos engineering is the act of throwing a wrench into the machine. You randomly introduce bugs and malformed requests and attempt to create as much havoc as possible to see what happens. This could mean pulling out cables and restarting servers to gauge how software systems respond.
“Software that can respond the best adapts the best,” said Razzaq. For example, if you manually remove a few nodes from the cluster but the system knows it needs seven to function, it could automatically spin up additional nodes. Or if the whole cluster goes down, ideally, the system would automatically restart and scale to the necessary requirements. In essence, anti-fragile software systems can understand and adapt to new environments without much, or any, human input.
Tips to Construct Anti-Fragile Software Systems
So, how can we construct anti-fragile software systems — especially when real-world challenges are often unpredictable? Razzaq shared helpful advice to consider when crafting anti-fragile software architecture.
1. Start With Your Most Critical Systems First
First, introduce chaos engineering to your most absolute critical systems. Define these vital components at the heart of your product which must remain functional. For a payments processor SaaS, for example, this would be the infrastructure supporting its transactional engine, such as the login and payment microservices.
2. Determine How to Test Fragility
Once you’ve identified this critical infrastructure, figure out how to test for fragility in your environment. Introduce chaos engineering to the component to see how it behaves. For example, returning to our payment platform example, this may include switching off a payment transaction type to see how the system functions in an extreme scenario.
3. Decouple Components
If systems are too closely coupled, errors and outages can ripple across multiple software services. To avoid that, it’s a good practice to prevent tight coupling between components—especially between critical systems and non-critical ones.
4. Introduce Alternate Paths
One method to decouple is by introducing alternate paths. For example, if a credit card verification component goes offline, the system should naturally accept other forms of payment and automatically suggest another form, such as a bank transfer. If all systems are down, the system could still accept unverified payments in a queue and let this continue until the threshold reaches a certain risk level.
5. Learn From Past Events
The next frontier of designing anti-fragile architecture is developing self-learning, autonomous systems. Of course, there is a limit to how automation can respond to unforeseen issues. But by feeding a machine learning model with more and more experiences, it could deal with untested new scenarios more gracefully. Of course, AI has the potential to make harmful decisions and even introduce unintentional bias, so organizations should have accountability for the algorithms they oversee, cautioned Razzaq.
Benefits of Building With Anti-Fragile in Mind
The goal is to able to thrive and function even under duress. By identifying the elements of core infrastructure to keep alive and putting it through the wringer, you can discover frailties before they cause significant consequences. And by decoupling components and introducing alternate paths, software systems can be better equipped for whatever may come. Uplevelling system operations with increased automation and AI is likely the future of constructing more fault-tolerant applications.
“Software is everything in this day and age,” said Razzaq. “It increasingly plays a critical role in life-enhancing capabilities.” As such, the end benefits of developing an anti-fragile style could have hugely positive effects. In health care situations, it could make a difference in saving lives. For other less dire scenarios, increased fault tolerance leads to less lost revenue during peak traffic times and more consistent end-user experiences.