devopsy.kinsta.cloud

Category: Features

  • AWS Unfurls Bevy of Automation Tools to Streamline DevOps

    AWS Unfurls Bevy of Automation Tools to Streamline DevOps

    At the AWS re:Invent conference, Amazon Web Services (AWS) added a bevy of tools to its portfolio intended to accelerate the pace of application development while simultaneously simplifying DevOps processes.

    AWS development tools unveiled this week include AWS Amplify Studio, a visual development environment that allows developers to create web application user interfaces (UIs) with minimal coding.

    Ken Exner, head of product for developer tools at AWS, said as an extension of an existing AWS Amplify Studio tool, this latest addition allows developers to customize application UIs at a higher level of abstraction using a library of components while enabling them to drop down to a lower level of coding to customize their application further whenever required. After the UI is designed, AWS Amplify Studio automatically generates the associated JavaScript or TypeScript code for the developer.

    In general, AWS is committed to improving developer productivity by providing, for example, automated reasoning tools that automate processes without locking devs into a layer of abstraction that, ultimately, may create a wall that blocks them from meeting customization requirements, he said.

    Other developer tools launched by AWS this week include updates to Amazon CodeGuru Reviewer that adds a secrets detector that uses machine learning algorithms to identify hardcoded secrets that can be secured using AWS Secrets Manager; a revamped Amazon Inspector vulnerability management tool that provides automated assessment scans, automated resource discovery, support for container-based workloads, improved risk scoring, integration with Amazon EventBridge and AWS Security Hub and the removal of the stand-alone Amazon Inspector scanning agent.

    AWS also revealed it has eliminated the need to manually pull images from the Amazon Elastic Container Registry (ECR) repositories by adding automated synchronization capabilities that keep private and public registries consistent with each other.

    In terms of IT operations tools, AWS added a managed database service to its Amazon Relational Database Service portfolio that allows IT teams to customize the underlying database and operating system using an RDS Custom option, a new table class for Amazon DynamoDB designed to reduce storage costs for infrequently accessed data and a service for RDS, dubbed Amazon DevOps Guru, that extends existing machine learning capabilities to automatically detect and diagnose performance bottlenecks and operational database issues.

    Jeff Carter, vice president for relational databases at AWS, said that as databases are increasingly managed by DevOps teams rather than traditional database administrators (DBAs), the cloud service provider is moving to reduce friction by automating as many manual tasks as possible. That’s critical at a time when IT teams are deploying a much wider range of purpose-built databases that are optimized for different classes of applications, he added. The overall goal is to rely more on automation to reduce friction across the IT environment, said Carter.

    Overall, there are two classes of AWS customers: The first are development teams that are increasingly relying on managed services provided by AWS to automate a wide range of DevOps processes. The second tend to be enterprise IT organizations that prefer to manage cloud services themselves.

    Mitch Ashley, a principal for Techstrong Research, an arm of the parent company of DevOps.com, noted that, in many cases, enterprise IT teams are managing DevOps workflows that span multiple clouds. As such, they tend to prefer tools and platforms that can be applied to both multiple clouds and on-premises IT environments. The more those organizations rely on managed services provided by a cloud service provider the more difficult it becomes to consistently manage DevOps workflows across multiple IT environments.

    It’s too early to say just how much IT organizations will rely on managed services going forward. However, at a time when more organizations are using more IT platforms than ever, it’s clear the DevOps management challenge increases with each new platform introduced into the IT environment.

  • Bridging the Developer Experience (DevX) Gap

    Bridging the Developer Experience (DevX) Gap

    They say our past experiences have an impact on how we view life and our general outlook. Routine behaviours and patterns become a program that we follow day in, day out. We stop using our minds to their fullest potential and become trapped in unquestioning, continual cycles of process and emotion.

    The same could be said of our experiences in the present—our worlds shaped by the environments around us. The people, practices, tools and culture. Only by challenging these accepted patterns can we start to think about changing our mindsets.

    DevOps is a mindset; a way of forming an agile relationship between development and operations teams to speed up delivery of software to support business operations and improve customer experience. By and large, it has been successful. Companies react to change more effectively; software is developed and deployed swiftly. It has become a necessity across most organizations.

    There are challenges, as we all know, however. The State of DevOps Report 2021 found that 83% of organizations are implementing DevOps practices, but most are still failing to scale adoption. It’s a situation that hasn’t changed for four years. Why? Among other things, the report states there are too many silos, not enough skilled people, a saturated market with too many tools—and a culture that prohibits scale.

    Given that the end goal of DevOps is to generate untrammelled innovation across an organization, getting the culture right is critical. Developers need an environment where they have the confidence to innovate and contribute back to their community, help grow the business, improve their skill sets and further their careers without being limited by the people, practices and tools around them.

    Above all, it’s about creating the best possible developer experience (DevX) in which to thrive.

    The Developer Experience Gap

    Stephen O’Grady at Red Monk, the developer-focused analyst firm, originally wrote about a DevX (developer experience) gap in this post back in October 2020.

    Stephen likened the experience you get from an iPhone to the expectations of developers. The iPhone’s closed SDK created an environment that allows all developers to work in a consistent way. Functions are tightly integrated so that, for example, it would be easy to ensure the music volume lowers when you receive a call. A basic function, but one that we expect to work seamlessly with any app we install.

    Since then, developers have been served a vast array of tools and services with more and more software arriving on a daily basis. The challenge is that, where the phone and music player teams were controlled in a way that would make them work well together, modern development infrastructure doesn’t have that capability. In most cases, the toolchain is a “patchwork quilt” of products and services from different suppliers. And it is up to developers to build and maintain that quilt.

    This means that developers are being forced to take time away from writing code—the reason why they are employed in the first place—to dedicate valuable time and resources toward understanding and managing issues associated with highly complex, multi-cloud toolchains that are often being held together by nothing more than sticky tape and string.

    In other words, the very same market that is telling developers they can have anything they want to do their jobs more efficiently is simultaneously telling them that piecing the whole thing together is their problem. This is the gap for developers: It’s a gap between expectations and current reality. And a bridge across that gap needs to be built for organizations to get the most out of DevOps.

    Delivering the DevX Factor

    Attracting the best people is about showing that the experience they will have in your organization is both rewarding and liberating, so expecting developers to spend the bulk of their time not developing is clearly not an optimal experience. Creating better DevX will not only influence your ability to attract new people but enhance your ability to hold on to, empower and upskill the teams you already have.

    Where to start? Well, contrary to popular belief, you don’t need an army of DevOps engineers to reach your goals. Instead, you need to empower your existing teams to reach their full DevOps potential, supporting their progress with the right tools and training for their experience and role.

    As with any work culture (remote or hybrid), the importance of knowing where to find what you need and how to collaborate with other teams has never been so important. Tooling is a good place to start. DevX is affected by the tools they use, so organizing them in a way that helps with the culture is a step forward. In a multi-cloud, multi-vendor environment, there isn’t going to be one tool that does it all so the key is to allow teams to use the tools they love and automate as much as possible to remove the repetitive tasks.

    Automation is also a critical element in the race to reduce time-to-market and improve quality where concepts such as shift left are adding complexity and increasing the responsibilities piled on developers. Having access to testing automation in a consistent way will improve the overall experience for DevOps teams.

    And finally, think about starting your DevX journey by using a self-service platform, where information, tooling and automation can be centralized and made available to developers on demand. For overworked specialists, access to a platform that connects to any tool and creates an on demand environment that automates much of the repetitive work will create more time for them to do what they were originally employed to do: Develop.

  • Will Automation Fill Gaps Left by the ‘Great Resignation’?

    Will Automation Fill Gaps Left by the ‘Great Resignation’?

    Between burnout, job-hopping and the ongoing pandemic, labor shortages will surely get worse before they get better, and automation is increasingly sought after as a way to mitigate the ripple effects of understaffed IT departments.

    For DevOps, automation has several benefits, from freeing up employees to work on more constructive tasks to minimizing software bugs and technical debt.

    However, starting off slow and learning how automation will help address staff shortages without complicating processes is a critical factor.

    “There’s still a lot of teams that are struggling with far too many manual or repetitive tasks,” said Rich Lane, chief strategy officer at Netenrich, a digital IT and security operations company. “Obviously, trying to do your day job of delivering features and functionality into the wild can be challenging with the speed that business needs demand.”

    Lane, a former senior research analyst at Forrester Research, pointed out that slowing down in the present to do foundational work that can lead to higher levels of automation in the future can be hard to sell to digital services stakeholders.

    “And yet, without doing so, it becomes increasingly difficult to meet commitments in the event a team becomes shorthanded or delivery requirements increase,” he said.

    Generalist or Specialist?

    Cody Michaels, application security consultant at nVisium, said when it comes to hiring people for a DevOps environment, you have one of two options: You can hire someone with a wide range of knowledge that spans the entire process or you can hire someone with expertise in one core area like programming or deployment.

    “At an individual level, a generalist will be better than an expert, however, a team of experts will outperform a team of generalists,” he said. “Not only can automation fill in gaps between an expert’s area of influence, but it can also serve as a permanent fixture on the team. As a result, the turnover rate won’t affect any automation so long as it’s implemented properly.”

    From Lane’s perspective, the main areas DevOps teams should be looking to automate are continuous integration and continuous delivery (CI/CD), IaC and AIOps-enabled incident management platforms.

    “By taking the manual nature of day-to-day work off of DevOps engineers’ plates, they are freed to focus on digital transformation,” he said. “The number-one stumbling block is not starting with process.”

    Lane noted unless you understand all the steps in a procedure that you’re trying to automate, it is very difficult to maximize the power of automation tools.

    “Much of the process that is still adhered to today is outdated for the digital age,” he said. “Spend the time up front to map out what you hope to achieve with an automation project, what all the touchpoints are and how one can measure the quality of automation when it’s implemented.”

    Michaels added that while the internet is flooded by companies shouting they have the “best” tools, that proclamation of “best” is going to be determined by budget and known languages.

    “The top tools that I would recommend are Jenkins, Docker, Git and Jira, since these four play very well together with regard to automation,” he said.

    Git, for example, has hooks that can trigger upon performing certain actions, like freshly submitted code, while Jenkins has automation as its middle name and it’s used in building, testing and deploying, which in turn facilitates CI/CD.

    Meanwhile, Docker provides virtualization of operating systems and running applications in a secure manner in conjunction with containers.

    “Lastly, Jira is one most don’t think about, but the platform has a lot of macros and filtering functionality that can update the work process automatically from events like code passing a test before it’s deployed, for instance,” he said.

    Michaels said a couple of core considerations organizations need to make when approaching the idea of automation and AI are time as well as safety nets in place.

    “Automation will take time to set up before you can benefit from it, but in the case of smaller teams, it won’t be too time-consuming to configure it all in one fell swoop,” he said. “But if you’re working with a lot of technical debt, unfortunately, it’s going to be payback time. If this is the case, my best advice would be to start small and not overhaul everything at once.”

    Minor Automation Improvements Have Major Benefits

    He said he thinks incremental improvement will be the best route for most organizations, and that even minor improvements will compound far faster than waiting a year to see if a redesign will work for you.

    “As I mentioned previously, it’s imperative to have safety nets in place,” Michaels said. “Make sure you have a contingency plan if something goes wrong in the process to recover back to a working state. However, strictly speaking, everyone should make it a practice to have a recovery plan in their software development life cycle (SDLC).”

    As the DevOps world moves into 2022, organizations choosing to embrace automation could find themselves in a win-win situation—for both developers and operations.

    “The more automation takes over the ‘housekeeping’ items of the process, the more time will be freed up to improve the process as a whole,” Michaels said. “As a result, taking time to go back and clean up technical debt can now be set aside between sprints and maintenance.”

    He pointed out that senior roles can tackle optimizing the architecture, which will ensure fewer bugs and a healthier work-life balance for all: Just imagine a world where developers can focus on perfecting their craft—instead of waiting hours or days for server reboots or disrupting their flow to update a ticket status.

    “It really comes down to the fact that organizations can no longer make automation an afterthought,” Lane added.

    He pointed out that given the difficulties in recruiting and retaining staff today, DevOps leaders must make their teams as efficient as possible—and automation can help achieve that goal.

    “The number-one way to achieve this is applying as much intelligent automation throughout the IT environment as possible so that the highly-skilled engineers can focus on transforming the business,” Lane said.

  • GraphQL as a Meta Layer

    GraphQL as a Meta Layer

    There’s a lot of interest in GraphQL lately. The query language is excellent for data retrieval, and it could act as a more usable mechanism to evolve the traditional REST API integration approach. But as adoption grows steadily, architects are now considering the implications of managing GraphQL at scale.

    Certain issues may hinder GraphQL in practice. To review, security concerns require a nuanced approach to authorization. This is because it wraps multiple data retrieval points into a single request. Without native caching, performance could also suffer. Changeability is also required to continually update the graph, which may ruffle feathers with those seasoned in the world of REST where a guiding tenet is to build for longevity using web-compliant methods.

    According to Jens Neuse, CTO, WunderGraph, there is a simple way to mitigate most of these issues and harmoniously work with various API styles. To Neuse, the core problem is that GraphQL is not meant to be exposed over the internet. Instead, he advocates for interacting through a gateway. Such an intermediary would enable more implicit calls and proper authentication and authorization for specific underlying functions and resources. Using GraphQL as an internal integration layer could also help combine disparate APIs into a similar language, exposing GraphQL-like usability for frontend developers while allowing them to hit any API style on the backend.

    Below, we’ll consider the drawbacks of exposing GraphQL directly and see how an intermediary layer could benefit development. We’ll also uncover what open source software is on the horizon that could power this setup.

    Why GraphQL Shouldn’t be Exposed Publicly

    Facebook, the creator of GraphQL, uses an internal GraphQL API to power all of its apps across Android, iOS and web browsers. (This is not to be confused with their public Graph API). Interestingly, if we look at Facebook’s use of GraphQL, we see these endpoints are not exposed to the public web at all.

    It’s true. Open up Developer Tools in Chrome, and you’ll see there is not a single mention of a root HTTP endpoint like https://api.graphql/. Functions name GraphQL, sure, but they call it more implicitly. Other companies, like Twitter and Airbnb, use a similar abstracted approach to interact with a private GraphQL.

    In fact, most GraphQL APIs are not public. APIs.guru only charts 35 public GraphQL APIs—notable services include GitHub’s GraphQL API, Shopify Storefront and GitLab. This number is minuscule compared to the total number of public APIs, at over 24,000 according to ProgrammableWeb.

    A Layer In Between

    If we treat it as Facebook intended (and how they use it in production today), it could reverse the issues above, said Neuse. “These problems come down to our core belief that GraphQL is not supposed to be exposed over the internet,” he said.

    But if clients do not access a GraphQL endpoint, what are they interacting with? According to Neuse, the ideal architecture involves a layer between the client and the server. Such a layer would validate queries and represent verbose calls with simple functions. The process would involve a handshake as follows:

    1. Take query, store it on a server.
    2. Validate it; make sure it’s not malicious.
    3. Send back a handle to the client.
    4. Then, execute the query.

    Such a process could also enact universal policies on a per-function basis, thus solving potential authorization issues when fetching data from multiple backend services with a single query.

    “The issue is that using the query language comes with a cost,” Neuse said. “Not exposing GraphQL directly but ‘compiling’ it to a JSON-RPC API when deploying solves these problems.”

    Thus, the scalability issue isn’t so much an issue of size but of access. “GraphQL can be very valuable, even in small environments,” said Neuse.

    The Next Evolution of a Virtual Graph

    Another potential benefit of using GraphQL as an intermediary is combining APIs; a “virtual graph,” as Neuse calls it. This virtual graph offers a common integration language to interact with various API styles, whether REST, GraphQL, gRPC or webhook, in a GraphQL-like manner.

    Consider the ease with which package managers like npm enable developers to instantly install dependencies. For example, installing a package like React.js is as simple as initiating the command npm install react within Terminal.

    On the other hand, the API integration is very cumbersome. Users must read through documentation, create an account on a third-party site and download an SDK to call the service in their preferred language. So, why isn’t looping in dependencies like APIs and cloud services as simple as other common dependencies?

    According to Neuse, one reason is that the REST paradigm is obsessed with the resource constraint—aside from HATEOAS, it lacks a concept of relationships between data and fields. A virtual graph could enable a straightforward process similar to package managers, said Neuse. Interestingly, the WunderGraph team plans to open source a project that allows just that ability.

    GraphQL as a Meta Layer

    Many folks assume that GraphQL and REST are mutually exclusive. However, as we’ve seen, there are ways to use them in combination. Plus, with an abstraction layer, the REST versus GraphQL comparison becomes just an underlying technical nuance.

    “If you compare GraphQL to other specifications, like OpenAPI spec, you see GraphQL has the fastest-growing community,” said Neuse. As of 2021, JSON Schema is the most popular, used by 75% of developers, followed by Swagger 2.0 (54%), OpenAPI 3.0 (40%) and GraphQL (31%) according to the Postman State of API Report. While this number may be small now, 35% of respondents also said they plan to use GraphQL in the coming year.

    “It’s a very, very interesting movement,” said Neuse. What began at Facebook as a framework for data fetching for React apps has now expanded into something much bigger. Within this second new movement, largely driven by companies like Apollo and Hasura, GraphQL becomes the new data layer, he said.

    “Still, everybody is looking at GraphQL thinking they must migrate from REST—this is the wrong direction,” said Neuse. Instead of migrating all existing REST APIs to GraphQL, he predicts the industry will coalesce on using GraphQL as the integration language for querying REST APIs. A meta layer, if you will.

    Of course, meta has been a bit co-opted lately with Facebook’s rebranding …

  • Fugue Marries Compliance-as-Code Tool to AWS Well-Architected Framework

    Fugue Marries Compliance-as-Code Tool to AWS Well-Architected Framework

    Fugue has infused the best practices defined by Amazon Web Services (AWS) within its software-as-a-service (SaaS) offering for provisioning infrastructure using its infrastructure-as-code (IaC) platform.

    Fugue CEO Josh Stella said that IT teams can now evaluate templates for provisioning AWS infrastructure that were created using AWS CloudFormation or Terraform tools to ensure they comply with the AWS Well-Architected Framework.

    The AWS Well-Architected Framework spans five pillars that AWS has defined for designing and operating reliable, secure, efficient and cost-effective systems in the cloud. AWS has defined a set of best practices for each pillar and rewards organizations for following those by making available additional cloud credits. The goal is to improve overall cloud security by providing guidance that ultimately reduces the number of misconfigurations so that cybercriminals can’t discover and exploit them.

    The Fugue platform complements that effort because it is based on Open Policy Agent (OPA), a general-purpose engine for managing compliance-as-code that is being advanced under the auspices of the Cloud Native Computing Foundation (CNCF). Fugue extended OPA by creating Regula, an open source tool that evaluates IaC files for potential security and compliance violations. Fugue then created a SaaS platform that makes extensive use of the AWS Lambda serverless computing platform to make it easier to enforce compliance policies.

    Those policies include turnkey coverage for specifications such as SOC 2, NIST 800-53, GDPR, PCI, HIPAA, ISO 27001, CSA CCM, CIS Controls, CIS Docker and CIS Foundations Benchmarks for AWS, Microsoft Azure, Google Cloud and Kubernetes.

    Stella said Fugue has now incorporated the technical aspects of the AWS Well-Architected Framework. However, it’s still up to each IT team to navigate the cultural aspects of the shared responsibility model of compliance and security that cloud service providers like AWS follow.

    Unfortunately, developers that employ IaC tools to provision cloud infrastructure often make some faulty assumptions about the level of security maintained by the cloud service provider. The result is often a raft of security and compliance issues that arise mainly because most developers don’t have a lot of domain knowledge expertise in either area.

    It’s not clear how many organizations have adopted the AWS Well-Architected Framework, but as various tools make it easier to implement these best practices defined by AWS the more secure and efficient cloud computing environments should become. In fact, in many cases, those tools are being embedded within a larger set of DevSecOps best practices that are being implemented across multiple clouds.

    Stella said he doubted there will ever be any standardization when it comes to security and compliance frameworks that could be applied to multiple clouds; however, it’s clear that cloud service providers are at least making available frameworks that borrow concepts from one another.

    Regardless of the approach, the overall state of cloud security and compliance will continue to improve as more guardrails are automatically implemented. The issue then becomes not only finding a way to ensure that cloud infrastructure is secure when deployed and also determining how much of the infrastructure being relied on today is less secure than it should be.

  • WhiteSource Report Finds NPM Vulnerabilities Fixed Fast

    WhiteSource Report Finds NPM Vulnerabilities Fixed Fast

    WhiteSource today published a report that found most of the vulnerabilities that affect node package managers (NPMs), widely employed to deploy JavaScript applications, are addressed long before they are assigned a Common Vulnerabilities and Exposure (CVE) in the National Vulnerability Database (NVD).

    The report, based on an analysis of the vulnerabilities that WhiteSource tracks in its own vulnerability database and revealed at the AWS re:Invent conference, found more than 90% of vulnerabilities in NPM packages are fixed before the vulnerability is published in the NVD.

    Susan St. Clair, director of product management for WhiteSource, said that fact suggests that, overall, the open source community is staying on top of vulnerabilities as they are disclosed rather than waiting for them to be assigned a CVE number.

    The WhiteSource report comes on the heels of a breach of UA-parser.js, a widely employed NPM based on a JavaScript library that detects browser, engine, OS, CPU and device type/model from User-Agent data. Unknown malicious actors found a way to inject malware into the NPM that modified the library. That vulnerability has since been remediated.

    In fact, the WhiteSource report noted 83% of the CVEs assigned to the most widely used versions of vulnerable NPMs could be remediated before the publication of the CVE.

    In the wake of a series of high-profile breaches of software supply chains, there is a lot more focus on how open source software is actually secured. Commercial software products have a distribution list that they typically notify of a security patch. Open source updates, on the other hand, need to be tracked by developers that use that software. The level of attention individual developers might pay to any number of open source updates can vary widely.

    Many developers are also hesitant to immediately update their applications because they are not sure what impact that additional code may have on their application, said St. Clair. For example, their application may have dependencies that are no longer backward compatible with the updated code, she noted.

    Developers also need to determine the degree to which that vulnerability may affect their applications because they often don’t use every component of an open source library, so the fix being offered may not be required.

    Open source project maintainers often depend on their community to help uncover vulnerabilities and create the fixes as quickly as possible. When a vulnerability is detected in an open source library, the more common practice is to report it to the NVD even after the fix for that issue has been made available.

    In general, St. Clair noted the most important thing for the open source community to remember is to remain proactive when it comes to tracking vulnerabilities and then implementing DevSecOps best practices. The one thing that is certain is that a wide range of malicious actors also are tracking those vulnerabilities closely. It’s not uncommon to see a wave of exploits being created once a vulnerability is disclosed via the NVD.

    The challenge, of course, is that the more open source software that is employed, the more difficult it becomes for individual developers to stay on top of every vulnerability in an application portfolio that only continues to grow. On the plus side, vendors have pledged more than $10 million to help better secure open source software. Hopefully, the return on that investment will become apparent sooner than later.

  • WTH? We Wanna WFH | DoD Dual-Sources JWCC | More Nvidia ARM Woes

    WTH? We Wanna WFH | DoD Dual-Sources JWCC | More Nvidia ARM Woes

    Welcome to The Long View—where we peruse the news of the week and strip it to the essentials. Let’s work out what really matters.

    This week: Working from home is de rigueur, JEDI redux, and more about Nvidia/Arm.

    Survey Says: Devs Stay Out of Office

    First up this week: GitHub’s annual survey of developers has some fascinating data on the workplace (ahem) “new normal.” The pandemic has radically changed devs’ expectations of where they can do their jobs.

    Analysis: WFH is SOP

    As with other roles, Dev people want to continue to work from home. But does this pose a threat to your DevOps culture? Make sure you can do Ops with equal flexibility.

    Paul Krill: Dev productivity is back to pre-pandemic levels

    GitHub’s “2021 State of the Octoverse” research … sees signs that work rhythm is returning to pre-pandemic levels. GitHub also found that 46 percent of developers who worked co-located with teammates now expect to work fully remotely or in a hybrid environment.
    …
    Based on data culled from more than four million repositories and surveys of more than 12,000 developers … JavaScript and Python remain the top languages, followed by Java and TypeScript. [And] GitHub found that development team performance can increase as much as 87 percent when reusing code and as much as 43 percent when using … CI/CD.


    Zeroing in on the WFH angle, it’s Richard Speed:

    Respondents who were in the office either full or part-time dropped from 42 percent before the pandemic to a mere 10.7 percent now. … That’s a lot of empty desks.
    …
    Fans of the ctrl-c, ctrl-v combo will be delighted to note that GitHub found that when code reuse was made “frictionless” at work, developer’s performance increased by up to 87 percent. … It’s all splendid stuff.


    But beware of statistical fallacies, thinks Henrik Ingo—@h_ingo:

    #WorkFromHome may have been more common than anyone realized. [But] is the population of GitHub users/respondents somehow skewed toward pioneers/early adopters? … This might be a case of correlation is not causation.

    Defense Dept. Orders New Clouds

    The DoD issued its latest cloud contract opportunity notice, with the snappy title of “Joint Warfighting Cloud Capability” (JWCC). Reading between the lines of the notice, it all sounds very DevOps-cum-Agile.

    The Pentagon has backed away from its previous insistence on a single supplier, and now wants at least two cloud providers.

    Analysis: JEDI redux, because AWS wants a do-over

    JWCC is the follow-on from the aborted JEDI procurement. Amazon Web Services was “disappointed” to have been passed over in favor of Microsoft. Now it appears the DoD wants them to share.

    What can you learn from such a multi-vendor DevOps strategy?

    Jordan Novet: Pentagon asks Amazon, Google, Microsoft and Oracle for bids

    Amazon and Microsoft were the finalists for a single Joint Enterprise Defense Infrastructure. … Microsoft won it in 2019, Amazon filed a protest and the Pentagon canceled the contract in July.
    …
    The new effort [is] known as Joint Warfighting Cloud Capability. … JWCC differs from JEDI because it allows the Pentagon to rely on multiple cloud providers.
    …
    The U.S. General Services Administration … said only two U.S. cloud infrastructure providers, Amazon and Microsoft, appear able to comply with all of the Pentagon’s requirements. … The value of the new contracts is not known, but the Defense Department estimates it could run into the multiple billions of dollars.


    In summary, heed this Anonymous Coward:

    Competing suppliers will try harder than one non-competing supplier. The DoD has had a lot of problems with One-Ring-to-rule-them-all systems like the F-35. … JEDI would have been the same.
    …
    Smaller steps with upgrades while testing/using offers better returns. Done right they can get a better deal by making the companies compete against each other in making sure the products are actually useful.


    But look carefully at the DoD contract opportunity, says bustinbrains:

    They’ll all get contracts. Then DoD will simply use the one that they actually like the best. No one can complain because it’s open-ended with no guarantees.

    Arm/Nvidia Deal on FTC’s Radar

    Last week, we talked about Nvidia’s goal to acquire Arm Ltd from SoftBank, and the regulatory challenges it poses—notably in the UK, home of the chip architect. After we went to press, the other shoe dropped.

    Analysis: Just Give Up Already

    As we said last week, Nvidia can’t be allowed to do this. Not only are ARM chips found in 99% of smartphones, but they’re an increasing fixture in the datacenter—especially places that value “performance per Watt.”

    Richard Waters: What’s next for the tumultuous takeover of Arm?

    Nvidia’s acquisition of UK chip design company Arm from SoftBank has provoked serious opposition on both sides of the Atlantic. … The US Federal Trade Commission had its own worries. China, meanwhile, is waiting in the wings.
    …
    The deal has shone a spotlight on the unique position Arm plays in the chip industry. [Nvidia] offered a guarantee that it won’t block any other companies from licensing Arm’s designs. The offer has fallen on deaf ears, with both the EU and UK ruling it inadequate. … Post-Brexit politics have also come into play.
    …
    At what point might Nvidia and SoftBank call it a day? … A stock market listing [is] the most likely alternative, with the UK a favoured venue.


    How’s it viewed from the inside? Here’s a hint from fivemack:

    I was at ARM when Softbank bought them; it came as a surprise to everybody, because we were absolutely confident that we were unbuyable because anyone interested in us would want to use differential licensing as a weapon against their competitors, and we … were confident that regulators wouldn’t accept that.
    …
    Lots of people went home with a year’s pay check in free money. And not a few people … went home with a decade’s pay check in free money.


    More and more DevOps workloads are running on ARM silicon. Here’s Biophoton:

    The low power expertise of ARM has to be an advantage in the data centre (DC) market. … Plus Linux is available on ARM so architectural lock-in of Intel could be eroded in the DC market.

    The Moral of the Story: You are known by the company you keep


    You have been reading The Long View by Richi Jennings. You can contact him at @RiCHi or tlv@richi.uk.

    Image: Lukas Mann (via Unsplash)

  • Digital Transformation Driving VSM Adoption

    Digital Transformation Driving VSM Adoption

    The number of organizations using value stream management (VSM) to bridge the divide between DevOps and the rest of the business is expected to dramatically increase over the next two years. The shift is due, in part, to investments in digital business transformation initiatives.

    A survey of more than 500 executives, IT and business leaders conducted by Dimensional Research on behalf of Broadcom, a VSM platform provider, finds the percentage of organizations employing VSM will rise from 42% today to 70% in the next two years. Nearly three-quarters of organizations (72%) that are not currently using VSM plan to do so in the next 18 months or more.

    The survey also finds that 85% of organizations are collecting some subset of value stream metrics today as part of an effort to deliver solutions faster (56%), improve decision making (55%) and increase transparency (54%).

    Laureen Knudsen, chief transformation officer at Broadcom, said that as organizations embrace DevOps best practices more broadly to drive digital business transformation initiatives, they need more visibility into the application development process. VSM surfaces the insights required to not only identify bottlenecks but also track how changes in application delivery timetables will impact key performance indicators (KPIs).

    VSM traces its lineage back to lean manufacturing methods, which called for each step of a manufacturing process to be continuously measured. As software development has evolved from being a craft to an automated process using DevOps best practices—and as organizations realize just how dependent they are on software—an appreciation for the value of monitoring the impact of missed software development deadlines on the business is growing steadily. VSM platforms automatically collect technical metrics from DevOps tools that are then correlated against the KPIs the organization decides to track.

    The Broadcom survey found more than 96% of respondents’ companies are already in the process of adopting digital transformation. Strategic objectives include increasing efficiencies (70%), improving product quality (67%) and improving customer value (63%).

    Most respondents (92%), however, are anticipating product development challenges for 2022. The biggest product development challenge cited is a lack of visibility throughout the product life cycle (45%) followed by inefficient processes (44%), lack of resources (42%) and enterprise-wide lack of understanding of how products deliver customer value (42%). A full 89% of the business leaders surveyed also noted they continue to struggle with organizational silos between strategic development and operations; 93% noted that better alignment between teams would improve their organization’s overall return on investment (ROI) in software development. A total of 95% said they believed VSM would optimize their product life cycle processes.

    Knudsen said that, in time, artificial intelligence (AI) technologies will increasingly be infused into VSM platforms to enable automated feedback loops between the business and DevOps teams. In the meantime, DevOps teams should be prepared to adjust project priorities based on the needs of the business and to endure a higher level of scrutiny as business stakeholders become more involved in setting the software development agenda.

  • Some Change is Bad

    Some Change is Bad

    Some time toward the end of the year, I start considering the volume of change we, as practitioners, have been under for … literally years, even decades. And I start to try and think of ways to keep you all from drowning in change. And then I write something that I hope helps you all focus on good change, and keeps moving your environment forward.

    This year I’m going to say “consider the reasons for the change.” I will attempt to find examples that do not require mentioning a specific product or vendor, so apologies if these read a bit convoluted as I attempt to illustrate this point without castigating.

    There are three basic reasons that a market or marketers want you to consider a change. There are, of course, variants and cases where it is a bit of this reason and a bit of that, but most demands that we change what/how we are doing things can be lumped into “mostly one of these”. They are:

    1. A great idea that is really a game-changer for some subset of the market.
    2. A shifting of who is in control.
    3. Locking customers in or expanding revenue.

    All of these happen at the vendor/project level, and all of them happen at the market level.

    Number one is best illustrated by any of the waves of server virtualization that have come through. From virtual machines to containers, each has solved problems and changed the way we do business. Notably, I’m only talking about server virtualization efforts here. Most of the other virtualization attempts have really not fit the “great idea that is a game-changer” category, though some have limited usefulness. Personally, I think the new version of storage virtualization will be in the game-changer category but am waiting to see it in use.

    Number two is one to watch out for and is most obvious when big-name OSS projects are forked. It is rarely about the technology being implemented (after all, a fork is a copy of the original; at first it is not at all about the technology), but rather about who is calling the shots. There is a bit of this in every public cloud vendor implementing their own version of container management (though this example is more about lock-in, in my opinion).

    Number three is multi-pronged. If your vendor went from selling you hardware or software to selling you a service, they want that recurring revenue and they want you to be linked into that service so deep that you can’t walk away. The much-reviled 1990s mantra of “embrace and extend” is the poster child for this kind of change. There might be a value-add in there somewhere and sometimes the added functionality makes the move a no-brainer. Often, though, the change is simply to the vendor’s advantage, not their customers. As most major Linux distros have shown, open source is not immune to this one, as most of them are making OSS users the de facto beta testers for their paid offerings.

    None of this is to say that the change in front of you is bad—even if the reason it was originally spawned is bad. It is reminding you that, after a constant stream of change, we’re more open to it than maybe we should be and you should remember the only important thing about new changes—do they make our environment more secure, faster, more agile? Everything else is just noise. Yes, all the cool kids are switching to coding in [insert flavor du jour], but does it offer your organization anything new? Yes, it seems everyone is using [insert current CI/CD flavor], but does it address the weaknesses of your current toolchain?

    You all are nailing it. Keep doing it. And don’t be afraid of change, but make sure you know if that new OSS database system is really new, or just a lot of installing the same code under a slightly different name. Because someone didn’t like the new owners.

  • WhiteSource Adds SBOM Tool That Lists Vulnerabilities

    WhiteSource Adds SBOM Tool That Lists Vulnerabilities

    WhiteSource has added a software bill of materials (SBOM) tool to its portfolio that, in addition to capturing the components of an application, also surfaces vulnerabilities that should be addressed.

    Many organizations are becoming more rigorous about making sure SBOM are attached to every software development initiative in the wake of an executive order issued by the Biden administration that requires federal agencies to employ them. While not every organization builds applications for federal agencies, the fact that the federal government is requiring them in the wake of a series of high-profile breaches has led other organizations to review their software supply chain processes.

    An SBOM facilitates those reviews by making available a formal, machine-readable inventory of software components and dependencies that can be used to track their supply chain relationships and identify dependencies and hierarchical relationships.

    Ori Bach, executive vice president for product at WhiteSource, said WhiteSource SBOM takes the concept a step further by also identifying open source libraries. It will also automatically update the SBOM when changes are made to make it easier to identify unintentional or malicious content being installed during application builds.

    Finally, WhiteSource SBOM also provides remediation guidance that ensures updates won’t break the build.

    Bach said it’s already apparent cybercriminals have significantly increased their effort to compromise supply chains as part of an effort to compromise as many downstream applications as possible. In many cases, cybercriminals are now specifically targeting open source projects that create software that tends to be employed across a wide range of use cases.

    An SBOM in and of itself doesn’t make applications more secure. However, it can provide visibility into the components of an application that can then be used to inform DevSecOps best practices, which organizations are implementing more broadly as responsibility for application security continues to shift left toward developers.

    There are two primary specifications that organizations can use to create an SBOM. The Software Package Data Exchange (SPDX) specification is an open standard for communicating SBOM information including components, licenses, copyrights, and security references. Defined by the Linux Foundation, SPDX is now recognized as the ISO/IEC 5962:2021 international standard. The second is the Software Identification (SWID) format that is today widely used by publishers of commercial software.

    Regardless of the format used to construct the SBOM, Bach said there’s an opportunity to more closely align the increased visibility it enables with the application security goals of the organization building the application.

    The primary reason SBOMs are not as widely used as they might be by now has as much to do with inertia as it does with technology. However, as organizations start to expect an SBOM to be made available, it’s clear application developers and builders will need to find a way to automate the SBOM process as applications are continuously updated. The challenge, of course, is that applications are now being updated with greater frequency—thanks mainly to widespread adoption of DevOps best practices—that now need to be integrated with any tool for creating and maintaining an SBOM.