Tag: NVIDIA

  • Secure Governance and Scalable Management for AI Models

    Secure Governance and Scalable Management for AI Models

    At swampUP 2025, Yuval Fernbach, vice president and CTO of MLOps at JFrog, and Adel El Hallak, senior director of product for NVIDIA AI Enterprise, discuss how JFrog is integrating machine learning operations (MLOps) into its platform strategy. Fernbach, who joined JFrog through its acquisition of Qwak, outlined how JFrog is extending its mission beyond artifact management to embrace AI-driven software supply chain automation. The goal, he noted, is to enable developers to build, deploy, and govern AI models with the same level of trust and traceability that JFrog has long brought to binary and container management.

    They discussed the rapid convergence of MLOps and DevOps, a shift that is redefining how teams manage the entire software and model delivery lifecycle. As organizations accelerate their adoption of generative AI and large language models (LLMs), the challenge is no longer just building or training models—it’s governing and managing them at scale. AI introduces new forms of technical debt, compliance requirements, and operational risk that traditional software management frameworks were never designed to handle. Every model carries dependencies, training data, and inference patterns that must be tracked, versioned, and secured across environments. Without unified governance, enterprises risk exposing sensitive data, violating regulations, or deploying unverified models that behave unpredictably in production.

    The energy at swampUP reflects an industry in transition—where every team, from developers to platform engineers, is rethinking their toolchains for the AI era. As Fernbach and others shared, JFrog’s latest innovations are designed to bridge that gap, ensuring that software—and now models—move from experimentation to production with speed, security, and confidence.

  • Sketch Coding and the Rise of MCP in DevOps

    Sketch Coding and the Rise of MCP in DevOps

    At swampUP 2025, JFrog’s Yonatan Arbel reflects on his journey inside the company and how developer DNA continues to shape JFrog’s culture. Arbel, who has spent nearly a decade with JFrog, began as a software engineer before moving into leadership and eventually into a developer relations role. But as he emphasized, “once a developer, always a developer.” He still codes daily, keeping a hands-on perspective that influences how JFrog builds and communicates its platforms.

    That commitment to staying close to the keyboard led Arbel to create JFrog’s first experimental MCP (Model Context Protocol) server as an open source project. Uploaded to GitHub almost as a side experiment, the project quickly gained traction — even attracting pull requests from NVIDIA and references from Cisco. For Arbel, it was validation that MCP represents a real turning point in how enterprises will connect APIs, AI models, and DevOps workflows.

    He described his process as “sketch coding,” akin to vibe coding, where ideas start as rough AI-assisted prompts before maturing into production-ready tools. That approach mirrors how many developers are now blending AI into their workflows: iterating quickly, validating ideas, and only then investing in hardening code for scale.

    Arbel also highlighted the excitement — and occasional sense of magic — that comes from integrating AI into day-to-day development. Whether it’s pulling runtime insights from an IDE or rapidly prototyping new capabilities, the goal remains the same: help developers work smarter while ensuring JFrog’s platforms support security, governance, and scalability across the software supply chain.

    The conversation underscored how JFrog continues to bridge its engineering roots with the demands of modern DevOps, proving that innovation often starts with developers who never stop building.

  • Bringing Trust and Governance to AI-Driven DevOps

    Bringing Trust and Governance to AI-Driven DevOps

    At swampUP 2025 in Napa Valley, Alan Shimel sat down with three industry leaders whose companies sit at the heart of today’s AI-driven DevOps transformation: Rahul Tripathi, general vice president and GM of ITSM at ServiceNow; Justin Boitano, vice president of enterprise AI at NVIDIA; and Shlomi Ben Haim, co-founder and CEO of JFrog.

    Tripathi, only months into his role at ServiceNow, stressed the importance of grounding product strategy in lived practitioner experience. Having led DevOps teams himself, he emphasized that ITSM remains the “bread and butter” of ServiceNow, but AI integration is reshaping expectations for productivity and security across IT operations.

    Boitano reflected on NVIDIA’s long journey from pushing CUDA adoption in 2008—when many thought GPU computing was a stretch—to today, where GPUs and AI frameworks have become central to enterprise computing. He noted that partnerships with companies like ServiceNow and JFrog are critical to building out the ecosystem, particularly as organizations look to infuse AI into every layer of their development lifecycle.

    Ben Haim underscored the urgency behind what he called “AI FOMO.” With board-level pressure driving 40% of CIOs to expand AI budgets, enterprises are rapidly embedding AI into software supply chains. For JFrog, that means positioning itself as the “system of record” for DevOps, launching new platforms like JFrog Fly to simplify AI agent adoption while strengthening governance, traceability and security.

    Together, the panel painted a picture of an industry balancing excitement with pragmatism. AI agents may not replace developers anytime soon, but they are already changing workflows, tooling expectations, and the economics of software delivery. As the panelists made clear, success will depend less on hype and more on how well organizations align AI adoption with trust, verification, and long-term resilience.

  • StarCoder2 LLMs Designed Specifically to Generate Code Arrive

    StarCoder2 LLMs Designed Specifically to Generate Code Arrive

    ServiceNow, Hugging Face and NVIDIA have once again combined their efforts to advance a generative artificial intelligence (AI) platform that has been specifically trained to generate higher quality code than rival approaches based on general-purpose large language models (LLMs) such as ChatGPT.

    StarCoder2 is a family of LLMs created by the BigCode community led by ServiceNow. Three iterations are starting now with a 3 billion‑parameter model trained by ServiceNow; a 7 billion‑parameter model was trained by Hugging Face, while NVIDIA built a 15 billion‑parameter model.

    The 3 billion‑parameter model is capable of matching the performance of the original StarCoder 15 billion‑parameter model.

    Nicolas Chapados, vice president of research for ServiceNow, said as AI research continues to advance, each successive wave of LLMs becomes more efficient. Each organization, depending on the use case and total cost, can then opt to deploy StarCoder LLMs either on a local machine or in the cloud. The more parameters an LLM has, the more infrastructure resources are needed to run any inference engine based on it.

    The StarCoder2 LLMs were trained using 619 programming languages and can be extended to, for example, provide summarizations of code. The foundation of StarCoder2 is a new code dataset called The Stack v2 that is more than seven times larger than version 1 of the dataset. In addition, additional training has enabled the model to understand low‑resource programming languages such as COBOL, mathematics and program source code discussions. Organizations can also fine‑tune models with industry or organization‑specific data using open source tools such as the NVIDIA NeMo platform or Hugging Face transform reinforcement learning (TRL) techniques. StarCoder2 was built using data under license from the digital commons of Software Heritage hosted by Inria.

    The overall goal is to provide development teams with LLMs that, in addition to being less likely to hallucinate, also generates code with much fewer vulnerabilities because the code generated is based on exampled vetted by the BigCode community, said Chapados.

    It’s not clear right now how much of the code finding its way into production environments is being generated by machines versus humans. However, as LLMs continue to advance, the percentage created by machines will undoubtedly increase. The challenge DevOps teams now face is determining how to apply AI to pipelines that will increase in number as codebases not only proliferate but also become inevitably larger.

    In the meantime, DevOps teams would be well-advised to become familiar with what type of LLM is being used to train the generative AI platforms being used to create code. The more general purpose an LLM is, the more likely it is to generate suboptimal code that DevOps teams are only going to need to eventually replace, especially if that code is consuming expensive infrastructure resources inefficiently.

    There is, of course, as far as generative AI is concerned, no going back at this point. The issue now is determining how to keep track of the provenance of all the code being generated to determine how best to fix it whenever needed.

  • Microsoft kills Python 3.7 ¦ … and VBScript ¦ Exascaling ARM on Jupiter

    Microsoft kills Python 3.7 ¦ … and VBScript ¦ Exascaling ARM on Jupiter

    Welcome to The Long View—where we peruse the news of the week and strip it to the essentials. Let’s work out what really matters.

    This week: VS Code drops support for Python 3.7, Windows drops VBScript, and Europe plans the fastest ARM supercomputer. (more…)