devopsy.kinsta.cloud

Category: DevOps Culture

  • CircleCI Adds Release Orchestration Capability to CI/CD Platform

    CircleCI Adds Release Orchestration Capability to CI/CD Platform

    CircleCI today added a release orchestration capability to its namesake continuous integration/ continuous delivery (CI/CD) platform to give developers more control over application deployments.

    CircleCI CTO Rob Zuber said a CircleCI releases capability provides development teams with the ability to release a controlled subset of their code in a live production environment in a way that provides immediate feedback on how that code is performing. That level of visibility can then be used to determine whether that code might need to be rolled back using automation tools embedded in the CircleCI CI/CD platform, he added.

    In the absence of that capability, noted Zuber, it takes DevOps teams, on average, 45 minutes to an hour to manually roll back an application.

    Initial platforms supported by CircleCI releases are Kubernetes and Amazon SageMaker. CircleCI releases is also compatible with the open source Argo CD platform that CircleCI integrated its platform with last year.

    CircleCI also plans to add support for blue-green deployments later this year, enabling DevOps teams to deploy one application version more gradually to replace another.

    The way CD is managed often varies widely from one organization to another. In highly regulated industries, CD is typically managed by a central team to ensure compliance requirements are met. In other instances, developers are directly responsible for building and deploying their code. Regardless of approach, CircleCI enables organizations to define a set of guardrails to ensure best practices are observed, noted Zuber. In effect, each organization can implement a set of checks and balances that are aligned to their specific needs, he added.

    As DevOps continues to evolve, more organizations want the ability to control deployments at a more granular level by, for example, only making a new application or an update available to a select number of users. Once deployed, developers need a more holistic view into not just how that application is performing but also how it impacted the business, said Zuber. If, for example, the number of logins declines after an update, a developer needs the information to determine whether that update should be rolled back as quickly as possible, he added.

    It’s not clear how many organizations are managing the entire CI/CD workflow on an end-to-end basis. While most DevOps teams make extensive use of CI to build an application, the deployment of applications is still often managed manually. However, as CD capabilities become simpler to employ across platforms with standard interfaces, the number of organizations automating CD should steadily increase. In fact, that capability will soon prove essential as, thanks to the rise of generative artificial intelligence (AI), the pace at which applications are being built only continues to accelerate.

    In the meantime, each organization will need to determine how closely it wishes to integrate CI and CD workflows. However, the more tightly integrated these processes become, the simpler it becomes for developers to assume full responsibility for building and maintaining the software they create.

  • The IaC Weight on DevOps’ Shoulders

    The IaC Weight on DevOps’ Shoulders

    It’s often challenging for established tech companies to adopt modern DevOps practices around infrastructure-as-code (IaC). It’s not so much in adopting the syntax, but rather the DevOps culture that IaC is part of for engineering organizations.

    Organizations often struggle to transition from traditional operations models to collaborative DevOps culture, which is a byproduct of transitioning infrastructure operations to code.

    In many cases, legacy tendencies persist, while IaC duties stay siloed within ops-centric DevOps teams—which is the exact opposite of the benefits of leveraging powerful technology like infrastructure-as-code. What this ultimately creates is bottlenecks in development velocity and product iteration.

    With all this in mind, it begs the question: What changes can enable a smoother IaC journey?

    The Pitfalls of IaC Silos

    Having DevOps teams shoulder the entire IaC burden is the exact anti-pattern we’re looking to avoid when truly transitioning to DevOps culture in our organizations. What many organizations start to experience is these familiar pain points that start to emerge:

    1.) Knowledge Gaps Widen

    Developers—by nature of their day job as software engineers—often lack access, visibility and context into infrastructure changes. Yet, with more and more infrastructure requirements shifting left, they are left (pun intended) with insufficient capabilities to manage these growing needs.

    This is compounded when they are faced with minimal cloud permissions. Limited permissions make it difficult to experiment and familiarize themselves with cloud tools and capabilities the way they would with code and its parts of the stack.

    Meanwhile, DevOps teams—as the exact mirror image and the nature of their role in the organization—don’t grasp application needs and objectives. This often results in a widening chasm of broken communication and collaboration around IaC initiatives.

    2.) Collaboration Suffers

    A common tool that exacerbates the problem is a ticketing and support platform. Developers, as their modus operandi, lacking a shared understanding of the domain, will often resort to opening tickets to achieve their goals. This creates a culture of inefficient cross-team communication via ticketing systems.

    This doesn’t mean the ticketing platforms are the problem, it just results in inefficient communication methods, as developers don’t always know what questions to ask.

    What this does mean is that DevOps engineers operate largely on instinct, being mostly in the dark on the real application requirements that can’t all be communicated in a simple Jira ticket. This results in final solutions that rarely satisfy needs on the first attempt and take longer to deliver.

    3.) Releases Slow to a Crawl

    DevOps teams hold the responsibility for conducting and orchestrating release engineering, on top of provisioning environments, maintenance, monitoring and other tasks that fall in the DevOps engineering domain.

    This is why DevOps tasks can become a bottleneck, as they not only dictate deployment timing (due to important considerations of uptime and client disruption, among others) but are also tasked with juggling infrastructure changes across many and multiple environments.

    This causes unpredictable delays in provisioning dev environments and other developer-oriented tasks that frustrate developers and stall production updates, and delivery velocity.

    The Obstacles of Change

    Recognizing these inefficiencies and shifting these responsibilities left became the popular way for DevOps leaders to distribute IaC duties. This allowed developers to gain sufficient autonomy to choose the solutions for their needs, while DevOps teams provided guidance and oversight for how to practically achieve these solutions.

    However, like all things shift left, this comes with its own set of unique challenges. Adopting the DevOps paradigm faces fundamental obstacles that add further complications and complexity to these cultural transitions, including:

    • Uncontrolled sprawl
    • Ballooning costs
    • Security concerns

    On the one hand, distributing the IaC load lessens the burden on the DevOps teams, but the downside is that it becomes difficult to understand which resources are actually in use and which have been temporarily created for testing purposes.

    With many owners creating resources on demand, once they are no longer needed, these leftovers create confusion around dependencies and make cloud platforms disorganized and difficult to maintain.

    Just like enabling more hands to touch IaC creates greater sprawl and disorder, more users with less governance invite careless sprawl in terms of costs as well. This often results in duplicate and unused resources accumulating, wasting budgets that are currently tight, and every penny counts. With a lack of automation and oversight, environments grow messy and expensive.

    The sprawl issues can also impact security, as expanding permissions raises valid security concerns that are intensified when clouds become disorganized and difficult to maintain.

    Well-intentioned developers may misconfigure resources or expose sensitive systems, and without proper methods to manage drift or misconfiguration, this can pose real risks to organizations and systems. Another important aspect that also increases with less oversight is intentional insider risk.

    From Silos to Shared Responsibility

    OK, that must have been really disheartening—and you’re probably thinking you should run for the hills before attempting any transitions to shared responsibility.

    But wait—don’t panic yet. It is possible for teams to redistribute IaC responsibility while avoiding these pitfalls.

    Below are some key factors to help you make the transition smoother in the short and the long term:

    • Automate everything—If DevOps practices have taught us anything, it is that manual cloud environment maintenance doesn’t scale. Automation has been the backbone of much of the evolution we have undergone when it comes to infrastructure operations, and this has been powered by the programmability and everything-as-code mindset—from IaC to policy-as-code (PaC). This automation-based optimization makes it possible to reduce neglected resources through regular and automated analysis and response. What you get is cost savings and the prevention of runaway sprawl.
    • Empower through governance—Remember when we spoke about automation and governance? Well, programmatic policy enforcement is a good way to constrain unwanted resource proliferation without manual intervention. Governance comes in to handle auto-remediation of misconfigurations or other anti-patterns that do not align with automated policies, which simplifies environment management at scale. Importantly, governance, if done right, can be a tool to empower teams, break silos and encourage autonomy. Having the right guardrails in place is how you instill confidence and provide helpful direction.
    • RBAC for the win—Role-based access control (RBAC) limits permissions to appropriate needs, establishing important guardrails that can help prevent some of the issues we spoke about above, including sprawl and security concerns.

    RBAC will enable developers to gain the necessary freedom while minimizing security risk. Minimum required access encourages responsibility and constraint.

    Now that you have a solid grasp of the principles, it’s time to put them into action!

    Fortunately, there are numerous solutions at your disposal. You can jump in and build them yourself using capable tools like Jenkins or Atlantis.

    Alternatively, for a more comprehensive approach, there are excellent commercial IaC management tools on the market. These can assist you in reaching your objectives swiftly and with greater efficiency.

    An Evolutionary Vision for IaC and DevOps

    As technology complexity increases, no single team can handle infrastructure alone. Dev and Ops must unite abilities to shepherd modern systems and the underlying code powering them. This requires closing knowledge gaps through collaboration and communication

    With a thoughtful process, IaC automation, governance guardrails and the right access policies, tech leaders can guide this cultural transition smoothly without exposing the organization to unnecessary risk and spending.

    When done right, Shared IaC responsibility will boost velocity, agility, and autonomy—the trifecta of crucial traits that quantify and qualify elite engineering teams and differentiate companies in a competitive landscape.

    Interested in learning more about env0 to manage your IaC at scale and confidently?

    Stop by Booth L12 at KubeCon EU to speak with an expert or see a demo!

    To hear more about cloud-native topics, join the Cloud Native Computing Foundation, Techstrong Group and the entire cloud-native community in Paris, France at KubeCon+CloudNativeCon EU 2024 – March 19-22, 2024.

  • StarCoder2 LLMs Designed Specifically to Generate Code Arrive

    StarCoder2 LLMs Designed Specifically to Generate Code Arrive

    ServiceNow, Hugging Face and NVIDIA have once again combined their efforts to advance a generative artificial intelligence (AI) platform that has been specifically trained to generate higher quality code than rival approaches based on general-purpose large language models (LLMs) such as ChatGPT.

    StarCoder2 is a family of LLMs created by the BigCode community led by ServiceNow. Three iterations are starting now with a 3 billion‑parameter model trained by ServiceNow; a 7 billion‑parameter model was trained by Hugging Face, while NVIDIA built a 15 billion‑parameter model.

    The 3 billion‑parameter model is capable of matching the performance of the original StarCoder 15 billion‑parameter model.

    Nicolas Chapados, vice president of research for ServiceNow, said as AI research continues to advance, each successive wave of LLMs becomes more efficient. Each organization, depending on the use case and total cost, can then opt to deploy StarCoder LLMs either on a local machine or in the cloud. The more parameters an LLM has, the more infrastructure resources are needed to run any inference engine based on it.

    The StarCoder2 LLMs were trained using 619 programming languages and can be extended to, for example, provide summarizations of code. The foundation of StarCoder2 is a new code dataset called The Stack v2 that is more than seven times larger than version 1 of the dataset. In addition, additional training has enabled the model to understand low‑resource programming languages such as COBOL, mathematics and program source code discussions. Organizations can also fine‑tune models with industry or organization‑specific data using open source tools such as the NVIDIA NeMo platform or Hugging Face transform reinforcement learning (TRL) techniques. StarCoder2 was built using data under license from the digital commons of Software Heritage hosted by Inria.

    The overall goal is to provide development teams with LLMs that, in addition to being less likely to hallucinate, also generates code with much fewer vulnerabilities because the code generated is based on exampled vetted by the BigCode community, said Chapados.

    It’s not clear right now how much of the code finding its way into production environments is being generated by machines versus humans. However, as LLMs continue to advance, the percentage created by machines will undoubtedly increase. The challenge DevOps teams now face is determining how to apply AI to pipelines that will increase in number as codebases not only proliferate but also become inevitably larger.

    In the meantime, DevOps teams would be well-advised to become familiar with what type of LLM is being used to train the generative AI platforms being used to create code. The more general purpose an LLM is, the more likely it is to generate suboptimal code that DevOps teams are only going to need to eventually replace, especially if that code is consuming expensive infrastructure resources inefficiently.

    There is, of course, as far as generative AI is concerned, no going back at this point. The issue now is determining how to keep track of the provenance of all the code being generated to determine how best to fix it whenever needed.

  • Survey Sees Limited DevSecOps Progress Being Made as Vulnerabilities Mount

    Survey Sees Limited DevSecOps Progress Being Made as Vulnerabilities Mount

    A survey of 2,037 IT and security professionals conducted by the Cloud Security Alliance (CSA) on behalf of Dazz, a provider of a platform for mapping cloud computing environments, finds well over a third of respondents (38%) estimated that 21-40% of their code contains vulnerabilities, while 19% estimated 41-60% of their code contains vulnerabilities. Another 13% estimated they have vulnerabilities in 61-80% of their code.

    As a result, organizations on average have 55.5 security vulnerabilities each day in their remediation queue, with typically at least one being deemed critical, the survey finds. Overall, the average number of vulnerabilities that could be addressed is 1,025 per month, but organizations typically only address 270 within a month.

    In general, there are four phases to a remediation with each phase, on average, requiring between three to six hours so it easy to see how some organizations with limited resources can reach a point where they never catch up. The report noted that more than half of the vulnerabilities that have been remediated reoccur within a month, often because the root cause was never adequately addressed.

    Hillary Baron, senior technical director for research for CSA, said the high number of vulnerabilities that need to be regularly addressed suggested that progress in adopting DevSecOps best practices remains limited. While not every vulnerability is equally critical, it’s clear application development and cybersecurity teams are not yet proactively addressing application security issues, she noted.

    In fact, the survey found that less than a third (30%) of respondents reported there is a good working relationship between the application development teams that typically provision cloud computing environments and the cybersecurity teams that usually are held accountable for maintaining cloud security. Just under half of respondents (49%) said their vulnerability remediation teams comprised four to six members. Most respondents (60%) found it somewhat difficult to identify code owners, with an additional 24% facing moderate difficulty and 5% that found it highly challenging.

    Dazz CTO Tomer Schwartz said the more dynamic application development becomes, the bigger the DevSecOps challenge. Most organizations lacked the visibility required to proactively correlate their DevSecOps practices, he added. For example, more than three-quarters (77%) of respondents said they lacked full transparency into their cloud computing environments, and 6% had no visibility at all.

    In total, the survey finds organizations typically employ a variety of scanning and detection tools to safeguard their cloud environments, with 32% using five to six tools while another 29% report using three to four tools.

    More than three-quarters of respondents (76%) also noted that one in 10 of the alerts these tools generate is a false positive.

    Just under 75% of respondents also noted security teams spend more than 20% of their time performing manual tasks when addressing vulnerabilities, even though 83% reported using at least some automation in their remediation process. A full 61% also noted their organization is using three to six different detection tools, with nearly half (45%) considering increasing their security budgets in 2024, so it’s likely that many will be adding additional tools. In total, 22% dedicated less than 20% of their budget to cloud security, compared to 31% that allocated 21-40% of their budget and 27% that invest 41-60% of their budget to securing cloud environments.

    Unfortunately, less than a quarter (24%) said they feel very prepared for the cybersecurity threats their organization faces. Much of that uncertainty stems from knowing just how many vulnerabilities there really are in production environments and that they are being attacked with greater frequency each day.