devopsy.kinsta.cloud

Tag: workflow

  • Mobb Extends DevSecOps Reach of Tool to Generate Patches

    Mobb Extends DevSecOps Reach of Tool to Generate Patches

    Mobb today added an ability to instantly surface patches for vulnerabilities at the time when code is being committed during a DevSecOps workflow.

    Previously, the tool provided by Mobb to create validated patches based on the scan results of third-party SAST tools would only surface issues to individual developers.

    Mobb CEO Eitan Worcel said organizations can now address these issues within the context of DevSecOps workflow.

    The Mobb platform combines proprietary security research and traditional semantic analysis with generative artificial intelligence (AI) to automatically create patches for vulnerabilities. The overall goal is to make it simpler for DevSecOps teams to consistently ensure applications with known vulnerabilities are remediated before they are deployed, said Worcel.

    The Mobb approach eliminates the need for IT teams to sort through static application security testing (SAST) scans themselves. Each vulnerability typically requires at least 30 minutes to fix, so the Mobb platform improves application security while simultaneously making developers more productive.

    Historically, one of the reasons for the wide divide between application development teams and cybersecurity professionals is that many of the vulnerabilities discovered don’t actually impact applications running in production environments. Development teams then find themselves wasting time investigating vulnerabilities and, when they do determine a vulnerability is an issue, spending time developing a patch. The Mobb tool automatically creates the necessary patch, so there is less of a need to debate which vulnerabilities are worthy of the time required to create a patch.

    For decades, application development teams have been attempting to find a way to remediate vulnerabilities as quickly as possible. With increased focus on adopting DevSecOps best practices to address those vulnerabilities, more organizations are now reviewing software supply chains as part of a larger effort to improve application security.

    There will, of course, be vulnerabilities that the Mobb tool will not be able to address simply because there may not be enough affected applications to warrant pre-building a patch. Even with the most robust set of rules, it is improbable that an automated tool can, with 100% certainty, safely fix every issue. However, given the number of routine vulnerabilities present in applications, such as SQL injections, there’s plenty of opportunity to improve developer productivity by relying on patches that address common vulnerabilities.

    The process by which patches are prioritized, developed and then applied has, of course, long been a source of DevOps frustration. Anything that streamlines that process will go a long way toward improving the overall state of application security at a time when cybercriminals are becoming more adept at exploiting vulnerabilities. The challenge is finding a way to remediate vulnerabilities that developers will embrace rather than resist. The Mobb approach still provides them with control over what patches are applied but the toil involved with creating a patch in the first place has been substantially reduced.

    It’s not clear how long it might be before automated approaches to building patches might improve the overall state of application security but, as always, every journey needs to begin with a first step in the right general direction.

  • Atlassian Extends Scope of Jira Project Management Portfolio

    Atlassian Extends Scope of Jira Project Management Portfolio

    At the Atlassian Presents: Unleash event today, Atlassian announced it is adding a Jira Product Discovery application to its portfolio. Jira Product Discovery makes it easier for organizations to keep track of the ideas that drive software development initiatives.

    At the same time, Atlassian is making available Jira workflow and toolchain templates to make it simpler for software development teams to extend the scope and reach of the project management software they use to track those projects. Rather than each organization developing these templates, Atlassian is now providing them in a way that allows organizations to customize and extend them as they see fit.

    Finally, Atlassian also announced that an existing Jira Work Management tool will be made available for free through March 2024.

    Megan Cook, head of product for software teams at Atlassian, said these additions will make it easier for organizations to collaborate around software development projects that businesses increasingly depend on to drive revenue. The overall goal is to make it easier for multiple stakeholders of the projects to more easily collaborate with software developers that rely on Jira to manage those projects, she added.

    These latest offerings are part of an ongoing Atlassian effort to enable organizations to simplify collaboration and enable organizations to embrace value stream management (VSM) without having to acquire a dedicated platform. In some cases, organizations may decide to adopt a dedicated VSM platform, but a project management application should make it simpler for everyone in an organization to keep track of how software development projects are impacting the business, noted Cook.

    It’s not clear just how many businesses are aligning their internal processes around DevOps workflows, but as more of them realize how dependent they are on software, there is a clear need to collaborate more efficiently with software development teams. In fact, many organizations are now trying to adjust to the rapid rate at which software is being delivered thanks to the adoption of DevOps best practices. In some instances, updates to software are now being delivered faster than many businesses can absorb given the impact those changes have on digital processes.

    The irony is many business executives used to routinely complain about how long it took IT organizations to adjust as business conditions changed. Now, many businesses are rolling out additional digital capabilities via software updates that can be made multiple times a month.

    The rate at which organizations will collaborate across departments will naturally vary, but the days when each department was a fiefdom unto itself are coming to an end. There may even come a day when organizations will need to restructure as the walls that once separated various fiefdoms become more porous.

    One way or another, the way organizations are managed is evolving in the digital age. Organizations are employing software to pivot more frequently as business conditions evolve. Each new idea can not only be tracked but also quickly evaluated for feasibility given the level of software development resources required, noted Cook. The challenge and the opportunity is as much about changing the culture of an organization as it is about modernizing the processes used to build and deploy software.

  • The DevOps Workflow is Broken. Here’s How to Fix It

    The DevOps Workflow is Broken. Here’s How to Fix It

    While great progress has been made in bridging the gap between developers and operations engineers, fundamental differences in roles and responsibilities, alignment, and access to information hinder further progress. 

    Everyone is familiar with the DevOps infinity loop: This is an ideal workflow where every stage of the software development life cycle feeds into the next. While the philosophy is well understood, organizations have worked on bringing developers and SREs closer together under the DevOps banner. The reality is most of the time, the DevOps infinity loop is broken. Developers continue to focus on building features without the benefit of data-driven insights that production data can provide. Developers often “push and pray” without fully understanding how their changes will affect performance and the user experience. As a result, companies miss the opportunity to improve products faster and delight the customer sooner.

    According to Gartner, 90% of DevOps initiatives will fail to fully meet expectations. And yet, the global DevOps market, which accounted for $7.3 billion in 2021, is expected to reach $37 billion by 2030. So to summarize, we’re all spending billions of dollars on the implementation of initiatives that are expected to fail in some meaningful way 90% of the time. The question is: What is failing and what can be done about it? 

    As the DevOps toolchain continues to expand, collaboration across tools is difficult and success depends on your organization’s ability to simplify and automate while achieving alignment. Developers use a variety of tools from various vendors in their daily work. The most widely used are VS Code from Microsoft, Jira from Atlassian and Slack from Salesforce. The most popular code hosts/repositories include GitHub, GitLab and Bitbucket. Bridging the gap are CI/CD vendors such as Jenkins and CircleCI. Ops engineers typically rely on other vendors for their daily work, such as New Relic, Datadog and Dynatrace. So, how do we ensure that all stakeholders in the SDLC have access to all the relevant information simultaneously, understand the objectives and get an integrated perspective of the different inputs and workstreams independently of the tools used?

    There are three essential areas that DevOps initiatives must tackle to create alignment, collaborate successfully and streamline workflows. Let’s tackle each of these areas individually.

    1. Collaboration. The first step is good collaboration. Most organizations rely on Slack or Microsoft Teams for this purpose, but they are both suboptimal when it comes to collaborating on code and incidents that point to code. Having code and data at the center of DevOps collaboration ensures that everyone has access to the information needed to tackle issues, reducing and even eliminating differences of opinion in the process.
    2. Visibility. Second, ops engineers and developers must have visibility into the performance of applications across the entire tech stack. These three barriers can occur in different aspects of an organization’s operations and their impact can limit the team’s ability to deliver value to the customer just as much as workflow inefficiencies. Visibility helps the organization reach alignment on common objectives.
    3. Context Switching. Third, given the number of processes and tools involved in a successful DevOps implementation, it’s important to work toward reducing context switching and integrating tools within established workflows. Integrating collaboration, tracking, history and observability into a single pane of glass can greatly improve clarity and time to resolution.

    For DevOps teams, increased velocity and transparency are always worthwhile pursuits. However, it’s important to avoid myopic focus on workflows when other nagging problems could be hiding in plain sight.

    By committing production telemetry, developer teams can adopt an objective, data-driven approach to software engineering—saving time and avoiding potentially misguided approaches based on opinions rather than facts. Visibility, transparency and velocity can help engineering teams meet ever-increasing requirements and provide their organization with the best possible business outcomes.

  • Red Hat Extends Reach of Ansible Automation Framework

    Red Hat Extends Reach of Ansible Automation Framework

    Red Hat this week launched a collection of downloadable workflows for automating IT processes that will be made available via the Ansible Automation Hub.

    At the same time, Red Hat has updated the Red Hat Ansible Automation Platform to add an automation services catalog to keep track of workflows and a return on investment (ROI) calculator that enables IT teams to determine which workflows are the most valuable based on usage.

    Red Hat also updated the core Red Hat Ansible Tower automation engine to streamline processes and reduce bottlenecks.

    Tom Anderson, vice president for the Red Hat Ansible Automation Platform at Red Hat, said Red Hat Certified Ansible Content Collections includes workflows from Red Hat and vetted third-party providers. As such, IT teams employing Ansible to automate the management of IT can have confidence these playbooks and workflows are both secure and supported, he said.

    Red Hat decided to separate the management of Ansible workflow content from the underlying platform because it’s become apparent new workflows are being made available at a faster cadence than updates to the core Red Hat Ansible Automation Platform, he noted.

    As part of that effort, Red Hat this week is also making available additional automated workflows that leverage Red Hat Tower to automate tasks and jobs via a centralized console, along with workflows that leverage Red Hat Satellite systems management software and the Red Hat Insights monitoring service.

    Anderson said interest in IT automation has expanded significantly in the wake of the economic downturn brought on by the COVID-19 pandemic. Many organizations have either frozen IT headcount or been forced to reduce the size of their IT staff. However, managing IT environments continues to grow more complex as emerging technologies such as microservices and Kubernetes clusters gain momentum.

    Some organizations may, of course, opt to put a hold on adopting new platforms, but just as many are continuing to invest in cloud-native technologies necessary to drive digital business transformation initiatives, which require applications that are both more flexible and resilient. Because of that, Anderson said, the urgency surrounding IT automation project has increased.

    In fact, he noted, Red Hat has created “office hours” during which the company provides IT teams with access to Red Hat personnel at specific times to provide additional guidance.

    As an open source project, Ansible is gaining traction as a management framework because it provides a declarative approach to automating IT that doesn’t require IT operations teams to learn how to code. Ansible is also being embraced by IT infrastructure providers including Cisco Systems and Dell Technologies, which makes it easier for IT teams to create playbooks that automate tasks across multiple platforms. Over time, those cross-platform capabilities foster the creation of an ecosystem that further increases Ansible momentum, Anderson noted.

    There’s no doubt IT teams were increasing their reliance on IT automation frameworks prior to the pandemic. Now, however, automating the management of IT is not just about increasing productivity and efficiency; in many cases, it’s become a matter of survival.

  • Trifacta Extends Data Preparation to DataOps with New Functionality for Data Engineers

    New features including RapidTarget and Automator allow data engineers to more efficiently operationalize data preparation workflows in production

    SAN FRANCISCO, 14th of November 2018 — Trifacta, the global leader in data preparation, today announced a new set of capabilities designed to enable data engineers within data operations (DataOps) practices to more efficiently develop, test, schedule and monitor data preparation pipelines in production. With RapidTarget, Trifacta customers can utilize an existing data model to intelligently guide transformations and accelerate the process of generating a new  output dataset that matches the predefined schema. Automator provides end-to-end management of scheduling, monitoring and refining preparation workflows. These new features, along with the new Deployment Manager framework, facilitate the work of data engineers in administering data pipelines that feed analytics, machine learning and data science initiatives.

    “Today’s organizations are increasingly adopting DataOps to more efficiently scale their analytics and data management practices ,” said Wei Zheng, VP of Products at Trifacta. “Data engineers are at the center of this shift, playing a critical role in the data preparation practices that form the backbone of their data efforts. Our team has extensively studied the emerging role of data engineers within our customers’ organizations and focused these latest enhancements on improving their ability to transition data prep workflows into scalable, repeatable data pipelines. Our introduction of these new capabilities demonstrates the expanding role Trifacta is playing within the data management practices of our customers.”

    DataOps is clearly on the rise, with 73 percent of organizations indicating they planned to invest in the methodology in 2018. In light of this growing adoption of DataOps practices, the role of the data engineer in configuring, managing and scaling data pipelines has risen as a critical position within data management. Trifacta, through its work with large-scale enterprises including New York Life, GlaxoSmithKline and Deutsche Boerse, has seen first-hand how data engineers are a driving force behind the maturation of data preparation from an ad-hoc activity to a standard enterprise data management process. These new features in the Trifacta platform further support the role that data engineers perform in promoting data preparation flows developed by analyst end users into scalable data pipelines that deliver value to the broader organization.

    “Data engineering is a critical function at Malwarebytes—ultimately, we’re the ones responsible for the secure adoption of new analytics use cases across the organization,” said Manjunath Vasishta, Director of Data Science and Engineering, Malwarebytes. “Trifacta’s data preparation platform has been instrumental to that objective, allowing us to provide our business users with increased accessibility to data, while also maintaining governance. As data preparation usage continues to grow, the platform’s functionality provides data engineers with even further assurance to scale and operationalize data prep workflows in order to keep pace with the demands of the business.”

    The new features from Trifacta to further support data engineers and DataOps-focused organizations include:

    • RapidTarget: Allows data engineers to set a predefined schema target that provides automated guidance for how diverse data sources must be prepared and joined together in order to map to that target. Aligning the transformation process to an existing data model is critical when integrating unfamiliar or external data into an analysis. RapidTarget offers intelligent suggestions to accelerate how users prep and blend data sources in order to align their output to fit a desired data model.
    • Automator: Trifacta’s system to intelligently manage the scaling, scheduling and monitoring of data prep workflows in production. Users can set the schedule for flows to automatically run in production at a given time, when data updates or programmatically using APIs. Upon scheduling, users can define parameters or variables to customize what data is input into a workflow and how outputs are published.  Finally, they can monitor the status and performance of jobs, and set alerts when input data changes or anomalies occur.
    • Deployment Manager: Provides a framework for testing, versioning and managing data preparation workflows as they transition into enterprise-wide data pipelines. Before a new workflow can bring value to the broader organization, the flow must be tested at scale. With Trifacta’s Deployment Manager, data engineers can seamlessly test new workflows across development, testing and staging environments prior to being transitioned into production pipelines. The framework also allows data engineers to manage the versioning of flows and even rollback to prior versions of flows if needed.

    According to a recent Gartner report, “Providing the right data to the right people at the right time is a constant challenge. Data engineers are emerging as a crucial role in addressing this challenge. Data and analytics leaders must therefore develop a data engineering discipline as part of their data management strategy.”*

    Learn more about how data engineers are using Trifacta as part of broader adoption of DataOps practices on Trifacta’s blog here.

    *Gartner, Data Engineering Is Critical to Driving Data and Analytics Success, Roxane Edjlali, Nick Heudecker, Ehtisham Zaidi, 4 October 2018

    Gartner Disclaimer

    Gartner does not endorse any vendor, product or service depicted in its research publications, and does not advise technology users to select only those vendors with the highest ratings or other designation. Gartner research publications consist of the opinions of Gartner’s research organization and should not be construed as statements of fact. Gartner disclaims all warranties, expressed or implied, with respect to this research, including any warranties of merchantability or fitness for a particular purpose.

    About Trifacta

    Trifacta is the global leader in data preparation. Trifacta leverages decades of innovative research in human-computer interaction, scalable data management and machine learning to make the process of preparing data faster and more intuitive. Around the globe, tens of thousands of users at more than 8,000 companies, including leading brands like Deutsche Boerse, Google, Kaiser Permanente, New York Life and PepsiCo, are unlocking the potential of their data with Trifacta’s market-leading data preparation solutions. Learn more at trifacta.com.

  • Continuous Delivery vs. Continuous Deployment

    Continuous Delivery vs. Continuous Deployment

    I am often asked, “if we are using a continuous integration (CI) server and adopting a continuous delivery pipeline, where does continuous deployment fit in?” Good question. In a nutshell, continuous delivery engines such as CloudBees or Blue Ocean are designed to trigger the execution of CI workflows across the dev, test and prod environments. Continuous deployment is called as part of the CI workflow to do the actual release of code out to endpoints. Continuous deployment can be managed by application release automation tools or supported by one-off scripts. Regardless of how sophisticated the deployment step of your CI workflow is, it is still continuous deployment.

    To understand both continuous delivery and continuous deployment, one must go back to their roots, continuous integration. CI is a key component of Agile practice.  Agile developers understand the importance of adopting a methodology that can manage smaller incremental software changes to release software, and therefore business value, to end users quicker and cheaper.  The activities of deploying to testing and production historically have been in the hands of testing and data center teams, separate from Development. But in the Agile practice, these activities cannot be separated from development. The concept of separate development, testing and production teams is a waterfall concept, not an Agile concept. In Agile, everyone is on the same team. This means that testing and production teams must become part of the Agile process.

    Mastering Agile’s last Mile

    Ultimately, the goal of continuous delivery is to accelerate release cycles and get new code out to production fast. However, for most companies this is not happening. Developers and testers may have embraced continuous delivery, but when it comes to production, the code is moved to a staging area where it languishes until production is ready to perform the update. The last mile of Agile is not achieved. And we are back to where we started: waterfall.

    There are many reasons for the production roadblock. Lets’ review the top four.

    • Agents: Both open-source and commercial CI servers are agent-based. An agent based system works fine for developers and can be tolerated by testing. However, production teams are not that excited about installing yet another agent on to the production end points. For this reason, production team use their own method of deploying code. Therefore, the deployment scripts used for releasing code to dev and test are not used by production. The way they see it: Production does not want a deployment process that requires them to re-configure their data center environments with agent from a development tool.
    • Scripts: Production teams want input and visibility. One-off scripts obfuscate the deployment process. For this reason, production teams will write their own scripts. Scripts simply do not have the intelligence needed to meet the audit, rollback and transparency requirements needed by production teams. Production teams perform many steps manually. This takes time and cannot be done in an agile cadence. The result: Great code languishes in staging areas. Production stays several releases behind what continuous test has approved.
    • Cost, cost, cost: Most of what developers create in a continuous delivery workflow comes from open-source or consumption-based licensing models (think Jenkins and Jira). Tools that can achieve continuous deployment such as application release automation solutions are far to expensive for one development team to acquire and incorporate into the CI workflow. These types of tools are sold at the CIO level and cost upwards of $250,000. Production teams are often hesitant to purchases these tools, as the high cost equals high risk. And ‘one ring to rule them all’ does not work. This type of operational tool acquisition is waterfall-based and does not fit into the leaner Agile DevOps tooling. The result: One-off scripts continue to be used and dev and test. Production does not adopt agile.
    • Sustainability: During waterfall, a production release was performed as infrequent as two times per year, or at most once per quarter (considered aggressive). In other words, production is staffed to support a production release on an infrequent basis. With agile, the demand for production releases increases 10-fold. Instead of doing at most four releases a year, they may be required to do four releases a month. Production cannot keep up the agile pace. They are staffed for waterfall.

    Automating Continuous Deployment: Key to Maturing Continuous Delivery

    Continuous delivery cannot be successful without continuous deployment. If you want to roll code out to end users faster, the deployment process itself must be modernized. Using the same one-off scripts that were used in the waterfall approach will not achieve your CD goals.

    Continuous deployment automation is performed by a new set of agile DevOps tools called application release automation, or ARA. ARA solutions are designed to support a model-driven software deployment that allows development, test and production teams to use the same process without the need for one-off deploy scripts. ARA adds all of the features and functions needed to support continuous delivery and data center deployment requirements. ARA is designed to fully perform the heavy lifting of software releases including application packaging, release versioning, database updates, server configuration management, calendaring, roll-forward, rollback, security access and auditing.

    ARA solutions are called by a templated CI workflow that is defined by the development team. The workflow is moved up the pipeline and adapts to the changes at the various pipeline stages (System Test, Integration Test, Pre-Production, Production, etc.)  Because the process is driven by the ARA tool and not a script, the changes are automatically addressed across the pipeline reducing the workload for all teams and supporting frequent changes that can be easily traced and audited down to the server inventory level, including the production environment.

    Developers, Get Your Hands on Deployment Automation Tool       

    To achieve a repeatable CD pipeline that supports production, developers must embrace the automation of continuous deployment at development. The dysfunction however is the outrageously high cost of ARA tools. Developers don’t buy expensive operational solutions. More ARA solutions must be offered on a consumption-based model or offered as open source with the ‘out of the box’ integrations into CI/CD. When considering the purchase of an ARA solution, look for agentless models that can be implemented one project team at a time. Embracing this level of corporate process change must be done project by project. The operational big bang approach does not work well in an agile practice. Not everyone can use the same solution. However,  if done correctly, there are big benefits.

    ARA is a process that strengthens the foundation of the continuous delivery pipeline and is core to the agile DevOps initiative.   It facilitates agile practices by allowing the testing teams and data center teams the tools needed to become more agile and accept the frequency of changes being pushed across the pipeline by Development. Ultimately, the goal is repeatability across the pipeline, pulling together Dev and Ops.

    About the Author / Tracy Ragan

    Tracy Ragan is CEO and Co-Founder of DeployHub. Tracy has extensive experience in the development and implementation of business applications. It was during her consulting experiences that she recognized the lack of build and release management procedures for the distributed platform that had long been considered standard on the mainframe and UNIX.  Tracy has worked with hundreds of development teams in implementing a team-centric standardized build to release process. Connect with her on LinkedIn.