devopsy.kinsta.cloud

Category: DevOps in the Cloud

  • Perforce Charts Testing Course After BlazeMeter Acquisition

    Perforce Charts Testing Course After BlazeMeter Acquisition

    Perforce Software has acquired BlazeMeter from Broadcom as part of an effort to extend the scope of its testing tools beyond the Perfecto tools for testing mobile and web applications it acquired in 2018.

    Broadcom gained control over BlazeMeter after it acquired CA Technologies in 2018. CA Technologies acquired BlazeMeter in 2016. Terms of the latest acquisition involving BlazeMeter were not disclosed.

    Tim Russell, chief product officer for Perforce, said BlazeMeter adds a complete platform that spans everything from load and performance testing to application programming interface (API) testing. The goal is to enable DevOps teams to push continuous testing further left as part of an effort to accelerate the development and deployment of applications, added Russell.

    The need for continuous testing has become more pronounced as more organizations launch multiple digital business transformation initiatives, noted Russell. The real challenge will be finding ways to embed continuous testing within DevOps processes at a sufficient level of scale, added Russell.

    In fact, the quality of the digital experience those applications provide ultimately determines the degree to which those efforts are likely to succeed, which Russell noted is why so many enterprise IT organizations are reengineering their application testing processes.

    In general, organizations are taking two paths to shift testing further left. Some are giving application performance testing tools to developers to employ during DevOps sprints while others are trying to bring application testers into the application development process earlier. In some cases, organizations are doing both as part of a comprehensive effort to discover issues before applications are deployed in a production environment. Those issue are naturally more expensive to fix after an application has been deployed.

    Perforce, for now, expects to continue to make BlazeMeter and Perfecto available as a set of complementary cloud-based services with some cross-pollination of capabilities occurring over time, said Russell.

    In the longer term, it’s apparent that more artificial intelligence (AI)-based analytics will be embedded within cloud-based platforms that can collect enough data to build meaningful models of development processes. IT teams should also expect to see testing tools make greater use of computer vision algorithms to surface issues.

    One away or another, the entire application testing process is about to become a lot more automated as more organizations are simultaneously building multiple applications. That may not necessarily reduce the need for either a developer or tester to create and manage that process, but it does mean the amount of testing time required for each application should continue to steadily decline in the months and years ahead.

    Hopefully, all that effort will lead to better application experiences being provided to end users more consistently. There is nothing quite as frustrating for any application development team than to discover that 90% of the functionality of an application is not being used because end users are not able to easily surface it. Worse yet, of course, is when an application turns out to be too slow once it is deployed. In either case, it’s difficult to regain that initial end-user enthusiasm for any new application experience after it’s been lost, simply because no one thought through all the use cases that should have been tested much earlier in the application development life cycle.

  • 4 Container Orchestration Security Concerns

    4 Container Orchestration Security Concerns

    Container orchestration is a must-have technical skill, according to 51% of respondents to the 2021 Upskilling survey from DevOps Institute. So, it’s no surprise, then, that container technology is becoming increasingly popular. However, container environments are far more complex and dynamic than other infrastructure, making container security essential. Security ensures containers are running as intended, protecting the infrastructure, software supply chain, runtime and everything in between. 

    If you’re looking to use container-based applications within your software infrastructure, you may want to keep in mind the security risks that come with this technology. To help, I asked DevOps Institute Ambassadors to weigh in on the security concerns they have with container orchestration. Here’s what they shared: 

    Anshul Lalit, head of technology and transformation, Kongsberg Digital

    “One of the most common security concerns with container orchestration is that it shares the host operating system and the host kernel. Thus, it raises the concern that a host OS or kernel vulnerability could potentially be taken advantage of and exploited in a container. An example of this is the recent attacks on Linux’s mmap syscall. Though you can avoid this by using defined best practices, it’s worth noting that any compromise in the host OS could compromise containers in that host. This scenario is in contrast to a virtual machine (VM), in which the guest OS and the host OS and kernel share no common binaries and are thus inherently separated.

    Similarly, a rogue process, a vulnerable container image that is running, and a security vulnerability in the orchestrator can all provide an attacker a means to get their foot in the door. For example, if a container is designed to run with root privileges, an attacker running a vulnerability scanner on the orchestration host will quickly find the vulnerable container.

    Users should follow best practices for securing the containers running in a container orchestration platform by providing a Linux security module, for example, that can enforce a policy that will protect against a rogue container or user from extracting sensitive data from the host system.”

    Supratip Banerjee, solutions architect, Principal Global Services

    “Containers are creating new challenges for cloud security. While these structures are dynamic and automated, new threat vectors are also present for precisely the same reasons containers are useful. DevOps and SecOps teams must be aware and prepared for these possible issues:

    • Containers, unlike virtual machines, share the host operating system on which they execute. If settings are not correctly maintained, the host and containers may be exposed to security concerns.
    • Orchestration automation adds a degree of complexity which can lead to misconfigurations that overprovision access and, as a result, an expanded attack surface.
    • Containers are constructed using pictures kept in public or private repositories; they are frequently reliant on other images, and a single flaw in any of them might spread across thousands of containers.
    • Because containers are ephemeral, IP address-based security measures are ineffective and complicate forensic investigations.”

    Parveen Arora, co-founder and director, VVnT SeQuor

    “If your project(s) involves the use of containers, you will have decomposed your application into its various constituent services, which, while beneficial, isn’t necessarily as secure as if you are using VMs. Since container platforms like Docker borrow code libraries from the host, they’re fundamentally not as secure as VMs. Vulnerabilities in a container environment can therefore allow malicious software to break out of a container and infect the host environment.”

    Vishnu Vasudevan, head of product engineering and development, Opsera

    “The security challenge, especially when it comes to container orchestration, is people think they can spin up their own image or they can download the image from the cloud and start using it. It’s not as easy as everybody thinks. There are vulnerabilities being introduced every minute. You need to have a security tool and processes that can automatically scan for every container that is being splintered. Furthermore, a team needs to create shared practices around blue-green deployments or rollbacks of the container changes that are being pushed into production. 

    From the pandemic alone, we have seen significant cloud movement, but on average, 50 vulnerabilities are introduced every day. In fact, there are people just waiting for loopholes where they can execute malicious scripts. And some people, especially people who are in mining, try to gain visibility into vulnerabilities within cloud accounts and penetrate them to run huge containers for their own cloud mining purposes. On average, it can take 280 days to even detect any vulnerabilities or a hack into the system. 

    Companies using containers must consider threat vulnerability management as part of their orchestration strategy. Do you think the team is experienced enough to scan 50 vulnerabilities a day and fix them? Probably not, since release velocity is shortening a release cycle to one or two weeks for modern application initiatives. This creates a lot of technical debt that can accumulate rapidly, which is why threat vulnerability tactics and processes must be considered when moving to the cloud.”

    Learn more about container orchestration and similar topics by registering for an upcoming SKILup Day. 

  • Global Next-Generation Software Engineering Conference

    Global Next-Generation Software Engineering Conference

    GNGSEC sounds the clarion call for software engineering change!

    The pandemic made 2020 the year where we see some of this decade’s most exciting digital technologies become commercially viable, and others finally go mainstream. Software is the core of digital transformation. In the digital era, the methods by which we produce software also need to be digitized. In the software engineering field, the adoption of AI, big data, cloud computing and cloud-native are accelerating the digitization for businesses.

    Global Next Generation Software Engineering Conference (GNSEC) focuses on a full range of software engineering technologies and is committed to defining the standard of the next generation of software engineering. The event will gather 1000+ Attendees and it will be live with virtual experience where senior software engineers, architects, and team leads connect, learn, and grow. Experts and developers from various backgrounds will be sharing their cutting-edge technical practical experience and cases.

    The majority of the attendees of this event will be from the Southeast Asia region (Singapore, Thailand, Indonesia, Philippines, Malaysia, Vietnam), India and China. Due to the current travel restriction, GNSEC 2021 will be virtually presented via Airmeet. It will offer attendees with virtual exhibitions, virtual conference, peer networking session, business matching session, virtual discussion group.

    Global Next Generation Software Engineering Conference (GNSEC) is co-presented by DAOPS Foundation and Escom Events. The event is part of digital innovation series events that features AI, DevOps, Cybersecurity and digital transformation. Previously the event was presented in Shanghai, Shenzhen, Beijing, California and Hong Kong.

    There will be more than 30 speakers from DBS Bank, ING, IBM, Daimler, Tencent, JD.COM, DevOps Institute, Gitlab, New Relic, SUSE and more to share ideas, experiences and case studies in software engineering.

    Register here to join​​ the event with us to rebuild software engineering.

  • Why an Embedded Analytics Maturity Framework is Essential

    Why an Embedded Analytics Maturity Framework is Essential

    Contextual analytics, or embedding analytics directly into a business application’s transactional environment and core workflows, is the most sophisticated stage of business intelligence (BI) currently available to organizations today. It is one level ahead of the traditional integration of analytics, which is a standalone capability presented as a separate dashboard or reporting module.

    Today, there is an increasing shift among organizations toward embedding transactional analytics into their enterprise applications. Because contextual analytics is the most advanced option in the market, there is also an increasing need for assessing our analytics readiness first.

    The fundamental difference of contextual analytics lies in how it offers analytics to the user to explore data or derive insight within their transactional environment rather than accessed as an external tool. This approach requires a deeper evaluation of the product’s analytical maturity level using a formal framework. Having such a framework can act as a deployment guide for your developers for every stage of embedded BI implementation, including contextual analytics.

    But first, what makes contextual analytics so different that it necessitates a readiness evaluation?

    Contextual Analytics Vs. Embedded Analytics

    With contextual analytics, dashboards, reports and charts are no longer separated but are part of the main user experience. This integral change necessitates a closer look at your data and integration capabilities to align what it brings with all aspects of your software, including existing analytics.

    Embedding these elements directly into core software workflows isn’t a replacement for any existing analytics, but is instead another big step toward making data and analytics a more seamless part of your end users’ everyday work.

    It should be stressed that contextual analytics accommodates various levels of implementation and business requirements. Analytics can be embedded lightly, such as in the form of basic in-line charts and graphs users can see as they explore your software, or embedded deeper to offer AI-assisted insights and real-time alerts that trigger action from users while they work. It can be integrated with an experienced BI vendor partner or in-house if you possess the skill set internally.

    Avoidable Pitfalls of an Improvised Approach

    The need to assess your application’s analytical maturity and readiness for embedding analytics directly into the transactional workflow comes down to avoiding unnecessary BI roadblocks. 

    For many organizations, BI and analytical capabilities are traditionally lower in priority than other aspects of the software. Intending to add BI features later is well-meaning and understandable, but leaving developers to add in features intermittently and leaving it to your users to demand them when needed creates an issue of visibility. Vendors typically lose sight of what their application currently is capable of analytically, or what development needs to make the addition of a new feature not only technically achievable but worth the time and cost investment.

    If you take the time to reassess your application’s analytical capability, your developers and leaders can better understand that your product could be at an earlier stage of readiness than previously believed, and prepare accordingly before seeking out a contextual or other BI solution.

    For example, approaching contextual analytics ad-hoc could mean you find an unexpected older dashboard module, a cost that could have been accounted for in a formal maturity assessment. 

    What it comes down to is this: Implementing a basic dashboard module or data export feature, completely separate product features, is fundamentally different from fully embedded analytics in your users’ core workflow experience. Pursuing a formal evaluation to guide the introduction of contextual analytics—or even a form of embedded analytics at an earlier stage—means your users and product receive the best type of analytical capability your business is ready for today.

    The Five Stages of Embedded Analytics Maturity

    Based on our experience, there are five possible stages of product development, as illustrated in the embedded analytics maturity curve below. As contextual analytics is the final, most advanced stage of an embedded analytics integration available to product owners, the need for a formal assessment of maturity is most important, should it be the target state.

    Accelerating the implementation of contextual analytics comes down to your skillset, which will allow you to identify and evaluate your state of readiness accurately and determine a BI solution’s flexibility. If you do not have the internal expertise, implementing contextual analytics is easier with the help of specialized BI vendors that can help you in your assessment, and offer a contextual analytics platform built around openness. We recommend you look for vendors and partners that allow for:

    • Advanced APIs that lower total integration time and ensure the need to rearchitect the software is minimal to zero.
    • Customized experiences to make your desired contextual linkages, such as showing a user a relevant embedded chart when they click on a table of a client, possible.
    • An open platform that allows for options for on-premises, cloud, multiple operating systems that guarantee long-term security, scalability and flexibility with your analytics.
    • Rapid adaptation to your product development roadmaps with automated processes and administrative tools to lower total cost of ownership
    • White-labelling to make the rebranding and blending of the analytical solution and your native application components seamless.

    Ultimately, having an awareness of your product’s readiness and taking the time to examine your application’s current state means you avoid costly integration mistakes, save time and ensure developers implement the best solution according to your exact needs now.

  • 4 Tips To Plug The Cloud IT Skills Gap

    4 Tips To Plug The Cloud IT Skills Gap

    Technical talent is few and far between. The programming world faces a developer shortage, and with this shortage comes a lack of cloud know-how. By now, most organizations have become multi-cloud, yet 86% of IT professionals believe a lack of these skills will slow down cloud projects.

    As more teams transition from older architectures to cloud environments, the IT skills gap could pose a severe bottleneck to software development and deployment. Cloud servers require ongoing maintenance, and managing security and compliance is challenging, even for seasoned developers. Furthermore, it’s easy to overspend in the cloud—without keen insight into vendor nuances, it becomes challenging to select the most cost-effective package for the application at hand.

    I recently talked with Joe Spurrier, chief technology officer and co-founder, cloudtamer.io, to see how teams can improve their cloud strategies while navigating a short supply of IT cloud know-how. According to Spurrier, increasing automation, sharing reusable templates, centralizing best practices and maintaining agnosticism are strategies to amplify cloud results. Below, we’ll expand on these areas to see how they maximize cloud benefits while retaining a lean engineering footprint.

    Automation, Automation, Automation

    “Automation is key to furthering your adoption and use of the cloud,” said Spurrier. The drive to automate comes as no surprise to many DevOps and SRE professionals whose goal is to automate their job away. When automating the cloud, infrastructure-as-code (IaC) is a fundamental necessity. “If you’re getting into cloud, you should probably have some scripting skills,” said Spurrier.

    Automation can significantly aid scalability when setting up additional accounts since engineers must replicate similar operations, such as server provisioning, deployment patterns, monitoring and so on. Automation is also essential for shift-left testing and to increase observability into production environments. “It’s one thing to deploy an application via automation, but another to have tests in automation—that’s the next level,” said Spurrier.

    Now that so many tools are API-driven, orchestration becomes programmable. A cloud service provider’s CLIs or SDKs could also be leveraged to hook into these integrations to construct such automation.

    On the flip side, too much automation for high-level privileges could backfire. “Someone could break into a data center, steal a server, disconnect it, then run off,” Spurrier explained. Thus, automation will require a higher sense of security to ensure that every change is appropriately authorized and logged.

    Reusable Templating

    Along with automation, documenting operational processes to be shared amongst engineering teams is vital for maximizing efficiency in a lean talent landscape. The key here is to make the process self-documenting and reusable, stresses Spurrier. This will most likely involve someone on the team creating shared libraries and repositories for other team members to leverage.

    One such example is Amazon’s golden AMI pipeline. Golden AMIs are virtual machines that teams can “standardize through configuration, consistent security patching and hardening.” Golden AMIs can also enable a repeatable process for performing monitoring, logging and even distribution to business units.

    Reusable patterns are important for building CI/CD systems so that when you commit code, it is automatically deployed to preferred cloud providers. In general, Spurrier recommends building shared repositories that are easy to clone, with step-by-step instructions for usage. Sample applications that demonstrate components in different languages can also aid developer usability and save time.

    Adopt Centers of Cloud Excellence

    Another way to encourage better adoption is by creating a center of cloud excellence (CoCE). For an enterprise, having a cloud-centric project management group can be very helpful to organize knowledge and offer support. A CoCE could centralize operations, supply engineers with pre-configured containers and help distribute access to cloud accounts.

    A CoCE is a good way to start to unite disparate teams around shared best practices, said Spurrier. Depending on your organization’s needs and workloads, a CoCE will focus on varying skills, he added. In the beginning, a company may have just one center for cloud knowledge. However, as adoption grows, dedicated centers may likely emerge around specific cloud providers, too.

    Retain Agnosticism

    Contrary to popular belief, multi-cloud steups aren’t always about lift and shift. Business units often adopt a particular cloud to suit specific workload requirements. For example, researchers may choose GCP for its data science functions, or Microsoft shops may prefer Azure.

    In a multi-cloud world, management layers could help retain a modicum of vendor neutrality. For example, take Cloud Custodian, a tool developed at Capitol One and recently contributed to the CNCF. Using Cloud Custodian, engineers can run security checks across AWS, Azure and GCP and generate metrics for reporting. Such control and insights could empower teams to automate some of the multi-cloud management burdens.

    When working with multi-cloud, it’s also good to consider agnostic tooling that can work across different providers, like Terraform or Kubernetes, Spurrier said.

    Templating

    Cloud transformation is ongoing at most organizations. With this rise, more cloud-native processes and tools are emerging. Yet, with talent in short supply, DevOps isn’t always fully prepared. To review, we covered four ways businesses can circumnavigate this issue to get an edge:

    • Increased cloud automation is a necessary response to a slimmer labor pool. Though building automation requires upfront technical skill, it pays dividends in increased efficiencies.
    • Reusable libraries and templates can rapidly configure new projects and do so consistently across business units.
    • Consider maintaining a center of cloud excellence to centralize policies and offer support.
    • Opt for agnostic tools. This helps to retain vendor neutrality and apply consistent controls across clouds.

    Of these above strategies, Spurrier emphasized the importance of creating shared templates. When used right, a templating approach can be very effective, he says. “If you have a templating model, it should be easy for DevOps teams to migrate to new features. If you’re leveraging tools that set up templates, it’s easier to figure out how much infrastructure is going to cost.”

  • CloudBees Acquires Neuralprints to Shift Compliance Left

    CloudBees Acquires Neuralprints to Shift Compliance Left

    At the online DevOps World 2021 conference today, CloudBees revealed it has acquired Neuralprints to provide the core technology for CloudBees Compliance, a real-time compliance and risk analysis platform that it will roll out in the first quarter of 2022.

    At the same time, CloudBees has enhanced the feature management capabilities of its namesake continuous integration/continuous delivery (CI/CD) platform to provide greater visibility into and control over feature flags that are now widely used to add new capabilities to applications.

    Prakash Sethuraman, chief information security officer (CISO) for CloudBees, said CloudBees Compliance will make it possible for DevOps teams to continuously enforce compliance policies as part of an effort to better secure their software supply chains. Those DevOps teams will be able to enforce compliance across code, binary artifacts, data, identity and infrastructure environments in a way that provides developers and IT operations teams with instant actionable feedback to enable issues to be addressed long before an application is deployed in a production environment.

    CloudBees Compliance is based on a mix of open source and proprietary technologies developed by Neuralprints and will be made available for on-premises IT environments as well as via a software-as-a-service (SaaS) application. The goal is to enable organizations to manage compliance as code as part of any effort to shift more responsibility for application security further left toward DevOps teams, said Sethuraman.

    CloudBees Compliance

    A survey of 500 C-level executives published today found nearly half (45%) admit that initiatives to secure their software supply chains are halfway or less-than-halfway complete. Nevertheless, 95% claimed their software supply chains are secure (95%), with more than half (55%) posting they are very secure.

    A full 93% said they are prepared to deal with an issue such as ransomware or a cyberattack on their supply chain. However, nearly two-thirds (64%) admitted they are not sure who they would turn to first if their supply chain was attacked; with 58% admitting that, if they experienced an attack, they have no idea what their company would do. An equal percentage (64%) said it would take more than four days to fix the problem if they did experience an issue.

    Almost all C-level executives (95%) said they think more about securing the supply chain now than they did just two years ago, and 92% said a security issue would impact their brand. A total of 83% said security issues cause their developers to drop everything to review code, with 82% noting that caused the organization to spend less time on innovation.

    On the plus side, 95% of executives said container images are checked for high or critical vulnerabilities, while an equal number said automation access keys are set to expire automatically. A total of 92% said their company only accepts commits signed with a developer GPG key, while 90% said dependencies to trusted registries are limited at their organization, while 89% said administrative access to CI/CD tools is restricted (89%).

    Of course, what C-level executives believe and what actually occurs inside an organization are not always one and the same. A series of high-profile software supply chain breaches resulted in a number of organizations reviewing their software supply chains. At the same time, the Biden administration has issued an executive order requiring all federal agencies to conduct similar reviews.

    One way or another, most organizations will soon find themselves embracing a wider range of DevSecOps best practices to ensure the integrity of software supply chains no matter how secure they may think their software development platforms are today.

  • The New(ish) Complexity

    The API economy and microservices architectures, led by Agile and DevOps, have introduced a massive amount of hidden complexity. I call it hidden complexity because we have a tendency in IT to focus on the problems in front of us, resolve them, then focus on the new problem in front of us. This leaves technical debt in little piles behind us, like the bits that fall off the ends of a snowplow blade (or the asphalt that squishes out the end of the steamroller for our southern friends).

    I was thinking about this when I was looking into how a majority of you handle scalability. The layers of complexity for scalability are surprising. In the past, we invested upfront to be prepared to scale to the levels we need. Today we allow expenses to grow as users grow. That is a great model if users are tied to revenue. If there is any fungibility at all in the correlation of users to revenue (and there almost always is), this is a potential disaster waiting to happen. “Our traffic spiked 200%, but revenue only grew 20%,” is pretty much the last thing your business leaders want to hear. Because in the “Pay as you grow” world, a 200% spike in users means a significant spike in overhead. How much overhead varies pretty wildly, of course, so I’ll just say “Guaranteed and significant” and leave it at that.

    This is a known bit of complexity, but those of you I’ve spoken with have done little to hedge against it. That’s … interesting. How many other hidden land mines are out there, being ignored by us?

    What can you do? That greatly depends upon your business. with internal systems, you have a lot of say in usage. External systems are where this is potentially an issue. You can pay your provider to block DDoS, so at least you’re not paying for that bandwidth if that happens—and most of the people I spoke with really do have some form of DDoS protection that stops traffic before it becomes a fee-generating hit. But for actual growth, the best you can do is get with business leaders and determine what influences the correlation between users and revenue. What makes those people upping your traffic actually spend money while they’re upping your traffic? What can you do to convince more of them to spend money? What services are you not offering that might be useful to those customers and not significantly increase overhead. Basically, this is the modern equivalent of the conversion problem. They’re on your site, how do you convert them to paying users of your site.

    We really can’t get more detailed than that—you know your business, and a media site is a different model than a retail site, for example—so you’ll have to develop beyond that. But have a plan that does what you can to make sure that when expenses go up, revenue goes with it.

    And keep rocking it. We’re outside the historical IT realm here—because you’re all rocking it so well that we have new things to worry about. Give those customers rockstar apps to work with and figure out how to make them paying customers. Develop those bonds with business leaders; you do indeed need each other.

  • DataOps: How to Turn Data into Actionable Insights

    DataOps: How to Turn Data into Actionable Insights

    Enterprises‌ ‌have‌ ‌struggled‌ ‌to‌ ‌collaborate‌ ‌well ‌around‌ ‌their‌ ‌data, which can impact everything from digital transformation to advanced concepts like AI and ML.‌ ‌DataOps is not without challenges; building, managing and scaling data pipelines requires careful thought around reusability, portability across infrastructure and applications and long-term maintenance and governance. And these are just a few of the issues facing enterprises. For this reason, DataOps technology stacks need to focus on providing key capabilities include data extraction, integration, transformation and analysis.

    The Importance of DataOps

    The enterprise is undergoing a seismic shift from siloed application development and data repositories to more composable and reusable architectures. Additionally, there is a growing demand for speed and agility as well as the influx of disruptive technologies such as the cloud, IoT and AI. Organizations that want to win big must act and adapt quickly to deploy and scale new software and solutions that provide customers with a superior experience and satisfy their rapidly evolving needs. To do that, they also must be able to rapidly aggregate, integrate and analyze data sources.

    Even with this need for speed, the tools and processes for the creation and processing of data are not standardized‌ in‌ a way that promotes rapid innovation‌ and ultimately helps organizations transform. DataOps is critical to address the challenges involved in acquiring, storing and governing data. In addition, companies are dealing with increasing complexity in their IT environments. In a recent survey, more than 80% of enterprises have a hybrid cloud or multi-cloud strategy. In order to have cost-effective and secure management of increasingly large amounts of data, enterprises must adopt DataOps.

    Managing Data Growth

    DataOps has been around for about a decade, but has recently gained momentum because of the overwhelming challenges companies face today in dealing with large, complex sets of data that are being generated at increasingly faster rates. With new technologies like the internet of things (IoT), cloud computing and the power of big data now integrated into everyday use, companies are generating at least 50 times more data than they were just five years ago. And with more data comes a need for greater efficiency and higher demand for data experts.

    DataOps potential impact on businesses might lead you to think of it as a radical new methodology. But many companies have already been using similar practices to deal with some aspects of data management, particularly around data warehousing and analytics. And just like DevOps, DataOps isn’t a product, but rather a cultural shift supported by many products—many existing products. So it’s important for businesses looking to adopt DataOps to consider what they already have, such as enterprise data warehouses and ETL tools, and what they may need to acquire, replace or modernize. In the end, companies will end up with several systems to support the data pipeline.

    DataOps and Big Data Challenges

    Most enterprises have invested in significant data infrastructure to extract, load and store their data to take advantage of Big Data analytics and technologies. However, these infrastructures are often layered, hard to manage and full of legacy tools that hinder the transfer and integration of data. Additionally, organizations have invested significant resources in tools such as data warehousing and data marts. These data warehouses have been used to model data but have been typically implemented with a predefined data model. A business intelligence (BI) system, an enterprise data warehouse or any other set of tools might assist in the execution of a data transformation job.

    The technological revolution is creating unprecedented opportunities to make things happen with data. By analyzing the data that powers modern business, we are able to provide value in every dimension of our organizations. As is the case with every transformation, the benefits of building an integrated, self-service analytics environment will be realized by users rather than IT. Leveraging the power of data science, business leaders can achieve operational excellence and compete effectively against the growing waves of competitors. For the data science and analytics staff to be successful, they must be exposed to all of the necessary technology and processes to support effective use of data.

  • Zero-Trust Network Access Platforms Slash DevOps Bottlenecks

    Zero-Trust Network Access Platforms Slash DevOps Bottlenecks

    In a prior article, I indicated why a new remote zero-trust platform is needed to support DevOps teams.

    Traditional remote access configurations are not well-suited for DevOps given a shifting user population comprised of employees, contractors and other third parties often using their own devices; the large number of tools and systems to which access must be provided and the hybrid nature of the infrastructure itself (on-premises, hybrid and multi-cloud).

    DevOps Pain Points

    Three pain points are associated with traditional approaches when used with DevOps.

    1. DevOps user experience is poor in traditional remote access configurations. Each DevOps service must be accessed as separate secured sessions, which is confusing and tedious.
    2. System management requires IP whitelisting rules in VPN, static SSH keys in Bastion, firewall rules controlled by different teams and application-specific authentication. The coordination and execution of so many touchpoints for common actions—including the onboarding of new team members, changing roles or adding a new service—invites mistakes that cause productivity issues and introduces unforeseen security holes.
    3. Security is complex and error-prone. DevOps users must be granted restricted access to the individual services they need to be productive. Administering restricted access for individuals and groups is time-consuming. Security policies typically require monitoring detailed audit logs of user access.

    How a Zero-Trust Remote Access Platform Works with DevOps

    In my whitepaper, Zero-Trust-Remote Access-For-DevOps, I explain how a zero-trust remote access platform can resolve the requirements for improved DevOps user experience, consistent and manageable administration and simplified, less error-prone security.

    Improved DevOps User Experience

    By replacing the need for the user to tediously set up separate sessions for each DevOps service they need to use, a client-side DevOps service catalog enables all DevOps services to be accessed with a single click. This service catalog approach is more convenient for DevOps users and provides a form of DevOps governance. It does this by encouraging DevOps teams to select choices that are available in the catalog instead of creating new choices that result in expensive DevOps tool sprawl.

    Simplified DevOps Management

    Management of remote DevOps users is simplified by consolidating access for all DevOps users and services under a single unified service manager and connector. The need to manage each user and service with different teams, each with their own processes, is unified and streamlined when the management requirements for DevOps users and services are consolidated under a common zero-trust remote access platform.

    In addition, this approach makes it much easier to implement more adaptive policies across the breadth of users and services.

    With a zero-trust remote access platform, onboarding new DevOps users and groups is accomplished within a single session—not separate sessions for each onboarding requirement.

    Simplified and Less Error-Prone Security

    Consolidating access to DevOps services under a common zero-trust remote access platform with unified services and services connectors enables simplified and less error-prone security controls, improved security monitoring and policy management. Audit logs of access are reported consistently for all services. Monitoring and changing access rules can be supported for individuals or groups when adjusting a policy. Trust-based access controls can be implemented consistently. Granular policies ensure access is predicated on appropriate device trust score thresholds being met. Rather than a single check prior to granting access, these production services policies continuously assess the security of the users and devices accessing the services.

    Transforming to a Zero-Trust Remote Access platform for DevOps

    Transformations of all types, and especially those for DevOps, are made more certain by following a
    seven-step transformation process, described in my book Engineering DevOps. This process serves as a framework for implementing the zero-trust remote access platform.

    The first two steps, vision and team alignment, involve getting the leaders, key influencers and implementers of an organization to recognize and establish actionable goals for the implementation of the zero-trust remote access platform.

    The third step, assessment, requires taking inventory of the current state of DevOps users, DevOps services and policies that need to be supported by the new platform.

    The fourth step, road map, requires determining the plan for implementation. Typically, it makes sense to start with one application. Start with a set of DevOps users and DevOps services needed for a single application or pipeline which can serve as a model to drive adoption to other applications as the solution is proved and accepted.

    The fifth step, realize, is the step in which the zero-trust remote access platform is installed and set up for the initial model application or pipeline. Banyan Security explains how an example tool vendor helps to simplify this step.

    Capabilities to look in for in zero-trust remote access tools include:

    • Deploys in minutes, with no integration requirements on any existing DevOps services or infrastructure.
    • No need to set up VPNs, open inbound firewall ports or manage DNS.
    • Use existing DevOps user devices without diminishing security.
    • DevOps service catalog provides users visibility and one-click access to DevOps services.
    • Support for bundled services, favorites, and autorun capabilities.

    In step six, operationalize, the procedures for commissioning and using the zero-trust remote access platform are proven, benefits are measured and playbooks are hardened for the organization.

    In step seven, expansion, the zero-trust remote access platform deployment is expanded to other DevOps applications pipelines, DevOps users and DevOps services across the enterprise.

    What This Means

    Traditional remote access configurations are not well suited for DevOps administrators or users given the dynamic nature of services, team members, and environments. A zero-trust remote access platform addresses the need for an improved DevOps admin and user experience, simplified access management to DevOps services and simpler, less error-prone security practices and monitoring.

    I want to express my appreciation for the time spent with the Banyan Security team, who has pioneered many of the concepts outlined in this blog and contributed valuable insight.

  • Google DORA Report Shows Modest DevOps Gains

    Google DORA Report Shows Modest DevOps Gains

    Google today published the annual Accelerate State of DevOps Report created by its DevOps Research and Assessment (DORA) team that finds 26% of DevOps teams surveyed can now be classified as elite performers. That compares to 20% the last time the study was conducted in 2019.

    DevOps teams, made up of a total of 32,000 IT professionals from around the globe, were evaluated based on their ability to frequently deploy, code, lead time for changes, time to restore service, change failure rates and operational performance. An elite team is expected to be able to restore service in the event of an outage in less than an hour.

    Dr. Dustin Smith, a user experience researcher for Google, said the increased rate suggests steady gains in DevOps maturity continue to be made despite all the challenges the COVID-19 pandemic presented. A full 96% of respondents said that prior to the pandemic they had not worked from home on a regular basis.

    The DORA report finds high and elite performers make up two-thirds of respondents. Elite performers were able to collectively decrease their lead time for changes when compared to previous studies. It’s also worth noting only elite performers have minimized their change failure rate. In previous years, medium and high performers were able to do the same.

    Overall, more than half of respondents (52%) report they have adopted site reliability engineering (SRE) best practices to some degree.

    Elite performers deploy code 973 times more frequently than low performers, who reported deploying less than once every six months. The normalized annual deployment numbers range from 1,460 deploys per year for the highest performers to 1.5 deploys per year for low performers. Not surprisingly, for elite performers, the lead times from commit to deploy are 6,570 times faster than low performers.

    Security practices have also improved with 60% conducting security reviews, with 58% reporting that the reviews include testing. Nearly two-thirds (63%) said they work closely with information security teams. The report finds teams with high-quality documentation were 3.8 times more likely to integrate security throughout their development process.

    Reliance on cloud platforms has also increased, with 56% of respondents indicating they employ a public cloud, a 5% increase from 2019. Only 21% of respondents reported deploying to multiple public clouds. Over a quarter (26%) of respondents did so to leverage the unique benefits of each cloud provider, followed closely by 22% that cited availability.

    Just over a third (34%) report they are using a hybrid cloud, while 29% have a private cloud. A total of 21% still don’t use a public cloud.

    The report also finds elite performers were 3.5 times more likely to have met all essential cloud characteristics defined by the National Institute for Standards and Technology (NIST). Those characteristics are defined as on-demand self-service, broad network access, resource pooling, rapid elasticity or expansion and measured service. Only 32% of respondents who said they were using cloud infrastructure agreed or strongly agreed that they met all five of the essential characteristics of cloud computing defined by NIST, an increase of 3% from 2019. Overall, usage of those characteristics of cloud computing have increased by 14% to 19%, with rapid elasticity showing the largest gain.

    The DORA report makes it clear there is still a long way to go before DevOps nirvana is more widely achieved. On the plus side, however, the number of organizations that have embraced DevOps to some degree continues to steadily increase.