devopsy.kinsta.cloud

Category: DevOps Practice

  • Pega Low-Code Platform on Display at PegaWorld iNspire

    Pega Low-Code Platform on Display at PegaWorld iNspire

    Low code enables developers to create applications using visual tools and models. Developers looking for resources to improve their productivity might find Pega’s low-code platform quite helpful.

    Jennifer Gill, senior director of technical product marketing at Pega, joined ASG’s Mitch Ashley for an interview on the benefits of low-code in designing applications.

    During the pandemic, Pega has helped customers develop apps such as a COVID-19 tracker for a large healthcare organization, which was built in less than a week. And in just five days, Pega worked alongside the Bavarian Ministry of Economic Affairs and T-Systems to build a new app that provides COVID-19 financial aid to self-employed workers and small to medium-sized businesses.

    Pega has its own DevOps tool, Deployment Manager, which enables organizations to execute their own unit testing.

    Also built into the Pega platform is the Pega Express Methodology, which guides users step by step from smaller implementations to complex transformations, laying the foundation to easily add new app functionality as business needs change.

    As for the future of Pega, Gill said we can expect more automation and more intelligence built into the platform.

    Meanwhile, you can join her at PegaWorld iNspire, an interactive virtual event taking place Tuesday, June 2, at 9 a.m. EDT, to learn more about the technology behind Pega.

    The event will also feature a live demo on low code, during which attendees will learn how to define core app components and what tools they can use to manage and govern low code at enterprise scale.

    Check out Jenn’s interview for TechStrong TV below to learn more about Pega’s low-code platform.

  • Survey Sees Increased Agile Adoption During COVID-19

    Survey Sees Increased Agile Adoption During COVID-19

    A report published by Digital.ai, the parent company of CollabNet VersionOne, finds 43% of organizations have increased their reliance on agile application development methodologies in the wake of the COVID-19 pandemic, with 15% saying it has increased significantly in the last 90 days.

    The report also notes 33% of respondents say they have increased or expanded Agile adoption in the last 90 days to help manage distributed teams, while 55% said their company plans to increase the use of agile in the next 12 to 14 months.

    Based on a pair of surveys conducted prior to and after the pandemic started, the 14th annual “State of Agile” report also finds 60% of respondents credit agile methodologies with having helped increase speed to market, while 58% say it has improved team productivity. Another 41% say they are also better able to manage distributed teams.

    Mike O’Rourke, chief research and development officer for Digital.ai, said there is a clear correlation between the ability to manage distributed application development teams and successfully navigating the pandemic now that most application development and deployment teams are working from home.

    Many of those teams are also seeing application development deadlines being moved up as organizations prioritize digital business transformation initiatives that now are core to any business continuity strategy.

    That shift is also accelerating a shift toward focusing more on business metrics for measuring success as defined by, for example, the Scaled Application Framework (SAFe), said O’Rourke, noting as a result, there’s also a lot more focus on value stream management across the enterprise as business and IT teams work closely together. The survey finds just over half of the respondents report their organization has either already implemented value stream management (VSM) or are planning to do so.

    The survey also finds more focus on risk and compliance, with the number of respondents identifying and measuring technical risk prior to deployment increasing 54%, while the importance of automated audit compliance and governance across control points increased 10% year over year.

    Perhaps most surprising, the survey finds reducing project cost is cited as an important reason for adopting agile by only 26% of the respondents, a 41% drop from the previous year. The number of organizations focused on release planning decreased by 11%; however, organizations focusing on product road mapping increased 9% over the previous year. The survey also finds slightly more respondents are using automated acceptance tools (3%). Another 12% said they plan to adopt agile project management tools in the future.

    It’s not clear whether increased focus on application development might result in bottlenecks when it’s time to deploy those applications. Organizations that have already embraced best DevOps practices alongside agile development methodologies will naturally be in a better position to deploy applications. At the same time, increased focus on digital business transformation is likely to accelerate the adoption of both agile development methodologies and best DevOps practices across the enterprise.

    In fact, it’s arguable that organizations that are not able to build and deploy applications adroitly may soon find they are unable to compete at a time when software development has become critical to their survival.

  • How FaaS Is Disrupting Workflows: New CSP vs. CDN Services

    How FaaS Is Disrupting Workflows: New CSP vs. CDN Services

    Function as a Service (FaaS), the popular incarnation of serverless computing, along with containers, has played an important role in the migration from monolithic application architectures to those based on microservices. Its introduction has changed the way developers think about infrastructure and application development. While at the same time, it has also driven evolution in the consumption-based infrastructure models we saw introduced with the advent of elastic computing.

    FaaS has simplified broad adoption of value-add platform services from the major cloud service providers (CSPs) such as database and authentication services. It has quickly grown to become a viable infrastructure option for developers and IT professionals alike, serving as another tool in the chest – perfect for short-lived or ephemeral tasks. In recent months, we’ve seen announcements from content delivery network (CDN) providers touting a new breed of FaaS offerings promising low latency, workflow customization and cost containment. 

    CSP FaaS vs. CDN FaaS

    CSP FaaS resides in regional hyper scale data centers with the number of data centers in a region typically in the low single digits. These are the same data centers where the myriad of infrastructure and platform services are offered. Enterprises spin up their applications in virtual private clouds in a single region, sometimes they configure a backup region but even with today’s tools it is still complicated to manage data replication/consistency and ensure environments are kept in lockstep.

    In contrast to CSP FaaS, CDN FaaS solutions are highly distributed with various PoP locations. These CDN FaaS architectures are typically integrated with the providers’ edge delivery locations. Depending on vendor architecture, there could be hundreds of edge locations. Whereas CSP FaaS benefits from being collocated with platform services, CDN FaaS benefits from being closest to CDN action, where end-user requests/responses are received and processed. This is where edge computing comes into play because it minimizes latency while allowing for custom content delivery workflows.

    Empowering Innovation by Enabling Customization

    Developers use functions to customize content delivery workflows to create more compelling user experiences. Up to this point, CDNs provided limited capability for developers to define custom workflows via their own code. Flexibility has been limited to predefined “rewrite rules” and custom behaviors are the product of professional services-led engagements, which is not exactly developer friendly. CDN FaaS changed this, handing power to developers to deploy code and create unique, differentiated user experiences.

    The Make Or Break Element: Latency

    It is important to minimize latency between application services that rely on FaaS to avoid program timeouts. CSP FaaS options typically offer a broad range of event-triggers and run-times/programming languages to support developer preferences and align with the other elements of the development environment. 

    CDN FaaS has request triggers that are integrated with end-user or origin server request/response flows. FaaS invocations are typically a result of an end-user requesting content or an origin server providing a response that needs to be customized on a per end-user basis to reflect their preferences. The scale and latency expectations are so demanding that the run-time options are often limited to those that provide the lowest memory footprints and highest performance. CDN FaaS can also take advantage of the caching technologies further improving latency to complete functions – especially when remote service or network calls are involved. 

    How Do the Costs Stack Up?

    For CSP FaaS, the pricing model is two dimensional. It is based on the number of requests combined with the allocated memory and the elapsed time of function execution. Some of the CSPs are offering generous free tiers to entice customer adoption – and remember, Function as a Service drives adoption of many of the CSP’s higher value platform services.

    When it comes to CDN FaaS, the pricing models are similar, some organizations are charging for requests only but they limit memory foot and the time a given function is allowed to run. However, there is typically a premium to run FaaS in those highly distributed edge locations. Which is no surprise because cost models are higher for distributed/global build outs than centralized models. Also, due to the nature and demands of web and video services, request volumes are likely to be an order of magnitude higher than CSP FaaS requests. 

    Not All Faas Services Are Created Equal

    Function as a Service is an important part of our infrastructure as a service (IaaS) choice but it’s unlikely enterprises make a CSP solely on the FaaS offering. Typically, the decision is based on what value-add platform servers are available to build and support the application, and how seamless can those services be integrated with FaaS.

    For CSP FaaS, it is all about the bigger picture, the infrastructure and platform choices. This is also true with CDN FaaS because it is unlikely a CDN purchasing decision will be made on that CDN’s FaaS capabilities in isolation. The selection criteria will include FaaS but the larger weighting factors will include the content delivery capabilities, such as global reach, capacity, origin offload, feature support and so on.

    Overall, CSP FaaS are optimized for flexibility, multiple run-times, broad range of request triggers and tight coupling with CSPs’ higher-value platform services. Latency is important but the focus is within the CSP region or customer’s VPC and the use cases are often enterprise applications for B2B technology companies. Meanwhile CDN FaaS is all about performance, run-time efficiency and latency. It’s about running the functions in conjunction with the CDN – closest to where end-user requests originate and media responses are delivered. CDN FaaS requires enormous scale – on a global scale versus a single, centralized region.

  • Netlify Makes Build Plugins Available for Jamstack Platform

    Netlify Makes Build Plugins Available for Jamstack Platform

    Netlify, at a Jamstack Conf Virtual event, announced the general availability of open source Netlify Build Plugins that provide IT teams with a set of continuous integration/continuous delivery (CI/CD) workflows for building websites and web applications using JavaScript, application programming interfaces (APIs) and static HTML files without employing web servers, otherwise known as a Jamstack.

    Designed to be launched from within a Netlify platform to enable IT teams to build a website and launch it from a Git repository to a content delivery network (CDN), the Netlify Build Plugins are intended to make it easier to employ DevOps processes using, for example, a plugin to run an end-to-end Cypress test.

    Netlify CEO Mathias Biilmann said DevOps teams also can make use of a Netlify API to create their own plugins. Ultimately, the goal is to foster the development of a wide range of open source Netlify plugins that would be built and vetted by the Jamstack community, he said.

    Other Netlify Build Plugins provide a means to enable Ghost, an open source headless content management system based on Node.js that can be deployed on any framework; launch a Sentry tool to automatically discover, triage and prioritize bugs; or enable a Gatsby tool to create incremental builds on Netlify.

    A survey of more than 3,000 Jamstack developers published today by Netlify finds one-third have worked on sites serving millions of users over the last year. Nearly two-thirds of survey respondents (62%) work outside the tech industry, including advertising and marketing (21%), media and publishing (14%), education (14%), financial services (13%) and business support and logistics (13%). Well over a third (38%) work for a technology company. About a quarter (26%) have more than 10 years of experience.

    Top Jamstack benefits cited by all types of developers are performance, uptime, speed of development, security, compliance and avoiding vendor lock-in, the survey finds.

    Biilmann said Jamstack as a philosophy enables IT teams to separate the development of front- and backend websites and applications in a way that allows organizations to embrace best DevOps processes. That approach is agnostic as far as any specific JavaScript framework is concerned, but it does define a way to automate the build process for a web application or website.

    In the wake of the economic downturn brought on by the COVID-19 pandemic, more organizations are investing time and resources in modernizing their websites and applications. With more employees and end users depending on those websites to drive digital business processes, the quality of the application experience has become a much bigger concern. At the same time, end users have now come to expect those application experiences to be updated regularly, which in many cases will necessitate the adoption of best DevOps processes.

    In fact, it’s arguable that organizations that don’t embrace best DevOps practices to build and deliver web applications at scale will soon discover they have bitten off more than they can chew.

  • Surviving Through the New Normal

    Surviving Through the New Normal

    As enterprises move from the first phase of the pandemic to the reopening phase, they’re going to have to determine what tech investments going forward will provide the most value in the their new normal.

    Emad Georgy, founder and CTO at consulting firm Georgy Technology Leadership, said that while overall tech spending will remain down, that pain won’t be evenly distributed. Georgy claimed that investments will continue to remain high in the obvious industries, of course, such as healthcare, retail and grocery chains.

    “However, multiple adjacent and dependent businesses investments will also remain high. Suppliers of materials required by these industries, companies involved in their supply chain, companies supporting remote collaboration tools will continue to thrive,” said Georgy.

    Coming off the heels of the economic shutdown, Alex Milowski, platforms researcher and evangelist at database management software provider Redis Labs, said that both people and systems have experienced significant disruptions, and this has hurt their productivity and strained technology platforms in some ways, while for other job functions platform and infrastructure are at over capacity.

    Milowski said organizations needed to strike a balance between the immediate needs of staff, so that they could adjust to the changes in their work and lives and still access critical work systems while also ensuring that customer needs are also met.

    However, now that enterprises are looking forward, how do they ensure that their digital transformation efforts remain in place?

    As enterprises get their footing as the new economic reality settles in, Milowski advised organizations to focus on small wins.

    “Prioritizing projects where one can increase the confidence of the team to function under uncertainty may be the best strategy. New projects may need to be created to address surges in demand via caching or acceleration systems,” said Milowski. “A team might use this time to address auto-scaling systems so they can scale-down as a cost-saving measure. The increased collaboration and creative problem solving will build team confidence in their ability to deliver. With each win, ensure readiness is improving.”

    Within those early successes, teams can then troubleshoot their processes with less disruption to their business.

    “As their confidence grows, they can ease into picking more business-critical and complex projects. In the end, they are more likely to succeed, with less disruption, as their team is now ready as they know how to deliver new digital initiatives in very different contexts. Successive small wins turn into big wins,” said Milowski.

    Georgy indicated that such uncertain times require a sharp focus and prioritization.

    “Financial insecurity often leads to sharpened discipline which one could argue we should have anyway in an organization. Unfortunately, many companies are plagued by ‘management by assumption.’ That is the practice of managing and deciding based on opinions or second-hand knowledge because they don’t have time to [dig into] the weeds,” he said.

    It’s time to prove concepts out. Georgy added that it’s more important than ever to make data-driven decisions based on experiential learning.

    “Opinions are no longer welcome. Every person at the decision table must justify their decisions with data, with real-world experience and with action,” he said.

    That can be done by CIOs as they track their changing customer base.

    “Customers are changing how they spend and re-evaluating what is important to them and this will be reflected in roadmaps as well. Now, more than ever, feedback loops, usability groups and touchpoints with customers to understand this changing dynamic is key to evaluate new approaches and priorities in the roadmap,” said Georgy.

    John Belden, project execution advisory services practice leader at UpperEdge, stated that now is the opportunity for CIOs to lead in a way that hasn’t been readily possible in more than 20 years.

    Belden advised CIOs to:

    • Take the lead: CIOs can win back the seat at the management table that they enjoyed during the Y2K era. By proactively reaching out to work with counterparts in HR, marketing and supply chain, CIOs can shift from order-taker to problem-solver, which will also help in prioritizing projects.
    • Determine an approach to road-mapping: There are likely to be three different strategies to road-mapping and navigating this time:
      1. Reinvent: This approach establishes a clean sheet of paper and takes a more traditional approach. Companies that could never get the cash or attention to build a roadmap, might have this work thrust upon them if corporate boards demand a pandemic risk mitigation plan.
      2. Revise: This would involve the shuffling of the existing portfolio, some reprioritization and confirmation of direction from the various vendor’s strategies.
      3. Accelerate: Some companies may have already charted a course that accounts for assumptions of a new world order. In these cases, it is likely that CIOs will be asked to accelerate to either leap-frog competitors or solidify their competitive position.
    • Transform vendor engagement strategy: The onslaught of vendor marketing initiatives and client targeting strategies could end up being like the wild west with cloud providers bypassing traditional IT channels to market their wares. CIOs need to work proactively with their business counterparts to set the ground rules for engagement and craft a strategy that motivates suppliers to assist clients in developing their own strategies that can be realized quickly and efficiently. Developing a long-term view in managing the short-term downtime will, in the end, pay dividends in executing during this economic downturn.

    Short-term survival with an eye on meeting long-term goals is key to survival, and in the months ahead those enterprises that successfully meet that balance will set themselves apart.

  • CTO.ai Launches SlackOps Tool for DevOps

    CTO.ai Launches SlackOps Tool for DevOps

    CTO.ai has launched SlackOps, a platform for managing DevOps workflows in real-time via the Slack communications and collaboration service.

    Company CEO Kyle Campbell said many DevOps teams are embracing Slack to share updates with one another. SlackOps takes that preference for DevOps a step further to enable complex DevOps processes, also known as ChatOps, to be launched from the Slack command line.

    That approach allows any member of the DevOps team to write a workflow once and securely share it regardless of what language is employed, added Campbell.

    SlackOps also provides access to a public registry that CTO.ai already makes available for sharing developer shortcuts within a specific community.

    Beyond streamlining communications processes across DevOps teams, Campbell said DevOps teams will be able to surface additional insights into workflows using DevOps research and assessment metrics to make them more efficient. SlackOps also makes it easier to onboard new members of a DevOps team, he noted.

    SlackOps is deployed as a container on the Slack platform and can be downloaded via the Slack marketplace in a couple of minutes.

    The goal is to enable organizations to embrace a lean approach to DevOps that enables them to quickly adjust processes as required, said Campbell. Rather than having to commit to a single tool or platform, developers should be able to advance the overall state of DevOps within their organizations without having to be concerned whether a new tool or platform will disrupt communications across the DevOps team.

    It’s too early to say to what degree Slack might emerge as a control plane for managing DevOps processes. It is, however, becoming more popular as a way to communicate across members who employ a broad range of tools. Rather than attempting to force DevOps teams to standardize on a single set of tools and platforms, Slack provides a more flexible means of communicating in real-time that is delivered as a cloud service.

    Of course, there are now plenty of alternatives to Slack. It will be up to each DevOps team to decide which communications medium best suits them. However, embedding a DevOps workflow application within a communications platform could serve to make Slack a more compelling option for many DevOps teams. Regardless of the communications medium, DevOps teams should expect to be more continuously engaged with one another. That may prove to be a critical requirement with most members of a DevOps team working from home for the foreseeable future in the wake of the COVID-19 pandemic.

    As workflows built around asynchronous email messages increasingly are giving way to real-time communications, the challenge many DevOps teams face is finding a way to embrace that higher level of interactivity without members of the DevOps team becoming overwhelmed by the volume of messages being shared. Making the transition to DevOps was enough of a cultural challenge for many organizations. Inundating DevOps teams with real-time messages, not all of which may be pertinent to a DevOps workflow, needs to be tempered by tools that help identify the DevOps signal within all the noise.

  • GitLab Releases Massive Update to CI/CD Platform

    GitLab Releases Massive Update to CI/CD Platform

    GitLab has updated its continuous integration/continuous delivery (CI/CD) platform with a raft of capabilities spanning everything from value stream management to cybersecurity.

    In addition, GitLab announced it is making generally available Gitaly Clusters, which enable DevOps teams to create a warm replica of a Git repository.

    Brendan O’Leary, senior developer evangelist at GitLab, said with release 13 GitLab continues to build on a single code base that allows the platform to be managed and accessed as an integrated set of applications rather than disparate components that DevOps teams need to pull together and then manage.

    In terms of core DevOps capabilities, the latest release adds the ability to customize the Value Stream Analytics module to specific workflows. GitLab is also planning to make it possible to visualize stages of a workflow.

    GitLab is also improving traceability between requirements and has added the ability to visualize the workflow associated with a specific release. A future release will enable DevOps teams to employ feature flags to conduct both A/B testing based as well as create feature flags from merge requests and to filter feature flags by status, according to the company.

    Other capabilities added include more customizable dashboards, with security dashboards now being shareable outside of GitLab to foster collaboration with IT security teams. Future releases will add Kubernetes Clusters to the dashboard to show all clusters and pods in use.

    DevOps teams now can route existing monitoring alerts to GitLab for triage and incident resolution. At the same time, GitLab has moved design management to the core platform.

    GitLab is also now making AutoDevOps, which provides pre-defined CI/CD pipelines, available on Amazon Web Services (AWS).

    In terms of cybersecurity, GitLab has revamped the way the platform manages vulnerabilities to make it easier in a future update to prioritize remediation efforts.

    GitLab also has static application security testing (SAST) support for .NET frameworks and has expanded support for offline environments. There’s also now support for dynamic application security testing (DAST) for REST application programming interfaces (APIs) and access to full commit history scan for secrets. Fuzz testing support will be introduced later this year, according to the company, along with an ability to secure secrets across services. GitLab also plans to add integration with security tools that track container behavior analytics, as well as an out-of-the-box container network policy.

    Thanks to GitLab’s collaboration with Okta, there is now also tighter integration with Okta’s System for Cross-domain Identity Management, which can be provisioned for a GitLab group.

    In terms of compliance, GitLab also plans to enable DevOps teams to establish a compliance framework, adopt its regulatory controls and simplify audit reporting. The company is also working on an initial security policy user interface to simplify setting up security guardrails.

    Finally, GitLab has become a Common Vulnerabilities and Exposures (CVE) ID Numbering Authority (CNA). GitLab users can request a CVE from GitLab, either for GitLab itself or for any project hosted on GitLab.com. In the future, users can request CVE IDs directly from within GitLab.

    O’Leary said GitLab is working toward increasing developer confidence in their security skills. By making it easier to embrace best DevOps practices, developers no longer will have to tiptoe around security issues that they today largely have little to no familiarity with, he said.

    In general, O’Leary said GitLab’s biggest challenge is getting organizations to move beyond simply using it to manage source code. Once organizations do make their first foray beyond source code management, however, it’s not too long before they are employing a wide range of modules, he noted.

    Of course, encouraging organizations to dive deeper DevOps has as much to do with the culture of the organization as it does the capabilities of the CI/CD platform employed.

  • CloudBees Extends SDM Service Reach

    CloudBees Extends SDM Service Reach

    CloudBees has extended the integrations embedded within its software delivery management (SDM) platform to add support for additional services provided by Google.

    Moritz Plassnig, senior vice president cloud and general manager for SDM at CloudBees, said these integrations with Google Cloud Build and Tekton pipelines are being provided as part of a long-standing alliance between the two companies to advance the adoption of best DevOps practices. The CloudBees SDM service runs on the Google Cloud Platform (GCP).

    As part of that effort, CloudBees has been making investments in an SDM platform that enables DevOps teams to pull metrics from any tool or platform. That data is then fed into an analytics cloud service through which CloudBees surfaces insights that identify, for example, how delays to a project might impact overall revenue goals.

    Interest in software delivery analytics has risen in the wake of the COVID-19 pandemic as organizations accelerate the building and deploying of digital business applications, said Plassnig. With employees and customers alike working from home to combat the pandemic, organizations are investing more in creating applications that provide digital alternatives to engaging end users, he noted, adding delays to any of those projects are likely to have a much bigger impact on business continuity strategies today than they did just a few short months ago.

    The CloudBees SDM service provides DevOps teams and business managers alike a real-time view of all product and feature development occurring across the organization, as well as a policy engine to trigger actions to better optimize the software delivery process. Organizations can also model and visualize the software delivery process while measuring overall DevOps performance.

    Rather than requiring organizations to standardize on a single platform to achieve that goal, Plassnig said CloudBees is leveraging application programming interfaces (APIs) to integrate metrics across a broad range of DevOps tools. That approach, he said, makes it possible to meet organizations wherever they are in terms of DevOps maturity.

    Support for Tekton pipelines is significant, Plassnig noted, because it’s expected that most continuous integration/continuous delivery (CI/CD) platforms in the future will be built using Tekton pipelines, which provide an open source framework for building platforms that are designed to be deployed on Kubernetes clusters. Jenkins X, originally developed by CloudBees, is one of the first CI/CD platforms to support Tekton pipelines, which were originally developed by Google. Jenkins X and Tekton pipelines are now being advanced under the auspices of the Continuous Delivery Foundation, an arm of the Linux Foundation.

    As more CI/CD platforms embrace Tekton, the platforms themselves will become less of a black box from an analytics perspective, noted Plassnig.

    In the meantime, CloudBees also expects Google to continue to gain momentum as a cloud service provider, he said. While Google trails Amazon Web Services (AWS) and Microsoft in terms of overall market share by a wide margin, Plassnig noted that with the arrival of Anthos, a cloud operating system capable of being deployed on multiple cloud and on-premises IT environments, it’s only a matter of time before services such as Google Cloud Build are employed more widely.

    Whatever the platform used, the one thing for certain is the rate at which software is being generated and deployed on various platforms will be watched much more closely.

  • DataOps and DevOps: A Match Made in Heaven

    DataOps and DevOps: A Match Made in Heaven

    The adoption of IT technologies has increased exponentially over the last few decades. Nowadays, the need to create quality web applications while simultaneously managing their extensive data effectively is a major concern for any organization. As such, there have been continuous efforts to come up with better approaches to software development, data analysis and data management.

    In order to meet the needs of today’s internet users, technology companies have intensified their efforts and are increasingly making use of all their available resources to find progressively efficient ways to create web applications within a shorter time-space and with superior quality, and to find more effective ways to integrate data-driven decision-making.

    In order to accomplish these feats when it comes to application development as well as data pipelining management, most of the larger organizations are now following a set of practices, DevOps and DataOps, respectively. Within this environment, developers must create applications that can function across various hosted providers, each with its own specific administrators and requirements. 

    This can only be accomplished by looking past the traditional storage capabilities. Instead, developers and organizations they work for are creating an underlying data fabric that facilitates what they’re trying to do, and any cloud provider they want to do it with.

    An Introduction to DevOps and DataOps

    DevOps and DataOps are a set of practices that relates to software development and data management, respectively. 

    These practices can be applied to a single project or can be followed at an organizational level. DevOps emphasizes the automation of the processes of the different teams (software development team, quality assurance team and IT operational team) so that they can easily complete their relative tasks (i.e., design, code, create, check, deliver, deploy, operate and monitor).

    DataOps, on the other hand, can be described as DevOps that includes data analysis, which empowers businesses to create processes to meet the needs of users throughout any data usage life cycle. 

    DevOps emphasizes the cooperation of the software development team (including testers) with system administrators (operational teams). In a similar way, DataOps relies on the cooperation of teams of data engineers, data scientists and other data practitioners who manage data-related processes (i.e., data storage, data processing, data decision-making, data archiving or data delivery).

    The concept underscores both communication and collaboration between the different teams within the same organization, ultimately removing all obstacles for seamless integration between the practices and the end product, including cross-device compatibility. 

    The combined team can make the project more visible and deliver faster results. It also increases the morale of the company by showing that, when it comes to project management, people will still be above processes and resources.

    Benefits of DevOps

    The benefits of DevOps include:

    • Transparency — By regular communication and coordination with the development team, the QA team and the IT operations team will be updated with the latest project details and will be more aware of the processes of other departments.
    • Quicker and high-quality deployment cycles — On any great DevOps platform, the pace of deployment will be improved by applying DevOps practices. The scope of the teams will be limited (i.e., for a small task/module) for a deployment period, as such their efficiencies will increase.
    • Faster bug fixes — Because tasks are smaller and continually distributed on the appropriate servers, the development team can receive input very easily from the QA team, allowing the development team to provide much faster resolution of any non-working components. Application stability would be a by-product of this method, because the cause behind any bug can be established very easily.
    • Flexibility — Because agile methodology insists on managing the raised change request, teams aligned with DevOps will be able to manage any unplanned work more effectively.

    Benefits of DataOps

    Now, let’s take a look at the benefits of DataOps:

    • Faster data processing — When different teams cooperate and interact on a daily basis and work on micro databases rather than on all at once, they not only avoid data overwhelm, they also accomplish much faster processing of data at all the different stages. 
    • Quicker problem solution — Similarly to DevOps, businesses can benefit from DataOps in multiple ways. DataOps data is processed in batches as this helps teams to recognize issues related to a batch of data faster as the emphasis is limited.
    • Better data security — We can protect our data in the cloud by using a range of safety features such as two-factor authentication and virtual private networks, the best of which provide a greater level of security than is available merely through your ISP. As data security professionals are in constant communication with data integrators and data analysts in a DataOps scenario, data security at each process level is made easier.
    • Improved resource output — When the team conforms to the basic principles of DataOps, seamless communication and transparency between the different teams will help the performance of each team member and lead to better resource output.

    The Lessons We Can Learn from eharmony

    The eharmony membership pool consists of singles in the U.S. as well as more than 200 countries around the world, making up about 5 million visitors per month.

    The company is continuously bringing out new features such as their most recent one-on-one video communication feature aimed at helping their members to maintain meaningful connections in the current era of social distancing. But the company has come a long way since its debut in 2000, learning very valuable lessons along the way.

    Overcoming Several Challenges

    eharmony is leading the online dating industry through innovation by continuously improving its two key applications — Singles and Matching. But they’ve had to make significant changes on their journey to remain at the top. Their challenges included:

    • The IT team needed a considerable amount of time to refresh the database, requiring the involvement of DBAs, storage specialists and network administrators.
    • The labor-intensive process of creating, scanning and storing database snapshots made it difficult to create test datasets for new releases. As a result, QA often carried out tests using stale data. 
    • Database improvements triggered development freezes, which hindered the team’s ability to deliver new features and functionality.
    • Their data volumes were increasing constantly, with several Oracle databases requiring between 20 to 70 TB of storage.

    Combining DevOps and DataOps

    eharmony has also implemented a self-service data management solution called Delphix, which enables the individual teams to develop and update environments as needed.

    Today, their refreshes can be completed within an hour, providing accurate and relevant information to development teams on demand. 

    Other solutions included:

    • They have saved over five thousand person-hours in just three years by automating database provisioning, refresh and rollback tasks. In the process, this has also saved the company over $3 million in operational costs.
    • Through compression and virtualization, eharmony has reduced their database storage footprint to a fraction of the original size, reducing their storage requirements by almost 80%.
    • As development teams adopt agile methods and more frequent releases, demand for non-production environments is expected to increase, further increasing storage cost savings. eharmony has also eliminated data related defects by providing accurate, up-to-date test data. Unrestricted access to test environments significantly improves the ability of QA teams to identify issues before they reach production.

    Why DataOps and DevOps Are a Match Made in Heaven

    It is easy to see why DataOps and DevOps go together so well. DevOps calls for continuous software monitoring, testing and deployment. DataOps are inherently modular because they are designed to execute a single function. Software that is scalable and easily fits into the DevOps framework.

    Using DataOps in the DevOps process will help to fully recognize the benefits of continuous integration by validating the implementation promised by DevOps. These tools can be combined to optimize computing in a way that should be used during development whenever possible.

  • Increasing Software Velocity While Maintaining Quality and Efficiency

    Increasing Software Velocity While Maintaining Quality and Efficiency

    At Compuware, we have been exploring the need for continuous improvement across three key dimensions of software development: velocity, quality and efficiency. Given today’s current market demands and the importance of digital services, this discussion couldn’t be timelier.

    On my previous piece, I focused on quality — how to maintain and improve it as (supposedly) competing demands for velocity and efficiency increase simultaneously. This article will focus on increasing velocity and how it is possible to do so without compromising quality or efficiency. As I discussed in my earlier piece on quality — automation, integration and measurement are once again the key.

    Compuware recently conducted a survey finding the overwhelming majority (97%) of application development managers said IT is under more pressure than ever to deliver software innovation faster. Higher demands for innovation are being imposed universally across platforms, from the cloud all the way back to the mainframe.

    Mainframe software delivery is a particular area of focus, with 95% of respondents noting their IT teams are under pressure to adopt modern development practices on the mainframe in order to accelerate overall software delivery and innovation. Adding to this pressure, organizations reported they expect to increase the frequency of new mainframe application feature deployments by an average of 41% over the next year, compared to 38% for non-mainframe applications. This shouldn’t come as a shock, given the proliferation of modern web-based applications placing unprecedented demands on back-end mainframes.

    Let’s take a closer look at how automation, integration and measurement are equally applicable to the goal of advancing velocity.

    Automation

    Throughout the software lifecycle there are many opportunities for automation — from software design, to development, to build, test, deploy and, ultimately, the production state. The more of these steps that can be automated, the faster developers can work.

    Perhaps the biggest area for potential time savings is testing, because of all the phases of the software lifecycle, testing assumes the most manual labor. The Vanson Bourne survey found test automation to be the single most important factor in accelerating innovation, according to 90% of IT leaders.

    On the mainframe specifically, development teams spend more than half their time on testing during the release of a new mainframe application, feature or functionality. If you can automate more testing, you can roll out applications faster while maintaining (or even enhancing) quality combined with an extremely high level of developer efficiency.

    Integration

    Integration helps software development technologies interoperate with each other. An integrated pipeline leverages existing investments and enables developers to build, test and deploy faster while providing the additional benefit of reducing errors resulting from human intervention (furthering quality). Integration also decreases the manual labor needed to execute and manage workflow within software delivery processes. When these technologies integrate out of the box, setup time is saved and productivity gains can be realized significantly faster. 

    For example, the integration of Compuware Strobe application performance solution with Jenkins, makes it easy for developers to take performance measurements early and automatically as part of a Jenkins CI/CD pipeline.

    Ongoing Measurement

    Measuring velocity is critical. Without measuring, it’s impossible to know how changes are impacting the software development process and what areas need to be improved. Measuring is the key to understanding if there are any activities that can be sped up across the phases of build, test and deploy. This enables development teams to find and address bottlenecks with a great level of granularity. Some common metrics for measurement include mean-time-to-build (MTTB), mean-time-to-delivery (MTTD) and mean-time-to-repair (MTTR).

    Conclusion

    Velocity does not need to exist at the expense of quality or efficiency — on the cloud, the mainframe or any other development platform. Like quality, the key to achieving and improving velocity lies in the disciplines of automation, integration and ongoing measurement. The next piece of this series will explore the keys to maximizing developer efficiency amidst near constant demands for exceptionally fast, high-performing rollouts.