devopsy.kinsta.cloud

Tag: digital transformation

Speed and Security: How to Find a Balance in Development
Time management can help developers realize the necessary combination of security and speed in application development

As software continues to create competitive advantages and market differentiation for organizations focused on successful digital transformation, developers will continue to be measured by how fast they can develop and release code. Business leaders encourage DevOps teams to push the pace of innovation to deliver features quickly while leveraging the newest technologies to gain any available advantage in the market. As a result, development and operations teams work at breakneck speeds to meet the deadlines of short and frequent development life cycles.

Security teams, however, are not measured by speed but rather by certain metrics of success, such as whether they achieve a certain security rating or reduce the number of incidents. Traditionally, security sets corporate standards and best practices centrally and tries to enforce them across the enterprise. With DevOps teams becoming increasingly distributed and decentralized, these conventional security methods no longer work and have actually slowed down security and/or blocked the software development process. Security tries to keep pace, but with numerous disparate reports to review and too many results to manage, they inevitably fall behind. In the rush to catch up, either critical vulnerabilities are overlooked or frustration sets in when releases are delayed.

Developer and security teams both know the importance of delivering secure software but struggle to find the right balance. Let’s take a look at some strategies that can help ease the tension between DevOps and security teams and create a happier work environment, where safer code is delivered to the market.

Incorporating Time Management for Speed and Security

Consider implementing the following time management strategies to make life easier for developers:

Integrate AppSec in the DevOps Pipeline: Application security scanning should be incorporated early and often within the DevOps pipeline. While this statement may sound like a broken record, the days of running scanning tools independent of your pipelines are gone. Studies have shown the earlier you detect an issue, the easier it is for developers to fix—with as much as 50% time reclaimed over late-stage remediation. By including AppSec in the pipeline, you ensure all issues are identified as quickly as possible. This effort will not only improve the quality of your software but also free up more time overall for developers to write code and create cool features. Sounds like a win-win for both teams.

Incorporate automation as much as possible: DevOps teams should look to leverage automation when implementing application security wherever possible. Many of the steps for securing your software development process are repetitive. For AppSec to be successful, the steps must work within the developer’s world and along with the other tools developers love. Developers look for tools that help them move faster.

Here are some areas where application security automation can unburden developers from repetitive tasks:
- Use AppSec tools that can be easily tied into your DevOps toolchains. As a matter of fact, a developer may not even need to know what security tools are being run, only that they are executed on every run of the pipeline.
- Automate the ingestion and correlation of the security findings. There will often be a boatload of redundancies between the findings from different security tools (i.e. SAST versus DAST versus SCA) and the potential compression ratios can be in the order of 10-50:1. There are tools to automate this process and dramatically simplify making sense of all the data. The onus shouldn’t be on the developer.
- Once a security issue has been identified, it should automatically create a ticket in the ticketing system that developers use today. We all know developers don’t like being forced to use a new tool. Give them the findings in the system they are already familiar with.
Tailor security training for your teams: Security training is a great way to reduce the number of security findings within an organization, which in turn frees up developers to work on other pressing issues. However, most security training is ad hoc and not tailored to meet the needs of the development teams. Even the most basic security testing can provide baseline metrics that enable security teams to recommend tailored training for development teams. These sessions can focus on areas that teams struggle with or areas known to be susceptible. Training is also a great way to develop security champions within the development teams, dedicated professionals who have shown to improve the overall security posture and awareness of organizations. Security champions—and the programs they advocate—have the power to improve both AppSec and relationships among people, a sentiment endorsed by 84% of industry professionals.

Summary

Developers know the importance of security and truly want to deliver safe code in the timeframes they are being judged on. By incorporating the strategies outlined above, they will have a fighting chance to reach the lofty goals set by the business, while releasing higher quality and more secure code. Security teams need to meet the developers in their world and help enable application security to be automated and work at the speed of DevOps.
December 11, 2020
Open Source: What IT Pros Wish the CIO Knew

Using DataOps and open source together, organizations can reimagine the way they deliver data and software-driven projects

Real-time data is the fuel that powers today’s most critical systems and applications, as well as the digital transformation initiatives organizations need to compete effectively. For the people on the front lines building them, having the right technology alone is not always enough to succeed. To make the most of these game-changing technologies, you also need the right tools and the right strategy in place.

If you’re not maximizing use of real-time data, you’re at risk of being disrupted by a competitor. For this reason, C-level leaders often adopt cutting-edge open source technologies that come with risks, making real-time data applications complex to build, deploy, operate and govern, and require a major investment in time and resources upfront to unlock their potential. You won’t get the most out of open source unless they can be operated and data can be accessed by a wider range of people outside engineering.

For example, open source data technology is often lacking, and documentation is sometimes sparse at best. You may need to ask your data and platform engineers to spend weeks or even months building custom management tools before you can get to work on your first streaming application. That kind of delay can seriously impact your time to market.

How Secure is Your Solution?

Streaming data projects with multiple open source components can also introduce security and compliance issues into the picture. Even if you’re working with the very latest technology, open source often lacks built-in capabilities for security and compliance—especially if they are industry-specific for use cases such as health care or financial services. In fact, a recent survey by Veracode found that 70% of applications have open source security flaws, and 47% of those flawed libraries end up in code indirectly. If your project suffers from increased risk due to security and compliance issues, it can also lead to additional delays in projects, further slowing your time to market.

In a world where the threat landscape is constantly evolving, your organization may wish to opt for a commercial offering for some types of use cases, especially if it includes the features you need to align with very specific compliance and governance needs.

Creating a Culture of Engagement

Arming IT professionals with the right tools and technologies they need is essential, but organizations also need to be sure they have a strategy in place to retain the best people—and keep them productive and satisfied. Fostering a workplace with the right level of engagement can provide a strong start. A recent Forbes article cited studies finding that highly engaged employees result in 21% greater profitability. On the other hand, the same article reported that 67% of employees reported feeling burned out at work—and were 2.6 times as likely to be looking for a different job.

Open source tools are a great way to attract talented engineers because people love working with the latest technology. But as you give your IT professionals access to bleeding-edge capabilities, the complexities and time spent working with them increase. The right tooling can help manage the shortcomings of these technologies and help retain the talent you need by encouraging a positive work-life balance and lead to more successful implementations. If your engineering team isn’t spending time fighting fires and consumed by open source technology, there’s more time to focus on bigger picture issues such as a well-defined career path that keeps team members engaged and more valuable to the business in achieving its goals. With the right level of work-life balance and keeping employees involved and satisfied on the job, you can be more confident that your best engineers will become more than technologists and are strategic business partners who won’t be spending their spare time dusting off their resumes.

Getting the Best of Commercial and Open Source

How can you enable IT professionals with the tools they need to deliver their projects on time and on budget while empowering them with a more strategic role? Employing a DataOps approach to your initiatives can provide a strong start. Spanning many disciplines and skills across the organization, DataOps encourages an environment where people can collaborate, reduce process complexity and fully optimize the use of their data. DataOps allows teams outside of IT and engineering to operate their data directly, without being dependent on a small engineering team. It enables business stakeholders in your workforce to get closer to data and bring their own knowledge and context to data initiatives and improve business outcomes.

For example, leading social and mobile gaming company Playtika needed to provide better visibility data streaming across Apache Kafka, its primary platform for the exchange of information between its microservices. Using a DataOps approach and platform, the company provided its analysts, product owners, operations and developers access to a fully audited GUI and CLI to explore and operate data in a compliant way without relying on data or infrastructure engineering. Providing direct access to data enables 600 developers to save 30 minutes a day as they work on delivering new features—which led to faster and more innovation when it came to developing new games.

When utilized together with a commercial solution based on open source roots, DataOps helps you get your business and technical people on the same page, so your organization can achieve closer alignment to strategic business objectives, with data serving the business. At the same time, you can free up technical staff and developers from tasks such as tooling, and enable them to utilize open source for R&D and other innovative priorities.

Using a collaborative DataOps strategy, your organization can reimagine the way it delivers data and software-driven projects, while also embracing new open source technologies.

December 9, 2020
Governance, Quality Critical in the New Digital Norm

The shift to digital is certainly not new. But the speed of digital adoption has been accelerated as organizations react to the COVID-19 pandemic and implement remote working and contactless or self-service channels for customers. What’s more, the evidence shows that trend is here to stay.

Analyst Gartner’s CIO Agenda 2021 report shows that 76% of CIOs reported increased demand for new digital products and services as a result of the pandemic. But far from being a temporary spike, 83% of CIOs said they expect that to increase further in 2021. There was also a 65% increase in the use of self-service by customers during the crisis, and 79% of CIOs expect that figure to grow next year, too.

These trends all put increasing pressure on DevOps teams to drive continuous improvement and development of products. Some 83% of developers say they are releasing code faster and more often than ever before, according to the “2019 State of DevOps Report.”

But with the pressure to develop faster, there is a battle to find the right balance between speed and quality. Move the needle too far toward either end of that spectrum and the results can be equally disastrous for the business.

In Digital, Quality Matters

Rushing releases and updates out too quickly can lead to poor quality or defective code. If this impacts the user experience, it can have a significant impact on adoption and ratings. A report on commercial and retail app testing by Testlio, for example, shows that 50% of users won’t download an application with a three-star rating from an app store. And that rises to 85% for an application with just a two-star rating.

In the second annual OverOps DevOps survey, over half of respondents said they encounter critical or customer-impacting issues in production at least one or more times a week. And more than 40% of critical production issues are first reported by end users or customers rather than by internal monitoring and alerting tools.

But 45% of developers in the same study said they are under pressure to move fast, which doesn’t give them enough time to ensure the quality of the code they are producing.

Manual Bottlenecks

While DevOps has tools to develop, test, ship and containerize software, there is often a lack of governance and orchestration. This makes it difficult to get that balance between speed and quality.

The major challenge for DevOps is the time-consuming and manual nature of testing and QA, which causes new releases and updates to be either delayed or of poor quality. The testing life cycle remains highly manual and updates must be tested across all devices, platforms and networks.

More than half (55%) of respondents to Kobiton’s “State of Test Automation” survey for 2020-2021 said automating testing would improve software quality. Yet just over three-quarters (76%) said they still conduct most of their testing manually.

And it is this lack of automation and orchestration that is really holding back DevOps efforts in many organizations today. In a survey by research analyst Forrester, “Face the Workflow Automation Gap Head On,” two-thirds (67%) of IT leaders said the lack of application workflow orchestration leads to services not being completed in time and 55% admit that it takes valuable time away from strategic initiatives.

Managing the DevOps Workflow

The key to ensuring better governance and quality in DevOps and overcoming the manual effort and time pressure is to integrate automated workflow orchestration into the software development lifecycle process.

This orchestration enables DevOps teams to test early and often, and ensure new releases and updates are ready to go to a live production environment.

For example, modern workload automation tools allow IT ops teams to easily standardize processes in development, test and deployment with the help of built-in wizards and templates. And rollback capabilities and automated process documentation take much of the pain out of managing DevOps processes. Workload automation also provides greater coordination with highly configurable automation that only runs processes when a certain set of conditions is met.

Ultimately, this helps DevOps teams automate and streamline the QA and testing processes so changes are carried over into the production environment without developers having to manually transfer new code from the testing environment.

And that means DevOps teams can automatically enforce governance and standards and maintain that essential balance between speed and quality in today’s digital norm.

December 2, 2020
To Accelerate Digital Transformation, Accelerate Data Transformation

Successful digital transformations are not possible without successful data transformation. After all, it’s data—and the ability to analyze that data—about an organization’s customers, employees, infrastructure, market dynamics, supply chain, business-technology systems and more that make digital transformation possible.

However, last year Gartner said that “information as an asset” is still in its infancy. “Data and analytics are the key accelerant of an organization’s digitization and transformation efforts. Yet today, fewer than 50% of documented corporate strategies mention data and analytics as fundamental components for delivering enterprise value,” the analyst firm wrote in its report, “Why Data and Analytics Are Key to Digital Transformation.”

Further, Gartner predicts change is coming and quickly: Gartner says that 90% of corporate strategies will unequivocally list information as a critical enterprise asset and analytics as an essential competency within two years.

Getting there and being successful with data and data analytics won’t come without significant challenges.

According to a survey released by IDG (sponsored by Matillion, a cloud data transformation provider for cloud data warehouses), the challenges are steep. The survey polled more than 200 IT, data science and data engineering professionals at North American organizations with at least 1,000 employees.

What are the challenges enterprises are facing? According to the survey results, 47% said data control issues are the biggest challenge to data analytics projects. In contrast, other top challenges indicated are lack of a scalable, reliable technology platform to process large data sets (45%), having too many manual processes (38%) and challenges cleansing and preparing data (36%). Additionally, 40% cited a lack of visibility and control of data silos as a challenge when collaborating with business users.

With that, it’s not surprising to learn that it takes on average one week to pull together and prepare data for useful analysis, and 45% of all time spent on data analytics initiatives is consumed by data preparation. About 60% said they lack the scalability and flexibility they need when preparing data for analysis.

It was almost unanimous when it came to the number of companies that sought ways to accelerate their approaches to data analysis improvement.

So how are enterprises accelerating their efforts to improve data analysis to better power their digital transformations? Respondents said data portability (57%), ease of onboarding (57%) and cost-effectiveness (52%) are the top features of an analytics platform that can help them move past current obstacles. The IT professionals surveyed favored user-friendly (50%) and easier connections to data sources (50%) as top features; however, data engineers, data scientists, data architects and others cited time-to-value (57%) and ability to provide self-service (51%) as essential capabilities.

A sizeable portion of those surveyed indicated they are already on their way, adopting the technologies and processes they need to improve their ability to analyze their data for better decision-making rapidly. According to the survey results, 38% are already using cloud data warehouses. And in the future, 43% said they expect to have all of their data in the cloud, with 57% planning to use both cloud and on-premises data strategies.

Those that master their data analytics more rapidly may enjoy an early mover advantage. “A company’s ability to compete in the emerging digital economy will require faster-paced, forward-looking decisions,” said Douglas Laney, distinguished VP analyst at Gartner. “Data and analytics leaders need to assert themselves into corporate strategic planning to ensure that data and analytics competencies are incorporated within the highest-level public-facing enterprise plans.”

November 25, 2020
Buildkite Tightens CI/CD Integration With GitHub

Buildkite has tightened the integration between its namesake continuous integration/continuous delivery platform and the GitHub source code repository.

Fred Showell, a senior product manager for Buildkite, said the integration will accelerate application development efforts using the Buildkite application in GitHub. Any connected GitHub account can be selected as a source when creating Buildkite pipelines, he noted.

Repository webhooks can also be created automatically, making it faster to kick off builds on a Buildkite pipeline from pull requests and then feed build information back into GitHub using the Status application programming interface (API).

The bulk of organizations that employ Buildkite are software vendors that are continuously updating software-as-a-service (SaaS) applications. Most of those organizations routinely employ software artifacts that are stored in the GitHub repository.

While GitHub continues to expand its DevOps capabilities in lockstep with its parent company Microsoft, Showell said there’s plenty of opportunity to extend capabilities of frameworks such as GitHub Actions, a set of APIs that DevOps tools can invoke to automate workflows.

In general, Showell noted the number of SaaS-based platforms continues to expand as organizations embrace cloud applications to drive digital business transformation initiatives. Some of those efforts are now also expanding to include use cases involving edge computing initiatives, which is expanding the reach and scope of the Buildkite platform, he said.

Of course, many enterprise IT organizations are now applying many of the same best DevOps practices employed by SaaS platform providers to their own application development efforts. Many of those organizations are realizing they are essentially software companies that also manufacture something or provide a service within the context of a specific vertical industry. As such, the number of organizations embracing best DevOps practices should substantially increase in the months ahead.

Most of those organizations are already trying to manage a large portfolio of applications. Digital business transformation initiatives may allow them to in time retire some of those applications, but chances are good legacy applications will need to be maintained for some time until the digital process that has been put in place proves effective. It’s likely then that sound software development lifecycle management practices will be more relevant than ever.

Competition among providers of CI/CD platforms to provide those capabilities is already fierce. Fueled by massive amounts of capital, providers of these platforms are expanding the scope of their offering to address a market that is expected to be valued in the tens of billions. Buildkite itself raised $28 million in additional funding last summer.

It’s too early to say how the next phase of the DevOps platform wars might play out. However, it’s apparent that when it comes to DevOps platforms the level of investment required just to keep pace is substantial.

November 20, 2020
3 Steps to Turn a Data Deluge Into Actionable Intelligence

Digital transformation is creating even more data for organizations to manage effectively. Here’s how to find the intelligence needle in the data haystack

Data and analytics transform how companies across all industries identify and act on opportunities to gain a competitive advantage. But many organizations are discovering that ever-growing volumes of the former make the latter increasingly difficult. They struggle to turn raw data into a strategic asset that informs decision-making and accelerates growth. Overcoming that struggle requires conceiving and implementing an effective data transformation initiative. Fortunately, there are three steps to implementing sustainable initiatives that any company can follow, no matter its maturity or size.

But before taking the first step, it is vital to secure the commitment at the board and management levels that the initiatives will be a companywide priority. That is critical to ensuring the allocation of adequate resources and budget dollars at the outset. Failure to secure senior leadership’s enthusiastic support will impede progress if not completely derail the entire project.

You might not think that would be a difficult sell. But research firm KPMG found that the opposite is true when it examined the fundamental role data analytics will play in the financial services sector over the next 10 years:

“(Many CEOs are) responding incrementally and, at best, tactically with better front-end digital services for customers or process improvements in the back-office. But they’re not addressing the more fundamental issue – that their very business model will need to change. And fast.”

If you encounter resistance, here’s the crux of your argument: Your company is a data company. It doesn’t matter if you sell software, cars, clothing, food or any other product or service; the ever-growing volumes of data you’re collecting inform all business decisions. Or at least, they should.

Curt Hopkins, editor in chief for Hewlett Packard Labs, summed up this point perfectly: “Data has become the new gold that backs the value of companies, though it is a gold that companies must locate, mine and refine before spending. Business has shifted from making things to knowing things, so realizing usable data is the order of the day if a company wants to remain profitable.”

Step 1: What Data Do You Have?

Once you have secured the full support of management, you’re ready to take the first step to launch a data transformation initiative: gain an understanding of precisely what you have and where it “lives.”

Take the time to conduct a thorough exploration of all your raw data sources and where those volumes are stored—likely some combination of individual users’ laptops and other devices, on-premises systems and in the cloud (public, private or hybrid).

Next, determine who has access to your data and how. If you decide to work with an analytics services provider, do you have the necessary extraction processes in place to deliver raw data to your partner? If not, you will need to build that pipeline or move the data into a cloud-based analytics database.

Step 2: What Do You Want to Accomplish?

Once you know what you have and where it is across your organization, the next step is to formulate your initiatives based on your objectives. Classifying all datasets under two broad categories—internal use or external use (i.e. commercialization)—will help narrow down the relevant data for a specific campaign.

Embed responsibilities for secure data handling and usage across all parts of the organization—don’t put them only on the technology teams’ collective shoulders. Assume you will need to organize, transform and enrich data to realize its full value. Remove any barriers to sharing data under NDA and considering a nuanced build-versus-buy approach. Finally, ensure the initiatives map to management by objectives (MBO) to maintain a focus on using data to help the company achieve its business goals. At this point, you’ll need to answer the build-or-buy question to determine if you have these transformation capabilities internally or need to find them externally.

Step 3: How Will You Track Progress and Evaluate Outcomes?

After executing the data initiative, you should regularly measure its outcomes. Hold your teams accountable by setting time-based goals and tracking progress versus plans. Anticipate the need to iterate because harnessing data is a fluid process. Create and maintain a dictionary to keep employees and teams aligned as they progress on their tasks and projects.

One additional and no less critical consideration is to ensure you strike a balance between gaining the ability to use data assets with protecting them from security threats, both cyber and physical. According to “The Cost of Insider Threats Global Report 2020” by IBM and the Ponemon Institute, the average cost to recover from a breach is between $3 million and $4 million, and that does not account for the loss of customer trust and brand loyalty. Protecting customer and employee personally identifiable information (PII) and all other sensitive data types and complying with all data rights agreements is paramount.

Fifteen years ago, British mathematician Clive Humby famously proclaimed, “data is the new oil,” and the analogy holds up today. Like the raw crude that comes out of the ground, raw data only represents potential value. Just as oil has to be refined into fuel to become useful, data must be analyzed and refined to become functional.

By understanding your supply of raw data, formulating your initiatives and executing and measuring those initiatives, you will be better able to identify opportunities and make more informed decisions that give your organization a competitive advantage.

November 18, 2020
Survey Sees Pace of IT Investments Continuing

A survey based on responses from 230 IT operations and DevOps executives in the U.S. and UK with at least 500 employees and $5 million in annual IT budgets published this week indicates large enterprises are continuing to make IT investments despite the downturn in the economy brought on by the COVID-19 pandemic.

In the second and third quarters of 2020, 60% of IT leaders significantly or moderately increased their annual technology budgets, according to the survey results. In contrast, 22% reported their organizations significantly or moderately reduced their IT spending during the same period.

Conducted by OpsRamp, a provider of a software-as-a-service (SaaS) platform for managing IT operations, the survey also finds nearly two-thirds of respondents (63%) have accelerated or maintained digital transformation initiatives in response to the economic downturn brought on by the COVID-19 pandemic.

Cost optimization, however, remains a major focus. IT leaders report they are using self-service tools (60%), open source software (51%) and rationalizing IT vendors (51%) to contain costs.

Financial managers who can help IT teams present a compelling business case for technology investments are the highest hiring priority (54%), followed by senior IT leaders who can execute digital transformation programs (47%) and cloud operators (44%).

From a technology perspective, top priorities are information security and compliance (59%), remote work and collaboration (55%), public and multi-cloud infrastructure (50%) and monitoring and management (42%).

More than half of respondents (57%) said they are also employing artificial intelligence for IT operations, while 50% have adopted both digital experience monitoring (50%) and network performance monitoring and diagnostics (50%) tools.

Bhanu Singh, senior vice president of engineering and DevOps for OpsRamp, said the survey makes it clear that IT spending in large enterprises is rebounding from last spring when the pandemic first hit. Overall, IT spending for the year will still be down from its historic highs, but larger organizations are prioritizing technology projects that enable the business to adjust to fundamentally different market conditions, he said. Most of those projects involve applications being deployed on cloud computing platforms using best DevOps practices, noted Singh.

Less clear is what the level of investments will be following the U.S. presidential election. Many business leaders are waiting to see what the stimulus spending might be approved by the new administration and Congress. That may not be known until a runoff in December for two Senate seats in Georgia, which will determine which political party controls the U.S. Senate.

In the meantime, it does appear that larger organizations are investing in IT projects they expect will either enable them to remain competitive or pick up market share at the expense of smaller rivals that may not have as many resources to invest in IT. Historically, smaller organizations have tended to be more nimble when it comes to leveraging IT to gain a strategic advantage. However, in an era when cash flow is constrained many smaller organizations may have to be more judicious when it comes to spending on IT.

Regardless of the outcome, the one thing that is certain is IT has never played a more crucial role in determining not only which organizations survive but ultimately thrive.

November 13, 2020
Accelerate Digital Transformation With Low Code

This year many organizations have gone through a lot of changes due to the COVID-19 pandemic. Businesses need to be more agile, adapt to the new normal and accelerate their digital transformation. We have a new future of work we need to be ready for, and that requires more automation than ever before. Low code enables organizations to accelerate their innovation, automation and transformation efforts. Low-code platforms enable developers and non-technical teams to quickly deliver applications by automating the creation of apps with a drag-and-drop interface.

In this interview, ServiceNow CIO Chris Bedi shares the main benefits of low code and talks about the ServiceNow Citizen Developer Program.

Chris is one of our distinguished keynote speakers presenting at Low Code Today Virtual Summit. During his keynote presentation, Chris will share a CIO perspective on rolling out low-code applications at scale, best practices from rolling out ServiceNow Citizen Developer Program and suggested guardrails to ensure a successful and secure implementation.

To find out more about the future of work with low code and how it can benefit your business, attend Chris’ session at Low Code Today Virtual Summit on Nov. 19. To check out the full agenda or to register for free, visit the Low Code Today website.

November 12, 2020
DataOps: The Key for Real-Time Data Application Development

Data engineers and developers face challenges every day to help their organizations digitally transform. To do this, they must deliver real-time data applications faster, better and cheaper. With businesses in every industry now data-driven, data professionals must work more efficiently and accelerate time to market for their products. That’s where DataOps comes in.

Development of real-time data applications slows down when strategic business project teams are dependent on small, centralized engineering or IT teams for basic data operations such as exploring data or even building most data processing applications. Needlessly, engineering teams are notorious for being a bottleneck for strategic digital transformation initiatives. Rather, they can become known as strategic project enablers that provide the best tooling to their customers—instead of building it and being known as gatekeepers.

Business users, outside of IT and engineering, need access to data but lack the necessary deep technical knowledge to work with the latest data technologies. DataOps brings people closer to data, helping organizations accelerate the delivery of their strategic projects, reduce costs and minimize the need for hard-to-source engineering skills.

To accelerate digital transformation initiatives, teams and organizations should onboard new users directly onto a data platform quickly with the right tooling and resources required to get to work on their projects. DataOps empowers these business-focused users to work with data tools on their own to self-serve. A DataOps approach makes it easier for more people across an organization to work with its data, resulting in better alignment for data engineers and developers with their business and technical partners and improved data access.

Serious new security, compliance and governance risks are introduced when less-technical users increase their interaction with technology, such as onboarding to a data platform. More people now understand that personal data has tremendous value. Initiatives such as the General Data Protection Regulation (GDPR) and California Consumer Privacy Act (CCPA) have brought the need to respect data privacy to the forefront. The stakes for safeguarding data are increasingly high and regulators are unforgiving—costing some organizations several billion dollars in fines, lawsuits and loss of market cap for some data breaches.

When providing access to data, stringent management, control and real-time monitoring must be applied from a governance perspective. A surprise compliance auditor visit can result in questions such as which applications are reading credit card data, how an organization is keeping customer data safe, who has access to data and what are they doing with it. While locking down for compliance, you must help your business stay agile and avoid any slow down in data initiatives.

Build vs. Buy to Streamline Operations

In addition to adopting a DataOps approach, a key step is to onboard users to a data platform. This will optimize your data strategy and will help take the worry out of managing data projects, putting you on a path to faster execution and accelerated time to market.

When onboarding users to a data platform, you need to choose between building your own or buying. It takes time and resources to develop your own system to onboard users and clients and can take years to build your own technology in-house. If your engineers are spending all their time working with open source technology to build tooling for onboarding, they’re not focusing on better ways to use data and get solutions to market fast—which is an inefficient approach.

When your users are onboarded, the cycle begins again, as users request access to more data sets. More time at the command-line console means less time for your engineering teams to focus on higher-value work and strategic initiatives.

Get a Leg Up on Competition With a DataOps Approach

Buying a solution requires an upfront investment to support onboarding and, when combined with DataOps principles, is formidable to remain competitive. Together they enable support and manage the onboarding of clients to a data platform at scale and to maintain security and compliance. A DataOps approach fosters improved communication between data and business professionals to share data more effectively and to build a data mesh architecture.

A Swedish digital bank, Avanza, needed to ensure the continuous improvement of the customer experience of its digital services to remain the leader in the Swedish market. Doing so required giving developers direct and constant access to real-time business data in Apache Kafka.

The organization applied a DataOps approach and intelligent data masking rules to meet GDPR and protect customer confidentiality, featuring a large single data platform to address how it delivered data access and applied data governance to support more than 20 strategic projects. Using the platform, tenants who build streaming applications gained visibility into the health of their Kafka clusters and improved insight into data flows. This self-service approach means fewer support calls to data engineers, minimizing bottlenecks and freeing them up to work on more strategic initiatives.

Data is the fuel that propels insights by tying back to analytics, decision-making and business systems. The better the shared data, the more it can be applied to supporting better business decisions, better customer experiences and execution of strategic business objectives and goals.

DataOps also builds compliance and governance into your processes. It provides the visibility needed to understand who is accessing data and how to maintain governance and ensure people are acting ethically with the data they are enabled to access.

The daily challenges faced in data-driven organizations only continue to grow. A DataOps approach helps data engineers and developers meet these challenges to digitally transform and build real-time data applications that help organizations gain a competitive business edge in their respective marketplaces.

November 5, 2020
COVID-19, Digital Transformation and Teams: Q&A With Red Hat’s Kevin Behr

Even prior to the novel coronavirus pandemic, enterprise teams found themselves strained as they pressed to move digital transformation goals from whiteboard to reality. Then along came the big push to work from home and many transformations that were planned to occur years from now were put into action. So how are enterprise teams faring in their mission? There are a lot of claims of success, yet surveys continue to show many, if not most, enterprises fall short.

To get a sense of the impact this is having on staff, and gain insights into what they could be doing better, we reached out to Kevin Behr, global senior director, global transformation office at Red Hat. Kevin is also the chief scientific officer at PraxisFlow and the founder of the Information Technology Process Institute (ITPI), and a co-author of “The Phoenix Project” and “The Visible Ops Handbook.”

devopsy.kinsta.cloud: Has the push this year, largely driven in response to the novel coronavirus, for organizations to rapidly “digitally transform” themselves had a substantial impact on staff and teams?

Kevin Behr: Yes, and one of the things that can be depressing is that management does not always understand how to manage during times of change and adapt their ways of working within the new economy. They are behind. Way behind.

Kevin Behr

And they think they can cost cut their way to success. During many transformations, they will try to cut the cost of their infrastructure to pay for the transformation. And they basically gaslight everybody in the organization that has to operate. It means management essentially just gave operations teams more things to do, more quickly, while deferring many of their priorities that, in all likelihood, contribute to reliability. And then management tells them that they’re going to have to cut costs in operations because there will be more things they have to do and manage.

Makes perfect sense, right? Of course not. Anyone who understands change theory understands that there is always a hump to get over when one changes behaviors. And that’s the other problem: When a transformation does not result in behavior change it burns people out.

Finally, many times people are saying one thing about the success of the digital transformation in public while they know the automation is just a series of Mechanical Turks and scripts executed between swivel chairs.

devopsy.kinsta.cloud: How do such organizations allow themselves to grow so dysfunctional?

Behr: A lot of that has been by leadership not framing their efforts correctly. I think that really burns people up because they get put into efforts, especially in operations, efforts that they know have no chance of succeeding—but they can’t ever say that because the organization only embraces happy talk. But it’s toxic happy talk and “toxic positivity” everywhere. And I think toxic positivity endlessly burns people out, especially when it reaches the point where one can’t actually say anything against it because they’re just viewed as being inherently negative.

People get disillusioned and competent people just sink. And then there are organizations with so-called meritocracies. Meritocracies can be unfair because it’s the privileged who decide what has merit. The key is who decides where merit resides. I think sometimes in a meritocracy, there can be a continual skewing of merit to the same people or to the same ideas and others begin to believe that they don’t fit and feel marginalized. And, when combined with toxic positivity, the organization becomes something people just can’t crack through. It is impenetrable.

This also happens at places that are, in theory, top-down. When you watch who decides has merit, it doesn’t have to be a hierarchical top, sometimes it’s influencers.

That just becomes impossible for people to manage their way through. They have to plan to influence a ton of people in the organization just so they have a shot at making something sane happen. They conclude it’d be easier to get another job.

I think it’s that combination of such organizational stupidity and the opportunities in the job market that give people more choices. I’ve just been blown away by how many smart companies are picking up talent during COVID right now and watching all the career changes happen with great people in our industry. People getting better lives.

So, when you’re an organization that doesn’t allow you to improve your own value, let alone improve anything in your organization, that’s a no-no. It’s a lose-lose, and people leave.

devopsy.kinsta.cloud: What impact do you think this has on enterprise transformations?

Behr: I think company actions speak louder than words. And I think a lot of these actions are very hurtful to people in technology, especially those who work in infrastructure operations. Operations is just under the scalpel’s blade all the time. And I think it’s blown up most of our transformations.

There are a lot of problems with transformations, generally. One of the areas is how we love to talk about all the happy talk about the future and serverless and all of the pizza box teams and product line orientation versus project orientation, and all of this stuff that they’re actually just distracting themselves with.

Yet, the failures come down to the fact that organizations just don’t do what they say they do and people get hurt. And reality catches up to them: that teams actually need to support all of this transformation so that they can literally run the applications and resources. Eventually, organizations have to literally deal with the fact that all they have to operate the infrastructure that actually runs all of the business and systems of record. And they realize that, despite all of the happy talk, much of the transformation was really playing around the periphery.

I think organizations may get this right over the long term. But they have to learn the lessons from what they are experiencing now. I don’t think employers understand it well enough, yet. I think they have an understanding regarding the talent problems they have. But they don’t get that the way they solve their talent problems isn’t through hiring. Long term they have to invest in their people, and they need to learn this lesson.

November 4, 2020