devopsy.kinsta.cloud

Tag: digital transformation

Becoming a Data Company: Back to the Future

Transformations are complex, and it’s often difficult to see the big picture when you’re in the middle of one.

The elegance of the end-state of harmonious synergy between software and data belie the complexity and chaos of the transition we find ourselves in the midst of. Not that long ago, there was a general balance between the pace and sophistication of software delivery and the speed and maturity of data analysis. Markets and customer bases grew steadily allowing organizations to keep pace in their software and data competencies to handle increasing mutual scope and complexity.

However, the consumerization of IT—the ability of lay consumers to consume sophisticated software experiences from browsers and mobile devices—unleashed access to a massive, eager and demanding customer base measured in the billions. This gave birth to the current wave of digital transformation. Along with it came the race to get to market, i.e. build and deliver high-quality software exponentially faster, in order to capitalize on massive global opportunity.

Innovation thus started in the software domain. With faster, better software came the need for purpose-built data stores (document databases, graph databases, etc.), the need to manage the velocity, volume and variety of data collected from millions (or billions) of users and the need to extract insights and value fast enough to further fuel the pace of software innovation. Thus, software innovation planted the seeds for the current focus on data innovation.

While the innovators quickly traversed this software journey, most enterprises are only now fully in the midst of embracing new software methodologies and technologies to deliver software quickly and effectively. Those enterprises at the vanguard of the software journey are now embarking on the data journey—to find ways to safely and quickly harness the value locked away in enterprise data assets. This includes everything from analytics to rapid provisioning of copies of production data for software testing. In fact, 451 Research states 86% of companies that consider themselves mature in DevOps journey are planning on increasing their investment in DataOps. That is, companies that have traversed the software competency curve are now embarking on the data competency curve.

The Dawn of the CDO

The growing traction by data science, AI, analytics and DataOps technologies and vendors point to the rising importance of developing a data competency. The recent emergence of the Chief Data Officer (CDO) is the culmination of this trend.

Given the nascent focus on data at most enterprises, most CDOs are still grappling with defining their role, navigating the dynamics of the C-Suite, defending against regulatory and compliance risk and justifying their role by identifying growth opportunities. Despite these immediate challenges, they do have one thing going for them: they know they’re in the midst of mind-bending change, have the benefit of hindsight on the software journey and the many parallels it carries, and are mentally prepared for the organizational, technological and cultural changes that will need to be navigated. Most companies were stunned by the sheer suddenness and velocity of change that came with the DevOps and the software transformation journey. With DataOps and the data journey, they are now aware, prepared and committed to the associated journey and changes for the long haul. They can be proactive and avoid being reactive.

Planning for the DataOps Era

As cycle times for going to market and learning from the markets shorten, and innovation cycles accelerate, CDOs can expect that data-centric organizations will need to drive net-new top and bottom-line results without incurring undue risk. This will come through adoption of new technologies, organizational structures, cultures and workflows. New roles focused on data-centric activities, and new forms of collaboration between different personas will emerge.

In the modern enterprise, production will no longer be the most important data environment in enterprises. There is a growing need for fast access to secure, governed data outside of production—where the work of the rest (non-IT part) of the enterprise is centered. What used to be non-prod for IT will now be prod for the rest of the enterprise. To succeed, IT will need to deliver data quickly and securely to the rest of the organization or even across organizational boundaries to partners and other ecosystem players. Embracing collaborative processes and workflows that support this new pattern of enterprise operation will be critical. Enter DataOps—the unique mix of people, process and technology approaches required across teams in an organization to become agile with data.

Given the similarities outlined earlier of the data journey to the software journey, organizations’ DataOps journey can benefit from the lessons learned in their DevOps journey. For instance, developing a data competency will require new layers of organization and workflow patterns supported by corresponding technologies. Proactively planning and laying down infrastructure for DataOps will yield far better results rather than adopting technologies ad hoc until a data competency blueprint emerges through trial and error.

Building a Data Company Technology Stack

As with any complex shift, it makes sense to deconstruct data competency into a series of layered capabilities, each building on the one that came before, that together help develop a sophisticated competency. For example, when building their software competency, in order to apply software in sophisticated ways, (e.g., cloud native architectures and microservices), organizations needed to manage distributed software elements (e.g., scale, heal, monitor) which in turn relied on the ability to test and deploy incrementally, consistently and rapidly.

A similar approach can be applied to data. Building collaborative processes around appropriate technological tooling can yield layered component capabilities that collectively define a sophisticated data competency. While the layering was emergent in the software journey, CDOs have the opportunity to be deliberate rather than reactive in developing a layered approach as the diagram below illustrates.

At the foundation of a successful data competency lies the capability to deliver data anywhere in the organization and secure it to meet regulatory and compliance mandates. Being able to quickly secure and move any type of enterprise data across a hybrid cloud context is hard work, but also table stakes for basic data management hygiene. These are critical prerequisites and this foundation needs to be solid in order to support the layers above.

Next, there are various data management activities to master, such as the self-service access, governance, data catalogs, data cleansing, collaboration and versioning, etc. These define the next level of data competency and must be mastered before data can be applied in different ways, e.g. in machine learning, test data management, analytics, data science, etc.

A thoughtful, layered approach to building a data company requires building the right abstractions between layers and will avoid tool sprawl consisting of a mesh of ad hoc point solutions that inevitably require human glue to drive data workflows.

CDOs should evaluate their organization’s needs through such a structured lens to evaluate the organization’s needs. Each of the activities will bring with them ecosystems of vendors, engendering further evaluation of vendor capabilities. Critical questions to ask include (among many): Is the solution enterprise ready? Does it work with all your data sources across hybrid cloud? Will it evolve with your changing needs? How will it interoperate with other solutions? Can it be automated?

As CDOs think through these infrastructure investments to connect different parts of the enterprise, the similarity to laying down railroad tracks to connect a large country are unmistakable and perhaps instructive: the resilience and stability of lower layers enable flexibility, variety and innovation at the higher levels just like the simplicity and reliability of simple railroad tracks enables running electric, steam engines, luxury bogies and cargo bogies alike.

Toward a Data Company

The refrain of digital transformation has been that every company is a software company. But to complete the transformation, every company must also become a data company. Software development practices and tools have evolved significantly in the last decade. We’re in the early stages of a similar evolution in data practices and tools that will take us through the next decade. The symmetry and coupling between software and data is unmistakable and there is much to be learned from the software journey.

As CDOs, CIOs and CEOs plan the transformation of their enterprises in this new era, they will need to consider building a foundation for their data competency that is stable, resilient, compatible with their software development processes and adaptable to inevitable change. They should be prepared to deliver their data from where it is produced to where it is used throughout the enterprise.

The data company is an organization where data is treated not just as a byproduct of operations, but as the lifeblood of the organization—that drives software development, improves quality assurance, informs decision making and potentially generates revenue in its own right.

— Karun Bakshi

December 4, 2019
The Road Ahead for Security, DevOps Transformation

Qualys hosted its Qualys Security Conference 19 this week in Las Vegas. Hundreds of attendees from around the world have gathered at the Bellagio hotel to learn more about the current state of cybersecurity and what the future has in store. As you might expect, given the technology landscape today, many of the sessions and discussions have revolved around DevOps and how cybersecurity can adapt to DevOps culture.

Scott Crawford, Research Vice President at 451 Research presents “The Road Ahead for Security, IT, and DevOps” at the Qualys Security Conference

The Shifting Technology Ecosystem

Scott Crawford, research vice president at 451 Research, opened day two of the Qualys Security Conference with his keynote, “The Road Ahead for Security, IT, and DevOps.” Crawford talked about the dramatic changes over the last decade or so—from monolithic to microservices, from standalone applications to integrated software, from Waterfall to Agile, enterprise networks to IoT / OT, from standard networks to 5G wireless, and from traditional IT to DevOps. He discussed the challenges organizations face and the dilemma of doubling down on past success and missing out on innovation.

With all of the changes from this shifting technology ecosystem, there are consequences. Organizations no longer have a single point of control and IT teams are faced with managing and protecting an increasingly complex web of interconnected platforms, applications and services.

Security and DevOps

Crawford also talked about the rise of zero trust. He described it as a sort of mashup of the concept of least privileged access, combined with behavioral analytics and an on-demand, just-in-time approach to managing access. The zero trust paradigm is based on the premise of never trust, always verify—and that means determining who is requesting access, under what conditions they’re requesting access and what actions they intend to perform with that access on a case0-by-case basis.

He shared the Xebia Labs Periodic Table of DevOps tools to illustrate the overwhelming number of options available and stressed the importance of automation. CI/CD (continuous integration/continuous deployment), RPA (robotic process automation), and SOAR (security orchestration, automation and response) solutions have been developed to help address and manage some of the complexity of both DevOps deployments and cybersecurity.

Xebia Labs Periodic Table of DevOps Tools

DevOps and Digital Transformation

Crawford’s keynote was followed by a presentation titled, “The DevOps Transformation of the Qualys Platform: Lessons Learned.” Dilip Bachwani, senior vice president of engineering and cloud operations for Qualys, talked about how the company has addressed the “innovator’s dilemma” and steps it has taken to deal with growing pains as it goes through its own digital transformation.

Bachwani explained that the engineering team at Qualys has expanded exponentially in a relatively short period. That comes with several benefits in terms of how quickly they can develop and implement new solutions, but the team also faced challenges with balancing that growth and trying to maintain velocity at the same time without sacrificing stability or reliability.

Just like every other organization going through a DevOps transformation, Qualys has had to break down silos with people, processes and tools. Bachwani explained that they needed to do things differently to automate as many functions as possible. They decided to view infrastructure and operations as software problems and find ways to fully automate deployment and configuration.

He also emphasized that security must be built in throughout the DevOps life cycle. Qualys adopted a hub-and-spoke DevOps model and developed a robust DevOps toolchain designed to combine and orchestrate an effective set of tools for developing, delivering and maintaining software. Having a standardized toolchain ensures that there is a common platform available to everyone.

Accelerating Transformation With Citizen Developers

At the end of Crawford’s keynote, he talked about the lack of people with the right knowledge and skills. Cyber Security Ventures predicts that there will be 3.5 million unfilled cybersecurity jobs by 2021. That’s just cybersecurity. What about developers and other key personnel? Crawford posed the question, “Where will we find the people to do it all?”

The answer, he said, is citizen developers. What is a citizen developer? It’s a movement to empower employees who are not developers by trade with the tools and support to implement ideas. They are the ones closest to the problem and, arguably, best equipped to understand how to solve it. They don’t need to be professional developers or need experience with coding environments. Low-code and no-code platforms give them the power to turn ideas into solutions.

A few years ago, George Hulme wrote an article on the growing citizen developer movement. Hulme noted, “The organizations that succeed with citizen development will be those that wholeheartedly embrace the movement and cultivate it, with IT’s help and guidance.”

The DevOps revolution isn’t going away. Companies that don’t embrace digital transformation will likely fade into oblivion as they are left in the dust by more innovative competitors, and organizations that don’t figure out how to integrate security into the fabric of their culture will struggle against a rapidly expanding and evolving threat landscape.

At this point, these things are like the ocean tide. It is coming whether you like it or not, and it will destroy those who are not prepared. Crawford wrapped up by asking the audience to consider what their role is going to be in building and maintaining the future of DevOps and cybersecurity.

— Tony Bradley

November 22, 2019
Five Tips for Solving the Technology Skills Gap

The insightful, and sometimes breathless, predictions regarding digital transformation are everywhere. We are right to be mindful of both the potential and the challenges posed by digital transformation. Digital transformation is increasing the rate of change in every business, in every line of business and even in every role.

Digital transformation requires new development practices, organizational culture changes, customer intelligence and focus, empathy and, above all, new ways of thinking. Only when everyone is committed to improving the way work gets done, can an organization thrive in a competitive environment. As organizations continue to grapple with how to prepare their workforce for an increasingly digital world, they have a technology skills development problem.

Below are five considerations for bridging the technology skills gap brought on by digital transformation.

Identify the Skill or Development Problem You Are Trying to Solve

It’s essential that the technology skills development program be aligned in purpose and design with the intended business objective. That may sound obvious, but organizations have a way of accelerating beyond the original scope and intentions.

Ultimately, what you are trying to do will determine the combination of tools, content and even organizational commitment needed to be successful. For instance, are you trying to deepen technical knowledge for a set of employees? Or, are you tasked with improving specific performance objectives in data, DevOps and/or security?

Before investing in training, it’s important to establish the targeted employees, the range of competencies to be covered, the degree of proficiency required and the time frame to complete the initiative. In all cases, build a program that can solve the problem in a time frame that meets the business needs.

Establish the Performance Objective

For nearly all technology skill problems, the goal is performance: the target learners becoming sufficiently competent to perform at the required level. Understanding what the performance standard or measure of success is or will be helps focus all stakeholders on the same goal.

If specific performance objectives are required, monitoring a specific key performance indicator (KPI) can be the most valuable approach to determining training success or impact. If the KPIs are not available, using post-assessments can be a useful measure of knowledge gained and can help demonstrate the impact of the training.

Leverage Authoritative Content

It’s important to understand the sources used to create the course material as well as who authored or collaborated on the material. This matters not just because the learning content should be accurate, but also because adult learners are sensitive to the credibility of the source material. Especially in technical areas, IT professionals expect to be provided with material from authoritative sources.

IT professionals consider current practitioners, or well-known consultants the most authoritative voices on how to use technology. In some cases, industry associations or trade groups have established themselves as authoritative sources in their specific technical areas. Regardless, the source material should be highly reputable and authoritative.

Put User Experience First

Making learning as convenient for the target learner as possible is essential to a successful learning program.

Convenience has often meant anytime/anywhere learning—learners can take courses at home, while commuting, from their desk or wherever they work. This experience should be much like watching a movie on Netflix or Hulu: Learners should be able to start the course at one location, on one device and continue it at another time on a different device from just the right point in the course.

Training can also be seamlessly incorporated into the workflow of some applications. Think of it like a more sophisticated, less annoying version of “Clippy,” the Microsoft assistant that provided in-context help for Microsoft Office through the mid-2000s. For some types of applications and technical roles, having a context-sensitive learning engine that can identify and serve up nuggets of learning appropriate to the task can be very useful.

Offer the Right Mix of Push Learning and Pull Learning

Employers are beginning to realize the benefits of pull learning where IT professionals identify and select their own learning paths. This model allows motivated professionals to acquire skills, gain more responsibilities and, in some cases, promotions and new job opportunities. Employers have realized that when employees self-select a course, it is a strong indicator of self-motivation, and a strong predictor of professional dedication and success.

But pull learning isn’t appropriate for every problem to be solved. For example, maybe you need a group to learn the same thing about cloud security or compliance. In such situations, it’s not practical to wait until the target learners are motivated to take the specific courses. It makes more sense to solve these challenges by assigning specific courses to the right groups of target learners, assigning completion dates and monitoring progress.

Depending on what problem you are trying to solve, either push learning or pull learning may be more appropriate.

Setting Your Organization Up for Success

Ultimately, successfully navigating digital transformation requires a team with well-aligned IT roles and skill sets. People are the make-or-break element of a high-performing IT organization. There is simply no replacement for people with the right skills, attitudes and traits. CIOs don’t have the luxury of hiring a totally new roster of people with “the right stuff,” so they will have to be creative in using a combination of hiring, mentoring and training, contractors, and partners to create the needed mix of capabilities.

Once you have hired and trained the talent in your organization, what’s your plan to help them continue their learning journey for the duration of their career?

— Mike Hendrickson

November 18, 2019
IT Enables Scale in Digital Transformation

Few corporate functions have seen their role disrupted as much as IT these last few years. The help desk days are long gone. As digital transformation accelerates, IT is becoming less about outright technology and more about creating capabilities that enable the overarching business strategy. As such, IT is the engine behind any company’s digital transformation.

Here are three ways IT can advance today’s digital journey by empowering the business from a consultative, strategic position.

Going Digital While Digitizing

First, let’s agree on digital transformation’s foundational terms, which understandably elicit some ongoing confusion: “digitization” and “digital,” which the MIT Center for Information Systems Research has described as two discrete yet complementary things. In short, digitization is about leveraging technology to improve or enhance business processes, operations and offer development. By contrast, going digital is about giving customer value to digitization—with the goal of revolutionizing the customer experience and creating new digital business models. What’s the point of digitizing a process or product if the user experience is poor?

Balancing these complementary efforts is essential. It is the role of IT to ensure that digitization and digital can happen at the same time.

Rethinking the Economic Model

To keep up, IT must move away from traditional demand-based models, enabling capabilities with a product mindset instead of a time-bound project mindset. The concept of capability (that is, a business area enabled by a consistent set of IT products) is quite critical to avoid creating one-off or point solutions that are just the means to answer to a specific demand. Instead, looking to IT as the enabler of corporate practices that are always evolving and improving (e.g., cybersecurity) is imperative. Embracing Agile methodologies will bring this approach to fruition.

Pivoting to this kind of a capability-based model requires a modern API-based architecture, which fundamentally is the way out of building and/or replacing monolithic systems. As today’s world requires a fine balance of speed and frugality, a mastery of resource allocation, and a prioritization rigor, we must unburden resources aimed at replacing systems that are working just fine.

To that effect, Amazon Web Services has a great architecture principle referred to as “two is better than zero.” In other words, it is better to have two similar IT components emerge from fast-paced, parallel development than nothing due to project bottlenecks and unmastered interdependencies. Even if we want to drive re-use every time we can (and have mechanisms for that), we should not bog down overall transformation due to one weighty project. So thinking in terms of not just tolerating diversity, but enabling it, is a good way to built an architecture that can empower both agility and resiliency.

Addressing Technical Debt Continuously

Technical debt is perhaps every incumbent company’s biggest challenge when going digital. The cost of maintaining legacy and outdated systems is high, and heavy technical debt is another obstacle to achieving full digital transformation. Accenture consultants see this every day, describing this burden as a price that “hinders [companies’] ability to innovate and adapt in the digital age.”

This is where IT can come to the rescue by having a structured approach and engagement model designed to streamline systems and processes in order to free up resources for reallocation. First focus on eliminating legacy custom-built applications and localized tools based on criteria that are well known and shared (think of a scoring system). Here, again, resource allocation can play its part by ensuring the full cost of those legacies is being recognized by the organization.

A huge added advantage of a continuous effort to remove technical debt (beyond cost, the reduction of obsolete systems and a simplified IT landscape) obviously relates to security. Especially at a time when the average cost of a single data breach globally is $3.9 million. In fact, potential cybersecurity and IoT-related physical safety concerns “will pressure CIOs at G2000 companies to increase IoT security spending by up to 25%, temporarily neutralizing business productivity gains” by 2020 (IDC). Redirecting technical debt can offset part of this essential spending on security.

Accenture notes that 79% of CEOs report that their organization is adopting new and emerging technologies faster than they can address related security issues. IT teams therefore must carry the security torch, determining and implementing ways to make security easier for employees while also protecting the end-to-end digital ecosystem. A continuous effort to reduce the technical debt is one way—among many others—to improve posture.

A Place at the Digital Transformation Table

Indeed, these are exciting times for every business. Digital is all about creating customer value, guided by a better understanding of customer needs deeper in the enterprise. Since providing a great experience is not only about the user interface but also about the ability to mobilize the internal systems in order to deliver at scale, IT experts now have a major seat at the table for revolutionizing the customer experience at scale.

The essential ingredients to full digital transformation are clear: digitizing and going digital, being able to embrace emerging technologies with an architecture that makes diversity efficient when required and creating cyber citizens that continuously reduce the corporation’s technical debt. Behind the scenes and now front and center, IT is pushing forward the transformation strategy for companies staking a competitive claim in today’s digital economy.

— Hervé Coureil

November 15, 2019
How DevOps Is Driving the Demand for Cross-Platform Tools

The demands of modern business mean companies today want choice when it comes to the technology they deploy. Gone are the days when they rigidly used a single environment–instead they want to be able to use the best of technology from vendors and be sure it will work across environments seamlessly.

Whether it’s on-premises or the cloud, Linux or Windows, SQL Server or Oracle databases or even at the development tool level, they want their people to be able to use the Integrated Development Environment (IDE) they’re most at home with, yet still be able to share third party tools and resources. This requirement for greater choice is being driven by three key factors.

The Growth of DevOps

Firstly, the spread of DevOps is leading to greater collaboration between departments within an organization, as is the rise of the cloud.

In the past, for example, database administrators (DBAs) and application developers worked in their own silos, with their own technology, without needing to interact. DevOps has radically changed this, bringing them together in teams where workflows merge and complement each other. They need to follow the same processes, such as continuous integration and continuous development, but have very different technology backgrounds and skillsets. DBAs may know Microsoft SQL Server Management Studio in-depth, while developers are used to creating code in IDEs such as Visual Studio. SQL Server experts may also choose to work in Windows, Linux or a combination of both environments.

Forcing everyone to use a single environment will clearly be counter-productive and lead to a drop in efficiency. At the same time, however, letting them use different approaches can harm the very teamwork that DevOps relies upon. Instead, vendors need to allow their software to be used and managed through multiple approaches, ensuring that companies get the best out of their people and investments.

Technology Everywhere

Marc Andreessen coined the famous phrase “software is eating the world“ back in 2011. Since then every business has seen technology spread throughout its organization, with digital transformation initiatives increasing the software development required for all departments and functions. At the same time, data has become central to better business operations, enabling brands to better understand their customers, provide an improved experience and deliver more tailored products and services.

This has resulted in the rise of full-stack developers who are now responsible for both application and database development. Redgate’s 2019 State of Database DevOps Report revealed that 77% of developers now switch between coding in application languages, such as JavaScript in one moment to writing T-SQL for the database the next.

A problem arises if different IDEs are required which inevitably leads to an increase in the need for training alongside a decrease in efficiency and performance.

Growing Cost Constraints

In today’s competitive marketplace, every business investment is subject to a high degree of scrutiny to check that it’s delivering a demonstrable ROI. While budgets in the past might have allowed organizations to use different tools or systems in each department, there is a growing need to avoid duplication and the costs that it brings. Open source and free tools are also being widely deployed in even the largest organizations. This drive for greater efficiency obviously needs to be balanced against the skills and experience of in-house teams, who want to be able to use tools they favor and are accustomed to.

Conclusion

All of these three factors are driving a new approach from software vendors. Whereas in the past they might have focused on integrating their tool with a single IDE, they now need to offer choice. This allows a common development approach to be adopted while still enabling collaboration across the business, particularly when it comes to DevOps. It preserves investment in the skills and tools that different groups know best, increasing efficiency, yet also eases the introduction of initiatives like full-stack development.

Driven primarily by DevOps we’re moving into a more heterogeneous world, and vendors need to ensure that they are supporting companies so they can work in the ways they want, with the tools they want, if they want to remain relevant and successful.

— Matt Hilbert

November 15, 2019
Achieving Organizational Change Through Scaled DevOps

The rapid pace of technological change has had a major impact on customer expectation, forcing IT decision-makers to adopt agile processes to ensure the continual delivery of innovation. This has led to the mass adoption of DevOps practices, with 77% of organizations indicating they rely on DevOps to deploy software, or plan to, in the near future.

DevOps is a key ingredient for building and deploying better software faster. It enables developers to spend less time on onerous tasks and more time creating new features that drive competitive advantage and elevate customer experience.

But, for DevOps to streamline organizational change, it can’t be a one-off initiative. Scaled DevOps requires aligning team members on common goals and best practices and establishing a DevOps maturity model tailored to a company’s individual needs. This is no easy lift and has led teams to abandon DevOps initiatives before they even get started. While the road to scaled DevOps can be daunting, the benefits can have a positive ripple effect across the organization.

Business

The business benefits that accompany DevOps are endless. Think: expanding your customer base, spearheading organizational readiness and achieving unmatched speed. As the consumer landscape continues to change, you will be able to adapt accordingly to generate new features and pathways to customers. Plus, DevOps provides a prime opportunity to get more information from your client base and relay it back to developers, equipping your organization with opportunities to learn from its failures and develop a company-wide, reflective model that better accommodates customer demands.

Once your organization establishes a successful DevOps practice, it will be better prepared to take on other emerging trends and undergo a full-scale digital transformation. With DevOps as the catalyst for change, you can surpass competitors and disrupt the industry with innovative ideas, enabling you to stay ahead of the curve. Once an organization is fully committed to DevOps practices, the company will also enjoy a competitive advantage due to increased speed by empowering your business to provide customers with capabilities faster than rival industry players.

Culture

DevOps may require a daunting corporate culture makeover, but it also allows businesses to enjoy unbeatable cultural benefits by improving communication, establishing a foundation for continuous learning and boosting corporate morale with more meaningful work. When effective processes for communication are in place, employees can avoid the misunderstandings and missteps that often lead to failed initiatives and low morale. Once DevOps improves communication between teams, it sets the stage for more collaboration and cross-departmental education, allowing an organization to develop new operational efficiencies and empowering staff with highly specialized knowledge. With reduced inefficiencies, team members are empowered with more free time to create and meet new goals, plus budding talent from competitors will be drawn to your organization’s innovative environment.

Operations

Empowered by scaled DevOps, businesses will enjoy huge operational and technical rewards with automation, controlling costs and revamping security practices. Since establishing a DevOps practice requires the successful adoption of automation processes, a DevOps initiative includes the same key benefits that automation boasts by delivering the coveted digital experiences that consumers, employees and stakeholders demand. Automation not only meets these demands, but also sets the groundwork for a DevOps journey with CI/CD pipelines that automate software releases to rid your organization of manual release challenges.

On the financial front, as your company moves toward a consumption-based economics model for things like public cloud, your DevOps practice will help control costs and environments by alleviating unused tools. In the long run, improved security will also prove profitable because there is no fiscal nightmare worse than a breach. Upon defining your DevOps model, compliance and control are not only maintained, but rapidly transformed through automated policies and configuration management.

With knowledge of the transformational qualities that come with an effective DevOps approach, IT leaders are eager to adopt DevOps to improve efficiency, boost profitability, deliver superior customer experience and establish a culture of empowerment. However, they must understand that successful DevOps initiatives do not happen overnight. DevOps is reliant on continuous testing and education to ensure that companies can keep pace with their new collaborative culture, rapid delivery process and new technologies. Nevertheless, motivated by the promise of positive business, operational and cultural transformations, organizations will be more willing to prioritize the key credentials necessary to start, and maintain, an efficient DevOps practice.

— Tim Curless

November 14, 2019
Becoming a Data Company: A Tale of Two Trends

Data is back, baby! Well, that seems like an odd statement. It never quite left. Nevertheless, Google’s recent acquisition of Looker and Salesforce’s acquisition of Tableau seems to highlight this mindset in magnified ways. With eyes set on new leaps in data science and artificial intelligence, it’s easy to understand how our views on data now feel like a shifting pendulum.

The theme of the last decade was “every company is a software company,” driven by ready compute availability from cloud platforms. In the coming decade, it will be time for our data practices to catch up with the ongoing revolution in software development practices–or in other words–for “every company to become a data company.”

Over the past few months, as I have conferred with customers, attended conferences and connected with colleagues, there’s a perception of growing vigor in data conversations. Developing a data competency is rapidly growing in importance to business now, just as establishing a software competency became a priority 10 years ago.

However, my most curious observation has been the discussions around data and software are insular. Conversations either focus on how to move, manage, cleanse and leverage data for analytics, or how to build and release high-quality software quickly. The people who hold these conversations are either data people or software people who live in isolated worlds, seem to have divergent concerns and connect only at the lowest common denominator–the database storage–when the software guy or gal needs to store the data somewhere. There is little talk of the deeper connection and synergy between the two worlds and how it relates to larger trends.

The Next Wave of Digital Transformation

If you zoom out a bit, there’s more than meets the eye. There is an elegant symmetry between the software and data worlds, and it alludes to how the modern enterprise will morph in the age of digital transformation. Just as functions and organizations were reshaped to support new cultural and technology trends such as DevOps, organizations shall be reshaped in a data-first world, too. As software competency grows and matures, the need to acquire data competency becomes more urgent. What seem like separate trends are a maturing continuation of the uber trend of digital transformation, an attempt to acquire competencies in software and data (and necessarily so in that order).

As the diagram below shows, enterprises have been pursuing a journey of developing a software competency and becoming software companies. Those that make sufficient progress in software competency quickly realize that the significantly increased data they now have acquired requires new competencies to harvest and extract value from it. A data competency becomes necessary to completing digital transformation and so begins the journey to becoming a data company. Software and data are two sides of the same coin of digital transformation with the echoing resonance of reassuringly familiar themes. The diagram below highlights some of these.

Mirror, Mirror on the Wall

Speed and safety are central drivers in both competencies. The DevOps movement was driven by these two (often at odds) requirements to deliver software quickly from development to production without breaking the production deployment by ensuring that what is traditionally known as “non-production” accurately mimicked production. The goal is to build and test code so when it is deployed to production it doesn’t break what is already working.

In the data context, as more organizations understand the value of their data, they realize the importance of speed to get data quickly from where it is produced (production) to those that need it (outside of production, i.e. non-production). Safety in the data context is not just about not disrupting production while extracting that data, but also about meeting regulations and remaining compliant with data privacy policies in the process.

The elegant symmetry here is further accentuated by the fact that while the primary concern in the software context has been around getting software from internal development environments to customer-facing production environments, in the data context, the concern is getting data from customer-facing production environments to internal non-production environments. This completes the virtuous cycle of experimentation and learning that modern enterprises must master (see below) in their journey of digital transformation. It requires that organizations rapidly iterate to product-market fit through trial (software release) and learning (data science, analytics, machine learning) and back again.

In this way, digital transformation becomes a virtuous cycle driven by the complementary movements of DevOps and DataOps–the cultural movements, philosophies, technologies and collaborations that underpin the software and data journeys respectively, and the primary drivers of organizational metamorphosis that enterprises are undergoing during the digital transformation journey.

— Karun Bakshi

November 6, 2019
Getting RPA Right the First Time

Remember when every article you read about IT focused on digital transformation (DX)? Now the broad concept of DX seemingly has taken a back seat in the headlines to a number of hot technologies. One of these, Robotic Process Automation (RPA), is frequently positioned as the new next big thing of the tech world. But if you look closely, you’ll notice that DX never really went away–RPA is really just the latest piece being added to that larger digital transformation puzzle.

Digital transformation is often explained as an ongoing journey that promises to help organizations break the traditional boundaries of enterprise IT, empowering business line workers to enable the delivery of faster outcomes and better experiences to customers. An important part of DX is eliminating the rote task work that plagues day-to-day operations. That can only be done with transformative automation technology. It’s no wonder that many organizations worldwide are focused on exploring, implementing or scaling RPA to help check that box.

But as the RPA hype cycle starts to crest, don’t be fooled into thinking that RPA is the single turnkey solution to your enterprise problems. There are ways to avoid common pitfalls that are starting to emerge in RPA projects and ensure its successful implementation and achievement of business results.

RPA projects need proper planning, a strategy and an implementation team, all tied into broader enterprise systems and process management. Without these, organizations are bound to face costly delays that can affect deployment plans and return on investment (ROI). In fact, a recent study from EY found 30% to 55% of initial RPA projects fail. In another industry survey, it was reported that 87% of respondents had experienced RPA bot failure while reporting that maintenance was the second biggest concerns users face.

Here are the most common mistakes organizations should avoid to achieve real results.

Thinking RPA is Easy Since It’s Everywhere

This is perhaps the most common myth around RPA–enterprises expect to purchase a solution, train business users to build automation and see the ROI start to accumulate. It is reported that only 39% of bots are deployed on schedule, and it typically takes 18 months to push bots into live production. This is hardly an out-of-the-box solution.

But we all know how difficult technology implementations can be. Very few things in our industry are truly plug-and-play–no matter what sales professionals might say. While RPA is pitched as a way to solve menial tasks, there is nothing menial about the process of automation and developing an application tailored to how work actually gets done. It is possible to train up employees to build simple automation, but the true results will not be realized without an end-to-end approach that takes into account the broader outcome the automated task is supporting.

Remember, your business has rules on security, compliance and governance–all critical to the success of an RPA deployment. Are you considering these factors and all the affected stakeholders across the entire organization when trying to fix a problem within one department?

There is a place for business users building automation, but to get to real scale and potential for transformation, IT needs to work together with stakeholders across different lines of business to design an end-to-end intelligent automation solution that ties into pre-existing business systems and accounts for their sophistication/complexity, while also accounting for where humans need to be involved.

Attended or Unattended?

RPA is the hottest trend since cloud computing, and it boils down to two basic forms: attended and unattended. Simply put, attended requires the help of human interaction on the desktop, while unattended takes over tasks without the need to interruption. Is one better than the other? Not necessarily. In the right situation, both can generate quick wins and task automation to assist the workers.

Consider the desktop of call center agents. These workstations are known for having deep integrations with legacy technology that are geared toward repetitive, scripted and manual tasks. This provides the perfect opportunity for attended RPA deployments to address these types of functions that take place across the entire call center, delivering better customer outcomes and generating business value.

There is a time and place for both attended and unattended solutions–it is important to understand which capabilities the RPA vendor is offering and how they will address the needs of your current situation. In many situations, attended can be an unexpected source of ROI opportunity, as the total volume of work automated in environments such as contact centers, can far exceed the amount of non-human assisted work that can be automated.

RPA Will Solve All My Problems

RPA is great—I wouldn’t be writing about it if it weren’t. But it’s not the end-all digital investment automation platform; it’s part of an organization’s digital transformation strategy and can create serious business value when used the correct way. Automating and streamlining the complexity of multiple, long-running processes that interlink internal and external systems, machine and human work, bespoke software and third-party apps and systems is not what RPA was created to do. That’s the work of a BPM solution supported by Case Management. Businesses need to realize how RPA fits into their overall digital transformation strategy and where specifically it can be applied across their organization to realize its true benefits.

You’ve likely heard of the old saying, patience is a virtue. Let’s apply that to RPA deployments. There is so much promise and opportunity for RPA within the enterprise, but the major ROI numbers that some expect may not be there–at least not yet.

Finding the right opportunities for RPA technology within the organization, getting company buy-in and developing and deploying the solutions all take time to create an impact. Businesses need to approach their projects with a strong understanding of where RPA can be applied and what it does well, like automating high-volume, low-complexity, rule-based tasks within an end-to-end enterprise process automation strategy, to realize the business benefits it brings.

— Nolan Greene

November 1, 2019
Red Hat Launches Holistic DevOps Training Initiative

Red Hat this week announced it has created a Global Transformation Office focused on bringing about digital transformation via broad adoption of best DevOps practices.

The Global Transformation Office initiative is comprised of Andrew Clay Shafer, vice president of transformation at Red Hat; Jabe Bloom, chief sociotechnical officer for PraxisFlow; Kevin Behr, general manager and chief science officer at PraxisFlow; and John Willis, CEO of Botchagalupe Technologies.

Shafer said Red Hat is trying to encourage organizations to take a more holistic approach to transformation by simultaneously addressing the need to understand how new technologies impact the organization in terms of skills required and the business model around which the organization is built.

Instead of focusing solely on IT skills, Shafer said the Global Transformation Office at Red Hat is also seeking to educate line of business executives how to engage with DevOps teams to advance digital transformation initiatives, many of which never come to fruition because the IT teams working on those projects don’t know what the end business goal really is. If line-of-business executives can communicate that goal in a way DevOps teams can appreciate, they soon discover how much faster those DevOps teams will be able to drive the organization toward that desired outcome, he said.

Rather than simply telling IT teams they will have a new role, Shafer said business leaders need to back up those claims by investing in training. Otherwise, IT professionals who today often associate their identity within an organization to the process they manage will resist that transition out of fear of losing their position. Right now, too many organizations are also asking IT teams to embrace DevOps without ever really informing DevOps teams why any given project matters to the business strategy, added Shafer.

Red Hat is investing in this initiative because the amount of demand for enabling technologies such as the Red Hat OpenShift application development platform will be constrained unless business and IT models evolve in tandem. It makes no sense for organizations to embrace containers only to try managing them using the same processes they previously employed to manage virtual machines and bare-metal servers, said Shafer. Organizations need to realize they will not get the maximum value out of such platforms unless they have the skills and expertise needed to apply them within the context of achieving a business goal, he added. In many cases, organizations may need to reorganize themselves because too many business leaders still value themselves in terms of the size of the organization they lead instead of the contribution the business unit makes to the company.

It’s not clear to what degree the DevOps tail may be able to wag the business dog. In many ways, it’s only a matter of time before the business needs to be reorganized in a way that makes it as agile as DevOps teams are becoming. The real question is how long that transition will occur. Of course, many organizations that don’t make that cultural DevOps transition won’t be around to find out how it all turns out.

— Mike Vizard

October 30, 2019
Cloud Native Tracing and Observability: Why You Care

Is observability the new monitoring? Or is observability, and tracing, fundamentally different? Like any IT industry trend, it can be difficult to discern as many jump on the trend bandwagon, appropriately or not. The Splunk .conf 2019 event presented Splunk with the opportunity to bring clarity to the terms and the reasoning behind the acquisitions of SignalFx and Omnition.

Monitoring is mostly about taking events, telemetry data and established data points and thresholds into monitoring, alerting and problem management processes and tools. Its tried and true, and has progressed with the evolution of typical monolith, distributed apps and systems. Enter cloud-native applications, composed containers, microservices and service meshes. With great flexibility comes complexity, and cloud native isn’t immune to such an adage.

What would have been a monolithic application is now shattered into hundreds or even thousands of smaller pieces of application and app technology software. The complexity of externally monitoring cloud native quickly surpasses the capabilities of most monitoring approaches.

Observability, as recently acquired SignalFx CTO Arijit Mukherji shared this week, is built upon three pillars: metrics, tracing and logs. Metrics show when you have a problem, tracing points you to the problem and logs help find the root cause—a reasonable way to define and segment observability. It also, not surprisingly, fits well with the logic of Splunk plus SignalFx plus Omnition.

To move beyond monitoring requires instrumentation, built into the software and APIs as part of the software creation process, a DevOps process. SignalFx in part brought Splunk auto-instrumentation, which during run-time, identifies frameworks and libraries in use within applications and can capture tracing instrumentation. Omnition brings even deeper tracing chops to perform tracing across large service meshes of microservices. Add Splunk’s capability to correlate data across the business, applications and operations data and you complete the picture with the ultimate goal of making observability easy for developers.

The above might explain why developers, ops and DevOps care about observability and tracing, but should the business care? SignalFx CEO Karthik Rau connected the dots nicely during a conversation this week. Digital transformation strategies require speed and agility, but also demand more risk-taking. Confidence in risk-taking comes when accompanied by the ability to respond rapidly to changes and failures. A software deployed 10 minutes ago may need to be backed out or corrected rapidly when the users’ experience goes negative. That requires a rapid determination of what the problems is, and the ability to take immediate corrective action, including automated action.

Cloud native and DevOps not only enable disruptive, digital transformation strategies but must be accompanied by rapid and automated responses when negative business and customer impacting conditions occur. Customers don’t care when CPUs are taxed, but they do care when a mobile app’s responses fail or are slow. The move to cloud native is served well when backed up by such capabilities to respond in near real-time when problems occur.

— Mitchell Ashley

October 29, 2019