devopsy.kinsta.cloud

Tag: enterprise

  • Automation is Key for Success of 5G at Scale

    Automation is Key for Success of 5G at Scale

    5G is more than a progressive step; it is a catalyst for individuals, society and industries to meet their digital aspirations. 5G goes beyond connectivity by providing a platform to collaborate across industries, and 5G networks will be at the center of an ecosystem that pushes society’s continued digital transformation forward. By design, 5G networks will be flexible and modular, with technologies such as network slicing, software-defined networks (SDN), cloud-native and cloud radio access networks (cloud RAN). A flexible architecture makes it feasible to increase overall network capacity as needed, completing the standardization of application programming interfaces (APIs) towards underlying infrastructure, which will be key for automated connectivity to a heterogeneous network. APIs have sparked a revolution in banking, finance, health care and other fields, and now it is communications’ turn.

    Communications service providers (CSPs) can’t lose sight of the imminent innovation coming from cloud and cloud-native software. 5G is where telco meets IT. Virtualization, cloud edges and containerized 5G deployment alongside a legacy network bring tremendous challenges with numerous interfaces, processes and complexities. The basic automation model that exists today will not be able to support the next-generation networks that come with a plethora of connected devices and the volume of traffic that must pass through. End-to-end, closed loop automation is going to be the catalyst that accelerates the space from network-centric operations toward customer-centric operations.

    Why Automation is Crucial in 5G

    5G drives incident-driven operations toward data-driven operations. It also drives human-triggered automation toward outcomes, triggers processes and initiates massive machine-to-machine communications. At the same time, the internet of things (IoT) has vastly increased the number of endpoints that need to be managed and, as network slicing comes to fruition, network slices will include SLA delivery models, which will be beyond humans’ ability to manage.

    As a result, CSPs must rethink the way their networks operate now and enhance their existing value-adds by complementing their existing, strong cellular expertise with software capabilities. As organizations integrate their networks with cloud assets, databases and application assets, they must focus on incorporating automation at every step to efficiently manage their 5G networks and enable new 5G offerings.

    AI/ML and Automation

    5G is evolving alongside AI and ML. The high speed and low latency of 5G along with AI and IoT are going to significantly impact the way we live and work. 5G and intelligent connectivity enable automation to go through a complete transformation from digitization to artificialization. ML is pushing CSPs from playing the role of digital service provider to becoming cognitive service providers. AI will augment human decision-making in telecommunications to achieve newer use cases. The power of AI can be used to identify features and qualities in the current environment from a large set of big data, then analyze the use cases and predict future network failures which will help to mitigate the impact of any possible failure.

    Enabling digital innovation is a high priority for CSPs as they transition from a prescriptive model to a declarative, API-driven and DevOps model. Success will ultimately depend on efforts to build the necessary organizational readiness. By leveraging cloud technology, managing risk, embracing openness and establishing a human-augmented AI model, CSPs can break silos in their network and enable this transition to happen.

    Are CSPs Ready to Embrace the New Era of Automation?

    In the 5G era, mobile will become even more central to our daily routines. According to a GSMA report, the economic value created by 5G connectivity will reach $1.4 billion by 2025 and 5G will contribute $2.2 trillion to the global economy over next 15 years.

    5G networks will require automation that can support multi-domain and multi-vendor technologies, as services such as network slicing will traverse across multiple network segments like radio, access, core and edge, which may contain a mix of virtual and physical network functions.

    On their journeys from communications service provider to cognitive service provider, many network operators today have orchestrated network and IT domains, but few have the capability to orchestrate end-to-end services, given the fact that physical legacy networks will coexist with cloud-native network functions. While network slicing and tailored connectivity are still in their early days, full automation will not happen without end-to-end service orchestration.

    To enable automation on every aspect of the network, CSPs must embrace fully interoperable, data-centric solutions that offer the visibility and monitoring that is expected of cloud-based applications. By working closely with customers, each network function can be built in microservices with fully automated testing and regression testing and by leveraging DevOps for carriers.
    5G is the catalyst that is converging telecom network engineering and operations with IT. The vast amount of data to be analyzed for insight-driven business models is forcing a cultural shift from pure network management to services management. Evolved, smarter networks, new agile and insight-driven business models all require that 5G automation be taken to the next level. Simply put, automation is not an option but a necessity.

  • Investing in Artificial Intelligence: How MLOps Drives Enterprise AI Wins

    Investing in Artificial Intelligence: How MLOps Drives Enterprise AI Wins

    Insight Partners had a banner year in 2021, with more than $50 billion in capital commitments across over 200 investments. While we invest in founders across a wide spectrum of businesses, this series focuses on outlining our theses on four verticals that we’re particularly excited about in 2022: Artificial intelligence (AI), fintech, cybersecurity and healthtech. 

    Artificial intelligence has remained a super-hot category for good reason: It has the potential to transform nearly every industry and business.

    At Insight, we’ve long been bullish on the many use cases for AI. In the past year, we’ve invested in image recognition software from Netherlands-based ScreenPoint Medical, which improves early detections of breast cancer, Covera Health, which provides a quality analytics platform to reduce medical errors in radiology, CARTO, which helps companies use and understand spatial analytics, and Laminar, a cloud data security platform to continuously monitor and protect against data leaks – among other game-changing companies.  

    In total, Insight invested in 49 different companies across a broad spectrum of artificial intelligence and machine learning use cases in 2021, which represents a 172% increase from the year prior.  

    As we look ahead into 2022, we expect artificial intelligence tools to continue to dominate. We see the ecosystem dividing into three primary categories:  

    • Layer 1 – Base platform companies: Algorithms; frameworks; infrastructure and workbenches for creating ML systems   
    • Layer 2 – Cross-industry capability companies: Turnkey machine learning-based systems that solve specific problems spanning multiple industries (e.g., cybersecurity)  
    • Layer 3 – Industry-specific companies: Applications powered by prediction or classification systems that target specific, niche use cases in one domain    

    Companies and investors will find valuable AI/ML software across all three layers. At Insight, we initially focused on layers two and three. We invested in startups creating robust ML systems that addressed specific problems, either vertically (like credit underwriting company Zest AI) or horizontally (like cybersecurity company SentinelOne). We thought that economic moats were hardest to build at layer one; in part as a result of robust open source ecosystems and because large public cloud vendors deliver many of these tools at low prices.

    Insight Partners’ three-layer framework of machine learning companies

    Throughout this year, however, we realized we were missing a piece of the puzzle. While there will always be market demand for cross-industry capabilities and industry-specific applications, 2021 taught us that as companies try to productize AI for the first time, base platforms will become increasingly important as well. 

    After all, more companies than ever are charging forward on artificial intelligence projects to try to guide business decisions and save time and money. A whopping 90% of enterprises are either actively running AI projects or plan to in the next 12 months, according to a recent survey.   

    But here’s the catch: Most of these projects will fail.  

    There are many reasons why including a lack of sufficient data or black-box models that spit out indecipherable results. Regardless of the reason, high failure rates can hurt the ecosystem if they result in corporations’ hesitation to move forward with future AI projects.  

    Accordingly, Insight is particularly excited about a subsection of AI tools that can significantly increase companies’ likelihood of succeeding at their project goals: Machine learning operations, or MLOps. 

    MLOps tools can improve a machine learning pipeline from start to finish by helping gather, manage and label data, experiment and test the model selection, deploy multiple models in production at once and protect against model and data drifts and attacks. 

    Holistically, the goal is to improve communication and collaboration between data scientists, data engineers and business analysts through the machine learning life cycle, similar to how DevOps tools help improve communication in the software development life cycle. Machine learning is an iterative process with constant feedback loops and the need for continuous monitoring, which makes MLOps tools to manage this process even more critical. 

    Overview of some of the key areas of an MLOps pipeline

    While every company rolling out AI projects can benefit from MLOps, the type of tools they require will depend on their needs, resources and strategies.   

    Companies should ask themselves: How sophisticated is our data science team? How mission-critical do we want our models to be? Are our data sources structured or unstructured? Do we want open source, commercial or homegrown tooling?   

    The answers to those questions will point companies to specific MLops tools that are best suited for their specific challenges. 

    For example, some platforms—like Dataiku or DataRobot—are end-to-end products with wide capabilities, which are typically easier to use and thus encourage model creation and experimentation from workers across job functions, regardless of technical knowledge. These so-called ‘citizen data scientists’ can boost analytic workflows for companies with fewer data science experts and create significant value, but there’s a risk that the deployed models are not as well understood or controlled. These types of tools are best for companies that want to use machine learning to drive business insights and analytics rather than as part of their core business function.  

    If a company has a sophisticated data science team, however, it may lean toward specific point solutions, or what Insight calls best-in-class MLOps solutions. While these products require greater technical knowledge to maintain and deploy, they allow for much more control, depth and sophistication in a system. One startup offered the apt analogy that using platforms versus best-in-class solutions is like driving an automatic car versus a stick shift. 

    There are multiple MLOps tools for each part of the pipeline, and we see a world where each tool has enough market space to support it becoming a large company. 

    At Insight, we spent 2021 learning about and investing in many of what we consider to be the best-in-class MLOps tools, as determined by highly satisfied customer feedback, clear market momentum and well-respected and knowledgeable teams. This perspective led to our investments in Explorium, Rasgo, Weights & Biases, Deci, Run AI, Fiddler, Tonic, Dataiku and Databricks, as well as several others that have yet to be announced. 

    Insight Partners’ MLOps market map 

     While Insight expects MLOps to continue to be an essential driver of AI success in the new year, we also expect to see several other trends take shape in 2022:  

    • First, we will see the emergence of two machine learning pipelines, one for structured and one for unstructured data. Unstructured data sources—like images, videos and audio—require specific data warehouses, data management, pipelines and model management tools that maximize productivity and accuracy. Given the nuances and complexities between the two data types, we expect to see more MLOps tools shift toward owning their category for one or the other. 
    • As enterprises scale from dozens of models to hundreds or even thousands of models, we expect the increased industrialization of AI. This will result in the rise of orchestration layers that sit on top of machine learning pipelines to help manage all the different tooling. These layers will integrate across multiple tools in the ML pipeline—whether open source, commercial or homegrown—and provide an integrated environment (or so-called “single pane of glass”) to better track and manage ML pipelines. Orchestration layers will help enterprises gain more control over their pipelines while still supporting best-in-class tools. 
    • Insight believes we are at an inflection point of moving from a model-centric to a data-centric world. In a model-centric approach, you ask how you can change a model’s code to improve the performance of the system, whereas, in a data-centric model, you ask how you can change the data to improve it [system performance]. We expect the most successful tools of 2022 to be focused on supporting the new tasks, workflows, and jobs spawned by the data-centric movement.
    • We will also see a new class of MLOps tools focused on closing the growing AI skills shortage and gap across businesses. These tools will focus on increasing the efficiency of the model production process and lower the bar for ML development.
    • Finally, we expect to see a significant increase in the overall enterprise adoption of ML, with a shift from experimentation to deploying models in full production. Last year, we saw many enterprises form AI research teams and hire data scientists to experiment with machine learning. In 2022, we believe these enterprises will start realizing the full capacity of their AI projects and driving significant business value. This increase in enterprise adoption will also be driven by the explosion of edge devices that will enable new use cases of ML adoption. 

    As active machine learning and artificial intelligence investors, we are excited to continue to watch the market and support new entrants in what will surely be another dynamic and exciting year. 

    George Mathew, Lonne Jaffe and Sophie Beshar co-authored this piece.

  • DevOps World 2021: 3 Sessions for Successful Scaling

    DevOps World 2021: 3 Sessions for Successful Scaling

    Your business has grown from a startup of five people to a full-blown organization of 100 … congrats! But, now what? The practices that worked for a smaller company may have worked as you first started growing, but now it’s time to scale operations to fit your new needs. Just like a kid outgrowing their clothes, your organization needs a new “wardrobe,” and scalability is key for long-term success.

    This year’s virtual conference—scheduled for Sept. 28-30—will offer a number of breakout sessions, panels and keynotes focused on scalability within enterprises. If you’re ready to scale within your business, here are a few sessions from this year’s conference to help get you started.

    From Startup To Enterprise—Sustainably Scaling

    Many organizations struggle to scale DevOps culture as they grow. Now it’s time to say goodbye to the startup phase and hello to thinking like an enterprise.

    Alston Rousseau, director of software delivery at Campspot and Kailene Cloud, director of talent acquisition at Bonusly, will walk attendees through the investment needed to sustainably scale operations, development and culture. Plus, they will share tips for finding more value in existing commitments to money spending, time and diversity in DevOps and how this approach applies to organizations with and without high compliance requirements.

    Scaling up Car Configuration: From a Statically Hosted Front End to Universally Rendered Services

    Over the past year, tech company Code Orange made the transition from a traditional API/SPA split to a much more technically challenging server-rendered SPA with full cache handover.

    In this breakout session led by Niels Hoiting, a developer at Code Orange, attendees will learn more about Code Orange’s transition from API/SPA split to server-rendered SPA while analyzing the pitfalls that can arise in the transition and how to optimize compute spend to best handle a specific workload.

    Leveraging CloudBees to Scale Satellite Ground System Emulation in Multifaceted Virtual Environments

    Aerospace and defense technology manufacturer Raytheon Technologies has successfully implemented CI/CD pipelines for the test and near-operational integration of both software and hardware for a global satellite ground system.

    In this breakout session led by Casey Jackman, deputy of GPS Product Factory at Raytheon, attendees will discover how Raytheon uses simulated satellite control stations in virtual private clouds for CI, test and check-out. Additionally, Casey will share how the team met challenges in initial pipeline generation between multiple disparate target environments with AWS and on-premise data centers.

    Think it’s time to scale your organization? Register now for DevOps World 2021 to get started.

  • IBM Unifies Cloud Services to Simplify App Dev

    IBM Unifies Cloud Services to Simplify App Dev

    IBM announced today that an IBM Cloud Code Engine service that unifies the cloud platforms on which applications are built and deployed is now generally available.

    Built on top of a serverless computing platform, IBM Cloud Code Engine is a managed service that unifies access to a platform-as-a-service (PaaS) environment, a container-as-a-service (CaaS) environment and batch workloads, that run on traditional virtual machines, to a single platform.

    At the core of that managed service is an instance of Kubernetes running Knative middleware for invoking an open source serverless computing framework, the Istio service mesh and Tekton pipelines for building DevOps workflows.

    Jason McGee, CTO and vice president for IBM Cloud Platform, said the goal is to eliminate the artificial constructs that developers are forced to navigate in the cloud today that can each trace their lineage to a different era of software development. The IBM Cloud Code Engine unifies the developer experience in a way that makes it simpler to build and deploy applications on the type of cloud service that best makes sense for a specific application workload, McGee said. Developers no longer need to configure code for a specific type of infrastructure; they can simply describe how they want source code or a container image to behave, and IBM Cloud Code Engine automates deployments accordingly.

    The serverless computing framework that IBM Cloud Code Engine is built on also makes it possible for IBM to only charge organizations when code is actually running, versus requiring them to commit to a specific amount of infrastructure to run a set of virtual machines, McGee added. Charges are based on the memory and vCPU consumed and incoming HTTP calls. That approach will, over time, substantially reduce the total cost of cloud computing, said McGee.

    It should also accelerate the rate at which applications are migrating to the cloud, added McGee. Today roughly 20% of application workloads run in the cloud. Getting the next 40% of workloads shifted into the cloud will require providing a simplified application developer experience, noted McGee.

    The challenge organizations are wrestling with today is that they are deploying a mix of monolithic and microservices-based applications in the cloud that, to varying degrees, rely on batch-oriented and real-time platforms. IBM Cloud Code Engine represents an effort to provide a layer of abstraction over a wide range of cloud services that they can flexibly invoke via an event-driven platform, as required.

    The DevOps implications of IBM Cloud Code Engine are potentially profound. DevOps teams today spend an inordinate amount of time provisioning cloud infrastructure on behalf of developers. IBM is now making a case for a platform that automates that process in few seconds. In effect, control over cloud infrastructure will shift even further left to application developers. DevOps teams, in theory, will spend more of their time optimizing cloud environments after an application has been deployed.

    It’s not clear to what degree platforms such as IBM Cloud Code Engine might force cloud service providers to consolidate distinct cloud services they provide to developers today based on the type of application being built. However, it’s already apparent that the writing on the wall heralding that consolidation have now been indelibly inscribed.

  • To Accelerate Digital Transformation, Accelerate Data Transformation

    To Accelerate Digital Transformation, Accelerate Data Transformation

    Successful digital transformations are not possible without successful data transformation. After all, it’s data—and the ability to analyze that data—about an organization’s customers, employees, infrastructure, market dynamics, supply chain, business-technology systems and more that make digital transformation possible.

    However, last year Gartner said that “information as an asset” is still in its infancy. “Data and analytics are the key accelerant of an organization’s digitization and transformation efforts. Yet today, fewer than 50% of documented corporate strategies mention data and analytics as fundamental components for delivering enterprise value,” the analyst firm wrote in its report, “Why Data and Analytics Are Key to Digital Transformation.”

    Further, Gartner predicts change is coming and quickly: Gartner says that 90% of corporate strategies will unequivocally list information as a critical enterprise asset and analytics as an essential competency within two years.

    Getting there and being successful with data and data analytics won’t come without significant challenges.

    According to a survey released by IDG (sponsored by Matillion, a cloud data transformation provider for cloud data warehouses), the challenges are steep. The survey polled more than 200 IT, data science and data engineering professionals at North American organizations with at least 1,000 employees.

    What are the challenges enterprises are facing? According to the survey results, 47% said data control issues are the biggest challenge to data analytics projects. In contrast, other top challenges indicated are lack of a scalable, reliable technology platform to process large data sets (45%), having too many manual processes (38%) and challenges cleansing and preparing data (36%). Additionally, 40% cited a lack of visibility and control of data silos as a challenge when collaborating with business users.

    With that, it’s not surprising to learn that it takes on average one week to pull together and prepare data for useful analysis, and 45% of all time spent on data analytics initiatives is consumed by data preparation. About 60% said they lack the scalability and flexibility they need when preparing data for analysis.

    It was almost unanimous when it came to the number of companies that sought ways to accelerate their approaches to data analysis improvement.

    So how are enterprises accelerating their efforts to improve data analysis to better power their digital transformations? Respondents said data portability (57%), ease of onboarding (57%) and cost-effectiveness (52%) are the top features of an analytics platform that can help them move past current obstacles. The IT professionals surveyed favored user-friendly (50%) and easier connections to data sources (50%) as top features; however, data engineers, data scientists, data architects and others cited time-to-value (57%) and ability to provide self-service (51%) as essential capabilities.

    A sizeable portion of those surveyed indicated they are already on their way, adopting the technologies and processes they need to improve their ability to analyze their data for better decision-making rapidly. According to the survey results, 38% are already using cloud data warehouses. And in the future, 43% said they expect to have all of their data in the cloud, with 57% planning to use both cloud and on-premises data strategies.

    Those that master their data analytics more rapidly may enjoy an early mover advantage. “A company’s ability to compete in the emerging digital economy will require faster-paced, forward-looking decisions,” said Douglas Laney, distinguished VP analyst at Gartner. “Data and analytics leaders need to assert themselves into corporate strategic planning to ensure that data and analytics competencies are incorporated within the highest-level public-facing enterprise plans.”

  • Don’t Let Your DevOps Transformation Be a Parade of Tools

    Don’t Let Your DevOps Transformation Be a Parade of Tools

    Choosing the right DevOps tools should be a careful, thoughtful process.

    On May 12, 1848, a store owner named Sam Brannan held a “one-man parade” to announce the start of the San Francisco Gold Rush. “Gold! Gold from the American River!” Brannan shouted up and down the market street in San Francisco. He held his hat in one hand and waved a bottle of gold dust in the other. San Franciscans had received false news of gold before. But by all accounts, Brannan’s performance sent residents running in search of riches. 

    Brannan had a good reason for spreading the news rather than panning for gold himself. The canny entrepreneur owned a general store that served the workers, and in the week between learning about the discovery and yelling about it in San Francisco, he’d bought all the picks and shovels in the city. Brannan’s announcement helped spur a seminal event in California’s history. As Brannan raked in money selling mining supplies, his actions also, years later, led to the coining of a famous maxim. (source)

    During a Gold Rush, Sell Shovels

    In the current age of DevOps and Agile rush of enterprises, there is a lesson to be learned from the gold rush: Don’t let your DevOps transformation become a parade of tools. While automation is critical for the success of modern ways of technology delivery and operations, the wide range of tooling choices can often lead to wrong decisions—especially well-intentioned, top down purchase of tools, duplication in toolsets and, most importantly, ignoring the voice of the teams on the ground.

    With such variety of tools available in the market today, we all have opinions regarding the tools that we would like to use. Our preferences are based on either our history of using certain tools, because we have invested time and effort to learn a tool, or partnerships your service vendors might have with tooling vendors.

    This leads to two general attitudes toward tooling adoption, with each having their own advantages and disadvantages:

    Autonomy for teams: Let teams decide the tools they feel are suitable for their context.

    The advantage is that teams can experiment and pick a tool that suits their context better. The disadvantages can, potentially, include security risks due to lack of patching and maintenance, and limited cross-team collaboration and sharing of tooling skills and know-how.

    Standardized tool suite: A centralized way of dictating tooling choices.

    The advantage is that it allows for capacity planning, integration between tools, larger pool of community for collaboration and create platforms-based solutions.The disadvantage is that this method at times prevents teams from using tools that best suit their unique needs and creation of avoidable dependencies on centralized tooling teams in the organization.

    These contrasting views lead to debates at every level for the choice of tools and, in some cases, slowing down decision-making—or even leading to the purchase of tooling licenses without input from technical teams on the ground. The decision-making process must be improved, and the need for autonomy must be balanced with standardization and optimization. Also, the two approaches of “autonomy for teams” and “standardized tool suite” do not always have to be binary and will change depending on the context of the team needs and area.

    By considering desirability, viability and feasibility of the DevOps tools, we can help achieve the balance between autonomy and alignment.

    Here are some tips to help ask the right questions:

    Desirability

    • Do teams want this? or is this a solution chasing a problem?
    • Do you anticipate that teams will need this in the near future?
    • Does this help improve team productivity and effectiveness?

     Viability

    • Does this make commercial sense?
    • Have we considered maintenance costs and total cost of ownership?
    • Do we have existing license arrangements with the tooling vendors?
    • Do we have comparable tools already in use and that can do the job?
    • Do we need to purchase licenses throughout the enterprise?
    • Are there any open source alternatives?

    Feasibility

    • Can we implement this in the enterprise IT landscape?
    • Have we considered the ease of setup and configuration of the tool?
    • Does it scale across the enterprise? Does it have to scale across the enterprise?

    — Aditya Vadaganadam

  • Challenges in Leading a DevOps Team at a Fortune 100 Company

    Challenges in Leading a DevOps Team at a Fortune 100 Company

    A manager, an evangelist and a godfather all walk into a bar.

    Okay, this isn’t a bad joke, this is an article about how managers can implement DevOps in large organizations, especially where culture and organizational change hamper efforts. But, I promise, we will talk about managers, evangelists and godfathers.

    Uldis Karlovs-Karlovskis (@UldisKK) works at Accenture Latvia. Accenture is a global consulting giant, employing 400,000 with teams and clients across the globe. Uldis helped bring DevOps into the organization and shared his journey—the good and the bad—at theAllDayDevOps conference in his talk titled, “Lessons in Leading a Fortune 100 Team to a DevOps Philosophy.”

    Uldis joined Accenture in 2012 and faced three top challenges:

    1. How to maintain quality at such growth?
    2. How to enable continuous learning?
    3. How to stay fast and flexible at their size?

    His team was small, they held daily stand-ups that were productive, and they were flexible. Life seemed good, and then his team grew into a “spider.” They still held daily stand-ups, but they were useless because it was just people reporting and no one understood what is going on.

    What did Uldis do? Well, he didn’t freak out, if that is what you are wondering. Actually, he did. One morning he, “woke up, realized I had 25 people under me, freaked out and skipped the job for the day.” He walked around, thinking about it and wondering what to do. He thought about his three to four types of DevOps projects and all of the non-project activities that were monopolizing his time. He realized he focused on the urgent over the important tasks, working 10 to 12 hours per day.

    What is a manager to do? Have people help him, of course. Uldis decided to create an unofficial role that sounds interesting: the subject matter expert (SME). They would work on non-project activities, increase team engagement, improve satisfaction and assess DevOps maturity in projects. Initially, people were motivated to be SMEs, but they were always busy with other things. So the effort failed.

    Uldis had a new idea: Instead of him leading the SME team, he let them decide who wanted to do what, which SME team they wanted to join and who would be the lead SME for each team, and let them set their own goals and how they wanted to work, etc. This helped, but still didn’t solve the problems. Uldis was still burdened with the tyranny of the urgent.

    Back to the drawing board, and a new idea was born: evangelists. Uldis broke down 50 SMEs into seven subgroups and added a little hierarchy because the flat structure wasn’t working. Each evangelist led a guild—one of seven. Team members were to talk to their evangelist before coming to Uldis with questions.

    But, alas, Uldis was still just handling escalations and talks.

    So, what do you do to bring order? Bring in Godfathers.

    Now, team members could always go to the Godfather for help. Uldis decided that, generally, each person would have four topics they may want to talk about. For each topic, they have a person to talk to:

    • SME addresses how to contribute to the company in general
    • Godfather addresses escalations, such as not happy with the team lead
    • Evangelist addresses questions about the person’s skills
    • Team Lead addresses project tasks

    Uldis now realized the benefit of this approach: In a way, now, everyone is focused at some level on sales. As we all know, this is important yet very difficult in any organization—but especially when everyone is an engineer.

    While his approach has helped him and his team, he also acknowledged, “If I did the same thing today, I would fail, but maybe for different issues.”

    Todat, Ulids is looking at improving his culture. As we talk about at AllDayDevOps time and time again, good culture is key to successful DevOps. Uldis, looking at Accenture, noted it doesn’t fit into a typical organizational structure. Reading literature, industry buzz, etc., he realized something was missing from these models, but he wasn’t sure what. He knew they needed to be better.

    He discusses three dominant types or organizational cultures:

    • Pathological – Eight hours of work and your job is done
    • Bureaucratic – Two to 12 hours of work if the process requires it
    • Generative – Two to 12 hours of work and I’ll deliver it

    Uldis is trying to be at the third approach, and the key to achieving it is knowing that if you are having a bad day, you should leave your work for tomorrow when you will be more productive. Tomorrow it may take more than eight hours, but you will put your best into your work.

    His journey is ongoing, but he finished his talk covering a few things they do on his team:

    • Onboard new joiners through a continuous delivery pipeline
    • Meet weekly at lunch
    • Half-yearly performance assessments

    He also provided four takeaways:

    1. Everyone by default is good
    2. Intrinsic over extrinsic
    3. Engagement is your work responsibility
    4. Let them lead

    Uldis told an honest and open story about his journey with the hope it will help others. To hear it firsthand, watch it for free here. You can watch any of the 2017 AllDayDevOps sessions free of charge here.

    — Derek E. Weeks

  • Top Challenges When Scaling DevOps in the Enterprise and How to Conquer Them

    Top Challenges When Scaling DevOps in the Enterprise and How to Conquer Them

    Preparing to adopt DevOps can be a challenge for organizations of any size. Scaling DevOps and bringing it to an enterprise demands a special strategy. Use these tips to build yours.

    DevOps principles apply to all organizations. Everybody has business goals that can be met and exceeded by following DevOps practices. However, an organization’s size will inevitably impact the level of difficulty it will face during implementing and scaling DevOps. A large enterprise organization will encounter more daunting obstacles than a smaller organization, even if their problems are similar in nature.

    Regardless of your organization’s size, you are probably wondering what to expect during your DevOps transformation and asking yourself which strategies you can use to overcome any complications. Luckily, you now have “The DevOps Handbook” to turn to for advice. This book is an excellent resource that distills the most important insights learned from difficult experiences into straightforward steps and solutions for overcoming common DevOps hurdles.

    DevOps organizations move much faster than many other organizations today without sacrificing quality, performance or security. This blog post will cover some helpful strategies from “The DevOps Handbook” that will help you overcome challenges your organization might face during its shift to DevOps.

    Changing Your Organization’s Culture

    The first hurdle in the DevOps process is managing culture and expectations. Scaling DevOps throughout the organization requires changing how your business views itself. It also involves rethinking your process for delivering value to the customer. The goal is to get the changes you make in your products and services to the customer faster. In technical terms, this boils down to how often deployments happen. The DevOps process uses positive feedback loops to deliver value to the customer quickly.

    Related Content:

    What is an Agile IT Organization?

    Everything Ops: Are DevOps Principles Being Applied Too Broadly?

    To achieve this goal, “The DevOps Handbook” says that it is necessary to shift your focus from cost-optimization to speed-optimization. Optimizing speed does not require a massive, top-down reorganization. Rather, it means giving teams more autonomy. Different skills (such as development, operations and information security) should be embedded into existing teams. This will make teams become self-serving, which will enable them to develop, deploy and operate their software more safely and effectively. Majorly successful organizations such as Netflix and Amazon attribute their success to this approach.

    However, building self-serving teams is just one part of the culture change. You also must change how your people think about themselves and their work. Let’s take Toyota, the pioneers of the lean manufacturing process, as an example. The “Toyota Way” has propelled the company’s success, as Mike Rother wrote about in “Toyota Kata.” In the book, he explains that while structural reorganization is an important step toward overall improvement, it is also critical to invest in the betterment of your employees. You have to build upon their technical work abilities and develop the habits of continual learning and experimentation. Toyota accomplished this by using what Rother called The Improvement Kata, which is a model for achieving goals that is based on coordinating theory and evidence.

    DevOps organizations integrate continual learning and experimentation into their day-to-day work and decision-making. Verifiable experimentation, such as A/B tests and cohort analyses, advance business goals. The challenge is building the data-driven learning framework. Product owners and other decision makers need a variety of business telemetry. Accordingly, engineers must collaborate and implement whatever is necessary to provide it. Everyone can—and should—use this telemetry to understand what is happening with the business and IT at any point in time.

    Etsy is a good example of an organization that has implemented telemetry well. According to “The DevOps Handbook,” in 2011, Etsy started tracking 200,000 business-layer metrics, displaying 30 of the most important metrics on its deployment dashboard. By 2014, the company had tracked more than 800,000 of these business metrics. This high level of dedication to telemetry made its engineers’ jobs easier with quick business and technical documentation, especially after deployments.

    The next hurdle is combining telemetry with the practice of continuous learning and experimentation. The two practices work best together when the process is moving as quickly as possible.

    Faster Iterations

    Addressing speed-related challenges requires more of a technical effort than the others. DevOps requires fast feedback loops at all levels, which begins and ends with a new set of engineering practices.

    Trunk-Based Development

    First, you should start using Trunk-Based Development (TBD). TBD is a model for source-control branching, in which multiple developers work together on code from one, individual branch. With TBD, developers check in their code to trunk one or more times daily. Using this method has powerful knock-on effects. Automated tests run against the entire system, so it is immediately clear if something is broken. TBD also reduces long-running feature branches that may eventually create difficult merge conflicts or introduce unexpected bugs.

    Another great thing about TBD is that it naturally keeps changes smaller, which means that they are easier to review, test and reason about, all while keeping your code in a releasable state. It can also help you reduce your total number of environments, which can easily save tens of thousands of dollars every month.

    Automated Deployment

    Next, you should automate your deployment process. This requires automating as many of the intermediate steps as possible, including developer workstation setup, test environment setup, environment promotion, the deployment itself and telemetry collection. This is a lofty goal, but taking it to the extreme creates powerful business outcomes. More people are able to deploy more often. Deployments become smaller, safer and more routine, and the number of production bugs is reduced.

    Making this happen is no small task since everybody involved must adopt the “automation first” mindset. Start by automating the more error-prone and repetitive tasks. The effort will be worth your time, since automation increases throughput and reduces errors.

    Etsy is a big fan of automated deployment. According to an Etsy case study from 2014 that can be found in “The DevOps Handbook,” “deployments are performed by anyone who wants to perform a deployment, such as Development, Operations, or Infosec. The deployment process at Etsy has become so safe and routine that new engineers will perform a production deployment on their first day at work — as have Etsy board members and even dogs!” You may not want actual dogs in your control room, but you certainly want that level of confidence and satisfaction.

    Integrating Information Security

    This amount of automation sounds fantastic for developers and operations, but what about information security? For them, it’s not necessarily true that topics such as compliance and security are treated as afterthoughts. DevOps is successful because it uses quick feedback loops to identify and fix problems early on in the software development process, but information security and compliance usually come into the picture at the very end, creating huge blockages and problems for everyone involved.

    Information security is another skill such as QA or operations, so it can be embedded in cross-functional teams. Then, teams can work together to address known security issues, such as testing for package vulnerabilities as part of CI. Testing for proper secrets, such as a database connection string, can be handled with automation, too. Teams can also integrate static analysis to detect possible problems before the code is executed.

    Continuous delivery practices also apply here. Security-related metrics such as total password resets, potential SQL injections, XSS attacks, core dumps and other strange events should be integrated into the deployment pipeline. Doing so provides visibility and immediately actionable information to the entire team during incidents.

    Bottom-Up Scaling

    Large enterprise organizations are structurally complicated. There are many leaders and smaller teams, each with their own mission and culture. As such, trying to change things top-down is a big mistake—it simply does not work. Scaling out from the bottom-up is a much better approach, and there are a few ways to accomplish this.

    Starting small is the best advice. We recommend that you find a small group of like-minded individuals and give them technical autonomy, goals and your trust to improve the business. This will likely help them tackle technical debt issues that have hindered them in the past. They’ll try new approaches and learn down in the trenches.

    Unfortunately, there is no such thing as DevOps boot camp—only principles and technical practices. It’s up to everyone in the team (and the organization in general) to figure out what DevOps means to them. Hopefully, their results will speak for themselves. Naturally, other teams will become interested in how they work and think. In time, some team members may even be ready to train their peers.

    Ross Clanton at Target uses a “technological innovation center,” which is more commonly known as a DevOps Dojo. The Dojo coaches help teams across Target’s IT department elevate their states of practice. The “30-Day Challenge” is their most intensive training. Blocked development teams bring their work with them to work alongside dedicated coaches and engineers. The goal of the challenge is to unblock your team in less than 30 days. According to Clanton, taking part in this challenge has helped many teams finish work that would normally take months in just a matter of days. Completing these challenges has solved many internal problems, and has ultimately made the organization as a whole more productive.

    Conclusion

    There is more than one way to scale DevOps in your organization. In this blog post, we have reviewed three of the main DevOps principles that will help you along your journey. First, you need to begin changing your organization’s culture and organizational structure to allow for a “divide and conquer” strategy. Next, establish protocols that will help you find errors early. Doing so will increase the speed of feedback loops, which will make for quicker delivery of customer value. Third, invest in continuous learning and experimentation of new ideas.

    Using this and other advice from “The DevOps Handbook,” you can work through your DevOps transformation with confidence. When in doubt, we suggest that you consult this excellent resource for more helpful tips.

    — Rami Sass

  • Scaling DevOps at Pearson

    Scaling DevOps at Pearson

    Pearson might be one of the more influential companies you have never heard of. Its footprint in the educational publishing marketplace is expansive, and its scope was just one challenge when it made the move to DevOps.

    Sean D. Mack (@SeanDMackNYC) is formerly the VP of Operations and Application for Pearson. To give you an idea of the scale of the company’s operations when it began implementing DevOps, Sean oversaw 110 development teams and 3,000 different products across the globe. To further complicate things, Pearson is a mature company with both legacy and new products. This is a case study of how the company approached implementing DevOps with the goal of helping you learn how to do it. Sean presented it at the 2016 All Day DevOps conference.

    As they say, you often learn your biggest lessons with your failures. Sean noted that, ultimately, DevOps was a failure at Pearson due to organizational change. But, he presented the case study with the hope you will learn from what Pearson did and take it forward. He continues to be a passionate evangelist for DevOps.

    Pearson had its challenges. It was a siloed organization operating at a large scale, and had a toss-it-over-the-wall mentality in dealing with problems. Out of sight, out of mind.

    seanmack1.png

    In addition, all of Pearson’s teams were at different levels of maturity, from legacy teams running .Net on Windows 2000 servers while others doing CI/CD to AWS. Obviously, this creates challenges when implementing DevOps, but we will see later how the company tried to tackle that, specifically.

    At the high level, the solution involved several aspects:

    • A DevOps practice area
    • A DevOps steering committee
    • Measuring progress based on global KPIs focused on measuring business impact
    • Measuring teams based on a DevOps scorecard
    • Push from the bottom – plenty of developers and operations who embraced and were eager for the change
    • Push from the top – support from key executives

    Pearson’s DevOps Steering Committee set up several workstreams:

    • Tracking tools
    • Metrics
    • Deployments automation and monitoring practices
    • Test automation
    • App engineering resource contention
    • Operationalization
    • Communications

    While these are specific to Pearson, many may be applicable to your organization. In any case, it is important to step back, look at your organization, and decide what your organization needs.

    Pearson also looked at four core competencies to track and measure: automation; independence (how independent is the feature); operationalization (how easy it is to maintain); and continuous improvement. The below chart outlines each area in more detail.

    Automation

    Continuous Improvement

    Independence

    Operationalization
    • Unit tests
    • Critical Scenarios
    • Dependent Systems
    • Continuous Build
    • Continuous Deployment
    • Test Automation
    • KPI
    • SLA
    • Metric Tracking
    • Metric Reporting
    • Remediation
    • Isolated Configuration
    • Code Dependencies
    • Isolated Infrastructure
    • Monitoring and Alerting
    • Tier 1 Service Desk System Knowledge
    • Tier 2 Service Desk System Knowledge
    • Documentation

    To help each team measure their competencies and improvement, Pearson built a self-reporting tool to empower teams but encourage visibility, and developed business metrics and then a competency model to measure teams against to see how they were progressing. As Sean noted, “We were not trying to get everyone to a 5. Some teams, yes, but we also had legacy products that were running for years, successfully, with little or no maintenance. The investment to bring the teams up wasn’t worth the effort. There had to be a business reason.“

    seanmack2.png

    In the end, Pearson raised DevOps maturity for all of the teams. As noted earlier, though, this was eventually phased out, but Sean presented this case study with the hope you found something to drive DevOps in your company.

    In the end, if you have just a couple takeaways, they are this:

    1. Begin by determining what your desirable outcomes are.
    2. Don’t just do DevOps to be cool. Do it to drive business results.

    You can watch Sean’s entire talk online here. If you missed any of the other 30-minute long presentations from All Day DevOps, they are easy to find and available free-of-charge here.  Finally, be sure to register you and the rest of your team for the 2017 All Day DevOps conference here.  This year’s event will offer 96 practitioner-led sessions (no vendor pitches allowed). It’s all free and online on Oct. 24.

    — Derek E. Weeks

  • AIOps: Managing the Second Law of IT Ops

    AIOps: Managing the Second Law of IT Ops

    Or, why IT systems keep breaking

    The tension between the two halves of DevOps is long-standing, going back long before the term was even invented. What underlies the split between Development and Operations is a fundamental difference, like an actual physical law. Let’s explore that analogy together.

    IT Ops is fundamentally a constant fight against entropy, meaning disorder. Sysadmins expend significant energy trying to keep everything up and running, in a process which can sometimes feel like trying to shore up a sandcastle while the tide is coming in. Systems are constantly breaking down; users need changes made that introduce risk; and, as if that wasn’t enough, the whole shebang is constantly under attack from malicious outsiders. It’s no wonder that sysadmins have a reputation for being a bit grumpy!

    Homer: In this house, we obey the laws of thermodynamics!
    It’s not just a good idea, it’s the law.

    In contrast, Dev actually increases the level of entropy. Every new release, be it as small as a one-line fix or finding that errant semicolon, is a change that has to be made, reducing the order that Ops fights so hard to maintain. Dev is entropogenic, embracing change as a positive force.

    The interaction between Dev and Ops is analogous to the second law of thermodynamics: Entropy can only increase over time, unless energy is introduced to the system from outside. In other words, unmanaged change will eventually cause something to break, unless people put in continuous effort to prevent those breaks.

    Thermodynamics: Lots of Heat, not Much Light

    These conflicting goals, taken to extremes, can get unhealthy, with Ops actively fighting change or Dev going around Ops to increase their velocity of change. Left unchecked, this tension can turn into a vicious cycle, as frustration with the “Department of No” drives developers into unmanaged “shadow IT.”

    The good news is that the analogy with hard laws of physics is not perfect. The process of increasing entropy is not irreversible, and does not require constant inputs of energy to prevent. New agile approaches to IT Ops are emerging that are light and flexible, taking their structure from that tension. In the physical world, this is called “tensegrity,” and was originally formulated by Buckminster Fuller, one of those annoying polymaths who seem to make breakthroughs in every field they touch.

    Tensegrity structure
    Tensegrity structure

    IT Ops needs its own equivalent to tensegrity, a flexible, holistic structure that is able to embrace change while maintaining its shape. Structures that try to resist change through rigidity are brittle and ultimately fragile.

    Today, this flexible strength is enabled by AIOps, a term coined by Gartner to describe a new way of operating IT, based on augmenting human skills with automated analysis enabled by modern algorithmic and machine-learning approaches.

    Complexity Breeds Complexity

    Modern IT is too complex for any single person or even team to keep track of, instead requiring multiple specializations. In turn, each of these functions requires its own specific support tools that enable that particular role. Beyond a certain point, it becomes very difficult to combine these partial views to gain a holistic understanding of what is actually happening in the environment.

    Because of the scale of modern IT environments and the sheer rate of change that businesses require, old-style rules-based “Managers of Managers” can no longer keep up, requiring more effort to build and maintain their worth. Especially as infrastructure becomes more and more self-modifying in response to changing conditions and demand, IT Operations needs to accelerate just to keep up.

    AIOps brings developing issues to the attention of sysadmins before they become visible to customers and end users, and gives them the tools to collaborate effectively between different teams and specialist roles to avoid outages or restore service quickly and efficiently. In this world, automation is used to augment the capabilities of the human operators to make them more effective, letting people concentrate on adding value, not just fighting fires.

    This is how Ops can get out of the business of fighting Dev, and instead become a supportive partner. AIOps can catch issues that are affecting services and flag them to the right people early on, including the DevOps engineers that are responsible for production support.

    For developers, one of the fears around DevOps is that in “picking up the pager,” they are opening themselves up to a constant stream of irrelevant alerts. Old Ops hands nod wryly at this, or maybe try to share their personal approach to filtering the noise. Instead, AIOps ensures that only valid, actionable alerts get as far as being shown to a human being and taking up their valuable time.

    Eyes on the Prize

    It’s not all roses. As the AIOps market begins to mature, there is a certain amount of “algorithm-washing” going on, with more and more vendors trying to claim that either their legacy tool was AIOps all along, or that their one-trick pony is actually a whole three-ring circus. What gets lost in the excitement is what all of those algorithms are actually supposed to achieve.

    Me too!
    Me too!

    In considering a shift toward AIOps, the goal should not be simply to build a better mouse trap, meaning a smarter filter for monitoring events. The success of the project can be measured not so much in the mathematical results of the algorithms, but in the smoother exchange between the human users. This easier collaboration is especially important across team boundaries, whether these be different specialist units within the Ops function or developers and software engineers who need to work with Ops to understand the performance of their code in production.

    The benefit of transforming operations this way—moving from sequential, waterfall Ops to abile, collaborative Ops—is to turn DevOps-enabled teams into a sort of perpetual-motion machine, able to continue operating without requiring constant massive inputs of energy from outside the system. Modern applications and the complex infrastructures that support them cannot be managed with the sorts of rigid, cobbled-together, industrial approaches that were appropriate in the mainframe days. AIOps is the agile, flexible approach that can not only accomodate that complexity, but thrive on it.

    — Dominic Wellington