devopsy.kinsta.cloud

Tag: devops

  • How DevOps Can Nurture the IoT Brownfield For Results

    How DevOps Can Nurture the IoT Brownfield For Results

    Your latest IoT project sailed through proof-of-concept but started running into problems during rollout. Fortunately, your DevOps superhumans got things running. Now, perhaps a year or more later, you’re seeing more and more issues crop up as the project grows. If this scenario sounds vaguely familiar, or you’re worried you may be headed there, you’re not alone. Welcome to the IoT brownfield.

    “Nah, can’t be. We selected and evaluated all-new hardware and software during the PoC. This has been a greenfield project from the start.” Be careful of this trap. Many IoT projects indeed start off as greenfield efforts, free of the constraints of integrating legacy hardware and software. Your team developed requirements with collaboration between IT and line-of-business folks. You introduced these new tools, and got results on a small scale. According to Cisco’s 2017 IoT survey of 1,845 IT and line-of-business decision makers, 60 percent of IoT projects stall at PoC, so you’re ahead of the game.

    Or so it would seem. Now comes the escalating commitment. Management assumes that if it worked on a small scale, it should just work when rolled out across the enterprise. Data and insight will just come pouring in, and everyone will have visibility to what’s happening everywhere in real-time. They write a big check for the rollout, and away you go.

    First Signs of Trouble

    At the first rollout project review, folks start splitting hairs over the definition of “complete success.” You rolled out X locations, Y of them are working, X-Y of them are in various states of triage. How can that be? You took the “same” configuration and scaled it. As the team investigates the non-working population, you’re finding little things that aren’t the same, calling for tweaks here and there.

    Tossing out the trivial, over scale and time there is no such thing as an IoT greenfield project. Every IoT project with an extended life cycle becomes a brownfield, with a mix of new and old hardware and software. Rigorously controlling a configuration, à la defense or medical applications, is prohibitively expensive and unnecessary for most IoT applications. Your vendors won’t sell you the exact same hardware or software two years from now, five years from now, 10 years from now. Unless, that is, you pay them for things such as lifetime buys of electronic components or freezing open source code releases.

    Ideally, we should send DevOps teams into IoT projects. With a big tool stack acquiring data, the line-of-business types figure out more things to do as the picture develops. The whole point is gaining insight and adjusting for optimized results. Cycles of learning and security enhancements are part of the game.

    Coping with an IoT Brownfield

    Here’s the paradox: the longer your IoT application survives, the more complex this situation gets. How can you improve your chance of success by applying IoT brownfield thinking? Technical requirements vary, but some core philosophical tenets can help.

    Roll with the changes. Your team is already skilled at quickly rolling application software releases. Get more aggressive with ongoing changes at the firmware and network level. Incorporate over-the-air (OTA) technology for updating code on any wireless devices. Deploy multi-protocol gateways that can deal with revised protocol stacks (or entirely new ones) as IoT specifications continue evolving. Update security vulnerabilities relentlessly.

    Abstract everything. You’ll read tons about IoT interoperability, and what people typically mean is wire-level interoperability of protocols. Those multi-protocol gateways have one job—getting real-time data onto an IP network and into the cloud. What keeps me up at night is data-level interoperability. Use edge or fog computing for pre-processing streams so the cloud isn’t stuck sorting out various formats of the same class of data. Abstract your data using JSON or other lightweight techniques so that if a next-generation sensor outputs data in a slightly different format, pre-processing can be revised close to the source. Consider a publish-subscribe protocol for easier routing between nodes.

    Optimize the cloud. Scale can bite hard in the cloud tier. Heterogenous, distributed architectures usually win. Deploy a hybrid cloud solution where there is real-time data. Let the public cloud do what it’s good at: presentation, business intelligence and archiving. In a private cloud instance for heavier analytics lifting, use a framework such as Apache Spark to minimize data movement, with faster nodes sprinkled in for critical algorithms. Try dedicating a small processing core to each data stream for predictable deadlines, or using FPGAs in workflow optimized nodes for boosting compute performance.

    After All, It’s Just Business

    Ultimately, the evaluation of complete success of an IoT application will be up to the line-of-business decision-makers who wrote that big rollout check. A DevOps model is ideal for nurturing the IoT brownfield over scale and time, featuring ongoing strong collaboration between technologists and line-of-business types. Early in the project, reach out for some external IoT expertise until you have a few cycles of learning under your belt.

    As that Cisco 2017 IoT survey alluded to, 61 percent of practitioners believe we’re barely scratching the surface of IoT transformation for businesses. Avoiding the greenfield trap and incorporating these IoT brownfield coping strategies sets you up for the best possible chance of success in the long run.

    — Don Dingee

  • How IBM’s $34B Bid for Red Hat Will Spur DevOps Adoption

    How IBM’s $34B Bid for Red Hat Will Spur DevOps Adoption

    IBM’s proposed acquisition of Red Hat for $34 billion will have an immediate impact in raising awareness for a programmatic approach to managing hybrid clouds based on best DevOps practices.

    A recent survey published by International Data Corp. (IDC) finds most organizations (64 percent) have already embraced multiple clouds to varying degrees. Only 7 percent of respondents said they have built a truly hybrid cloud. Another 24 percent, however, said they have achieved a high degree of interoperability across multiple clouds. IBM made it clear this week that its primary motivation for acquiring Red Hat was to tap into a trillion-dollar opportunity that will manifest itself as hybrid cloud computing continues to evolve. In fact, IBM is betting that Kubernetes clusters will play a key role in advancing the state of hybrid cloud computing.

    But for hybrid cloud computing to take hold, it’s also clear the level of mastery over DevOps within enterprise IT organizations will need to improve substantially. In fact, hybrid cloud computing might not even be the ultimate destination, said CloudBees CEO Sacha Labourey. Enterprise IT is increasingly being defined by programmatic services rather than platforms, he noted, and those services need to be weaved together to drive modern applications. But where each of those services resides is becoming less relevant as organizations become more DevOps proficient, Labourey said, noting the rise of hybrid cloud computing will drive more organizations to embrace DevOps to bring order to the services being consumed.

    “Within a hybrid cloud DevOps will be required,” said Labourey.

    While platforms are still relevant, they increasingly are being hidden behind services that expose an application programming interface (API), added Chip Childers, CTO of the Cloud Foundry Foundation, which oversees the development of the open source Cloud Foundry platform-as-a-service (PaaS) environment. A developer consuming a service might have little to no idea what platform that service runs on.

    “IT teams are building on the abstractions that have gone before,” Childers said.

    The ability of one organization to differentiate itself will come down to the maturity of the DevOps processes they rely on to most efficient consume a service, he added.

    Demand for that level of DevOps expertise is already starting to accelerate, said Tim Beerman, CTO of Ensono, a provider of managed IT services. As IT continues to transform into microservices enabled by containers and serverless computing frameworks, the complexity of the overall IT environment requires a more sophisticated approach to managing the overall environment, he said.

    But, he added, “A lot of organizations simply don’t have the talent.”

    Of course, IBM is betting that once it owns Red Hat, more organizations will look to IBM Services for that expertise. But as always, competition across the IT services space remains fierce. IBM is also betting that it will provide the management plane through which hybrid cloud services will be unified. But even there, competition from rivals small and large will be significant. Vendors such as Rackware are already providing frameworks for managing hybrid clouds.

    While hybrid cloud computing is still relatively immature, organizations that embrace DevOps will be much further down that path than others, said Todd Matters, chief architect at Rackware.

    “DevOps will absolutely be a critical part of hybrid clouds,” he said.

    IT has always been about striking a balance between people, process and products. It’s too early to say whether IBM will recoup its $34 billion investment to acquire Red Hat. But the deal inevitably will lead to a greater awareness for a more comprehensive approach to DevOps capable of spanning a hybrid cloud computing environment that includes multiple platforms.

    — Mike Vizard

  • Simplifying Through DevOps in Banking

    Simplifying Through DevOps in Banking

    There are a great many assets available on the internet that talk about what DevOps is and how far DevOps adoption has come. So we will not repeat the “what” and the “how far.” Instead, we will focus on the road ahead and future of simplification and transformation at banking organizations and the role DevOps is likely to play in the same—in all three areas of “Run the Bank (RTB),” “Change the Bank (CTB)” and “Transform the Bank (TTB).”

    Figure 1: Change-Run-Transform Bank – POA with DevOps Maturity

    In this article, we take a look at what shape we think DevOps-driven simplification in financial organizations can take in the near future and also some opportunities for such simplification. As depicted in Figure 1, with DevOps maturity, the siloed processes of CTB, RTB and TTB will come closer together, allowing common activities and processes to be carried out in one place. The others will get simplified and automated thereby reducing elapsed time of cycle from ideation to production and application management drastically. This will yield benefits not only in terms of cost and time, but also in terms of feature availability to business.

    Banking Focus

    In banking, the early adopters are leading the march with a lot of good practices such as standardized tooling for specific technology areas, orchestration, feedback loop setup, process and technology rationalization, etc. The focus needs to continue on the same.

    We think that the challenge that DevOps architects need to now focus on is to be able to use DevOps to help financial organizations bring down the non-discretionary spend by simplifying the previously ignored maintenance, operations and infrastructure realms. The core of this DevOps driven simplification is needs to be move cost of maintenance to its lowest possible and hence releasing funds to discretionary areas in “Change the Bank” and “Transform the Bank.”

    While the point of arrival for DevOps-led transformation should be “full automation” and “zero maintenance” but they are a bit far in the future because the return on investment (ROI) on the monies and effort required to make that happen needs to justify the same. We see the focus shifting to value delivered to business by transformation of an application or suite of applications. Figure 2 depicts the core repeatable activities of a DevOps-led simplification exercise while keeping in mind these improvement cycles need the setup of feedback loops at each stage.

    Figure 2: Repeatable activities for DevOps-led simplification

    This brings us to our next idea, that the road map to DevOps-enabled simplification needs to be paved with metrics and measures and these metrics cannot be randomly chosen. These metrics have to be actually measurable and be divided into two areas of business metrics and IT metrics as we explain below and need to be diligently tracked to aid course correction.

    For example, release orchestration processes for applications are mostly complicated, need a lot of manual effort, have strict compliance requirements as well as security mandates, need to be carried out more frequently nowadays because of shorter deployment cycles and have to manage ever-increasing complexity of component interaction (upstream and downstream). Now, release orchestration is also a very important step in the entire CI/CD pipeline. So standardization and simplification of release orchestration process makes immense sense. But how does one measure success and quantify improvement?

    As already stated, intelligent insights (metrics and measures) are important in this process for both business and technical team members so that everyone has single source of truth for current status and distance from end goal. This kind of automation also helps focus on the continuous improvement pillar of DevOps adoption by logging everything that matters, have frame-by-frame data for everything that helps. This collection of metrics aids analysis of success as well as failure thereby furthering the cause of continuous improvement.

    Continuing with the example of release orchestration, there are many important best practices that help ensure that wheel is not reinvented. Some of the important learnings that only practitioners will be able to let you in on are as follows:

    1. Tools that need scripting will not be able to help you scale and create low-maintenance solutions.
    2. You need to invest in tools/orchestration solutions that help you visualize, get intelligent insights in a jiffy and can report in a variety of forms and shapes.
    3. Tools need to be able to capture and track business metrics, too.
    4. You need to spend effort on organization level compliance and security.
    5. Strive to automate your most effort-intensive and/or lengthy processes.

    Each step on DevOps maturity scale will have many such learnings and best practices that must get shared across the length and breadth of the organization.

    DevOps and Banking: Future Trends Prediction

    Based on our understanding & knowledge of financial organizations, we believe that the following areas will see focus in the next few years:

    Figure 3: Trends in short-medium term in DevOps adoption

     

    Continuous Improvement within Federation

    • Technology teams will need to enable and encourage localized innovation within a federated boundary of toolsets that will, in time, bring Dev and Ops toolsets to near mirror images of each other.
    • By federation, we mean a detailed homogeneous blueprint that gives flexibility but within an overarching boundary of tools and procedures while encouraging continuous improvement.
    • This must also inform relevant stakeholders about areas for continuous improvement that make the most evolutionary impact to business and use metrics suitably to track and reward improvement.

    Outcome Management

    • We also think that DevOps adoption needs to be driven by outcome management with evidence of impact taking the driving seat for road map management.
    • Focus will be on point of arrival (POA) and continuous measurement of business and IT metrics for outcome management.
    • We also believe that Culture-Automation-Lean-Measurement-Sharing (CALMS) model of DevOps will find more believers.

    Evidence of Impact

    • Metrics and metrics-based course correction will become the Holy Grail in the simplification exercise. The metrics collection will need to focus on areas of both business and IT. The metrics and measures will need to be redefined as:
      • Business Metrics for business value delivered or improved such as new products / services launched, conversion rate, net promoter score (NPS), customer retention rate, CSAT, ESAT, etc.
      • IT Metrics for value improvement and value management such as cost performance index, reduction in defect debt, reduction in lead time to change, etc.

    Design Thinking

    • We believe that the next big disruptor in this space will be bringing design thinking into the mix to ensure that DevOps-led automation is more customer-centric and iterative.
    • We see focusing on test data management (TDM) across the organization as an important step to ensure continuous delivery.
    • In our opinion, the testing process will undergo complete rethink with support of processes such as risk based priority and risk based test case design, test bed splits, TDM and so on.

    KM, Training and Governance

    • Tools and processes for knowledge management, training and governance need to ably support the DevOps-led simplification.
    • These processes will need systemic focus to create and nurture self-learning technology teams.
    • Transformation of the way of working to rationalize processes such as collaboration for lean and improved handover of work packages.

    DevSecOps

    • Security will be extremely crucial in the process of DevOps-led simplification. All the automation, cloud adoption, pipeline creation, IoT adoption and more is throwing up many new security concerns, thereby leading to a focus on DevSecOps.
    • Security processes will potentially need to move their execution points up in the CI/CD pipeline and be divided into stages based on iterations sizes and deployment cycles.
    • Focus on shifting left application security and role-based security.

    Conclusion

    The next wave of simplification of IT spaces in banking organizations will be aided by DevOps. To look for opportunities, a systematic approach toward DevOps will help—it will be perilous to ignore detailed assessment of existing spaces that will detail the current state, point of arrival simplified state, road map for simplification and the key guiding principles for the transformation.

    Carrying out pilots and proof of concepts in DevOps is not a problem; the challenge is with scaling—in all areas: tooling, training and processes. The key is to manage the horizontal and vertical spread of same/similar toolsets that also help drive efficiency associated with scale and standardization. DevOps-led simplification will need changes to processes, technology and people management (culture). The current role structures might need to be demolished to ensure there are no fiefdoms and cross-functional teams handle all requirements of Dev-Test-Ops. Finally, all the changes need to be able to define the new way of working  so that Change-Run-Transform organizations can be brought together.

    This article was co-authored by Babu Unnikrishnan, Head, Banking & Insurance Technology Group at Tata Consultancy Services.

    — Anshu Premchand

  • Defining DevOps: How Do You Know When You Have Achieved DevOps?

    Defining DevOps: How Do You Know When You Have Achieved DevOps?

    Key to understanding where you are in your organization’s journey is by defining DevOps

    Much has been written about the compelling benefits of DevOps. However, the lack of a commonly agreed definition and clear prescription for implementing DevOps have left many people puzzled. Ask a group of experts, “How do you know when you have achieved DevOps?” and you are likely to get different answers.

    It is tempting to dismiss definitions as academic. However, there are many incomplete, floundering and costly DevOps implementations that fall short of expected business benefits. This article explains some of the key challenges of defining DevOps, and the importance for having a definition at least within the scope of an enterprise. This article includes suggestions to help enterprises navigate their DevOps journey toward successful business outcomes.

    Defining DevOps

    An enterprise needs to know when it has accomplished DevOps to justify investments that support business goals. Pinning down a universal definition that satisfies the diverse community of stakeholders and survives the test of time has proven elusive. DevOps spans an evolving body of knowledge covering multiple humans, process engineering and technology disciplines. DevOps is never really “done” because the highest level of maturity, “The Third Way,” continuous improvement, defies setting boundary conditions.

    To reconcile this dilemma, some have defined DevOps using vague or transient terms such as a “cultural movement.” Others have skirted the problem of definition with rambling statements describing what is involved to do DevOps, rather than state what it is in concise and concrete terms. Despite these challenges, acceptance of a definition by enterprise stakeholders is a key first step in any successful DevOps transformation. Any meaningful project progress measurements hinge on clear definitions.

    I have found the following DevOps definition useful, relatively concise, and stable: “DevOps is the application of continuous flow, feedback and improvement to people, processes and technology for the benefit of agility, stability, efficiency, quality, security, availability and satisfaction.” Defining DevOps in this way is broad enough to stand the test of time, yet specific enough to answer key questions addressed by a good definition including “What is it?” “How it is done?” and “Why do it?” without too much rambling.

    Whether you create of choose another definition, or decide to use mine, it is important that your cross-functional teams agree on one to align their goals and implementations and to have a basis to measure your DevOps progress and business outcomes.

    DevOps and Continuous Delivery Partnership

    DevOps is often said to refer to cultural aspects of high-performance software development and operations including leadership, teams and collaborative practices. Nevertheless, technical mechanisms involved in architecting applications and toolchains are essential for continuous delivery pipelines. Culture, without the technology associated with continuous delivery pipelines, will not achieve the expected benefits of DevOps and vice versa. Therefore, a practical measure of DevOps maturity really must include the technology and not only the culture.

    Continuous Flow is a Practical Baseline

    DevOps benefits are derived primarily from applying lean manufacturing methods to software development and operations. The “First Way of DevOps,” continuous flow, is first because it is a prerequisite to the Second and Third Ways. Therefore, the realization of continuous flow for at least one application in an enterprise can be considered a baseline for “having DevOps.” If you accept this point of view, then the answer to the question, “How do I know when I have DevOps” can be equivalent to the question, “What is the minimum requirement for continuous flow for an application?” In the context of continuous delivery toolchains, a continuous flow pipeline is achieved when it allows code submissions from development to successfully transit all pipeline stages to deliverable packages acceptable for production deployment.

    A DevOps Model Application

    Enterprises typically have many applications moving toward DevOps at different paces. Identifying one application as a model for DevOps is useful. The model provides a focal point for DevOps investments, and experiments. Other application product teams learn from the model application as it matures from discontinuous flow to continuous flow and higher levels of DevOps maturity.

    Summing Up

    A practical answer to, “How do you know when you have DevOps” depends on having a definition. While defining DevOps itself has been elusive, an enterprise definition is needed for alignment and progress measurement. Considering that DevOps needs continuous flow to accomplish business goals, it can be said you have DevOps when you have implemented continuous flow for at least one model application.

    — Marc Hornbeek

  • IBM Acquires Red Hat to Create DevOps Behemoth

    IBM Acquires Red Hat to Create DevOps Behemoth

    In a move that has massive implications for DevOps teams across the enterprise, IBM today announced it is acquiring Red Hat in a deal valued at $34 billion.

    Red Hat will join IBM’s Hybrid Cloud group as a distinct independent unit. Red Hat will continue to be led by Jim Whitehurst and Red Hat’s current management team. Whitehurst will join IBM’s senior management team and report to IBM CEO Ginni Rometty. IBM intends to maintain Red Hat’s headquarters, facilities, brands and services practices.

    IBM expects to close the deal, which will be financed using both cash and debt, in the latter half of 2019. IBM also said the deal will be accretive within 12 months after close, as the combined entity moves to address a total market that spans application development and deployment, IT services and security and collectively exceeds $1 trillion, or more than one-fifth of the total global IT market valued at $3.7 trillion in 2018 by Gartner.

    IBM and Red Hat have had a partnership in place for 20 years, spanning everything from reselling and supporting Red Hat Enterprise Linux (RHEL) to collaboratively working on open source projects such as Kubernetes.

    IBM officials said they expect Red Hat to significantly bolster IBM’s hybrid cloud business, which already generates $19 billion a year in revenues. IBM noted that about 80 percent of application workloads have yet to move to the cloud.

    The move to acquire Red Hat comes on the heels of Red Hat acquiring CoreOS, which augments the company’s existing open source expertise with a container management platform built on top of Kubernetes.

    Once the deal is closed, IBM said the two companies will remain committed to open source software licensing, including initiatives such as Patent Promise, GPL Cooperation Commitment, the Open Invention Network and the LOT Network.

    IBM and Red Hat also plan to continue to build and enhance existing Red Hat partnerships, including those with major cloud providers Amazon Web Services, Microsoft Azure, Google Cloud, Alibaba and others that compete directly with IBM Cloud.

    It’s too early to say to what degree those cloud service providers will be open to continuing to work with Red Hat as unit of IBM. In recent months Red Hat increasingly has been positioning itself head to head against VMware, which operates as a unit of Dell Technologies. IBM also has a strong relationship with VMware that could change in the years ahead. What is clear, however, is that any deal of this magnitude usually begets others in very short order.

    — Mike Vizard

  • DOES 2018: DevOps Providing Greater Business Outcomes, Yet Challenges Remain

    DOES 2018: DevOps Providing Greater Business Outcomes, Yet Challenges Remain

    The DevOps Enterprise Summit turned 4 years old this year, and what a difference four years makes: At the inaugural summit, perhaps a few hundred attendees participated. This year, there were roughly 2,000, according to the organizers.

    The energy at the first year’s event felt like that of a skunkworks team that knew they were on to a great new way—they only had to sell the rest of the world on their radical ideas. This year, it became clear that the battle of ideas is over: The DevOps evangelists have won, and DevOps is here to stay.

    According to a DBmaestro study, more than 80 percent of organizations have adopted DevOps to some degree, but only 30 percent have done so across their entire organization. That study shows the popularity of DevOps, but also that there’s much more work to be done.

    That state of work was reflected at this year’s DevOps Enterprise Summit. This year the focus was on improving the pipelines that have been built, pushing the limits of DevOps even further and learning how to cope with the limitations of legacy business-technology systems that refuse yet to be retired.

    Greater Focus on Business Outcomes

    It seems DevOps has arrived. As Scott Prugh, chief architect and VP software development at CSG, said during his DevOps Enterprise Summit talk, “Product Management Meets DevOps,” it’s about business leaders from across the organization coming together to deliver better outcomes.

    That’s an aspect of DevOps that stood out boldly this year. DevOps practices in the enterprise are starting to move beyond the common cloud-native toolsets and are being applied to other areas in the organization, from the cloud back to mainframe and many systems in between.

    This unleashing of systems detached from the typical DevOps workflows turned out to be a central theme of the summit. For instance, Eric Robertson, VP product marketing management at CollabNet, explained how customers are exposing services on their mainframes and enterprise resource planning systems. As a result, customers are better able to deliver capability and value without having to re-create and re-release services.

    “It’s about knocking down the silos and optimizing functionality around business value,” said Sam Fell, VP of marketing at Electric Cloud. Those who aren’t doing it right, he said, are those who are not aligned well enough to provide the best business outcomes, the right user experience or the most optimal architecture, security and performance necessary to succeed in creating those user experiences.

    “You need to take a multidimensional approach and put the right people in the room early enough so that the right focus is maintained on outcomes,” said Anders Wallgren, CTO at Electric Cloud, during the DevOps Enterprise Summit.

    Both Fell and Wallgren agreed that more organizations have embraced the kind of architectures necessary to support the agility necessary to succeed. This includes container architectures that can be changed rapidly and safely so that in the event errors are introduced, systems can be remediated quickly.

    Challenges Remain

    In his talk, “DevOps for the Enterprise: 2018 Trends and Insights,” Wesley Pullen, chief strategy officer at Electric Cloud, detailed how enterprises are working to manage IT complexity. According to findings he shared from 451 Research, enterprises are taking a hybrid approach to cloud and container adoption, with 81 percent of organizations adopting a multi-cloud strategy, 58 percent adopting containers and 48 percent traditional on-premises systems. “What we have today are multiple teams working on independent and interdependent systems and target platforms with overlapping tools, platforms and release cadences. Managing all of this effectively is a challenge,” he said.

    One area where these challenges show themselves is with security. According to the “Sonatype2018 State of Software Supply Chain” study, since the Equifax data breach in 2017, there has been a 55 percent increase in open source breaches and 1 million downloads of vulnerable open source libraries. Fortunately, more organizations are embracing DevSecOps and making sure security processes are integrated into their DevOps processes, Pullen said.

    When it comes to extracting value from their existing enterprise systems, Pullen noted more organizations are making self-service catalogs available to their teams so that if a service is needed and it’s already been built, teams can access these resources quickly.

    Finally, he said more organizations are applying artificial intelligence and machine learning to their development pipelines. About one-third of organizations are making machine learning a top investment priority. They are using the technology to identify patterns using in-depth statistical analysis, predict future risk using previous patterns and provide recommendations to reduce risk, he said.

    While DevOps has outgrown its skunkworks roots, this year’s DevOps Enterprise Summit made it clear there is still much more work to be done before most DevOps organizations reach their full potential.

    — George V. Hulme

  • Integrating the Database into DevOps for Business Success

    Integrating the Database into DevOps for Business Success

    The benefits of DevOps have been known for some time, but this year’s “Accelerate: State of DevOps” report shows that successful companies are extending the advantages they get from adopting it. The longest-running research of its kind, the report from DevOps Research and Assessment (DORA) represents five years of work surveying more than 30,000 technical professionals worldwide.

    Based on their input, the report previously split organizations into low, medium and high performers. This year it added a new category—elite performers—who are embracing DevOps to raise the bar when it comes to the benefits they gain. Elite organizations are able to deploy code 46 times more frequently than low performers, lower lead times from committing to deploying changes to less than one hour (compared to one to six months in low performers), and recover from incidents 2,604 times faster.

    The Importance of the Database

    For the first time the report also focuses on the vital importance of the database to successful DevOps, echoing research from Redgate. As it points out, deploying changes to the database is often a bottleneck, slowing down DevOps. To overcome this, the Accelerate report recommends integrating database work into the software delivery process, based around four good practices:

    Strong Communication and Cross-Team Collaboration

    Traditionally, the teams involved in application and database development have had different priorities and responsibilities. The biggest concern with the database was to limit change to keep data safe, while application development was all about creating features, often without regard to what impact they would have on the database.

    With DevOps, this has changed. The faster releases of smaller changes it encourages means the database cannot be a logjam at the end of the development pipeline. Instead, it needs to be part of the same workflow, so good, ongoing communication is vital.

    In many organizations this is already happening: Fifty-eight percent of respondents in Redgate’s 2018 “State of Database DevOps” survey said that collaboration between developers and database administrators (DBAs) was either good or great, showing that the processes and teamwork are in place to break down any walls between departments.

    Comprehensive Configuration Management

    Using configuration management tools for software development is a central part of achieving successful DevOps results. To avoid issues, these same practices and tools need to be extended to cover the database as well. Handling changes systematically creates a seamless flow that enables faster releases and dramatically reduces issues—vital when more than a third of respondents to Redgate said they deployed database changes either daily or more than once a week.

    Extending Version Control to the Database 

    As part of taking a holistic view, the Accelerate report found that extending version control from application code to database code delivered major benefits. Elite and high performers store database changes as scripts in version control and manage these changes in the same way as application changes. This is a task that is shared between DBAs and developers—75 percent of respondents to the Redgate survey said that application developers now build database deployment scripts, and were responsible for deploying these database changes to production in nearly half (47 percent) of organizations.

    Building a Strong Culture

    Culture is a strong factor in whether DevOps succeeds or fails in an organization. This is unsurprising given that DevOps is a very different process to those it replaces, focusing on speed, collaboration between teams and a willingness to be open and continually learn and improve.

    Most people are now familiar with the work of sociologist Ron Westrum, who characterized the culture within organizations as one of three types:

    • Pathological (power-oriented) companies work in silos, don’t cooperate and are quick to apportion blame if something goes wrong.
    • Bureaucratic (rule-oriented) companies are more open, but are still suspicious of novelty and tolerate limited co-operation between teams.
    • Generative organizations are most suited to successful DevOps, with a focus on co-operation, surfacing problems, breaking down silos, holding postmortems to understand why things have not gone well and continually experimenting to drive improvement.

    To create the right culture for DevOps, leaders need to build trust with their teams, empower them with autonomy and rely on strong communication to ensure that everyone understands their role and overall objectives.

    Summary

    The latest “Accelerate: State of DevOps” report has a simple, top-line message: Higher software delivery performance enables better business outcomes. Achieving this requires taking a development-wide approach to DevOps, incorporating the database seamlessly into DevOps flows, communicating and collaborating between teams, sharing technical practices, and building a strong, open culture across organizations. Only then can businesses become high or elite performers, with an agile infrastructure that underpins their success.

    — Matt Hilbert

  • Where is the Next Generation of Mainframe Talent?

    Where is the Next Generation of Mainframe Talent?

    Are we potentially facing a shortage of mainframe talent?

    “Three of us set up this tool in 1989. One is retired, and the other person is somewhere in heaven!”

    That was the remark of Alf (name changed), the senior mainframe architect at a large European enterprise, as he pointed toward the ceiling of the meeting room. Everyone else in the room let out a half-nervous smile, unsure how to react to the statement and hoping for some visual cues from Alf. We were all talking about a specific version control and packaging tool on the mainframes, which was heavily customized to suit this particular enterprise. The attendees were a mix of DevOps enthusiasts and mainframe architects and engineers with the purpose of re-evaluating the existing tooling stack on mainframes with a view to modernize them as part of a wider agile/DevOps initiative. Alf’s remark in the meeting has stayed with me ever since.

    Later that week, I was having lunch with a group of new joiners in the technology department, who were all part of a greenhouse initiative to in-source digital skills. During the conversation, I asked which areas where they focused on and the career direction they had in mind. The answers were predictable: data science, python, Nodejs, UX, blockchain, JavaScript etc. If all the new kids on the block are learning the newer technologies, then who will take over from mainframe people over the next few years when they retire?

    The reality is that 96 of the world’s top 100 banks, 23 of the 25 top U.S. retailers and 9 out of 10 of the world’s largest insurance companies run IBM’s System z mainframes. Most digital transformations have barely scratched the surface of the legacy stack.

    Like many other enterprises, this enterprise had outsourced most of its application estate to IT service providers. In few years’ time, Alf will move to a well-deserved, retirement phase of his life and take his mammoth knowledge base that the enterprise  had always taken for granted. There are many more Alfs across various enterprises today with significant mainframe know-how who are set to disappear from workplace over the next few years, making these enterprises entirely dependent on external service providers—so much for those bold visionary statements for the CXOs such as, “Technology is business and business is technology.”

    Most enterprises today have ambitious digital transformation agendas, investing millions on various initiatives ranging from digital, DevOps, agile, UX and blockchain. This will manage to “DevOpsify” the enterprise skin (aka front end, customer experience layer, or mode 2 apps) and, in many cases, possibly, their non-legacy middle-end system as well. However, the majority of the back-end stack, especially in financial services, remains in the legacy world (zSeries  or iSeries).

    One might argue with a seemingly coherent logic that, “Well, we won’t have mainframes in a few years as we will move to newer platforms soon.” But it helps to remember Mark Twain’s quip, “The rumors of my demise are greatly exaggerated,” applies so well to the mainframe world!

    In simple terms, your enterprise most likely will adopt a combination of the following strategies to deal with the legacy challenge:

    • Retire mainframe app, in some cases.
    • Extract business logic and move it to distributed platforms.
    • Componentize the monolithic mainframe code and hook them to APIs for external consumption.
    • Build new platforms to replace legacy apps.

    There is one least common denominator for any of the above strategic direction that your enterprises take: ever-diminishing mainframe knowledge! So, I ask again: Where is the next generation of mainframe talent?

    — Aditya Vadaganadam

  • Rollout Adds Configuration as Code Support and Integration with GitHub to Treat Feature Flags as Mission Critical Operations

    Plans to Open Source Flagship Platform to Drive Innovation and Provide Operational Benefits of Software Production Changes Without Sacrificing Management Processes

     San Francisco, October 24, 2018 – Rollout.io, a feature delivery and management company, today announced its feature management platform will leverage Configuration as Code (CaC) and a powerful integration with GitHub, so developers can treat feature flags as they treat any other mission-critical infrastructure.

    Additionally, Rollout plans to open source its feature management platform, providing greater flexibility and stronger ownership of feature changes in production. Both moves allow development organizations to gain the benefits of using feature flags – such as increased speed of development and deployment, and reduced risks of deploying new code – without sacrificing any of the known management processes that govern the software development lifecycle (SDLC).

    “Rollout’s CaC approach, the integration with GitHub, and the decision to open source its feature flag technology means that developers now enjoy the benefit of both worlds – faster development and deployment of new features without the risk of breaking things,” said Kyle Daigle, director of Ecosystem Engineering for GitHub.

    Feature flags modeled with CaC means that for the first time, software developers can design, implement, and deploy application features using the same processes used to manage code changes, like full change management, ability to rollback, track ownership of changes, and more, to reduce the inherent risks of breaking production. The ability to treat feature flags like configuration as code and use the same tools as any other software project allows dev teams to be more agile, and rapidly deploy features that add value to a business much faster than before.

    “Until now, control of feature flags has circumvented existing software development and deployment processes or known checks and balances for software development and production changes,” said Erez Rusovsky, co-founder and CEO of Rollout.io. “The decision to integrate our configuration as code with Git & GitHub and open source our platform means that product and development teams have consistent and reliable control of every feature in production, resulting in applications that are easier to develop and maintain.”

    Rollout’s feature management solution with CaC will initially leverage GitHub as an integration point with support for Atlassian, Gitlab and other hosting services coming soon.

    “We use feature flags to continuously deploy new features to production without risk so, it was important to find a solution that is engineering-friendly. One platform met those requirements, and it was Rollout,” explained Ron Shoshani, vice president of R&D at Testim.io. “Integration with GitHub allows our dev teams to work faster in collaboration on dev and deploy goals. Rollout’s plans for open source will ultimately provide us with more flexibility and ownership over the code that drives our business.”

    Using Rollout, developers, DevOps and product managers can speed the development and deployment of software, while reducing risk and improving customer satisfaction and experience. Rollout designed its management system with feature flags and controls for gradually releasing features in development and production with the ability to rollout back features when needed. This is possible by separating version release from feature deployments, and it provides companies with flexibility to deliver features to specific users at a granular level.

    Rollout supports multiple clients and backend platforms. It adds feature level support across an organization’s entire stack. The company also recently announced its integration with Atlassian’s Jira Software.

    Pricing for Rollout’s feature management is based on seats and MAU and starts at $75.00. To try Rollout, please visit www.rollout.io.

     Learn more:

    For more information on feature flags, check out Rollout’s eBook The Ultimate Feature Flag Guide or follow Rollout’s blog, or on Twitter @RolloutIO and LinkedIn.

    About Rollout.io

    Rollout.io is a feature delivery and management company that accelerates software development and release and minimizes the risk of deploying new code. It is the only unified platform for feature delivery, experimentation, and application-layer remote configuration. Built for engineers, developers and product managers, to rollout, rollback, test and adapt features at scale, in real time. Rollout gives teams full visibility and control of all application features to deliver the right feature to the right user at the right time. Founded in 2014, Rollout has offices in San Francisco and Tel Aviv, Israel. Learn more at www.rollout.io.

  • Redgate enhances the appeal of compliant database DevOps with new provisioning dashboard for SQL Clone

    Cambridge UK, Wednesday 24 October 2018 – The leading Microsoft SQL Server tools vendor, Redgate, has introduced a new at-a-glance dashboard to its database provisioning solution, giving users an easy way to manage, organize and make available masked copies of databases.

    Copies of production databases are commonly used in development to test changes against and ensure they are not breaking changes. Shifting left the testing in such a manner is one of the ways a DevOps approach can be applied to the database in order to speed up development and minimize deployment errors.

    Redgate’s provisioning solution uses the tried and tested virtualization technology built into 64-bit Windows to create database copies, or clones, which are a fraction of the size of the original. While they work just like normal SQL Server databases and can be connected to and edited with any program, they typically take up only megabytes of space, even for databases of 64TB in size.

    As importantly in an age when data breaches are prompting the introduction of stricter data privacy legislation, the provisioning solution also enables sensitive data in the clones to be masked at the time they are created.

    Rather than simply randomizing the data, it uses pseudonymization and anonymization techniques to ensure the database is truly representative of the original and retains its referential integrity and distribution characteristics. The sanitized copies can then be used to speed up development, accurately test code, and fix issues faster. Their small size also encourages a dedicated development model where developers are free to have their own personal copy of the database to work with.

    The increasing demand for and use of masked copies, however, can make managing multiple copies across disparate teams a complicated task. To address this, Redgate has developed a dashboard to make the searching, grouping, renaming and management of copies fast and easy.

    The copies in use can now be grouped by location, the database version they were created from, and the instance they are on, and searched by name, creation date or size from the dashboard. Importantly, if a developer tests a change which breaks their copy of the database, it can be reset to its original state in seconds rather than having to be deleted and recreated.

    The activity view additionally gives Database Administrators an instant picture of what has happened over the last seven days, who did it, and any errors that occurred. For errors that occurred previously, the log files can be retrieved in a couple of keystrokes.

    To increase security and further demonstrate compliance with data protection legislation, Active Directory role-based access can also be used so that users without appropriate permission only have access to masked copies of the database.

    As Rebecca Edwards, Redgate Product Marketing Manager, concludes: “At Redgate, we’re constantly talking to companies to discover what challenges they face in their desire to develop applications and databases faster while also keeping data safe. This latest addition to our database provisioning solution will help them comply with data protection legislation – and demonstrate compliance – while making their jobs easier.”