devopsy.kinsta.cloud

Category: DevOps Practice

  • The Move to Platform: Unleashing a Wealth of Opportunities in Fintech

    The Move to Platform: Unleashing a Wealth of Opportunities in Fintech

    Closed, legacy systems and requirements around availability and security remain major obstacles to innovation in the financial services industry. But there’s now an appetite for change and a growing recognition of the benefits to be gained from a platform-based approach. This article explores the legacy challenges that still plague financial services, examines the case for a move to platform and looks at the opportunities for developers in fintech moving forward.

    Overcoming Legacy

    Would you be surprised to learn that most financial organizations are still using the same legacy core banking systems they were using in the 1980s? Developed using languages such as COBOL, these proprietary back-end systems were deployed to speed the efficiency of specific tasks, including bank transactions and financial trading.

    As organizations added more applications over subsequent decades, they retained their core banking systems as the foundation on which everything else was built. This approach maximized ROI and achieved the best possible performance because mainframes were still faster than x86 servers.

    Each application was selected for its “best of breed” capabilities that solved specific business challenges, increased operational efficiency and protected against downtime. However, the new applications introduced new file formats, new programming languages such as C++ and new operating systems including Unix, Windows and flavors of Linux.  Banks now had even more silos and the need for even more technology experts to maintain them.

    Fast forward to a few years ago, and the next wave of innovation was driving digital workflows. Banks invested significant time and money to create custom connections between all their silos. Building on legacy was still faster and less risky than ripping and replacing the back end. However, point-to-point connections and new web-based apps introduced even more technologies such as Python, Ruby, Java and Javascript. Banks’ IT departments were now almost completely absorbed with just maintaining their legacy environments and meeting the needs of employees.

    Large banks today can have thousands of applications, and some of them can have 10 million lines of code. Even a small change can cause stability issues and downtime. A bank can lose millions or even billions of dollars a day if just one application goes down. This makes testing extremely critical. But the complexity of legacy environments translates into slower development cycles. Not only do projects require experts that understand legacy technologies, but also it often takes significant time to stand up the complicated systems to match the complex production environments. If a new application makes it to production, it’s often too late to make a competitive difference.

    What’s the Way Forward?

    Consumers today expect modern apps and services and they’re not seeing them fast enough from their banks, lenders and investors. Financial organizations must find a way to meet their customers’ expectations for anytime, anywhere access to their data and services, while maintaining their legacy investments.

    One way to accelerate development of modern customer-centric services with legacy systems is to use a cloud-based platform, based on open technologies, to facilitate communication between legacy systems and all new apps and services. This platform-based approach makes it possible to innovate and meet other requirements including security while retaining legacy as long as needed.

    We’re starting to see a platform revolution in financial services. By the end of 2019, financial experts predict that 25 percent of retail banks will be working with startups to replace legacy and mobile systems.

    An open-standards based platform that connects developers with global financial services companies is crucial in allowing collaboration and innovation on a global scale. So, instead of always having to develop dedicated solutions for individual silos, developers can first check and see if someone else has developed an app with an open API that solves all or part of their challenge.

    What Are the Opportunities for Developers?

    It’s an ideal time for developers to get involved in bringing new ideas and innovation to financial services. Banks are finally facing up to the threat posed by the tech giants and the need to digitally transform the services they offer.

    Banks’ willingness to open up and adopt a platform-based approach is growing and they are also starting to recognize the value in being able to tap into all the IT creativity and innovation that’s happening in other industries. Importantly, developers will no longer require specialized knowledge about the proprietary APIs and technologies supporting their legacy systems.

    Studies show that some of the best solution ideas come from people from analogous fields—fields that are different from each other but the same on some structural level. So, a developer who works on genomic data models may be able to help a hedge fund manager who needs to uncover patterns in data. Or a retail developer might offer new perspectives on how to manage transaction processing.

    In the past, if a bank wanted to evolve their technology in some way, they needed to engage a fintech, and there was a lot of back and forth. The process was so complicated. It could take months and many times it wasn’t successful. Using a platform-based approach that provides connectivity to a standard set of pre-built and fully integrated open APIs can dramatically optimize the software development environment, allowing apps to be developed in a matter of days.

    Giving developers the ability to interact with open APIs in a sandbox environment will allow them to experiment online in bringing data into their applications and to test the inputs and outputs in a safe environment. Using a low code development environment, they can create UX and business logic in a few clicks and test them in real time. Publishing those apps in a store helps developers to get visibility and solve the distribution problem. Finally, banks can purchase them and run them in the cloud through automated delivery, supported behind the scenes by CI/CD pipelines. Banks don’t need to worry about provisioning any additional hardware as the apps run in the cloud, and support hybrid scenarios as well, where the core system remains on-premises.

    The platform ecosystem is the future of how apps will be developed, deployed and consumed. It enables such convenience for the user that it is inevitable. Being able to reuse tools that other people develop instead of reinventing the wheel in every project will also save a tremendous amount of time.

    My advice for developers today? Seriously consider what value you can bring to financial services—banks and other providers need your support!

    — Félix Grévy

  • Threat Modeling: The Why, How, When and Which Tools

    Threat Modeling: The Why, How, When and Which Tools

    You may have heard of threat modeling as a structured activity for identifying and managing threats. Threat modeling can be applied to a wide range of things, including software, applications, systems, networks, IOTs and business processes.

    Threat modeling’s motto should be, “The earlier the better, but not too late and never ignore.” Without threat modeling, your security is a gamble—and in today’s business environment,  you’re sure to lose.

    When you design an application, you will face several security issues during different phases of the software development life cycle (SDLC), and so having threat modeling in the SDLC from the beginning can help to safeguard that applications are being developed, with security built in.

    Simply put, threat modeling is a procedure to identify threats and vulnerabilities in the earliest stage of the development life cycle to identify gaps and mitigate risk, which guarantees that a secure application is being built, saving both revenue and time.

    Why Threat Modeling?

    • It is better to find security flaws when there is time to fix them.
    • It can save time, revenue and the reputation of your company.
    • To build a secure application.
    • To bridge the gap between developers and security.
    • It provides a document of all the identified threats and rated threats.
    • It offers knowledge and awareness of the latest risks and vulnerabilities.

    How To Do Threat Modeling

    Many people think only security engineers can do threat modeling. That’s not true. Anyone, from developer to software project manager, can threat-model. In fact, I would suggest they should also know a little bit of threat modeling as part of their work.

    Let’s look at the elements of threat modeling:

    Assets: What valuable data and equipment should be secured?

    Threats: What the attacker can do to the system?

    Vulnerabilities: What are the flaws in the system that can allow an attacker to realize a threat?

    Steps to Threat Modeling

    Step 1: Identify the assets (database server, file servers, data lake stores, Active Directory, REST calls, configuration screens, Azure portal, authenticated and anonymous web user, Azure AAD client apps, database users, DB administrators)

    Step 2: Outline details of architecture on which the valuable asset is being processed. It may include the software framework, version and other architectural details (ASP.net web application connection to cloud data stores and third-party services using JWT tokens).

    Step 3: Break down the application regarding its process, including all the sub-processes that are running the application. We create a data flow diagram (DFD).

    Step 4: List identify threats in a descriptive way to review to process further.

    Step 5: Classify the threats with parallel instances so that threats can be identified in the application in a structured and repeatable manner.

    Step 6: Rate the severity of the threat.

    Threat Modelling Methodologies

    STRIDE (Uses application-centric approach)

    PASTA (risk-centric approach): Process for Attack Simulation and Threat Analysis

    TRIKE (risk-based approach with unique implementation and risk-modeling process)

    VAST (Visual, Agile and Simple Threat modelling)

    OCTAVE (focused on assessing organizational (non-technical) risks that may result from breached information assets.)

    When to use Threat Modeling

    In simple words, at the early stages of the SDLC, perform threat modeling:

    • Every time there is a change in the system’s architecture.
    • After a security incident has occurred or new vulnerabilities are introduced.
    • As soon as the architecture is ready.

    It is not a mandate to perform threat modeling at the early stages of the SDLC; you can still pick up threat modeling at any stages even if it is close to deployment.

    Threat models are continuously changing, and the models you prepared today may not be efficient tomorrow. And it is difficult to say that you are covered from all the threats. But if you have performed threat modeling and done whatever it takes to minimize your exposure to security risks, at least the impact of something very bad happening will be manageable (again, hopefully, but not a guarantee).

    Tools to Perform Threat Modeling

    I have used two tools for threat modeling, both of which are free to use.

    1. OWASP Threat Dragon
    2. Microsoft Threat Modelling Tool 2016.

    Let’s take a look at both the tools:

     

    Microsoft Threat Modelling Tool 2016         OWASP Threat Dragon

     
    Full version available for free (as of now) Alpha version available, flaws are still there. It is an OWASP incubator project, so it is at its early stage.
    Installable desktop application available. The tool comes in two variants: an online web application for GitHub and second an installable desktop application.
    Documentation easily available on Microsoft website. Default threat template is available. You have to search, default threat template is present
    I found the UX better, unique methodology available, the user can understand threats better. Guidance and feedback in drawing a model. Good UX. Fun and engaging user experience.
    Presently using this tool for threat modelling. Maybe I will use. I think will continue to improve.
    End-user documentation is available. End-user documentation is also available. Automated threats.
    Threat Grid, Template editor, Mitigating existing DFDs. Mitigating existing DFDs. Powerful threat mitigation. Rules have not yet been coded as it is the Alpha stage.
    Comes with a base set of threat definitions using STRIDE categories. Even in its current stage (alpha), you can create threats from those STRIDE categories.

     

    — Debarghya Pandit

  • Artificial Intelligence: Coming to the Rescue of ITOps

    Artificial Intelligence: Coming to the Rescue of ITOps

    According to McKinsey’s Global Institute Report of 2018, artificial intelligence (AI) has the potential to create an annual value of $3.5 billion to $5.8 billion across different industry sectors. Today, AI in finance and IT alone accounts for about $100 billion; hence, it is becoming quite the game changer in the IT world.

    With the onset of cloud adoption, the world of IT DevOps has changed dramatically. The focus of ITOps is changing to an integrated, service-centric approach that maximizes business services availability. AI can help ITOps in early detection of outages, potential root cause prediction, finding systems and nodes that are susceptible to outages, average resolution time and more. This article highlights a few use cases where AI can be integrated with ITOps, simplifying day-to-day operations and making remediation more robust.

    Predictive Analytics of Outages

    False positives can cause threat alert fatigue for ITOps teams. One survey indicates that about 52 percent of security alerts are generally false positives. This puts a lot of pressure on the teams, as they have to review each of these alerts manually. In such a scenario, deep neural networks can predict whether an alert will result in outages.

        Alerts                                                                                Layers                                                                                 Yes/No

    Feed Forward back propagation with two hidden layers should yield good results in terms of predicting outages as illustrated above. All alert types within a stipulated time can act as inputs and outages would be the output. Historical data should be used to train the model. Every enterprise has its own fault line and weakness, and it is only through historical data that latent features are surfaced; hence, every enterprise should build its own customized model, as a “one size fits all” model has a higher likelihood of not delivering expected outcomes.

    The alternate method is a logistic regression where all “alert types” are input variables and “binary outages” would be the output.

    Logistic regression measures the relationship between the categorical dependent variables and one or more independent variables by estimating probabilities using a logistic function, which is the cumulative logistic distribution. Thus, it treats the same set of problems as probit regression using similar techniques, with the latter using a cumulative normal distribution curve instead.

    Root Cause Classification and Prediction

    This is a two-step process. In the first step, root cause classification is done based on keyword search. From free-flow root cause analysis fields, natural language processing (NLP) is used to extract key values and classify into predefined root causes. This can be either supervised or unsupervised.

    In the second step, random forest for multi-class neural networks can be used to predict root causes while other attributes act as input. Based on the data volume and the datatype, one can choose the right classification model. In general, random forest has better accuracy, but it needs structured data and right labeling and it is less fault-tolerant to data quality. While a multi-class neural network will need a large volume of data to train, it is more fault-tolerant but slightly less accurate.

    Prediction of Average Time to Close a Ticket

    A simple weighted average formula can be used to predict time taken for ticket resolution:

    Avg time (t) = (a1.T1 + a2.T2+ a3.T3 )/(count of T1+T2+T3)

    Where T1 are ticket types.

    Other attributes can be used to segment the ticket into right cohorts to make it more predictable. This helps in better resource planning and utilization. Weightage of features can be done heuristically or empirically.

    Unusual Load on System

    Simple anomaly detection algorithms can inform whether the system is going through a normal load or it has high variance. A high variance/deviation from average on time series can inform the unusual activities or resources that are not freeing up. However, the algorithm should take care of seasonality, as a system load is a function of time and season.

    Given the above scenarios, it is obvious that AI has a tremendous opportunity to serve IT operations. It can be used for several ITOps including prediction, event correlation, detection of unusual loads on system (e.g. cyberattack) and remediation based on root cause analysis.

    — Vivek Singh

  • The Importance of Soft Skills in Engineering

    The Importance of Soft Skills in Engineering

    In college, professors regularly told us that in the real world, we need to be able to work in groups. As a young skeptical undergraduate, I naturally rolled my eyes every time I heard this. Most of us hated group projects. More often than not, people would go missing or not do their part with little to no consequence.

    After nearly five years of reflection on my college learning journey, I realize that while my professors were right about group work being important in the professional world, they were wrong about the way we go about working in groups. Instead of teaching us what is commonly called group work, I’ve found that honing my soft skills has best served me in group project settings.

    Google defines soft skills as, “Personal attributes that enable someone to interact effectively and harmoniously with other people.”

    Group projects in university teach us about scrum and agile principles—the concept of creating tasks and delegating them with everyone on the hook (to a certain degree) for the delivery of the project. It’s very clean, it’s very simple. It’s almost like a business contract; there is very little human emotion involved. It’s entirely possible to successfully deliver a project while being completely disgruntled at your team and the process.

    I’ve found that being forced to “learn” group work skills often leads to it being devalued as you leave school and step into your first professional role. This is ultimately detrimental over time, as soft skills are vital to successful engineers (especially as they become more senior).

    When I joined LinkedIn right after graduation, I knew it was my role to help build reliable infrastructure. I instinctively had an “us versus them” mentality with developers. Over time (and through several unfortunate experiences), I realized that this doesn’t actually work: I found myself in need of building some non-technical skills.

    Unfortunately, the “real world” is not cut-and-dry business transactions, as I came to learn. Saying “no” to developers and then asking them for things that site reliability engineers (SREs) consider sensible just doesn’t cut it. You have to be able to effectively communicate with others. Over the last five years, I learned a few soft skills can go a long way in working with others.

    Empathy

    One of the greatest traits you can have as an engineer is empathy. There’s a great quote from theologian Ian McLaren, who said, “Be kind, for everyone you meet is fighting a hard battle.” As invincible as we think we are, at the end of the day, we’re all human and it’s easy to forget that. Co-workers may have family or personal issues that they don’t speak about, or they may struggle to understand a concept. Empathy can help you better communicate with others. Approaching your colleagues with a kind tone rather than a grumpy or aggressive tone goes a long way in building a harmonious work environment.

    During university, I had a team member in a group project who was not able to fulfill his commitments by the required time. My immediate reaction was to let him know I was upset at him. Little did I know that his wife was ill and he was struggling to keep up with his study commitments. My frustration toward him didn’t help the situation and, given the extra context, is actually insensitive. Asking your peers how you can help them is far better than lashing out when things don’t go according to the plan. 

    Clear Communication

    One thing that struck me as an engineer is how many disagreements are based on communication failures. This includes assumptions, misreading intentions or not being specific enough with asks. Many of us work in poor-quality conference calls and text-based conversations. It’s very easy to lose context and intent in digital communication and equally easy to disagree because something wasn’t as clear as it was intended to be.

    Part of my role at LinkedIn is to review all site issues on a daily basis and ensure that we’ve addressed all concerns. When raising these concerns (usually during a phone conference), good intent can be lost quickly. Over time, I learned to ensure that my colleagues understand where my concerns come from and offer support to resolve any outstanding questions.

    Very simply put: Over-communicate. Make sure your tone and intent are very clear to the recipient and be sure to add as much detail to the feedback or request that you’re making.

    Humility

    One important thing I’ve come to learn is that, as SREs, we are expected to have all the answers; after all, our job skills are multi-faceted.

    Over the last four years at LinkedIn, I’ve worked on many different teams, from the traffic stack to the database platform, and amassed a wealth of knowledge on how those systems work—to the point where I could almost be considered an expert for all of those pieces of infrastructure. Naturally, people come to me asking questions about these subjects and unfortunately, I don’t always have all of the answers. It’s very easy to end up in an imposter syndrome position where you feel inadequate because you don’t have the answer to every question.

    This is actually counterintuitive. While great engineers have a solid understanding of the infrastructure, tools and teams around them, the best engineers will learn who the best subject matter experts are and will direct relevant queries to that person.

    This becomes a critical skill as you scale as an engineer. If 30 people are coming to you asking you about things that aren’t in your subject matter expertise, you can get slowed down and may not be able to give the best possible advice, effectively slowing the person asking the question. Knowing when to refer someone and whom to refer them to is part of being a great engineer.

    Mentorship

    The general career growth chart for SREs roughly looks like this:

    • You do toil tasks
    • You work on projects
    • You lead projects and delegate to others
    • You invisage projects and ask people to lead them
    • You create a team or company vision and get groups of people to lead them

    As you become more senior, your role becomes less dependent on day-to-day execution of toil and more focused on empowering those around you. You become less directly in control of the outcome of a project and your success comes from the success of those to whom you’ve delegated. This is where soft skills become important.

    Over the past two and a half years, I’ve been lucky to have a mentor who has helped my personal development. He has been a great role model, especially in teaching me how to lead and influence others. He has often said to me that you need to break through the eight-hour barrier. What he means is that you need to up-level those around you so that your impact on the business goes beyond just your work. Your contributions are also to others to whom you’ve scaled your knowledge and skills. As well as technically leading my team, I have the privilege of being a mentor to others at LinkedIn.

    These four skills are just a small set of a large base of soft skills. Mastering each of these takes time and practice, but ultimately makes you both a better engineer and person.

    — Michael Kehoe

  • An Easy Way to Optimize Network Performance Monitoring

    An Easy Way to Optimize Network Performance Monitoring

    Network performance monitoring, and especially network optimization, is more of an art than a science because there are so many factors that figure into network and application responsiveness. In addition, because there is a vast amount of data traversing the network, determining the right kind of data you need to monitor (and where you should you be capturing the data from) can become very difficult. As an example, according to research conducted by Enterprise Management Associates in October of 2016, 41 percent of IT personnel spend more than 50 percent of their time working on network and application performance problems.

    The data collection process is further complicated by the fact that tactical data loses up to 70 percent of its value after 30 minutes. This makes the speed and accuracy of data analysis critical if you hope to resolve your network or application issues.

    There is one easy way to improve your lot in life, though: Add a network visibility architecture. Most IT personnel have a security architecture and a network architecture, but very few have a visibility architecture. Network visibility is what enables you to capture the right data at the right time, so you can do something useful such as quickly isolate potential problems.

    Setting up a visibility architecture is straightforward. There are four main components:

    • Data access
    • Data manipulation
    • Application intelligence
    • Purpose-built analysis tools

    Data access can be provided by physical taps, virtual taps and bypass switches. These devices give you timely access to the data you need. A SPAN port also can be used; however, there can be issues with the quality of the data from this type of device, so it is not recommended.

    After you have acquired the monitoring data you need— probably along with a bunch that you don’t need—the next step is to send it to a network packet broker (NPB). This device provides granular filtering capability to maximize the flow of relevant information to your monitoring tools. Specific types of data can be segmented out and sent to specific tools to improve efficiency and faster time to resolution. NPBs can provide the following services as well: data aggregation, deduplication and load balancing of Layer 2 through 4 (of the OSI model) packet data.

    Application intelligence is another capability available through an NPB. This functionality provides additional filtering and analysis at the application layer, i.e. Layer 7 of the OSI stack. For instance, you can leverage application intelligence to prevent application bandwidth overloads on your network. You can literally see the amount of traffic on your network based upon application type (e.g. Hulu, Netflix, Pandora, FTP, HTTP, RTP, etc.). You also can use it to identify slow or underperforming applications.

    The final layer of the visibility architecture is made up of your security and monitoring tools. These devices typically are special-purpose tools (e.g., sniffer, network performance monitoring [NPM], application performance monitoring [APM], etc.) that are designed to analyze specific data. For instance, APM tools can provide real-time data analytics to help you manage your network and lets you see problems before your users do.

    However, standalone deployments of these tools can run into different problems, such as overloaded disk space and processing, the need for different interface ports based upon network traffic speed and the need for lots of input ports to capture data across the network. An NPB can be used to capture network data and filter that data before it goes to the NPM tool. This increases the efficiency of the tool by reducing clutter. The additional filtering of duplicate data further enhances the efficiency and also removes the storage waste associated with storing irrelevant data.

    Network administrators need network visibility solutions to help them discover, isolate and solve problems related to the network and its applications. This architecture gives your network tools a complete picture of the network so they can resolve issues faster.

    — Keith Bromley

  • Kubernetes Jenkins Master-Slave: Scaling the Scalability Issue

    Kubernetes Jenkins Master-Slave: Scaling the Scalability Issue

    LinkedIn, Sony, Dell and many other companies use Jenkins as a CI/CD tool and builds happen with substantial code bases that run 365 days in parallel. Generally, you may not have as much code to build, but the slave that’s created during the build process is in an upstate even after the build is completed. That can result in higher cost, higher unnecessary resource utilization and a more complex delivery pipeline.

    Is there the process to overcome this situation? Yes. The solution is scalability.

    Jenkins Scalability

    One of the strongest features that Jenkins has is scaling—pushing beyond its normal limits of productivity. In this article we’re taking a look at the Jenkins master/slave model with one central Jenkins instance, referred to as master, and couple of slave executables called slaves, with the master responsible for scheduling the jobs across the slaves. Some features of slaves are:

    • Running multiple build tasks in parallel.
    • Replacing falsify Jenkins instances automatically.
    • Spinning up and terminating the slaves based on the requirement and need, which leads to cost reduction.

    There are a few ways to implement Jenkins scaling; perhaps the easiest one is to implement is Jenkins scaling on Kubernetes.

    This article will answer certain questions regarding the scaling process on Kuberenetes, including:

    • How is a slave pod created when a master generates a build?
    • How is the build assigned to the newly created slave?
    • How does the slave communication happens with the master during and after build?

    Jenkins Master Installation

    Prerequisites:

    • Jenkins Docker file
    • Kubernetes-Jenkins-deployment.yaml file
    • Kubernetes-Jenkins-service.yaml file
    • Kubernetes persistent volume
    • Kubernetes persistent volume claim ref:blazer.com

    The preferred way to install Jenkins on Kubernetes is to create a custom Docker image based on the Jenkins base image. For now, you need to generate a Docker file that has the base image taken from Docker public repo, on which you can fine-tune your Jenkins application by adding plugins and installing Maven (RUN command) and alternative dependencies of your build along with Jenkins deployment.

    The Docker file will prepare the image such that you don’t need to install any plugins after Jenkins deployment—it’s all done when the Jenkins Docker image is created.

    Create a Jenkins-deployment.yaml file with deployment name, Docker custom Jenkins image, persistent volumes and other inputs, then deploy the application.

    Since each deployment creates a pod, let’s check the status of the pod and how the Jenkins was deployed on the Kubernetes cluster.

    Now, let’s create the service file for the Jenkins deployment to access the dashboard.

    From the above image we can access the Jenkins dashboard with node ip and port number generated by the service file (32683).

    Slave Configuration on Jenkins

    The moment the Jenkins dashboard is up, checking to make sure the Kubernetes plugin is installed. If plugin installation is done, it will provide the options to configure slave details in  Manage Jenkins–>Configure System–>Kubernetes–>Add Cloud

    Follow the below steps:

    Then go to  Manage Jenkins–>Configure global security–>agents–>check the protocols JNLP agents fixed:50000 and (Java Web Start Agent Protocol/1) as mentioned in the below image.

    Now, create a freestyle job and in the general tab check “Restrict where this project can be run” and give the slave-name that we have configured in the settings:

    Workflow

    • When the job is triggered from master it will look for the slave configuration details.
    • Based on the configuration given in master using the JNLP Jenkins slave image, a slave pod is created and assigned to the build generated from the master until the build gets completed.
    • On a successful build, the slave will check for other build schedulers for a certain period of time. If there is no response from the master, the slave will post existing build results to the master and terminates.

    — Lucky Kumar Sappa

  • Digital Transformation Strategy? Think Cloud

    Digital Transformation Strategy? Think Cloud

    A few years back companies used innovation and digital transformation mostly to differentiate themselves and to stay competitive. The drastic growth in digital and cloud computing over the last couple of years have changed this mindset.

    Today, organizations have to be innovative and leverage the latest technologies just to sustain and stay in business. Enterprises that implement online retail business, online banking and several other online services aren’t considering those channels simply as another route to increase their revenue—they have realized that online services are fast becoming their primary revenue channel. According to Forrester’s Report, “The Digital Business Imperative,” data analyzed just for three months showed that 84 percent of U.S. banking customers had used online banking for their transactions and 43 percent of the customers had used a mobile phone for these activities.

    Organizations are working at a very fast pace reviewing and analyzing their processes to seize opportunities for digital transformation. It is important to understand that to realize digital transformation, companies may need to completely re-engineer their current processes to make use of technologies such as the internet of things (IoT), big data analytics and artificial intelligence instead of doing patchwork on existing processes to adapt to digital technologies. It is also important for senior IT executives to consider digital initiatives in tandem with their cloud strategy instead of treating them in isolation. There is absolutely no doubt that the scale, speed and massive infrastructure requirements such as compute power, storage and other associated platforms to support these technologies are best provided by the cloud. The question to ask is, “Are we going to develop and host the infrastructure we need to support digital, or are we going to leverage the constantly improving robust infrastructure and services that cloud providers are offering?”

    Organizations that plan for massive growth and transformation are usually the ones that invest considerably in technology to support innovative ideas. Advancements in the digital space help fuel innovation, but without a solid cloud strategy accompanied by agile development processes, ideas are likely to remain simply as ideas on paper and will take forever to get materialized into products or solutions that provide value. In today’s context, digital and cloud are almost inseparable due to the robust infrastructure, services and tools available on the cloud to support digital initiatives. All leading cloud providers offer competing solutions and services to help organizations move forward with their digital initiatives at a rapid pace.

    In my previous article, “Transforming with the Cloud: If Not Now, When?” I touched upon some of the key areas that organizations should be considering with regards to transforming their businesses with the cloud. I have attempted to structure this article in a similar fashion, with the focus being on the role played by cloud to support some of the key digital technologies such as artificial intelligence, big data and analytics and IoT. The idea is to provide a very high-level overview of the digital landscape and how the leading cloud providers are helping enterprises with their digital initiatives.

    Artificial Intelligence

    The digital revolution, which centered around mass production of computers and communication devices, has changed the way businesses have operated over the last several decades, resulting in constant improvements ranging from generating new ideas for products and services to innovative product designs to improving the customer experience. At present, the world is going through another, possibly even stronger wave of revolution. It is the use of artificial intelligence to perform complex cognitive tasks to solve business problems in ways that previously were either highly complicated or extremely resource-intensive. Most organizations deal with business propositions that yield small to medium value and require a high volume of human work, such as reviewing large numbers of documents such as RFPs to understand requirements and estimate costs versus returns to determine whether to proceed with those proposals. AI is proving to be the best alternative to handle cases such as these that are currently being dealt by human beings, but the volume of work involved challenges businesses on the feasibility of continuing those without AI.

    AI systems try to mimic the human brain, which uses patterns to generate perceptions, and logic to drive the structured approach of analyzing a situation from a rational perspective. AI systems process large volumes of data that could come in from various sources such as sensors, online applications, textual data from social media and the like. AI processes the data using perception to analyze patterns and incorporates machine learning to utilize structured evaluation methods and rational decision-making, not only to extract pieces of meaningful information, but also to assemble such pieces of information to make valuable decisions.

    Cloud plays a significant role in enhancing the power of applications that incorporate AI. Almost all major players in the cloud business have developed AI services that use powerful cognitive engines to process structured data such as relational databases and unstructured data from NoSQL databases, sensors and other technologies that get uploaded to the cloud. The robust pattern-matching algorithms and the powerful logic component built into these cognitive engines are highly sophisticated. Data and compute power are the two most critical requirements to make these engines effective. The engines predict more accurately with larger datasets. AI applications such as image recognition, video analytics, natural language processing and speech recognition leverage machine learning using highly sophisticated neural networks that perform detection and prediction from large volumes of data. Parallel processing with the use of graphics processing units (GPUs) make these data processing and computations faster. Building and implementing such robust GPU-based parallel processing engines on-premises would be a very expensive and resource-intensive initiative. Cloud addresses this problem by providing APIs to access several machine learning services such as video analytics, speech recognition, process automation, vision detection and natural language processing. Behind these APIs are complex infrastructures that combine the power of clusters of GPU-based compute engines, neural networks and data lakes.

    Amazon’s AWS Marketplace offers machine learning and AI software solutions from several major software vendors to help AWS customers increase the efficiency and productivity of their businesses. AWS’ artificial intelligence solutions offer several pretrained services that can be utilized by developers to add intelligence to their applications via simple API calls. Advanced deep learning functionalities are provided by Amazon Lex, which chatbots could incorporate for automatic speech recognition and natural language processing abilities. Similarly, AWS provides Amazon Polly to build applications that can convert text into speech delivered in natural sounding male or female voices. Amazon Rekognition, meanwhile, uses technology to analyze billions of images, so applications can be developed to search images, detect and compare faces, objects and scenes.

    Google Cloud also offers competing products and services to support their customers’ AI initiatives. Google Cloud Natural Language provides powerful machine learning models that analyze text to provide meaningful insight such as customer sentiments about a product via conversations on social media. Google provides Cloud Speech-to-Text, a service that understands about 120 languages and converts audio to text. Similarly, Cloud Text-to-Speech can convert text to natural-sounding speech in 30 different voices and in several languages. Google also provides products such as Cloud AutoML, a suite of machine learning products, and Cloud Machine Learning (ML), a managed service that can be leveraged by developers and data scientists to build powerful machine learning models.

    Microsoft Azure provides the infrastructure, tools and services to build smart applications with data that can live in the cloud, on-premises or on edge devices. Several products and tools are available via marketplace, and Azure provides sophisticated AI services to integrate the tools to develop intelligent business applications. Powerful chatbots could be developed by integrating Azure Bot Service, language understanding and external systems such as Office 365 and Dynamics CRM to streamline and automate common activities. Another example is an automated failure detection system for assembly lines built based on Azure’s image classification system using a convolutional neural network to help improve the identification of faulty electronic components. Azure also provides Visual Studio Code Tools to build, test and deploy AI solutions. The powerful AI Toolkit for IoT Edge from Azure enables IoT devices with AI that runs locally through pre-built deep learning models. Last but not least, Azure’s Machine Learning Studio helps develop and manage predictive-analytics solutions.

    Big Data and Analytics

    With the growth of internet, cloud and social media, there has also been an exponential growth of data across the world. According to statistics on big data generation in the last five years, the average volume of data created in the world every day is about 2.3 trillion Gigabytes. When the nature of the data was more structured and organized, companies relied on data warehouses and BI applications to help make important data-driven business decisions. Traditional data warehouses were built based on relational databases that could be queried using SQL; the data could be extracted, transformed and loaded from one or more data sources via ETL jobs that ran up to several times a day. This overall approach has proved to be ineffective when it comes to handling and modifying continuous streams of real-time data flowing in from multiple dissimilar sources such as social media, IoT, the public web and relational databases. Big data analytics, which helps examine large structured and unstructured data sets, has become a major enabler for enterprises to make critical business decisions by providing insight and knowledge with data mining, predictive analytics and forecasting. As big data processing evolved, the concept of data lakes came into existence. A data lake is a centralized repository that allows the storing of structured and unstructured data as is and enables the use of a variety of tools and approaches to answer business questions.

    The ability of the cloud to scale vertically and horizontally makes it the ideal platform for big data hosting and analytics. With vertical scaling, it is possible to increase the capacity of a server by adding resources as needed by the applications. Horizontal scaling allows expanding hardware as the processing requirements increase. Hadoop, which leads the big data revolution, is designed as a distributed system so that it can take advantage of scaling. Parallel processing is an important part of its design so that the system can process several independent small tasks such as serving data stores and file systems, processing streaming data and handling queries in tandem. Cloud-based systems offer high bandwidth, enormous amounts of memory and scalable processing power to help big data applications with improved real-time processing and analysis of streaming data. Cloud undoubtedly is an obvious choice for applications running large workloads and storing enormous volumes of data. Cloud providers offer highly scalable database services coupled with tools and services to support information-management, business intelligence and analytics.

    All major cloud service providers offer competitive and comprehensive tools and approaches to process big data at the volume and velocity that your business demands, while at the same time being conscious to reduce the associated operational costs. To perform faster and efficient transfer of petabytes of data to the cloud, AWS provides solutions that employ different techniques such as network multiplexing, reduced storage footprint via data deduplication and compression and advanced data replication. AWS offers services including Amazon Elastic Compute Cloud (EC2) and Simple Storage Service (S3) to provide the elasticity needed for your data lake along with your choice of database products, ETL and reporting tools via AMS Marketplace solutions. Similarly, with Microsoft Azure HDInsight, solutions can be developed using open-source frameworks such as Hadoop, Spark and Hive to analyze streaming or historical data. To process large volumes of data, HDInsight offers a broad range of cost-effective solutions for ETL and data warehousing. Azure HDInsight offers specific cluster types such as Apache Hadoop, Apache Spark, R Server and Apache HBase, with cluster customization capabilities such as adding components, utilities and languages.

    Internet of Things

    IoT is the concept of connecting devices such as security sensors, surveillance cameras, smartphones, smart watches and other wearables and even household appliances such as washing machines, refrigerators and several others that have the ability to communicate and transfer data over the network without direct human interaction. This disruption in technology has not only empowered individuals and families with greater control of their household systems and appliances, but enabled organizations with data that provide greater insight into critical areas of business interest, opening the doors for innovative products, solutions and new business opportunities.

    The rate of adoption of IoT by businesses is growing exponentially as the number of connected devices continues to increase. As per Gartner’s recent study, businesses could already be using as many as 3.1 billion IoT devices today, and by 2020 this number is likely to rise to about 7.6 billion. According to an article in Forbes, IoT will have an impact in all industries, from manufacturing to logistics to health care. We are not too far from the days where nearly everything in the world will be connected. IoT is at present playing a dominant role in optimizing production, managing supply chains, tracking assets, making financial decisions and improving the customer experience. Digital transformation of the fitness industry has been remarkable. Wearable devices with built-in sensors can constantly collect data on physical activities such as distance travelled, calories burnt and sleep patterns to provide detailed analyses and insight for continuous health care. IoT applications are designed to use data from connected devices, and the sophisticated tools available in the cloud let you visualize, explore and build complex analytics. In complex IoT applications that use several devices, it is important to understand the state of the devices and to frequently communicate to the application components that leverage those devices. It is also essential to ensure secure identity and access between devices and applications.

    As with any revolution in technology, IoT also has its associated challenges. With more devices being connected to the internet, the volume of data generated is also immense, which in turn puts significant pressure on the internet. So there obviously is a need to support infrastructure that can transmit and store this data more efficiently. Due to the vast number of connected devices, and with that number constantly increasing, there is a push toward making these “edge” devices intelligent to perform some degree of processing and then send the results to the servers rather than sending massive amounts of data to central servers to perform all the processing. For example, surveillance cameras in the past would send all the videos to the central recording device, which in turn would record only when it detects motion. Consider the scenario where hundreds of high-definition surveillance cameras are sending video feeds constantly to the central server—the impact this could have on the network is unimaginable. It should also be noted that as the physical distance between the devices and the server increases, network transmission latency increases. With edge computing, the video cameras are smart enough to sense motion and they send videos to the cloud-based central recording system only whenever they detect motion. This drastically reduces the volume of data that is transmitted over the network and makes it a more efficient system. Cloud and IoT complement each other—connected devices generate huge amounts of data and cloud provides the infrastructure to store, process and analyze the data. It is also interesting to note that IoT is an important data source for big data analytics.

    Cloud IoT Core service, built by Google Cloud Platform (GCP) partners, offers connectivity to devices by providing the necessary hardware, software and solutions. These services include secure connection of devices (via identity and authentication) over industry standard protocols including MQTT and HTTP.  Cloud IoT Core also allows remote controlling of devices from the cloud by maintaining a logical configuration of the devices. Since this IoT service is serverless, there is no need for upfront software installation and the horizontal scaling of the GCP can instantly be leveraged when required. A host of tools such as Dataflow, BigQuery, Bigtable and Data Studio are available to help with data analytics and insights.

    Microsoft provides services for IoT via the Azure IoT Hub. Features offered by the Azure IoT Hub include ensuring bi-directional communication with billions of IoT devices, establishing device-specific access control, secure and scalable provisioning of devices and integration between cloud and edge to distribute intelligence across devices. Azure IoT Edge provides features for optimized performance of the edge and cloud such as ensuring the lowest latency between edge devices and the cloud. IoT Edge helps control IoT costs by reducing the volume of data transferred from the edge devices to the cloud. It leverages services such as Azure Stream Analytics to process data at the edge and transmit only the bare minimum data that is needed by the cloud for further analysis.

    Amazon provides a fleet of services such as AWS IoT Core, AWS IoT Device Management, AWS IoT Analytics and several others to support IoT initiatives. For easy deployment, programming and management, Amazon offers the FreeRTOS lightweight operating system for small edge devices. The AWS Greengrass software further enhances the power of edge devices by offering local compute, secure messaging and data caching. AWS IoT Analytics, a fully managed service, provides all the capabilities required to run complex analytics on IoT data to obtain insights and to build machine learning use cases.

    Taking Everything Into Account

    Technology is certainly a major driver in any business today and companies that struggle to integrate technology effectively will eventually find it difficult to sustain or do well in their marketplace. Connected devices, social media and massive volumes of several forms of structured and unstructured data feeds have paved the way to further leverage technology and transform businesses. This “digital transformation,” which includes use of artificial intelligence, big data analytics and IoT, is fast becoming a key requirement for organizations to be innovative and competitive. Digital has opened doors for better analytics and decision-making, leading enterprises to explore, analyze and obtain new insights and ideas on growing their businesses.

    With cloud reaching a high level of maturity with regards to services, tools and security, it has become the ideal platform for digital technologies. All leading cloud providers have comprehensive and competitive solutions, tools and associated services to address even the most complex digital transformation initiatives for their customers. Furthermore, the horizontal and vertical scaling of infrastructure offered by the cloud makes it highly suitable for the compute requirements demanded by digital technologies.

    Escaping from or ignoring the digital revolution will be disastrous to organizations as well as individuals. Humans and businesses are generating trillions of gigabytes of data every day. Whether we have realized it or not, digital is transforming our lives. It is important for organizations to get on board soon and devise a digital strategy that works together with their cloud initiatives and strategy.

    — Sridhar Asvathanarayanan

  • DevOps in the Age of Blockchain

    DevOps in the Age of Blockchain

    The current state of software development results in a lot of redundant efforts. The same functionality is built repeatedly, which is a waste of time and resources for a company. Open source has been a partial solution, but doesn’t fully solve the problem of reliable code upgrades, which are critical for sustaining a high speed of innovation. This especially acute in decentralized networks, where new releases must be pushed to potentially thousands of independent nodes. While blockchain has become one of the biggest buzzwords of 2018, it’s important to explore how this technology will apply to developers’ workflow in the coming years.

    The DevOps culture has been making inroads to fixing this central issue by streamlining and automating the delivery processes. By taking ownership of all levels of the development cycle—increased release frequency, construction, testing, integration—software developers have been better able to keep up with business objectives.

    From a business perspective, the principles inherent in DevOps are great, but they are only the first step. While automation and continuous integration has helped speed up the development process, utilizing blockchain technology can become one way to help make the process both instantaneous and transparent.

    Upgrading Contracts

    A major distinction between a DevOps team in its current iteration and blockchain-era continuous integration is smart contract upgradability. A traditional software-as-a-service (SaaS) company has full control over versions of software it publishes to its servers. In distributed blockchain-based networks, upgrading smart contract code is not that trivial and often is not even supported natively (e.g. in Ethereum).

    Distributed governance is still in the early days, and since every upgrade to smart contracts has to go through the governing procedures it is difficult to come up with a single solution addressing most models. This leads to manual error-prone implementations of smart contract migrations that can cause bugs, financial losses and even inability to make further upgrades. Standards and tools are yet to emerge in this arena.

    That said, it is easy to see how blockchain-based code migrations adds transparency and accountability comparing to centralized systems, where a single entity has full control over the application logic and can alter it at any time without regard for consumer interests. Use of the blockchain in this case is not meant to take over a DevOps team, but rather to integrate with the work of the team and make the process more transparent to all parties involved.

    Supporting Developers

    Not all developers currently have the skill set to create on the blockchain. But this is an issue of interest in protocol-level details, and time. Instead of expecting developers to learn every line of code and configuration used in blockchain applications, we need tools for common use cases in enterprise software so that developers aren’t reinventing the wheel every time. Again, this speaks to the complementary nature of blockchain technologies to DevOps. We want to create a suite of tools to make the DevOps experience even more effortless.

    In the emerging age of blockchain, DevOps still contributes the best practices needed in getting software released. By adding blockchain applications and tools to the development cycle of DevOps teams, DevOps will reach new heights of productivity.

    — Andrei Anisimov

  • Why Digital Transformation Can’t Happen without Experimentation

    Why Digital Transformation Can’t Happen without Experimentation

    A sure sign of a company’s successful digital transformation is when it can leverage data to optimize every area within the enterprise. As Forrester puts it, for a company to realize the return on investment (ROI) of a digital investment, “benefits from digital investments need to be aligned with the business requirements for a specific value chain stage.” All along, data-driven insights need to be tied to measurable gains and real-world business key performance indicators (KPIs).

    The reality, however, is that data-centric principles are easier to apply in some areas of the business than others. The ethos has certainly taken hold in the development world—with its whole culture around DevOps, continuous delivery (CD), continuous integration (CI) and related agile methodologies that emphasize constant, data-driven experimentation and frequent reassessment and adaptation of plans.

    Unfortunately, too many companies drop the innovative mindset once things progress to production and customer rollout phases. In doing so, they fail to spread the ROI from digital investments more fully across the organization. In other words, they fail to mount a genuine digital transformation.

    True digital transformation requires companies apply a stronger, more agile approach to product adjustments across the value chain. Nowhere is this more critical than the need to embrace “full-stack experimentation” to realize business value from the world of data-based product development and testing.

    The Limits of A/B Testing

    Especially in an era when the customer experience sits at the center of the value proposition for most industries, the ability to fine-tune everything about how we bring products and services to market—and how our customers use and feel about those products and services—is mission-critical, especially in the quest for true digital transformation.

    Unfortunately, no matter how agile and data-driven the development phase may initially be, we lose visibility, insight and, most importantly, revenue whenever we fail to infuse those same values into the later stages of production, QA and analysis around how customers are actually using what they buy.

    Perhaps the biggest sign of this is the over-reliance of marketing teams on signals from basic A/B testing.

    Industrywide, A/B testing is the most common approach to experimentation, but it’s far from ideal. A/B testing is typically used by marketing teams to test front-end changes at the user interface-level. It’s limited in its efficacy, as it does not take into consideration the deeper statistical nuances and resultant effects across the application stack—for instance, swapping out code to roll out an entirely new feature to a small set of users and measuring the results of that single deployment across both business and system metrics.

    Worse yet is the false sense of security that can come from the narrow scope of A/B testing. Any outcome can be easily misunderstood as the only improvement possible. That leaves countless other variables and opportunities for ROI unexamined. If Plato remarked, “The unexamined life is not worth living,” what does that say about the unexamined product development process? How do we know if it’s worth building?

    A Better Way

    While A/B testing is still appropriate for some limited uses, leading-edge companies are learning to expand inquiry across the “full stack” of their applications. Full-stack experimentation differs from traditional A/B testing in scope, purpose and the potential impact on a business. It examines not just user interfaces but also execution logic, technical services, APIs and the data infrastructure underlying it all. This fosters a market-driven approach that brings more insight and less guesswork to decision-making.

    To be clear, full-stack experimentation does not optimize part of a product or feature; it is about optimizing the process around experimentation and allowing development to more directly support and inform the evolution of the business. In this way, full-stack experimentation helps gain deeper value from the organization’s use of agile, DevOps and CI/CD so we can garner critical insights into how a product or feature is actually used by customers. More importantly, it measures the impact of every release on important key business metrics—which may be radically different from the developer’s intent.

    Full-stack experimentation unlocks value across the entire data ecosystem. It involves collaboration among the key stakeholders involved in the development of a product or application, from engineers to product managers, as opposed to primarily marketers. You can use full-stack experimentation to answer deeper questions and test deeper assumptions. For instance, does adding the ability to tag friends in a photo increase their likelihood of posting photos in the future? Does this result in more sessions per user and does that ultimately impact the ad revenue generated per user?

    Delivering Digital Transformation to More Companies

    The benefits of full-stack experimentation are well-known among organizations with the resources to build and manage this in-house. Microsoft, Etsy and Netflix, for instance, test everything from multiple angles, sometimes running 50 tests a day or more on a single feature. These companies have powerful instrumentation to capture and analyze the multitude of variables and metrics—including those that may have been unanticipated. This surfaces valuable insights to be quickly spotted and harnessed for value.

    Fortunately, modern software-as-a-service (SaaS) solutions are increasingly saving companies from having to create their own in-house, custom-built systems to get the benefits of full-stack experimentation. Unified platforms now exist that help engineering and product teams accelerate the pace of product delivery and make data-driven decisions without investing valuable development resources to build the tools themselves. Critically, these platforms have robust feature flagging and extensive experimentation capabilities that allow testing to happen well into production and rollout to customers—with strategic and incremental tests to examine and adjust logic, APIs, UX or other elements across the entire application ecosystem.

    On a tactical level, these new platforms make experimentation processes simpler and more accessible to more companies. But the bigger picture is that these new options for full-stack experimentation effectively remove the friction that exists in achieving true digital transformation—allowing more companies to completely revolutionize how they do business.

    — Kavitha Mariappan

  • SDN/NFV DevOps: Release Automation for Network Operators

    SDN/NFV DevOps: Release Automation for Network Operators

     A powerful force is waiting to be unleashed. Software defined networking (SDN) and network functions virtualization (NFV) are already acknowledged as exciting possibilities in the world of telecom and media. Leading providers, such as Verizon Enterprise Solutions and My Republic, along with original equipment manufacturers including Cisco Systems and Ericsson, have unveiled their SDN-based services.

    In the near future, all physical devices in the network will undoubtedly be replaced by SDN/NFV. Analysts predicted millions of additional devices on network because of the internet of things (IoT) and smart home intelligent devices, which necessitates the need to automate release and deployment on the networking layer to deploy components and upgrade patches to both the southbound network components and northbound applications. So far, release and deployment is successful on the application layer, but network administrators will be challenged in the near future to manage the surge in demand without a seamless automation solution to manage new devices on the network.

    In this article, we’ll take a look at various real-life scenarios to build a business case to solve bigger problems in networking—including reducing overall provisioning time, mean time to recover from failure, round-trip time for a packet to enable a usable application or time to free up/repurpose a network resource—that apply to all edge technologies. The solution lies in leveraging DevOps container-based solutions and scaled agile and SDN/NFV controller engine module customizations, and a non-prescriptive approach to products or solutions.

    These technologies are the new beasts in the market, but how can SDN go beyond merely saving money for telecom and media companies? How can it provide an enhanced, immersive digital experience, reduce customer churn rate and create new revenue streams? That is where DevOps steps in.

    The DevOps Approach to SDN

    With mobile, cloud, and IoT creating futuristic applications, consumers require strong underlying network speed and reliability.

    The DevOps solution to SDN/NFV is an ideal means for network operators to transition seamlessly from physical networks to futuristic SDN/NFV networks, to provide uninterrupted digital experiences for users, and to achieve high customer retention. DevOps enables a close collaboration between developer and network operators to eliminate many of the pains of introducing new software and services to market. Similar to the system development life cycle, the automated release and deployment process of DevOps enables frequent and incremental updates of network components in near-real time.

    The symbiotic relationship between SDN/NFV and DevOps is a dream. The software-defined and virtualized network elements remove the hassles of configuring and upgrading the physical networks’ devices. DevOps adds to this agility and programmability to streamline and shorten the implementation cycles, paving the way for automated service delivery. In turn, SDN and NFV create an elastic network environment, managed from a central location, to program and rapidly deploy new functionality on the network.

    Service providers can drastically reduce the time it takes to create and provision those network services, gaining a significant time-to-market advantage that helps them acquire new customers and retain existing ones. In the process, they can achieve higher revenues and cost savings, too.

    Creating New Dimensions in the Customer Experience

    Tech support has moved from phone to video, but before long, a centralized monitoring system will be able to auto detect and self-heal all issues related to virtual assets on the network. This will provide a more continuous experience for customers. For example, in case of any unexpected service interruption, TV channel providers’ AI-enabled systems will be able to record live programs and play back those interrupted periods to their viewers.

    Deploying a vendor-agnostic reliable SDN is a wait-and-watch situation for TV channel operators and telcos. Replacing their physical devices (including the conventional routers and switches) with software-enabled virtual devices will bring down printed circuit board (PCB) replacement costs and other associated maintenance fees. As reliable network bandwidth becomes all the more critical to improving channels’ television rating points (TRP), telecom carriers and TV channel operators are converging.

    Digital operators looking beyond TV and telcos anticipate millions of smart home appliances (such as security systems, microwaves, refrigerators and temperature control devices) controlled through the network by users from any remote location. Network administrators want to apply patches and upgrades to any SDN/NFV device, and to proactively fix any network-related issues without bothering their customers. Consumers may be controlling those home appliances from connected cars using mobile devices, so they may not be available physically to read glowing and blinking lights or to reset undecipherable, hard-to-access conventional modems and routers. Network providers are expected to manage millions of devices and provide uninterrupted service, so this paradigm shift, aided by technology transformation, has become imperative not only to reduce the call volumes of help desks but also to retain customers and acquire new ones. 

    Future-proofing for Technology Shifts    

    It is important to realize that telco carriers and TV channel operators are no longer merely confined to the communication and entertainment industry. With their scope vastly expanding into managing and controlling millions of household devices on their networks, it is imperative to future-proof technology with automated release-and-deployment processes over the network to support those devices.

    Some of the latest DevOps zero-touch release automation (RA) solutions can reduce the overall service provisioning time from months to a few hours. The scope is not just limited to configuring the southbound virtual software devices but may also be required to configure and manage northbound SDN/NFV applications to fulfill the ever-growing market. Having a graphical user interface (GUI)-based orchestration dashboard for network management can enable existing admins to continue without needing to acquire additional, niche skill sets to manage the new, massive networks.

    While release automation has been successful on the application layer, it has yet to prove itself on the networking layer. For SDN/NFV, release automation is just an engine that can be fitted into any controller to do virtual local area network provisioning, virtual management of firewalls and network functionalities management. Since there will be a shift from physical devices, there will be a desperate need to automate release and deployment of software versions on such networks.

    Adopting a DevOps strategy into SDN deployment requires a cultural shift from the traditional “big-lab,” acceptance-testing mindset to the “here-and-now” daily incremental changes in a live network. It calls for engineers to collaborate in managing the critical network-applications relationships. Consulting skillsets might be needed to assess the extent of customization required for legacy systems and to understand which technology solutions can deploy SDN, NFV and DevOps.

    It is clear that this industry will undergo a paradigm shift, and this technology is just crossing the chasm. Early adopters can give themselves an edge by leveraging compelling value propositions.

    — Ramesh Ganapathy