Category: DevOps Practice

  • How Can IT Admins Maintain a Grip on Secure Access?

    How Can IT Admins Maintain a Grip on Secure Access?

    Today’s IT environment is evolving at a rapid pace, requiring production environments to be highly scalable and elastic and requiring faster software release cycles and cloud adoption.

    As a result, there’s a broad move toward Docker containers to make software deployment faster, easier and more consistent across computing environments. Bundling an application and its supporting code, libraries, settings and assets into a single container enables the speed and flexibility that today’s business environment demands.

    At the same time, many large enterprises are also experimenting with creating internal startups, or labs, that are given latitude to innovate, housed under the company umbrella and resources. Similarly, some companies are hosting startup-like “innovation incubators,” where developers are empowered to think and work like startups, within the relative safety of the corporate environment. They can quickly test new ideas, using the “fail fast, succeed fast” startup mentality, without incurring the usual risks of an independent startup.

    It’s a new agile way of working that companies, in the race to innovate, are widely embracing. In fact, more than half of Fortune 500 companies have some version of these startup-like efforts housed within at least one business unit. 

    Replicating the unstructured movement of a small–say, six-person–startup is difficult to do within an enterprise environment. Nevertheless, the rapid pace, combined with the flexibility to innovate, is what the vast majority of developers desire today. But, it does mean that enterprises’ IT environments have to be ready to adapt.

    IT Administrators Under Pressure

    Developers on these innovation teams are often given the freedom to select the tools they’d like to work with. That can–and often does–include Docker containers, which adds to the complexity of the IT environment. 

    The environment that fuels developer innovation also creates additional concerns for the IT administrators supporting them. Developers can work so rapidly–creating hundreds of new servers each day–but often only do the bare minimum in terms of security. How can IT admins maintain secure access in the face of these changes? How can they keep track of who has access to what application and what data, when those applications and data are constantly changing? That’s a massive issue when it comes to compliance.

    As containers make enterprise IT environments even more complex, IT admins are under pressure to carefully manage access without slowing down development. The more complex the environment becomes, the more important it will be to have a holistic view of all of their underlying infrastructure, including the hybrid or multi-cloud environments that support platforms such as Docker and Kubernetes.

    Meanwhile, the enterprise executive team, often the CISO, overseeing both the development and IT departments is hard at work trying to satisfy the requirements of both groups. Of course, it’s in the business’s best interest to innovate and explore. But, all that innovation means nothing if the business is no longer secure and open to compliance concerns.

    Below are three steps executives should follow in order to keep their developers and IT admins happy.

    Consider the Developer Experience for Secure Access

    Off-the-shelf single sign-on solutions (SSO) may be good enough for business employees who need to access Outlook or Salesforce, but they aren’t robust enough for privileged IT users, like developers, who need to access secure environments. 

    Privileged access management (PAM) software sprung up to serve IT users, but even this isn’t a perfect solution, because traditional PAM tools are often too clunky to use, nonintuitive, or hard to configure. A bad user experience simply encourages developers to bypass PAM however they can, which is a compliance risk.

    Businesses need a way to deploy secure access very quickly, so they can holistically manage access to critical IT resources without slowing down development. That means choosing secure access solutions that are built with the developer’s needs in mind.

    Lean PAM solutions prioritize automated, instant access to secure IT environments with the click of a button, combining the convenience of modern SSO solutions with the security and fit of a PAM solution. If sysadmins have a great PAM user experience, they’re more likely to play by the rules. 

    Opt for Role-Based, Time-Bound Access

    To maintain the highest levels of security, access must be controlled on a role-by-role basis. Your rock-star in-house developers should be able to quickly spin up access to core IT infrastructure whenever they need it, while a third-party development contractor should only have limited access to those same resources for the time in which they are working in your environment–and no longer than that. 

    This is where we begin to see time-limited, credential-less secure access, enabled by ephemeral certificates, start to play a role in privileged access management. Ephemeral certificates are short-lived access tokens that are automatically generated and automatically expire–so access is granted only for as long as it is needed to authenticate and authorize privileged connections. 

    In other words, privileged access is able to move toward a just-in-time model, eliminating the need for passwords and clumsy credential management. IT users no longer need to authenticate using credentials at all–instead, based on predefined roles and security policies, they are granted access to resources only as and when required.

    Ultimately, ephemeral certificates help streamline access processes, ensuring that IT admins don’t have to worry about revoking access when access roles or needs change.

    Make It Easy on the IT Admin, Too

    Speaking of IT admins, it’s important to consider their experience, too.

    Admins want to give staff the best tools for the job without compromising corporate security. The friction between admins and developers begins when the developers feel that the tools admins force them to use are slowing down their work. In reaction, admins become frustrated when more unmanaged, unmonitored tools enter the corporate network, requiring more of their time to implement, configure or remediate if something has gone wrong.

    The right technology solutions make life easier for IT admins, too. That means secure access solutions that can be installed and deployed within a single day–not weeks–and that introduce automated routines and maintenance to eliminate manual work in the access management process. Accountability, in the form of session monitoring and logging, is also important. 

    Ultimately, it’s about giving IT admins some distance. They can retain the oversight they need to feel comfortable about organizational security, without needing to be on top of every single secure access decision in the organization. At the same time, developers retain the autonomy they need to work quickly without feeling like they’ve got someone looking over their shoulder. 

    By enabling rapid development, while keeping a watchful eye on enterprise security, enterprises can satisfy both their developers and IT admins, and ultimately, the business’s bottom line.

    — Markku Rossi

  • Shifting Left – But How Far Left Do Companies Need to Go?

    Shifting Left – But How Far Left Do Companies Need to Go?

    For developers, there’s nothing more satisfying than looking at a web app or service and being able to say “I made that.” Or at least part of that. Being able to develop code that works and helps millions of people do what they need to get done is an excuse for bragging rights.

    But the opposite is also true. You don’t want to be the developer whose code inadvertently enables hackers to take control of online apps or services, for instance. There are many examples of these kinds of errors. Hackers are better and more sophisticated than ever, and no programmer wants to be the culprit in the ever-increasing number of attacks based on poor coding practices.

    Where and when do errors like these creep in? It could actually be at any stage of development. Vulnerable code could be the basis for a main module in an app, including in the very early stages of development. Once implemented, bad code travels through all stages of development, making apps vulnerable to exploits. Hence the importance of shifting left–deploying security at as early a stage in the process of code creation as possible.

    What exactly does shifting left mean in the context of code development today? Tools need to be integrated in as early in the process of development as possible, and at every stage of the software development life cycle. Today, programmers will work on code and commit their changes several times a day. They need to be able to examine that code for security issues before committing that code to the repository that other teams will be basing their work on. That, of course, is what they would do if they were checking for functional defects, and the same must be done to exclude insecure code.

    What’s the best approach to shifting left in the early stages of development? Developers need tools that can supply them with actionable information, highlighting specific issues in specific lines of code. Sort of like a “spell checker” for security risks, the tools would point out the location and issue in a line of code, and even make recommendations for how to mitigate the risk.

    With that, there are some caveats that come with tools that report risks at such a granular level. Tools should be flexible enough not to over report issues. One general problem with security tools is the possibility of false positives. Security testing tools are notorious for generating very long lists of reported issues, with many of them turning out to be false positives or issues that are not critical to fix. So, to be taken seriously and provide real value, security testing tools need to effectively highlight the high-severity alerts that you cannot ignore and also separate out true vulnerability signals from noise (i.e. false positives).

    Another caveat is that other tools need to be deployed further on in the development process. When a module is completed, it becomes a building block for other modules that will eventually be part of the full program. Testing needs to be done to determine if there are security risks when modules work together. Thus, you should not consider security as shifting left, but rather expanding left, maintaining a presence alongside the development process as it moves forward. With this kind of strategy, programmers can be sure they will have those bragging rights, proudly pointing to what they built.

    — Shahar Sperling

  • Can We Do Without DevOps?

    Can We Do Without DevOps?

    Recently, DevOps has received so much hype and every respectable company sees it as its duty to have a DevOps team on and running. It is considered to be so beneficial to any company that a non-technician scrolling down posts at any tech website would think of it as a universal remedy for all the corporate diseases ever possible. However, given that a definition of DevOps varies from company to company and no one exactly knows what a true DevOps team should do, isn’t there some misconception between the praise that DevOps has been receiving and the real output? Let’s dig deeper and decide for ourselves.

    Yet, despite the buzz around the notion, DevOps is said to be a working thing. It guarantees a company-adopter 200 times more frequent deployments, 24 times faster recovery and three times lower change failure rates, according to Puppet’s State of DevOps Report.

    That is why, to get a non-biased view on why one should “DevOps-ize” corporate processes, in our article we give the answers to the following:

    • What DevOps is?
    • What issues is it supposed to help resolve?
    • What does a DevOps engineer do?
    • What skills does a DevOps engineer need?

    All of these questions, when answered, will help us see why the industry has adopted DevOps as a regular practice, why your company needs to try it out and why we can’t do without it the way we used to before—or can we?

    What Is DevOps?

    According to the AWS, DevOps is the combination of cultural philosophies, practices and tools that increases an organization’s ability to deliver applications and services at high velocity: evolving and improving products at a faster pace than organizations using traditional software development and infrastructure management processes. Well, this definition may not be as clear as the waters of Sardinia, so we’ll clear it up for you a little bit.

    The message is that once (before DevOps) the product release was slow and troublesome, while today, with DevOps practices in place, it almost reaches cosmic speeds and high-end quality. All through maturation of work approaches and practices–merge of development and operations–along with the automation, cloud technologies, virtualization and abundance of top-notch tools. DevOps culture to a certain extent enables developers to perform the work of the IT team, and vice versa, which breaks down the operational walls between the teams, eroding a “not my task” mindset at the same time.

    What Does a DevOps Engineer Do?

    If that’s what is happening today–programmers deal with the configuration issues and the IT teams code–why is a DevOps engineer on the tip of everyone’s tongue? Why come up with a new term if you can write some more new responsibilities into the dev’s/IT guy’s contract and leave the names as they are? The thing is, the transformation of the processes and roles was so substantial, that in time they merged into a completely new position, which is neither a coding IT guy, nor a configuring developer.

    Roles Before DevOps

    Developers

    • Built features, tried to merge separate pieces of code together (usually resulted in a huge amount of errors and ended up in a long and troublesome integration).

    Integrators or Production Facilitators

    • Known to have been engaged on huge banking/insurance projects.
    • The phenomenon had a limited application within the industry and this intermediary work piece was often shared between the devs and ops with varying success.

    IT/Operations

    • Fixed the developers’ mess and squeezed the available code (often requiring a lot of massaging) into the available infrastructure.
    • Supported tech life in the offices (repairs, tech support, configuration and security, etc.).

    Roles Today, Approximate Responsibilities Breakdown

    Developers

    • Specification-based code production.
    • Software components integration.
    • Troubleshoot, debug and upgrade of existing software.
    • Creation of technical documentation for reference and reporting.
    • Integrations implementation.

    DevOps Engineer

    • Infrastructure planning, testing and development.
    • Integrations implementation.
    • Updates and fixes.
    • Build tools to minimize errors and improve customer experience.
    • Develop software to integrate with internal back-end systems.
    • Production errors root cause analysis,
    • Investigation and resolution of technical issues, Level 2 tech support.
    • Script development to automate visualization.
    • Procedure design for system troubleshooting and maintenance.

    IT/Operations

    • Install and configure computer hardware, software, systems, networks, printers and scanners.
    • Monitor and maintain computer systems and networks.
    • Service issues and requests.
    • Technical support across the company.
    • Set up accounts for new users.
    • Repair and replace equipment as necessary.

    To sum it all up, a DevOps engineer is a so-called “centaur” performing duties of a system administrator between coding sessions. His/her primary objective of the project is to find a solution that will make product deployment fast and efficient and people’s (devs and ops) lives easier.

    What Skills Does a DevOps Engineer Need?

    According to the Qulix’ practice, an ideal skillset for a DevOps is:

    • A technical graduate.
    • Intermediate or advanced English.
    • General programming skills.
    • Proficient PC-user knowledgeable about networking, Git, Linux, Docker, Nginx, Bash.
    • Strive for constant perfection and thirst for the new.
    • Curios, with inner drive to get to the bottom line, enthusiastic.
    • Strong work ethic and time management skills.
    • Open and a great team player.

    These will do for a start. At the same time, within the same company various departments may require from their DevOps engineers to have a command of different technologies. For example, another department of ours expects from their candidates to know TCPIP, DNS, Nginx, Tomcat, Git, Bash, Groovy, Ansible, Docker, Ubuntu, Debian, Macos, Gradle, Maven, CI/CD.

    As you can see, an ideal fit for the position of a DevOps engineer within a specific department is something rare. All because DevOps engineers have to have so much on their plate, both from technical and soft skills perspective. Why? First of all, as a system administrator to a certain degree, a DevOps engineer has to operate numerous technologies, that sometimes far outweigh the number of those a regular developer uses on a daily basis. In this connection, I once asked a friend of mine working in a DevOps team, “Of all the difficult things that you have to deal with as a DevOps guy, which one is the most difficult one?” And his response was “Everything.” 

    The “everything” of my friend needed further explanations, for which I headed to another DevOps engineer of Qulix Systems, Denis Golovnev. He told me he opted for a DevOps position because he wanted to get to know the whole SDLC closer and in depth, as “this position requires a 360-degree knowledge of the SDLC processes.” DevOps guys are among those few specialists who stay on the project from start to finish, to maintenance. That is why “a DevOps engineer won’t only have to dig deeper into his/her own area of knowledge, but also target adjacent spheres to see how the product is created and how his/her actions can create more value for the client,” explained Golovnev. Thus, this made the tech part clear to me.

    What about soft skills?

    “The most difficult thing about DevOps is to arrange the processes in a manner that will suit everyone; the one that will encourage the initiative and individual decision-making process, to a reasonable extent,” said Golovnev. “To achieve that, highly developed soft skills are a must to get to the root of the problem and enable an engineer to outline clearly his/her point of view to everyone.”

    On the other hand, DevOps team will inevitably face both technical and organizational issues, which should not be confused.

    “One should not try to settle human-related issues through technical means,” said Golovnev. This is another reason why a candidate without highly developed soft skills is very unlikely to get the position. Soft skills are the absolute killer thing for a DevOps candidate.

    To Wrap It Up

    We’ve been gradually migrating from slow and troublesome product releases and delivery of separate work chunks to a speedy, orderly and consistent performance. Most definitely, this transition isn’t the consequence of DevOps practices implementation alone. But without it, without the culture of speedy collaborative delivery, dynamic issues resolution and ultimate automation, which are typical of the DevOps philosophy, the shift would be impossible.

    Furthermore, the revolution starts from minds, whereas the means comes later. If a typical DevOps engineer is, as according to Golovnev, “the one in love with his/her job, insane as to automate everything within his/her reach, get to the core of any constrain that impedes fast product delivery or reduces effectiveness,” no wonder we, as consumers, and the whole industry need them. Together with the outstanding tools and technologies, a DevOps engineer is a soldier that, in 100% of cases, shoots to kill (any production constraint).

    One issue, however, remains unclear. With all the variances as to what true DevOps means even within the same company, how will we reach a unified DevOps definition? Most definitely, some unanimity would be useful for the candidates that try to land the position but end up rejected due to the above-mentioned differences in DevOps visions among the companies.

    On the other hand, lack of unambiguity is justified due to a wide range of technologies as well as different approaches used by different companies. So far, skillset variance will stay a key problem in regards to DevOps hiring. Although our main question–can we do without DevOps?–is out of the question.

    — Veronica Chizh

  • DevOps Tools for the Monitoring Ecosystem

    DevOps Tools for the Monitoring Ecosystem

    DevOps is a cultural and process transformation—the goal of it is to achieve a higher rate of change and quality of software. There are categories of tools to help facilitate this transformation through automation and increased measurement and visibility. You can’t have or observe progress without measurement, and throughout your transformation you need to maintain a high degree of visibility in order to move quickly, safely. Not only is the idea of cultural and process transformation daunting for many organizations, adopting the tools to help you on that journey is even more so, as it requires a great deal of learning. 

    An integrated set of DevOps tools for monitoring has the power to improve visibility and productivity, achieve higher-performing systems and establish cross-functional collaboration. The right toolset is more than the tools themselves—it’s about developing the culture, discipline and practices that come to define your product/service and your workplace.

    This article will outline some of the best DevOps tools and practices when it comes to the monitoring ecosystem that help developer and operations teams work together effectively.

    Monitoring Tools

    A good monitoring platform lets you monitor infrastructure and application performance, whether on-prem, in the cloud or across containerized environments—so you have complete visibility into every system, all the time. Whether you want to monitor Kubernetes, IoT devices or bare metal, the right monitoring tool helps make it possible.

    An effective monitoring tool improves system performance and productivity, and helps you reduce downtime. You can adequately plan for upgrades and new projects, and better allocate your time and resources. You can detect problems—and solve them—before they impact users.

    There are a ton of great monitoring tools out there; I won’t go into them all here, but I’ll highlight the top three in terms of popularity.

    Love it or hate it, Nagios is still very much a widely used tool and formed the foundation for monitoring as we know it today. Plus, the Nagios service check is actually awesome (and underappreciated). Where Nagios falls short, however, is when it comes to monitoring ephemeral environments, such as Kubernetes and Docker, at scale. In reality, Nagios falls short when it comes to monitoring at scale in any environment, whether that’s virtualized or not. Enter Prometheus, an open source monitoring tool (and, like Kubernetes, a part of the CNCF) that uses a pull-based architecture. Instead of pushing metrics to the monitoring tool, it pulls metrics from services. Due to Kubernetes’ built-in exporters, Prometheus is often the go-to for monitoring Kubernetes, but it falls short when it comes to monitoring older infrastructure (see more pros and cons of Prometheus here).

    That brings me to Sensu, which offers the best of both worlds: you can reuse Nagios service checks in Sensu and monitor ephemeral environments at scale. If you have a strictly container-based environment, then Prometheus is a good choice; if you have a mix of multi-generational infrastructure (which we’re finding more often than not with our customers), then Sensu is the way to go.

    Configuration Management Tools

    Configuration management tools allow you to automate the provisioning and deployment of systems, enforce desired configurations and remediate configuration drift. By modeling your infrastructure as code, you can apply software delivery practices such as version control, automated testing and continuous delivery to infrastructure and applications.

    Automating work that used to be manual, repetitive and error-prone results in greater speed, predictability and scalability—and the assurance of standardized configurations across test, developer and production environments. Eliminating snowflake servers reduces time (and headaches) and lets you deploy software faster and more reliably.

    Popular configuration management tools include: Ansible, which is written in Python, agent-less and utilizes an imperative (as opposed to a declarative) approach. Puppet is another popular choice—it relies on a declarative config management approach and uses domain-specific language and an agent/master architecture. Finally, there’s Chef, which is written in Ruby and Erlang, and is modeled similar to Puppet.

    Alerting Tools

    Alerting tools provide both actionable and informational system alerts, and can be customized to fit the complexities of your systems. For example, your alerting system needs to be sensitive enough to cover an outage—but not so sensitive that you’re catching frequent, intermittent problems that users aren’t going to see and that would inundate you with needless alerts.

    Alerting tools help lay the foundation for your alerting policies, so you can determine who to notify, how to track issues and outcomes and how to prioritize remediation. Popular tools include PagerDuty, which offers an on-call management platform with add-ons for analytics, event intelligence and automated incident response. There’s also ServiceNow, which utilizes automated workflows for ITSM as well as customer service and business processes. You can send your alerts into Slack to consolidate alerts into the same platform you use for collaborating with your team.

    Metrics Storage

    Once you’ve automated configuration management, alerting and monitoring, you’ll have a whole lot of data at your disposal to learn from. The challenge: How do you securely store and analyze it? You need a storage system that lets you aggregate and learn from system capacity, user behavior, service levels, security risks and more.

    The insights you gain from your metrics inform decisions across all layers of your business, improving your ability to meet SLAs, satisfy customer expectations and make the case for new strategic investments. Data-driven decisions promote a culture of continuous learning and improvement.

    Popular tools for storing metrics includes InfluxDB and TimescaleDB, time-series databases (TSDBs) that are well suited for long-term storage, and Splunk, which uses a search engine database model to store and query your data. Amazon Web Services (AWS) supports a wide range of storage purposes, including relational and non-relational databases, a data warehouse for analytics, a TSDB, a leger database to store transactions and more.

    Visualization Tools

    A visualization tool might be considered the pièce de résistance of your DevOps toolchain for monitoring: You get to combine all of your data, sort and visualize it, and display it on customizable dashboards.

    Visualization tools provide context and meaning, allow you to track changes and improvements over time and give management a real-time view that helps guide strategic decisions. Customization options make it easy for team members to design and share their own dashboards.

    Grafana is a popular open source visualization tool and can be used on top of a variety of different data stores, including Graphite, InfluxDB and Elasticsearch. For more on how Grafana can integrate with your monitoring and TSDB, check out this use case.

    Evaluating Your DevOps Tools

    No matter where you are in your DevOps journey, it’s wise to re-evaluate the tools you’re using and identify where you can fine tune. Think about the DevOps tools in the monitoring ecosystem as more than their capabilities. How you use them begins to define your habits, values and work culture—and accordingly, the quality of your product or service and the value you bring to your users.

    — Sean Porter

  • The Next Frontier for Enterprises: Moving to a Product IT Operating Model

    The Next Frontier for Enterprises: Moving to a Product IT Operating Model

    In the world of technology, there are few things as dramatic and impactful as enterprise IT’s recent ascent from an enabler to a strategic business driver. Tasked with powering next-gen digital business models, IT teams today are fast realizing the harsh truth that they must make a pervasive strategic shift from traditional project-based operating model to a product-centric one. 

    Traditional project-based operating models focus resources on a specific application or project, leading to fragmented experiences and a disconnect with the rest of the IT organization or company. For instance, in many cases, one team may be working on a back-end system, while entirely different teams work on the front-end application or updating databases. A product-based operating model takes a more holistic approach to ensure projects are integrated and teams work concurrently toward a uniform goal. 

    The days of plan/build/run are numbered as digitization forces IT leaders to act fast, think outside the norm and embrace the blurring boundaries between the digital and physical worlds. A product-centric operating model, sometimes also referred to as a DevOps operating model, accelerates organization-wide digital transformation and consolidates a business’ competitive standing. Companies must transition to a product-centric model to scale-up their agile programs to support end-to-end innovation and reduce siloed operations and projects. 

    The companies that have embraced a product-centric operating model are quickly realizing the benefits, but for many IT departments, they have yet to experience or fully understand why the shift is imperative.

    What’s Driving the Shift to Product IT Operating Model?

    As digitization takes hold and becomes more and more pervasive, CIOs are realizing that intertwining IT and business objectives is quintessential to respond to market changes quickly and remain competitive. Since 2018, volatility has returned to global equity markets in a big way. With the economy in a constant state of flux, coupled with rapid globalization and unprecedented disruption from technology, the average lifespan of a company is 20 years, compared to 60 years in the 1950s, according to Credit Suisse. Overall, the changing business environment makes few things as abundantly clear as this–agility, an appetite for new markets, new products, bias for action and a fail-fast approach will be the new mandates for IT leaders going forward. 

    To this end, most CIOs are replacing portfolios of discrete, siloed projects with product lines that enable a specific business capability. The key factors driving the shift toward a product IT operating model include:

    • Need for Holistic Transformation: Siloed functions have no place in the digital business world that thrives on integration of data and information across structure, culture, processes and technologies. To enable holistic transformation, organizations must draw a balance and establish cross-collaboration between organizational guidelines versus autonomy of teams for process, architecture, technology, tools, etc.
    • Lack of Alignment in Traditional IT Operating Models: Lack of alignment across large teams and multiple organizational groups (such as IT, finance and business) prevents organizations from executing an end-to-end digital strategy. Shifting to a product IT operating model can help steer organizational units toward a common goal. It can facilitate greater collaboration between different DevOps teams that support different products and domains. Recognizing the failure of existing IT governance and the pressing need for a shared digital transformation vision, IDC predicts 40% of CIOs will adopt new digital governance models.
    • Need for Speed: Most companies cited that their digitization efforts were hindered by inflexible or slow processes with a traditional IT operating model. Adopting a product-oriented management approach for IT can help organizations define and agree on a pace of change congruent with their readiness and appetite. According to IDC, 65% of CIOs will expand Agile/DevOps practices into the wider business to achieve the velocity necessary for innovation, execution and change by 2021.
    • Lack of Integration Between Old and New Technologies: Lack of integration between new and existing technologies is a key factor that keeps organizations from realizing the full potential of digitization. Modern companies need to balance innovation, experimentation, adoption of new technologies and tools with the enterprise’s need for business results, cooperation, risk management and compliance. Because a product-centric model takes a holistic, end-to-end approach, it is ideal for integrating technologies, processes and teams at scale.

    Embracing a Next-Gen Operating Model

    To compete with newer, nimbler companies such as Facebook, Uber, Spotify and others, traditional companies that are serious about continued growth are leaving no stone unturned and embracing a product IT operating model. From being providers of reliable IT services, CIOs have emerged as strategic partners in driving business transformation. Modern day CIOs are driving business innovation and identifying opportunities for competitive differentiation, not delivering ad-hoc, siloed services. As cloud-first and AI-first become the new mandates for organizations, re-engineering the IT operating model is quickly becoming central to creating value, competitive advantage and growth.

    — Rohit Nand

  • Predictions 2020: The Future of APM Strategies and Tools

    Predictions 2020: The Future of APM Strategies and Tools

    As 2019 comes to an end, it’s safe to say that multi-cloud strategies are here to stay. We can predict that 2020 will bring an ever-increasing demand for application performance management (APM) tools and services.

    So what does the future of APM look like? Let’s dive in.

    Multi-Cloud Strategies Increase the Demand for APM Tool Adoption

    Companies are increasingly adopting more than one platform—either for financial leverage or to create a time-to-market or feature race between the platforms. To remain competitive, public cloud providers must offer unique features or capabilities differentiating them from competitors. This has created an upsurge in new and more complex technologies, increasing the need for APM tool adoption.

    Logs Will Be Recognized as Key Aspects of APM

    APM is a puzzle made up of user experience, metrics and traces, and logs—and each piece must be incorporated to achieve full-stack visibility and ultimately seamless app performance. However, a healthy app is not characterized solely by top performance, but also availability—making a log management solution that monitors availability, enables proactivity and reacts quickly to remediate problems key.

    In 2020, implementing solutions that achieve integrated application performance and availability management for any type of organization, including those operating on-premises and/or cloud-based applications across all mixes of cloud environments, will become increasingly important and prioritized.

    APM Skills in a Cloud-First World

    The nouns have changed, but the verbs remain the same: APM skills are among the top skills tech pros want to develop in the coming year, according to the recent SolarWinds IT Pro Day survey, Building Confidence for Tech Pros of Tomorrow. However, in 2020 tech pros will begin to recognize that many of the skills they already possess are sufficient to manage the new realm of integrated solutions in the cloud. Tech pros can leverage some of the same skills they could before as the nouns have changed, but the verbs remain the same. Where we used to have server or virtual machine, now we have containers. But the verbs? They’ve always been focused on optimizing performance, ensuring availability and planning capacity.

    In 2020, the extent to which tech pros will discover they don’t need to fear new nouns (more than they did when they moved from client virtualization) will increase.

    Universal Adoption of APM Tools

    As environments become more dynamic and complex, the opportunity for applications to break increases—creating the need for robust monitoring tools. In 2020, as buyers see the benefit of leveraging APM tools on a regular basis rather than only in critical moments, the market will adjust accordingly. We can expect easier-to-implement solutions that are more highly valuable and affordable. The paradigm that APM tools have to be expensive will be broken once and for all.

    Want to learn more about what to expect in 2020? Join us Jan. 23 for our Predict 2020 Virtual Summit  featuring discussions from some of the industry’s best and brightest offering up their visions for the future. Sign up today for this free daylong virtual event.

    — David Wagner

  • The Role of DevOps in Custom Software Development

    The Role of DevOps in Custom Software Development

    Hundreds of tech giants and big retail companies such as Etsy, Netflix, Amazon and Target have adopted an ecosystem called DevOps. It establishes a good collaboration between cross-functional teams such as development and operations which simplifies the processes. DevOps has calibrated a new standard for building custom applications.

    Do you know any predefined technique to check whether developers have adhered to all coding principles? A quality check is not the right answer. What if you want to check and optimize their entire system of writing code, understanding the project and following coding guidelines?

    You must have dug into the details of not being able to accomplish the projects before the deadline. Well, there is not a single reason for this failure. There is a need for restructuring the entire ecosystem of project development, which will include polishing of coding rules, eliminating duplicates of code, unit tests, bug removal and understanding and improving the project architecture. DevOps aims for such a practice that includes neat coding, bug-free testing and the fastest deliverables.

    This blog explains everything about how DevOps plays a significant role in custom software development and makes a super-productive project development environment.

    The term “DevOps” is the combination of two different terms: development and operations. Thanks to Patrick Debois and Andrew Clay Shafer for originating the idea of DevOps in 2008. They have altered the traditional techniques of software development. Gone are the days when the software was written once and used for years. For the era of rapid software development and extreme personalization, you need smart software that updates in seconds.

    Custom Software and DevOps

    Now is the time where generic needs seem limited and are absolute off-the-shelf in the software industry.

    DevOps is the process of creating custom software that fulfills everyone’s demands in no time. It chooses to streamline the project development phases by automating the process, making faster deliveries and upscaling the software quality in the world of high-end customizations.

    How Does Automation of Software Development Plays a Predominant Role?

    Software development life cycle comprises of various components such as time, manpower, expenses, knowledge resources and problem-solving capabilities. It continuously evolves around bringing a unique solution that stays reliable over time.

    The optimization of the expenses, time and manual efforts is possible using the automation of SDLC. Especially when customizing the features, manual efforts make the tasks tedious and slow. Automation overcomes these drawbacks and makes customizations super easy to fulfill.

    Automation testing has carried out the responsibility of making project development glitch-free, as it detects the errors at a much faster pace. For instance, a custom software/application, such as the social media platform Instagram, carries a lot of features and functionalities to add. To make this a glitch-free development, automated test runs stand as the rock-solid solution.

    DevOps not only improves the programming practice of the coders but also scrutinizes the testing. It streamlines communication among the coders, front-end engineers, clients and project managers. Thanks to its immediate nature of validation and rectification.

    The automated workflow of SDLC saves a lot of business expenses as it requires less time for rebuilding the applications. Also, it optimizes the time required for running the manual tests. It brings in the high quality of work that decreases the glitch fixing need and hence, reducing the overall project costs of custom software.

     How Is Continuous Delivery or Faster Delivery Important?

    Continuous delivery means delivering the small chunks of code a developer has built, in short cycles. It lowers the project development costs significantly of custom software as the hidden fixed costs with the primary release of projects will not exist.

    Smaller release, smaller risks. Yes, there will be nothing like once the project delivery has been done, you can not opt for a small change request as a part of the suggestion. Because the deliveries won’t stop when you opt for custom software development. This sort of development is required to adopt changes very fast and hence, needs deliverables to be delivered at a faster pace for the rapid feedback implementation and live changes.

    So, hypothetically, you will not need to be afraid of how it will turn out. For instance:

    1. Amazon deploys code at every 11.7 seconds.
    2. Popular social media platform Instagram follows the same practice 30-50 times a day.
    3. Netflix pushes its code thousands of times a day.

    What led them to this?

    • To avoid bad commits.
    • Freedom to deploy the code whenever possible.
    • Avoiding the mess for going through the thousands of deploys for one error.
    • To speed up the work and make changes in seconds.
    • To respond to users unpredictable queries and turn it into a feature.

    This can be easily achieved through DevOps in custom software.

    There are many reasons for more occurrences of deployments. All the reasons lead one focus (i.e., faster delivery for custom software and small but speedy updates for the custom software that is used by millions of people online).

    Faster bug fixes and updates are the new demand for sure. All thanks to DevOps methodology. The most valued part is, developers and delivery managers will come to know users’ preferences at the very early stage of development. As the project goes on, there will be zero chances of bug generation and a smooth operation will be established. 

    Thus, the successful glitch-free customized software development is possible with a DevOps strategy. The teams will get to learn from peers and end-users about the project at an earlier stage using DevOps.

    Responsiveness Is the Best Software Quality 

    The main advantage of the DevOps technique is its quick responsiveness to clients’ feedback and rapid execution.

    If you can quickly look into the client’s feedback in the small chunk of code you’ve submitted, you can respond to them faster. Thus, can lead the fastest execution possible.

    The integration of a quality assurance team will add a lot of potential in improving product quality.

    Responsiveness plays a crucial role in decreasing the frequency of a module collapsing (i.e., maintains the module dependency because of the continuous delivery and quick implementation of feedback).

    Purpose and Role of DevOps in Custom Software Development

    You need a super key to stay ahead of competitions and it always comes by streamlining the process. Be it a business revenue, cross-selling of products, increasing the target market and anything that comes up with competitors.

    DevOps is that key here. Yes, you read it right. Bring in this methodology at your information technology firm to witness the enormous growth.

    Custom software development has become pivotal in the IT industry. To create the best-customized experience, consistency is required. That is the reason why DevOps is gaining popularity so fast. DevOps is the master key to connecting resources with the same thread. 

    You might have wondered how Netflix updates and personalizes the experience for everyone so rapidly. Netflix does thousands of deployments per day. That’s the reason it can give its users a top-quality personalized experience. 

    “The rate at which this entertainment game-changer has adopted new technologies and implemented them into its DevOps approach is setting new standards in IT,” said Coman Hamilton, editor at JAXenter.

    Netflix had started nailing this modern approach from 2015 itself. Now you can understand how technology has come so far approaching 2020. So, what next doable thing is up for today’s need for deployments using DevOps?

    DevOps is an exemplary technique that enables IT to showcase its power. It empowers the dynamic nature of custom software and stabilizes the work environment by keeping everyone on the same page at all instances of time.

    For a company to be responsive and relaxing throughout the year without overloading the work, adopting DevOps is a must. The purpose of enabling development and IT operations together serves as time-saving and an in-budget procedure of building applications with the utmost quality.

    Where to Start?

    DevOps is not a new software to learn or a new framework to set in. It is the way of working and adopting it—everyone must be involved. Educating resources is important when you are planning to adopt a big change. Conduct a conference, state the pain areas and showcase the possibilities with DevOps. You can start with teams that have great adoption skills, the ones who can take challenges and are ready to learn.

    Essentials to Consider for the Best Practices of DevOps Implementation

    Breaking the Silos

    If you want to make DevOps an organizational practice, you will have to first break down the functional silos. As DevOps will make everyone come closer, the communication gap among the teams will not be questioned. In an information technology organization, the more communication engineers keep, the more successful they will be in project development.

    Make this a daily practice, “exchange of ideas is an exchange of solutions.”

    Using the Right Tools

    There are several tools to help your team automate the process and track down the work done. You will need a tool such as Jira when entering a DevOps environment. Also, choosing the right tools for the right needs is necessary. For example, Jira is a very good tool for tracking the issues and for managing the projects, while Querysurge is used to perform continuous testing and data analysis purposes.

    As per the project and company needs, one can go for the right set of tools to use. Make your organization resourceful with the automation and project tracking tools that are already overcoming manual efforts.

    User-Centric

    Adopting DevOps is a cultural change and it listens to its end-users. The end-user of any particular industry will want the best features and robust functionality. Firms will have to step in with the altering demands and stay committed to user satisfaction. Start making customer-centric applications.

    Automation at Its Best

    As we know, DevOps detects diseases at an earlier stage. It has the lowest percentage of risks involved as it uses automated testing of a small snippet of code. Keep it in a practice to code and test. It will automatically follow the test-driven development practice used in agile teams. Use the most of automation practices that eliminate the slow process of software development. This way the teams get faster development completions without any hampering of deadlines.

    Conclusion

    You may end up thinking DevOps works for IT firms and is not going to help non-IT industries or businesses. It’s not like that.

    Think about the software you have used for years in your non-IT firm. Are you using a software that drives your business but it’s not up to date? Think about customizing it to improve the way it works. These customizations come from experienced custom software development companies. Prominent companies are nailing DevOps nowadays.

    — Hardik Shah

  • Predictions 2020: Ushering in 2020 Data Predictions

    Predictions 2020: Ushering in 2020 Data Predictions

    2019 brought us more data organizations running more advanced analytics, AI and ML workloads than ever before. 2020 is the year where we’ll see a spike in both the number of technologies and data teams that support these types of workloads internally. We’ll see AI and analytics teams merge into one as the new foundation of the data organization, focused on areas such as moving to the cloud while maintaining on-prem Hadoop, “Kubernetifying” the analytics stack and Hadoop compute. These are the trends I believe we’ll see come about in 2020.

    Rise of the Hybrid Cloud

    We’ve been hearing people talk about the hybrid cloud for the past three years now. For the most part, that’s all it’s been talk—but in 2020 it gets real. We are seeing large enterprises refusing to add capacity on-prem to their Hadoop deployments and instead invest in the public cloud. But they are still not willing to move their core enterprise data to the cloud. Data will stay on-prem and compute will be burst to the cloud, particularly for peak demands and unpredictable workloads. Technologies that provide optimal approaches to achieve this will drive the rise of the hybrid cloud. 

    One Machine Learning Framework to Rule Them All

    Machine learning with models has reached a turning point, with companies of all sizes and at all stages moving towards operationalizing their model training efforts. While there are several popular frameworks for model training, a leading technology hasn’t yet emerged. Just like Apache Spark is considered a leader for data transformation jobs and Presto is emerging as the leading tech for interactive querying, 2020 will be the year we’ll see a front-runner dominate the broader model training space with pyTorch or Tensorflow as leading contenders. 

    Kubernetifying the Analytics Stack

    While containers and Kubernetes works exceptionally well for stateless applications such as web servers and self-contained databases, we haven’t seen a ton of container usage when it comes to advanced analytics and AI. In 2020, we’ll see a shift to AI and analytic workloads becoming more mainstream in Kubernetes land. Kubernetifying the analytics stack will mean solving for data sharing and elasticity by moving data from remote data silos into K8s clusters for tighter data locality. 

    Hadoop Storage (HDFS) is Dead but Hadoop Compute (Spark) Lives Strong

    There is a lot of talk about Hadoop being dead, but the Hadoop ecosystem has rising stars. Compute frameworks such as Spark and Presto extract more value from data and have been adopted into the broader compute ecosystem. HDFS is dead because of its complexity and cost and because compute fundamentally cannot scale elastically if it stays tied to HDFS. For real-time insights, users need immediate and elastic compute capacity that’s available in the cloud. Data in HDFS will move to the most optimal and cost efficient system, be it cloud storage or on-prem object storage. HDFS will die but Hadoop compute will live on and live strong.

    AI and Analytics Teams Will Merge Into One as the New Foundation of the Data Organization

    Yesterday’s Hadoop platform teams are today’s AI/analytics teams. Over time, a multitude of ways to get insights on data have emerged. AI is the next step to structured data analytics. What used to be statistical models has converged with computer science to become AI and ML. So data, analytics and AI teams need to collaborate to derive value from the same data they all use. This will be done by building the right data stack—storage silos and computes, deployed on-prem, in the cloud or in both, will be the norm. In 2020 we’ll see more organizations building dedicated teams around this data stack.

    Talent Gap Will Inhibit Data Technology Adoption

    Building the stacks that enable data technology into practice is hard, and this will only become more obvious in 2020. As companies discuss the importance of data in their organizations, they’ll need to hire the data, AI and cloud engineers to architect it. But there aren’t enough engineers who have expertise in these technologies to do that. This super-power skill is the ability to understand data, structured and unstructured, and pick the right approach to analyze it. Until the knowledge gap closes, we’ll continue to see a shortage of these types of engineers—many companies will come up short on their promises of “data-everywhere.”

    China Is Moving to the Cloud on a Scale Much Larger than the US and Will Leap Frog From On-Prem to Massive Cloud Deployments for Advanced Workloads

    Over the past five years, while enterprises in the U.S. have been moving in leaps and bounds to public clouds, enterprises in China have been investing mostly in on-prem infrastructure primarily for data-driven platform infrastructure. 2020 will be the inflection point where this changes. China will leapfrog into the cloud at a scale much larger than the U.S. by adopting the public cloud for new use cases, bursting in the cloud for peak loads and over time move existing workloads. Public cloud leaders in China will see dramatic growth that might outpace the growth of the current cloud giants.

    2020 is the year where the rubber meets the road when it comes to advanced analytics and AI. Companies that back the types of technologies that enable and support this kind of data and workloads will emerge as leaders in the space. On the other side, companies that structure their data teams to meet the requirements of the new data stack will emerge as leaders as well. I’m excited to see how advanced analytics and AI opens up new and innovative applications and use cases.

    Want to learn more about what to expect in 2020? Join us Jan. 23 for our Predict 2020 Virtual Summit  featuring discussions from some of the industry’s best and brightest offering up their visions for the future. Sign up today for this free daylong virtual event.

    — Haoyuan Li

  • SMBs Struggle with Digital Transformation

    SMBs Struggle with Digital Transformation

    It’s a cliché, but it’s as true now as ever: The only constant in business is change. Digital transformation efforts are as essential for small and mid-sized businesses as they are for large enterprises—and SMBs are engaged in their transformative efforts for largely the same reasons as large enterprises: to engage more effectively with customers, streamline business processes and improve productivity.

    What is surprising, however, is many of the challenges small and mid-sized businesses face when it comes to their digital transformation efforts are also similar to those of larger companies.

    According to Insight 2020 Technology Report: IT Trends for Midmarket and Small Business, a survey-based report from IT services provider Insight, even as 95% of IT professionals who work at SMBs have implemented, or plan to implement within the next year, digital transformation initiatives, 49% said integrating new technology with legacy systems is very challenging to extremely challenging.

    Understandably, when survey respondents cited their top challenges when deploying the best available technology 52% said information security concerns, while 46% pointed to their ability to find the tools that best fit with their existing technology. Surprisingly, all three of those concerns surpassed budget constraints in importance.

    Respondents also cited cloud computing as the most important technology when it comes to the growth of their organization. Next up were data analytics, artificial intelligence and IoT.

    Fifty-six percent of small and mid-sized businesses said comparing cloud costs to current costs is their biggest barrier to cloud migration. That was followed by security requirements (50%). When it comes specifically to cloud, 65% of respondents said security and compliance are their primary challenges.

    The survey also highlighted that small and mid-sized businesses may need to focus on improving how they engage with IT services providers. The survey found 41% of respondents said they use from six to 10 IT vendors, with another 24% saying they use more than 10—not all is rosy in these relationships. Many respondents cited hurdles when it came to working with their service providers, and they rated the following as “very” to “extremely” challenging when working with their vendors:

    • Customizing solutions to business needs and objectives (44%).
    • Coordinating internal and external resources on complex IT projects (43%).
    • Vendors cater more to larger enterprises (43%).
    • Asset managing and juggling licenses from multiple providers (40%).
    • Vendors do not understand small business needs (40%).

    The survey did find that 64% of respondents believe their IT program is very flexible at adapting to organizational changes and growth, while 57% of respondents believe their IT program is well-equipped to implement new technologies.

    These challenges echo the challenges faced by large enterprises, such as those we covered in our post, Legacy Systems Prove to be Digital Transformation Headwind. And, of course, large enterprises have challenges with budget, security and talent constraints.

    The research was conducted online by M/A/R/C Research among 408 IT decision-makers at small and mid-sized organizations with 10 to 1,000 employees in the U.S. and Canada. The Data was collect from Aug. 29 through Sept. 6, 2019.

    — George V. Hulme

  • The Automated Data Center Emerges

    The Automated Data Center Emerges

    Hyperscale and automation seem to go hand-in-hand, at least when it comes to the data center. Synergy Research reported that, at the end of 2017, there were 390 web-scale data centers worldwide, up from 300 a year earlier. IDC identified that hyperscale data centers “have a minimum of 5,000 servers and are at least 10,000 sq ft in size but generally much larger.”

    Today, hyperscale data centers are driving a sense of urgency for automation, but many smaller corporate data centers are just as complex, even if they are less physically expansive. In either case, automation takes data from across the data center, applies analytics and machine learning to that data and helps to identify both inefficiencies and trouble spots in the data center–as well as their origins. 

    Some data center functions are well on their way to becoming automated, making new deployments and changes to existing deployments simpler, faster and more efficient than they are today. Among those functions, orchestration, change management, software-defined architectures and DevOps stand out. 

    Meanwhile, another data center function is lagging: problem resolution. The challenge revolves around the siloed nature of the data center, where the multiple, disparate tools and methods used to sleuth out inefficiencies and pinpoint trouble spots lack a unified view of the data center infrastructure. In addition, silo-centric infrastructure monitoring tools lack application context. Short of automation, today’s data centers are stuck in the past. Especially as a data center grows and must interact with cloud-based services, no amount of human intuition in using those tools and methods can overcome the barriers of siloed systems. 

    However, data center infrastructure planners are making strides in identifying the requirements for accelerating the move to the automated data center. 

    Three Keys to the Automated Data Center

    For years, the automated data center was just a dream. And as recently as late 2018, Forrester spoke of islands of automation.

    “For the enterprises I’ve talked to, it’s not like any of them are completely manual at this stage, but they’re all at varying degrees of trying to integrate all these pieces,” said Chris Gardner, Forrester senior analyst. He added that fragmentation was due to a lack of holistic solutions that could connect all aspects of the data center.

    But now, three forces are shaping up to help make the automated data center a reality. Let’s look further. 

    Real-Time Visibility 

    Real-time infrastructure visibility has become a critical element of automating the data center. Why? Because an issue can occur anywhere on the virtualized, hybrid infrastructure, and the issue can ripple to create problems elsewhere, making issue resolution a perennial challenge. 

    For that reason, your best defense is to build deep, real-time visibility into the compute/virtualization stack and the storage/storage network stack. This will help you quickly ensure the performance of mission-critical applications accessing the data from those stacks.

    For ITOps, it is about proactively pinpointing the source of the issue and resolving it before application users are impacted. That requires stitching data across the full stack of technologies supporting the business: application services and infrastructure services, both on-premises and in the cloud. 

    Real-Time Analytics

    Why is real-time analytics crucial in the data center? In short, it’s because real-time responsiveness is critical for supporting anomaly detection, root cause analysis, remediation, prevention and planning use cases, where processing and correlating massive amounts of information is critical.

    For ITOps, it means distributed real-time architectures must combine cloud-based and on-premises analytics that leverage AI and machine learning. It’s the analytics and correlation that helps you make sense of all that real-time data so it becomes actionable.

    Closed-Loop Systems

    IT organizations are acutely aware of the perils of introducing changes to critical production environments. In fact, they take great pains to establish a governance of change. Closing the loop is about humans delegating decision-making to technology.

    For ITOps, that means ensuring integrated control of change processes across the entire technology stack. Data and analytics should lead to optimal real-time decision-making that empowers the automation for closed-loop operation.

    How Automated Is Automated?

    In 2016, researchers at the University of Pisa argued that hyperscale data centers would have to rely completely on analytical, automated systems to have to any chance to manage the huge number of day-to-day issues their servers generate. They added that “large computer systems in general, and data centers in particular, will ultimately be managed using predictive computational and executable models obtained through data-science tools, and at that point, the intervention of humans will be limited to setting high-level goals and policies rather than performing low-level operations.”

    Now, more than three years later, critical technologies and methodologies are in place to make the automated data center a reality, even if humans are still required for guidance, if not for sleuthing out infrastructure issues.

    — Len Rosenthal