devopsy.kinsta.cloud

Tag: on-premises

  • Building a Services and IoT Platform with DevOps, Part 4

    Building a Services and IoT Platform with DevOps, Part 4

    IoT platforms and their solution development have crossed different maturity levels. At first, devices were primitive, isolated and working with limited functions before slowly evolving to communicate and connect with other devices using lightweight communication protocols, and then stepping into advanced level with remote management of devices, harvesting the data from devices and smart decisioning and analytics. Beyond that, IoT has moved to optimized levels of using multiple technology options and adopting cloud and DevOps disciplines to build intelligent IoT solutions.

    This article focuses on giving insights on choice for IoT platforms and factors influencing build or buy options for IoT solutions, different hosting options, factors on IoT development in the cloud and more.

    IoT Platforms: Build, Buy or Both?

    The dilemma of whether to build an IoT system from scratch or buy an off-the-shelf platform still continues. Below are some of the key factors to consider before making the critical decision.

    1. Price: If you plan to build an IoT platform, the infrastructure setup and operation costs can be high. And, as you plan to scale, there will be additional infrastructure cost in setting up additional data centers in that region. If you buy an IoT platform, prebuilt infrastructure is available across global data centers to scale out on a per-usage basis. Plus, you get the the benefit of buying devices from a certified partner ecosystem with discounted device prices. You can buy a few devices from cloud vendors or other device manufacturers, and use mix of devices, mix of cloud/multi-clouds or on-premises, using mix of platform as a service (PaaS)/infrastructure as a service (IaaS) at competitive prices.
    2. Time to Market: A house-built IoT systems will typically take about six to 12 months to complete, including infrastructure setup, device development and software development and testing. Depending on the complexity of the business case, the release cycle may take even more time to get to production, especially with ongoing device technology updates, lack of protocol standards and new business demands.
      Under the buy option, you can buy preconfigured end-to-end solutions available for common IoT use cases such as remote monitoring, predictive maintenance and connected factory, and these solutions can be customized based on need. You can even try to build only few components of IoT solution which may not be available from the vendor, such as the devices/edge, build a field gateway and use preconfigured solutions for data ingestion and data processing, data analytics and data storage components. There are starter kits available to quickly prototype, evaluate and deploy, eliminating spending more cost and effort, before hitting to the ground.
    3. Skillset/Expertise: Building an IoT platform requires a skilled team with expert knowledge of software and hardware and development, which costs more. Currently there is lack of skilled IoT experts with a mix of software and hardware knowledge in traditional industrial-based enterprises.
      When buying a IoT solution from vendors, the solutions are prebuilt and guided by standards and best practices. The platforms are built by experts who are proven with cloud infrastructure, based on their experience working with large customer base and customer support—necessary for IoT platform development.
    4. Scalability: When you plan to scale your business, you need to scale your infrastructure as well. With millions of devices connecting to your infrastructure to send and receive data and firmware updates, monitor the devices, support multiple range of network providers and store large volumes of data being collected and processed, you need a scalable infrastructure on-premises or in the cloud to meet those demands. Setting up a scalable infrastructure on your own requires skilled labor, effort and cost.
      Real-time streaming and data processing capability is available for large volumes of data workload, enabling scaling of individual components of IoT solution, as well scaling as scale units (group of components for data ingestion, data processing and data storage). Choices of different cloud vendors give you options to scale based on demand, horizontally or vertically, adding/removing devices with portability options and providing on-demand simulated test environments with the flexibility to choose multiple clouds or a mix of PaaS/IaaS options from different clouds or hybrid infrastructure.
    5. Security: For in-house IoT development, teams should be aware of various security attacks and develop systems with current security strategies and technologies to both protect data and devices and maintain data privacy. A large infrastructure is needed to support the sandboxed hosting of data in separate geographic regions for secure data storage, device security and user security.
      IoT platform vendors provide security standards. For example, Microsoft provides SDL (Security Development Lifecycle), which enables developers to follow the standards during device development and ensures development happens with security built at all parts of system. These platform providers have prebuilt infrastructure for secure data storage, cloud security, user security, application security and device security, and has separate security teams to protect the data at all levels.
    6. Analysis & Visualization: It’s estimated that 90 percent of the large volumes of data collected from millions of devices remains unused in most enterprises. To tap the data for valuable business metrics, data analysis/algorithms should be updated as industry and device requirements change and extensions are needed to integrate data with data analysis tools to churn the data. There are open-source and proprietary tools available for data mining and data analytics and custom dashboards must be built to monitor the telemetry, setup rules and actions.
      Preconfigured solutions and most IoT platform providers have out-of-box data analysis tools integrated and rules can be defined to analyze data with customizable dashboards, enabling you to focus more on business outcomes and how to churn/mine data, rather than focusing on building algorithms/tools/dashboards from the scratch.

    Factors Driving Cloud-based IoT Development

    Considering the various factors, a hybrid option with mix of buy and build for IoT platform components is better compared to building the platform from scratch, as you get out-of-the-box features from the off-the-shelf platforms that are customizable and extensible to business needs.

    The following are some of the factors driving more and more IoT development in cloud (single/multi-cloud/hybrid), making IoT grow exponentially:

    1. Increasing number of devices – billions/trillions of devices interconnected
    2. Need for remote processing power and high compute
    3. Need for hyperscale systems for processing hot/cold/warm/fast data pipeline with low latency
    4. Need a platform that can quickly evaluate, prototype and deploy IoT applications
    5. Need a platform that provides security by design across devices, gateways, networks, data processing, users and applications
    6. Usage of advanced analytics and monitoring solutions available in cloud
    7. Leverage interdevice communications, interservice communications
    8. Utilize preconfigured IoT solutions for most of the popular use cases such as remote monitoring, predictive maintenance, with proven practices/patterns.
    9. High data storage, highly scalable with minimum cost to deploy
    10. QoS features in cloud—security, performance, usability, backup, etc.
    11. Cloud gateway providers have support for various devices and provides extensibility
    12. Cost of devices, cost to deploy and internet usage has dropped
    13. Out-of-box support and availability of big data and machine learning techniques in the cloud with mobile data visualization
    14. Ease of integration with DevOps tools in cloud, end-to-end testing with simulated devices as containerized environments in cloud

    Expanded internet connectivity, reduced cost of devices, increased spread adoption of remote devices such as smartphones, usage of big data, machine learning techniques and cloud adoption, competitive pricing models and more are the key drivers for IoT’s growth in cloud.

    About the Author / Lavanya Subbarayalu

    Lavanya Subbarayalu is Senior Architect working with Technology Office in HCL Technologies. She has expertise in IOT, Azure, DevOps consulting & Microsoft technologies. She is associated with DevOps COE, working on design and Development of DevOps solutions and consulting tools. Connect with her on LinkedIn.

  • The Need for Speed Demands New-Era APM

    The Need for Speed Demands New-Era APM

    Applications have become the primary conduit by which customers experience a brand, making the application experience critical to business success. As such, continuous delivery and highly available systems that deliver optimal end user experiences at lightning speed have become top priorities for organizations. Modern IT is all about speed, efficiency and customer experience. Simply put, companies that develop new applications and services the fastest will win.

    That’s where containers and microservices come in. Containerized microservice applications enable organizations of all sizes to approach the ultimate goal of continuous integration and continuous delivery. This accelerates everything within the application life cycle, from initial development to component updates and new functionality.

    To match the development speed that microservices and containers enable, Ops teams need tools that can also operate hyper-fast, providing an immediate understanding of the impact of changes made, real-time analysis of vast amounts of data across applications and technologies, visibility across a range of languages and infrastructures, and knowledge of how the application is performing for end users.

    In fact, we are at a critical inflection point in application performance management (APM), where a generational shift is needed to keep pace with the new hyper-dynamic application architectures consisting of microservices, containers and hybrid cloud environments. To fully understand the impact of this shift, we can examine the genealogy of APM.

    APM Evolution

    The first generation of APM, established at the end of the last century, collected statistics and metrics out of system and application components, providing a first generation of user-defined transaction profiling by using instrumentation technology. APM 1.0 required a lot of effort to implement and maintain, but helped to monitor and analyze the performance of newly developed 3-tier applications.

    The second generation of APM was driven by more distributed, service-oriented architectures (SOA) and the rise of more elastic, virtualized or cloud-based infrastructure. APM 2.0 added application topology discovery and could analyze distributed applications. User experience monitoring of web and mobile apps quickly became part of APM 2.0 as more businesses depended on the service quality of their applications. With a primary focus on application monitoring, not management, APM 2.0 required an even broader set of subject matter experts to configure and interpret monitoring data from dashboards and reports. Similar to APM 1.0, the second generation still required way too much human analysis and setup to fully understand application problems and derive actions.

    The continuous change in today’s modern dynamic applications makes it difficult, if not impossible, for those older tools to keep up. Throw in additional architectural challenges such as hybrid clouds, containers and orchestration (such as Kubernetes and Mesosphere) and the picture gets murkier, all in the name of high-speed delivery. Meanwhile, the microservice architecture and development strategy spawns wide technology diversity—each engineer can choose the programing language, middleware and databases best suited to code their individual microservice. Scale, complexity and constant change is the new reality for today’s dynamic applications.

    Wanted: A New Breed of APM

    In this environment, traditional IT monitoring tools won’t work. These new applications use a vastly larger set of technologies, must scale dynamically and are in a constant state of flux.

    In this effort to engineer for speed and efficiency, complexity and scalability have increased exponentially, requiring continuous monitoring and more visibility than ever before. As organizations embrace agile methodologies for development and containerized microservices for deployment, developers have shifted their thinking in technology choices. Developers now continually select the most optimal technology for their individual goals regardless of DevOps’s ability to configure, deploy and manage the specific technology.

    With several teams across the enterprise constantly publishing new software releases, understanding the impact of these new releases and changes to the overall environment has become a significant challenge. Today there is no single source of truth assisting communication between various organizations, resulting in what seems a wild west–like environment for IT. With so many groups and technologies in play, everybody seems to be speaking a different language, especially when it comes to delivering high-quality applications. In this environment, a new breed of APM  is needed that can provide a “Truth Machine”—complete with deep visibility across all application components in the technology stack.

    This new breed of APM must be automatic and dynamic to deal with the ever-changing world of applications running in containerized microservices. To deliver these critical capabilities, the tool must provide real-time visibility into dynamic, containerized, microservice applications, regardless of language or infrastructure. New thinking is required to meet the challenges of these complex, large and dynamic application environments; a new monitoring and management approach that can handle dynamism by:

    • Delivering real-time analysis of vast amounts of data across all applications and technologies.
    • Conducting continuous discovery and automatically mapping every component, host and container, (hosted on-prem, private cloud, public cloud), to the exact architecture of complete technology stack.
    • Visualizing the application architecture and delivering actionable suggestions that assure the operational health of systems and applications.
    • Providing real-time accuracy to visualize, structure and identify problems immediately.
    • Monitoring the complete set of diverse technologies that run in microservice environments.
    • Operating in a heterogeneous environment and automating deployment across nearly all programming languages.

    Artificial Intelligence for Next-Generation APM

    DevOps and IT organizations are struggling to match the speed of agile development, and this is creating exponentially more challenges around new application architectures. The only constant in these modern application environments is change. Whether change is due to microservice architectures, container orchestration or simply more agile delivery execution to effectively manage microservice applications, DevOps needs up-to-date visibility, a precise picture of the application’s structure and an immediate understanding of its health. To accomplish that, DevOps must be able to apply artificial intelligence to the act of monitoring modern dynamic applications.

    Rather than spending time configuring and reconfiguring monitoring tools, organizations must automate management steps with as little human setup and configuration as possible. To have complete control and visibility of their environments, regardless of language and infrastructure, organizations must use a modern application management solution that is automatic and AI-powered to deal with the ever-changing world of dynamic containerized microservice applications.

    About the Author / Pete Abrams

    Pete Abrams is a co-founder and the chief operating officer at Instana, the leader in application performance management for containerized microservices applications. Formerly vice president of customer centric innovation at AppDynamics, Pete has spent more than two decades in the technology industry focused on introducing new products to market and ensuring customer feedback and desired capabilities are incorporated into future product innovation. Connect with him on LinkedIn.

     

  • IBM: DevOps Challenges Require Managed Services

    IBM: DevOps Challenges Require Managed Services

    The DevOps complexity associated with determining which classes of application workloads are best-suited to run on certain platform types will push an increasing number of IT organizations to embrace managed services. That’s the premise put forth by IBM, which provides such a service.

    Steve Robinson, general manager for IBM Cloud Platform Services, says it’s becoming less likely that internal IT operations teams will have the tools and expertise required to analyze and optimally deploy workloads. In fact, the number of platform choices have already exceeded the ability of most internal IT organizations to master. They include:

    • Traditional virtual machines: The most widely deployed form a virtual machine software comes from VMware. But there’s already a lot of interest starting to emerge in so-called “micro VMs” that promise to reduce the memory footprint of VMs. However, rather than replacing traditional VMs, it’s probable micro VMs would be deployed alongside traditional VMs.
    • Containers: Most containers today are deployed on top of virtual machines because of a lack of tools as well as security concerns. But as IT organizations become more adept at managing containers, many more of those containers will wind up running on bare-metal servers that will be able to attain much higher utilization rates by eliminating the need to deploy traditional virtual machines.
    • Serverless Computing: Based on event-driven architecture, these platforms can be invoked natively as functions or simply be used to host containers. In either case, the need to provision servers, storage and networking resources is eliminated because IT infrastructure resources are made available on demand in real time.
    • Graphical Processing Units (GPUs): As an alternative to Intel Xeon class processors, GPUs now are taking on many workloads involving machine-learning algorithms and big data because of their ability to process many of those workloads in parallel.
    • Field Programmable Gate Arrays (FPGAs): Based on custom integrated circuits, FPGAs are being more widely employed within a variety of cloud services.

    IT organizations today have the option to deploy any of these technologies on-premises or invoke them in the cloud. The issue most are contending with is a lack of advanced IT skills. In the absence of those skills, many IT organizations are in peril of falling behind their rivals. To overcome that issue, Robinson says it’s easier for many organizations to rely on third-party providers to manage their infrastructure while they concentrate their limited resources on application development.

    Any increased reliance on managed services to gain the benefits of a more sophisticated approach to DevOps will vary widely by organization. Crafting DevOps process that span third-party organizations takes time and effort. There’s no way to analyze a workload and then automatically deploy it on the optimal platform without any human intervention. But as IT organizations continue to struggle with mastering DevOps processes, many may conclude it makes the most sense to rely on the expertise of a third-party service provider to manage a hybrid approach to DevOps. Regardless, the pressure to show results by increasing the amount of high-quality code being generated across the enterprise is mounting.

    — Mike Vizard

  • Application Development: Ripe for Encryption

    Application Development: Ripe for Encryption

    As businesses look to accelerate application innovation and move more and more workflows to the cloud, their needs can be at odds with the need to secure the data. This article will outline current challenges faced by application developers to protect data in this new cloud/DevOps environment. It will also provide insights on how to eliminate the burdens from the DevOps team while significantly improving the security of data stored in cloud databases.

    Cloud Migration

    Businesses are looking to accelerate application innovation and move more workflows to the cloud.  Cisco’s Global Cloud index projects a 26 percent growth rate of cloud workloads between 2015 and 2020. As this trend accelerates, IT project owners increasingly run into audit and compliance requirements that prevent transfer of sensitive data outside the traditional security perimeters. The overwhelming benefits of the cloud including the ability to obtain unlimited scalability of infrastructure, set up and tear down rapidly and gain cost advantages due to economies of scale make this trend impossible to contain.

    Application Development Challenges in the Cloud Era

    As applications migrate to the cloud, application developers are using microservices to accelerate application development because they dramatically reduce development time and help them focus on their key value proposition, be it hailing a cab or booking a place to stay. The success of microservice players including Twilio and Braintree demonstrate the benefits of delegating non-core features such as calling, messaging and payment processing to domain experts.

    Surprisingly, when it comes to something as critical as data security, enterprises still leave the burden on the application developers. This distracts them from working on business objectives as they worry about security problems such as how best to encrypt at data and when to change encryption keys—problems best handled by security domain experts.

    Securing Cloud Workloads

    As it turns out, encryption is not easy to incorporate into enterprise workflows. Encrypting application data prevents databases from processing it. Then there is the problem of managing keys, which could be lost or become stale. Encrypting cloud workloads brings a whole new set of challenges as the issue of control comes up. If the cloud vendor has access to the keys, what prevents them from being subjected to a subpoena from a government that is using national security as justification to decrypt the data? Add to that the complexity of migrating from on-premises databases to the cloud without risking the data.

    Pragmatic Approach

    A pragmatic approach to solving this problem is to use existing encryption solutions that provides the detection, classification, policy enforcement and monitoring of sensitive data at all times. These solutions can protect application data at a columnar level but easily integrate into existing enterprise workflows. Application developers can identify sensitive data based on a data classification matrix and the sensitive data is automatically protected as it is inserted into the database. Using existing cryptography techniques, that data can even be processed without ever being decrypted, ensuring the highest level of security.

    About the Author/ Ameesh Divatia

    Ameesh Divatia is CEO and Co-Founder of Baffle, developer of the world’s first solution for end-to-end database encryption. Ameesh is a serial entrepreneur with a track record of converting innovative ideas into successful businesses. Connect with him on LinkedIn.

  • Protecting Data in Public Cloud and Hybrid Cloud

    Protecting Data in Public Cloud and Hybrid Cloud

    Backup, business continuity (BC) and disaster recovery (DR) have all been a critical part of IT for over 30 years—ever since we began relying on technology to run our businesses. Traditional solutions were designed for the world of on-premises infrastructure and structured applications and relational databases. But the world is changing. In the last blog, I talked about the era of digital transformation and the impetus it has on rethinking and reinventing the fundamental backup and recovery architecture for workloads moving to the cloud and applications being born in the cloud.

    What Changed? Reinventing Backup and Recovery

    Application and data platforms are undergoing the biggest transformation since the dawn of computing. There are several forces at work:

    • New applications. The third-generation applications are geo-distributed, scale across multiple systems, always on and typically are deployed in a cloud-first model.
    • Existing applications are moving to the cloud. They aren’t going away, but companies are moving some or all of them to the cloud. They still need backup and recovery.
    • RPO and RTO windows are shrinking: Enterprises want “always-on” and gone are the days when you could do nightly backups.
    • Smaller companies will go all-in on public cloud. SMBs don’t want to be in the business of IT. They’ve been driving the rapid growth of cloud applications and platforms.
    • Enterprises will build hybrid clouds. Enterprises will deploy applications and data across on-premises and public cloud environments. Scale, compliance and other factors mean they’ll need to keep some systems on-premises.
    • Everyone will use multiple clouds. No one will bet their business on one cloud or one provider. Even now, enterprises are splitting workloads across clouds or cloud and on-premises. Development and test may use one cloud, while the same application might be deployed in a private cloud or a different public cloud.

    Cloud’s Impact on Backup, Recovery and Continuity

    Cloud gives organizations much more agility, operational savings, and a pay-as-you go model. Public cloud providers can also build much more resilient infrastructure. Amazon guarantees 99.95 percent availability for EC2 and 99.99 percent for S3; S3 is designed for 11 nines of data durability, with multiple availability zones. Because the cloud is so reliable—and cheap—it’s quickly becoming a backup target for on-premises data. But that shouldn’t trick us into believing backup and recovery are “built-in” when we run applications in the cloud. Even Amazon itself recommends backup services for all AWS-native applications and cloud databases.

    While service availability and data resiliency addresses infrastructure business continuity and disaster recovery, it doesn’t provide point-in-time recovery or application level intelligence for backup and recovery. As good as the cloud platforms are, they don’t protect against logical errors. And research shows 8 out of 10 errors are logical errors—data corruption, user errors.

    Existing Backup Products and the Cloud

    As we noted above, traditional backup and recovery products don’t meet the needs of cloud applications—even for existing applications that have moved to the cloud, and not just because they were built in a different era. Cloud and distributed architectures present several other challenges:

    • Cloud breaks the media server based architecture of traditional solutions. Cloud applications and data don’t reside on a specific array or disk, so you can’t easily back up what you can’t see. Backups also don’t capture configuration data in the cloud, such as AWS Cloud Formation templates.
    • Cloud doesn’t speak the same language. Legacy solutions talk to tape, disk or virtual disks. Backup and recovery in the cloud means integrating with the right protocols, such as the S3 API or Google Cloud Storage.
    • Backup appliances can’t be moved to the cloud. Existing backup appliances such as EMC Data Domain or NetBackup that work extremely well on-premises can’t be picked up and moved to the cloud.
    • Traditional backup agents won’t scale. If you could get a backup agent running in the cloud, it wouldn’t scale well across dozens or possibly hundreds of nodes.
    • VM is not the right layer of abstraction: This is precisely why the core principle of the Datos IO CODR architecture is a scalable application-centric view of data management and data protection that distinguishes it from conventional approaches. This is the exact reason why the CODR architecture introspects application data and uses global semantic de-duplication to achieve storage efficiencies, instead of relying on traditional de-duplication techniques which treat data as an opaque object (such as a VM or a LUN). The benefit of this approach is fine-grained and highly space-efficient data protection that can span clouds over network links.
    • Cloud Gateway or Migration Services: Unidirectional only.

    Data Protection Must Be Reinvented

    The problem of backup and recovery of cloud applications is novel as there are three critical things a cloud backup and recovery architecture should have:

    • Elastic compute only. The architecture should scale efficiently on elastic compute instances. There shouldn’t be any CapEx costs for servers or appliances.
    • No media servers. Backing up large, scale-out databases requires direct a parallel streaming architecture for data movement between the database and secondary storage. Legacy backup architectures rely on media servers that quickly would become choke points. Direct parallel streaming also allows the data to remain available in native formats.
    • Semantic deduplication. Scale-out application databases typically have a 3x replication factor. If you back up individual nodes or even manage to snapshot the whole database, two-thirds of the backup data is redundant. Over time, backups will be 75 percent to 80 percent inefficient without semantic deduplication that works across a distributed architecture.

    — Shalabh Goyal

  • How Healthy is Your Office 365 Environment?

    How Healthy is Your Office 365 Environment?

    The IBM Domino environment was mostly replaced by Microsoft on-premises technology, then later replaced by the Microsoft Cloud Services. One thing that hasn’t been replaced is the need for administrators to have control and visibility of service delivery to end users.

    Whether you’re using IBM applications, Microsoft on-premises servers or Microsoft Office 365, as the messaging administrator you need to provide availability and performance metrics of the service delivered to your management. You also need to understand issues when they arise.

    In short, you are still responsible for the service that you deliver, even if the servers reside in Microsoft’s data center. So, how do you measure the health of your Office 365 service delivery and how do you anticipate issues before they impact your end users?

    Measuring Office 365 Health

    Microsoft works to maintain the 99.99 percent availability it promises from its data center as well as improve both the features it provides and the performance it delivers. The issues that arise generally impact only some features and usually only affect a subset of users with a specific configuration or in one location. It’s difficult to know exactly how these issues impact your global network, and it’s impossible to predict, but it helps to know that Microsoft offers a health dashboard. The dashboard provides an easy-to-understand view of the availability and performance of each Office 365 service out of the data center, and lets users configure alerts. The disadvantage is that the dashboard only shows service out of the data center, not any visibility into the service your users are experiencing. It also doesn’t provide information on how issues are impacting the availability and performance in your company. Nor does it take into account user complaints that can stem from many issues within the Office 365 environment.

    Microsoft provides two tools that can be useful for measuring the health of the Office 365 services delivered to your users. First, the Remote Connectivity Analyzer allows you to run a series of checks from the cloud to the cloud that will tests different part of the Office 365 services. This allows you to check if your tenant has problems with the basic functions of Office 365 by running Outlook connectivity tests, opening a mailbox, performing a mail routing or connecting to Skype to Business Online. The disadvantage is that it is performed from Microsoft Azure to Microsoft Office 365—from one cloud to another. It doesn’t provide you with any information that could help you troubleshoot what is happening at the user location.

    Microsoft’s other tool that can be useful when it comes to checking the connection is the Support and Recovery Assistant. This tool lets you perform tests to help you understand what’s happening when a location is experiencing poor Office 365 performance. However, these tests aren’t automatic or continuous, and they don’t alert you when an issue begins to arise. You still have to rely on user complaints for that.

    That leaves manual testing as the last option for administrators. Manual tests are available in PowerShell. When conducting tests, start by testing pings to Office 365, DNS resolution time, latency and end-user egress points to understand the impact of the network on your Office 365 service delivery.

    Even if these tools are not enough to measure, alert, report or troubleshoot the service to your end users, they still provide something to check the health of the Office 365 service. Microsoft understands that the service out of the data center is not what matters most; it is the user experience. That is why when people think about their Office 365 health, they should always consider the entire service route, from the Microsoft data center to the user sitting at his desk.

    When we measure health, we understand availability and performance of the service.The border between poor performance and unavailability is small, especially for end users. To understand the health of your services you need metrics, facts and alerts.

    What really matters is not poor performance but a degradation of the performance, because that is what users perceive. To measure performance degradation, you need to know the baseline, or what’s normal. This requires a tool that continuously measures the performance delivered to your end users and that can alert you when the performance is declining.

    Finally, because most Office 365 services are used in conjunction with internal infrastructure applications and appliances, they also can impact the service delivered. If you don’t monitor these applications at the same time and on the same dashboard, you won’t be able to troubleshoot the issue effectively. Some companies have hybrid Exchange, some still have Domino application servers, some are using gateways, some have proxy servers and so on.

    It all comes down to service delivered to your end users. Whether issues arise from the data center or your network, it’s important to monitor and understand what users are experiencing so you can maintain the highest level of service. Understanding the relationship between availability and performance from the user’s perspective helps you achieve better insights and oversight of your cloud or hybrid cloud health.

    About the Author / Jean-François Piot

    Jean-François Piot is vice president of Product Management at GSX, responsible for articulating the vision of the company’s product range and translating these into detailed specifications. His experience in IT processes and service management projects comes from previous business development positions at IBM and Beijaflore (IT Service Provider). He holds a master degree from the Grenoble Graduate School of Business.

  • DevOps Q&A: SUMO Heavy’s Brodie on the Hybrid Cloud

    DevOps Q&A: SUMO Heavy’s Brodie on the Hybrid Cloud

    Not every expert says DevOps will enable the hybrid cloud. One says it never will. Find out why in this Q&A with Bob Brodie, partner and CTO of SUMO Heavy, a digital commerce design and development firm.

    David Geer: How will DevOps enable the hybrid cloud?

    Brodie: I do not feel that DevOps enables the hybrid cloud. Hybrid clouds are very complex, and DevOps would need to be supported by a team of network and system engineers. The beauty of being on the cloud is that it’s API-driven. You can provision servers and configure and deploy with tools that are easy to set up.

    Bringing dedicated servers into that environment which do not have tooling to match the cloud counterpart means that it can’t be solely maintained by a DevOps team, but rather needs much more support. We have worked in hybrid cloud environments, and have experienced firsthand the amount of effort and resources required to create and maintain a hybrid cloud environment.

    Geer: What makes hybrid clouds more complex than being either fully in the cloud or fully dedicated?

    Brodie: When mixing dedicated servers with cloud servers on the same network, you will likely need parallel VPNs set up so you are not sending your data unencrypted over the internet and so that if there is a networking issue there is a fallback. Setting up full-duplex, parallel VPNs can require both expensive hardware and specialized resources to configure and maintain that hardware.

    Geer: Why have tools such as some of the ones AWS is building (OpsWorks On-Premises) not hit the inflection point where these are making it easier to enable the hybrid cloud? What other tools are examples of this?

    Brodie: OpsWorks On-Premises is just one tool that AWS has. They also have VPNs to connect your dedicated hardware to your virtual private cloud. These tools aren’t necessarily to make it easier to run a hybrid cloud, but they allow you to utilize existing AWS provisioning and deployment tools to set up and deploy to your existing infrastructure.

    Geer: Why do you think these tools never will make it easier to enable the hybrid cloud?

    Brodie: Hybrid cloud infrastructure is inherently more complex than being fully dedicated or fully cloud-based. They are two different types of ecosystems, and are managed differently. They require different skill sets and levels of experience within those ecosystems.

    Geer: Please expand on your following statement: “The problem is that the methodologies behind the two schools of thought (Separate sysops and dev versus DevOps) are vastly different in thought and practice, and to really bring dedicated hardware into a cloud-based architecture is very cumbersome, especially if the dedicated environment is not utilizing complex virtualization.” What does the difference in schools of thought have to do with it? What does bringing dedicated hardware in have to do with it?

    Brodie: My point here is that “DevOps” is often misused. It isn’t a tool, it is a mindset. It’s a combination of development and sysops. A “hybrid cloud” is a mix of dedicated hardware and cloud infrastructure. My main point here is that DevOps practices do not make managing enterprise developments easier across hybrid cloud environments. Hybrid cloud environments are inherently more complex than the alternatives (dedicated hardware or fully cloud-based).

    The reason for this is that development is becoming more and more intertwined with sysops. For example, five years ago you could download MAMP onto your Mac and be fully set up with a PHP & MySQL development environment. Modern PHP development requires a developer to understand specific versions of PHP and MySQL, how to manage them, how various applications communicate with each other and, often times, many other applications (Redis, MongoDB, Queueing systems, etc.). These can be complex to maintain, and even if they are distributed to developers from the ops team, it will often be in the form of Vagrant boxes or Docker files, which the developer will be required to understand.

    This is the nature of the “DevOps” school of thought, versus the traditional “SysOps separate from development” school of thought. The target environment needs to be considered when developing an application. For example, if the application will reside on AWS then the developer will need to have a solid understanding of the tools available on AWS (RDS, EC2, ElastiCache, etc.). The maintenance of these becomes part of the developer’s job, which was not present in previous times. Adding the extra complexity of a hybrid cloud means that the developer needs to understand the underlying network architecture of that system.

    Geer: In what way is the hybrid cloud a settling point, mostly built on marketing?

    Brodie: Hybrid cloud environments are not ideal. They are costly to maintain and provide more points of failure because of the networking between them. What it can do is provide a halfway point for CIOs that are reluctant to move to the cloud, so they can see some of the benefits on the cloud side and hopefully move their organization to the cloud.

    Geer: What are organizations afraid to let go of?

    Brodie: Many organizations are scared to move to the cloud. They may have a CIO/CTO that is afraid of “losing control.” I have seen many environments such as this.