Tag: data center

  • How Healthy is Your Office 365 Environment?

    How Healthy is Your Office 365 Environment?

    The IBM Domino environment was mostly replaced by Microsoft on-premises technology, then later replaced by the Microsoft Cloud Services. One thing that hasn’t been replaced is the need for administrators to have control and visibility of service delivery to end users.

    Whether you’re using IBM applications, Microsoft on-premises servers or Microsoft Office 365, as the messaging administrator you need to provide availability and performance metrics of the service delivered to your management. You also need to understand issues when they arise.

    In short, you are still responsible for the service that you deliver, even if the servers reside in Microsoft’s data center. So, how do you measure the health of your Office 365 service delivery and how do you anticipate issues before they impact your end users?

    Measuring Office 365 Health

    Microsoft works to maintain the 99.99 percent availability it promises from its data center as well as improve both the features it provides and the performance it delivers. The issues that arise generally impact only some features and usually only affect a subset of users with a specific configuration or in one location. It’s difficult to know exactly how these issues impact your global network, and it’s impossible to predict, but it helps to know that Microsoft offers a health dashboard. The dashboard provides an easy-to-understand view of the availability and performance of each Office 365 service out of the data center, and lets users configure alerts. The disadvantage is that the dashboard only shows service out of the data center, not any visibility into the service your users are experiencing. It also doesn’t provide information on how issues are impacting the availability and performance in your company. Nor does it take into account user complaints that can stem from many issues within the Office 365 environment.

    Microsoft provides two tools that can be useful for measuring the health of the Office 365 services delivered to your users. First, the Remote Connectivity Analyzer allows you to run a series of checks from the cloud to the cloud that will tests different part of the Office 365 services. This allows you to check if your tenant has problems with the basic functions of Office 365 by running Outlook connectivity tests, opening a mailbox, performing a mail routing or connecting to Skype to Business Online. The disadvantage is that it is performed from Microsoft Azure to Microsoft Office 365—from one cloud to another. It doesn’t provide you with any information that could help you troubleshoot what is happening at the user location.

    Microsoft’s other tool that can be useful when it comes to checking the connection is the Support and Recovery Assistant. This tool lets you perform tests to help you understand what’s happening when a location is experiencing poor Office 365 performance. However, these tests aren’t automatic or continuous, and they don’t alert you when an issue begins to arise. You still have to rely on user complaints for that.

    That leaves manual testing as the last option for administrators. Manual tests are available in PowerShell. When conducting tests, start by testing pings to Office 365, DNS resolution time, latency and end-user egress points to understand the impact of the network on your Office 365 service delivery.

    Even if these tools are not enough to measure, alert, report or troubleshoot the service to your end users, they still provide something to check the health of the Office 365 service. Microsoft understands that the service out of the data center is not what matters most; it is the user experience. That is why when people think about their Office 365 health, they should always consider the entire service route, from the Microsoft data center to the user sitting at his desk.

    When we measure health, we understand availability and performance of the service.The border between poor performance and unavailability is small, especially for end users. To understand the health of your services you need metrics, facts and alerts.

    What really matters is not poor performance but a degradation of the performance, because that is what users perceive. To measure performance degradation, you need to know the baseline, or what’s normal. This requires a tool that continuously measures the performance delivered to your end users and that can alert you when the performance is declining.

    Finally, because most Office 365 services are used in conjunction with internal infrastructure applications and appliances, they also can impact the service delivered. If you don’t monitor these applications at the same time and on the same dashboard, you won’t be able to troubleshoot the issue effectively. Some companies have hybrid Exchange, some still have Domino application servers, some are using gateways, some have proxy servers and so on.

    It all comes down to service delivered to your end users. Whether issues arise from the data center or your network, it’s important to monitor and understand what users are experiencing so you can maintain the highest level of service. Understanding the relationship between availability and performance from the user’s perspective helps you achieve better insights and oversight of your cloud or hybrid cloud health.

    About the Author / Jean-François Piot

    Jean-François Piot is vice president of Product Management at GSX, responsible for articulating the vision of the company’s product range and translating these into detailed specifications. His experience in IT processes and service management projects comes from previous business development positions at IBM and Beijaflore (IT Service Provider). He holds a master degree from the Grenoble Graduate School of Business.

  • Deep Art of Scaling for Cloud Environments, Part 2

    Deep Art of Scaling for Cloud Environments, Part 2

    In Part One of our two-part series on scaling for cloud environments, we looked at some of important factors for scaling and best practices for code storage. Let’s continue the discussion.

    Auto Scaling for Next-Generation Applications

    There are a few very good-looking cloud hosting service providers that provide the features mentioned in Part One: AWS, Google Cloud Platform (GCP) and Microsoft Azure. If you are going to auto scale, it’s best to use one of these cloud hosting services. There are other cloud hosting service providers in my knowledge that follow similar approaches, but they are not widely used or are platform-as-a-service (PaaS) ecosystems so we will not cover them.

    Also, it’s important to remember that in auto scaling, all app nodes are changing and created dynamically, with a new host, new IP, etc., so your synchronization mechanism needs to determine how to update the code with the new nodes that may change or be deleted later.

    Options to Auto Scale and Not Die Trying

    Lsyncd

    Lsyncd allows you synchronize nodes across the cluster that are spinning up/down new instances, Work using this script, which monitors the auto scaling group for new nodes and updates the Lsyncd master with new auto scaled slaves. Discovering which IPs or instances are within the auto scaling group is the preamble auto scaling faces, but this script will help resolve the discovery.

    GlusterFS

    Another approach is to use GlusterFS, which ensures high availability and syncs between your web nodes. You need a minimum of two instances in case one fails. The other instance can jump in and still be tolerant for failures. Once the failed instance is back online, it’s added again to the cluster. GlusterFS works similarly to master/master or master/slave. I won’t cover how to configure GlusterFS, but I’m sure this approach was a very good option before AWS EFS was released. If you are not using AWS, this is an awesome option.

    On the other hand, if you are using AWS environments with EFS, consider registering your web node to the master node using “AWS User data,” so every time a new EC2 instance is created by the ASG, it will register this new volume to the master and keep data synced with the rest of the nodes.

    Pre-Baked AWS AMIs (Old Approach)

    Given the fact that your code won’t change or barely changes, baking an AWS AMI and updating the auto scaling launch configuration will do the trick—this is an older but still-used approach. When you have a defined code release schedule and a defined development workflow, you could bake an AMI for every new code release and, once ready, recycle the nodes to use the new AMI with the latest code update. There are plenty of tutorials on this method.

    Admin Server (mainly used for WordPress, Drupal, Magento and CMS-based apps)

    Another wild approach is to use an admin or master server out of your auto scaling group. This is used commonly to sync code from the master to the scaling group, but I believe this a waste of server resources that can be used inside the auto scaling group. I would recommend this approach for WordPress or Magento sites; in addition to this, consider proxying the wp-admin or administrator URL to the admin server, or update the CMS directly to the admin server. As with NFS, AWS EFS will facilitate everything with WordPress, Magento, joomla and CMS-based apps. Click here for an example of what I’m talking about.

    Implement AWS EFS

    Finally, the easiest approach is AWS Elastic File System – EFS. Amazon Web Services released EFS a few months ago as a very promising technology—a distributed file system similar to NFS but improved—think of it as a sophisticated NAS solution. During AWS Re:invent, the company said EFS is based on NFSv4.1 but with better network performance, and is a pNFS workaround. I have waited for such a component in the arsenal of AWS for months, though early access was only available for the Virginia region. Today, though, it is supported across all AWS regions – Yay!

    EFS can be mounted as a local mount point within fstab, similar to NFS, and you can share your code across your AWS auto scaling group: Set your EFS volume in/etc/fstab to load automatically in each reboot. Therefore, If you are in AWS, use EFS; it will facilitate everything—really. To accomplish this approach, make sure to update your user data in each EC2 instance with the EFS mount point, so every time a new instance is created it automatically will have the EFS volume.

    Note 1: All EFS mount points should be in the same region; if an EC2 instance tries to mount from another EFS region, it won’t be allowed.

    More Details can be found here.

    Google Cloud Platform

    Well, what do we have for Google Cloud Platform? I suggest GlusterFS, Lsyncd, rsync or Google Cloud Storage—although it shares a lot of features with Amazon S3, it is faster. Google states that its cloud storage is faster than Amazon S3 due to its infinite edge points across the world and, yes, it is correct. The easiest way to share the code across your auto scaling group is to use Google Cloud storage and GSutils. Additionally, you could use GSutils and call the bucket in each Google Instance—just make sure the visitor/user does not change the code or static content in the instances. Also, Gsutils helps sync the bucket data to your instance data to stay in sync. If your code changes dynamically with your traffic, you need to use gluster or Lsyncd, given that it is very complicated to deploy since Google auto scaling is always recycling instances/IPs and needs some workarounds.

    Don’t Forget Scaling Databases!

    Another topic that we didn’t touch was scaling a database, for AWS RDS, DynamoDB, Postgresql, Mysql and Nosql databases. All these modern databases are easy to scale; some of them are PaaS such as DynamoDB or RDS. For Postgresql, MySQL exists only for manual scaling, which is adding new database nodes to the master/slave cluster. In AWS RDS, with some clicks you can get a slave “replica” and you are scaling already. But I won’t go further about it as it’s out of scope for this piece.

    Scaling Microservices and Serverless Apps

    Definitely, these topics need another talk. There will be another time to explore this new architecture approach of splitting your application into pieces. Microservices is changing the IT paradigm, but to survive for this new paradigm, you need to understand first scaling for cloud before evolving to microservices with Docker, Consul, Compose, RancherOS, AWS ECS, Kubernetes and Serverless apps architectures.

    Conclusion

    diagram

    As you can see, scaling an application can be challenging when you are dealing with a custom app or CMS-based application. With this article,  you have the options and their pros and cons in tackling this constant preamble of how to scale with custom apps. Everybody uses their own scaling process and seriously, they never fit with other applications’ needs. Now that AWS EFS is out there, I’d recommend it every time you can, but in case you are not using AWS, implement Lsyncd or GlusterFS, but don’t forget the other methods that also can help accomplish your scaling goals.

    About the Author / Alfonso Valdes

    alfonsovaldesAlfonso Valdes is the CEO and Founder of ClickIT Smart Technologies, a IT agency focused in the architecture and implementation of Modern Cloud Applications highly resilient in Linux and Open Source technologies. He combines more than 8 years of IT experience in the field of scalable architectures, distributed systems and developing cloud-native applications, and has worked for different U.S companies such as: Sprint Nextel, Texas Instruments and Nike USA. Nowadays he is encouraging enterprises to join the cloud. Connect with him on LinkedIn.

  • Deep Art of Scaling for Cloud Environments, Part 1

    Deep Art of Scaling for Cloud Environments, Part 1

    The number of resources to scale an application are abundant, but every application has its own challenges to scale successfully, and the solution to this problem doesn’t come handy. This two-part article focuses on the best approaches to build a scalable application, software-as-a-service (SaaS) application, mobile app, or WordPress-, Drupal- or Magento-based website. The scope of the approach extends to both auto scaling and manual scaling.

    In my experience, there are two types of applications: ones that use an admin and Admin panel to modify the code, layout and functionality (WordPress, Joomla, Drupal, etc.) and modern, next-generation applications that are stateless and follow the 12-factor app approach.

    But first, we have to decide whether to use auto scaling or manual scaling. It is an important decision to make in the very beginning, as it changes the whole approach and complexity of developing your app. We will be discussing both, but in essence of auto scaling, you will need to consider how to maintain the code synchronized across the web nodes and how to maintain new web nodes that constantly are being created/deleted in the auto scaling group. (With stateless apps this is not an issue, as the images or static content can go to Amazon S3, CDN or isolated services. Later on will explain a bit about these technologies.) Working with manual scaling will facilitate your work—you just need your code synced across all the web nodes. All of the approaches mentioned below apply to manual scaling.

    But on the other hand, auto scaling with an admin application is a headache for some common reasons: Every time a front-end developer  changes something in the admin, code will be inconsistent across multiple web nodes along with the Git versioning.  Because of this issue, I don’t  recommend using Git for such applications—there are always discrepancies between the last commit/version in Git and the last update in the admin CMS, which makes auto scaling with admin even more complicated.

    Let’s cover some approaches to solve this problem. Using an NFS distributed file system will resolve everything for a CMS app, but as everybody knows, this approach pulls down the performance.

    What’s Important for Scaling:

    • Designing a deployment workflow that is easy for your team, your manager and developers—that’s why you are here, to improve development productivity by defining a good mechanism to deploy code to your cluster or auto scaling group. Here, you can think of Webhooks, rsync cronjob, bash scripts, Jenkins, CircleCI, DeployBot, manual pulls or AWS CodeDeploy.
    • Isolating and centralizing all services. Databases, caching session, integrate s3 for static content, implement CDN, worker servers, cronjobs, Load Balancer, app server, web server, etc., to name a few.
    • Dynamically synchronizing code in a live application with many web nodes behind a load balancer or AWS auto scaling Group.
    • Ability to use the 12 factor approach for next generation applications, or stateless apps. You may be using this approach already but you don’t know it!
    • Bootstrapping your application with a stateless approach—this will facilitate all your work when you implement a scalable infrastructure. There are many ways to bootstrap your base application using bash scripts, docker files, vagrant file, AWS code deploy, git, user data file from AWS and baked AWS AMI, among others.
    • Using immutable servers in your environment. Make sure your applications and servers are in a pre-configured image state. Every moment a new server/instance is created in an auto scaling mechanism, the instance should be identically cloned of any other instance in production without any further configuration. As you know, AWS, Docker, Google Cloud, and cloud-based providers are much more easier for immutability. This really helps to deliver software faster and decrease time to market.
    • For CMS based apps, stay out of git for now, unless you have a very good code change control with your CMS admin and a very good discipline to push your changes with every dashboard/back-end change in the CMS.
    • A load balancer for auto scaling, such as AWS ELB, Google Cloud LB, Nginx or HAProxy.

    Advised Practices for Code Storage

    Understanding how your application works in terms of how visitors dynamically change your application data is very important. It’s imperative to understand how the session is saved, either locally or in Redis/Memcached. If it’s saved locally, you need to add this data to your synchronization mechanism. Another storage change is the user image uploads for WordPress or Magento-like CMS-based websites; use of plug-ins such as Wp super cache or w3 total cache makes image upload configuration easy. Plus, the option to integrate with a CDN, Amazon s3 or a cloud storage pushes the boundaries even further.

    But if you have a modern application, you need to understand and guide your development team how to change local image uploads path to a cloud storage or CDN. Make sure all your dynamic data is saved in a SQL or NoSQL database, and all cached database data (db queries, objects, sessions, keys, etc.) is saved to redis/memcached, Taking care of these factors will facilitate your scaling journey.

    Why not use NFS?

    Well, we hear a lot around the internet that the easiest way to achieve scalability is integrating NFS. Though it is the simplest way to look for and implement a fix, reducing the work for a service provider/developer  by almost 50 percent, it’s a real sacrifice in terms of application performance. Autoscaling an environment with a slow response speaks against the whole cause of the setup! It is very irritating to a client whose application speed is 1 second to 2 seconds when switching to a scalable environment with NFS slows the application by at least 50 percent.

    It’s also very disappointing when your small EC2 instance is much faster than your auto scalable environment with NFS. Additionally, using NFS has a point of failure, unless you have a failover in the cloud that is rarely used. It is just not worthwhile to have a complete standby environment waiting for a failure, and it’s better to use this environment within your cluster and have more resources to consume your traffic, or even better use this redundant environment to balance between different regions or data centers.

    Why not use Amazon S3 to Share Code?

    What about using Amazon S3, Google Cloud Files or any cloud storage to share code?

    Implementing Amazon S3 to share code between nodes will slow down your site similar to NFS. There’s not much of a difference accessing the S3 URL to mounting the S3 to a Linux directory, so don’t use Amazon S3 as the document root or code base folder. Keep the use of S3 limited to static content that needs to be fetched over time. Yes, there are websites that can survive and work decently using Amazon S3 for static web pages, but from my experience, I don’t recommend it for websites that support an application responsible for driving revenue to the company.

    Stay tuned for Part Two of our conversation on scaling for cloud environments.

    About the Author / Alfonso Valdes

    alfonsovaldesAlfonso Valdes is the CEO and Founder of ClickIT Smart Technologies, a IT agency focused in the architecture and implementation of Modern Cloud Applications highly resilient in Linux and Open Source technologies. He combines more than 8 years of IT experience in the field of scalable architectures, distributed systems and developing cloud-native applications, and has worked for different U.S companies such as: Sprint Nextel, Texas Instruments and Nike USA. Nowadays he is encouraging enterprises to join the cloud. Connect with him on LinkedIn.

  • Software Defined Data Centers Enable True DevOps

    Software Defined Data Centers Enable True DevOps

    I have been talking a lot recently about the bleeding edge technologies that are offloading the complexity of DevOps management, and software quality. My last post on service virtualization is the latest example. Not lets talk about the software defined data centers. A dense topic, but a massive leap into automating, and unshackling a previously immutable component in the software delivery pipeline.

    (more…)