devopsy.kinsta.cloud

Tag: Cloud Computing

Achieving True Automation is a Marathon, Not a Sprint

But It Starts in the Cloud

Arguably, the most important goals for companies seeking to get into cloud today are agility and flexibility to innovate at speed and scale. Adding cost savings to the mix makes the possibilities all the more attractive. And these three pillars can only be achieved with automation.

But while many companies may believe they’re automating (and thus attaining these three goals), they’re really not. In the rush to cloud adoption, they fail to see that this type of investment takes time and work—a lot of work. Think of it as a marathon, not a sprint.

While it’s a significant step toward achieving scale and speed, simply moving an application to cloud is not enough to attain true automation. Adding genuine automation to your server means enabling it to spin up or down as needed, react to scaling requests and shut down when not in use. Only when these processes are in place can you then start seeing the cost savings and agility promised by cloud.

While there is no simple road map to achieving true, seamless automation (near-Nirvana), there are clear and well-worn steps that organizations seeking these goals can take. Looking at it from a broad perspective, automation in the cloud depends on two major initiatives.

Long-Term Value is in App Automation

The first step is to figure out where automation will provide the most value (from an application perspective). In most cases, at least as far as what we’re seeing across the marketplace, the answer to this will likely revolve around infrastructure.

The second step gets a bit trickier. You essentially want to connect automation to the applications themselves (replacing the manual task of writing a script). The challenge then becomes pulling the application back into the larger architectural context so that it can respond to an event without human intervention. This phase involves making the application cloud-ready—that is, furnishing it with the ability to trigger automation without someone sitting at the controls.

This second phase bears closer scrutiny—it’s here that the people on the ground start to lose faith, because with some applications automation may seem all but impossible. Let’s face it: A fair majority of applications were not designed with automation in mind. Legacy applications in particular (which typically constitute the bulk of most companies’ IT estate) are fairly fussy when it comes to automation. This is when organizations start debating the value of opening code and making changes—whether the investment of time and capital is really worth it.

Which brings us to the two biggest barriers to automation—costs and culture. Yes, it can be expensive to implement the processes and skills to achieve game-changing automation in the short-term. It requires moving application development specialists to new roles, a massive shift in culture and skill sets within IT. This is particularly true for code development, which has been manually focused for so long. Many companies fail because they don’t see automation as an ongoing true software development program (which, of course, it is).

Attaining DevOps

The Holy Grail of automation is achieving a seamless Development-Operations environment (DevOps), one in which software developers and the IT folks are continuously collaborating and communicating to automate software delivery and infrastructure changes. One of the chief goals of DevOps is to create a culture and environment where building, testing and releasing software happen frequently, rapidly and reliably.

Achieving a true DevOps environment means that you are a functioning automated shop. Automation does not end with the first deployment. In fact, it is only the beginning. You will continually face new upgrades and changes in service, requiring near-constant tweaking. But automation smoothes the process, loosening the friction of life cycle management. Automation becomes embedded into the very fabric of your entire IT estate.

For some companies (mostly startups), automation is not a luxury but a necessity. Look at Netflix (which, in some respects is less of a startup and more of an established player). The only way the company could scale and stabilize to meet customer demand with the success it has over the past few years was to automate. It would be physically impossible for Netflix to serve its customers any other way.

Same with Amazon. Jeff Bezos figured out early on the value of automation in the cloud and built the flexibility into Amazon’s infrastructure to grow and scale according to exponential customer expansion.

We’re seeing this happen increasingly across industries—from the financial sector to the healthcare industry.

One large global financial institution is changing its very culture by putting automation at the very core of everything it does, from loan applications to payroll. It has taken the bold step of only adopting applications that can be automated. The simple rule of thumb here is, “If we can’t automate it, we’re not going to do it.” This has resulted in the financial institution being able to provide an improved customer experience, providing features for customization of services via their mobile banking app.

Another is Avalon Healthcare Solutions, a laboratory benefit management company. The organization wanted to accelerate service delivery with a 100 percent cloud-based claims editor. So it developed a portal for providers, patients and health plans to offer appropriate access to data, claims and prior authorization. Avalon can now edit from 28,000 to 38,500 per hour in a single environment with 99.99 percent uptime and 99.95 percent accuracy,

The lessons here are clear. By pacing their race with an eye for the long-term goal, Avalon, Amazon and Netflix—among others—trailblazers across industries, are reaping the benefits cloud-automation offers. Namely, they are scaling, becoming more agile, innovating and delighting their customers all at the same time.

About the Author / Chris Wegmann

Chris Wegmann is Managing director for Amazon Web Services Practice & Relationship lead, responsible for helping Accenture cross-industry enterprise clients leverage the power of Amazon Web Services including through the Accenture Cloud Platform. He has 20 years of experience in technology consulting and specialization in technical architecture, infrastructure design and engineering, legacy system migrations, cloud application architectures and systems operation. Connect with him on LinkedIn and Twitter.

June 2, 2017
Quali CloudShell Rises to Multi-Cloud DevOps Challenge

Multi-cloud computing is now an everyday reality for most IT organizations. Because of that, most DevOps teams are trying to figure out how to apply a repeatable set of processes against multiple clouds; rather than having to manage a unique pipeline for each cloud, DevOps teams want to be able to deploy application workloads on any cloud as necessary.

With that goal in mind, Quali has extended its CloudShell tools for modeling production environments in a sandbox to include support for both Azure and OpenStack environments. Previously, CloudShell only supported Amazon Web Services (AWS) and VMware.

Shashi Kiran, chief marketing officer for Quali, says version 8.0 of CloudShell makes it possible for developer, test, QA and compliance teams to create a truly authentic replicas of a production environments. That replica can then be used to create a blueprint of an application spanning all the physical or virtual infrastructure, data, applications, tools and services in that environment.

CloudShell 8.0 also includes support for application configuration management using either a custom script or playbook created using the open-source Ansible IT automation framework. In addition, DevOps teams now can edit some app details directly within blueprints.

Via a CloudShell web portal, shells now can be centrally managed, updated and removed. The new version also offers enhanced single sign-on (SSO) support, enabling more SAML security features and integration with third-party identity management tools. Finally, CloudShell 8.0 includes increased control of blueprint max reservation time, advanced search improvements and access to new table views.

Kiran says CloudShell is unique in that it gives DevOps teams access to a model that eliminates the surprises that commonly occur when developers attempt to move an application into a production environment. Given the level of scale involved in modern applications, it’s not possible for developers to effectively replicate a production environment on their own machines. The result is a lot of rolling back of applications after they’ve been deployed into production. CloudShell eliminates that problem, says Kiran, by giving developers and IT operations teams access to a sandbox in which to make sure that applications work as expected once they are deployed in production.

Rolling back applications in a multi-cloud world has become especially problematic. IT organizations now need to deploy applications locally on a private cloud and in a public cloud. But from an IT operations perspective each of those clouds is becoming more complex, which means the opportunity is greater for something to go awry when it comes time to deploy an application into production. Include the number of target platforms in the age of the cloud and the probability that an application will work seamlessly the first time it’s deployed is virtually nil. Multiply all the application deployments and updates an IT organization is trying to manage, and it quickly becomes apparent how much time and effort is being lost.

Production environments need to be as pristine as possible. The only way to make sure they remain that way is to make sure that every application that runs on them doesn’t ever get the chance to adversely affect their condition.

— Mike Vizard

May 10, 2017
Is Cloud Adoption on Your Company’s Horizon?

Under pressure to increase the speed of innovation in the digital space and outpace competitors, companies large and small are turning to the cloud. Moving to the cloud provides flexibility of infrastructure, transparency of infrastructure costs and many other benefits.

However, making the transition to the cloud is far more intricate for regulated industries—especially large financial organizations—where security and compliance concerns often trump innovation efforts. According to Cloud Security Alliance, the financial industry is still in the early stages of cloud adoption. In fact, 61 percent of financial leaders say their cloud strategy is in the formative stages. Those financial companies that can shift to the cloud properly will see long-term gains, from both a performance and economic standpoint.

In 2015, while working at a large international bank, I assisted on a project to help transform how the bank approached digital for retail banking and wealth management. The bank was still approaching most projects as large, monolithic endeavors, where six-month delivery cycles were standard. Infrastructure was owned by IT Operations, and provisioning lacked automation. As such, almost every project began with designing environmental needs from scratch—adding time to both creation and delivery. It was quickly evident that cloud was the bank’s best option, allowing for increased transparency, greater scalability, infrastructure on demand and improved organizational efficiency.

Shifting to the Cloud

After working on this project, I learned that to effectively move to the cloud, organizations must understand how to maximize benefits and minimize risks. Here’s how:

Apply a two-pronged approach

We realized right away that we needed a system where we could meet our project objectives, while simultaneously exploring the cloud options. To avoid jumping in too quickly and running into non-technical challenges, we adopted a two-pronged approach: we’d build the platform internally using Cloud Foundry PaaS, while exploring the use of external cloud in parallel—without impacting the delivery of the internal platform. To avoid re-architecture and rebuild later on, we had a guiding principle: Whatever is built internally has to be available in the external cloud as well.

Navigate legal and compliance hurdles

Regulators are still on the fence when it comes to certain industries adopting the cloud, with regulations still lacking specific policies. For example, in the financial space, lack of close regulation leaves many institutions on their own to sort out contracts with cloud providers based on their individual risk appetite. When regulations are published, they are very high-level and generally state that managing the risk and responsibility for integrity of data is with banks, not with cloud providers. Instead of creating an equal playing field, this tilts the scale toward smaller fintech partners with larger risk appetite±and who more regularly fly under the regulator’s radar.

Expect roadblocks

In so many organizations, IT operates in silos. For example, all infrastructure is owned and operated by the IT Operations group, which then recharges that infrastructure back to the businesses based on complex metrics. For cloud to succeed, we need to answer the question of ownership. But that’s not all: Many folks may not believe that organizational issues are a serious problem, and thus, will lack a commitment to be involved. To ease this problem, strong sponsorship from senior leadership is essential.

Prepare for technical issues

When implementing a technology solution, things don’t always go smoothly from the start. For example, there might be approved ways to connect with third-party providers that were created before cloud providers came into play. While this kind of issue is easy to solve, it requires a change in internal procedures that must be accounted for.

Focus on development and testing first, then on production

While working toward final approval and negotiating the agreement, which can take significant amount of time, conducting a trial run is an effective way to work out any potential problems before full implementation. Signing an agreement to use cloud for development and testing only is less of a hurdle, as no customer data will be transmitted or stored. It’s also important to train anyone who might be involved throughout the process—no matter how big or small their role.

Conduct a formal project launch and set clear goals

To drive continuous improvements, it’s essential to establish clear objectives and goals. This provides metrics for benchmarking success and ROI, and encourages the brainstorming of new ideas on a case-by-case basis. Prior to the formal launch, don’t forget to identify key stakeholders and advocates—they will stand behind the project and provide valuable insight into the potential outcomes of decisions.

Create a transparent culture

Full transparency is key, even if it means dealing with resistance. Resistance, especially early on, often is unavoidable. However, it is better to deal with resistance, gain support and handle any issues early in the project. Being transparent gains you more support. Or, if the opposition is insurmountable, you can stop the project before wasting too much time or effort. Most important, you want to avoid criticism suggesting that the project is putting the company at risk. By looping in the right people from the start, you’ll have advocates in your corner who will raise red flags and anticipate problems on the horizon.

Remember that people will want to be involved

While cloud may still be an unfamiliar concept for many, almost always someone will come out and say, “Actually, I have experience, and already thought about how to do this. How can I get involved?” Bring those people on board, leverage additional insights and increase commitment to the project. This is an exciting change—people want to be on the team that introduces it!

Large, regulated companies historically have been slow to implement the cloud, due largely to perceived compliance concerns and security risks. However, in an environment marked with rapid innovation, companies are quickly realizing that to stay ahead, adopting a cloud-first business model is a must.

About the Author / Alexander Kalinovsky

Alexander Kalinovsky is Director of Engineering at Wipro Digital. He is a senior IT leader with extensive experience managing successful application development and operational teams, along with establishing global development practices and partnering with global organizations. Alex has spent over 10 years working in Financial Services in development and infrastructure, specializing in payment systems and FX solutions. Alex brings his strong FS background and enthusiasm for Agile development to his role at Wipro Digital, driving the digital transformation agenda for many of the world’s leading brands.

February 15, 2017
Top Reasons to Use Node.js

When considering the architecture of a new web app, it might be a good idea to include Node.js in the discussion as a key part of that architecture. Node.js could provide you with additional speed, can be used on numerous platforms and has an active community from which to get modules and other assistance.

What is Node.js?

Node.js is a server-side platform developed by Ryan Dahl in 2009. It was built on Chrome’s V8 JavaScript engine and designed with a non-blocking, event-driven I/O. It is also cross-platform, able to be run on Windows, Linux and OS X. Its non-blocking I/O is due to its use of asynchronous functionality, which keeps the server from waiting on data to be returned before being able to move on to other tasks.

Why Use It?

Given the architecture of Node.js, a number of advantages are provided both to developers and the resulting apps they develop. A few of these advantages include speed, programming language consistency, real-time apps developed easily and a large number of modules/packages for developers to use while creating new apps.

Speed

Node.js offers speed advantages in many cases. First, it uses the Google V8 engine, which is optimized for JavaScript speed. Also, its use of non-blocking asynchronous behavior can ease the time it takes for users to see actions as complete, since things can be going on in the background while the user moves on to other tasks.

One Language

JavaScript is listed as the most popular programming language. Source: Thenewstack

Using Node.js, front-end and back-end code can both be in JavaScript, so your developers don’t have to transition between languages as they move through the stack. Some architectures, such as the MEAN stack (MongoDB, Express.js, Angular.js, Node.js), are designed around this principle. In this case, the stack uses JavaScript for Express, Angular and Node. MongoDB uses BSON for database queries, which uses the same syntax as JSON (JavaScript Object Notation).

Seeing full-stack developer as the largest occupation and JavaScript as the top language among both back-end and front-end aligns with what we’ve been seeing in overall JavaScript adoption.

Source: Mikeal Rogers, Node,js Foundation Community Manager

For developers, this can help both with moving across the stack during development as well as help communication among developers, as they will be referencing the same language (JavaScript) while discussing programming decisions.

Real-Time Apps

Node.js is a great platform for creating real-time apps. With a module such as Sockiet.io, you can create real-time chats, games and more. Along with its non-blocking asynchronous architecture, real-time apps can be developed with plenty of speed and responsiveness.

Modules

The Node Package Manager (NPM) makes it easy to install packages that others have developed to perform any number of tasks. This helps to encourage code-sharing among developers and to reduce the amount of custom development that has to be done when one or more packages can take care of that coding for you.

Notable Examples

Some notable examples of Node.js use are Paypal and Uber, showing that it can, indeed, be useful for large projects.

Paypal

Node.js and an all Javascript development stack helped PayPal bring efficiencies in engineering and helped rethink and reboot product, design and operational thinking.

Source: Sameera Rao, Paypal Sr. Business Products Engineering Manager

Back in 2012, Paypal had a single app running all things, including UI, API and more. This included a lot of duplicated code that was copied and pasted from one app to another to make apps for different locations.

Using Node.js, the company was able to modularize and clean up its code and processes, which in turn sped the development of apps it needed at any given time.

According to Paypal’s engineering team, they were able to build the Node.js app with 33 percent fewer lines of code and 40 percent fewer files, and were able to build it twice as fast and with fewer people than before. Certainly, Node.js proved to be advantageous to their development.

Uber

Uber used Node.js from the beginning, and its distributed system was a perfect fit, since a lot of network requests are made. The non-blocking asynchronous I/O used by Node.js helped to ensure those requests could be made and handled quickly to provide the best service possible.

Conclusion

The benefits of Node.js have been proven. It helps speed development, can be used on numerous platforms and has an active community from which to get modules and other assistance. Include Node.js in your future conversations about website architecture.

— Brian Wheeler

January 4, 2017
12 Tips for Managing Your Multi-Cloud Environment
No single cloud vendor has the perfect solution for your unique business situation. The only way to maximize the benefits of cloud services to your business is by dealing with several cloud providers and monitoring all of them simultaneously. That’s the IT equivalent of singing an aria while standing on your head and scrambling eggs. (Not really, but it can seem that way.)

Here are 12 things to keep in mind when planning your company’s multi-cloud strategy:
- Map your entire network, and then identify where the cloud fits in it. Get a clear picture of the cloud’s role in your overall system-management strategy to avoid having to backfill gaps in the data needs of business managers and other customers. Different lines of business will be best served by different cloud providers.
- Work with vendors that respond quickly and positively to the unexpected. There will always be unforeseen situations when you’re stitching together multiple cloud services. Your partners should report regularly on the state of their services, enhancements, and new features.
- Automate low-level monitoring and maintenance tasks. Two primary reasons for adopting cloud services are cost savings and more efficient operations. One way to achieve both is by reducing the amount of human oversight required to keep your systems humming.
The potential complexity of multi-cloud management is shown in this flowchart encompassing the many elements that need to be stitched together precisely. Many of these low-level operations must be automated to optimize your strategy. Source: Carl P. Lehrmann, 451 Research
- Follow your users. Pay close attention to how the “consumers” of your apps and systems are actually putting the tools to use. They are the best indicators of which processes are the biggest productivity boosters. You can also learn about any new services the users discover that help them get their work done faster, more efficiently and/or less expensively. This lets you get a jump on the kinds of tools the users are most likely to need in the future.
- Double down on cost controls. You can’t control what you can’t see. Dealing with multiple vendors complicates the process for requisitioning new services. To avoid unnecessary impediments to quick implementation of useful new services, devise a purchasing process that is as flexible as the services themselves. However, not long after the service honeymoon period, evaluate whether each new service is proving to be worth the cost and whether there are better and/or less-expensive alternatives.
- Application life cycle continues to dictate cloud migration. Data centers may be shrinking, but they are definitely not disappearing. For reasons related to security, compliance, insurance, and any number of other factors, some systems are staying in house for the foreseeable future. You certainly want to avoid attempting to lift and shift an in-house app to a cloud service, particularly one that is only a couple years old and thus hasn’t yet paid for itself. Once an older system is ready for a reinvention, it’s easier to make a business case for going cloud-native.
- Take advantage of the power of pre-emptable virtual machines. Because they persist for no longer than 24 hours, pre-emptable VMs are a good choice for batch processing and fault-tolerant deployments, according to ComputerWeekly’s Matthew Cooter in a November 2016 article. Pre-emptable VMs are as much as 80 percent less expensive than Google’s regular instances, and their fixed price help companies plan and manage their costs.
- Adopt an integrated data center management system designed for virtual environments. This is the only way to ensure your server, storage, network, security, operations and applications teams are working as a single unit toward common goals, according to Jay Litkey, president and founder of cloud service provider Embotics, in a November 21, 2016, article in Virtual Strategy magazine. Any organization structured around traditional physical infrastructure boundaries is destined to struggle with the shift to hybrid and multi-cloud environments.
- Identify the apps in your organization that are best suited to cloud-native implementation. Traditional apps are monolithic, run as VMs, use a scale-up architecture and are generally more difficult to develop, deploy and maintain. By contrast, cloud-native apps are more modular and service-oriented, comprised of collections of containers and services, based on a scale-out architecture, and easy to automate, move, and scale.
Focus on the apps in your organization that will benefit most from a cloud-native approach, and wait on those that are better served by deployment on a traditional data-center platform. Source: TechTarget
- Match your apps to the services that play to their strengths. Big-name services such as VMware and AWS have bifurcated to meet the diverse needs of their customers. For example, VMware’s vSphere-based Enterprise Hybrid Cloud is intended for traditional apps, and the company’s Native Hybrid Cloud is designed for modular, microservice-based systems. Similarly, AWS’s Elastic Compute Cloud, Elastic Block Store, EFS file system and VPC private networks mimic in-house data centers, while the company also supports higher-level services such as NoSQL databases, BI processing, Hadoop-like clusters, message queues, push notification services, media transcoders and search engines.
- Automate policy across cloud environments. The only efficient approach to use of multiple cloud services is to have a single standard for policies that are applied automatically to each environment. The policies cover such areas as virtual servers, workloads, data storage, traffic flows, compliance/regulation, reporting and security. It’s also easier to apply updates and changes when you have a single configuration across different infrastructures because the changes propagate seamlessly from environment to environment.
- Vendor lock-in becomes a thing of the past. When you rely solely on one public cloud service, it can be difficult to migrate virtual machines and other resources to another service. Among the factors to consider are comparable read/write loads, data transfer needs and network latency. By relying instead on your custom combination of cloud services, all of which share policies and configurations, you find yourself managing the change instead of the change managing you.
— Brian Wheeler
December 5, 2016
Taming 2 Cloud App Performance Enemies

Whenever you hear the words “the network’s slow today,” or, even worse, “the network’s down,” you know somebody miscalculated. It may have been the person who decided to buy the wrong hardware, or maybe the one who signed up the wrong telecom provider.

Often, that slowdown or outage was the result of somebody underestimating server demand or overestimating the amount of redundancy built into the network setup. It’s always tricky finding the right level of redundancy that will keep users from waiting for their data without overspending on cloud capacity.

A common mistake IT managers make when switching from in-house infrastructure to cloud infrastructure is failing to take advantage of the flexible, affordable, redundant capacity offered by Amazon Web Services, Microsoft Azure and other cloud providers. That’s the conclusion of Yan Cui in a April 20, 2015, post on his “The Burning Monk” blog.

Cui points out that latency is inevitable because things go wrong—and usually, those things are beyond your control. That’s why you have to design your applications with latency in mind. Too often developers work inside a bubble, where all hardware and network parameters are geared to the tasks at hand. In the real world, hardware, networks and software break. If your app isn’t designed to accommodate such failures, you’re leaving your users in the lurch.

Get Real: Inject Latency in Your Test Environment

To model app performance in real-world settings, it isn’t enough to test servers and databases under anticipated loads. You have to take the extra step of mimicking the glitches that are as inevitable as they are unpredictable. Cui proposes injecting random latency delays on every request. He even provides a simple PostSharp attribute that automates the process.

Cui applies the concept of error-planning even further by introducing errors in the mechanism used to throttle requests to prevent bad guys from spamming their servers using such tools as Charles and Fiddler, or any of the spammers’ own bots. By making the errors visible in the dev environment, designers can anticipate them and engineer ways to handle the throttling errors “gracefully.” Once again, a PostSharp attribute is applied to randomly inject throttling errors; Cui states that the same approach can be used to plan for and respond automatically to session expiration, state out-of-sync, and other service-specific errors.

These and other error-detection and -correction methods fall under the umbrella of Design for Failure, a principle championed by Simon Wardley as a key characteristic of next-generation tech companies.

Simon Wardley’s Design for Failure contrasts the development approaches of traditional shops and those he identifies as “next generation.” Source: The Burning Monk

Cui explains how modeling latency and other errors within the development process has been applied by Netflix’s use of Chaos Monkey and Chaos Gorilla, as well as the mechanism Uber devised to allow its client applications to fail over seamlessly in response to a data-center outage.

Learn How to Combat Latency from the experts: Gamers

Nowhere does server performance matter more than in the world of online games, where one-microsecond hiccup can spell the difference between winner and loser, champion and chump. In an April 17 article on Venture Beat, Cedexis’s Pete Masden explains that a 500-millisecond network delay makes an online gamer twice as likely to walk away, while a 2-second delay in load time leads to abandonment rates as high as 87 percent.

Addressing latency begins by measuring it, according to Masden. That’s why we added native monitoring to Morpheus when we built the cloud management platform. Being able to see the status of all your organization’s apps and business services at a glance isn’t just a nice convenience—it’s a competitive advantage. To be able to glance the status of all your organization’s apps and business services, group databases, app servers, messaging queues and web servers by system dependency and impact to view an overall status as well as each app’s current state is crucial to knowing what’s happening in your organization.

One approach to cloud-server latency is to create separate instance, which is common for failover but less so for addressing regional network demand. Rather than the traditional active-passive model, however, Masden recommends an active-active approach when implementing multiple redundant instances to ensure all servers are up to date and optimized. Atop the multiple instances runs the global traffic analyzer to ensure no single instance becomes overly congested.

Is ‘Zero-Latency Computing’ Even Possible? Why not?

If you think network latency is a problem now, just wait until such technologies as 5G, the internet of things (IoT), virtual reality (VR) and robotics go mainstream. Equipment and bandwidth upgrades, and more sophisticated software-defined networking won’t be sufficient to address tomorrow’s application-performance needs. That’s the conclusion of Nitin Serro in a Sept. 29 post on the Channel Partners blog. Serro identifies three technologies as key to meeting future performance requirements: network functions virtualization infrastructures (NFVi), fog computing and zero-latency computing.

The applications made possible by 5G and related technologies come close to eliminating latency by shifting more intelligence directly to the network, not merely by providing faster speeds. Source: GSMA Intelligence, via The Unwired People

While NFVi and fog computing focus primarily on the network and data layers (compute, storage and network infrastructure), zero-latency computing targets the application layer, where information has to be exchanged in real time between diverse components, most of which link directly to users. According to Serro, nothing short of an entire re-invention of the network infrastructure—from backbone to edge—will prevent today’s bottlenecks from becoming tomorrow’s blockages.

Move the Network Intelligence Closer to the Data

The network-capacity challenge we now face is unprecedented, according to Silicon Republic’s John Kennedy, who writes in an Oct. 3 post that even the much-touted 5G network will come out of the gate hobbled by the need to accommodate its many predecessors. Rather than the “step change” in network speed that most analysts expect, 5G will deliver a tremendous boost to network intelligence. This may be the technology’s greatest asset: a smarter network means the data doesn’t have as far to travel.

From an application-design perspective, latency can be traced to three sources: the need to link to legacy infrastructure, the need to link to third parties via APIs and the innate unpredictability of the internet itself. Marqeta’s Dave Matter examines these situations from a payment-processing perspective in a Sept. 13 post on App Developer Magazine.

As with the network infrastructure itself, the solution to application latency is to move more of the intelligence into the app itself and then rely on open APIs to link data sources in real time in an environment that’s closer to users. Once again, the data and associated analytics tools have less distance to travel, and much less latency to deal with. Ultimately, the solution to latency—in hardware and in software—will be fewer, shorter trips.

— Brian Wheeler

October 10, 2016
Fog Computing Meets Cloud Computing, and the Forecast Is Sunny

Fog computing is more than just cloud computing meeting edge networks, but for practitioners more interested in results than in new buzzwords, the question is whether fog’s IoT promise deliver real-time data storage, processing and analysis.

One thing all new technologies share is the lack of a consensus on what exactly defines the technology. Ask some folks what “fog computing” is, and they’ll tell you it’s synonymous with edge networks. That’s the claim made in a July 29 post on Nanalyze, which traces fog computing concepts back to the days of mainframes and dumb terminals.

On the contrary, ReadWrite’s Ryan Matthew Pierson explained in an August 5, 2016 article the many ways that fog computing is distinguished from edge networks. The primary distinction, according to Pierson, is that edge networks connect sensors to programmable automation controllers (PACs) for data storage and processing, while fog computing requires that the sensor data be transferred to local gateway at the end of the cloud “chain.”

Pierson claims the fog computing model is more scalable than edge networks because each gateway has more power to “make more complex and dynamic decisions.” The gateway approach also accommodates a range of data at various flow rates. However, he said, this design creates more potential failure points.

Fog Computing = Edge Networks + Analytics?

As you might expect, there’s a third, middle-ground approach that defines fog computing as edge networks to which real-time analytics have been added at the node level. That’s how FogHorn Systems CEO David King explained the technology in a Sept. 13 article by CIO’s Stephen Lawson. FogHorn’s Lightning software platform is designed to deliver real-time analytics to IoT gateways, as well as to existing industrial products, including hard-wired controllers deeply embedded in equipment such as locomotives. Some of these embedded controllers are based on proprietary OSes that have been around for decades.

The key feature of Lightning’s design, according to King, is its close ties to cloud servers, which ensures that the data collected at the edge of the network is available to analytics tools that reside in the cloud. CIO’s Lawson quoted Machina Research analyst Aapo Markkanen as stating that linking IoT data sources to cloud analytics is the only way to get the “big picture” view required to ensure accurate analyses.

In an Aug. 19 interview with App Developer Magazine’s Richard Harris, King explains how fog computing addresses the shortcomings of previous distributed edge networks in terms of bandwidth use and the resulting “performance-killing application latency.” The FogHorn system is designed to move the “intelligence layer” closer to the machine data, resulting in faster and more cost-effective analysis. King claims the company’s software makes possible “real-time remote monitoring, asset optimization, proactive maintenance and operational intelligence applications.”

Fog computing adds a new layer that connects mobile, IoT (sensors, embedded controllers), and vehicle control systems to the depth and breadth of data resources in the cloud. Source: SpinalCom via SlideShare

Many of the features promised in fog computing systems—remote monitoring, asset optimization and proactive maintenance—are available today in best-in-class tools. You should be able to monitor SSH and Agent-based connectivity to apps running in private, public and hybrid clouds.

Fog Computing Delivers Analytics for Time-Critical Applications

The cloud’s centralized nature has never been a good match for the short response times required by data processing and analytics in IoT environments. Ben Dickson wrote in an Aug. 2 TechCrunch article that fog’s decentralized architecture moves the analysis tools and other resources closer to the edge, where the data is being collected and actions applied. Telemedicine, autonomous vehicle control and remote security systems are examples of time-critical applications requiring near-real-time analytics.

Another advantage of maintaining data at the edge of a widely distributed network is the privacy and security benefits of keeping sensitive data off the internet, where it would be subject to strict privacy regulations. Dickson presented the “fog layer” of the network as offering the compute, storage and network resources required to “mimic cloud capabilities.” By ingesting data locally, you can analyze it and convert it into action almost instantly.

The complete IoT model extends from the data source (human or machine) to the data center in the cloud, with much analytics-driven decision making occurring in the “fog layer.” Source: Moor Insights & Strategies, via Forbes

Dickson pointed out that the fog layer serves as a complement to cloud computing rather than as a replacement. In fact, fog computing couldn’t exist without all the data, tools and other resources now residing in the cloud. The larger data sets and deeper analyses being done in the cloud serve as the foundation for the fog layer’s quick-turnaround analytics. The fog’s capabilities are constantly being updated and enhanced based on the big-picture view the cloud affords.

Fog Computing Standards Take Shape

Today’s mobile networks simply can’t meet the needs of tomorrow’s data operations. Dropped connections, bandwidth bottlenecks and latency—the time it takes for data to travel from point to point on the network—are all too common. Upgrading the “infrastructure” at the edge of the network requires a concerted effort to ensure open access for diverse tools and resources—some of which we can’t even imagine, let alone prepare for.

Princeton University’s Edge Lab is taking the lead in bringing together tech vendors, organizations and customers of all stripes to create the Open Fog Consortium, which is intended to design an architecture for fog computing. The official announcement of the group was announced at the Internet of Things World Forum in Dubai Dec. 6, 2015; John Sullivan reported on the announcement in a Dec. 16 post on the Princeton University blog.

Open systems have proven their worth many times over, but an open architecture for IoT networks is even more important. Estimates are that by the end of 2018, there will be 22 billion connected devices generating an unending stream of unstructured and semistructured data, according to numbers from IDC research quoted by Forbes’ Gil Press in a Nov. 10, 2015 article.

The flood of data would overwhelm the bandwidth and network capacity organizations currently have, as Jelani Harper explains in a March 10 article on Data Informed. Only an open, decentralized model that keeps the bulk of data and its supporting resources at the edge of the network will prevent a “datapocalypse.”

For many network requirements, cloud computing and fog computing complement each other. Source: Cisco Systems

How much of a company’s data can be maintained in these edge-network nodes? Harper projects that as much as 90 percent of the data that will be collected by IoT devices will never reach the data center. Not only does this reduce strain on the network, it also speeds up “time to action” because analyses are performed where the data resides: on the edge.

The advent of fog computing can be seen as an expansion of the cloud via the creation of a “fog layer” in which many data assets and resources are reallocated. That layer is where IoT’s incredible data reach meets the cloud’s infinitely scalable compute, storage and big data analytics. What is fog, after all, but cloud you can feel?

— Brian Wheeler

September 27, 2016
Cloud Computing in 2016 and Beyond
Few activities are more perilous than predicting the future of the tech industry. That goes double for cloud computing forecasts. Yet, with thoughtful deliberation and the appropriate amount of caution, you can get a fairly accurate read on what’s to come in the increasingly competitive cloud services market.

Here’s a quick look at several trends evident among leading cloud vendors and technologies through the first half of 2016, along with a look ahead at what awaits us for the second half of the year, and on through the next decade.

Google Makes a Major Cloud Play

The company synonymous with search isn’t used to looking up at the leaders in its markets, yet according to figures compiled by Synergy Research Group, Google’s 4 percent share in the cloud services market in fourth quarter 2015 makes it an also-ran behind Amazon (31 percent), Microsoft (9 percent) and IBM (7 percent). Market Realist’s Puneet Sikka reports on the share numbers in a May 31 article.

Google’s first-quarter 2016 earnings call highlighted three recent actions by the company that signal its intention to grow its cloud business:
1. Google has gained experience through its internal cloud operations that it is now ready to share with its customers.
2. The company is integrating its artificial intelligence (AI) and machine-learning expertise with its cloud services for enterprises.
3. VMWare co-founder Diane Greene was appointed to lead Google’s cloud business in late 2015.
AWS, Microsoft Azure Continue to Raise the Cloud Services Bar

Even with its depth of technical expertise and seemingly endless resources, Google faces a formidable challenge in taking share away from Amazon and Microsoft in the skyrocketing cloud services market. Among the 722 new features added to AWS in 2015 was the Aurora relational database that Amazon claims is its fastest ever, according to Sikka. The Amazon QuickSight (AWS data analytics and visualization) tool has helped AWS sustain quarterly revenue growth near or above 50 percent for five consecutive periods.

Amazon Web Services racked up five straight quarters of revenue growth at or exceeding 50 percent through first-quarter 2016. Source: Market Realist

Meanwhile, Microsoft’s 2015 revenue from Azure, Office 365 and Dynamics CRM increased a whopping 140 percent year over year. Microsoft’s continued ascendance, as well as that of other cloud service providers, is attributable in part to a trend highlighted in RightScale’s 2016 State of the Cloud Survey: Companies using the cloud run apps on an average of 1.5 public clouds and are experimenting with another 1.5 public clouds. They’re also using 1.7 private clouds on average, and they’re experimenting with 1.3 private clouds.

Companies running apps in the cloud use an average of three different public and private services. Source: RightScale

The RightScale survey notes that while AWS’s share of the public cloud market was flat at 57 percent in 2015, the number of workloads per customer increased, which helps account for the service’s 69 percent jump in revenue for the year over 2014’s figures. Venture Beat’s Jordan Novet digs into AWS’s 2015 financials in a Jan. 28 article.

Vendors Called Out for ‘Cloudwashing’

The rosy outlook for cloud services through this year and beyond is echoed in Gartner’s Cloud Computing Primer for 2016 (Gartner account required to access). Yet, cost and security continue to be two primary obstacles to cloud adoption. One other impediment to success identified by Gartner researchers is the tendency of cloud vendors to make unsubstantiated claims about their offerings. Gartner refers to this as “cloudwashing.”

The industry has matured to a point where most obstacles to cloud adoption are non-technical, according to Gartner: adapting the organization’s culture, skills and management to a cloud framework; and winning the hearts and minds of end users, who often are hesitant to overcome their innate fear and mistrust of the cloud model. (Yes, your sensitive data is indeed safer on a cloud server than it is on your local hard drive.)

Much of the confusion that Gartner lumps into the “cloudwashing” category relates to a lack of clear definitions of cloud terms such as “infrastructure as a service” and “platform as a service”—two categories whose distinctions will continue to blur.

They’re All ‘Hybrid’ Clouds—or Soon Will Be

Also becoming less discernable are differences between public and private clouds, both of which are morphing into versions of hybrid cloud, according to Gartner. In fact, it’s not unusual for enterprises to have multiple cloud strategies being implemented in parallel. An overarching aspect of this trend is that the fundamental role of IT is changing from a service provider to a service broker.

The Gartner survey points out that IT pros now wear several hats and switch between roles in the course of a given day, from provider to broker to consumer. The hybrid model itself is evolving from “public/private” to “fast mode/safe mode,” according to Gartner. Confusion surrounding the term “hybrid” as it relates to cloud services is likely to continue with the rise of mobile devices, where “hybrid” refers to apps that are both web-based and native to the mobile platform.

One way to ensure cloud customers and vendors are speaking the same language is to focus on cloud outcomes rather than the means of achieving those outcomes. Customers need to define what results they expect from their cloud efforts—financial effectiveness, tapping the expertise of cloud providers, delivering immediate results to business consumers and making overall IT operations more efficient, for example.

What’s in Store for 2017 and Beyond?

The popular buzzwords resurface when you scan the prognostications of the cloud experts: microservices, containers, open source. Yet customers—particularly those in enterprises—have the same concerns about cloud computing now that they had when the technology first emerged a decade ago: security, risk management, governance, compliance and disaster recovery. How much progress is being made on these fronts?

Perhaps those areas will be addressed not by traditional IT providers but instead by web-centric services and other non-IT companies that develop and market cloud solutions for their vertical. You can’t blame non-cloud service providers from salivating at the prospect of grabbing a share of the booming cloud market. According to Forbes’ 2016 Roundup of Market Estimates, cloud forecasts are dominated by a single preposition: “Up.”

CLICK TO TWEET
- Public cloud IaaS hardware and infrastructure software revenue up to $38 billion in 2016 from $25 billion in 2015 and jumping to $173 billion by 2026 (Statista)
- Worldwide public cloud revenues up to $117 billion in 2017 from $80 billion in 2015 and $167 billion in 2020 (TBR)
- Cloud’s share of annual IT budgets up from 29.8 percent as of January 2016 to 43 percent by January 2018 (IDC)
No matter your organization’s current state of cloud adoption—whether dipping your toe, or in with both feet—you’ll future-proof your cloud investment by relying on vendor-agnostic application-management services. There’s no better way to take advantage of what the cloud has to offer now while ensuring you’re positioned to benefit from tomorrow’s cloud enhancements.
August 24, 2016
The Maturing OpenStack Cloud: Just Do it
We’ve been thinking about the maturing cloud a lot lately. In fact, we’re participating in a panel discussion on the topic at the upcoming OpenStack East in August in New York. As we’ve spoken with OpenStack users and vendors throughout the world we’re hearing a common theme about the maturation of OpenStack clouds—the results seldom look like what was originally planned. Disruptive technologies, changes in personnel, the evolution of business strategies … these are all unplanned events that can change what the mature cloud eventually will look like. The challenge for today’s DevOps teams is to how to maximize the opportunities for success, despite the likelihood of unanticipated changes. After all, as many DevOps folks know, between the design of an application or system and the completion of that project, they often are faced with a change in the initial business requirements of the project.

Lao Tzu famously said, “The journey of a thousand miles begins with one step.” While that is true, we’d add that the journey will be much smoother if that one beginning step relies on careful consideration of the best shoes for the journey: shoes that are comfortable, durable and multi-purpose. Shoes that see you through any type of weather. The right shoes won’t necessarily get you to the right destination, but they certainly will make your journey easier and better ensure you complete your journey—even if you end up somewhere other than where you thought you would.

So what do shoes have to do with the OpenStack cloud? We liken choosing the right shoes to making the right architectural decisions at the beginning of your OpenStack journey. One of the most important architectural decisions DevOps can make early in the development of the OpenStack cloud is to include a comprehensive data protection strategy, accompanied by the appropriate tools to facilitate that. There are three primary reasons to include data protection from the start:
- It’s a “given” for VMWare and other enterprise applications for a reason. One might argue there’s no need for data protection, since everything lives in the cloud. But we all know files, applications and systems can fail for no apparent reason. It’s much easier to recover from a failure if you have a data protection strategy in place, rather than having to suddenly (and often quickly) re-create the lost resources.
- It will make it easier to manage OpenStack and cloud resources. As your OpenStack cloud grows, you may need to move workloads to different availability zones or data centers. While you may have a general idea of your resources needs at the beginning of the DevOps process, chances are things will change along the way.
- It makes for a smoother upgrade process. Chances are that you are not rolling out your OpenStack cloud with the exact OpenStack version you started with. For example, one of our customers started on Kilo and now is migrating to Mitaka for its rollout. Their migration is much easier because the customer knows everything is backed up, available to use and restored in its new, upgraded environment.
Sure, the OpenStack and the cloud paradigm is enabling DevOps and business users to upend many “traditional” approaches to IT. But that doesn’t mean you should toss out every strategy that’s considered “traditional.” After all, if you were buying your shoes for your thousand-mile journey, would you only buy one pair? Or make sure you have a spare just in case?
August 1, 2016
Financial Drivers for Cloud Migration in Enterprise

Clouds are a vast pool of usable and manageable virtualized means such as development platforms, hardware and services. These assets can be animatedly reconfigured to adjust to a flexible load, which helps in the optimal utilization of resources. Your enterprise breathes through IT—to keep your business moving forward, you must modernize and be more flexible to break the shackles of the orthodox technologies that are holding you back.

The cloud’s pay-as-you-go business model enables enterprises to cut capital spending by leveraging the cloud service. It also offers several benefits required by the enterprise, including lower operating cost, no upfront investment, great scalability and much more.

To gain the benefit of cloud computing and guard the current investment to the legacy system, companies are keen in migrating their legacy systems to the cloud. The cloud provides a cost-effective corridor to transforming legacy systems to more modern technologies. It offers modern efficient development structures, improved supervision of IT resource supply and demand and enhanced scalability, and it provides access to these assets from almost any part of the world. With this upgraded dexterity, you are able to respond more swiftly to the changes in the market, exploit newer business prospects, surge financial growth and drive innovation.

However, to benefit from what the cloud could offer, you are required to spend money and time on launching your cloud and migrating data and applications to the newer environment.

Migrating databases from on-premise machines to the cloud has a few crucial advantages: Cloud elasticity delivers instant flexibility in scaling up or down. And, the cloud model saves money by shifting companies from early heavy capital expenses (CapEx) of on-premise hardware to constant, known operational expenses (OpEx) on cloud facilities and more.

Migration Ahoy

But migrating live applications to the cloud is not an easy task. Listed below are few methods to migrate to the cloud.

Rehost: This term refers to redistributing applications to a diverse hardware setting and changing the application’s structure configuration. Rehosting the application while making no changes to its construction can offer a faster migration to the cloud. The key benefit of infrastructure as a service (IaaS) is that the systems can be migrated by the teams quickly, with no modification to their architecture.

Refactor: This term refers to applications run over the cloud provider’s infrastructure. The main benefit is combining innovation with awareness, as “backward-compatible” platform as a service (PaaS) means developers can reuse frameworks, languages and containers in which they have invested. Its biggest drawbacks include transitive risk, missing capabilities and lock-in framework.

Revise: This method requires extending or modifying the present code base to assist the legacy transformation requirements, and then using the refactor or rehost options to position the cloud. This method enables enterprises to enhance the application to influence the cloud features of providers’ infrastructure. However, this method requires upfront development work, which needs upfront outlays to assemble a development team.

Rebuild: This term refers to rebuilding the solution on PaaS, discarding code for the current application and restructuring the application. Though rebuilding involves dropping the familiarity of present code and frameworks, the benefit of rebuilding the application is admission to advanced and innovative features in the platform of the provider. They increase the productivity of the developer. Still, lock-in is a major drawback: Should the provider make a pricing or technical change, breaches the service level agreements or goes out of business, the customer is required to switch, possibly deserting some or almost all its application resources.

Replace: This method requires discarding the present application and using the commercial software offered as a service. This method reduces or eliminates any investment in assembling a development team when there is a change in the business function. Issues may include issues in accessing the data, unpredictable data semantics and vendor lock-in.

The Bottom Line

Projects relating the migration be of any size entails careful planning. Evaluating its existing applications portfolio could help an organization understand the complexity, challenges and amount of effort required in migrating its databases to the cloud. Many tools assist application and database migrations, and each method differs in the amount of automation they provide and in the accurateness of the migrated system. Carrying out proofs of concept using these tools to understand their abilities will be constructive in the longer run for larger projects related to migration.

June 28, 2016