devopsy.kinsta.cloud

Author: Brian Wheeler

Top Reasons to Use Node.js

When considering the architecture of a new web app, it might be a good idea to include Node.js in the discussion as a key part of that architecture. Node.js could provide you with additional speed, can be used on numerous platforms and has an active community from which to get modules and other assistance.

What is Node.js?

Node.js is a server-side platform developed by Ryan Dahl in 2009. It was built on Chrome’s V8 JavaScript engine and designed with a non-blocking, event-driven I/O. It is also cross-platform, able to be run on Windows, Linux and OS X. Its non-blocking I/O is due to its use of asynchronous functionality, which keeps the server from waiting on data to be returned before being able to move on to other tasks.

Why Use It?

Given the architecture of Node.js, a number of advantages are provided both to developers and the resulting apps they develop. A few of these advantages include speed, programming language consistency, real-time apps developed easily and a large number of modules/packages for developers to use while creating new apps.

Speed

Node.js offers speed advantages in many cases. First, it uses the Google V8 engine, which is optimized for JavaScript speed. Also, its use of non-blocking asynchronous behavior can ease the time it takes for users to see actions as complete, since things can be going on in the background while the user moves on to other tasks.

One Language

JavaScript is listed as the most popular programming language. Source: Thenewstack

Using Node.js, front-end and back-end code can both be in JavaScript, so your developers don’t have to transition between languages as they move through the stack. Some architectures, such as the MEAN stack (MongoDB, Express.js, Angular.js, Node.js), are designed around this principle. In this case, the stack uses JavaScript for Express, Angular and Node. MongoDB uses BSON for database queries, which uses the same syntax as JSON (JavaScript Object Notation).

Seeing full-stack developer as the largest occupation and JavaScript as the top language among both back-end and front-end aligns with what we’ve been seeing in overall JavaScript adoption.

Source: Mikeal Rogers, Node,js Foundation Community Manager

For developers, this can help both with moving across the stack during development as well as help communication among developers, as they will be referencing the same language (JavaScript) while discussing programming decisions.

Real-Time Apps

Node.js is a great platform for creating real-time apps. With a module such as Sockiet.io, you can create real-time chats, games and more. Along with its non-blocking asynchronous architecture, real-time apps can be developed with plenty of speed and responsiveness.

Modules

The Node Package Manager (NPM) makes it easy to install packages that others have developed to perform any number of tasks. This helps to encourage code-sharing among developers and to reduce the amount of custom development that has to be done when one or more packages can take care of that coding for you.

Notable Examples

Some notable examples of Node.js use are Paypal and Uber, showing that it can, indeed, be useful for large projects.

Paypal

Node.js and an all Javascript development stack helped PayPal bring efficiencies in engineering and helped rethink and reboot product, design and operational thinking.

Source: Sameera Rao, Paypal Sr. Business Products Engineering Manager

Back in 2012, Paypal had a single app running all things, including UI, API and more. This included a lot of duplicated code that was copied and pasted from one app to another to make apps for different locations.

Using Node.js, the company was able to modularize and clean up its code and processes, which in turn sped the development of apps it needed at any given time.

According to Paypal’s engineering team, they were able to build the Node.js app with 33 percent fewer lines of code and 40 percent fewer files, and were able to build it twice as fast and with fewer people than before. Certainly, Node.js proved to be advantageous to their development.

Uber

Uber used Node.js from the beginning, and its distributed system was a perfect fit, since a lot of network requests are made. The non-blocking asynchronous I/O used by Node.js helped to ensure those requests could be made and handled quickly to provide the best service possible.

Conclusion

The benefits of Node.js have been proven. It helps speed development, can be used on numerous platforms and has an active community from which to get modules and other assistance. Include Node.js in your future conversations about website architecture.

— Brian Wheeler

January 4, 2017
12 Tips for Managing Your Multi-Cloud Environment
No single cloud vendor has the perfect solution for your unique business situation. The only way to maximize the benefits of cloud services to your business is by dealing with several cloud providers and monitoring all of them simultaneously. That’s the IT equivalent of singing an aria while standing on your head and scrambling eggs. (Not really, but it can seem that way.)

Here are 12 things to keep in mind when planning your company’s multi-cloud strategy:
- Map your entire network, and then identify where the cloud fits in it. Get a clear picture of the cloud’s role in your overall system-management strategy to avoid having to backfill gaps in the data needs of business managers and other customers. Different lines of business will be best served by different cloud providers.
- Work with vendors that respond quickly and positively to the unexpected. There will always be unforeseen situations when you’re stitching together multiple cloud services. Your partners should report regularly on the state of their services, enhancements, and new features.
- Automate low-level monitoring and maintenance tasks. Two primary reasons for adopting cloud services are cost savings and more efficient operations. One way to achieve both is by reducing the amount of human oversight required to keep your systems humming.
The potential complexity of multi-cloud management is shown in this flowchart encompassing the many elements that need to be stitched together precisely. Many of these low-level operations must be automated to optimize your strategy. Source: Carl P. Lehrmann, 451 Research
- Follow your users. Pay close attention to how the “consumers” of your apps and systems are actually putting the tools to use. They are the best indicators of which processes are the biggest productivity boosters. You can also learn about any new services the users discover that help them get their work done faster, more efficiently and/or less expensively. This lets you get a jump on the kinds of tools the users are most likely to need in the future.
- Double down on cost controls. You can’t control what you can’t see. Dealing with multiple vendors complicates the process for requisitioning new services. To avoid unnecessary impediments to quick implementation of useful new services, devise a purchasing process that is as flexible as the services themselves. However, not long after the service honeymoon period, evaluate whether each new service is proving to be worth the cost and whether there are better and/or less-expensive alternatives.
- Application life cycle continues to dictate cloud migration. Data centers may be shrinking, but they are definitely not disappearing. For reasons related to security, compliance, insurance, and any number of other factors, some systems are staying in house for the foreseeable future. You certainly want to avoid attempting to lift and shift an in-house app to a cloud service, particularly one that is only a couple years old and thus hasn’t yet paid for itself. Once an older system is ready for a reinvention, it’s easier to make a business case for going cloud-native.
- Take advantage of the power of pre-emptable virtual machines. Because they persist for no longer than 24 hours, pre-emptable VMs are a good choice for batch processing and fault-tolerant deployments, according to ComputerWeekly’s Matthew Cooter in a November 2016 article. Pre-emptable VMs are as much as 80 percent less expensive than Google’s regular instances, and their fixed price help companies plan and manage their costs.
- Adopt an integrated data center management system designed for virtual environments. This is the only way to ensure your server, storage, network, security, operations and applications teams are working as a single unit toward common goals, according to Jay Litkey, president and founder of cloud service provider Embotics, in a November 21, 2016, article in Virtual Strategy magazine. Any organization structured around traditional physical infrastructure boundaries is destined to struggle with the shift to hybrid and multi-cloud environments.
- Identify the apps in your organization that are best suited to cloud-native implementation. Traditional apps are monolithic, run as VMs, use a scale-up architecture and are generally more difficult to develop, deploy and maintain. By contrast, cloud-native apps are more modular and service-oriented, comprised of collections of containers and services, based on a scale-out architecture, and easy to automate, move, and scale.
Focus on the apps in your organization that will benefit most from a cloud-native approach, and wait on those that are better served by deployment on a traditional data-center platform. Source: TechTarget
- Match your apps to the services that play to their strengths. Big-name services such as VMware and AWS have bifurcated to meet the diverse needs of their customers. For example, VMware’s vSphere-based Enterprise Hybrid Cloud is intended for traditional apps, and the company’s Native Hybrid Cloud is designed for modular, microservice-based systems. Similarly, AWS’s Elastic Compute Cloud, Elastic Block Store, EFS file system and VPC private networks mimic in-house data centers, while the company also supports higher-level services such as NoSQL databases, BI processing, Hadoop-like clusters, message queues, push notification services, media transcoders and search engines.
- Automate policy across cloud environments. The only efficient approach to use of multiple cloud services is to have a single standard for policies that are applied automatically to each environment. The policies cover such areas as virtual servers, workloads, data storage, traffic flows, compliance/regulation, reporting and security. It’s also easier to apply updates and changes when you have a single configuration across different infrastructures because the changes propagate seamlessly from environment to environment.
- Vendor lock-in becomes a thing of the past. When you rely solely on one public cloud service, it can be difficult to migrate virtual machines and other resources to another service. Among the factors to consider are comparable read/write loads, data transfer needs and network latency. By relying instead on your custom combination of cloud services, all of which share policies and configurations, you find yourself managing the change instead of the change managing you.
— Brian Wheeler
December 5, 2016
Taming 2 Cloud App Performance Enemies

Whenever you hear the words “the network’s slow today,” or, even worse, “the network’s down,” you know somebody miscalculated. It may have been the person who decided to buy the wrong hardware, or maybe the one who signed up the wrong telecom provider.

Often, that slowdown or outage was the result of somebody underestimating server demand or overestimating the amount of redundancy built into the network setup. It’s always tricky finding the right level of redundancy that will keep users from waiting for their data without overspending on cloud capacity.

A common mistake IT managers make when switching from in-house infrastructure to cloud infrastructure is failing to take advantage of the flexible, affordable, redundant capacity offered by Amazon Web Services, Microsoft Azure and other cloud providers. That’s the conclusion of Yan Cui in a April 20, 2015, post on his “The Burning Monk” blog.

Cui points out that latency is inevitable because things go wrong—and usually, those things are beyond your control. That’s why you have to design your applications with latency in mind. Too often developers work inside a bubble, where all hardware and network parameters are geared to the tasks at hand. In the real world, hardware, networks and software break. If your app isn’t designed to accommodate such failures, you’re leaving your users in the lurch.

Get Real: Inject Latency in Your Test Environment

To model app performance in real-world settings, it isn’t enough to test servers and databases under anticipated loads. You have to take the extra step of mimicking the glitches that are as inevitable as they are unpredictable. Cui proposes injecting random latency delays on every request. He even provides a simple PostSharp attribute that automates the process.

Cui applies the concept of error-planning even further by introducing errors in the mechanism used to throttle requests to prevent bad guys from spamming their servers using such tools as Charles and Fiddler, or any of the spammers’ own bots. By making the errors visible in the dev environment, designers can anticipate them and engineer ways to handle the throttling errors “gracefully.” Once again, a PostSharp attribute is applied to randomly inject throttling errors; Cui states that the same approach can be used to plan for and respond automatically to session expiration, state out-of-sync, and other service-specific errors.

These and other error-detection and -correction methods fall under the umbrella of Design for Failure, a principle championed by Simon Wardley as a key characteristic of next-generation tech companies.

Simon Wardley’s Design for Failure contrasts the development approaches of traditional shops and those he identifies as “next generation.” Source: The Burning Monk

Cui explains how modeling latency and other errors within the development process has been applied by Netflix’s use of Chaos Monkey and Chaos Gorilla, as well as the mechanism Uber devised to allow its client applications to fail over seamlessly in response to a data-center outage.

Learn How to Combat Latency from the experts: Gamers

Nowhere does server performance matter more than in the world of online games, where one-microsecond hiccup can spell the difference between winner and loser, champion and chump. In an April 17 article on Venture Beat, Cedexis’s Pete Masden explains that a 500-millisecond network delay makes an online gamer twice as likely to walk away, while a 2-second delay in load time leads to abandonment rates as high as 87 percent.

Addressing latency begins by measuring it, according to Masden. That’s why we added native monitoring to Morpheus when we built the cloud management platform. Being able to see the status of all your organization’s apps and business services at a glance isn’t just a nice convenience—it’s a competitive advantage. To be able to glance the status of all your organization’s apps and business services, group databases, app servers, messaging queues and web servers by system dependency and impact to view an overall status as well as each app’s current state is crucial to knowing what’s happening in your organization.

One approach to cloud-server latency is to create separate instance, which is common for failover but less so for addressing regional network demand. Rather than the traditional active-passive model, however, Masden recommends an active-active approach when implementing multiple redundant instances to ensure all servers are up to date and optimized. Atop the multiple instances runs the global traffic analyzer to ensure no single instance becomes overly congested.

Is ‘Zero-Latency Computing’ Even Possible? Why not?

If you think network latency is a problem now, just wait until such technologies as 5G, the internet of things (IoT), virtual reality (VR) and robotics go mainstream. Equipment and bandwidth upgrades, and more sophisticated software-defined networking won’t be sufficient to address tomorrow’s application-performance needs. That’s the conclusion of Nitin Serro in a Sept. 29 post on the Channel Partners blog. Serro identifies three technologies as key to meeting future performance requirements: network functions virtualization infrastructures (NFVi), fog computing and zero-latency computing.

The applications made possible by 5G and related technologies come close to eliminating latency by shifting more intelligence directly to the network, not merely by providing faster speeds. Source: GSMA Intelligence, via The Unwired People

While NFVi and fog computing focus primarily on the network and data layers (compute, storage and network infrastructure), zero-latency computing targets the application layer, where information has to be exchanged in real time between diverse components, most of which link directly to users. According to Serro, nothing short of an entire re-invention of the network infrastructure—from backbone to edge—will prevent today’s bottlenecks from becoming tomorrow’s blockages.

Move the Network Intelligence Closer to the Data

The network-capacity challenge we now face is unprecedented, according to Silicon Republic’s John Kennedy, who writes in an Oct. 3 post that even the much-touted 5G network will come out of the gate hobbled by the need to accommodate its many predecessors. Rather than the “step change” in network speed that most analysts expect, 5G will deliver a tremendous boost to network intelligence. This may be the technology’s greatest asset: a smarter network means the data doesn’t have as far to travel.

From an application-design perspective, latency can be traced to three sources: the need to link to legacy infrastructure, the need to link to third parties via APIs and the innate unpredictability of the internet itself. Marqeta’s Dave Matter examines these situations from a payment-processing perspective in a Sept. 13 post on App Developer Magazine.

As with the network infrastructure itself, the solution to application latency is to move more of the intelligence into the app itself and then rely on open APIs to link data sources in real time in an environment that’s closer to users. Once again, the data and associated analytics tools have less distance to travel, and much less latency to deal with. Ultimately, the solution to latency—in hardware and in software—will be fewer, shorter trips.

— Brian Wheeler

October 10, 2016
Fog Computing Meets Cloud Computing, and the Forecast Is Sunny

Fog computing is more than just cloud computing meeting edge networks, but for practitioners more interested in results than in new buzzwords, the question is whether fog’s IoT promise deliver real-time data storage, processing and analysis.

One thing all new technologies share is the lack of a consensus on what exactly defines the technology. Ask some folks what “fog computing” is, and they’ll tell you it’s synonymous with edge networks. That’s the claim made in a July 29 post on Nanalyze, which traces fog computing concepts back to the days of mainframes and dumb terminals.

On the contrary, ReadWrite’s Ryan Matthew Pierson explained in an August 5, 2016 article the many ways that fog computing is distinguished from edge networks. The primary distinction, according to Pierson, is that edge networks connect sensors to programmable automation controllers (PACs) for data storage and processing, while fog computing requires that the sensor data be transferred to local gateway at the end of the cloud “chain.”

Pierson claims the fog computing model is more scalable than edge networks because each gateway has more power to “make more complex and dynamic decisions.” The gateway approach also accommodates a range of data at various flow rates. However, he said, this design creates more potential failure points.

Fog Computing = Edge Networks + Analytics?

As you might expect, there’s a third, middle-ground approach that defines fog computing as edge networks to which real-time analytics have been added at the node level. That’s how FogHorn Systems CEO David King explained the technology in a Sept. 13 article by CIO’s Stephen Lawson. FogHorn’s Lightning software platform is designed to deliver real-time analytics to IoT gateways, as well as to existing industrial products, including hard-wired controllers deeply embedded in equipment such as locomotives. Some of these embedded controllers are based on proprietary OSes that have been around for decades.

The key feature of Lightning’s design, according to King, is its close ties to cloud servers, which ensures that the data collected at the edge of the network is available to analytics tools that reside in the cloud. CIO’s Lawson quoted Machina Research analyst Aapo Markkanen as stating that linking IoT data sources to cloud analytics is the only way to get the “big picture” view required to ensure accurate analyses.

In an Aug. 19 interview with App Developer Magazine’s Richard Harris, King explains how fog computing addresses the shortcomings of previous distributed edge networks in terms of bandwidth use and the resulting “performance-killing application latency.” The FogHorn system is designed to move the “intelligence layer” closer to the machine data, resulting in faster and more cost-effective analysis. King claims the company’s software makes possible “real-time remote monitoring, asset optimization, proactive maintenance and operational intelligence applications.”

Fog computing adds a new layer that connects mobile, IoT (sensors, embedded controllers), and vehicle control systems to the depth and breadth of data resources in the cloud. Source: SpinalCom via SlideShare

Many of the features promised in fog computing systems—remote monitoring, asset optimization and proactive maintenance—are available today in best-in-class tools. You should be able to monitor SSH and Agent-based connectivity to apps running in private, public and hybrid clouds.

Fog Computing Delivers Analytics for Time-Critical Applications

The cloud’s centralized nature has never been a good match for the short response times required by data processing and analytics in IoT environments. Ben Dickson wrote in an Aug. 2 TechCrunch article that fog’s decentralized architecture moves the analysis tools and other resources closer to the edge, where the data is being collected and actions applied. Telemedicine, autonomous vehicle control and remote security systems are examples of time-critical applications requiring near-real-time analytics.

Another advantage of maintaining data at the edge of a widely distributed network is the privacy and security benefits of keeping sensitive data off the internet, where it would be subject to strict privacy regulations. Dickson presented the “fog layer” of the network as offering the compute, storage and network resources required to “mimic cloud capabilities.” By ingesting data locally, you can analyze it and convert it into action almost instantly.

The complete IoT model extends from the data source (human or machine) to the data center in the cloud, with much analytics-driven decision making occurring in the “fog layer.” Source: Moor Insights & Strategies, via Forbes

Dickson pointed out that the fog layer serves as a complement to cloud computing rather than as a replacement. In fact, fog computing couldn’t exist without all the data, tools and other resources now residing in the cloud. The larger data sets and deeper analyses being done in the cloud serve as the foundation for the fog layer’s quick-turnaround analytics. The fog’s capabilities are constantly being updated and enhanced based on the big-picture view the cloud affords.

Fog Computing Standards Take Shape

Today’s mobile networks simply can’t meet the needs of tomorrow’s data operations. Dropped connections, bandwidth bottlenecks and latency—the time it takes for data to travel from point to point on the network—are all too common. Upgrading the “infrastructure” at the edge of the network requires a concerted effort to ensure open access for diverse tools and resources—some of which we can’t even imagine, let alone prepare for.

Princeton University’s Edge Lab is taking the lead in bringing together tech vendors, organizations and customers of all stripes to create the Open Fog Consortium, which is intended to design an architecture for fog computing. The official announcement of the group was announced at the Internet of Things World Forum in Dubai Dec. 6, 2015; John Sullivan reported on the announcement in a Dec. 16 post on the Princeton University blog.

Open systems have proven their worth many times over, but an open architecture for IoT networks is even more important. Estimates are that by the end of 2018, there will be 22 billion connected devices generating an unending stream of unstructured and semistructured data, according to numbers from IDC research quoted by Forbes’ Gil Press in a Nov. 10, 2015 article.

The flood of data would overwhelm the bandwidth and network capacity organizations currently have, as Jelani Harper explains in a March 10 article on Data Informed. Only an open, decentralized model that keeps the bulk of data and its supporting resources at the edge of the network will prevent a “datapocalypse.”

For many network requirements, cloud computing and fog computing complement each other. Source: Cisco Systems

How much of a company’s data can be maintained in these edge-network nodes? Harper projects that as much as 90 percent of the data that will be collected by IoT devices will never reach the data center. Not only does this reduce strain on the network, it also speeds up “time to action” because analyses are performed where the data resides: on the edge.

The advent of fog computing can be seen as an expansion of the cloud via the creation of a “fog layer” in which many data assets and resources are reallocated. That layer is where IoT’s incredible data reach meets the cloud’s infinitely scalable compute, storage and big data analytics. What is fog, after all, but cloud you can feel?

— Brian Wheeler

September 27, 2016
How to Avoid Trouble when Deploying Apps to Multiple Clouds
Deployment is where Dev meets Ops. The glue that binds the two is orchestration, which places operations guidelines in the application itself. Here’s how to avoid trouble when deploying your apps to multiple clouds.

Things move faster when your infrastructure is the cloud. Applications go from concept to production in hours rather than weeks or months. Everything happens at the same time: development, testing, deployment, updates. Your tools have to be as quick and nimble as your virtual environments.On the open-source side of orchestration, two scripting tools dominate–Chef and Puppet. As Tom Nolle explained in a July 2016 article on TechTarget, Chef is considered the more “programmer-friendly” of two, while Puppet traces its roots back to the operations side. In practice, however, the imperative or prescriptive Chef model and the declarative Puppet model are “growing closer together,” according to Nolle.

Chef and Puppet: Two Paths to the Same Destination

With Chef, you create “recipes” and “cookbooks” in a programmatic manner; each deployment step is described separately, and the independent recipes are combined into a repeatable app deployment process. By contrast, Puppet has you build a model that describes, or “declares,” what the end state will look like. Puppet models are intended to simplify these deployment descriptions. The Puppet approach is a natural complement to application lifecycle management (ALM) because it accommodates a complete ALM function.Other orchestration tools that take a purely declarative approach are CFEngine and Juju, while Ansible and SaltStack support both declarative and imperative models. The Organization for the Advancement of Structured Information Standards has released the Topology and Orchestration Specification for Cloud Applications (TOSCA), a declarative-based model intended to support both the definition of end-state application deployment and specific modeling of virtual resources and pools.

DevOps as an ‘Easy Button’ for Cloud App Deployment

Today, the components of any application may reside in-house, in a public cloud or anywhere in between. Microservices, virtualization and similar innovations have turned the concept of “infrastructure” on its head. Now the infrastructure adapts to the needs of the applications and services running on it rather than the software adapting to the hardware it runs on. DZone’s Mark Jackson wrote in an Aug. 1 article that IT organizations are evolving from builders and managers of infrastructure to “security-driven cloud service brokers.”

The ultimate goal is to adopt a new approach: a hybrid-cloud stack model based on a programmable, policy-driven infrastructure. In addition to the stack’s infrastructure, management layers are hybrid cloud orchestration and cloud-native stacks that combine to support nearly any type of data-driven app, whether mobile, IoT, hyperscale or vertical use cases.

In a next-generation data center reference architecture, a hybrid-cloud stack accommodates deployment of apps whose components may exist in-house or in public clouds at any given time.

Take the Sting out of Deploying to Multiple Clouds

Organizations that rely on one single cloud service are the exception rather than the rule in today’s IT operations. Flexibility is the name of the game as companies deploy to public, private and hybrid clouds using a range of services: OpenStack, AWS, Google Cloud Platform, HPE and Microsoft Azure, among others. Cloud Technology Partners’ David Linthicum wrote in a March 28 article that deploying to multiple clouds presents three challenges:

• Taking advantage of each service’s native capabilities using generic, relatively new DevOps tools and processes;

• Ensuring your DevOps tools keep pace with the rapid changes being made to the platforms; and

• Maintaining security and governance when each service’s rules for these vital operations are platform-specific.When deploying to multiple clouds, that same code set—usually teamed with data—has to be marked for the target platform at the point of staging. As the app runs through the platform-specific testing, it is checked by the tool or process to verify it will work with the platform’s native features. While this feature-checking can be automated to a degree, it often entails manual rechecks as potential problems are identified and corrected.

Deploying apps to multiple clouds requires platform-specific testing that may not be possible to automate completely.

Once the platform-specific testing is completed, the actual deployment to production on each platform can be automated by placing it on a machine instance in a public or private cloud, and then spinning up the “continuous operations” process. The five components of the process are monitoring, management, resource governance, service governance/service catalog and security:
- Monitoring the application during execution reports any stability or performance glitches when preset thresholds for various subsystems are exceeded.
- Management acts in response to alerts of potential problems by interpreting the monitoring data, usually via a cloud management platform for resource management, as well as primary application and subsystem components.
- Resource governance abstracts the performance data received from individual cloud services through a single console, or dashboard, linked to all the target platforms. User policies can be set once and applied to the various clouds. This is the approach taken by the Happy Apps performance and uptime monitoring service, which provides status and real-time visibility across all your apps and business services.
- Service governance lets you create policies for the use of APIs and services that serve as common services for all the clouds you use, allowing apps to leverage the service on whichever cloud they’re deployed to. A service catalog is used to track infrastructure-level and application-level services in all your clouds as well as on premises.
- Security attempts to minimize complexity in a multicloud model by reducing the number of security services involved, which usually include user authentication, access management and breach detection.
The nature of individual cloud services ensures that deploying to multiple clouds will remain complicated into the future regardless of your preferred set of deployment tools. One way to minimize the complexity is to plan your DevOps strategy from the start with a multicloud environment in mind. There will be times when getting an application to pass data seamlessly across many distinct cloud platforms will require a good deal of manual coding.Adopting a service orientation can mitigate the costs associated with deploying to multiple clouds by sharing services among apps. This way, a single application could run hundreds of remote services from any number of clouds. The service and resource tracking may be daunting at first, but microservice apps prove their business value by making developers more productive and opening doors to new sources of revenue.
September 12, 2016
Cloud Computing in 2016 and Beyond
Few activities are more perilous than predicting the future of the tech industry. That goes double for cloud computing forecasts. Yet, with thoughtful deliberation and the appropriate amount of caution, you can get a fairly accurate read on what’s to come in the increasingly competitive cloud services market.

Here’s a quick look at several trends evident among leading cloud vendors and technologies through the first half of 2016, along with a look ahead at what awaits us for the second half of the year, and on through the next decade.

Google Makes a Major Cloud Play

The company synonymous with search isn’t used to looking up at the leaders in its markets, yet according to figures compiled by Synergy Research Group, Google’s 4 percent share in the cloud services market in fourth quarter 2015 makes it an also-ran behind Amazon (31 percent), Microsoft (9 percent) and IBM (7 percent). Market Realist’s Puneet Sikka reports on the share numbers in a May 31 article.

Google’s first-quarter 2016 earnings call highlighted three recent actions by the company that signal its intention to grow its cloud business:
1. Google has gained experience through its internal cloud operations that it is now ready to share with its customers.
2. The company is integrating its artificial intelligence (AI) and machine-learning expertise with its cloud services for enterprises.
3. VMWare co-founder Diane Greene was appointed to lead Google’s cloud business in late 2015.
AWS, Microsoft Azure Continue to Raise the Cloud Services Bar

Even with its depth of technical expertise and seemingly endless resources, Google faces a formidable challenge in taking share away from Amazon and Microsoft in the skyrocketing cloud services market. Among the 722 new features added to AWS in 2015 was the Aurora relational database that Amazon claims is its fastest ever, according to Sikka. The Amazon QuickSight (AWS data analytics and visualization) tool has helped AWS sustain quarterly revenue growth near or above 50 percent for five consecutive periods.

Amazon Web Services racked up five straight quarters of revenue growth at or exceeding 50 percent through first-quarter 2016. Source: Market Realist

Meanwhile, Microsoft’s 2015 revenue from Azure, Office 365 and Dynamics CRM increased a whopping 140 percent year over year. Microsoft’s continued ascendance, as well as that of other cloud service providers, is attributable in part to a trend highlighted in RightScale’s 2016 State of the Cloud Survey: Companies using the cloud run apps on an average of 1.5 public clouds and are experimenting with another 1.5 public clouds. They’re also using 1.7 private clouds on average, and they’re experimenting with 1.3 private clouds.

Companies running apps in the cloud use an average of three different public and private services. Source: RightScale

The RightScale survey notes that while AWS’s share of the public cloud market was flat at 57 percent in 2015, the number of workloads per customer increased, which helps account for the service’s 69 percent jump in revenue for the year over 2014’s figures. Venture Beat’s Jordan Novet digs into AWS’s 2015 financials in a Jan. 28 article.

Vendors Called Out for ‘Cloudwashing’

The rosy outlook for cloud services through this year and beyond is echoed in Gartner’s Cloud Computing Primer for 2016 (Gartner account required to access). Yet, cost and security continue to be two primary obstacles to cloud adoption. One other impediment to success identified by Gartner researchers is the tendency of cloud vendors to make unsubstantiated claims about their offerings. Gartner refers to this as “cloudwashing.”

The industry has matured to a point where most obstacles to cloud adoption are non-technical, according to Gartner: adapting the organization’s culture, skills and management to a cloud framework; and winning the hearts and minds of end users, who often are hesitant to overcome their innate fear and mistrust of the cloud model. (Yes, your sensitive data is indeed safer on a cloud server than it is on your local hard drive.)

Much of the confusion that Gartner lumps into the “cloudwashing” category relates to a lack of clear definitions of cloud terms such as “infrastructure as a service” and “platform as a service”—two categories whose distinctions will continue to blur.

They’re All ‘Hybrid’ Clouds—or Soon Will Be

Also becoming less discernable are differences between public and private clouds, both of which are morphing into versions of hybrid cloud, according to Gartner. In fact, it’s not unusual for enterprises to have multiple cloud strategies being implemented in parallel. An overarching aspect of this trend is that the fundamental role of IT is changing from a service provider to a service broker.

The Gartner survey points out that IT pros now wear several hats and switch between roles in the course of a given day, from provider to broker to consumer. The hybrid model itself is evolving from “public/private” to “fast mode/safe mode,” according to Gartner. Confusion surrounding the term “hybrid” as it relates to cloud services is likely to continue with the rise of mobile devices, where “hybrid” refers to apps that are both web-based and native to the mobile platform.

One way to ensure cloud customers and vendors are speaking the same language is to focus on cloud outcomes rather than the means of achieving those outcomes. Customers need to define what results they expect from their cloud efforts—financial effectiveness, tapping the expertise of cloud providers, delivering immediate results to business consumers and making overall IT operations more efficient, for example.

What’s in Store for 2017 and Beyond?

The popular buzzwords resurface when you scan the prognostications of the cloud experts: microservices, containers, open source. Yet customers—particularly those in enterprises—have the same concerns about cloud computing now that they had when the technology first emerged a decade ago: security, risk management, governance, compliance and disaster recovery. How much progress is being made on these fronts?

Perhaps those areas will be addressed not by traditional IT providers but instead by web-centric services and other non-IT companies that develop and market cloud solutions for their vertical. You can’t blame non-cloud service providers from salivating at the prospect of grabbing a share of the booming cloud market. According to Forbes’ 2016 Roundup of Market Estimates, cloud forecasts are dominated by a single preposition: “Up.”

CLICK TO TWEET
- Public cloud IaaS hardware and infrastructure software revenue up to $38 billion in 2016 from $25 billion in 2015 and jumping to $173 billion by 2026 (Statista)
- Worldwide public cloud revenues up to $117 billion in 2017 from $80 billion in 2015 and $167 billion in 2020 (TBR)
- Cloud’s share of annual IT budgets up from 29.8 percent as of January 2016 to 43 percent by January 2018 (IDC)
No matter your organization’s current state of cloud adoption—whether dipping your toe, or in with both feet—you’ll future-proof your cloud investment by relying on vendor-agnostic application-management services. There’s no better way to take advantage of what the cloud has to offer now while ensuring you’re positioned to benefit from tomorrow’s cloud enhancements.
August 24, 2016
Cloud Security Tips: How to Prevent Hackers from Breaching Your Cloud
Recent data system breaches at Oracle and Sage reminds us all that the continuous threat of criminal hacking of cloud security systems is not abating. Rather, it’s becoming routine. Business managers, lawmakers and computing professionals must understand the motivation behind this activity if they want to protect business interests and thwart attacks effectively. Perhaps the biggest challenge is that the hacking community is a diverse and complex universe—a large variety of skilled players and several motivators. Only by understanding the motives of criminal security hackers is it possible to profile computer crimes. With solid profiles and some basic cloud security tips in hand, security professionals can better predict future activity and install the appropriate safeguards.

Most security professionals are likely to spend much more time analyzing the technical and mechanical aspects of cybercrime than the social and psychological dimensions. Of course, it’s critically important to dissect malware, examine hacker tools and analyze their code. However, if we want to understand the nature of the cyberthreat, security professionals need to act more like criminal investigators. We no longer live in a world of mere glory-seekers and script kiddies. Some very serious thugs are now lurking in virtually every sector. So, it’s critically important to understand their motives and signatures, since these point to their targets and reveal their methods of operation.

As you consider your business context, it’s important to frequently ask yourself this question: What exactly are the means, motives and opportunities for potential criminal hackers of my business computing systems? Getting a solid answer to this question is the key to identifying your most vulnerable assets and developing a security plan.

The Sage Breach

Earlier this month, Sage Inc. announced that hackers, using an internal login, had gained access to sensitive customer information including the names, addresses and bank account information of a few hundred U.K. customers. It’s undetermined at this point whether the hackers gained access to everything needed to cash in on their attack. But the Sage attack is just the latest in a wave of high-profile hackings in recent months, with victims ranging from enterprise IT vendor Oracle to cryptocurrency Bitcoin and even The College Board, creators of the ACT exam.

According to many IT and computing system analysts, the top three hacker motives are financial, corporate espionage and political activism. In the remainder of this article, we look closely at the financial motive, and then we help you consider the best approaches to securing your cloud computing assets.

Financial System Hackers

You’re probably most familiar with this type of hacker, since they cause the most damage and are often featured in the news. The motive here is pretty obvious: make money the easy way, by stealing it. Financial system security hackers range in size from a few lone actors to large cybercrime organizations—often with the backing of conventional criminal organizations. Collectively, these thieves are responsible for extracting billions of dollars from consumers and businesses each year.

These threats go well beyond the hobbyist community to a very high level of sophistication. All criminal attackers immerse themselves in a complex underground economy: a vast black market in which participants buy and sell toolkits, zero-day exploit code and malware botnet services. Vast quantities of private data and intellectual property are up for sale—highly valuable data that has been stolen from victims. A recent market trend is the sale of web exploit kits such as Blackhole, Nuclear Pack and Phoenix—which they use to automate drive-by download attacks.

Some financial system hackers are opportunistic and focus on small businesses and consumers. Larger operations go to great lengths to analyze large enterprises and specialize in one or two industry verticals. In a recent attack on the banking and credit card industry, a very organized group was able to pull off a global heist of $45 million in total from an ATM—with an extreme degree of synchronization. These secondary attacks were feasible because of a previously undetected breach of some bank networks and a payment processor company.

The Next Wake of Hackers

Malicious hacker attacks are quite common nowadays, and often have tragic and highly disruptive outcomes. And these attacks are also inevitable, as more internet users utilize cloud computing and storage. This raises more concerns about combating the effects of hacking, and it will become increasingly critical in the future. There is ongoing debate as to whether basic cloud security tips are enough to prevent hacking threats. After years of extensive industry debate, it’s been found to be the same problem in a different location. So, if businesses can build reliable security and recovery methods, then cloud computing can be a serious consideration. Most importantly, the freedom, accessibility and collaboration that is available through cloud computing can far outweigh and mitigate the risks to your data security.

Many cloud computing users assume their data is held safe by the security measures of their cloud vendor. But, hackers use code-cracking algorithms and brute force attacks to acquire passwords, and they can also access data transmissions that lack proper encryption.

Ask yourself this question: Do you have solid infrastructure, processes and procedures to ensure reliable, high-security backups of your sensitive and business-critical data? If you can’t answer this question with confidence, then we invite you to read on a bit further as we consider various aspects of a top-tier cloud backup service.

Your cloud backup service should process all data through encryption to ensure that it’s entirely unreadable by unauthorized users. It should only be possible to decrypt your data when you decide to retrieve it. Minimally, this means that data transmission should be done only through the SSL protocol and that strong passwords are necessary for information access and decoding.

No system is hacker-proof, but the greatest benefit of cloud a backup service is the high-degree of readiness for recovery from a hacking event. Companies that specialize in cloud backup services reduce threats to your data by enabling full recovery of all business-critical data to its original state in just a matter of clicks. These backup companies replicate your cloud data and safeguard it in a separate cloud so that the likelihood of data loss from natural disasters and other threats remains infinitesimally small.

Modern Cloud Security Tips

Rock-solid facilities. Today, the best cloud application management solutions have standard built-in backups. Look for a provider that has robust, extreme-security data centers utilizing precise electronic surveillance and multifactor access control systems. The design of its environmental systems is important, also, to minimize the impact of any disruptions to operations. What’s more, multiple geographic locations and extensive redundancy add up to a high degree of resiliency against virtually all failure types, including natural disasters.

Protection from the bad guys. The best providers not only provide super-strong physical protection for your backup data, but they also lock everything down with extensive network and security monitoring systems. Their systems include essential security measures such as distributed denial of service (DDoS) protection and password brute-force detection on all accounts. Additional security measures can include:
- Secure access and data transfer – all data access and transfers go through secure HTTP access using SSL.
- Unique users – identity and access management features to allow you to control the level of access.
- Encrypted data storage – encrypt your backup data and objects using Advanced Encryption Standard (AES) 256.
- Security logs – extensive, verbose logs of all activity for all users of an account.
- Native Support – Native support for multiple platforms and systems including MongoDB, MySQL and Linux/Unix/Windows files.
August 19, 2016
4 n00b MySQL Mistakes Every Programmer Makes

Even experienced programmers can sometimes be thrown for an infinite loop–or other code failure–with these common MySQL mistakes.

When a database application goes belly up, the cause often can be traced to sloppy coding. Every now and then, the reason for a misbehaving app is an idiosyncrasy in the platform itself. Here’s how to prevent your database from tripping over one of these common MySQL mistakes.

Handling Numeric Value Overflows

In an Aug. 11, 2014, article in the Database Journal, Rob Gravelle describes a “gotcha” (not a bug) in the way MySQL handles numeric value overflows. When a value supplied by an automated script or application is outside the range of a column data type, MySQL truncates the value to an entry within the acceptable range.

Generally, a database system will respond to an invalid value by generating an error instructing the script to proceed or abort, or it will substitute the invalid error with its best guess as to which valid entry was intended. Of course, truncating or substituting the entered value is almost certainly going to introduce an error into the table. Gravelle explains how to override MySQL’s default handling of overflow conditions to ensure it generates an error, which is the standard response to invalid entries in most other databases.

Misuse of Primary Keys

According to Justin James on the Tech Republic site, the most common database programming no-no is the misuse of primary keys.

James insists that primary keys should have nothing at all to do with the application data in a row. If they’re not system values managed by the system, you’re likely to encounter problems when you change the underlying data or migrate the data to another system.

Using Stored Procedures as a Crutch

Another frequent cause of program problems is overuse of stored procedures, which James describes as a “maintenance disaster.” There’s no easy way to determine which applications are using a particular stored procedure, so you end up writing a new one when you make a significant change to an app rather than adapting an existing stored procedure. Instead, James recommends that you use advanced object-relational mappers (ORMs).

Not every developer is sold on ORMs, however. On his Experimental Thoughts blog, Jeff Davis explains why he shies away from ORMs. Because ORMs add more lines of code between the application and the data, they invite more semantic errors. Davis points out that debugging in SQL is simpler when you can query the database as if you were an application. The more lines of code between the application error and the database, the more difficult it is to find the glitch.

Overuse of BIGINT

One of the common database errors identified by Thomas Larock on the SQL Rockstar site is playing it safe by overusing the BIGINT data type. If you’re certain no value in a column will exceed 100,000, there’s no need to use the 8-byte BIGINT data type when the 4-byte INT data type will suffice. You may not think a mere 4 bytes is significant, but what if the table ends up with 2 million rows? Then your app is wasting 7.8MB of storage. Similarly, if you know you won’t need calendar dates before the year 1900 or after 2079, using SMALLDATETIME will make our app much more efficient.

Misuse of GUIDs

On the SQL Skills site, Kimberly Tripp highlights another common database design error: use of non-sequential globally unique identifiers, or GUIDs. In addition to creating fragmentation in the base table, non-sequential GUIDs are four times wider than an INT-based identity.

Is your app’s failure to launch due to poor typing skills?

Maybe the best way to start your hunt for a coding error is via a printout rather than with a debugger. That’s the advice of fpweb.net’s tutorial on troubleshooting SQL errors. The problem could be due to a missing single quote mark, or misuse of double quotes inside a string. If you get the “No value given for one or more required parameters” error message, make sure your column and table names are spelled correctly

Likewise, if the error states “Data type mismatch in criteria expression,” you may have inserted letters or symbols in a column set for numeric values only (or vice versa). The FromDual site provides a complete list of MySQL error codes and messages, including explanations for many of the codes, possible sources for the errors, and instructions for correcting many of the errors.

The FromDual site explains how to find sources of information about MySQL error messages.

For many of the MySQL error messages, the FromDual index provides an explanation, reasons for the error message’s appearance, and potential fixes.

With a little work and a lot of sticky notes, these common MySQL mistakes are fairly easy fixes. There’s nothing worse than looking like a nOObie programmer and damaging the integrity of your application. So stay sharp, and take a second look at your MySQL queries.

August 11, 2016
What is JSON? How Do I Use It? Does It Beat BSON?

TL;DR: If you’ve ever wondered about JSON—what it is, how to use it or what it has to do with BSON—then you’ve come to the right place. In this article I’ll explore what these acronyms stand for and what these formats do in the world of programming and databases.

What is JSON?

JSON stands for JavaScript Object Notation. Basically, it uses syntax derived from JavaScript to represent data in a way that is readable to both humans and machines. For instance, an example used on the JSON website describes a glossary in JavaScript Object Notation, as shown in the figure below.

How to Use JSON

As you can see, a hierarchy is created that encloses each level within a set of curly braces ({}). At each level, you have name/value pairs, such as “title”: “example glossary”. In the same way that JavaScript objects work, the value part of the pair can open yet another level by starting a new set of curly braces. This creates a well-structured layout of the data that can be scanned easily for information by machines, as well as be looked at by humans to find what is needed.

This notation allows a number of data types to be represented—numbers, strings and Booleans, hashes (enclosed by {}) and arrays (enclosed by []) all can be used in this format.

Once data is represented in this notation, a JSON parser can be used in most any programming language to turn this notation into the native data types for that language, thus allowing programmers to access the data using that language’s natural syntax. Since this notation is so prevalent, most programmers have a great deal of experience in storing and retrieving this type of data.

Does It Get Used in Databases?

Document databases typically store data in JSON format or something based upon that format. Rather than the tables and rows used by a relational database, a document database will store and query information through the use of the JSON format.

These databases refer to each set of JSON data as a document. Since the structure is hierarchical, each document can have subdocuments on any of its keys. This setup makes it very easy for a programmer familiar with the language to both store and query the data within the database.

What is BSON?

BSON stands for Binary JSON. This format is specifically used by MongoDB for its document database. The idea behind it is to provide the utility of the JSON format, but also be able to use that format with the speed that comes with a binary format.

JSON vs. BSON

While JSON simply represents a data structure, MongoDB can use BSON to build database indexes on keys at any level, thus helping improve speed in that area. MongoDB also provides a specification called GridFS that can be used in conjunction with BSON to store data over BSON’s 16MB document size limit. This can be useful to store large files, as it divides them into “chunks” to allow the storage to work properly in the database.

For the most part, JSON and BSON are very similar, but BSON is can provide an extra punch of speed as well. Both are certainly very handy ways of storing and retrieving data for applications.

June 10, 2016
Will MariaDB Soon Replace MySQL?

MariaDB is a database growing in popularity. Its lead developer is Michael “Monty” Widenius, who was part of the team that created MySQL. When MySQL was acquired by Oracle, Widenius and some of the other original MySQL developers decided to launch MariaDB as a fork of MySQL and keep it as open source to mitigate any issues around Oracle’s ownership of MySQL.

In 2009, Sun Microsystems acquired MySQL. Sun’s subsequent acquisition by Oracle brought about a movement headed by Widenius to “save MySQL.” Widenius and many others in the open source community felt Oracle’s ownership might be a conflict of interest, since Oracle already had a competing closed source commercial database. They believed Oracle would be slow to further develop the MySQL database, given the potentially greater focus on the commercial Oracle database.

In response to this, Widenius began working on MariaDB and helped form the MariaDB Foundation, which uses a community governance model. Because of the separation between the open source and commercial sides of the foundation, its governance generally is seen by the open source community as positive. This is in contrast to Oracle, which has both its commercial Oracle database and the open source MySQL both under the same governance. Often this is perceived as a conflict of interest, especially when it comes to keeping MySQL up to date.

MariaDB vs. MySQL

While MariaDB isn’t yet replacing MySQL, it has brought about good competition between the two, which can be good for innovation. With its backing of Oracle, MySQL may reign supreme for at least the near future; however, technically speaking, MariaDB could replace MySQL if a company decides it wants to make the switch.

The developers designed it as a drop-in replacement of MySQL. Since it was forked from MySQL, basically all of the structures MariaDB uses are the same. This means switching from MySQL almost surely won’t require a company to alter its application in any way, since the data and data structures won’t need to be adjusted.

In fact, the command-line tools such as mysqladmin, mysqldump, the daemon and others all keep their original names, so MariaDB is very much a drop-in replacement for database administrators as well. Indeed, many organizations have already either switched to it or simply began using it on its own: It replaced MySQL as the default database in Fedora and Red Hat. It is also used by Booking.com, CraigsList and Wikipedia, among others.

What Else Does MariaDB Offer?

Community improvements to MariaDB have been made by notable organizations such as Facebook, Twitter and Google. For example, Google contributed data-at-rest encryption to release 10.1.3, which is quite helpful given the security issues that can arise quickly in this ever-changing environment. In addition, it offers performance improvements such as subquery optimization and faster joins. It also has added row-based replication.

While MariaDB may not replace MySQL immediately, it is worth watching: It is gaining traction as a potentially serious contender in the marketplace.

June 8, 2016