devopsy.kinsta.cloud

Tag: virtualization

Database Virtualization – Driving the Next Big Transformation

Data is both the most important asset and the biggest challenge for most corporations. It impacts every aspect of the business from financial considerations to AI/ML training. Data is used by development and DevOps teams, engineering teams, quality assurance, business intelligence teams and database administrators. Despite its central role in operations, traditional forms of database delivery remain complex, costly and time-consuming.

In many cases, enterprise data is growing exponentially, increasing management challenges and driving up the cost of data storage. Duplicating data to each new environment as use cases arise is overwhelming DBA teams and incurring a significant storage footprint. This has resulted in data becoming a significant bottleneck for DevOps teams, highlighting the importance of finding a quick and automated solution for the replication process across non-production environments.

What’s Changed?

In the past, development teams would work with the same dataset for upwards of six months. This left DBA teams with plenty of time to meet all database delivery needs. However, as technology became more sophisticated and consumers grew more demanding, the data delivery life cycle was cut down dramatically. Today, with agile development, new features may be added every few days, if not hours. Coupled with QA teams’ requirement for testing data continuously, AI/ML teams requiring millions of data points for accurate training, and data analysts constantly needing fresh data for organizational growth, this means DBA teams are flooded with demands they are incapable of managing.

What Now?

Database virtualization is an automated and easy solution to the dataset bottleneck. By simplifying the creation and distribution of database replicas, DevOps teams will be able to create fully functioning virtual environments via a self-service process–allowing DBA teams to focus on more critical duties relating to production databases.

Speed Up

The first—and most obvious—advantage of this approach is the increased speed at which data is delivered. To keep DevOps teams productive and avoid the use of old, stale data, new environments need to be produced on demand. By integrating the production of new environments into the CI/CD pipeline, enterprises will be able to automate the process and provide DevOps teams with full autonomy. By cutting out the dependency on the DBA, teams will be able to access the data they need, when they need it.

Scale

Agility is fundamental to the success of database management. Each team requires an independent environment, unaffected by other use cases. The creation and supply of virtual databases allow for quick provisioning. Once created, these databases look identical to the original physical database and function completely independently of each other. Any changes made to a vDB remain unique to the environment it’s in.

Save

As the need for data grows, so does the cost of storage. Creating a physical copy for each team and use case results in a significant storage footprint. This means that these datasets come with a hefty cost. However, the creation of virtual databases led to a dramatic decrease in storage needs. Where traditionally, physical datasets can weigh upward of several TB, vDBs only weigh a few hundred megabytes. Further, virtualization solutions also compress the weight of physical copies – such as the golden copy – actively decreasing the weight and cost of storage across all fields.

Secure the Database

Even after the database delivery problem has been solved, enterprises are faced with another major issue – security. With so many different teams accessing data in virtual and low-level testing environments, the risk of ransomware and other cyberattacks increases dramatically. Generally, the investment in securing non-production environments is significantly weaker than in production.

A common approach to address this issue is masking sensitive data so it is rendered useless to outside sources. However, traditional masking requires a data expert to manually sift through all existing data and implement data masking algorithms on any sensitive information found. This time-consuming solution creates a new data bottleneck. By automating the process and applying it to the golden copy, corporations can be confident that all vDBs created are completely secure and privacy compliant.

Looking Ahead

Database virtualization is the future of database replica management. Virtual databases are delivered in a matter of minutes, significantly accelerating the speed of delivery. These environments are easy to use, independent, and can be maintained via a self-service portal. The ongoing sync between the original database and the vDB ensures continuous access to the latest data without the need for timely or expensive processes.

Effective database virtualization will allow well-established enterprises to more easily compete against those agile startups that face significantly fewer barriers to entry.

Image Source: joshua-sortino-LqKhnDzSF-8-unsplash

April 4, 2023
VMware Backup Guide for Hyper-V Admins

At first glance, the Hyper-V and VMware virtual environments may seem similar, but upon closer inspection, a number of important differences between these two virtualization platforms become apparent. Initially, performing administration for VMware vSphere could present some challenges for administrators who are familiar with Hyper-V. There are several reasons a business might be switching from Hyper-V to VMware. The company could be deploying a multi-hypervisor environment – e.g., using Hyper-V for Windows-based virtual machines (VMs) while introducing VMware vSphere for VMs running Linux, Free BSD, Solaris, etc. This blog post is intended to help Hyper-V administrators understand the differences and similarities between these two major virtualization platforms, making the transition to VMware easier.

Comparing Hypervisor Design

The Hyper-V Role for Servers and Workstations

Hyper-V users can set up the Hyper-V role not only on the Windows Server operating system (OS) but also on desktop versions of Windows, such as Windows 7, 8 or 10. The installation is similar for both server and desktop solutions. Hyper-V is classified as a type 1 hypervisor, meaning that it is run directly on hardware. There is also a lighter-weight alternative to installing the Hyper-V role on Windows Server: Microsoft Hyper-V Server. Hyper-V Server is a standalone product that is installed on bare metal.

VMware Workstation for Workstations and VMware ESXi for Servers

VMware provides VMware Workstation for installation on desktops with Windows and Linux-based operating systems. VMware Workstation is a type 2 hypervisor. A type 2 hypervisor is installed on the underlying OS. As you can see, both Hyper-V and VMware support installation on users’ PCs. If you want to install a type 1 hypervisor on bare metal, you should use VMware ESXi Server. VMs running on VMware Workstation can be migrated to ESXi Server with the VMware vCenter Converter.

Management Tools

vCenter as the Analog to SCVMM

System Center Virtual Machine Manager (SCVMM) is a special universal product created by Microsoft for centralized virtualization management of Hyper-V hosts, clusters, virtual networks and storage. SCVMM is useful for the management of complex multi-site and multi-domain environments.

Hyper-V Manager is a common tool that is available by default upon installation of the Hyper-V role. With Hyper-V Manager, you can connect to an available Hyper-V host. You can manage the host, the VMs residing on that host, the virtual devices of those VMs and VM replication. You can also create VM checkpoints, start or stop VMs and connect to VMs in order to interact with the user interface of a guest VM.

Failover Cluster Manager is a tool for managing failover clusters and VMs within a cluster. With this tool, you can create a failover cluster. Failover Cluster Manager is not a substitute for Hyper-V Manager; you still need the latter to manage non-clustered VMs, Hyper-V hosts, virtual switches, etc.

The equivalent of SCVMM in VMware vSphere is vCenter Server, which can perform the tasks that Failover Cluster Manager and Hyper-V Manager handle in Hyper-V environments. With vCenter Server, you can manage almost all components of vSphere – hosts, clusters, virtual networks (including standard vSwitches and distributed vSwitches), datastores, VM virtual devices, etc. vCenter is installed on a virtual machine, and the vCenter web interface can then be accessed through a browser. You can use vSphere Web Client (which comes in HTML5-based or Flash-based versions) to access vCenter via a web browser. To do so, simply enter the IP address of your vCenter server in the address bar of your browser.

If you have a standalone ESXi host, you can similarly access that host using a browser with the vSphere client. Enter the IP address of your ESXi server in the address bar of your browser. Note that you cannot create data centers, configure distributed virtual switches or perform VM migrations when accessing a standalone ESXi host with the vSphere client.

Below, you can see vCenter v.6.5 accessed via browser with vSphere Web Client. If you want to access the VM’s user interface, click Launch Web Console.

A new tab is opened and you can interact with the guest OS.

Another way to access the user interface of a VM is using VMware Workstation. For connection to a VM running on the ESXi host from VMware Workstation, click File > Connect to Server.

Input the IP address of vCenter or ESXi Server, as well as the user name and password required to access that server. Click Connect.

In the left pane of the VMware Workstation interface, you can see a list of VMs residing on the ESXi host to which you are connected. If you connected to vCenter, you would see the objects such as Datacenters, ESXi hosts and VMs that are managed by vCenter Server.

PowerShell Vs. PowerCLI

With the Hyper-V platform, the alternative to using a GUI for management of the virtual environment is using PowerShell. PowerShell is command-line shell that is available on Windows-based systems, allowing users to manage system components. PowerShell provides automation using combinations of cmdlets and scripts.

In VMware vSphere, you should use PowerCLI as the alternative to a GUI. PowerCLI is a scripting tool built on PowerShell. It extends PowerShell and includes a set of more than 600 cmdlets. You can automate installation and configuration of any features, virtual machines, system patching, etc.

Clusters

VMware HA and DRS Clusters Vs. Hyper-V Failover Cluster With PRO

The Hyper-V virtual environment offers failover cluster functionality. A failover cluster provides automatic failover of VMs from failed hosts to healthy hosts within a cluster. Upon host failure, VMs from the failed host are restarted on other Hyper-V hosts in the cluster. The Performance and Resource Optimization (PRO) feature collects data about resource usage in the virtual environment and generates PRO tips, recommending migration or reconfiguration of VMs for more rational usage of hardware resources. Since Windows Server 2012, PRO was replaced with Dynamic Optimization.

There are two types of clusters commonly used in VMware vSphere – High Availability (HA) clusters and Distributed Resource Scheduler (DRS) clusters. The HA cluster provides high availability of VMs and carries out automatic VM failover in case of host failure within a cluster. The VMs that were running on the failed ESXi host are restarted on healthy ESXi hosts. The DRS cluster is used for load balancing. vCenter server monitors the usage of hardware resources such as CPU and memory for VMs and ESXi hosts within a cluster. If, after calculations and analysis of these metrics, the DRS decides that some of VMs can migrate from one ESXi host to another (e.g., from an overloaded host to a host with free resources that is almost idle) for more rational resource usage, then migration recommendations are generated. These recommendations can be applied manually or automatically depending on your DRS configuration.

Power Optimization Vs. Distributed Power Management

Power Optimization is a feature of Dynamic Optimization in Hyper-V. Power Optimization allows the migration of VMs from Hyper-V hosts with low workloads to other hosts that have enough resources within a cluster. The unused hosts are then powered off in order to save energy. The powered-off hosts are powered back on as soon as their hardware resources are needed for the cluster.

Distributed Power Management (DPM) is a component of the DRS cluster in the VMware vSphere virtual environment. Similarly to Dynamic Optimization, the DPM executes VM migration from less loaded ESXi hosts to ESXi hosts that have sufficient resources to run the migrated VMs. The freed-up ESXi hosts are powered off (put into a standby state) to save power. When load is increased and the cluster needs more hardware resources again, those ESXi hosts are powered back on.

Fault Tolerance in VMware HA Clusters

VMware also provides a useful feature for HA clusters: Fault tolerance. If you use an HA cluster, in the event of ESXi host failure, there is a period of VM downtime between the VM’s failure and the VM being loaded on another healthy host. Using the fault tolerance feature in the HA cluster helps avoid any downtime. This is because a second VM (ghost replica), which is an exact copy of the VM protected with fault tolerance, is already running on another ESXi host within the cluster. The ghost replica is in an inactive state (its network connections and output are disabled) until the primary VM fails. When the host on which the primary VM was running fails, the ghost replica becomes active in an instant, and the network connections of the replica become enabled. Switching to the ghost replica is smooth; you would only notice a slightly higher latency for one packet if you were to ping the IP address of the protected virtual machine. There is no data loss or service interruption if you are using fault tolerance.

Migration

Live Migration Vs. vMotion

There are two types of VM migration for Hyper-V: Live migration and quick migration. Quick migration puts the VM into a saved state, which is similar to hibernation. The VM’s memory is saved to the disk where the other VM files are located. There is a small period of VM downtime once the VM is put into a saved state until it is registered on the new host. Live migration is used for VM migration between Hyper-V hosts (including failover clustering) without losing service availability. The migration procedure consists of different stages. The most important stage of live migration is the iterative transfer of VM memory pages.

The analog of Hyper-V Live Migration in VMware vSphere is vMotion. With vMotion, running VMs can be migrated from one ESXi host to another without interrupting the VMs and services that are running on the hosts. When the VM migration process from a source ESXi host to a destination host is initiated, the following operations are performed.

First, compatibility is checked to ensure that the VM can be run on the target host. Hardware resources are reserved on the target ESXi host and the VM process is started. A VM memory checkpoint is created; all further VM memory changes are written to a special area. The data stored in memory before creating a checkpoint is copied to the target VM. The source VM is running and some memory pages are changed (these pages are called dirty pages). They are written to change buffers, which are copied to the target VM and integrated into the memory of the target VM. The process of copying dirty pages is repeated over and over (an iterative process) until there are no memory changes in the source VM’s memory compared to the destination VM’s memory. vMotion then stops the CPU of the source VM, copies the last change buffer, and integrates the memory changes stored in that buffer to the virtual RAM of the target VM. Disk access is discontinued on the source ESXi host and is started on the target ESXi host. vMotion sends a reverse ARP (Address Resolution Protocol) to the physical switch for the purposes of updating the MAC and IP address mappings. The source VM is then shut down and the VM process is deleted on the source ESXi host.

Storage Migration Vs. Storage vMotion

Hyper-V Storage Migration can move VM configuration files, checkpoint files, virtual hard disks and mounted ISO files to a new location. Storage migration can be used when the VM must continue running on the same Hyper-V host but must migrate to another storage location. If the VM must be run on another host and placed on another storage location, then live or quick migration must be used along with storage migration.

Storage vMotion is the vSphere analog of Hyper-V’s Storage Migration. Storage vMotion can perform live migration of VM components such as VM virtual disks (VMDK files) and VM configuration files to another storage (e.g., ESXi local datastore, shared SAN datastore, etc.). First, the VM metadata (including configuration, swap and log files) stored in the VM’s home directory is copied, then the virtual disks are copied.

Changed block tracking is used to track the changes after the first replication of the source VM’s virtual disk. While the source VM is running, the disk changes are copied in several iterations to the target virtual disk until there are no differences between the source and target virtual disks.

The virtual disks can be transformed from thick-provisioned disks to thin-provisioned disks or vice versa. During migration with Storage vMotion, the ESXi host on which the VM is running does not change. Storage vMotion is used together with vMotion when the VM must change hosts and datastores at once.

Storage

Cluster Shared Volumes (CSV) Vs. VMFS Cluster File System

Cluster shared volumes are developed to provide simultaneous access to the same disk by multiple Hyper-V hosts in a cluster for reading and writing data. Disk volumes with NTFS or ReFS file systems must be used for CSV.

VMFS is a virtual machine file system that is optimized for virtual machines and is suitable for clusters in vSphere. A VMFS can be shared by multiple ESXi hosts in a cluster for reading and writing data. This high-performance file system holds files with metadata and has specific locking mechanisms to prevent data corruption caused by writing metadata by multiple hosts concurrently.

SMB 3.0 Vs. NFS

SMB 3.0 is a network-sharing protocol developed by Microsoft that can be used for accessing file-level shared storage by Hyper-V hosts in Hyper-V environments as well as Windows environments in general.

Network File System (NFS) is a sharing protocol developed by Sun Microsystems that provides access to files in file-level shared storage. NFS can be used by different vendors, by different operating systems, and in networks that have different architectures. NFS can be used by ESXi hosts in vSphere for accessing VM files located on shared storage.

VHDX Vs. VMDK (VM virtual disks)

VHDX (VHD) is a file format of Hyper-V VM virtual disks. Gen1 VMs can use either VHD or VHDX virtual disk files. Gen2 VMs can use only the VHDX virtual disk file format.

VMDK is an open file format of VMware VM virtual disks developed by VMware.

Dynamic Disks vs. Thin-Provisioned Disks

Dynamic disks are dynamically expanding virtual disks for Hyper-V VMs. The main purpose of using dynamic disks is to save storage space. When Dynamic Disk is created, not all provisioned space is occupied by the virtual disk file. Initially, the virtual disk is a small file. It grows as data is written to the disk.

Thin-provisioned disks in the VMware virtual environment are similar to Hyper-V’s Dynamic disks. A thin-provisioned disk has a minimal size upon creation and grows as needed as data is written to the virtual disk.

Checkpoints Vs. Snapshots

A Hyper-V checkpoint is used to save the state of a virtual machine at a given point in time. You can create a checkpoint, work with a VM, and then revert the VM back to the state it was in when the checkpoint was created.

The VMware snapshot is the analog to the Hyper-V checkpoint; it, too, is used to save the current state of a VM. You can take a snapshot at any time, e.g., before updating software configuration, updating installations, testing new software, etc. A snapshot can be taken even while a virtual machine is running. The snapshot captures the following VM data: Virtual disk states, virtual memory content, and VM settings. The following files are created and stored in the VM directory when a snapshot is taken: .vmdk, -delta.vmdk, .vmsn, and .vmsd.

Pass-Through Disks Vs. Raw Device Mapping

A pass-through disk is a physical disk that is mounted to a VM and used instead of (VHD/VHDX) virtual disks in Hyper-V environments.

Raw device mapping (RDM) is an alternative to the use of virtual disks for accessing the VM disks in vSphere. Depending on your settings, you can let virtual machines access the physical storage device directly when RDM is used. RDM can be used in virtual compatibility mode or physical compatibility mode.

Virtual compatibility mode provides full virtualization of the mapped device; snapshots can be used.

In physical compatibility mode, the guest OS can access the hardware directly; minimal SCSIvirtualization is provided. If a VM snapshot were taken, the RDM disk would not be included in that snapshot.

For example:

Suppose DiskName.vmdk is a disk heading file.

DiskName-rdmp.vmdk is the mapping file (as evidenced by the -rdmp suffix).

A mapping file resembles a symbolic link to a mapped device.

RCT Vs. CBT

Resilient Change Tracking (RCT) makes it possible to copy only those data blocks that have changed since the last backup when an incremental backup of Hyper-V VMs is performed. RCT technology was released with Windows Server 2016. Files with .RCT and .MRT extensions are created when RCT is enabled.

Changed Block Tracking (CBT) is the vSphere analog of RCT for Hyper-V, helping perform incremental backups. CBT allows backup software to track the blocks that were changed since the last backup and copy only the changed blocks. Files with .CTK extensions are created for each VMDK virtual disk and snapshot file. Block changes are tracked on the virtualization layer. The VMware vSphere API for Data Protection allows third-party software to access the CBT feature for the purposes of performing backups and replications more efficiently. Changed Block Tracking was introduced by VMware in 2009 with ESX 4.0.

Storage Spaces Vs. vSAN

Windows Storage Spaces is a technology that abstracts storage from physical disks, used for storage virtualization and provisioning. With Storage Spaces, you can create cost-effective, scalable, highly-available software-defined storage for Hyper-V virtual environments by using servers with HDD devices.

VMware vSAN (Virtual Storage Area Network) is the equivalent of Windows Storage Spaces. vSAN provides a distributed concept of data storage that is integrated with vSphere at the cluster level and abstracted from hardware disks. ESXi hosts managed with vCenter must be used to deploy vSAN. This approach allows disks of ESXi hosts to be used for vSAN storage while ESXi hosts are used to run VMs. SSDs can be used for caching data, while HDDs are used to store data. Disk groups are aggregated into pools that are accessible for the entire cluster. The data can be mirrored on one or more hosts, which ensures protection against failure. This is defined by the number of failures setting. Using vSAN, you can achieve reliable and scalable enterprise-grade storage.

Dynamic Memory Vs. Virtual Memory Overcommit

Dynamic memory allows a VM to use the amount of memory that it currently needs, rather than all virtual memory provisioned for that VM. The hardware RAM on Hyper-V servers can be used more rationally with this feature.

VMware Virtual Memory Overcommit is the equivalent feature in VMware. Both features use dynamic memory allocation methods, but the methods differ somewhat. With VMware, you can allocate more memory than the ESXi host has for VMs, thereby increasing the density of VMs on the host. The idea is to optimize physical memory consumption – thus, more VMs can run on the ESXi host, but there is no guarantee that all the memory provisioned for it can be provided to a particular VM at a given point in time. Using Memory Overcommit presumes that a memory can be rearranged between the VMs depending on demands. 25% of unused memory is reserved on an ESXi host in case the memory requirements of its VM suddenly increases.

Hyper-V Integration Services Vs. VMware Tools

Hyper-V “Integration Services” is a set of tools and drivers for providing proper functionality of child VMs and time synchronization. When Integration Services are installed, the standard drivers for mouse, keyboard, disk controllers, and network controllers are replaced by drivers included as part of the Integration Services suite for better performance and user experience.

VMware Tools is a collection of drivers, modules, and utilities that is the VMware equivalent of Hyper-V Integration Services. VMware Tools offers advanced functionality to improve performance, user interaction with guest VMs, and integration. When VMware Tools is installed, you can shut down the guest OS directly from menu in the vSphere client or VMware Workstation. Video resolution and graphics performance are increased. Other useful features like Drag & Drop, shared folders, time synchronization, copying/pasting between VMs and hosts, and grabbing/releasing of the mouse cursor also become available. VMware Tools is usually provided as an ISO image or Operating System Specific Package (OSP).

Migration from Microsoft Hyper-V to the VMware vSphere Platform

Transitioning from Hyper-V to vSphere can be useful if the vSphere virtualization platform better meets your company’s needs. First, you should prepare the hardware and software. Check VMware’s hardware compatibility list to ensure that your hardware is officially supported. Install ESXi servers, vCenter, and other software that you need. Download and install VMware vCenter Converter. Run VMware vCenter Converter and click Convert Machine.

Select Hyper-V Server as a source type. Select the IP address or server name of your Hyper-V Server. In this walkthrough, vCenter Converter is installed on the Hyper-V Server and localhost is input as the server name.

Next, input the username and password of an account that has sufficient permissions to perform the operation. The Windows Administrator account is used in this example. Click Next.

Select the virtual machine to be converted. Click Next.

Select VMware Infrastructure virtual machine as the destination type. Type the IP address of your vCenter Server or ESXi host, then enter the username and password. Click Next.

Select the folder and the name that should be applied to the destination VM after conversion. Click Next.

Select a destination location, including the ESXi host and datastore. Click Next.

Configure the options for the destination VM. You can see how to change the type of virtual disk provisioning in the screenshot provided below. Click Edit in the Data to copy section.

In the dropdown list, select the disk provisioning type (thick or thin). Click Next.

Check the summary and, if you are satisfied, click Finish.

Wait until the conversion task is complete.

Once the VM conversion is finished, you can open vCenter with vSphere Web client in your browser and find the converted VM. In this walkthrough, the VM name is Server2016-01. As you can see on the preview screenshot, the VM is successfully running. Click Launch Web console to access the user interface of the VM. You can now tune the virtual machine and install VMware Tools.

You now know how to migrate from Hyper-V to VMware vSphere.

Conclusion

Knowing the differences and similarities between features of VMware and Hyper-V environments makes migration from one platform to another easier and is particularly useful for administrators of a multi-hypervisor environment. Understanding the architecture as well as the working principles of both virtualization platforms, along with their feature sets, helps administrators work comfortably and effectively when they are tasked with maintaining the less-familiar platform. There is one important rule that applies to both VMware and Hyper-V environments: The VMs must be backed up regularly. Robust VM backup can protect your business-critical data against natural disasters, technological failures, human errors and virus attacks.

November 22, 2022
Broadcom’s Software Strategy: Milk the Cash Cow, Exploit ‘Synergies’
The week began with rumors that Broadcom would acquire VMware and ended with the news that the deal is all but done. The VMware board, along with Michael Dell (who still owns about 41% of VMware’s stock), have a 40-day “go-shop” option; however, I don’t believe there will be other suitors.

On Tuesday, I published an article about Broadcom’s desperation to gain cloud relevance and transform itself into a software company. I detailed the fact that VMware’s profitable business model and its nearly $4 billion of free cash flow makes it an attractive acquisition target. Finally, I said that Broadcom needed to find innovative DNA to make this acquisition work.

After thinking further about this proposed acquisition and talking to a number of VMware customers, as well as customers of CA Technology, which was acquired by Broadcom in July 2018, my outlook on the acquisition has become decidedly more cynical and likely more realistic.

Milk Profits, Minimize Expenses

We have seen this scenario play out before. Several former CA Technology customers and employees I spoke to was quick to say, “CA—where software goes to die.” That phrase has become so synonymous with CA that even the Wikipedia page for the company has a section on the phrase! Joking aside, most CA customers I spoke with over the last week agreed that since their acquisition, the company has “gone dark.” CA Technologies was a profitable company with important software that enterprise clients relied on and couldn’t easily transition away from. Sound familiar? VMware is in the exact same spot.

Soon after the CA acquisition, Broadcom slashed about 40% of the CA workforce. On an analyst call to discuss the acquisition, Tom Krause, president, Broadcom Software Group (the Software Group will be renamed VMware) said that the company will increase VMware earnings from $4.7 billion to $8.5 billion within the next three years. How will the company nearly double earnings? “Synergies,” of course. The drivers of growth will be savings generated by eliminating duplicate functions in the combined company along with projections of more workloads running on the VMware platform.

Do I expect Broadcom to change its corporate DNA to become an innovator? Will VMware innovators change Broadcom? Based on Broadcom’s past behavior, sadly, I think the answer is clear—No.

I expect VMware to quickly cut costs across the board. Everything from headcount to R&D will see an immediate slash.

This is Not Dell 2.0 For VMware

You might argue that VMware has gone through this before when they were acquired by Dell and later set free by Dell Technologies in a 2021 spinoff. Why is this acquisition different? The Dell deal was, at its core, financial maneuvering to finance a deal and enrich investors with the goal of selling or taking the businesses public at a much higher market valuation. There was little cross-pollination between the Dell Technologies companies, and in this deal, Dell acted more like a private equity firm than an acquiring company.

Broadcom is different. VMware is going to be milked as a cash cow as Broadcom tries to leverage VMware’s importance in the data center. VMware has huge cloud computing customers (AWS, for one) that are reliant on the company’s technology. If you are a VMware customer, get ready for higher prices as the company shifts to more of a subscription model. I also suspect that we will see large layoffs and the diminution of VMware as an entity. This is the end—VMware is not going to have another opportunity to go public on its own or go private in the future.

I also don’t expect huge technology advancements from VMware over the next decade; but the company won’t die. The company will continue to exist because of the mere fact that enterprise software never dies.

What Should Partners and Customers Do?

This isn’t the end of technology acquisitions; nor is it a one-off event. Within the current market conditions, software vendor valuations have come crashing down to a more realistic level. I suspect that many large businesses with cash on hand will be looking to buy profitable software businesses that are viewed as essential. So, what should technology and business leaders do?
- Understand vendor roadmaps and business. If you choose to go with an independent vendor, especially a smaller company, you need to understand their product roadmap along with their overall business strategy.
- Don’t become overly reliant on a single vendor. This isn’t the first time VMware has forced customers to rethink their relationship with the company. Years ago, VMware attempted to drastically increase licensing costs. Customers revolted and the company reversed course; however, it was a wake-up call and many businesses began to look at alternatives like OpenStack. You also need to ask technology leaders within your business what happens if a vendor changes their pricing, becomes a competitor or goes out of business. These “what if” scenarios are critical to reducing your risk.
- Explore open source options. Open source isn’t a cure-all, but many businesses have adopted open source in part to reduce the risk of vendors changing strategies, increasing prices or no longer supporting your software. With open source technology, even if a project’s community is no longer active, a business that uses the technology can continue to contribute code and maintain the project.
May 27, 2022
Database Virtualization: Breaking Vendor Lock-In

Every enterprise is battling vendor lock-in—constantly. Ask any IT leader about what keeps them up at night and vendor lock-in is certainly near, if not at, the top of the list. And while network, storage and compute have long been democratized by virtualization, one discipline has held out as the last bastion of vendor lock-in: Databases.

How bad can it be? Take Amazon, for example. They celebrated their retirement of the last internal Oracle database exuberantly. People familiar with the matter suggested that replacing Oracle was at least a decade in the making, probably longer. It is anybody’s guess how much effort it took to accomplish this feat. A big celebration was in order, indeed.

Now, let’s take a look at database virtualization. As a relatively new discipline, the idea of virtualizing critical systems like data warehouses made a big splash recently. But does it apply to your systems? And the ubiquitous question: Does it come with performance penalties?

What Is Database Virtualization?

Database virtualization is the concept of abstracting applications and databases from each other. Using virtualization, an application written for, say, an Oracle database can run on an open source database like PostgreSQL without making changes to SQL or to associated APIs. As an aside, this is not to be confused with data virtualization—which is really a data integration play.

A database virtualization platform translates queries and data in real-time. Through emulation, it bridges functional gaps between the source and destination system so applications remain unchanged while moving to a different database. Database virtualization doesn’t make migrations easier; it makes them effectively obsolete.

The idea is rather compelling. So much so that Amazon developed and released their own version of a database virtualization product recently that targeted small standalone database systems.

Why You Need Database Virtualization

Here’s a simple rule of thumb. The cost of migrating a typical data warehouse—without virtualization—is about the total cost of the database over the next five years. So, if your current data warehouse runs you, say, $5 million a year, (which is typical for a mid-range system) your cost to migrate it will probably be close to $25 million. Moreover, the project could take up to five years.

But wait; there’s more. The true downside of conventional database migrations is actually their abysmal failure rate. Analysts put it typically above 80%. For large data warehouse systems, it seems to be higher still. That’s what makes databases the ultimate bastion of vendor lock-in.

As an IT leader, you naturally don’t have the time, the budget or the appetite for such a risky undertaking; you’d probably rather deploy that kind of money toward creating new revenue. Database virtualization lets you do just that: Sidestep these risks and, instead, focus on what’s really relevant.

When to Use Database Virtualization

Database virtualization is particularly suitable for replatforming data warehouse workloads. These workloads are notorious for their complexity and for the effort a rewrite would entail. Often, these workloads have been grown and curated for years—sometimes decades. Maintaining the workload while replatforming makes economic sense: It means preserving and protecting long-standing investments.

However, for database virtualization to be successful, both the source and destination systems need to be of comparable performance and scale. This should be obvious: Moving a workload from, say, a Teradata system to a Postgres instance is unlikely to satisfy your business users.

But there is good news. Over the past few years, cloud data warehouses have increasingly reached competitive performance targets. They have become viable alternatives to legacy on-premises appliances. From an IT leader’s perspective, that is great timing. Most IT leaders today are currently tasked with sunsetting these appliances over the course of the next few years. Database virtualization couldn’t be any timelier.

Goodbye, Vendor Lock-In

Virtualization, as a general concept, has revolutionized the IT stack over the past 20 years. Server virtualization, which started as a niche for QA and developers, became the foundation of cloud computing; virtualization then spread to network and storage. Databases are one of the few remaining disciplines that have bucked the virtualization trend—until now.

Not surprisingly, the premise of database virtualization resonates greatly with practitioners. And even though the initial success in the data warehousing arena is yet to be replicated in other areas, it seems like it’s just a matter of time before the concept becomes ubiquitous.

February 3, 2022
VMware Brings Hybrid Cloud Computing to Virtual Desktops

VMware has updated its desktop and application virtualization platform to make it easier for IT teams to support end users who need to regularly shift between working in the office and from home.

The latest version of VMware Horizon adds support for an instant clone capability that rapidly provisions personalized virtual desktops and applications enabled by the VMware vSphere platform.

VMware Horizon is also now available on the Google Cloud VMware Engine service as well as on-premises servers from Dell EMC, a sister company of VMware. It is also committing to adding support for Microsoft Azure in addition to existing support for Amazon Web Services (AWS).

VMware is also adding REST application programming interfaces (APIs) to make the platform more programmatically extensible.

The company has also optimized audio and video support for Microsoft Teams and is making it possible to publish Linux applications on the platform.

Finally, VMware has launched a beta of VMware Workspace ONE Proximity, which leverages Bluetooth Low Energy (BLE) beacons to give employees insight into how to better maintain social distancing in the workplace. A forthcoming Workspace ONE Campus will also enable employees to leverage that same beacon infrastructure to book meeting rooms and determine how best to optimally move about a larger work environment.

Jeff McGrath, senior director for VMware End User Computing, said interest in hybrid cloud implementations of VMware Horizon has risen sharply. IT teams want to be able to drive application traffic to on-premises servers to minimize latency whenever employees are in the office. When those same employees are at home, however, it may make more sense to drive that traffic to a cloud-based instance of VMware Horizon. As employees move between environments, McGrath said VMware Horizon makes it possible to maintain a consistent digital workspace experience for end users.

Interest in virtual desktops has been rising since Microsoft changed Windows licensing terms in advance of launching its own Windows Virtual Desktop (WVD) service. The VMware Horizon platform is significantly more robust in terms of both the hybrid cloud computing scenarios it supports and the management tools provided to IT teams, said McGrath.

In the wake of the COVID-19 pandemic, it remains to be seen how many desktops will shift into the cloud. Most IT organizations were simply content to provide access to desktops applications over a virtual private network (VPN) when the pandemic first hit. Now that it appears more employees will be working from home on a semi-permanent basis, many IT organizations are revisiting their entire IT strategies.

Of course, there is no shortage of options when it comes to desktop and application virtualization. VMware is betting a large installed base of IT organizations that have already standardized on VMware vSphere in both on-premises and now cloud computing platforms will result in more enterprise IT organizations embracing VMware Horizon. Regardless of the outcome, the one thing that is clear is virtual desktops in the age of COVID-19 are now well on their way to becoming the preferred default option for many organizations.

August 6, 2020
Red Hat Updates Virtualization Platform

Red Hat today announced an update to its virtual machine platform that now can run on the latest version of Red Hat Enterprise Linux (RHEL).

Version 4.4 of Red Hat Virtualization can now run on version 8.2 of RHEL. In addition, Red Hat has added Grafana-based dashboards for its Data Warehouse (DWH) service that IT teams can employ to track performance and analyze workloads.

Finally, Red Hat Virtualization 4.4 simplifies configuring network services using its NetworkManager console.

Peter Lauterbach, a product manager in the Cloud Platforms group at Red Hat, said support for RHEL 8 is core to the company’s overall strategy to enable IT teams to run microservices-based applications simultaneously with existing monolithic applications.

Red Hat envisions IT organizations will deploy RHEL on bare-metal servers running Red Hat OpenShift, an application development and deployment platform based on Kubernetes that can run applications built using containers. In addition, Red Hat OpenShift supports kubevirt, open source software that enables kernel-based virtual machines (KVMs) such as Red Hat Virtualization to run on top of a Kubernetes cluster. Made available as Red Hat OpenShift Virtualization, that software makes it possible to run legacy monolithic applications alongside microservices-based applications on the same Kubernetes cluster as needed, said Lauterbach.

That approach not only is easier to manage from an infrastructure perspective, Lauterbach noted it also can reduce latency between monolithic applications and microservices. Many IT teams in the wake of the COVID-19 pandemic have accelerated the rate at which they are moving applications into the cloud by encapsulating entire monolithic applications into containers that then can be ported to a cloud platform without the need to refactor them. They then can begin to modernize those applications by breaking them up into smaller microservices.

Lauterbach also noted it has become a lot easier to leverage Kubernetes to move applications between platforms because interoperability between storage and networking services has improved significantly, thanks to the Container Storage Interface (CSI) and Container Network Interface (CNI) that have been made available by the Kubernetes community. In fact, Lauterbach said standards in infrastructure are living up to their interoperability promise.

Red Hat, of course, has been making the case for open source virtual machines for decades now. However, in the wake of the economic downturn brought on by the COVID-19 pandemic, there will be many more organizations looking to reduce costs by relying on commercial virtual machine software. In addition, Lauterbach noted that now that cloud service providers such as Amazon Web Services (AWS) support KVM, it is a lot easier to create a hybrid cloud computing environment by deploying Red Hat Virtualization in a local data center than it is to replicate a VMware environment in a public cloud.

It’s too early to say how the battle for hybrid cloud computing will play out in the months and years ahead. The one thing that is certain is the more diverse the IT infrastructure environment is, the more difficult it is to manage.

August 4, 2020
How to Secure the Intangible — Cloud Native Security in the New Age
People don’t want to buy a quarter-inch drill. They want a quarter-inch hole.

Companies aren’t interested in managing IT; they’re more interested in results. They want to deliver software quickly and focus on their core product or service. This is where the concept of cloud native computing comes in—and why securing it has become critical.

Traditionally, companies had many servers and resources. Today we’re moving toward a simpler reality. Developers are focusing on what’s needed to deploy business logic, regardless of what it’s deployed on. That’s why it’s particularly appealing to move to software without the server and the added complex management of IT overhead.

If a company can push its offerings without hardware, patch management, storage and so forth, and it can do it more affordably and faster, it’s a great win.

The Paradigm Change

2012 was the golden age of IT, where admins ruled the world and developers got in line. Moving from that era to 2016, we come to the age of the cloud, where developers were kings.

For many years, the IT department led—and perhaps stifled—innovation within organizations. Developers’ paths for innovation were limited to the area which they had control, i.e., the application layer. Anything beyond that was the IT department’s call. This included hosting, load balancing, database management and whether to adopt virtualization or advanced networking technologies.

With the emergence of NFV and cloud computing, that changed. There has been a continuous erosion of the abstraction layers that developers were faced with in the past. In 2020, developers are the masters of their own destiny, pushing their organization toward innovation. All of this is possible not only due to virtualization and cloud computing, but also to the relative simplicity of a developer setting up a full application stack on their own laptop.

In many organizations, the CTO focus has moved from an infrastructure orientation to an application focus. Developers are the new golden children of these organizations. They hold the keys to increasing company profits.

What Cloud Native Brings to the Table

There are many numbers being thrown around about the efficiency of cloud native. Whether it’s 300% reduction in deployment time or a mere 30%, the reality is that the time savings are huge. Many developers will never see a traditional data center again in their lives. In fact, new applications are developed as microservices or at the very least as containers, simply because they’re faster.

There’s no going back. This new approach enables developers to invest in scalable solutions without making companies preallocate unnecessarily large sums of money. Both applications and infrastructure can scale up or down quickly, using the right amount of resources to provide the best service.

Businesses that are seasonal or peak-oriented gain a lot from the movement to cloud native architecture. Users also stand to gain from the move, as they receive the best service possible, without downtime. Everyone comes out on top.

A Look at the Future

This movement to broader use of cloud native architectures is a trend that will continue to grow over time.

By 2025, VMs and bare metal will be as anachronistic as mainframe and token ring networks are today. In fact, most medium-to-large organizations already have some kind of a migration project from VMs to containers in progress. Many developers don’t need to spin up VMs anymore, since they can have a whole application stack running on their laptop using containers. When they are done, they simply lift and shift it to staging and production environments.

In addition, on-premises and cloud environments are becoming a single entity rather than two separate infrastructures. People used to think that the entire data center was going to move to managed clouds for economical reasons. Today, however, most deployments are becoming hybrid. Organizations are opting to keep some of their on-prem systems (mostly legacy applications that cannot be easily moved to the cloud). IT administrators are starting to think about the deployment in the cloud as an extension of the on-prem data center rather than a separate network that needs to be managed.

This trend will most likely continue, making it all the more important to understand the changing expectations around how to secure the hybrid network environment. The rate of change for cloud native environments is monumentally faster than on-prem, initiating code in milliseconds.

Most legacy solutions weren’t built to cope with ephemeral networks and compute entities that go up and down within seconds, one million times a day. So, what’s the best way to future-proof risk management in this ever-changing environment? There is no singular answer, but in general prevailing advice suggests you consider the long view, exploring the options best suited for your approach and then look at each through a one, three and five year lens.

Cloud Native Security

When thinking about the future of security, the basics stay the same and the threat modeling concepts stay the same. However, there are a number of security issues to consider, including policies, permissions and authorizations. As networks are converging, the security tools are as well. Looking forward, only solutions that can provide a genuine single pane of glass to control your digital estate will have a real impact on security.

When looking at solutions to help apply security to cloud native architectures, ask the following questions:
- Can I improve visibility in my cloud service? Applications may be deployed over multiple cloud instances and on servers in different sites, and even different regions, making it more difficult to define clear security boundaries. The right tool can help you to understand the scope and layout of your cloud resources, particularly if it automatically discovers applications on your cloud deployment and maps the data flows between them, making it easy to understand applications that you have running in the cloud and how they interact with one another.
- Can I achieve a single view of multiple environments? The tools used to apply security need to adapt to be able to control hybrid networks across on-prem and multicloud organizations. For example, if your architecture requires policies to be deployed to the edge, how is it managing convergence? Numerous network solutions are barely aware of layer 7 process level, let alone containers.
- Can I set and maintain policies across these environments? Consider how solutions address network policies and understand potential challenges from both a technical and procedural point of view. Securing application traffic within cloud native architectures or between applications and external networks is a good start, but in the cloud, it’s necessary to implement micro-segmentation so you can define policies at the application level. By defining which applications are allowed to interact and the types of interactions that are permitted at a granular level, it’s possible to provide the level of security necessary for applications operating in the cloud. Look for tools that provide automatic policy recommendations that can be effectively applied on any cloud infrastructure, streamlining your organization’s security policy for deployments.
- Can I future proof my cloud native security posture? When planning your future roadmap, think about what your network is going to look like. What type of workloads and network constructs are going to be used? Make sure the security vendors you select provide full coverage for your roadmap.
When implemented correctly, modern micro-segmentation techniques can offer a simple way to secure a cloud native environment, including solving the unique challenges of containers and providing the ability to create dynamic application policies down to process level. Look for tools that provide a single view and foundational visibility, automatically discovering all network flows and dependencies. This allows your business to take advantage of the best of serverless computing without increasing risk or complexity for security.
May 15, 2020
Rackn Extends Reach of IT-as-Code Framework

Rackn announced the Digital Rebar platform it created to automate the provisioning of bare-metal infrastructure now can be applied simultaneously across multiple physical sites.

Company CEO Rob Hirschfeld said Digital Rebar 4.3 makes it possible to manage IT infrastructure as code across highly distributed computing environments. Previously, IT organizations would have needed additional versions of Digital Rebar for each physical data center.

Hirschfeld also noted IT teams can also set up mirrors of data centers, now that Digital Rebar can be employed across multiple sites.

IT teams that adopt Digital Rebar can define workflows through which, for example, instances of virtual machines or Kubernetes can automatically be deployed on hundreds of thousands of servers in about an hour, said Hirschfeld.
Accomplishing that same task using manual processes would take most IT teams six months, he noted, adding IT teams can even use Digital Rebar to deploy instances Kubernetes on virtual machines at scale.

Hirschfeld noted many IT teams assume they need to have a layer of virtualization in place before they can start automating the management of their IT environment. Digital Rebar provides a framework around a common set of application programming interfaces (APIs) through which servers, storage and networking resources can be provisioned consistently. In addition, IT teams can set up a to manage IT infrastructure on an ongoing basis, including firmware and operating system updates and cluster configurations.

Digital Rebar doesn’t necessarily replace the need for frameworks such as OpenStack or VMware vSphere. However, it does provide a simpler mechanism for installing them. Alternatively, IT teams can employ Digital Rebar to deploy software such as Kubernetes on a bare-metal server.

Hirschfeld said Digital Rebar is designed to be deployed on-premises or in a public cloud rather than be accessed as a software-as-a-service (SaaS) application. That approach means IT teams operating in highly regulated industries can still take advantage of the framework, he said.

While the concept of IT as code has been around for a while, IT teams that are being asked to now manage IT at an unprecedented scale are turning to IT automation frameworks in larger numbers. Hirschfeld said Digital Rebar is now designed to replace other frameworks as much as it is to provide a complementary framework for specifically managing bare-metal infrastructure. In fact, Digital Rebar can be access vis Terraform or Ansible using a plug-in that Rackn makes available.

Rackn also plans to extend the reach of Digital Rebar out to edge computing platforms as well as endpoints based on, for example, Raspberry Pi platforms, he added.

Most IT teams will eventually wind up employing multiple IT automation frameworks in place of thousands of custom scripts that many IT professionals have developed themselves. The issue with most of those scripts is they don’t tend to scale especially well. In addition, they are usually poorly documented so when the author of the script leaves the organization the remaining IT staff wind up having to write another custom script because no one is quite sure how the script in place works. Whatever the path forward, the one thing that is certain is the days of being able to get by manually provisioning platforms is finally coming to a merciful end.

— Mike Vizard

February 27, 2020
Mainframes: The Cloud Before the Cloud

Here’s something to think about: Despite the fact that IBM stopped manufacturing personal computers in 2005, deciding instead to focus on higher margin business services, the company continues to produce mainframes and by all estimates will continue manufacturing them well into the future. The question is, why? The answer: because there is still significant demand for this type of hardware.

The fact is that a mainframe is a monster when it comes to computation and processing transactions. One mainframe can process 2.5 billion transactions daily. That’s the equivalent of 100 Cyber Mondays in a single day. To give you a clearer idea of the immensity of scale, keep in mind that Cyber Monday sales in 2018 were $7.9 billion. This translates into a mainframe being able to handle a bit more than three-quarters of a trillion dollars in transactions each day, which is almost twice the yearly GDP of Austria!

As I said, the mainframe is a workhorse. Yet, its power and potential are hidden from most people who work in the modern cloud. This is surprising because there’s a good argument to be made that, given its power and scalability, coupled with its support for a large number of concurrent users, the mainframe was the cloud before there ever was a cloud. However, the technology is practically unknown among younger developers, mostly due to cultural reasons and a lack of historical awareness. That more people are not taking advantage of the tremendous opportunities at hand is a misfortune. When it comes to mainframe computing, there’s a lot of interesting tech to be learned and a good amount of money to be made. It’s just a matter of increasing awareness to do this. A good place to start is to redefine the narrative.

Allow me to elaborate.

A Special Club

When mainframes appeared on the commercial landscape, few people interacted with computers and even fewer worked with them directly as programmers or system administrators. Just as in a modern data center, the machines were isolated in climate-controlled rooms to which few had access. Access to the actual physical machine was restricted to a special few. But unlike today, when restricted access is due more to security concerns because there are a lot more people around who are potential black hats, back then access was also restricted due to the exclusivity of the profession.

In the 1950s, 1960s and well into the 1970s, not just anybody could program or be a systems administrator. You could go to a technical school to learn to become a computer operator, which meant you could work at a terminal as a bank teller or travel agent. Or you could learn to become a data processing clerk and work in a back office at a brokerage firm. But to learn to program, you had to go to college. Then after college, you had to go to work for a company or organization that actually used computers in day-to-day operations. These were banks, insurance companies, major manufacturers such as General Motors and General Electric, airlines, very large retailers and government. Computing was a club and you had to be well-educated to get in and be well-liked by the organization to keep your membership. Yes, there were the outliers—Gates, Allen, Joy, Wozniak and Jobs—but for most people, the career path to being a programmer was college and then into big business or government.

As a result, computing had an air of mystery and magic to it. Remember, until the mid-1980s, most people had never touched a computer. It was the stuff of moon landings and election night predictions on network TV. Then the PC came along and changed everything. Within 10 years computers became commonplace. Even elementary schools had them—at least, those that had the budget and foresight. The power that business applications such as the spreadsheet program Lotus 1-2-3 brought to small business motivated even the dry cleaner on Main Street to plunk down the thousands of dollars it cost to buy a PC. The productivity boost was worth it. Also, since the dry cleaner actually owned the machine, the dry cleaner’s kid could learn to program on the box unencumbered, when it wasn’t being used during business hours. Remember, the Microsoft DOS operating system that shipped on every IBM-compatible PC came bundled with the programming language BASIC. You didn’t need to go to college to learn to program; all you needed was a book.

So, PC programming took off. Instead of having to go through the effort required to get into the special club the mainframe programmers belonged to, PC programmers started their own clubs and anybody could get in. All you had to do was show up. PC programming became a meritocracy. Also, it became extremely lucrative work. There were a lot of PCs being shipped and the appetite for applications to run on them was enormous. The unlikeliest of people, such as Lotus Software founder Mitch Kapor and his sidekick, Grateful Dead lyricist John Perry Barlow, became wealthy, thanks to the PC.

The PC became cool and programming for it became cooler. But there was a fact that got obscured as the popularity of the personal computer grew: Compared to the mainframe, the PC was a toy—a productive toy, a liberating toy, but a toy nonetheless, very much in the same way that a go-kart is a toy compared to a Formula One race car.

Feature for feature, the mainframe ate the personal computer’s lunch. The IBM System/370 Extended Architecture Mainframe released in 1983 was a 31-bit machine that could support up to 8MB of memory with a processor running at 38 MHz. By comparison, the IBM PC-XT, also released in 1983, was an 8-bit machine that shipped with 128KB of RAM and an 8088 processor that ran at a speed of 4.77 MHz.

But, as the saying goes, “You should never underestimate the value of cool,” and the PC was very cool. It got a lot of attention while the mainframe stayed hidden away in climate-controlled rooms in the basements of banks and insurance companies. Yet, the mainframe was more powerful, and whereas the PC was meant to meet the needs of a single user, a mainframe can service thousands.

Built to Scale Built from Inception

Mainframe computers from the start were designed to support hundreds (if not thousands) of users. However, more importantly, mainframes were and are designed to support millions of concurrent transactions. When you think about it, while it’s possible for a high number of user requests to exhaust CPU utilization in a typical commodity machine—which is the reasoning for being able to scale up CPU count by increasing VM allocation in the cloud-based computing cluster—the real burden on the CPU is not so much the request count but the computing burden that each request incurs. For example, one request getting a .JPG of a cat will incur nowhere as much burden as a request that needs to convert an audio file in .WAV format to MP3.

The need to alleviate request burden has been around since bank tellers working in the various branches of a big bank had to make deposits and withdrawals at the same time. Mainframe engineers addressed the problem decades ago. Technologies such as batch processing and message queues, which make computational activity asynchronous and are optimized to fully utilize CPU capacity efficiently, were and are commonplace in the mainframe landscape.

IBM’s 360 mainframes supported batch processing in its OS/360 system starting in 1964. Batch processing allows thousands of instructions to be performed on terabytes of data at pre-scheduled times (usually “off-hours”), all without human intervention.

Message queuing allows processes to take data in the form of a message stored in a message queue and perform computational activity according to the data in the message. Processing is performed independent of a request in an asynchronous manner. When processing is complete, a new message is created and sent on to another queue, where another process will do more processing or create a notification indicating that work is complete. IBM MQSeries was an early message-queuing technology first used in 1993 at State Street Bank well before the internet became commonplace.

The reason that support for user and transactional concurrency was possible in mainframes is due to a technology that is commonplace in today’s cloud environment but was unheard of among PC users back in the days when most data was exchanged between standalone PCs using floppy disks. That technology is virtualization.

Virtualization Takes Mainframe Computing to the Next Level

Today we take virtualization—the capability to have software simulate hardware—for granted. Without it, we couldn’t have native cloud providers such as Amazon Web Services, Google Cloud and Microsoft Azure. These services rely on virtualization to provide the segmentation and computational isolation required to meet the needs of the modern computing. Yet, as essential as virtualization is in the world of commodity cloud computing, putting a layer of virtual machines on a commodity PC is a fairly recent event in the history of computing. VMWare, VirtualBox and KVM became mainstream in the mid-2000s. On the other hand, virtualization has been part of mainframe technologies since the mid-1960s. Virtual memory appeared in the mainframe in 1965. Machine virtualization was introduced on the IBM 370 mainframe in 1972 and is still an important part of the mainframe computing experience today.

Every single person who logs onto the mainframe when working in a native mainframe operating system gets an identical address space. You can think of each address space as a standalone virtual mainframe system. These address spaces are created instantly and are protected from interfering with each other. Also, many companies create logical partitions of their hardware called LPARs (logical partitions). The logical partitions are controlled by a native firmware hypervisor called PR/SM (pronounced Prism).

The value that LPARS provide is a further separation between environments. This means not only can you separate users, but you can also create distinct computing environments such as those created when virtualizing X86 machines. For example, you can have a shared development environment as well as environments for testing, staging and production. You can allocate resources and security access to a particular environment based on need, just like you would in a data center full of commodity machines. This means you can configure production systems to be more secure and have higher execution priority and more resources such as memory. In fact, according to the U.S. government, from a security perspective, virtual machines created on the mainframe are considered distinctly separate machines.

LPARs running on the same hardware or different hardware under the native mainframe z/OS operating system can be linked together as a Sysplex for redundancy and to easily provide additional capacity. Dynamic Virtual IP Addressing (DVIPA) is used to dynamically route IP traffic across multiple LPARs in a Sysplex. This is analogous to being able to tie together a variety of web servers behind a load balancer or organize them within a Kubernetes cluster. Being able to tie computers together is an important feature for making systems scalable. Also, linking computers across geographic regions increases performance. It’s always faster to work with a machine that’s closer to you than one that’s further away. As mentioned earlier, being able to scale systems up and down is a key feature of cloud-based computing. Dynamic connection, which is part of mainframe architecture, is critical for scaling to work.

The important thing to understand is that the stuff we consider to be modern in terms of the machine virtualization that we rely upon to make the cloud work has been in play for years on the mainframe. Also, virtualization on the mainframe makes it possible for programmers and system administration staff to work in a variety of operating systems and programming languages as we’ve come to expect in commodity cloud instances. Machine virtualization allows mainframes to support Linux. Whether the computing environment is virtualized Linux or native mainframe, developers can code in Java, C/C++, Python, Perl and a variety of other languages, including something as new as Swift. But, then again, the mainframe was always intended to support many programming languages.

Even back in the ’60s and ’70s, programmers could write in COBOL, PL/I, job control language (JCL) and Customer Information Control System (CICS). Also, writing low-level code in Assembler has always been possible. Clearly, the versatility that modern developers enjoy programming in a variety of languages in a cloud environment has been part of mainframe computing for a while. Yet, the benefit has never been at the forefront of developer awareness. Maybe it’s time to change the narrative.

Changing the Narrative About Mainframe Computing in the Cloud

As I stated at the beginning of this article, IBM continues to make mainframes because there’s a growing demand for the technology—particularly as smartphones and IoT devices proliferate worldwide. Netflix might be spinning up containers on a commodity machine in a data center to let you view past episodes of “Orange is The New Black.” But when it comes time to pay for your Lyft ride using your iPhone, that credit card transaction will most likely be processed on a mainframe that’s providing service via some sort of API gateway.

There is little benefit to arguing that mainframes should or can be used interchangeably with commodity servers. That’s like saying a fleet of automobiles can be used interchangeably with a city bus. You can do it, but it doesn’t make much sense. Each technology has benefits that when used together, make the overall transportation system more efficient. The trick is to understand where and how the technologies fit together in the big picture. In this case, the big picture is the cloud.

The old narrative around mainframe technology is based on perceptions that are no longer applicable. The modern mainframe is no more centralized than the single building that houses a regional data center. In terms of programming languages, mainframes moved beyond FORTRAN and COBOL decades ago. Current mainframe virtualization technology makes it possible to run programs written in just about any programming language. And, mainframe emulators, such as ZD&T, make it possible to get the hands-on experience necessary to get familiar working in a mainframe environment.

The Perceptions of the Past are No Longer Applicable

The modern mainframe is a powerful resource that provides extraordinary benefits for cloud computing. The new narrative is that the mainframe is modern, adaptable and well-suited to accommodate a very high volume of transactions at lightning-fast speeds. And, it’s a good way to make good money.

As the current workforce (which is admittedly aging) approaches retirement, a new breed of mainframe developer will be needed to continue the work of making the mainframe a mission-critical resource in the modern cloud. In addition to understanding the essentials of mainframe computing, these developers will need to be well-versed with the modern tools and techniques that drive cloud computing. It’s an investment that I’ve urged many up-and-coming developers to make.

Putting It All Together

Let me share a piece of personal history: A while back I needed to do some programming. So, I went to my desk, logged into my system, wrote some code and ran it. It worked like a charm! I saved my work, finishing up just in time for dinner. A while later, it was again time to program. I went back to my desk, logged into the system and coded away. Just another day of slinging the code, right? Well, not really.

My first coding session was in 1975. “A while later” is today. Forty-four years have passed. Back then, I was working on a mainframe that was 50 miles away. Today, I logged into an online programming environment that was also miles away, maybe even halfway around the world. That’s how it is with cloud computing: The resources are any way at any time.

Interestingly, despite the passage of a significant amount of time, my current developer experience is not that different than the one I had decades ago. It’s still just me and a terminal. Back then, the terminal was a screen full of green characters. Today, my terminal is able to accommodate a variety of media and input devices. But, at the other end of the wire, who knows what’s going on? For all I know, I might still be connecting to the same mainframe I used back in 1975. Which is my point.

Much of what we consider new and wonderful about cloud computing today has actually been around for a while in mainframe technology. In many ways, the mainframe was the cloud before the cloud. And, it’s still a powerful resource for what we have come to know as modern cloud computing. Given its power, versatility, scalability and support for modern DevOps tooling, the mainframe is a perfect resource for today’s and tomorrow’s cloud computing infrastructure. It’s a computing resource that we’ll do well to take advantage of as we move forward in our development efforts on an increasingly connected planet.

— Bob Reselman

May 1, 2019
Red Hat Takes Virtualization Aim at VMware

Red Hat, as part of on going drive to wean enterprise IT organizations off proprietary virtualization technologies, today launched a set of Red Hat infrastructure migration tools and services.

Joe Fernandes, vice president of cloud platforms products at Red Hat, said that as organizations move to modernize their application environments by lifting and shifting them to the cloud, they are simultaneously looking to take greater advantage of open source virtual machines and containers specifically as an alternative to VMware.

The Red Hat initiative includes a consulting engagement as well as a pilot project that enables legacy application workloads to be deployed on kernel-based virtual machines (KVMs), an instance of OpenStack or the Red Hat OpenShift platform based on the open source Kubernetes container orchestration software. The core tool used to migrate those workloads is the Red Hat Ansible Automation, Fernandes said.

As management software stack the company now provides on top of those foundations has matured, Red Hat is finding more customers willing to consider transitioning away from commercial virtualization software, he noted. At the same time, he added, wider adoption of KVM by cloud service providers (CSPs) is making open source technologies a much more attractive option to foster deployment of application workloads across a hybrid cloud computing environment without having to refactor them.

Those two trends in combination, Fernandes said, are making it easier for Red Hat to make a case for organizations to move way from VMware.

Red Hat may have lost the virtual machine wars to VMware early on, but it’s clear the company is gearing up for a new phase. Many organizations have adopted an open source-first philosophy that Red Hat clearly is trying to exploit at VMware’s expense. At the same time, rising interest in alternative approaches to virtualization in the form of Docker containers and Kubernetes is creating another wedge that Red Hat is looking to exploit. For example, the company is testing Kube-Virt, an open source project that makes it possible to run KMVs within a container. Red Hat has already signaled its intention to apply that technology to other hypervisors as well.

It’s unclear to what degree containers and Kubernetes will be deployed on virtual machines and used in place of virtual machines altogether. Red Hat is prepared for either eventually, but Fernandes said virtual machines will continue to remain relevant in enterprise IT environments for many years to come. Reliance on container and virtual machines will come down to both the nature of the application workload and the relative cloud-native maturity of the organization deploying them, he said.

It also remains to be seen the degree to which VMware can stave off a new virtualization challenge from Red Hat. VMware has allied with Amazon Web Services and IBM in the public cloud, while partnering with sister company Pivotal to embrace Kubernetes running on top of VMware. The issue, of course, will be how much enterprise IT organizations are willing to pay for commercial virtual machine licenses in support of the initiatives.

— Mike Vizard

August 23, 2018