devopsy.kinsta.cloud

Tag: cloud

Survey Finds More Data Disruption Occurring
A global survey of 1,000 IT decision-makers with more than 250 employees conducted by Dell Technologies finds on average organizations managed 13.53 petabytes (PB) of data in 2019, a nearly 40% increase since 2018.

The same survey also finds that as the amount of data has increased, so, too, has the cost of downtime. Those costs increased by 54% in the same period, resulting in an estimated total cost of $810,018 in 2019.

The survey also notes 82% of respondents suffered a disruptive event in the last 12 months, with the total cost of a data loss now on average exceeding $1 million.

Not surprisingly, the survey finds that a lot of the data that organizations are trying to protect resides in the cloud, thanks mainly to the rise of a variety of application use cases. The top five areas of investment cited by survey respondents include cloud-native applications (58%), artificial intelligence (AI) and machine learning (ML) (53%), software-as-a-service (SaaS) applications (51%), 5G and cloud edge infrastructure (49%), and Internet of Things/endpoints (36%).

Nearly, three-quarters of respondents (71%) noted these emerging technologies create more data protection complexity and 61% said they increase risks. The survey also finds 81% of respondents said their existing data protection solutions will not be able to meet all future business challenges.

Specific areas where existing data protection platforms are expected to fall short include being able to recover data after a cyberattack (69%), recover data from a data loss incident (64%), comply with regional data governance regulations (62%), and meeting backup and recovery service level objectives (62%).

Among organizations that have adopted these emerging technologies the percentage of organizations struggling with adequately protecting data is as follows:
- 5G and cloud edge infrastructure (67%)
- AI and ML platforms (64%)
- Cloud-native applications (60%)
- IoT and endpoint (59%)
- Robotic process automation (56%)
The survey also finds organizations prefer public cloud/SaaS (43%), hybrid cloud (42%) and private cloud (39%) as deployment environments for new applications. In the case of edge computing deployments, 62% cited private clouds as being the platform that will be relied on most to protect data.

Bob Ganley, senior marketing consultant for cloud at Dell EMC, said the biggest challenge for organizations will be finding a way to centrally manage backup and recovery across multiple cloud computing platforms. Those platforms inevitably will span multiple types of public and private clouds accessed as a service and deployed in local data centers. Most organizations are relying on a “mash-up” approach to cloud computing where there is no unified control plane, so as they add each new cloud platform their total cost increases, he said Ganley, noting it’s now only a matter of time before IT organizations start to centralize the management of those multiple clouds to rein in those costs.

It’s not clear to what degree that centralization effort will revolve around IT automation enabled by application programming interfaces (APIs) and best DevOps processes versus traditional data protection tools based on graphical user interfaces (GUIs). However, given the amount of data involved, it is more than likely that existing approaches to data protection are not going to scale, so a fundamental change in how backup and recovery is managed is now all but inevitable.

— Mike Vizard
March 16, 2020
Why Organizations Are Adopting or Avoiding Serverless

For the inaugural O’Reilly survey on serverless architecture adoption, we were pleasantly surprised at the high level of response: More than 1,500 respondents from a wide range of locations, companies and industries participated. The high response rate tells us that serverless is garnering significant mindshare in the community.

Below we examine three of the most notable findings from the survey.

Why Organizations Adopt Serverless

When we asked if respondents’ organizations had adopted serverless—defining “adopted” as entering into a contract with a vendor to provide serverless resources—we expected a low take rate for this relatively new and developing technology. Interestingly, a higher-than-expected 40% of respondents said they had adopted serverless (Figure 1). Of those respondents whose organizations have adopted serverless, more than 50% made the jump one to three years ago and 15% adopted serverless more than three years ago.

Figure 1: Serverless adoption among survey respondents’ organizations.

When asked about the benefits of adopting serverless (Figure 2), there were some stand-out issues. Reduction of operational costs was the number one reported benefit. Rather than buying racks and racks of servers that need to handle the maximum potential traffic and be idle most of the time, it seems that serverless’ method of paying by compute is proving to be beneficial to the bottom lines of organizations.

This likely goes hand-in-hand with the second most popular reason for adoption: scales with demand automatically. Instead of planning for average or maximum usage, you can deploy serverless and it will scale to current usage. You only pay for what is used. This scaling takes away the worry from random and unexpected traffic spikes or big seasonal traffic.

Figure 2: Serverless benefits among survey respondents.

Serverless Concerns

Educating current staff was the number one serverless concern among respondents whose organizations have adopted serverless (Figure 3). That makes sense—with serverless being relatively young, formal training is difficult to find, specific documentation must be generated and case studies to learn from (while growing) are harder to come by. In addition, organizations sometimes discover that success requires custom features. These features evolve quickly as vendors vigorously compete to attract and retain customers—making up-to-date formal training difficult to maintain.

The second biggest challenge was vendor lock-in. Writing code for one vendor platform does not make it portable or simple to move elsewhere. Because serverless is a nascent space, it seems the market is waiting to see how the issue of portability among vendors plays out.

Figure 3: Serverless challenges faced by survey respondents’ organizations (respondents whose companies have adopted serverless).

Turning to the 60% of respondents whose organizations have not adopted serverless (Figure 4), they cited security concerns as the chief reason they have avoided serverless. Because we’re in an environment where security is a paramount concern, the adoption of any new technology carries a lot of weight. Moreover, serverless introduces a different paradigm for data management where sensitive data is much more dynamic. With valuable business data not being managed or controlled by your company, there will always be concerns around who has access, how safe is it, whether or not the vendor looks at the data or metadata, and whether or not the software is patched or vulnerable to attacks.

Adjacent to security is also the challenge of meeting regulatory requirements such as GDPR. In the cloud on third-party servers, can you guarantee the same levels of security and quality as your own on your own infrastructure?

Figure 4: Reasons why survey respondents’ organizations have not adopted serverless.

Serverless Vendors and Tools

The results in Figure 5 reflect what we know of the cloud market and mirror what we found in our cloud native survey in 2019. Amazon’s early-to-market serverless offering and dominant market position has given them a functional leg up on their primary rivals in the cloud market. However, both Microsoft and Google have ramped up their own serverless offerings and likely retain whatever market differentiation they’ve already developed in the wider cloud market—i.e., customers who already like and use Azure or Google Cloud Platform are likely to find the same advantages hold with Azure Functions and Google Cloud Functions.

Figure 5: Serverless vendors used by survey respondents’ organizations.

Custom tooling ranked number one in serverless tools used by respondents’ organizations (Figure 6). This could mean there’s a market for standardizing or improving existing tools, getting companies to either migrate away from their in-house tools or to convince new customers to avoid spending the time to build something custom. It could also mean that current tools are not addressing everything needed to properly manage and deploy to serverless infrastructure.

Figure 6: Serverless tools used by survey respondents’ organizations.

Conclusion

Based on the survey, we expect the demand for serverless to grow in the near term as another worthwhile infrastructure option for many organizations. With no ties to specific technologies or programming languages, serverless can work with a wide range of tasks, and getting started has gotten easier over time.

You can see additional findings and analysis from the O’Reilly serverless survey here.

To learn more about containerized infrastructure and cloud native technologies, consider coming to KubeCon + CloudNativeCon EU, in Amsterdam. The CNCF has made the decision to postpone the event (originally set for March 30 to April 2, 2020) to instead be held in July or August 2020.

This article was co-authored by Chris Guzikowski. Chris is the senior content director at O’Reilly, where he manages the acquisition and development of content in software architecture and software development. He’s also the co-chair of the O’Reilly Software Architecture Conference. Chris has been working on technical content and technology marketing for more than 30 years.

— Roger Magoulas

March 13, 2020
Better Apps and Better Security When You Shift Left

Tens of thousands of people and hundreds of cybersecurity vendors descended on San Francisco at the end of April. While the RSA Conference was the primary draw, there are a number of peripheral events that go on throughout the week as well, such as the Cloud Security Alliance Summit, the CIO/CISO Interchange, BSides San Francisco and the Qualys Security Conference.

I was able to attend some of the Qualys Security Conference and I had the opportunity to see Alex Mandernack, security solution architect for product management at Qualys, presenting on the need to shift left when it comes to cybersecurity.

Mandernack outlined some of the issues organizations face today when it comes to security. Migrating to the cloud and embracing DevOps culture means more ephemeral, elastic environments and compressed development time frames. He pointed out that these trends make security more challenging, but also described the opportunity they present to change our approach to streamline and improve security.

The Traditional Pipeline is Broken

The traditional model of application development is much more linear. Each app team builds their own image—often in a silo or vacuum disconnected from other teams. Penetration testing is conducted on a sporadic or infrequent basis, and eventually the app is deployed. Vulnerability scans are run on the app in production, and issues or deficiencies are reported back to the developers to address or incorporate into the next build.

The process is slow and cumbersome. It also has the potential to introduce redundancies that waste resources and make the organization less efficient. As companies embrace DevOps and move to the cloud, the traditional development pipeline simply doesn’t work—especially not for security.

DevOps, and Cloud, and Containers — Oh My!

The cloud and DevOps fundamentally change the development lifecycle. Development time frames are compressed, and the process is a self-feeding, iterative circle rather than a repeating linear approach. The traditional approach of identifying and resolving security issues after an application is deployed won’t work in a DevOps environment because by the time issues are identified and fixed, the app itself may have already moved on to the next version.

Cybersecurity and the development pipeline have enough challenges with the cloud and DevOps, but there is also a newer technology trend that exacerbates the problem—containers. Containers are designed to get code quickly and to operate fast. They are also built for scalability—spawning and spinning down hundreds or thousands of containers constantly to keep pace with demand.

There is a diverse array of tools and platforms associated with cloud, DevOps and containers. Organizations often employ a complex collection comprised of some combination of AWS, Azure, Docker, Kubernetes, OpenShift and others. The result can quickly become a black hole when it comes to security—which is why it’s important to push security left into the CI/CD pipeline.

Shift Left to Help Security Teams Do Better

In a DevOps culture, developers and security teams need to think about security sooner. It’s important to get the security tools into the process earlier. Thankfully, with the DevOps lifecycle and the tools commonly used within a DevOps environment, it can be done.

For starters, companies should build a golden image. Developers shouldn’t have to patch and update the base operating system before they even start developing. The golden image takes a baseline operating system such as Ubuntu or CentOS, and completely patches and configures it so everyone has a standard, consistent image to develop on.

With the golden image in place, the next step in shifting security left is to implement testing and scanning in the CI/CD pipeline. Put vulnerability information at the fingertips of developers as they’re developing and implement vulnerability gates in the pipeline to ensure that applications that don’t comply with policies and meet minimum requirements are not allowed to be deployed.

New Role of the Security Team

Part of shifting security left includes providing tools that make security self-service for the DevOps team. Automating actions with scripting, APIs and CI plugins ensures that security is simplified and streamlined to provide value for developers.

In this shift left scenario, the security team has to redefine its role in the process. The security team is still vital, but rather than being the police or a roadblock in the development process, the role of security focuses on verifying and auditing the process. The security framework should have dashboards and live data that enable the security team to monitor activity and identify trends that may need further attention.

Better Security, Faster

There is often conflict between developers and security teams, but there doesn’t need to be. Mandernack stressed that developers don’t want to create insecure apps either.

Shifting security to the left into the CI/CD pipeline streamlines the whole process. Ultimately, the result is higher quality, more secure applications that are developed faster.

— Tony Bradley

March 10, 2020
Extending Cloud to Enable Intelligent Edge
In the last few years, with the emergence of 5G, IoT and AI, cloud providers have recognized the value and the importance of edge, and started building the infrastructure technologies to provide computing and connection between cloud and edge, as well as edge based intelligent services as part of their total cloud value proposition. However, this is a relatively new area of cloud computing with great opportunities as well as challenges.

Top Scenarios of Cloud-Edge Computing

The new set of applications such as 5G, IoT and AI, and maximizing cloud computing resources have been driving the development of edge computing as extension of cloud.

Here are the major scenarios for such cloud-edge computing development:
1. Low latency: For latency sensitive services such as 5G and IoT, running on the edge provides much lower latency and faster response time. We have seen more and more latency sensitive services being built around the public cloud ecosystem.
2. Massive data: With the rapid growth of devices and users, we’re generating massive amount of data every day. It is very costly to move device or user generated data for services to run on the cloud in real time or on a regular basis. Instead, it is more efficient to run services on the edge and to perform local data processing. In some cases, it is the only viable option and can save tremendous network cost.
3. Privacy/security: For some businesses or regions, it is required to store sensitive and/or confidential data locally on the edge (without moving to cloud) to comply with privacy law and/or security regulations.
4. Distributed cloud architecture: At the same time of adopting public cloud services, many enterprises have needs to run part of their business on locally edge servers, which challenge cloud providers to design and provide a distributed cloud architecture to their users.
Top Use Cases: Edge Analytics and Edge AI

One of the top cloud-edge computing opportunities is edge analytics, which allows data analysis and decision making closer to where the data is generated such as IoT devices, edge and gateways while the cloud is still responsible for the overall service life cycle management, service scheduling, data storage or warehousing, and more comprehensive service or data analysis such as big data analytics. The key benefits of edge analytics are reduced latency and increased security of the analytics, and hence, quick business decision making and so on.

Similarly, edge AI enables applications such as autonomous navigation, remote monitoring using NLP or facial recognition, video analytics, and etc. by embedding AI algorithms in AI devices connected to distributed, low-latency and reliable AI services on the edge, while cloud resources can be pulled in for more in-depth analysis. Here is an image recognition example using both edge and cloud resources and technologies: The edge renders the initial image collected from the device, performs initial analysis and determines it is a dog in the image, then the edge sends the extracted dog image, rather than the whole original image file, to the cloud for further analysis. The cloud then utilizes a bigger data set and determines the breed of the dog and sends the information back to the edge as a more detailed description of the image.

Top Challenges

Cloud-edge computing obviously brings many technical challenges to cloud providers, especially in support of heterogeneous computing, network instability and security on the edge.

Here are top challenges we have seen:
1. Limited amount of resources on the edge. The services deployed on the edge need to be more lightweight and less resource intensive.
2. The IoT devices on the edge can be massive and diverse, which can lead to device management challenges for the edge.
3. The connectivity between the cloud and the edge can be unstable. It is important to implement well thought-out network protocols to ensure the reliability of data transmission between the cloud and the edge.
4. There needs to be a streamlined security strategy connecting IoT to the edge, and to the cloud.
5. Since this is still a relatively new area with great promises and competing visions, there are many disconnected technologies, point solutions and fragmented communities addressing this space. This can be challenging for users to determine the best architecture and solution stack for their edge cloud computing use case.
KubeEdge and Volcano, Cloud Native Projects that Extend Kubernetes to Edge and AI

Cloud native computing has been considered as the de facto approach for the cloud, edge and AI application and service development, and Kubernetes has been proven to be the cornerstone of cloud native computing today. KubeEdge, a CNCF Sandbox project launched in July 2018 with increasing user and developer support, extends Kubernetes to the edge. KubeEdge 1.2, just released in February 2020, includes many reliability features that addresses the cloud-edge network challenges and improves the cloud-edge connectivity. Volcano is an open source project that extends Kubernetes to meet the AI and deep learning workloads. Volcano has recently received the CNCF Runtime SIG approval, and is currently applying for CNCF Sandbox status.

Please join KubeEdge and Volcano, and let’s work together to complete cloud-edge-IoT streamlined solutions to meet the opportunities that 5G and AI can bring us today and in many more years to come.

— Anni Lai
March 5, 2020
The Cloud Is Booming — But so Is Cloud Waste

Fueled by the growth in cloud spend, cloud waste will exceed $17.6 billion in 2020.

Over 90% of organizations will use public cloud services this year. In fact, public cloud customers will spend more than $50 billion on Infrastructure as a Service (IaaS) from providers such as AWS, Azure and Google. This boom is, of course, due to the broader adoption of public cloud services. But, it is also caused by the expansion of infrastructure within existing accounts. Not surprisingly, the growth in spending often exceeds the growth in business. That’s because a huge portion of what companies are spending on cloud is wasted.

Cloud Computing Is Expanding

Before getting to the waste, let’s look a little closer at the growth in the cloud sector. Gartner recently predicted that cloud services spending will grow 17% in 2020 and will reach $266.4 billion. While Software as a Service (SaaS) makes up the largest market segment at $116 billion, the fastest growing portion of cloud spend will continue to be IaaS. This will increase 24% year-over-year and surpass $50 billion in 2020.

Typically, I find that about two-thirds of the average enterprise public cloud bill is spent on compute, which means about $33.3 billion this year will be spent on compute resources. Unfortunately, this portion of a cloud bill is particularly vulnerable to wasted spend.

Cloud Waste Is Also Expanding

As cloud computing growth continues and cloud users mature, we might hope that this $50 billion is being put to optimal use. While I personally do find that cloud customers are more aware of the potential for wasted spending than they were just a few years ago, this does not seem to be correlated with cost optimized infrastructure from the beginning. The fact is, it’s simply not a default human behavior.

I frequently run reports for companies having issues with cloud waste. Invariably, I find wasted spend in these accounts. For example, one healthcare IT provider was found to be wasting up to $5.24 million annually on their cloud spend—an average of more than $1,000 per resource per year.

Below is where the total waste is coming from.

Idle Resources: Idle resources are VMs and instances being paid for by the hour, minute or second, that are not actually being used 24/7. Typically, these are non-production resources being used for development, staging, testing and QA. Based on data collected from our users, about 44% of their compute spend is on non-production resources. Most non-production resources are only used during a 40-hour work week, and do not need to run 24/7. That means that for the other 128 hours of the week (76%), the resources sit idle, but are still paid for.

So, I find the following wasted spend from idle resources: $33.3 billion in compute spend, times 0.44 non-production, times 0.76 of week idle, equals to $11 billion wasted on idle cloud resources in 2020.

Overprovisioned Resources: Another source of wasted cloud spend is overprovisioned infrastructure—that is, paying for resources are larger in capacity than needed. That means you’re paying for resource capacity you’re rarely (or never) using. About 40% of instances are sized at least one size larger than needed for their workloads. Just by reducing an instance by one size, the cost is reduced by 50%. Downsizing by two sizes saves 75%.

The data I see in client infrastructure confirms this—and the problem may well be even larger. Infrastructure I see has an average CPU utilization of 4.9%. Of course, this could be skewed by the fact that resources I deal with are more commonly for non-production resources. However, it still paints a picture of gross underutilization, ripe for rightsizing and optimization.

To take a conservative estimate of 40% of resources oversized by just one size, we see the following: $33 billion dollars in compute spend, times 0.4 oversized, times 0.5 overspend per oversized resource, equals to $6.6 billion wasted on oversized resources in 2020.

The Extent of Wasted Cloud Spend

Between idle and overprovisioned resources alone, that’s $17.6 billion in cloud spend that will be completely wasted this year. In fact, it could well be higher. Other sources of waste include orphaned volumes, inefficient containerization, underutilized databases, instances running on legacy resource types, unused reserved instances and more. Some of these result in significant one-off savings (such as deleting unattached volumes and old snapshots) whereas others can deliver regular monthly savings.

That’s a minimum of about $5 million wasted per day, every day this year, that could be reallocated toward other areas of the business.

It’s time to end wasted cloud spend. Perhaps acknowledging the extent of this problem is a good first step.

— Jay Chapel

March 4, 2020
Castles in the Cloud: How to Take Your Business Kingdom Off-Prem with Confidence
Cloud transformation requires migrating all or part of an organization’s virtual assets off-premises, typically to a public cloud. This enables greater efficiency and cost savings, among other benefits, but it brings heightened risk as well. At the same time, many organizations moving to the cloud don’t fully understand the implications in terms of security. Consequently, successful cloud migration requires a shift in mindset.

A Medieval Metaphor for Cloud Infrastructure

Think of your organization as a kingdom surrounded and protected by a castle. Traditionally, this castle would be outfitted with defensive layers designed to protect the residents, the royal family and the possessions within. Protection of the castle is straightforward. A moat surrounds the castle with drop bridges in and out at strategic points (firewalls).

In addition, guards are stationed at each entry point, conducting a thorough inspection of the people, carts and the content that they carry both in and out of the castle (IDS/IPS). The guards also check credentials at the gates and within the castle to ensure patrons are cleared for access and use of rooms and resources (IAM). The perimeter and interior walls are continuously checked for weaknesses that could introduce potential blind spots that are unguarded (vulnerability management).

The castle has a variety of rooms, each with its own function, content and worth. Each door has a lock (AV). Within each room and the correlative hallways exist guards who check credentials, monitor the people who enter and the contents they carry, etc. (behavioral detection and endpoints). In addition, multiple other measures are in place that bolster prevention, detection and response capabilities (DLP, deception, etc.). Information from all of these sources is noted in a central location by a scribe. Guards analyze the information, looking for additional indicators of potential threats to the kingdom (SIEM) and respond accordingly.

Comprehensively, protection of the kingdom requires an overall model and policies that accounts for new threats and tactics from rival kingdoms and thieves. Architecture of security measures are continuously updated and adaptable. In the event of an incident, the guards have designed and exercised a plan to respond.

Understanding the Three Different Models of Cloud

Clearly, this is a basic metaphor for the on-prem approach. Moving to the cloud introduces a new level of complexity to protection spread across three different models: Infrastructure as a service (IaaS), Platform as a service (PaaS) or Software as a service (SaaS).

In the context of our castle analogy, let’s assume the castle has reached productive capacity. Neighboring rivals are creating competitive pressures, offering residents (who pay taxes) the possibility of a better life if they relocate. To keep residents happy, productivity requires expansion beyond castle walls. As production expands, neighboring kingdoms offer different models to achieve productivity. However, each model presents different levels of shared responsibility in protection (referred to as the Shared Responsibility Model).
- IaaS: In this model, the on-premises kingdom (kingdom A) decides to expand all operations to a neighboring castle (kingdom B). The castle itself and the moats are all controlled and provided by kingdom B. However, the infrastructure, moats, hallways, rooms, access lists, monitoring of activity and protection and usage of its contents are the responsibility of kingdom A.
- SaaS: Kingdom A decides to have blacksmithing conducted in a neighboring castle (kingdom B). The castle itself, the moats, infrastructure, hallways and blacksmithing room are all protected by kingdom B for a charge. Kingdom B’s responsibility is to control access to the room, but maintaining who’s on the access list to the room and the protection of its contents and usage are the responsibility of kingdom A.
- PaaS: Kingdom A decides to house and build military operations in a neighboring castle (kingdom B). Kingdom B is responsible for the basic controls, but the configuration and monitoring remain the responsibility of Kingdom A.
With this transformation, Kingdom A faces a more complex security landscape. Innovation and efficiency are critical to meeting the growing and changing demands of their kingdom, but not without additional risks. New operating models bring new risks and potential vulnerabilities to their system(s). Methodologies and processes used to secure their own kingdom do not necessarily work or apply to protecting their new partner kingdoms. Additionally, people–and experience–that understand these new complexities are harder to come by while the thieves and rival kingdoms discover vulnerabilities and weaknesses and begin to exploit and steal valuables that put the entire kingdom and its residents at risk.

Protecting Your Castle in the Cloud Takes a Deep Understanding

Moving to the cloud is almost inevitable at this point, and it carries so much potential to accelerate business outcomes. It takes a different mindset to secure it than what we are used to as an industry–the landscape has been changed from a security perspective. By fully studying and understanding the differences, organizations can continue rapid and secure adoption of the cloud, while reaping the benefits that it promises to bring.

— Sean Blenkhorn
February 28, 2020
HashiCorp Updates Terraform Tool for Azure Cloud

HashiCorp today announced it has updated the instance of Terraform it created to automate the configuration of virtual machines on the Microsoft Azure cloud to give IT teams more granular control.

Version 2.0. of Terraform (TF) AzureRM Provider will allow IT teams to separately configure virtual machines by type for instance of both Windows and Linux.

In addition, version 2.0 of TF AzureRM Provider will allow users to specify custom timeouts for resources. Previously, timeouts were set for one hour by default.

Finally, TF AzureRM Provider 2.0 removes resources, data sources and fields that have been deprecated. TF AzureRM Provider currently provides access to 374 resources and 119 data sources.

Amith Nair, vice president of product marketing for HashiCorp, said the goal is to provide IT teams with a set of tools for automating the configuration of virtual machines on Azure that is more accessible than the tools currently provided by Microsoft.

Nair said HashiCorp is seeing increased adoption of Microsoft Azure, which in turn creates demand for tools that make it easier to deploy virtual machines at scale. In fact, Nair notes that as more workloads move into the public cloud the less resistance there is to IT automation as administrators look for ways to manage IT at what is now unprecedented scale. Of course, many organizations gain their first experience with tools such as Terraform when they first embrace a set of best DevOps practices.

Less clear is how many virtual machines are being configured on top of instances of Windows versus Linux. There is no doubt Microsoft has emerged as the strongest challenger to the dominance of Amazon Web Services (AWS) in the public cloud. Much of that success, however, comes from the fact that Microsoft supports distributions of Linux alongside Windows Server on the Azure cloud. There are, of course, many organizations, such as Walmart, that prefer to standardize on Azure simply because they view Amazon to be a competitor. At the same time, however, there are many companies that employ both AWS and Azure.

Nair said most organizations that have adopted more than one cloud tend to manage them in isolation from one another. However, it is expected that one day more organizations will seek to unify the management of multiple clouds within the context of a truly hybrid cloud computing environment as the total cost of cloud computing continues to rise with each additional workload deployed. Naturally, many of those organizations eventually will gravitate toward tools that can be employed across multiple clouds.

In the meantime, whether it’s IT administrators or developers themselves configuring virtual machines in the cloud, it’s thanks to tools such as Terraform that the amount of time and effort required to set up virtual machines in the cloud is now as negligible as it is. The challenge now is not so much finding tools to automate IT as much as it is navigating all the cultural changes that naturally ensue.

— Mike Vizard

February 24, 2020
DevSecOps: A Renewed Commitment to Secure Delivery, Part 2

If done right, DevSecOps eliminates the cultural roadblocks that often prevent organizations from getting IT security proactively involved with development and operations. Barriers are the last thing any organization should contend with when comprehensive security is an absolute priority at a time when data breaches, intellectual property theft and other cybercrimes can almost immediately cripple a business.

Still, companies are far from perfect. Misunderstandings, ill-defined policies and turf wars can prevent development and operations from collaborating in a way that naturally bakes-in security into their work. To effectively transform a DevOps pipeline into a DevSecOps one, organizations need to change the way people work and collaborate; make processes inclusive of quality, security and compliance activities; and integrate the right tools.

Those three steps are the focus of this two-part series on becoming a true DevSecOps enterprise. We reviewed processes in part one and now we’ll cover people and tools in this article.

For enterprises with deep-rooted DevOps, a renewed focus on security might entail modernizing the release pipeline and involving IT security people even more. For enterprises that are modernizing heritage-monolithic application portfolios and moving to cloud native architectures, pipeline modernization should include a DevSecOps approach that considers containers, microservices and APIs.

DevSecOps done right gives organizations a level of visibility and accountability that helps accelerate coding, testing, production deployments, problem determination and defect resolution–all accomplished because of an organization-wide acceptance and commitment to security that should shatter any and all roadblocks.

Don’t Forget About the Humans in DevSecOps

When was the last time you heard a developer say, “Hey, let’s get the security guys in here, to hear what they have to say?” Reality is far harsher. Collaboration with IT security often triggers visceral reactions that are not exactly positive. IT security is often stereotyped as the organization of “no” and “you can’t do that.”

DevSecOps aims to tear down those cultural barriers. DevSecOps is all about embracing your IT sec organization and even inviting auditors to collaborate early on in the release process.

Ultimately, security is everyone’s responsibility. Being a developer, there was nothing like the mention of security to make me feel super uncomfortable. I don’t believe that developers set out to purposely create insecure code. However, when IT sec is not involved in a collaborative, cooperative relationship with dev and ops, there is an uncertainty that creeps in, and trust and confidence are shattered.

We know developers develop and release code, while operators control and manage the runtime environment. What is the expectation of IT security in this process? What are they looking for? What risks do they need to mitigate when a digital product is delivered? Understanding the objectives of IT security in the delivery process will clarify the steps that must take place in the process and reveal which tools should be used to create a product that’s secure and delivered with velocity and quality.

Understanding what stakeholders do along the pipeline reconciles personal and organizational accountability with tangible value-add in the release process. DevSecOps can be a catalyst to re-examine and provide clarity on who is doing what, when it is being done and why the effort is being exerted.

Clarity might reveal that personas need to be updated to more modern roles. For example, an operations system administrator might become more of a site reliability engineer (SRE). The SRE role involves more proactive activities, earlier in the release pipeline, to infuse greater fault tolerance in code and greater resilience in the environment configurations–and to solve problems by engineering solutions with developers and IT security.

Determine who does what and where. Develop personas that outline responsibilities along the pipeline, from the sourcing of code to production, to create necessary transparency. Value-stream mapping is a good way to capture the current state of a release pipeline and an excellent way to construct a desired state pipeline with the necessary people performing value-add activities to releasing a product.

Use Tools that Provide Necessary Visibility

Tools are the enablers. These are the cogs in the machine that will be running test cases, static analysis, vulnerability scans and integration tests. Tools will automate DevSecOps processes and provide the most critical function: visibility. The ability to observe all DevOps processes and determine their origins in the pipeline is critical. Visibility lessens risk and supports compliance and audits. The process that defines how code is governed through the promotion path should drive the tools that will be integrated into the release pipeline. Most tools serve to automate build, testing, scanning, movement through the promotion path and integration with other tools. It only makes sense that end-to-end visibility in the release pipeline should be engineered in.

Tools, though, must integrate and communicate to achieve end-to-end visibility. In the heritage era of DevOps (basically, the time before cloud native), it was tough to gain visibility because tools came from different vendors and often couldn’t talk with one another. Fortunately, cloud native platforms offer more integrated capabilities. Build and CI tools naturally share information because these capabilities are being refactored into the cloud platforms themselves.

An example of this is the cloud native CI/CD capabilities of Tekton, which leverage and extend the operator framework in Kubernetes. Translation: CI/CD tools are starting to look like extensions of the cloud native platform, in stark contrast to heritage CI/CD tools which were standalone third-party software products. The result of this tool trend is tighter integration and better visibility built into the platform–you don’t have to engineer it.

Companies need to review how effectively their tools communicate and close any visibility gaps. When possible, consider some of the next-gen automation tools and technologies such as Tekton, Kabanero.io and Jenkins-x that bring greater visibility, integration and consideration for cloud native apps which heritage tools have struggled with.

DevSecOps Is a Catalyst for Pipeline Modernization

A modern security framework will let companies take full advantage of the cloud native technologies that are changing DevOps. Granted, most companies won’t completely shift their entire infrastructure to the cloud, nor should they. Most organizations need to identify a partner to help them assess which processes and tools are worth moving to the cloud, and which ones–like a mainframe that runs core business functions–should be left alone.

For those enterprises that have a large heritage application portfolio, DevSecOps could mean retrofitting automation into systems and applications that were often designed and developed without the scaffolding of a secure DevOps pipeline. DevSecOps applies to heritage application, too. After all, what pipeline wouldn’t benefit from the visibility and traceability of tracking a user story to its production release? By automating the delivery of these heritage apps, you’re optimizing the effort of manual activities such as build, test and deploy, and retrofitting the capability of pipeline visibility.

Cloud has changed the DevOps game, newly defining the fundamental technical underpinnings of what an application really is. And it is cloud that’s driving companies to look at the security of DevOps in an entirely new way. With much on the line–the purity of products, compliance, brand reputation–companies need to rebuild their security frameworks so they can take full advantage of cloud and their on-premises programs. They don’t have to do DevSecOps alone.

— Andrea C. Crawford

February 20, 2020
Never Ignore the E-Factor

When cloud burst upon the scene, we were treated to wild-eyed prognosticators telling us about the death of the traditional data center. This is a trend in IT that you would think we’d learn from, but we repeat every couple of years. 4GLs, no, XML, no…Were all going to end development, and yet here we are, with more lines of code and more programming languages in use. It just keeps happening.

These all share two common failings. First, technology in use today may not be perfect, but it is getting the job done, so dumping it for the new silver bullet is an unlikely result either in the long or the short term. For example, we’re many iterations beyond the conception of the C programming language, and yet for some purposes it is still going strong–because it does the job.

But that’s a little tangential to today’s discussion. Today, the other failing is where I’d like to focus. The E-Factor.

Enterprises are a lot of things, but one of the things the successful ones are not is careless. Cloud use in the enterprise was always going to be both multi-cloud and hybrid. Why? Multi-cloud because sole supplier arrangements are riddled with possible issues, and hybrid because every enterprise has something they want to maintain sole control of or something that simply can’t be moved but they want to push toward newer environments.

We were always going to have multiple choices, and successful enterprises were always going to use multiple of them. Containers facilitate this truth a lot. The ability to run containers pretty much anywhere makes them portable, as long as staff is familiar with the infrastructure of a given environment so they can support the container management system in that environment. But even without containers, a cloud portability solution was inevitable.

There is a massive amount of automation going into IT right now, all for the good. Freeing up traditionally scarce IT hours to focus on more and better services is all to the positive. Inevitably, one or more of these automation tools will be hailed as “The End of X.” Ignore them, and keep rocking at your job. AIOps seems most likely to gather this type of following right now. It will offer a flood of new data and consolidated views of that data, but not much else in the near term. Even when decisions start to be automated, full-on automation based upon AIOps feedback still needs code to run against and infrastructure to run on, so no doom and gloom there.

Apply the E-Factor to anything hailed as a silver bullet. If you’ve worked in the enterprise, you know there are so many variables in the datacenter that any silver bullet would have to be a splintering round, each splinter behaving in a completely different manner to address current IT needs. More options are always good, and I, for one, go after new tech because I’m non-stop curious. But it is indeed just more options for your IT needs. Use the ones that will help, ditch the ones that make things worse and ignore the cheering “This time we mean it!” crowd.

— Don Macvittie

February 12, 2020
3 Lessons in Network Visibility from the A-List

The network can be host to multiple performance killers. Here are three worth noting

Like Ed Sheeran as a secret stormtrooper in “Star Wars: The Rise of Skywalker,” some cameos are hard to spot. They can appear out of nowhere halfway through the film or remain a complete mystery until behind-the-scenes footage reveals the masked actor.

The same can be said for guest appearances on your network. It’s tough to spot the hidden malware, application interactions, network delays and other “cameos” popping up, especially when organizations today are grappling with the complexities of public, private and hybrid cloud infrastructure and multi-tier applications. To adequately support a robust and secure digital infrastructure, you need network and application-layer visibility.

With movie awards season upon us, I invite you to sit back, relax—pop some champagne, if you’d like—and I’ll tell you about three cameos that can be easy to miss in the network environment, and what you should do about them.

Hidden Malware

At a pivotal moment in the Oscar-nominated “Joker,” the evil protagonist studies a tape to prepare for his appearance on a popular TV show. On the tape, he sees a comedian appear on the same show—played by Justin Theroux. It’s a blink-and-you’ll-miss-it cameo from Theroux, very much like hidden malware traversing the network without detection.

Whether they’re sophisticated or non-targeted, high-volume criminally motivated attacks can move throughout the network without your knowledge and cause enterprises significant damage and cost, to the tune of $6 billion by 2021. Emotet, described by the Department of Homeland Security as one of the most costly and destructive instances of malware, likely comes to mind as one of the most pertinent examples. It has the ability to move laterally through the organization, spread other damaging malware and constantly evolve.

Spotting hidden malware requires network visibility that spans the entire distributed digital infrastructure, as well as having access to network data to ensure visibility for all security and performance tools. Specifically, organizations need reliable access to data in motion to tap into traffic across cloud and on-premise infrastructure and traffic flowing across 5G and 100GB networks. This type of pervasive visibility allows enterprises to keep up with the speed at which networks and businesses run, as well as with security demands.

Interactions Between Applications or Microservices

When Brad Pitt’s character in “Once Upon a Time in Hollywood” unknowingly ends up at the Manson Family compound, he comes across Dakota Fanning, who plays a key member of the infamous cult. Their meeting is short but meaningful, and sets the stage for additional character development and exchanges down the line. Interactions between applications or microservices are just as meaningful, with 80% of network traffic consisting of such communication on average.

Understanding these interactions is critical for guiding digital transformation (DX) efforts, which are a top priority for 87% of business leaders. Among the goals of DX are streamlining business processes, cutting costs, improving productivity and introducing new business models that redefine industries—or build new ones. These are ambitious goals, and they’re ultimately realized through new digital applications built on intricate microservices-based architectures that need to be managed and secured.

Only with a clear view, and the right kind of data, can you understand the applications’ interactions, performance and security characteristics to accelerate DX. And the only way to get to this view is to look at the data in motion on the network. The ability to isolate specific applications or microservices communication streams for deeper inspection allows infosec teams to easily understand access patterns and put in place effective micro-segmentation strategies. Application developers, too, can benefit from this by better understanding communication bottlenecks between applications and microservices, as well as troubleshoot them.

Traffic Bottlenecks

The latest rendition of “Little Women” follows a non-linear storytelling structure, constantly jumping between two timelines. It can be jarring to viewers when the girls’ father returns from war, revealing himself as the actor Bob Odenkirk. While this moment is meant to be an intimate glimpse into a family coming together after a long separation, it brings the movie-watching experience to a standstill, since the audience associates Bob Odenkirk less with a serious scene and more with his previous comedic roles.

Networks today are similarly prone to experiencing abrupt slowdowns or bottlenecks that limit network performance and waste bandwidth. Managing the network to ensure constant availability is a major challenge when data volumes are growing exponentially. It’s estimated that about 1.7MB of new information will be created every second for every human being on the planet this year. This data explosion paired with digital transformation technology investments and lightning-fast 5G and 100GB networks has led to a very complex enterprise infrastructure setup—and network blind spots across physical, virtual and cloud environments.

The solution lies in having the right kind of data to help you understand network and application performance. By analyzing the network traffic, you get immediate actionable data points that can be used to address trouble spots.

Whether we’re talking big screen or enterprise network, surprise cameos are common and they are easy to miss—and this is particularly problematic when things like malware and bottlenecks pose a threat to network security and speed. For the NetOps and infosec pros tasked with complex infrastructure development, implementation, maintenance and security, you need a clear line of sight into what’s happening within the network. Armed with insights on traffic patterns (both cloud and on-premises), delays in data flow and insecure areas, you can use this intel to write the screenplay (or plan) to support a robust and secure digital infrastructure.

— Shane Buckley

February 10, 2020