devopsy.kinsta.cloud

Category: DevOps in the Cloud

Distinguishing Between Cloud Washed and Cloud Native
Don’t fall for the marketing jargon: Running legacy infrastructure on a cloud instance is not actually cloud computing. Unfortunately, “cloudwashing” is common among enterprises as they attempt to jump on the cloud trend.

So, how do we dispel the hype to distinguish pure cloud software as a service (SaaS), platform as a service (PaaS) and infrastructure as a service (IaaS)? In this article, we’ll define cloud washed and cloud native, citing NIST standard definitions and industry expert opinions. We’ll add some cues to help decipher cloud washed versus cloud native architecture as you search for new tech. No whitewashing here; prepare for some hard truths.

What is Cloud Washing?

In recent years, services that simply work over the internet were rebranded with the “cloud” buzzword, perpetuating a tenuous, vague perception of the cloud.

Cloudwashed solutions are legacy, on-premises software held in a virtualized data center and rebranded as cloud software. Such tools originally were not built for the cloud and do not satisfy the NIST definition of true cloud computing.

While non-local hosting can reduce maintenance and server costs, by adopting cloudwashed services you sacrifice the benefits of cloud native. As Clint Boulton, writing for CIO describes:

“… the implementation doesn’t provide automated, continuous upgrades, let alone the multitenancy, self-service, auto-scaling and usage-metering capabilities that have come to characterize cloud services …”

Cloud Native: What is True Cloud Computing?

Cloudwashed is diametrically opposed with cloud native. To understand the true native cloud, let’s turn to the NIST official definition of cloud computing:

“Cloud computing is a model for enabling ubiquitous, convenient, on-demand network access to a shared pool of configurable computing resources (e.g., networks, servers, storage, applications and services) that can be rapidly provisioned and released with minimal management effort or service provider interaction.”

NIST also defines three service models of cloud computing (SaaS, PaaS and IaaS), and specifies four types of deployment models (private, community, public and hybrid).

As you can see, the NIST definition of cloud computing is specific. Authentic cloud services are configurable, providing hands-on optimization. They are elastic, able to provision and support multiple application requirements. Such cloud architectures are able to efficiently scale to the unique computing workloads of various applications.

Furthermore, resources for cloud services are pooled. According to HiTechNectar, “True cloud platforms are built on a true multi-tenant model, with different physical and virtual resources dynamically assigned and reassigned according to consumer demand.”

A cloud-native application (NCA) is thus an application specifically designed for the cloud that satisfies the definition of cloud computing. A true cloud solution would be a proprietary service built on a platform such as CloudStack or OpenStack. Google Cloud, Amazon Web Services and Microsoft Azure are all examples of authentic cloud providers.

Cloudwashing Warning Signs

Cloudwashing is rampant in the ERP realm. As HiTechNectar reports, “Many enterprise software vendors make claims that their applications are built for the cloud when they are not.”

Thus, to avoid cloudwashed software, take heed of these warning signs:
- Overuse of “cloud” in vague marketing jargon.
- The need for custom integrations (this is against self-service).
- No resource pooling.
- Lack of developer tooling.
- Lack of configurability.
- Tooling not planned with redundancy in mind.
- Server arrangement is not distributed.
Good Signs of a True Cloud

Thankfully, finding true cloud computing is made possible with the help from NIST, which lists five essential elements of cloud computing:
- On-demand self-service.
- Broad network access.
- Resource pooling.
- Rapid elasticity or expansion.
- Measured service.
Another good sign is the vendor’s involvement in the Cloud Native Foundation. Companies also dedicated to open source projects tend to be more progressive and transparent.

Dissipating the Foggy Cloud

A recent CompTIA Report found that half of the companies in 2018 have 31 percent to 60 percent of their IT systems in the cloud. Cloud computing is still driving the rise of cutting-edge trends including AI, blockchain and IoT.

Unfortunately, though, not all purported cloud services are authentic. Ruled by an incessant need to stay relevant, tech marketing adopts and overuses certain jargon, and the “cloud” is not immune. Only by understanding the dichotomy between cloud washed and cloud computing can CIOs make educated decisions regarding their tooling choices.

The NIST cloud computing criteria is valuable because NIST is an impartial standards agency. NIST computer scientist Peter Mell stresses the importance of evaporating this fog to reveal the truth:

“… by adopting an authentic cloud, [businesses] are more likely to reap the promised benefits of cloud—cost savings, energy savings, rapid deployment and customer empowerment.”

To avoid being cloudwashed, seek out multi-tenant architectures with self-provisioning, pay-per-use models, virtual instances and linear scalability. Only then will you reap the rewards of an authentic cloud-native environment.

— Bill Doerrfeld
January 2, 2019
AWS re:Invent 2018 Product Launches: An Overview

AWS re:Invent is always rich in product launches. Some products are entirely new, while others are updates or enhancements to existing tools. The 2018 event was no exception. In case you missed it, here’s a drive-thru on some of the best products—and one important statement—from this year’s show.

AWS Marketplace for Containers

Announced at the Global Partner Summit keynote, the AWS Marketplace for Containers is the next logical step in the Marketplace ecosystem. Vendors will now be able to offer container solutions for their products, just as they do with AWS EC2 AMIs. The big takeaway here is just how important containerization is and how much of a growth we see in the implementation of containerized products and serverless architectures in general. Along with the big announcements around AWS Lambda, this solidifies the push in the industry to adopt serverless models for applications.

AWS Marketplace – Private Marketplace

The AWS Marketplace has added the Private Marketplace to its feature set. You can now have your own marketplace that’s shared across your AWS organizations. This is neat and all, but I think what’s even more interesting is what it hints at in the background. It seems to me that to have a well-established marketplace at all, your organization needs to begin its journey on the DevOps trail. This news shows that a good deployment pipeline is really the best way to handle a project, for both external and internal customers.

Firecracker

This looks really cool. Firecracker is a virtualization tool that is built specifically for micro VMs and function-based services such as Lambda or Fargate. It runs on bare metal … wait, what? I thought we’re trying to move away from our own hosted servers? That’s true. However, consider all the new IoT products and features that were announced at the conference and you’ll see there’s still a lot of bare metal, both in use and in development. I don’t think Firecracker is meant solely for large server farm-type setups, but quite possibly for items in the IoT space. The serverless/microservice architecture is a strong one, and this allows that to happen in the IoT space. In fact, I’m currently working on installing it onto my kids’ Minecraft microcomputer.

Andy Jassy Says What?

In the fireside chat with AWS CEO Andy Jassy in the partner keynote, there were several things I found interesting. But one comment stood out from the rest:

I hear enterprises, all the time, wanting help thinking about how they can innovate at a faster clip. And, you know, it’s funny, a lot of the enterprise EBCs I get to be involved in … I’d say roughly half the content of those are enterprises asking me about our offering and how we think about our business and what we have planned in the future, but a good chunk of every one of those conversations are enterprises trying to learn how we move quickly and how we invent quickly, and I think that enterprises realize that in this day and age if you are not reinventing fast and iterating quickly on behalf of your customers, it’s really difficult to be competitive. And so I think they want help from you in how to invent faster. Now, part of that is being able to operate on top of the cloud and operate on top of a platform like AWS that has so many services that you can stitch together however you see fit. Some of it also is, how do people think about DevOps? How do people think about organizing their teams? You know … what are the right constraints that you have but that still allow people to move quickly.

He said DevOps! Apparently, larger companies that are looking to change don’t just want fancy tools and new technology. They also need help getting better at effecting change. That’s absolutely outside the wheelhouse of AWS, and clearly a call-to-action for the partner community.

— Lars Cromley

December 21, 2018
Revolutionize Software Delivery with DevOps and AWS

Technology is constantly evolving—every time we think we’ve got a complete hand-holding experience on some technological aspect, there emerges a better innovation.

When the DevOps solution emerged, organizations were more than happy to incorporate its principles. But somewhere down the line, there came DevSecOps. Innovation is an upward curve that doesn’t see a decline but rather starts another journey of its own.

And that is what most DevOps consulting service providers are striving to produce—innovation. Ever thought about getting a cloud service to operate and innovate a DevOps process? Well, that’s exactly what AWS does to innovate the DevOps process and services. Plus there’s more. That’s Innovation + Benefits!

Benefits the Team ‘DevOps AWS’ Can Offer

Automate – Automation is vital to the DevOps process and more so with the DevOps processes for cloud instances. It can drastically reduce developers and operators work hours dedicated to creating bug reports, generating scripts for specific functions, performing calculations and testing code. AWS helps automate DevOps approach including software testing, adding innovation, besides accuracy, consistency and repeatability. It’s human to err but it is technology that removes any scope for error.

Scale – AWS was designed for scaling operations. AWS brings scalability to operations allowing you to do as many things as your businesses require. It facilitates easily availability of resources to fulfill the requirement at hand. The best example of this is handling traffic spikes and downtimes without the need for manual monitoring.

Collaborate Better – Collaboration is at the very heart of the DevOps process. The unification was essential to make the development process a systematic and organized one. AWS can quickly accelerate this process and make it the way it’s intended to be. It unifies automation and enhanced operations to deliver the promises of DevOps. Gone are the days when different departments would wait on each other for communication; this paves way for improved software delivery.

Deliver Quality with Performance – DevOps as a working principle and AWS suite of services are individual names of efficiency. So when they come together, it’s a powerful combination of quality, performance and remarkable output. Even before the output is out, developers and operators are rewarded with automated testing, continuous delivery and better infrastructure to work with. The entire process is knitted to facilitate a smooth DevOps process.

Secure – Just as we discussed earlier, security has definitely become a part of DevOps. With AWS automation and managed cloud computing, it becomes easier to introduce updates and deploy them frequently. In turn, it makes easy to track changes in the code and ensure that security measures are enough and running correctly.

Some DevOps-AWS combination services that enhance the DevOps process for cloud computing:

AWS CloudFormation: DevOps-for-cloud professionals need to create and release cloud instances/services faster than general DevOps professionals. AWS CloudFormation increases the pace of creating cloud instances by creating templates of AWS resources such as EC2 instances and ECS containers to let you construct the entire setup without much manual intervention.

AWS EC2: AWS EC2 brings security and management features to your DevOps process, making it a more the DevOps AWS duo a more desirable combination to work with. It can run containers in EC2 instances.

AWS CloudWatch: This is an AWS monitoring tool that helps you track every AWS resource. Additionally, it enables DevOps experts to use third-party monitoring tools.

AWS CodePipeline: This service is a popular AWS feature that eases the management of continuous integration and continuous delivery tools. It allows you to integrate tools such as CodeDeploy, GitHub and Jenkins so that you can visually control the flow of apps from scratch to the end.

Not a service or tool but the next on the list is more like a functionality feature:

AWS Instances
AWS creates and adds new instances to its list. The flexibility to customize along with these instances makes it easy to use AWS DevOps together.

Conclusion

If DevOps is a solution, then AWS further enhances that solution to make businesses deliver their best. No wonder our world is being dictated by speed and quality!

— Veritis

December 14, 2018
Understanding Pillar 1 of AWS’ Well-Architected Framework
AWS’ Well-Architected Framework provides a wealth of incredibly valuable best practices and guidelines structured around five distinct pillars that more cloud-native organizations would do well to heed. The first of these pillars, Operational Excellence, recommends a number of particularly valuable techniques. However, successfully adopting the design principles and practices within Pillar No. 1 requires a strong foundation: a company culture that is willing to continuously work to improve.

In practice, this can mean holding high-profile “Game Days” focused on advancing internal procedures, nurturing an environment where engineers and product leaders really collaborate to promote business value while pursuing rapid iterative release cycles, championing operational automation and related approaches. Fortunately, adopting the tenants of AWS’ Operational Excellence Pillar can naturally assist in implementing cultural changes where they are needed.

Design Principles for Operational Excellence

To achieve operational excellence within AWS, incorporate these six design principles:

Perform Operations as Code

The cloud makes it possible to apply the same rigorous practices used to reduce risks in software development (such as version control and automated testing) across your full application environment. Cloud infrastructure allows all aspects of operations to be defined, managed and implemented in software code, spurring a revolution in their automation and consistency.

Annotated Documentation

Without effective automation, maintaining documentation is tedious to keep in sync with reality—ultimately making it challenging to troubleshoot issues when they inevitably arise.

Fortunately, the cloud makes it possible to create documentation automatically as an artifact of the build process. This documentation is detailed and annotated, always up-to-date and can be read by us humans (as well as systems). With automated documentation, engineers can react to issues more rapidly and accurately, and enlist automated systems to do the same.

Make Frequent, Small and Reversible Changes

While agile, iterative development methods have become commonplace in many areas, product-driven organizations tend to stick to large releases. However, operational excellence calls for a systems design that incorporates tightly focused, failure-resistant components in an environment designed for constant change and improvement. Implementing this flexibility allows for more rapid deployment of new iterations—but easy rollback if issues pop up.

Refine Operations Procedures Frequently

Operational excellence requires holding regular meetings to reflect, suggest and implement improvements to operations procedures, the same as those held when practicing any agile methodology. Game Days, where procedures are tested and iterated on, will help build and strengthen excellence as well.

Anticipate Failure

Rather than waiting for failures to strike and then studying the causes in a post-mortem, practicing routine “pre-mortem” exercises that recognize failure points before any harm is done. Game Days can then be used to investigate these scenarios and ultimately bolster resilience. For example, premier cloud-native organization Netflix demonstrates this design principle with its Chaos Monkey project, which is purpose-built to strengthen application resiliency against random instance failures.

Learn From All Operational Failures

Operationally excellent cultures have the openness to share post-mortem findings across the organization without pointing fingers. This broadly shared knowledge can then inform improvements in all areas of the business, not just in engineering and product departments but marketing, finance, and others as well.

Critical Components of Operational Excellence

Embracing the six design principles above paves the way for you to practice operational excellence in three key areas defined by the Well-Architected Framework: Preparation, Operation and Evolution.

Preparation

Operational excellence requires operations teams to possess a clear and complete understanding of the goals for each workload within the complete system, and how those goals will be achieved.

Operational Priorities

This clarity must include knowledge of the workloads the operations team is responsible for, the shared business goals and their role in achieving them and any relevant regulatory or compliance requirements. This information allows teams to correctly prioritize those workloads with greater importance or complexity.

AWS offers these helpful resources for setting operational priorities:
- AWS Support. This including the AWS Knowledge Center, AWS Discussion Forums, and AWS Support Center.
- AWS Documentation, now available on GitHub as an open source project.
- AWS Cloud Compliance.
- AWS Trusted Advisor, which includes core checks for environmental improvements.
Businesses can also enlist AWS Certified Partners—experts selected by AWS to provide consulting, and professional and managed services as an extension of a company’s own operations team—to make sure priorities are set wisely.

Design for Operations

A well-architected workload is designed for operations from the beginning, with consideration for deployment, updates and similar needs. AWS provides the ability to model entire workloads as code, from applications to infrastructure, policy, governance and operations. It also offers a robust set of tools and services for operation design, such as CloudFormation and the AWS Developer Tools. Businesses can also create systems featuring the observability and insights needed to operate workloads by using AWS Cloudtrail, AWS VPC Flow Logs, AWS CloudWatch and more.

Operational Readiness

Beyond the technology used, operational excellence is also a function of elegant, consistent processes and procedures for deploying and operating workloads. This means maintaining accurate documentation, building a well-trained and resourced team of experts and having governance that makes sure everything is done the right way.

It’s also smart to utilize automation when evaluating environments. CloudWatch events can be automatically addressed by scripting procedures created with AWS Systems Manager’s Run Command and Lambda. Configurations can be automated and benchmarked against best practices using AWS Config rules.

Operation

The first step to operational success is clearly defining it with key metrics based on shared business goals, so that events can be addressed with those goals in mind and success can be recognized when (or when not) achieved.

Understanding Operational Health

Because no two businesses are the same, the definition of operational health and success will vary. For example, one company might have workloads that demand low-latency and high-throughput performance to be prioritized over cost. Another might require a focus on cost-effectiveness and high availability. Operations teams must first understand those priorities and how they serve business goals to deliver what is actually needed.

AWS offers tools and services for analyzing workloads to serve specific goals. CloudWatch Logs and Dashboards can provide system and business level views of key metrics. Amazon ElasticSearch and Kibana can offer further visualization of operational health metrics, as can outside tools and services like Logstash and Grafana.

Responding to Events

When events occur, operational excellence means having prepared and even anticipated failures and putting metrics, processes and procedures into action for a quick and effective response. At the same time, teams must have metrics to understand the business impact of workload components to resolve the highest priority issues first. AWS enables teams to script automatic event responses, such as automated rollbacks triggered by failures. Amazon CloudWatch offers a powerful service for these automated responses.

Evolution

Curiosity and a desire to learn and improve are essential traits for operations teams pursuing operational excellence. Every experience should be viewed as an opportunity to learn and to share that new knowledge base. The best teams enjoy learning from failure and by asking for the perspective of other business units.

AWS offers an incredible platform for the analysis and experimentation needed to evolve practices. Amazon CloudWatch and CloudTrail can be combined with Amazon ElasticSearch with Kibana. Data exported to Amazon S3 can enable analysis with Amazon Athena and Amazon QuickSight. AWS also helps share best practices with CloudFormation templates, Chef Cookbooks, Ansible Playbooks, Lambda functions and more.

— Jonathan LaCour
December 11, 2018
Developers: Key to the User Experience in a Multicloud World

It’s amazing how many applications behave as if the network is always available, bandwidth is infinite and latency is zero. While optimistic, this approach to applications is ultimately rooted in fantasy. As it stands, there is a major proficiency gap between application developers and the networks in which they operate—but this can’t continue as our world becomes increasingly multicloud. It’s time for developers to understand the inner workings of the network to close the chasm between applications and the network.

Ships in the Night to Star-crossed Lovers

For the most part, applications and the infrastructure on which they operate have little to no interaction. If you’re not sure about this assertion, try asking any network operator what applications are running on the network. It’s possible they can tell you about key mission-critical applications, largely because at some point the person has had to troubleshoot an application issue. But beyond the top few applications, it’s likely that they are considered largely nameless bits and bytes that fill up wires.

Similarly, if you ask an application developer to explain anything about the network, you are likely to get blank stares. Yes, developers are aware that the network provides connectivity to critical applications resources, but it is atypical for them to know much beyond that. The acronyms that make up a network engineer’s livelihood are largely just jumbled letters—a bit of a nod to the dark arts required to actually make networks work.

This missed opportunity has the potential to be the perfect match. There have been various efforts over the years to bridge the gap between these worlds with movements such as application-aware networking and the somewhat lesser-known network-aware applications. The central thesis of these approaches is: The more these two co-dependent but largely separate organisms know about each other, the more they can adapt and adjust to deliver a superior end user experience. To achieve peak usability, each side must know something about how the other operates.

App Developers, Meet the Network

While technology is making networking more interesting than it has been in years, excitement about the network is largely contained to those directly involved in designing, deploying and maintaining it. This is no fault of anyone, though, because the main appeal of the network is that it remains essentially invisible to the user. The network is a success when you don’t know it’s there.

Consider the network as a power grid. When you turn on a light switch, you’re not concerned with how power is generated and distributed—you only care that it can light up the room, and you are concerned when it won’t. The physical infrastructure that makes this all possible is largely hidden to users and visible only when something goes wrong. Just as physical infrastructure is invisible, the network should be, too.

That said, it is important for application developers to “see” the network and not let it fade into the background that same way that users do. Awareness of the network and its functions is critical when developing applications that will run on this system, as simply invoking an application should also include the requisite infrastructure.

Broader Network Shifts are Required

Of course, no change happens in a vacuum. Application development must be considered alongside the broader industry move toward distributed workloads and the proliferation of multicloud, as it is increasingly likely that applications will be hosted in different environments.

Application developers need to not just understand the network, but also the different contexts in which that network exists. For the infrastructure to be invisible, this means those different environments—on-premises, in public cloud A or in public cloud B—must deliver the same experience, inclusive of both policy and security. This creates a bit of a challenge if the future is to be a tight mesh of interwoven applications and infrastructure. For a group that today assumes 100 percent availability and infinite bandwidth, this would be a pretty massive shift in thinking, and a correspondingly massive additional burden to bear.

Abstraction is the Answer

In the same way that containers serve as an abstraction above the compute environment, there must be a level of abstraction that sits above the cloud environment. This is really what underpins the multicloud movement.

Application developers must ensure that application requirements are agnostic to the underlying infrastructure to ensure the system remains invisible to the user. Likewise, multicloud management must create an abstracted platform capable of translating application intent into device behavior regardless of where that device resides, whether it is physical or virtual, or even whether it is from vendor X or vendor Y.

Taking on the Future of Multicloud

Much too often trends in IT favor silos. As we make the shift toward multicloud, networking will be brought into focus like never before, requiring a different way of thinking about cross-domain management. What is required from application developers and network operators alike is communication and collaboration.

Having a baseline understanding of how the distributed network functions will allow application developers to create products that will ultimately run better. Developers should also be transparent about what they are expecting from the network and what they need. This is the only way to get us closer to where we want to be: operating on a network that is omnipresent and wholly capable of anything we can throw at it.

— Mike Bushong

December 6, 2018
Google Cloud Platform: The Basics for Absolute Beginners
Google Cloud Platform provides powerful tools and services. Why should you pay attention to one of the leading cloud service providers? More and more companies are moving to the cloud to help create software applications and manage data, and Google Cloud Platform is one of the many platforms available for cloud computing, an industry that is poised to break $159 billion in market value by 2020.

There are four basic options for creating applications on Google’s infrastructure:
- Google Compute Engine
- Google App Engine
- Google Kubernetes Engine
- Google Cloud Functions
What You Need to Know About Google Compute Engine

Google Compute engine lets you create instances. These instances are virtual machines with their own set of hardware and software. Basically, your personal instance, or VM, is a virtualized environment that sits on top of the very real partitioned environment on servers in a Google data center.

Google Compute Engine is an infrastructure as a service (IaaS). As such, you’ll be able to control networking, firewalls, disk storage and a lot of granular aspects of how, when, where and why your application works.

The Essentials of Google App Engine

Google App Engine is a platform as a service (PaaS). You don’t have to worry about the infrastructure—this is taken care for you. Your project will scale as necessary according to your needs. Simply vendor your APIs, add custom code, load it into the app engine and voilá! You’ve got yourself a working Google Cloud application.

Simply set up billing, add some developers, put in a dash of code and you’ve got the gist of the full project arc for App Engine.

Google Kubernetes Engine: The Absolute Basics

Google Kubernetes Engine lets you create containerized applications with the open source Kubernetes. Containerized application management is just another way of talking about a kind of specialized VM management.

With containerized apps, you don’t need a full VM to launch distributed software applications. In other words, it’s a way to boost memory and efficiency by having multiple containers sitting on top of a single VM instance. It’s a relatively new way of developing applications.

Google Cloud Functions

Serverless applications aren’t without their dedicated service, either. For all your serverless application needs, Google Cloud Functions has got you covered.

Getting Started With Your First Project

Setting up your first project within Google Cloud Platform can be done with relative ease.

First, you’ll want to get an account with Google Cloud Platform if you haven’t already. This can be done in a few easy steps:
1. Sign up with a Gmail account.
2. Agree to the terms of service.
3. Sign up for the free trial (one year of free membership and $300 credit toward your account).
4. Enter your credit card information.
After you’ve completed those steps, you’ll be prompted to an in-browser tutorial. Don’t skip this. It’s an easy way to get oriented to the platform right off the bat. Follow the tutorial so you can get a feel for how the developer console is organized.

Creating Your First Project

Now it’s time to create your first project, add team members and enable billing. This is fairly straightforward, but it’s important to get acquainted with these pages, dashboards and controls early on.

To create a project, simply click on “create a new project,” then follow the prompts. Every Google Cloud Platform project needs a unique title so make sure you supply one or accept a generated one. Then scroll down to IAM policies. You’ll want to make sure you have the right privileges in place.

After you’ve gone through a few of the tutorials and created your first project, open up the hamburger menu in the upper left corner. When you expand the menu you’ll see a long list of services. App Engine, Compute Engine and Kubernetes should all be visible.

Conclusion

There are four major options when deciding to employ Google Cloud Platform: Google Compute Engine, Google App Engine, the Kubernetes Engine and Google Cloud Functions. All are great options for starting your first cloud project.

Google Cloud Platform is a service that gives you access to cloud client libraries and enterprise-grade hardware. As more and more companies move to the cloud, and more and more data is produced, it’s important to start investigating which cloud solution is right for you.

With the Google Cloud Platform, you have many options, from building atop an infrastructure, a platform or building and managing containerized applications. What you choose to employ is completely up to you.

— Charles Dearing
December 5, 2018
AWS Finally Embraces On-Premises IT

Amazon Web Services (AWS) announced this week it will make available instances of its IT infrastructure available in on-premises IT environments, which it will then manage on behalf of customers. The announcement was made at its AWS re:Invent 2018 conference.

AWS CEO Andy Jassy said AWS Outposts platforms, scheduled to be available in 2019, will come in two flavors: one will run the same stack of software AWS uses to manage its public cloud, while the other will be based on the software stack VMware currently makes available on the AWS cloud.

VMware announced it will make available a managed service of its own for AWS Outposts based on its entire platform stack, in addition to making its management and networking software in the form of VMware Cloud Foundation available on instance of AWS EC2. That move represents a significant expansion of the VMware approach to hybrid cloud computing because AWS EC2 is based on the open source Kernel-based virtual machine (KVM) rather than the VMware hypervisor.

Regardless of path chosen, each Outposts platform will function as an extension of the Amazon Virtual Private Cloud (VPC) service, with connectivity being provided via the closest AWS Region located to that on-premises platform. Both Amazon VPC and AWS Outposts will be tied into the AWS Control Tower service to create a common management plane.

After years of making the case for moving all workloads into the public cloud, Jassy admitted this week that there are instances when performance and compliance concerns will result in some workloads never moving to the public cloud. In response to demand from customers who want to be able to deploy and manage the same infrastructure on-premises as they have in the cloud, Jassy said AWS is now committing to making racks of systems available via an AWS managed service.

The challenge DevOps teams will now face is determining the degree to which they will want to integrate their DevOps processes with managed services provided by AWS, VMware or any number of third-party service providers. In some cases, DevOps teams may simply decide that additional layer of management adds to much complexity. In other cases, the decision might lean more toward allocating more resources to application development, as IT operations spanning hybrid clouds become more of an automated service provided by an external vendor.

Heading into 2019, it appears the specious debate over public clouds versus on-premises IT environments is coming to an end. Not only will there be workloads running in both environments for the foreseeable future, but a recent study conducted by International Data Corp. (IDC) notes that 80 percent of IT organizations surveyed have repatriated workloads back into on-premises IT environments from public clouds. To make matters even more complex, a whole new generation of so-called cloud-native applications based on microservices enabled by containers running on Kubernetes clusters is adding even more complexity to IT environments.

The goal, of course, is to make modern businesses more agile. The irony is that achieving that goal appears to requires paradoxically making IT environments more complex than ever.

— Mike Vizard

November 29, 2018
5 Questions to Ask When Choosing a Cloud-Native Security Platform for DevOps

If you work in DevOps, you’ve likely heard the mantra that DevOps success is all about the right people, tools and processes.
Below, we take a look at how you can address the “tools” part of that equation when it comes to security. Specifically, we’ll discuss which questions to ask and what features DevOps teams should look for when choosing a security tool for today’s cloud-native infrastructures.

What is Cloud-Native, and What Does It Mean for DevOps?

Our discussion is informed by two main considerations. First, we’re going to focus on the security challenges that arise in cloud-native environments—which means those built with technologies that include, but are not limited to, containers, serverless functions and (of course) virtual servers.

Second, we’ll focus on the security needs of DevOps teams. Although DevOps positions may not explicitly involve security, we now live in the age of DevSecOps, and everyone on the DevOps team has a role to play in keeping infrastructure and applications secure. In this article, we’ll consider which types of tools are best-suited to help DevOps teams do that, while also promoting the visibility and collaboration that are essential parts of a healthy DevOps strategy.

So, those are our goals. Now, let’s look at the questions you should ask as you evaluate cloud-native security platforms for a DevOps organization.

What Are You Actually Securing?

This question may seem obvious—so obvious that you may not give it much real thought. But because today’s infrastructures and environments vary so widely, it’s important to step back and figure out what, exactly, you need to secure.

Are your workloads running in containers? Are you also using serverless functions, or do you plan to add them? How are you orchestrating your workloads? (With the native orchestrator provided by your cloud vendor, Kubernetes running as a service, your own Kubernetes build or something else?) Which new cloud-native technologies do you expect to adopt in the future?

Answering questions such as these is important to ensure you choose a security platform that can support all of your current and future cloud-native environments. In most cases, you’ll find that security platforms that are purpose-built to secure a range of environments (not just containerized ones, which are usually the focus of most self-proclaimed “modern” security platforms) are the best and safest fit for your needs.

What Are Your Security Threats (and How Can You Stop Them)?

This is another question that might seem overly obvious. But here again, the fast-changing nature of threats means that it’s worthwhile to take some time to assess what your threats actually are.

Keep in mind, too, that the threats faced by your particular team, or the app you deliver, may be different from those that threaten other teams. This is one place where the DevOps principle of cross-organization communication is crucial for effective security management.

Which Layers Does the Security Platform Secure?

Scanning container images for known vulnerabilities is good. On its own, however, it hardly amounts to a complete container strategy. The same could be said for setting up a firewall or locking down access control.

To achieve true security, you need to secure all layers of your infrastructure (including those managed by other teams) against all vectors of attack. For that reason, you want a cloud-native security platform that is designed for holistic security, not a tool that only secures one or two layers.

Where Does the Security Platform Get Its Vulnerability Information?

When it comes to identifying vulnerabilities, security platforms can get their information from lots of sources. They could look at a public CVE database or at a list supplied by the tool’s vendor.

The best security platforms, however, will pull vulnerability data from multiple sources. After all, if you’re only relying on one data source to figure out where the threats are, you are unlikely to catch them all—and just like in the world of Pokémon, catching them all is one of your main priorities for DevOps security.

How Automated is the Platform?

Automation is the mother of DevOps (or something like that).

What I mean is that without automation, you can’t do DevOps very effectively.

You also, incidentally, can’t do cloud-native security unless you rely heavily on automation. That’s because the highly dynamic nature of containerized, serverless and other cloud-native environments means that trying to interpret all of the data they generate, identify vulnerabilities and react to them manually just doesn’t work. That’s why you want a cloud-native security platform that automates wherever and whenever possible.

You should still expect to have to perform some tasks manually, of course (which is actually a good thing—if we could automate everything, DevOps engineers wouldn’t need to exist anymore). But to the extent possible, your security platform should automate your security-related workflows.

— Sonya Koptyev

November 29, 2018
DevOps and Security: Focus on Importance

Why have I written “DevOps and Security” and not DevSecOps or one of the other names that floats around? Because this post is aimed to apply to everyone—not just those far enough along to have melded some of their security practices into DevOps.

Note: After I wrote this article but before I posted it, Bill Doerrfeld wrote this excellent example/solution post. It’s worth a read.

From the beginning, DevOps has struggled with a couple of touch points that don’t give themselves easily to the “frenetic movement” of DevOps. Security is one of those items. If your organization does manual vulnerability assessments, it is unlikely that this testing slips well into the DevOps process chain.

Most organizations that are dead set on making DevOps their primary dev/test/deliver platform have simply put limitations on security—limiting what long-running processes can be part of DevOps or dropping unaccommodating processes altogether. Some spectacular failures have occurred because of this type of thinking—and more will come—but for many the benefits of DevOps outweigh the risks. Will I get flamed for this paragraph? No doubt. That doesn’t make it untrue.

Given that IT has almost never given security the full leeway/authority that they want—and in shops where IT did, business often stepped in to say no—this really is not a massive change from how we have always done business. Just in the modern development environment, where a ton of libraries come from outside and configuration is software or data, there is a greater risk from running roughshod over the security team.

So what can security do to minimize the risk? Start looking more at enterprisewide and architectural risk. Accept that app or app portfolio team X is not going to give you time/resources to thoroughly vet their product and focus on things such as configuration files and malicious code embedded in that external library “Bob’s Really Good GIF Drawer™” that the team insists on using. Focus on the database configuration being in GIT or a shared (public/private) repository and what mischief might come about should attackers get at that repository. Run some of the increasingly sophisticated checkers against external libraries and review the results. As has been the growing case for years now, use automation to at least minimally check apps for things such as SQL Injection, but focus actual man-hours in places that might impact more of the organization. It might be worth the fight to get a full-on security review for applications or an application portfolio that touches the customer database, but perhaps not so much for the app used to schedule a conference room, even if customers are using it (unless it touches the customer database, of course; then see the first part of that statement).

Worry about network configuration as code, database configuration as code and the whole slew of unique security issues that come from public cloud. They’re manageable, and there are tools to help. Just make sure you have it down, because architecture is going to be the new target. A hacker doesn’t have to hack your DC if you left a public cloud storage bucket misconfigured and the data they want is in that bucket.

In too many organizations, the CISO is increasingly treated as chief compliance officer (assuming compliance does not interfere with DevOps cycles) with less to do with security and more to do with legal protection should security fail. Don’t be that organization. Security doesn’t have to be perfect; it just has to make you a hard enough target that only someone who really has it in for your company will keep trying.

As for Devs doing Agile and Ops doing DevOps, as has been true from the beginning, some of the security burden needs to fall to you. Security doesn’t have its hands in what you do every day, so ask them for guidance, learn from them; make it so they need to be active in every step, then include them in every step, because the security environment changes faster than almost anything else in IT. And when you are implementing monitoring, include security monitoring. In most DevOps shops, the question isn’t, “What can we monitor?” but rather, “In this wealth of data, what do we want to monitor?” Well, the answer should include, “What security thinks will help protect us and our customers’ data.”

As automation continues to increase, this tension will continue to ease, and focus can increasingly shift to locking down the really important bits such as database configurations and repositories. Until then, keep fighting the good fight and protect your customers’ ass(ets).

— Don Macvittie

November 29, 2018
Toyota Research Institute Discusses DevOps Automation and Rapid Innovation

How TRI is using DevOps automation to drive its research and engineering

Toyota Research Institute (TRI) is working to build a future where everyone has the freedom to move, engage and explore with a focus on reducing vehicle collisions, injuries and fatalities. Working tirelessly behind the scenes is TRI’s Infrastructure Engineering team, responsible for designing, deploying and maintaining the infrastructure that makes this possible.

I recently sat down with Mike Garrison, the technical lead for Infrastructure Engineering at TRI, to talk about DevOps automation and cloud-based deep learning. (Full disclosure: TRI is a customer of Flux7)

Note: Mike Garrison and Adrien Gaidon, Machine Learning Lead at Toyota Research Institute, will share more about their experience with Distributed Deep Learning & High Resolution Driving Data at this week’s AWS re:Invent.

Can you start by giving us a broad idea of the project you’re working on at TRI?

Our mission at TRI is to improve the quality of human life through advances in artificial intelligence, automated driving and robotics. The Infrastructure Engineering team supports the researchers and engineers at TRI by making it easier for them to utilize the power of the cloud in a secure, automated and reliable fashion while not slowing them down. We see ourselves as partners who can help reduce the time and burden, so they can focus more on their deliverables.

Have you evolved as a team to enable the rapid innovation that this kind of project needs?

Yes. The biggest change we’re making is an evolution to embrace more DevOps ideas, methods and processes. Automation is a core tenet of DevOps. We are applying that in two ways. First, to reduce tactical, manual IT operations activities; second, to make it faster and easier for our researchers to test their ideas.

For example, we are creating a self-service portal that will allow researchers to easily and quickly provision the AWS assets they need to test new ideas. This helps researchers become more productive as they won’t wait for the infrastructure team to spin up resources. Having a secure cloud sandbox environment enables them to try new ideas, fail fast, destroy the sandbox if needed, and start over. It’ll enable our researchers to innovate at velocity and at scale. Modern cloud infrastructure and DevOps automation is empowering us to quickly remove any barriers that get in the way, allowing the team to do their best work, advance research quickly, push boundaries and transform the industry.

How are you applying the latest technology advancements to help produce cars that are safer, more accessible and more environmentally friendly?

The process of developing autonomous vehicles is data intensive. At TRI we log a massive amount of data—terabytes of data per car per day—from our fleet of test vehicles. Researchers analyze the data and continuously refine machine learning algorithms to ensure the most accurate outcomes possible across a myriad of driving scenarios. Researchers train new models and update test cars for new test runs. We’re also using simulation to virtually recreate traffic situations, adding adverse conditions such as bad weather and difficult traffic patterns. It’s a process of continuous improvement that simply couldn’t be achieved without the high compute power of the cloud.

Supporting this high-powered data analysis is a cloud infrastructure that has been optimized to minimize the amount of time researchers take in this gather, learn, tune and test process. For example, at TRI, we have two driving technology modes—Guardian and Chauffeur. Guardian is designed to intervene only when a driver needs help avoiding an accident, whereas Chauffeur is meant as a truly autonomous vehicle. We developed a single technology stack to support both modes.

And, using the PyTorch deep learning framework, my teammates on the machine learning side developed a series of computer vision models that monitor and control both modes as well. Working together, we are able to collect information from their deep learning models, spinning up new compute and storage resources on demand. We’ve married this elasticity with advanced management and orchestration services to ensure that collected data can be retrieved and analyzed quickly. Through advanced automation, we are able to shrink model training times so they are measured in hours—or even minutes.

What lessons have you learned along the way? Any barriers or surprises that others could learn from?

While DevOps automation requires more upfront work, it pays dividends in time savings, staff being able to focus on strategy rather than tactics, and provides logged history which can be critical for security, problem resolution and much more. If I had any words of advice for others looking to adopt DevOps automation, I would advise them: Don’t try to go back a year or two later and try to reverse engineer what was done. Start on the right foot and automate as much as you possibly can.

— Aater Suleman

November 28, 2018