devopsy.kinsta.cloud

Tag: cloud

Report Pins Cloud Security Woes on Flawed DevOps Processes

By now it’s become apparent that the bulk of security incidents involving cloud platforms revolve around some type of configuration issue. An analysis of those cloud configurations published today by the Unit 42 research arm of Palo Alto Networks suggests that the root cause of the issue is most likely the templates many developers and DevOps teams rely on to configure cloud infrastructure. In fact, according to Unit 42’s analysis, more than 199,000 templates have medium-to-high vulnerabilities in use on public clouds.

Matt Chiodi, chief security officer for public cloud for Palo Alto Networks, said the issue stems from the rate at which organizations now want to be able to deploy applications on public cloud. Templates greatly accelerate the process but very few of those templates have been vetted by a cybersecurity team.

According to the report, the most commonly used templates are created using YAML files on instances of Kubernetes (39%), Terraform and CloudFormation (24%). The most vulnerabilities were discovered in templates created using CloudFormation (42%), Terraform (22%) and YAML for Kubernetes (9%).

Chiodi noted most templates are created through a simple three-step DevOps process: design, code and deploy. Unfortunately, in the rush to deploy most DevOps teams forget to scan for vulnerability issues. As a result, Chiodi said cybercriminals have become very adept at siphoning off cloud resources to mine cryptocurrencies at low-enough consumption rates to go unnoticed by many IT organizations.

Other significant cloud security issues surfaced in the report include the fact that 43% of cloud databases are not encrypted and that 60% of cloud storage systems have logging disabled, which Chiodi noted makes it impossible to know whether those systems have been compromised. In addition, the report finds 76% of cloud workloads expose SSH (port 22), while 69% of organizations expose RDP (port 3389) and 27% are using outdated versions of Transport Layer Security.

Overall, the goal now needs to be to make it as simple as possible for DevOps teams to do the right cybersecurity thing going forward, said Chioti.

Unit 42 researchers employed a publicly available GitHub Searching application programming interface (API) to find AWS CloudFormation files, Kubernetes YAML files and HashiCorp Terraform files. Hundreds of thousands of templates were then downloaded and analyzed by the Prisma Cloud infrastructure-as-code (IaC) Scanner to identify the insecure configurations. Each IaC file was also sent to an IaC resource checker to analyze the cloud resources being used in the template. Researchers also used Palo Alto Network’s proprietary AutoFocus and Wildfire tools to identify and investigate cybercrime groups through their usage of custom toolsets (malware samples). Those samples were dynamically and statically analyzed to ensure indicators of compromise.

The Unit 42 research makes it clear there’s a lot of room for improvement when it comes to employing best DevSecOps processes in the age of the cloud. The challenge, however, may have little to do with the tools available. Rather, it’s the absence of a structured set of processes through which DevOps teams and cybersecurity professionals can collaborate that appears to be inflicting the most damage.

— Mike Vizard

February 5, 2020
Ten Questions Data Professionals Should Be Asking When Moving to the Cloud

You have finally decided to move your data into the cloud. Whether it was the ability to leverage the power of cloud compute, the promise of cost-savings or the scalability and flexibility that a cloud data warehouse can bring, it’s important to have a strategy in place. In order to set yourself up for success, here is a list of questions that will help you organize your data and ensure your data migration efforts provide business value.

What Is the Outcome I Need to Achieve?

When you are creating a data migration strategy, you need to have an end goal in mind in order to make the best decisions on technology and approach. Simply getting your data into the cloud won’t be enough to take action on that data.

How Many Data Sources Do We Currently Have?

Once you have your outcome in mind, take a comprehensive inventory of all your data sources before you begin the migration process. It reduces your risk of discovering halfway through the process that you have no access to a data source–or that you forgot to include it in your migration plan.

Who Is the Administrator of Each Data Source?

It is likely that you may not have access to every single data source that you are looking to migrate. Make sure to find out who can provide you with access to that data, and fully understand your requests. This person will be able to explain the necessary requirements for that data source and will be your contact to help integrate that source into your migration efforts.

What Data Is Most Important to My Key Stakeholders, and What Do They Consider Important in Getting that Data?

This question might also be: What are the KPIs that drive the business from the data sources? You’ll need to have a good grasp on the metrics that are used for business-critical KPIs so that you have the right data in the cloud to ultimately be transformed for analysis. For example, a retailer might value click-through-rate on emails as a key indicator of attribution for sales data. In order for them to get that information, they would need to provide data from their email marketing system and their eCommerce platform. Keep stakeholders involved in the success and failure criteria of your project so they can help set you up for success. Identify the system of record for the metrics that each KPI is calculated from. Quite often, more than one system in your organization can have a representation for that key metric, but with different values. Align with the related stakeholders as to which is the system of record in those cases.

How Can I Show Business Impact Quickly?

A proof of concept (PoC) in the cloud is always associated with the desire to show results quickly, as the speed and performance that the cloud brings means that you should be able to move faster, right? Asking yourself what you can do to help the business recognize the value of your project can help you assuage any productivity concerns. To create a quick PoC, start by first identifying a small number of typical use cases to test out. Picking a high-value KPI that is calculated from metrics from well known source systems could be a good candidate for that first iteration.

What Are My Security Concerns?

Considering that the cloud can house petabytes of data that enterprise companies collect every day, it is safe to assume that some of that information will be sensitive and confidential. Ask yourself if the data you migrate contains personally identifiable information (PII) and if that information is necessary for the project you are working on. In most cases, PII is not needed for analytics work and therefore, you wouldn’t need to migrate that. You might need to know the brand of a credit card to make a data-driven business decision but perhaps not the actual credit card numbers.

Who Has Access to My Data Now that It Is in the Cloud?

If you are migrating sensitive data into the cloud, these are the security and compliance requirements that come into play. How much data security management you entrust to a cloud provider entirely depends on the needs and requirements of your business. There are several different security models, some of which give you more control, some that hand-off that control to cloud providers—it is important to choose the model that works best for your business. Ensure the vendors you use are SOC II compliant and have all the necessary certifications in place to keep your data safe. Work with your organization’s security group on this. They may have defined an overarching Enterprise Cloud Strategy which can help to provide guidance around the acceptable use of cloud resources for your analytics workloads.

What Do I Do with My Data Once It’s in My Cloud Data Warehouse?

Once your data is in the cloud, what do you plan to do with it? Create a strategy to structure your data in the correct way for your business outcome. Different cloud-built technologies can help structure your data for analytics, prepare your data for machine learning or artificial intelligence projects, or perform data cleansing so you have the most accurate data available. It is important to remember that different cloud data warehouses, data integration tools and data visualization products need to complement one another. Make sure you build your stack for your project to ensure success.

How Do I Save Costs in the Cloud?

In addition to faster time to value, one of the main reasons companies migrate data to the cloud is to save costs. But, how can you ensure you are saving costs while migrating data? You can take advantage of free trials offered by many purpose-built cloud data tools that allow you to allocate, compute and storage as needed. The beauty of a cloud data warehouse is that you don’t need to provision for future workloads. As your needs grow, you can scale up easily and manage costs effectively. Try to avoid using multiple products that do similar things. This will reduce cost, increase adoption of the products and keep your architecture simple.

Is My Team Equipped to Handle Cloud Migration?

Cloud data migration is different than on-prem migration and in such, you and your team will be using technology that you haven’t used in the past. To set your team up for success, inquire about their comfortability using the tools you’ve selected and ensure they have the level of access needed to do their jobs efficiently. If there are gaps in knowledge, you can look to provide the training to help them get up to speed and make onboarding as painless as possible.

If you answer these ten questions, you will be on the right track to begin your cloud data migration project. Keep your business goals in mind and start small—the flexibility of the cloud allows you to scale your project later if needed.

— Arawan Gajajiva

February 4, 2020
IBM Doubles Down on Cloud, But Is Krishna an Interim CEO?

Big news from Big Blue yesterday: IBM CEO Ginni Rometty is stepping down after eight years at the helm. In her stead, IBM is doubling down on the cloud by appointing Arvind Krishna as CEO and member of the board. The real move to watch, though, is the appointment of Jim Whitehurst, Red Hat CEO, as president of IBM and a board member. It’s the first time that someone has held the title of president singularly, and it may portend where the real power will be at Big Blue (and Red).

Krishna was previously head of the cloud and cognitive software division at IBM. He is an IBM veteran, having been with the company since 1990. His appointment sends a strong signal about how important the cloud is to IBM’s future. In some ways, it is similar to Satya Nadella’s appointment at Microsoft. But perhaps it’s not as strong a signal about the importance of the cloud than the $33 billion IBM spent acquiring Red Hat.

It is that $33 billion investment and the importance of Red Hat to the new IBM that leads me to believe Krishna may, in fact, be a transitional CEO. The eventual appointment of Whitehurst as CEO and the rise of Red Hat leadership within IBM may in some ways be a reverse-merger type of thing, with Red Hat eventually running the show.

IBM is a huge beast. A company of Red Hat’s size will find it difficult to change the culture and overcome the inertia at IBM. But frankly, IBM has been transitioning for almost all of Ginni’s time there. While the Enterprise Systems group (Z systems mainframe) is still the cash cow, bringing in a large percentage of revenue, IBM has increasingly tried to be a major cloud player.

The SoftLayer acquisition didn’t quite give the company what it needed, as it was not a public cloud play. It left IBM unable to compete with Google, Microsoft Azure and AWS in true cloud services. After a few fits and starts, IBM decided that Red Hat, with its open source pedigree and cloud strength in both public and private cloud settings, could be the engine that could as it pertains to cloud.

The street has not been kind to IBM over the last few years as a result of the botched cloud play, either. At a time when its tech competitors and the market in general have soared, IBM has actually fallen, as has its revenue.

Mitchell Ashley, founder and managing analyst at Accelerated Strategies Group, agrees with this assessment. “We are seeing a sea change in big tech CEO strategy. Like Microsoft’s elevation of cloud exec Satya Nadella to CEO, IBM is rebuilding its executive ranks from the open source cloud software world with the appointment of Red Hat’s Jim Whitehurst as president and board member,” he said. “How soon the Krishna regime including old-line IBM executives gives way to Whitehurst and other Red Hat executives remains to be seen.”

IBM CEO Change Elevates Open Source from Digital Anarchist on Vimeo.

Ashley was not the only analyst to weigh in.

“IBM bet the farm on cloud with the $34 billion Red Hat acquisition,” said Steve Duplessie, founder and senior analyst at Enterprise Strategy Group (ESG). “It was only a matter of time for the succession. Now the two top dogs running IBM are cloud purists. The old IBM died a while ago and they had to change—I like the move! Their stock lost 26% while the S&P rose over 150%. This lets them remake themselves before it’s too late.”

Changes at the top don’t always portend success. There is a lot of hard work ahead at IBM as the company continues this journey. Competing with AWS, Google, Microsoft and other players in the cloud space is going to be challenging, given the head starts these players have already. But you really can’t count out IBM, ever.

One other thing I wanted to mention: With IBM’s appointment of Krishna at IBM, there now are CEOs of Indian descent at Alphabet (Google), Microsoft and IBM. This is very positive for all persons of not only Indian heritage but for all people of color and nationalities. Kudos to these companies and these men for their success. Now, how about some women of color in these roles?

— Alan Shimel

January 31, 2020
SIOS Technology Automates Rebooting of AWS EC2 VM Instances

SIOS Technology Corp. this week launched SIOS AppKeeper, a tool that automatically reboots instances of EC2 virtual machines anytime they fail.

Michael Bilancieri, senior vice president of products and marketing for SIOS, said SIOS AppKeeper is an extension of the work SIOS has historically done to enable servers and clusters to failover. However, many organizations don’t require that level of availability, so SIOS developed AppKeeper to provide a way to automate the process of restarting virtual machines in the cloud, said Bilancieri.

SIOS AppKeeper also provides a dashboard through which DevOps teams can centrally manage the stream of alerts that are generated when there is an EC2 issue, he said. SIOS AppKeeper is a software-as-a-service (SaaS) application so there is no software to be installed. Instead, organizations log into their AWS account and select which instances and services to monitor and the level of protection they want SIOS AppKeeper to provide.

Pricing for SIOS AppKeeper starts at $40 per EC2 instance. Monitoring and alert capabilities are provided by default. Restart services, reboot instances and reboot instances only if a restart of services fails are all options. Organizations can opt to access SIOS AppKeeper either via the graphical user interface provided or via an application programming interface (API), Bilanieri said.

Previously, SIOS AppKeeper was only available in Japan. Based on data from those customers, the average customer with only three AWS EC2 instances experienced downtime at least once a month. With SIOS AppKeeper, customers were able to reduce downtime by 90%, Bilancieri noted.

SIOS will consider adding support for other public clouds based on customer demand, he said.

Cloud service providers, in keeping with the “pets versus cattle” ethos, tend to view virtual machines as being disposable. However, whenever there is an issue someone on the IT team still needs to take the time to restart a virtual machine, Bilancieri said.

Despite the drive to ruthlessly automate in the management of IT in the age of DevOps, there are still many tasks being performed manually. As the number of workloads in cloud computing environments continue to increase, those tasks conspire to slow IT operations. There are, of course, many ways to automate IT processes in the cloud. SIOS is making a case for a simpler approach to managing EC2 instances that doesn’t require IT teams to set up an entire IT automation framework.

Many IT teams will opt to create their own scripts to manage EC2 instances. However, it’s all too often the case that those scripts are poorly documented and don’t scale especially well over time. In addition, each time the IT professional who wrote a script that the organization depends on leaves the organization, there is no one left to update and maintain the script and new members of the IT team wind up writing yet another script to perform the same function.

There will always be some type of script floating around the IT environment. However, in the age of DevOps, the time to start reducing the number of those scripts that need to be supported might be at hand.

— Mike Vizard

January 30, 2020
DevOps Chat: Security Graph SDK Powers Better Policies, with vArmour

The increasing breadth and complexity of the environments we manage are nothing short of breathtaking. Understanding the security risks and applying policies across the cloud, multi-cloud, private clouds and a plethora of software technologies is a challenge faced by every enterprise.

vArmour recently announced version 5 of the vArmour Application Controller, which opens more significant access to vast amounts of information the Security Graph represents and organizes through its Security Graph and SDK. By streaming cloud, network and agent information into the Security Graph, enterprises now have a centralized understanding of application relationships across their multi-cloud infrastructure, enabling centralized risk and policy management that is simple, accurate and secure.

In this DevOps Chat, Marc Woolward, vArmour CTO and CISO, and I discuss how establishing and assessing policies becomes more manageable through expanded access to information about software infrastructure, systems, environments across the organization.

As usual, the streaming audio is immediately below, followed by the transcript of our conversation.

Transcript

Mitch Ashley: Hi, everyone. This is Mitch Ashley with devopsy.kinsta.cloud, and you’re listening to another DevOps Chat podcast. Today, I’m joined by Marc Woolward, who is CTO and CSO at vArmour. We’re talking about policy and risk of cloud applications, how to manage it. Marc, welcome to DevOps Chat.

Marc Woolward: Thanks, Mitch. It’s great to be here.

Ashley: Well, thanks for flying over from across the pond, too. [Laughter] You’re in the U.S., with your U.K. accent, so thanks for being on.

Woolward: There’s quite a few of us out here in Silicon Valley. To be honest, I think it’s half British.

Ashley: There are, there are quite a few of you here. [Laughter] Would you do us a favor and just introduce yourself, tell us a little bit about you, what you do at vArmour, and just a brief on what vArmour does.

Woolward: Yeah, of course. Alright, well, hi, everybody. As Mitch said, I’m Marc Woolward, I’m chief technology officer and also, in my spare time, the CSO at vArmour. My background is, I’ve been at vArmour for five years or so and I look at kind of the strategic direction, I speak to the senior technologists and our customers to understand their challenges with cloud security, and I lead some functions such as, you know, the data science team, and handle things like CCPA and GDPR in my role as CSO.

So, my background is, before I joined vArmour, I was a technology fellow, and I was the CTO for telecommunications and networking at Goldman Sachs. I think I was there for 18 years. Did lots of things there. One of the interesting parts of the role of a tech fellow at Goldman Sachs is, you kinda get involved in all sorts of things. So, I was a member of the private cloud architecture team, you know, almost a decade ago, we built one of the first private cloud environments. And also, I had an interest in application resilience and application design patterns, despite my background keynote infrastructure. So, that kinda set me up really well for a job in a product company in Silicon Valley.

Ashley: Yeah, very much so, especially the proliferation of cloud and all of that. So, let’s jump into it. I think we’re gonna talk about an announcement that was made back in October—what was it, version 5 of the application controller from vArmour?

Woolward: Yes.

Ashley: Can you tell us a little bit about what that announcement was, and then we can get into some of the effects that you’ve seen in the market from it?

Woolward: Yeah, of course. Well, I mean, first of all, I think I probably should tell everyone a little bit about what vArmour does.

Ashley: Oh, yes. Please.

Woolward: So—yeah, no problem. I didn’t answer that in your last question. So, really, we focus on helping our customers to understand how their applications behave within cloud and multi-cloud environments. So, we have a risk and a policy engine that kind of builds a picture of your application behavior, its dependencies, and suggests the ways in which the application should be secured from a policy standpoint, and we help to make it—we make it simple for our customers to implement safe, secure, simple policies within cloud environments.

So, what version five was all about, underpinning the application controller function itself is a security graph, and affecting that is a graph that describes all of the nodes, you know, the compute instances, the Kubernetes pods, the servers within a multi-cloud environment and the relationships between them. And what we’ve been doing is enriching that graph with all sorts of information about the application, compliance requirements, information about what the application does, is it there, what ___, what realm does it run in, what region does it run in?

And really, what version 5, release five was about was opening that up and exposing more of the information within the graph so that customers can benefit from it more. So, what that means is the ability to plug more and more information into the graph to gain more value out of it. Things like information that reside in CMDBs, information from systems that handle vulnerability management, information from control planes, things like Kubernetes so that that information within the graph becomes richer and richer and more valuable. And it can be used for all sorts of use cases, and it also allows companies to build the policies based upon their view of risk and their view of their applications, whether that’s compliance driven or risk driven or business unit driven.

Ashley: Now, can you kinda dig down and open up the hood a little bit more for us? Talk about how all this information gets into vArmour addition the application controller. Are you taking it from logs, configurations, active applications? What are some of the sources for this?

Woolward: Yeah, and—great question. And what I do, I’ll start out by answering that by saying, in the cloud, we do not have a deficiency of data. So, if you’re running a reasonable sized cloud environment, you have tremendous volumes of logs—you know, cloud trail logs, flow logs, all sorts of event based information. And that’s—well, event based data. And that’s data.

The problem with that is data—it’s unstructured, and it’s in huge volumes and it’s very difficult to kind of glean too much value from it. And what the application controller does is, it takes the data and it transforms it into knowledge. So, it provides you with insight into your application, which makes it a lot easier to make good security decisions.

Ashley: Mm-hmm.

Woolward: So, I’ll kinda come back to that in a minute. But the way that we do it is, we have a distributed telemetry pipeline where we use edge compute functions distributed out to the cloud environments that consume the logs and also consume events off of event buses and APIs. So, when a compute instance is deployed, we pull all of the metadata associated with it.

The edge compute function we call message bus summarizes that data, enriches it, and then distributes it into the centralized graph where we effectively are transforming that data from being the individual flow log information into adding information into an edge within a graph, which represents the relationship. So, all the time, we’re enriching and summarizing, and that enrichment provides context, which makes it a lot easier to understand what your application is all about.

Ashley: Well, and given the dynamic nature of our applications—cloud native, all of that, whether it’s Kubernetes clusters instances, microservices, just knowing what you have is half the battle, more or less what’s this information telling you? So, that’s got to be a value, too.

Woolward: For sure. And, you know, Kubernetes is a fantastic example. Because if you’re capturing that information at the wrong level, if you’re looking at the IP address level or even the pod level, there’s a tremendous amount of noise in there and it’s very temporal and it doesn’t make any sense. If you can consume that at the abstraction, that makes sense to the application, so that would be at the deployment or at the service level and you ingest that into the graph you can build a picture of your application over time.

And then when the, kind of the instantiation that’s just been deployed now varies from that model, you can see it very quickly. So, what we’re trying to do is summarize the information and represent it as—I say represent it as knowledge while continuing to process kind of the very dynamic and very high volume data and look for the anomalies and look for places where it varies.

Ashley: That sounds, too, like a perfect application for a graphing technology to be able to visualize, explore, dig in deeper, back out, look at it at different levels.

Woolward: Yeah. For sure, and you know, like I said, my background of working at Goldman and working in financial services, graph technology is fantastic for describing relationships, and that relates to risk. So, if you look at financial risk modeling, that has been almost the archetype application for graph technology for over a decade. I really would take that same approach and that same principle to cyber risk and cloud security risk.

Ashley: Very good. Now, I know also that you’ve introduced an SDK with this. What are some of the applications where customers might want to use the SDK?

Woolward: The whole point of the SDK—so, we integrate with all of the common cloud environments and we integrate with SDNs and the other product is designed to integrate into these different environments, just as part of the product out of the box.

Ashley: Mm-hmm.

Woolward: But what we’ve been finding with our customers is, they have different repositories for this context that would enrich an understanding of risk. So, if you have a vulnerability scanner, you have a tremendous wealth of information about the posture of the individual systems running in your environment. By plugging that into the graph, and by using the SDK, it’s very easy to do that for whatever system you’re using, you can begin to ask questions like, “Well, for my application, do I have any dependencies on systems with, let’s say, a CB severity of seven or above.” So, the SDK is a way of pulling in the vulnerability information, pulling the specific fields that are of value to you from your CMDB from, say, your service now, effectively plugging in that context that kind of illuminates your whole kind of understanding of your application and your risks.

So, really, it’s about allowing our customers to integrate the data sources that matter to them into this graph that plugs into their cloud.

Ashley: Mm-hmm. So, both in kind of an in and out application—bring in more data, also be able to pull it out for other uses reporting.

Woolward: Yeah. That’s a superb point. I missed that. Another application that customers use the graph for is, embedded within the app controller, we have classification engines. So, what we do is, we will classify systems based upon our understanding of their behavior, based upon the relationships they have and the systems they have relationships with, we’ll classify them.

So, one great use of the SDK is to compare the results of our systems classification with the information within a CMDB. So, you know, does your intent, which is in the CMDB match what’s actually happening in the cloud environment?

Ashley: Mm-hmm.

Woolward: And by doing that, what you can do is, you can validate that your inventory information is correct, is accurate, continues to be so—which, again, improves your security posture because, you know, one of the fundamental pillars of security is understanding what you have, and having an accurate understanding of the systems that you want to secure, so yes, for sure.

Ashley: Absolutely.

Woolward: But, you know, it’s a two-way thing, the SDK.

Ashley: Well, and you’ve got a little bit of time under your belt with GDPR and we have the California Consumer Privacy Act coming up here. How does this kinda technology help satisfy, meet, report compliance—what are the various ways that you could leverage vArmour and the application controller to help you with those challenges to meeting regulatory requirements?

Woolward: For sure. Yes, so, I mean, there were different regulatory frameworks that are more or less specific about the security controls required. So, if you take something like PCI, it’s very specific and it’s very directed in terms of the way in which you structure your controls. So, the app controller can allow you to scope your in scope infrastructure around your card holder data environment and build the layers of controls around the tiers of systems, according to the PCI DSS. So, it’s very good at doing that.

If you look at something more abstract, which talks more about the outcome rather than the means of getting to the outcome—so, if you say it’s something like GDPR where you’re looking at protecting personal information relating to data subjects in Europe or CCPA where it’s more around protecting the personal information around California based consumers, that really comes down to managing risk and ensuring that the control—suitable sets of controls are placed around the systems that either store or process this information. And the first step to do that is to understand the scope of the problem and the scope of the relationships of the systems that are processing this personal information.

So, the app controller, first and foremost, can allow you to understand, you know, the scope of these environments and begin to put controls and reporting around their behavior and access to those systems. So, it all starts from scoping and understanding your environments and ensuring that you can put the controls around the—you know, place the controls around the edge of them, and you continue to monitor that those controls are effective and that, effectively, in the cloud is what the app controller does.

Ashley: Mm-hmm. Very good. Well, I certainly don’t profess to be an expert on either of those two regulatory frameworks. [Laughter] I’m really curious, we’re about a month, month and a half in since you’ve released version 5. As a former product creator myself, it’s always interesting to see what people do with new technology, new versions. But also, what you didn’t expect them to do with it. Have you seen kinda instances of both, you know—yep, we thought we’d help customers this way, it’s worked out that, and then we’ve also seen some interesting ways they’re using it they didn’t think they would.

Woolward: Yeah. So, I’ll start with kind of what we expected and kind of the, if you like, the continued illustration of the value that we designed the product to offer. One of the things that we never—well, I never fail to be amazed by is, you deploy our system within a cloud environment and it begins consuming the telemetry. And within an hour or so, the graph is already beginning to hydrate and you’re finding hundreds or thousands of workloads.

And all of a sudden, the extent of the interrelationship and often the external dependencies, the relationships outside of the VPC or outside of the VNET kind of are visualized. And almost every time, as we sit there for the first few hours with the customer, we’ll get into the same discussion about, “What’s that relationship? Oh, I have a dependency on something in GCP.”

So, generally, that kind of uncovering the unknown and unsuspected dependencies opens up a whole kind of, you know, can of worms for understanding what’s going on here and ensuring that the correct policies are in place and ensuring that everybody understands the extent of the attack surface and the blast radius. So, that, we always see, and that’s just accelerated with version 5.

In terms of what was less expected, that’s the beauty of the SDK. So, the SDK allows the customer to plug in the information and the metadata that’s important and pertinent to them. An area where I didn’t really expect it was actually the application not just to your traditional cyber risk and your attack surface and your blast radius, but asking some interesting operational questions.

So, this use case came up about two weeks ago where a customer was taking kind of an operational term for their applications and plugging it into the graph, and asking the question around, for my critical tier applications, my systemically critical applications, if one of their dependencies, one of the applications that they depend upon for, I don’t know, reference data or for other types of communication, like communications gateways failed, what would be my recovery time?

And all of a sudden, what you can do is, you can draw within the graph the relationship between your application and the dependencies and plug in that recovery time objective, and all of a sudden, I understand that if application A, B or C failed, I might be waiting eight hours for recovery, whereas my system has a recovery time requirement of one hour.

Ashley: Mm-hmm.

Woolward: So, they’re beginning to ask these questions that you would not expect from, necessarily, a cyber tool around operational risk. So, I think that’s the beauty and the power of the graph—it allows the customer to think about, what does risk mean to them, and they can plug the information or plug the data in that they require and then they can ask these questions that lead them to knowledge.

Ashley: Excellent, yeah. It sounds like both information, operational information, but also behaviors of what’s happening between applications and greater insight into it.

Woolward: Absolutely, yeah. Yeah, for sure. And I think the other thing, I mean, the final thing I’d say is, if you look at most of these data breaches over the last year or two, it kinda comes down to, if you look at cloud and you understand the way that security works, it’s this shared responsibility model. So, the cloud service providers provide a tremendous array of tools—you know, micro-segmentation, identity access management policy enforcement. There’s a tremendous suite of security capability there.

The problem is always that the lack of knowledge about the application means that the policies that are written and the permissions that are provided by the customer, because that’s their side of the, kinda the shared responsibility model have been deficient. And what the application controller allows you to do is to build the policy in a simple fashion that’s secure and meets your business requirements, and then that’s allowing our customers to meet their part of the shared responsibility model for cloud security.

Ashley: Mm-hmm. Wow. Well, I feel like we’ve barely scratched the surface and we’re already out of time. Hopefully, we can get you back and we can explore this some more as we get closer to the CCPA coming into effect in January.

Woolward: Yeah, and as the CSO of a California based company, I’m obviously spending a bit of time on that as well, so.

Ashley: I’m sure you are.

Woolward: That’d be fun.

Ashley: Yeah, exactly. Well, Marc, thank you so much for being on DevOps Chat with us.

Woolward: Yeah, it’s been a real pleasure, Mitch. Thank you—thank you for inviting me.

Ashley: Absolutely. The pleasure has been ours. I’d like to thank my guest, Marc Woolward, CTO and CSO with vArmour and of course, thank you, our listeners, for taking your time to join us and listen to this podcast episode. This is Mitch Ashley with devopsy.kinsta.cloud. Have a great day, and be careful out there.

— Mitchell Ashley

January 15, 2020
Perfect Sense Open Sources Gyro to Automate Cloud Provisioning

Perfect Sense has launched an open source project dubbed Gyro that provides an alternative approach to automating cloud infrastructure provisioning and management.

Gyro was developed as an alternative to other open source projects such as Terraform and Pulumi, said CTO Jeremy Collins. It is designed from the ground up around a programming language that allows DevOps teams to define infrastructure as code, and features logic built into configurations to make it easier to create more precise workflows, he said.

Adding logic to create equivalent workflows in rival platforms is more complex, Collins noted, adding the ability to add logic makes it easier to not only provision cloud infrastructure but also to roll it back when necessary.

Perfect Sense developed Gyro to make it easier to manage the infrastructure for its Brightspot content management system (CMS). As such, Gyro can support multiple cloud computing platforms.

In addition to making it possible to add cloud platforms to an existing configuration, Collins said it can be used to create custom commands and define multiple extension points directly within the Gyro language.

Perfect Sense developed Gyro to automate the provisioning of virtual machines. Since then, the company has extended its capabilities to add support for emerging cloud-native platforms such as Kubernetes.

Collins said Gyro provides a single extensible tool spanning a broad range of DevOps processes, from provisioning infrastructure to discovering what services already exist.

Despite the popularity of rival tools such as Terraform, it’s still early days in the adoption of infrastructure as code. Most IT organizations are a long way from standardizing on any one platform for automating the provisioning of infrastructure. In fact, it may not be uncommon for different DevOps teams within the same organization to employ different platforms for provisioning infrastructure.

Gyro’s ultimate success will depend much on Perfect Sense’s ability to foster an open source community. The Brightspot CMS is used by the Los Angeles Times, Walmart, Mattress Firm, Univision, Sotheby’s and US News and World Report, so Collins said there is a base of support for Gyro on which to build a community. Perfect Sense is looking to build a community that will continue to extend the reach and scope of the platform by adding additional plugins.

Regardless of the approach taken, as the number and rate at which workloads are being deployed on cloud platforms increases, it becomes critical for IT organizations to find a way to automate processes. It’s not clear to what degree those workloads will be provisioned by developers themselves versus traditional IT operations teams that have learned how to program. However, one way or another the provisioning of cloud infrastructure is going to be automated.

— Mike Vizard

January 13, 2020
It’s Almost Here: Predict 2020 Virtual Summit
We’re a little less than two weeks away from Predict 2020 Virtual Summit, our annual online event during which we ponder the future of DevOps, serverless, cybersecurity, digital transformation and IT. This year’s event will be held Jan. 23, featuring an impressive lineup of industry luminaries offering up their visions for 2020 and beyond.

Among the keynote speakers for this daylong event are analysts, practitioners, venture capitalists and thought leaders, each with their perspective on the trends, tools and technologies most likely to impact the IT space. Those scheduled to speak include:
- Emmet Keeffe III, Founder, Insight IGNITE, Insight Partners
- Charles Betz, Lead DevOps Analyst, Forrester
- Dr. Chenxi Wang, Managing General Partner, Rain Capital
- Sid Sijbrandij, Co-founder & CEO, GitLab
- Mitchell Ashley, CEO, Accelerated Strategies Group
- Mike Bursell, Chief Security Architect, Red Hat
We also have a full slate of panel discussions featuring industry speakers on topics ranging from IoT security and continuous delivery in a Kubernetes world to the state of analytics and scaling enterprise DevOps. Each panel is sure to be chock-full of lively discussion and insight from those in the know.

Similar to last year’s event, Predict 2020 also will feature a virtual environment with a conference lobby, presentation theater, expo hall and more. There will also be an exhibit hall where you can interact with other attendees and sponsors, watch videos and download more information. It is truly a conference experience from the comfort of your desk.

Registration and Sponsor Info

You can register now at https://predict2020.io. Attendance is free. Download the Program Guide here: https://bit.ly/predict2020ProgramGuide

If you are curious about what the experts, pundits and luminaries think will be the big trends and stories of the coming year, mark your calendar and attend Predict 2020.

— Alan Shimel
January 13, 2020
Five IT Ops-tackles that Stifle Business Growth

I was recently asked about the top five IT obstacles that stifle business growth, and I want to help clear a path too often lined with fear, uncertainty and doubt. In the not-so-distant past, IT was all about the tools. Solutions were more straightforward, teams were less globally connected, and the ever-present data deluge was not changing our business landscape by the millisecond.

With this in mind, I have created a list of the five most common IT obstacles and will pinpoint the single tool you can implement to help alleviate these pain points.

IT Staff Lack Time to Learn New Technologies

This is not to slam our tech teams or leaders; this is just the byproduct of condensed customer schedules, tight purse strings and managing the MOOSE (maintenance and operations of the organization, systems and equipment). The demand on IT teams is high, and we don’t always have the opportunity or the resources to restructure operations for a new tool, as upgrading requires learning new APIs, UI/UX and deployment architectures.

Difficulty Supporting New Technologies Quickly

It is hard to keep pace with IT complexity and manage it properly. New services, applications and infrastructures are constantly being added and there are plenty of vendors out there that will get you locked into solving a single issue versus addressing your macro problems. While the benefits of upgrading are there, the difficulty lies in understanding the complexity of these systems and how to leverage them most effectively.

Lack of Support for Hybrid IT

Hybrid by definition adds complexity, and multi-cloud, on-prem and private/public clouds are only a few ways to make your hybrid dream a nightmare. No matter the combination, understanding different architectures and their components is a must. Additionally, we need to understand how we manage our own data as well as how our vendors are managing it.

IT Complexity

We need to know our machines and architectures well enough to focus limited resources on what impacts the business most. IT will always be complex—there’s always something new, and the amount of information to process is insurmountable and exhausting. Adding context to IT data can help you identify the root cause of overly complex systems. The most difficult task will be to understand the technology well enough to know what to do next.

Inability to Understand Business Impact in Order to Prioritize Tasks

Prioritization is by far the most important aspect of IT. There is a lot of noise, lack of visibility, lack of context, and—the squeakiest wheels of them all—legacy tools. Prioritizing means routing the right people to the right problem to get it solved quickly and elevating IT service visibility, health and risk into the business function.

Let’s Get Digital

IT pros can start to tackle these challenges through prioritization and then automation, or AIOps—the principal of AIOps being to add context to data, using machine processes to enrich the data and bring it to a business level. Understand what is important to the organization and think about the outcomes important to IT. To do this, you can create a realization plan to tackle the low-hanging fruit and create achievable milestones and outcomes. For example, look for opportunities and create a crawl-walk-run phased approach driven by business priorities. Each organization has different needs and different timelines—and this process can be applied to customers, too.

Next, take into account how your team does work and focus on automation as a principle within how you do work. One of the biggest impediments to digital transformation is culture. Remove manual processes, capture tribal knowledge and figure out how to repeat this approach in a consistent way. Additionally, absorb feedback and learn from it. This will drive acceleration when done right, and we all know that better digital experiences boost enterprise value.

Finally, automation is useless if your data is flawed, and accuracy is a key fault for legacy configuration management database systems. Data must be accurate for effective machine-enabled automation. Leverage the tech you have to build an interconnected ecosystem that is fully integrated and automated—and that avoids tool sprawl. Have an open mind to consider new approaches and be brave enough to shed the legacy mindset. Additionally, take into consideration your partners and vendors, and how that team collaborates, to determine your strategy.

The Light at the End of the Tunnel

The path for today’s CIO can be unclear. But you can take bite- (or “byte-”) sized steps to put machines to work and leverage processes that deliver on AIOps. By understanding business outcomes, prioritizing automation as a principle and leveraging trustworthy internal and external relationships, you will have more time in the day for more important tasks. I measure outcomes based on customer experiences, and I ensure great customer experiences by running a great system. While your mileage may vary, moving forward is the most important part.

— Erik Rudin

January 13, 2020
DevOps Chats: InfluxDB Cloud 2.0 Managed Service for Time Series Data

We can’t seem to generate enough data. And as they say, you haven’t seen anything yet! Cloud-based managed services are a natural solution to ingest, store and access large amounts of data from a variety of sources across private and cloud locations. An enterprise solution hosted in the cloud is good, but a developer-friendly, API-based solution with non-enterprise usage-based pricing reaches an even broader audience.

Enter InfluxDB Cloud 2.0. The name just about says it all. InfluxData VP Products Tim Hall joins DevOps Chats to discuss the company’s new cloud-based time-series database offering.

InfluxDB Cloud 2.0 represents that shift to a cloud-based, developer-friendly offering with much greater accessibility. We discuss the use cases; the free and usage-based pricing; the new FLUX language for querying, analytics and data processing; common APIs; and that TICK stack. TICK stands for Telegraf, InfluxDB, Choronograf and Kapacitor. It is a feature-rich conversation, so join us.

Also, check out InfluxData’s Oct. 16th webinar; Optimizing Time Series Performance in the Real World.

Transcript

Mitch Ashley: Hi, everyone. This is Mitch Ashley with devopsy.kinsta.cloud, and you’re listening to another DevOps Chat podcast. Today, I’m joined by Tim Hall, Vice President, Products, at InfluxData, and our topic is their new offering, InfluxDB Cloud 2.0. Tim, welcome to DevOps Chat.

Tim Hall: Hey, Mitch. Thanks for having me.

Ashley: Excellent. Great to have you on the podcast. Would you start by telling us a little bit about yourself, what you do at InfluxData and maybe for somebody who doesn’t know InfluxData does, give us a little bit about Influx.

Hall: Sure. So, I’m the Head of Products here at Influx. I joined almost three years ago. I’ve got a background in big data open source technologies. I spent some time at Hortonworks running the Product Management Team there. They’re focused on the Hadoop space. Prior to that, I spent time at Product Management at Oracle addition HP, mostly in the monitoring and data integration spaces.

So, Influx is a great opportunity to join a small company at the time. We’ve grown to almost 160 people now over the past 3 years, and it was sort of at the convergence of all the things I’d done in my career, from monitoring technologies to integration to big data. And so, it was great to be invited by Evan Kaplan, our CEO, and Paul Dix, our Founder, to join the team and see what I could do to help build a great product.

Ashley: Fantastic. And tell us just briefly—InfluxData, what do you do?

Hall: So, InfluxData is focused on providing a platform for time series data, which is about anything with a time stamp. Time series can be broken into two categories—regular series, which are things that you sample. Think of it like sticking a coffee cup in the river and pouring it into a bucket, and we’re the bucket, and you do that on a frequent basis—every 30 seconds, every minute, every 5 minutes, but it’s regular. It’s a regular time interval. And irregular series are also supported by Influx, which are things like events or logs, things that happen at any point in time that could be somebody doing key card access to a building, putting their ATM card into an ATM machine—those sorts of things. Those are all events.

And so, we handle both of those two things and we have lots of folks using us, both in the open source community, on prem, or wherever they decide to play the software and now our cloud offering as well across a wide range of use cases, everything from DevOps monitoring use cases and building and assembling platforms for observability across to IoT use cases, both for consumer and industrial products.

Ashley: Well, in a world where we just can’t seem to generate enough data, it sounds like you all are well positioned in the market. [Laughter]

Hall: There’s more coming every day, right? So—

Ashley: Yeah, it’s not getting less.

Hall: Yeah, what do you do with it is really the question. And I think, again, we’re focused on providing a platform for developers. And our goal is really to drive developer happiness. And our motivation is trying to allow developers to solve problems quickly from the time of install to problem being solved. We call that the time to awesome. We want that to be as short as possible, and that’s not always the case with a lot of technologies out there. You’re fiddling with dependencies and other things and Influx comes out of the box with no dependencies and install it, get it running very, very quickly and get going and get your problems solved.

Ashley: I do really appreciate your focus on the developer community. Actually, we—I just did a webinar a few weeks back with Tom Crow, your Community Manager, and he did an excellent job, by the way, of talking about how he worked with the developer community.

So, let’s jump to InfluxDB Cloud 2.0. Is this your first SaaS online offering, is that correct?

Hall: It actually isn’t, Mitch, but I’m glad you asked. So, previously, we’ve had InfluxDB Cloud 1.0, which was really taking our enterprise edition and providing a managed service on top.

And the challenge we had with that is, we’ve seen good adoption and nice traction. We’ve had a large number of customers purchase and buy it, but the challenge is that it was a little on the expensive side, and it existed as a dedicated instance of our enterprise software that we ran and managed for them, but it was constrained on specific resources. And so, you would buy a plan, pay for it, and then if you continued to grow and use it, then you would have to upgrade, typically by contacting support, and we could do that in a matter of seconds, but it really required a lot more high level interactivity than maybe you might get from the vast majority of cloud services that you could think of, more consuming like a utility.

And so, our focus with InfluxDB Cloud 2.0 was to deliver the first completely serverless time series platforms, deliver that elasticity for our customers to just adopt and go.

Ashley: So, truly a cloud service, not just an application in the cloud, if you will.

Hall: That’s right, that’s right. And all the capabilities we’re exposing in a multi-tenanted fashion, which means we’re obviously testing and validating across a large group of folks all at the same time, and keeping that platform up and available for everybody is our goal.

Ashley: And I know you’re doing more than just storage in the cloud. You have visualization, UI—other capabilities? You wanna talk a little bit about that?

Hall: Yeah, that’s right. So, you know, there’s sort of three parts to this story, right? Part number one is, you need to be able to feed the data in, and you need to be able to do that with velocity, and we need to store and land that data and we need to do that quickly and make it available for query and access and all the way through to visualization.

And so, we have all those parts of the story together. The most recent addition also includes the ability to create monitoring checks that allow you to create queries against the data that you’ve stored and decide if they, let’s say, exceed a certain threshold or create a change of state in some way, shape, or form. And then from there, you can create a notification that can be sent to you through a variety of channels including HTTP POST, Slack—which is available in our free tier—and then, obviously, PagerDuty, which is a super popular mechanism for alerting your DevOps teams.

Ashley: Mm-hmm. And I think, in addition to kind of integration level type functionality, you’ve also updated the API to InfluxDB in the cloud, correct?

Hall: That’s right. Actually, one of the goals is, you know, through the 1.0 line of the TICK stack, T, I, C, and K, which many DevOps people might be familiar with, there were four parts—Telegraf, InfluxData (the database itself), Chronograf (which is our visualization tool) and Kapacitor (which was our anomaly detection).

And we’ve really taken the I, C, and K portions of that stack and integrated them all together in our InfluxDB 2.0 offering. This is both open source as well as cloud. And what we wanted to do was bring together the language, first and foremost, that you use to interact with data, both from a query and a task perspective, and we’ve done that through a new language you’ve introduced called Flux—not surprising, the name of the company is Influx. So, Flux is the new language we’ve introduced. And then we’ve obviously kept Telegraf, which is our data collection agent that can be distributed and used to send data in.

Now, one of the other things about T, I, C, and K is, each one of those elements had its own API. And so, if you worked with one, it didn’t necessarily translate to the other. So, we’ve unified and have created a common API as part of InfluxDB 2.0 that allows you to do everything from drive the creation of dashboard sales and dashboards themselves all the way through to submitting queries, checks, notification rules, et cetera, all behind a common API.

And that API is consistent across the open source and the cloud editions. That allows [Cross talk]—yeah, sorry, that allows developers to essentially write an application, either against the cloud or against open source. And if they need to swap it around for any reason—for example, if your journey starts with the open source and you’re like, “Okay, now I need the scale and elasticity offered by InfluxDB Cloud 2.0,” they don’t have to make any changes to their application code. They can just change the end point and point out our cloud.

Ashley: Very interesting. You know, we’re building applications that are API centric much more today versus let’s expose the application through an API. Did that change how you redesigned the API the way you’ve integrated all these components and does just kind of InfluxDB in the cloud more or less use its own APIs because it works together across these functions?

Hall: It does. We’re actually consumers of our own APIs. And frankly, that’s been our philosophy for a while. The challenge had been previously that it was fragmented across the four components of the TICK stack and we’ve really tried to unify that and think about, again, delighting developers and giving a consistent API experience across them all.

Ashley: One of the things you mentioned, too, about—and thanks for that information about TICK stack. I’ve got another question about that in a moment. But back to your 1.0 offering versus 2.0—have you changed kinda the pricing model, done anything to make that more accessible to either developers or enterprises

Hall: Yeah, completely. In InfluxDB Cloud 2.0, it’s a completely usage based pricing model. We’re also starting developers off with a free tier, and so that free tier is rate limited, but you have access to most of the features and capabilities. We’ve obviously held off some of those things behind the paywall, but generally speaking, it’s now all pay as you go and we’ve got that pricing model published and up on the website as well as in the documentation that describes exactly how much you pay for reads, rights, storages, and so on, and you only pay for what you use.

And that’s really the goal of providing that serverless offering. People wanna use it as a utility and, you know, compare and contrast with what we did with the managed service in the 1.0 line, you know, they were paying for things they weren’t using, right? It was a fixed price offering and so, you know, that’s why it looked a little more expensive. So, I think now, we’re, I think, price optimized for developers to grow as they’re successful.

Ashley: And one of the things that probably in any developers mind listening to this podcast is thinking about developers definitely don’t like crippleware. So, you’re focused around data retention limitations, rate, that kind of thing. I’m sure you’ve heard that from your developer community about how to properly construct a free tier offering that’s still very usable to them.

Hall: Yep, yep. I think we’ve got a pretty good mindset around that. On the data retention side, we’re currently allowing the storage of data for at least 3 days at 72 hours. That can be a lot of data in the world of time series. And the ingest rates are also, I would say, fairly reasonable exposed. Just to give you an insight, I actually am using the free tier myself. I haven’t arm wrestled by Product Manager to give me access to all of the features and capabilities, although I suppose that’s possible, but I wanted to see what I could do.

Ashley: Mm-hmm.

Hall: And so, one of the things I happened to do with my son recently is, we built a gaming PC for him. And, you know, one of the most expensive components, the two most expensive components in the machine are the graphics card and the CPU.

Ashley: Mm-hmm, yep. I would’ve said the same thing. [Laughter] Do the same project with my sons, but yes.

Hall: That’s right. And so, one of the challenges, of course, is that I wanted to make sure that we installed the CPU correctly and I was worried about it overheating. And so, we’ve got a nice, big, you know, heat sink on there. But I enabled the, what’s called the sensor’s plug in within Telegraf to basically give me access to the CPU temperature data. And I’m taking that and I’m actually sending it to Influx Cloud. And then what I’m doing is, I’ve created a check to check on the temperature and I’ve created warning and critical threshold levels, and then I send the notifications to myself in Slack.

And so, generally speaking, it’s worked quite well, Mitch, but there’s been an interesting side use case and benefit that I’ve found, which is, my son comes home and decides not to do his homework.

Ashley: [Laughter] Yep. I was just wondering.

Hall: [Laughter] So, now I have a very uniquely tuned child monitoring service that allows me to recognize when he is gaming when he should be doing his homework. And he still hasn’t figured out how I’m doing it, so I’m definitely not sending him this podcast.

Ashley: You’re too sneaky, dad. Parental controls and he doesn’t even know it. [Laughter] Too funny.

Well, I’d like to touch back on the TICK stack, because I know you’ve been very active with providing open source software through the Telegraf and InfluxDB, Chronograf, Kapacitor. What’s the plan, then? You said you’ve kinda done a lot to the open source or to the code and built that into this cloud offering. What’s the plan with the open source going forward?

Hall: Yeah, thanks for asking. I mean, one of the core values of our company is that we are strong believers in open source. And, as part of that, we’ve been working on the 2.0 offering for more than a year, and it’s gone through a series of alpha editions, and I think we’re currently up to alpha 18, which matches the features and capabilities that we have available as a GA instance for InfluxDB Cloud 2.0. And we may have some community members out there scratching their head, saying, “Um, so, what gives? How come this hasn’t gone GA?”

And I wanna be really, really clear, because we have such a large community that we value their feedback, but I also value not giving them something that’s broken and something that’s not ready for prime time. And so, we’ve been very careful about labeling the 2.0 code line in our GitHub repository as alpha code as we work to complete the features and capabilities that we want to have within the 2.0 line.

And so, the key things that are coming up for us next, there are sort of three things. Number one is, we’re moving towards sort of the feature completeness for Flux, our new language. We’re learning a lot of things about running Flux within the cloud environment and getting feedback from community members on that and there’s probably a handful of things that we’d like to finish there before we’re gonna call that sort of release one of Flux.

Second is, we really need to create migration tooling, and there’s two things that we’re doing in that regard. Obviously, folks that want to land and store their data for longer periods of time will want to move their data from InfluxDB 1 to InfluxDB 2. And we’ve created sort of a new model inside of InfluxDB 2, which includes the notion of a bucket, which is where you place your data, and how to secure that bucket. And so, there are some migration tools that are required to bulk move that data across.

And then last is, I mentioned, we’ve introduced a new primary language for working with data inside of InfluxDB 2, and it unifies the ability to both query that data for dashboard creation and report generation and these other use cases that people might build on top, but it also is the same language that’s used for tasks and sort of creating batch operations.

And that was certainly not the case between InfluxDB and Kapacitor, let’s say. Kapacitor used something called TICK Script and Influx was using InfluxQL. And InfluxQL is a SQL like language and it was an easy onramp for gaining access to the data within InfluxDB. And we’re certainly not abandoning that, but what we’re doing is we’re going to create the ability for you to use InfluxQL, but behind the scenes and on the fly, we will transpile that into something that’s executable by the Flux engine, and then return the results to the customer. And this should allow for existing dashboards and queries just to work natively and seamlessly with InfluxDB 2.0 and obviously InfluxDB Cloud 2.0 when we offer it there. And when we reach that point, then we will declare beta, which means we’ll have feature completeness which include those migration tools and then that’s the time for the community really to start piling in and trying it out, and then we’ll be focusing more on performance and sort of user experience changes between the beta declaration and when we declare 2.0 GA.

But in the meantime, we’ll continue to add features to the cloud edition, they will show up in the open source through our alpha cadence. But I would say that those three big things, you know, the migration tooling, InfluxQL support, and just completing Flux as a generally available language in version one is where we’re focused.

Ashley: Okay, very good. Well, certainly a well thought out strategy and you’ve done a lot of work that we’d look forward to seeing in the open source path as well.

You mentioned buckets, remind me to ask—what types of clouds are you offering the Cloud 2.0 service in?

Hall: Yeah, great question, Mitch. So, currently, we’re available in AWS and there’s two regions we’re available in—U.S. West 2, which is up in Oregon, and we just started the beta process in Europe, which is available in Frankfort, and it’s currently using the same code, but we just haven’t turned on the ability for folks in Europe to pay us yet. But we’ll run that beta probably for four to six weeks in Europe before announcing GA there.

In addition, we announced at Google Next earlier this year that we would be available on GCP. And so, our intent is to continue to drive forward. Most likely, that will appear in the U.S. East region, just to sort of fill out a geographic strategy, but we’re very excited about working with the team at Google. Some of my former colleagues from Oracle are now there, which is fun to reconnect with them.

Ashley: Mm-hmm.

Hall: And we’ll see that land hopefully by year end, and then we’ve been spending time with Azure on the Microsoft team, and so, I would look for that in Europe and Cuba.

Ashley: Okay. Very good. I’m really tying back to the very beginning when you told us about your background—I’m really interested in what do you see are the differences in the approach with InfluxData takes with the developer community being such a focus for the company versus a more traditional database company like you worked at at Oracle? Not that they don’t have developer products, they do, but clearly, this is the center of the universe for you. What are some of the things that you’ve learned or maybe differences in the approach you’re taking now?

Hall: I’d say it’s a mix, Mitch. Like, there are definitely some things in the Oracle landscape that were powerful and can apply to the open source community.

So, for example, feature flag items—so, don’t ship a new version of the database with all your latest and greatest and coolest features turned on by default. That really prevents people from moving rapidly from, you know, from older editions to newer editions. So, we try to keep those things turned off and provide highlights to folks and release notes about how to activate them and turn them on. And it is a double-edged sword, because people who are new that come in, you’d like them to use the latest and greatest and best technology and we do try to highlight that. But we also want people to move and eliminate that sort of long tail of support as quickly as possible.

So, a very smooth process. And this sort of came from Oracle—yeah, feature flag those new things, definitely highlight it so they can take advantage. But let the developers—they’re smart. They’ll take advantage of those new features if you just tell them where they live and how to turn them on.

Ashley: Mm-hmm.

Hall: On the flip side, on the community side, you know, listen to the community feedback. And I would say that’s one thing that’s been top of mind, if not totally obvious is, you know, the motivation behind introducing Flux as a new query language was not done lightly. But it really is in response to all of the outstanding query related questions and issues that were opened by our community members. They could not move certain workloads, they couldn’t do certain kinds of functions, they needed more flexibility with the language. And so, Flux is the genesis of listening to all that feedback.

You know, unfortunately, some of those issues have definitely stood out in the community for, you know, now going on two or three years. In some cases, I finally feel like we’re at the point where we can deliver on those requests from our community members.

Ashley: Very good. I feel like we’ve covered a lot of ground on our podcast today, certainly giving our listeners a great feel for what’s unique and new and also how to get started with Cloud 2.0 InfluxDB. I imagine on your website, there’s a pretty easy place to get started?

Hall: Yeah, absolutely. I think it’s the primary call to action on the website today. If you go to InfluxData.com, it’ll pop up and say, “Get started with Cloud 2.0 immediately.” Also, you can get the open source by going to our Downloads page, or you can find us on GitHub.

Ashley: Excellent. Well, Tom—Tim, [Laughter] I knew I would do that—Tim, thank you for being on the podcast today.

Hall: [Laughter] You’re welcome, Mitch.

Ashley: Okay. I owe you one. [Laughter] I’d like to thank my guest, Tim Hall, Vice President, Products, at InfluxData for joining us today—and, of course, thank you, our listeners, for joining us as well. This is Mitch Ashley with devopsy.kinsta.cloud. You’ve listened to another DevOps Chat podcast. Be careful out there.

— Mitchell Ashley

November 20, 2019
Microsoft Extends Azure Reach to On-Premises IT Environs

Microsoft today at its Ignite 2019 conference previewed an Azure Arc management service that extends the reach of the Microsoft cloud to on-premises IT deployments.

Azure Arc will be made available on both local Windows and Linux servers as well as Kubernetes clusters or an instance of Kubernetes that Microsoft is making available on Azure Stack for on-premises IT environments. With these announcements, Microsoft is, for the first time, extending its hybrid cloud computing strategy beyond systems running a version of the Azure operating environment on an on-premises IT environment.

Jason Zander, executive vice president for Microsoft Azure, told conference attendees Azure Arc creates an opportunity for IT organizations to extend the best DevOps and cloud management practices employed on Azure to on-premises IT environments. Rival cloud service providers, he noted, are only paying lip service to hybrid cloud computing.

In addition to extending the Azure management plane to on-premises IT environments, Microsoft today extended its portfolio of edge computing platforms capable of running instances of Azure Stack to include previews of commercial series and rugged series for Azure Stack Edge. Configurable with graphical processor units and field-programmable gate arrays (FPGAs) capable of running machine learning algorithms in harsh environments, the ruggedized instance of Azure Stack Edge even includes a portable server designed to be carried in a backpack.

To complement those systems, Microsoft is also previewing a range of additional services for Azure Stack, including a fully managed, real-time data ingestion service designed to stream millions of events per second from any source to build data pipelines the same way the company does on the Azure cloud; a cloud database management service for Microsoft SQL Server and Postgres; and an Arc-enabled application programming interface (API) management platform.

While Microsoft generally would prefer organizations to adopt Azure Stack as an on-premises extension to Azure cloud, not many organizations have the robust modern server infrastructure in place required to run Azure Stack. Azure Arc is an effort to extend the reach of Azure to multiple types of edge computing platforms without requiring organizations to deploy yet another operating system. Alternatively, organizations can employ a diverse range of Kubernetes distributions, of which there are now more than 70 that have been certified by the Cloud Native Computing Foundation (CNCF). Most organizations at this stage have yet to standardize on a single distribution of Kubernetes.

It’s unclear what the appetite is for extending the management plane Microsoft created for its Azure cloud to on-premises environments. However, as the scope of the hybrid cloud management challenge facing organizations grows as various new forms of edge computing manifest, Microsoft clearly sees an opportunity to increase the scope and reach of its cloud platform strategy. Each edge computing instance is essentially a smaller instance of a cloud platform for which most IT organizations will not have the tools required to manage at scale. In fact, many IT organizations might not even attempt to deploy workloads at all on those platforms without something equivalent to Azure Arc to manage and secure those environment first being in place.

— Mike Vizard

November 4, 2019