devopsy.kinsta.cloud

Category: IT as Code

  • Veritas Looks to Reduce Data Protection Costs

    Veritas Looks to Reduce Data Protection Costs

    Veritas has extended the capabilities of the company’s NetBackup software as part of an effort to reduce the total cost of data protection.

    Version 8.3 of NetBackup now includes Veritas Resiliency Platform (VRP), a disaster recovery tool, and CloudPoint, a tool for backing data up on a public cloud. All of these tools are part of what Veritas calls its Enterprise Data Services Platform (EDSP).

    In addition, Veritas has integrated hardened NetBackup appliances and vendor-agnostic immutability for storage via its NetBackup OpenStorage Technology (OST) to help organizations recover from a ransomware attack more easily.

    Veritas is also adding support for 2048-bit encryption and integration with third-party key management systems (EKMS) along with more flexible enhancements to the role-based access control (RBAC) it provides.

    Other capabilities in this release of NetBackup include more granular recovery of virtual machines, the ability to orchestrated rehearsals to verify effectiveness of recovery procedures, and the ability to move data between cloud platforms.

    Veritas has also extended discovery, backup, restore and automation at scale for Microsoft SQL, Oracle and Oracle Real Application Clusters (RAC) to simplify data protection processes spanning multiple classes of databases.

    Finally, Veritas claims with this release, data protection via auto-discovery of resources and load balancing is now 25% faster and the discovery of VMware vCenter and vCloud platforms is 50% faster.

    Doug Matthews, vice president of data protection and compliance for Veritas, said with the expansion of EDSP it’s now much more cost-effective to protect data residing on multiple platforms. In the wake of the economic downturn brought on by the COVID-19 pandemic, there’s much greater sensitivity to IT costs, he said.

    At the same time, Matthews said Veritas is seeing DevOps teams become much more involved in automating data protection processes. Veritas has defined more than 600 application programming interfaces to enable DevOps teams to programmatically invoke its data protection software, he said.

    Cybersecurity teams are also becoming more involved as backup and recovery becomes a core element of ransomware defense strategies, Matthews added. In an ideal world, anytime a ransomware attack occurs, a pristine copy of any impacted data would be readily available on another unaffected platform.

    The convergence of data protection and cybersecurity has been a long time coming. Symantec once owned Veritas in the hopes of achieving that goal. Now Veritas is working with a much broader array of third-party partners to achieve that same goal, Matthews said. It may wind up being the rise of DevOps that finally forces that issue as data protection becomes one more IT task to be automated.

    In the meantime, IT organizations may want to revisit their entire approach to data protection. Chances are good that most of them have multiple redundant data protection tools and services in place that have been acquired over time. Consolidating those functions ultimately not only reduces licensing costs, but it also creates an opportunity to automate a set of mundane tasks that all too often IT staff forget to perform consistently at a time that will prove crucial.

  • RackN Transforms Bare-Metal Servers Into a Cloud

    RackN Transforms Bare-Metal Servers Into a Cloud

    RackN is making available an update to the RackN Digital Rebar Platform that will make it easier to employ tools such as Terraform to configure bare-metal servers.

    Company CEO Rob Hirschfeld said version 4.4 of the platform adds an application programming interface (API) that enables bare-metal servers to be abstracted into resource groups. IT administrators can request a machine based on an attribute map to provision a collection of bare-metal servers as if they were a cloud platform, he said, noting RackN is also making available workflows to enable IT teams to automate the allocation and reallocation processes more easily.

    At the same time, the latest release of RackN Digital Rebar Platform adds support for Trusted Platform Module (TPM) Secure Boot, an industry standard that ensures only software that is trusted can be loaded onto a machine. Most automation platforms don’t support TPM Secure Boot, which forces IT teams to choose between the benefits of automation and having a secure platform. RackN Digital Rebar Platform is the only automation framework to support (TPM) Secure Boot in VMware environments, Hirschfeld claimed.

    Out-of-band management passwords, TLS certificates and VLANs will also be secure by default with this latest release, he said.

    Hirschfeld said IT teams want to be able to provision bare-metal servers as if they were code in much the same way they provision virtual machines. Just as importantly, they require an automation tool that enables them to just as easily de-provision those machines when it comes time to reallocate them to another workload.

    Achieving that goal, however, should not require IT organizations to hire a site reliability engineer (SRE), he said, noting the RackN Digital Rebar Platform is designed to be employed by the average IT administrator. An SRE could define workflows within the RackN Digital Rebar Platform that are reused by IT administrators; in fact, RackN plans to add support for Git repositories that will make it easier to share workflows, Hirschfeld added.

    While the reasons for deploying workloads in the cloud versus on-premises IT environments are varied, the automation advantage cloud platforms historically have enjoyed over on-premises IT environments is narrowing at a time when interest in deploying Kubernetes clusters on bare-metal servers is rising. It may be a while before most IT teams are able to manage bare-metal servers as if they were just another cloud resource. The main obstacle for achieving that goal now, however, is likely to be a lot more cultural than it is technical.

    In the meantime, IT teams should expect to see bare-metal servers employed more widely in the age of containers. There will, of course, always be a need for virtual machines, at least in the foreseeable future. The challenge is the IT infrastructure environment itself is becoming more diverse as new classes of cloud-native application workloads emerge.

  • 3 Key Issues With Hybrid Cloud Transformation

    3 Key Issues With Hybrid Cloud Transformation

    The cloud has fractured. Nowadays, it is nearly impossible to scale a cloud-agnostic digital transformation initiative. The reality is that most enterprises adopt a hybrid cloud arrangement. Whether designed strategically or throughout years of competing vendors and technology influences, hybrid cloud environments, and the problems therein, seem a mainstay for modern IT.

    Many digital transformation efforts are culminating in hybrid cloud scenarios. 451 Research recently found 72% of enterprises are already using more than one cloud vendor. As one can surmise, competing standards and inconsistencies among cloud tooling can cause headaches for developers, security teams and potentially even end users.

    3 Common Cloud Transformation Problems

    I recently met with Jeff Kukowski, CEO, and Grant Ho, CMO at CloudBolt Software, to identify issues within today’s cloud transformations. According to CloudBolt, custom coding, governance concerns and cybersecurity incompatibilities are among the top problems that hold back cloud transformation (or make it very costly). Below, we’ll see what strategies they recommend to solve these problems.

    1. Custom Coding

    In 2020, a software-defined approach seems the obvious preferred choice for most IT professionals. Yet, custom code is still a significant struggle for hybrid clouds. The Standish Group research shows 19% of projects will be canceled before they ever get completed. Further results indicate 52% of projects will cost 189% of their original estimates. Custom code decreases reusability and creates redundancy.

    2. Lack of Governance

    With many disparate cloud applications, it can be difficult to set standards. There is not much control. This free-for-all can directly impact the business. Lack of visibility into resources, a related concern, could cost a business significant expenses. Citing a recent Gartner study, ParkMyCloud estimates that companies waste $8.8 billion on idle cloud resources each year.

    3. No Role-Based Security Policies

    Gartner found that 33% of enterprise security attacks are happening from people using resources in the cloud and not necessarily putting the proper governance and security parameters in place. Without role-based access control (RBAC) and a standards-based way to apply RBAC across all your resources, hackers can more easily penetrate a hybrid cloud.

    How Hybrid Cloud Affects DevOps

    These above problems are quite broad, but they reflect a culmination of many roadblocks along the development pipeline. In terms of DevOps, juggling the ever-growing variety of hybrid cloud tools could easily stunt deployment velocity.

    Kukowski described how cloud IT faces a plethora of tooling styles. They include on-premises tooling, serverless, infrastructure as code, Terraform, Kubernetes and backup as a service. “Greater than 70% of organizations are not only multi-cloud, but they are multi-tool,” he estimated.

    Naturally, supporting nuances between them becomes tricky. Not only must you recognize governance and security inconsistencies between platforms, but you must maintain them “into Day 2,” he said.

    Solution: Policy-Based Automation

    With hybrid cloud tooling now a reality for most enterprises, managing them becomes the next logical step. Solutions primarily revolve around better automation to increase self-service capabilities. For Kukowski, the answers lie in automated governance, heightened tooling visibility and implementing enterprisewide security policy services to automate security compliance and decrease custom coding.

    One way to achieve these benefits is by utilizing a Cloud Management Platform (CMP). Gartner’s 2020 Magic Quadrant for Cloud Management Platforms includes CMPs from CloudBolt, Embotics, Flexera (formerly RightScale), HyperGrid, Morpheus Data, Scalr, and VMware with vRealize Automation Cloud plus CloudHealth. A CMP layer could help automate self-service IT, assisting engineers to do their work with governance and security capabilities in place.

    Using a CMP, companies could eliminate custom coding for things such as access control. Such increased visibility could, for example, significantly decrease AWS server cost by reducing unused deployments. A CMP can benefit CI/CD too. Home Depot, for example, uses a CMP to greatly improve efficiency, decreasing infrastructure deployment time from weeks to hours.

    “Who is authorized to perform that action, on that K8s container, local VM, or cloud machine? We need visibility into that,” Kukowski said. “What is it doing? When is it running? Who has visibility to shut it down?” He recognizes that deep industry expertise is required to abstract complexity and standardize governance across all these situations.

    Streamlining the Polymorphic Cloud

    Saturated within an overabundance of cloud tooling, it’s easy to lose sight of your enterprise’s IT policies. Without global DevSecOps standards, security for hybrid cloud becomes risky.

    Enterprises with multi-tool environments are also at risk of wasting development time and effort. “We want to make sure the market does not repeat the problems of the past,” Kukowski noted. “Why custom code the fundamentals that operate your business?”

    Instead, companies should seek to reduce redundancy and eliminate custom code for each point to point integration. However, abstracting such complexity involves porting governance standards across all environments—a challenging feat for internal teams.

    The solution could lie in utilizing an external CMP layer. Such a layer could offer a software-defined approach to eliminate custom coding, offering policy-based security controls that consider the organization’s needs.

  • How to Achieve AWS Security in 10 Steps

    How to Achieve AWS Security in 10 Steps

    Looking to lock down your AWS cloud service? Here are the 10 steps to take for AWS security

    As more companies embrace the cloud, security is becoming a more significant challenge. Irrespective of size, organizations are increasingly realizing the importance of having proper cloud security practices in place. More and more companies are creating a culture where digital engineering teams are making cloud security a priority and checking whether their service providers can provide the correct security levels for their cloud workload. It is up to organizations, in partnership with their cloud service providers, to understand the different security roles, responsibilities, tools and best practices.

    It turns out that getting the balance right security-wise is becoming easier. For example, AWS has published its cloud security best practices to help customers implement cloud services in a highly secure manner. These best practices provide guidance to organizations unsure of the tools or services they require. The AWS cloud platform allows customers to scale and innovate while maintaining a secure environment; customers only pay for the cloud security services they are using without any upfront expenses, unlike the on-premises environments.

    When it comes to cloud security SNAFUs, misconfigurations that accidentally and unnecessarily expose services are the biggest culprits. Any misconfigurations during implementation can potentially lead to big problems down the road. In the cloud, resource provisioning, management and monitoring happen via APIs, consoles and command-line interfaces (CLIs). So, it’s essential to secure all these connections during the early stage of cloud adoption.

    Learning From Experience

    Teams also must ensure that virtualization, identity and access management (IAM), workload protection and network security and encryption are a part of this architecture blueprint. We are all familiar with catastrophic data breaches that make the news, but there are many others that do not, and the vast majority of these are avoidable. Let’s examine the following three scenarios, all derived from real experiences, and then look at how the vulnerability could have been avoided.

    Case 1: Passwords Matter

    An AWS account was compromised due to a weak root user password, combined with MFA not being configured. An attacker removed EBS, EC2 instances, S3 data backup and asked for a ransom to provide the database backup file.

    This could have been prevented by utilizing an IAM user with a specific policy instead of using the root account. 

    Case 2: Visible Access Keys

    The developer posted the development account credentials for the AWS access and secret access key in the script, which was on the public Git repository. Using those keys enabled the attackers to deploy multiple resources in all regions and created multiple dummy resources. For the six to eight hours that the account was hijacked, the attackers racked up a bill of $4,000 to $5,000.

    This could have been prevented by using an IAM role instead of hard-coding the access keys into code. Access keys should be rotated periodically.

    Case 3: Misconfigurations

    The security group was not properly configured. In particular, port 22 was allowed globally, ports 80 and 443 were also open from all. The application which was hosted on EC2 instance was front-ended by the application load balancer. The attackers found a basic cross-site scripting vulnerability and SQL injection, tried injecting malicious scripts and removed all the data from the server.

    This could have been prevented by following a trusted advisory, putting WAF on top of the application load balancer and utilizing the AWS inspector for security assessments.

    AWS Security in 10 Steps

    These three cautionary tales are all examples of where a small security vulnerability caused big losses, and they all demonstrate how easy it would have been to deploy the secure cloud option. Hindsight is a wonderful thing, but it is even better to avoid making the error in the first place. Here is a 10-step guide to understanding security best practice structures and how users should implement the AWS platform and its services in the best way.

    1. Understand the Shared Responsibility Model

    • Amazon’s responsibility: AWS is focused on infrastructure security, including computing, storage, database and intrusion prevention networking services. AWS is also responsible for the security of the software, hardware and the physical facilities that host AWS services.
    • Customer’s responsibility: AWS customers are responsible for the secure usage of AWS services that are considered unmanaged. The customer is responsible for managing user base access-authentication methods, encryption of data stored inside AWS.

    2. Follow IAM Best Practices

    • The AWS Identity and Access Management Service enables users to manage access to AWS services and resources securely.
    • The AWS account can be administrated by creating groups and users and applying granular permission policies to users to provide limited access to APIs and resources. This video takes a deep dive into IAM policy management.
    • When granting IAM roles, follow the “least privileges” approach to security.
    • Rotate access keys and passwords.

    3. Manage OS-level Access and Keep Ec2 Instances Secure

    • Run an inspector assessment to generate an OS-level vulnerability report.
    • Use System Patch Manager to keep OS packages updated.
    • Patch the EC2 instance periodically to protect your infrastructure from newly discovered bugs and vulnerabilities.
    • Follow the security advice provided by OS vendors RedHat, Suse, Microsoft, etc. This helps to keep all OS-specific packages updated from a security perspective.

    4. Encryption

    • There are two methods for encryption in AWS: in transit and at rest.
    • Use AWS KMS for storing at rest encryption keys, which can be AWS-generated or customer-generated.
    • Use Cloud HSM to provide hardware-encrypted devices for storing keys.
    • Most AWS services provide in-transit encryption by providing https endpoints that provide encryption end to end.
    • AWS Certificate Manager allows you to create an SSL certificate for the public domain.

    5. Follow Security Best Practices for AWS Database and Storage Services

    • RDS storage should be encrypted at rest.
    • Restrict access to RDS instances to decrease the risk of malicious activities such as brute force attacks, SQL injections or DoS attacks.
    • S3 storage should be encrypted at rest.
    • S3 policy should be used to restrict access to S3 content. Keep your S3 bucket private if you don’t have any requirement to expose objects.
    • Use AWS Macie to detect and secure sensitive data within AWS-S3.
    • Use the AWS Parameter Store to store environment-specific credentials and secrets, which you can easily achieve using secrets management for your cloud-native application.

    6. Network Security

    • Intrusion detection systems (IDS) or Intrusion prevention systems (IPS) allow the detection and prevention of attacks on critical infrastructures such as payment gateway of banking-related transaction applications.
    • VPC flow logs should be enabled to monitor network traffic.
    • Restrict access by security group (EC2, RDS, Elastic Cache, etc.)
    • Use Guard Duty to monitor AWS accounts and infrastructures continuously.

    7. Web Application Security

    • Web application firewalls (WAF) provide deep packet inspection for web traffic.
    • WAF can also help to prevent platform and application-specific attacks, protocol sanity attacks and unauthorized user access.
    • Amazon Inspector is an automated security assessment service that improves security and compliance of applications deployed on AWS.
    • Use AWS Cognito to authenticate application user pools securely. It also supports federated access from Google, Amazon and Facebook.

    8. Enable Configuration Management

    • AWS Config helps to audit, assess and evaluate the configuration changes within AWS.
    • Customers can also rely on third-party configuration management tools, which can provide more granular visibility.

    9. Monitoring and Alerting

    • CloudTrail enables auditing and monitoring of authorized and unauthorized activities within the AWS account.
    • CloudWatch alerts can be set up for malicious activities within the AWS account and infrastructure deployed inside AWS and application logs.
    • Set billing alarms to keep your team aware of the cost utilization of specific accounts or infrastructure.

    10. Compliance, Training and Certification

    • AWS Artifact helps customers to get a no-cost, self-service portal for on-demand access to AWS compliance reports.
    • Train and educate teams using the AWS cloud platform for deploying or are responsible for the infrastructure.

    Implementing these cloud security best practices guidelines provided by AWS helps organizations implement the proper security measures, irrespective of where they are in their digital transformation journey or how large their organization is.

  • Xen Project Team Updates Open Source Hypervisor Project

    Xen Project Team Updates Open Source Hypervisor Project

    The team behind the open source Xen hypervisor announced today provided an update to the project that adds better nested performance, more robust live patching capabilities, improved security and support for Linux subdomains that make it possible to execute code within a specialized domain.

    In addition, Xen Project Hypervisor 4.14 adds the ability to run Xen as a guest under Hyper-V, the hypervisor Microsoft employs on the Azure cloud.

    George Dunlap, Xen Project Advisory Board Chair, said that latter capability will make it easier to create centrally managed hybrid cloud computing environments that span lightweight instances of Xen at the network edge and instances of Xen running in the Microsoft Azure Cloud. Xen is a core component of the Edge Virtualization Engine (EVE) being developed by LF Edge, which, like the Xen Project, is being developed under the auspices of The Linux Foundation.

    Dunlap also noted IT teams can employ Xen Project Hypervisor to run containers on top of virtual machines, which makes it possible to centrally manage the deployment of containerized applications from the edge to the cloud.

    Xen Project Hypervisor 4.14 also extends Raspberry Pi 4 support for devices with 4GB and 8GB of RAM, in addition to supporting the next-generation AMD EPYC processor, codenamed Milan.

    Finally, Xen Project Hypervisor 4.14 adds support for faster introspection of malware; the ability to live-patch security fixes in a way that ensures they are implemented in the right order; and control flow tools to combat Return-Oriented Programming (ROP) attacks.

    Dunlap noted the Xen team continues to work on Secret-free Xen, which prevents memory from being mapped as part of a side-channel attack.

    Golang bindings are also been expanded to make it easier to develop code on top of Xen using the Go programming language, and add the ability to migrate virtual machines with no drivers or broken drivers.

    While there is no shortage of options when it comes to virtual machines, Dunlap said the more they are viewed as a commodity the more likely organizations will opt to rely on an open source project, as many organizations are deciding there isn’t enough differentiated value to warrant paying for a commercial license.

    It’s too early to say how much organizations will continue to rely on virtual machines. Many IT teams do so now to provide a layer of isolation that is often deemed critical for security. Other IT teams are starting to rely more on containers deployed on bare-metal servers or a lighter-weight instance of a virtual machine that serves only to isolate workloads.

    Regardless of the path forward, the more platforms that run a common layer of virtual infrastructure, the easier it is to manage a hybrid IT environment. The real challenge is being able to enforce that level of IT discipline in an era where developers often spin up virtual machines wherever they find them.

  • Arrcus Expands Network Services Spanning Hybrid Clouds

    Arrcus Expands Network Services Spanning Hybrid Clouds

    Arrcus has added virtualized distributed routing (VDR) and multi-cloud networking (MCN) capabilities to ArcOS, a network operating system (NOS) that can be deployed on-premises or in the cloud.

    Company CEO Devesh Garg said these additional capabilities provide the programmatic network services that IT organizations require to foster deployments of hybrid cloud computing environments.

    Arrcus MCN provides the ability to securely connect workloads and data across any cloud, while VDR eliminates the need to rely on physical routers that require their own chassis, he said.

    Arrcus has been making the case for a disaggregated approach to networking that eliminates dependencies on physical infrastructure. Instead, IT teams can deploy ArcOS on a cloud service or any white-box appliance they choose in an on-premises IT environment. That approach eliminates the possibility of becoming locked into specific network hardware platforms, he said.

    Garg added it also provides DevOps teams with a consistent foundation on which to deploy distributed applications. Rather than having to navigate different NOSes running in the cloud and on-premises IT environments, ArcOS presents a consistent set of application programming interfaces (APIs) for accessing network services.

    In fact, the need to programmatically invoke those network services is what’s driving the convergence of network operations and DevOps teams, he noted. As applications become more distributed in the age of the cloud, IT organizations are looking for more efficient ways to expose network services on-demand.

    That convergence, however, will take time. Many networking teams still tend to prefer to acquire routers and switches as physical devices that include a NOS rather than deploying an operating system on infrastructure they select and manage. However, as DevOps teams continue to push for access to network services that can more easily scale up and down as required, it’s now only a matter of time before software-defined networking services become a default requirement.

    In fact, Garg said, the rise of microservices will force that issue as DevOps teams look to deploy applications that will be updated frequently.

    In addition, he noted, the rise of edge computing platforms and 5G wireless networking services will simultaneously push organizations toward virtualized network services.

    It’s not clear at what rate network services are being transformed. Software-defined networking services are already readily available in the cloud. Progress in on-premises IT environments has been tied to the willingness of IT organizations to fund upgrades to core network infrastructure. The challenge many IT organizations now face in the wake of the economic downturn brought on by the COVID-19 pandemic is finding a way to achieve that goal cost-effectively. White-boxes provide a less costly alternative to switches from vendors such as Cisco Systems and Arista Networks. At the same time, the total cost of delivering network services across a hybrid cloud computing environment can decline when a common NOS is employed across all platforms.

    Regardless of the path forward, the one thing that is certain is legacy network infrastructure has become a major bottleneck at a time when IT organizations need to be more agile than ever.

  • DevOps Chat: Software-Defined Storage, With StorPool

    DevOps Chat: Software-Defined Storage, With StorPool

    The whole Data/DataOps space is becoming very hot. For StorPool, it is more of a case of the rest of the world catching up to what the company has been doing for a while now. It is the leading distributed block storage provider globally.

    In this DevOps Chat, we speak with Boyan Krosnov, co-founder and chief of product at StorPool. Boyan gives us the background on StorPool as well as a good look at the state of the distributed storage/data market.

    As usual, the streaming audio is immediately below, followed by the transcript of our conversation.

    Transcript

    Alan Shimel: Hey, everyone, this is Alan Shimel. You’re listening to a DevOps Chat here on devopsy.kinsta.cloud, and my guest in this DevOps Chat is Boyan Krosnov from StorPool.

    Boyan Krosnov: Hi, Alan.

    Shimel: Boyan, I know it’s not perfect, but it’s the best I could do with your name, I apologize. But why don’t you say it for our audience?

    Krosnov: Sorry—repeat, please? I did not hear.

    Shimel: Why don’t you say your name for our audience?

    Krosnov: Yeah, so, my name is Boyan—Boyan Krosnov, but in, kind of for English speakers, most people say Boyan Krosnov, like, with the stresses at the wrong places, but it’s perfectly okay, you know, I understand.

    Shimel: As long as they don’t call you late for dinner, as we say here, right?

    Krosnov: Yeah, okay.

    Shimel: So, Boyan, your company is StorPool, yes?

    Krosnov: Yeah.

    Shimel: And what’s your position with StorPool?

    Krosnov: In StorPool, I am one of the Co-founders, one of the three Founders in the company, and my role as a Co-founder from the very beginning has been kind of the glue between technology and business and the product person in the company, the Chief of Product of StorPool.

    Shimel: Sure.

    Krosnov: And the other two Co-founders, you know, respectively take, are focused on technology development and the other Co-founder on strictly business, who is our CEO, right.

    Shimel: Someone has to do that. Okay.

    Krosnov: Someone has to do the business stuff, yes.

    Shimel: Someone has to. Before we jump in, Boyan, to what StorPool is about and what you guys do, why don’t we talk a little bit about your own personal journey? How did you come to found StorPool? What’s your background?

    Krosnov: Yeah, right. So, back in the ‘90s, I used to do competitive programming, so like, from a fairly early age, I got very interested in informatics or computer science, right? And from that, later on, I went into networking and ISPs, later on, I had a startup around something called packet processing and nowadays software-defined networking.

    And from that, at the later stages of that startup before we folded that, we found this gem of an idea of software-defined storage before it was called software-defined storage, right? So, we didn’t know it was called software-defined storage; the term wasn’t coined yet. That’s, like, in 2011.

    Shimel: Mm-hmm.

    Krosnov: And, from that, we founded StorPool. So, I got with the other founders and did, like, a whiteboard exercise of what an ideal storage system would look like. And that became the architecture of our product afterwards and we’ve been building on that for the last years, right? It’s eight years now.

    Shimel: Yeah.

    Krosnov: And along the way, we got—yeah, so, I’ve lived in a few places, not just in Bulgaria. I’ve lived in Iceland and the U.K. and Malaysia and a few others. And so, in StorPool, you know, we started with the idea, implemented it, got it to first production use some five, six years ago, right, and since then, we’ve been expanding it in a very kind of real business organic way, meaning pretty high growth, but at the same time, very customer-oriented. So, it’s very—the things we build into the product are things that our customers demand, right? It’s not like a completely greenfield.

    Shimel: Yeah, everyone already has some data.

    Krosnov: We do stuff that matters for our customers, right?

    Shimel: So, Boyan, listening to your story, I’m reminded of my own. You know, I got into technology in the mid-‘90s when the web first started coming in, it was just starting to get commercial, and we were storing websites. It wasn’t called hosting. I thought I was a virtual real estate person, [Laughter] because as I needed more land, I put another hard drive in the machine, right?

    Krosnov: Yeah.

    Shimel: And by about 1997, people started calling that web hosting.

    Krosnov: Yeah.

    Shimel: And we sold the company in 1998, we ended up doing a whole IPO and that whole thing. But, you know, at the end of the day, what we were doing is, we were just, we were storing people’s websites. I was selling virtual real estate, renting virtual real estate out. And it sounds like a similar thing you, before there was a thing called software-defined storage, you recognized that there was a need in the market for people to do, to be able to store stuff without the expensive, I mean, you know, back—I remember in those days buying EMC boxes, right?

    Krosnov: Yeah, yeah.

    Shimel: And some of the Sun and IBM—was it IBM Sharp? I forget. But all of these huge SAN—you remember the days.

    Krosnov: Yeah.

    Shimel: Millions—millions of dollars in SANs and all of that. And then, [Snaps fingers] it was like a flip—like a switch flipped. People started saying, “Wait a second, we could do this in software, for a lot less money with a lot more advantages.”

    Krosnov: Yeah, yeah.

    Shimel: And it sounds like StorPool happened to be right there at the right time when this was coming together.

    Krosnov: Yep, so, kind of the reason we started it was a few conversations with, actually, hosting companies.

    Shimel: Really?

    Krosnov: Actually hosting companies and some banks, but it started with hosting companies, and at that time, there was this big movement in hosting—everyone wanted to be the next DigitalOcean, right?

    Shimel: Right.

    Krosnov: Everyone wanted to have, like, a public cloud service with virtual machines and be like Amazon and DigitalOcean. And there were architects in these companies that were trying to figure out how to do it and they couldn’t and the one piece that was missing was software-defined storage. It was, like—they couldn’t make the business model work with standard SAN from HP or IBM or EMC, right? And we had to find a solution that worked for them which runs on standard servers—like, it’s not a proprietary box.

    Shimel: Understood, understood. So—I mean, but that was 2012. Now, here we are, 2020. Software-defined storage is a much better-understood term.

    Krosnov: Mm-hmm.

    Shimel: You know, I think it’s gone mainstream, if you will, right? People understand it and use that.

    Krosnov: Yeah.

  • Serverless Computing: Taking DevOps to the Next Level

    Serverless Computing: Taking DevOps to the Next Level

    Serverless computing enables you to outsource information, thus freeing teams to focus on front-end work. Organizations choose serverless when they look for faster and easier ways to release updates and patches. DevOps teams often use serverless when they are looking to automate infrastructure tasks. However, each provider offers different capabilities, and each service does require a level of training pre-adoption.

    What Is Serverless Computing?

    Serverless computing is a method that outsources infrastructure and infrastructure tasks to external providers. Organizations choose serverless computing because it enables development and operations teams to focus on the front-end of products and services.

    There are a few traits that distinguish serverless infrastructures from traditional cloud automation models:

    • Serverless computing services eliminate user responsibility for the management or operation of infrastructure.
    • Serverless computing services enable users to run code on-demand according to requests or event triggers. Infrastructure use scales transparently based on how many requests or events occur.
    • Serverless computing is pay-for-use and can be used to ensure that users pay only for actively used resources.

    The benefits of serverless in enterprise environments include:

    • Better control over production deployments.
    • Faster and easier updates or patches.
    • Increased productivity and easier release rollouts.

    Popular Serverless Computing Services

    All three major cloud providers offer serverless computing services. Below you can learn about these services and how each operates.

    AWS Lambda

    AWS Lambda is a service for serverless computing with other AWS services or via API. It can be connected to applications or run as standalone code. When using Lambda, users pay for active compute time although instances can be reserved to ensure their code runs with menial latency.

    Lambda can be triggered by a variety of AWS services, including Kinesis and DynamoDB. It can be monitored through Amazon Cloud Watch. Common use cases for Lambda include data processing, stream processing, machine learning and application backends.

    Azure Functions

    Azure Functions is Microsoft’s serverless service. You can use it for end-to-end development, including building, testing and deploying code. Functions can be used with a variety of Azure services and integrate with Visual Studio and Visual Studio Code for easy use.

    In addition to Functions, Azure offers Logic Apps, a service for creating visual design workflows and orchestrating Functions activities, and Event Grid, a message routing service to integrate Functions with other Azure services.

    Google Cloud Functions

    Cloud Functions is Google Cloud Platform’s functions as a service (FaaS) (i.e. serverless) offering. It can be used in the cloud or with hybrid deployments and it includes built-in monitoring, logging and debugging features.

    Cloud Functions supports a variety of environments including Cloud Run, Kubernetes and other Knative-based environments. Use cases for Cloud Functions include integration with third-party services, application backends, real-time processing, virtual assistants and machine learning.

    Why Serverless for DevOps

    If you are already using a DevOps approach or are considering adopting one, serverless can be a good option. Many DevOps teams, especially those operating in the cloud, are already working with modular components managed via a pipeline. Moving these modular services to serverless can enable teams to eliminate some of the tasks related to pipeline management and focus on development and deployment.

    This capability is especially valuable for teams that are just getting started or are looking to switch to DevOps practices. Serverless systems can enable teams to get started with development and testing right away on a comfortable scale. Then, as productivity increases, the serverless infrastructure can smoothly scale with minimal to no effort on the part of operations.

    Another benefit of serverless infrastructures is the ability to easily enable DevOps automation. With serverless, you can deploy your pipeline without hosted solutions. You are only responsible for “glue code.” This infrastructure is flexible and you can easily modify it as needed using infrastructure as code (IaC) solutions and automated events.

    Despite these benefits, there is a potential downside to serverless. Your DevOps team does not have the same visibility into or control of the functioning of the infrastructure. While you can integrate monitoring, teams are limited in how much of the infrastructure can be customized or manually affected.

    How Serverless Can Take DevOps to the Next Level

    There are several ways serverless deployments can be leveraged to improve DevOps productivity. Below are a few examples to consider.

    Infrastructure as Code

    Provisioning infrastructure is often a significant portion of deployment processes. Many applications use a variety of serverless services as building blocks through all stages of development and release. Making sure that these services are correctly connected and configured is key to a successful deployment.

    IaC can help users accomplish these tasks smoothly by leveraging declarative languages and tools such as Terraform. With these solutions, users can version your infrastructure along with their code, keeping tasks in sync and deployments consistent.

    Flexible Deployments

    With serverless services, infrastructure is immutable. Any changes result in new instances being spun up and activated. This means both old and new versions can run simultaneously.

    This immutability can be used to perform flexible deployments that are not possible or are much more difficult with traditional infrastructures; for example, canary or blue-green deployments. With serverless infrastructure, users can switch from one version to another with minimal fear of service interruptions and the ability to easily rollback updates.

    Distributed DevOps

    Serverless services can enable DevOps teams to work across a variety of environments and locations with minimal impact on others. These services allow the distribution of workloads at scale and can ensure that infrastructure is always available.

    The catch is that greater distribution and the ephemeral nature of serverless can make infrastructure much more difficult to monitor effectively. To account for this, teams need to ensure centralized monitoring with real-time alerts. Teams should also periodically audit infrastructure availability and performance to ensure that resources are functioning as expected.

    Conclusion

    Serverless computing enables DevOps teams to eliminate time and cost on the back end. Each serverless provider offers different services that can help achieve different capabilities. The main capabilities are provided via the use of functions, which enable the automation of infrastructure. However, in most cases, teams do need to learn how to use functions to properly configure customized IaC workflows. Once teams understand how to use the software, they can create flexible deployments and a distributed DevOps pipeline.

  • IaC and Kubernetes: A Natural Pairing

    IaC and Kubernetes: A Natural Pairing

    Using IaC with Kubernetes helps standardize Kubernetes cluster configuration and manage add-ons

    Infrastructure as code (IaC) is the ability to provision and manage infrastructure using a configuration language. It offers repeatability, transparency and the application of modern software development practices to the management of infrastructure including networks, load balancers, virtual machines, Kubernetes clusters and monitoring.

    IaC helps to reduce error and configuration drift while allowing engineers to spend time on higher-value tasks. IaC defines what the end state of infrastructure should look like, instead of defining a process to be executed.

    So Why Aren’t All Companies Using IaC?

    There are three main reasons why companies may not yet be embracing IaC.

    Many industries and engineers have not fully embraced or adapted to the paradigm shift of infrastructure APIs and code

    A lot of us think about infrastructure as touchable, involving physical installation and cabling. It’s only relatively recently in enterprise IT years that the industry has adopted APIs that allow borrowing and configuring infrastructure on-demand without the physicality. Engineers are good at standing up infrastructure using a time-tested, battle-hardened process. Incremental learning and integrating IaC into processes and culture helps to digest an otherwise daunting and challenging project.

    Industries are hard at work adapting security controls and regulations to incorporate the opportunity of cloud and automation without compromising high standards for privacy, safety and security. Engineers using IaC to improve their own efficiency can help champion its use in wider circles. One such example of this success is the 18F office which helps improve how the government serves the public through technology.

    IaC may feel like a loss of security or control

    Increasing the awareness of how and why infrastructure is configured can invite questions or feedback that initially may feel uncomfortable to those who have made infrastructure decisions. The ease with which others can make changes to infrastructure may feel dangerous or devaluing to engineers who have always pushed those buttons.

    IaC provides an opportunity for improving single human points of failure and knowledge bottlenecks without jeopardizing necessary controls. Access to infrastructure code and IaC tools can still match organizational structure, culture and approval processes. Infrastructure engineers remain integral to its evolution and success, and they are more available to tackle next-level problem-solving.

    ‘We move too quickly to take the time to ramp up with IaC, and we’re fast at clicking in user interfaces’

    The repeatability achieved by IaC means artisanally created infrastructure does not need to be kept around because it may be needed next month and parallel infrastructure can be created for testing and destroyed tomorrow. Start using IaC for something small such as spinning up that occasional temporary VM, then create a limited-access service account that someone else can use along with your IaC when they need that VM. Your IaC efficiency will grow with your organization’s body of IaC knowledge and code.

    Kubernetes and IaC

    Using IaC with Kubernetes helps standardize Kubernetes cluster configuration and manage add-ons. Infrastructure now can be versioned and committed to a Git repository, and it can more easily be “stood up” elsewhere when the company needs to expand to new regions or clouds. IaC benefits are vast and include:

    • Reduced human error and increased predictability: Create new test environments without impacting production and make changes across multiple environments using code review processes. Focus and attention to detail are less impacted by repetitive manual work.
    • Consistency: The repeatability of IaC with Kubernetes allows for the rapid creation of consistent infrastructure. This frees up time to address other challenges.
    • Improved changed tracking and auditability: When infrastructure is represented in a versioned repository such as Git, commits to the Git repository can reflect who, when and why changes were made. The code provides a record of how environments were built and evolved.
    • Faster speed to recovery: The reliability of an application is impacted by the ability to pivot and the speed to redeploy. Infrastructure as code can help you know what the recovery process looks like and practice that process more often.

    Start Using IaC and Kubernetes

    As a company moves into the world of containers, cloud-native apps and Kubernetes, IaC will help teams improve application reliability and stability, streamline development and operations and support a better user experience.

    In a Kubernetes environment, reliability becomes much easier to achieve with IaC because teams are deploying standard configurations. DevOps teams can use IaC to embrace the ephemeral nature of containers and Kubernetes pods, as well as avoid single points of failure.

  • observIQ Lauches Open Source Log Agent Project

    observIQ Lauches Open Source Log Agent Project

    Blue Medora, after selling its core monitoring integration platform to VMware, today changed its name to observIQ and launched an open source agent project that will serve as the foundation of a log management tool to be delivered as a software-as-a-service (SaaS) application.

    Company CTO Mike Kelly said the observIQ agent, which is written in the Go programming language, will provide IT teams with access to highly configurable agent software that doesn’t adversely impact performance. In contrast, existing open source log agent software requires IT teams to sacrifice one for the other, he said.

    Because the observIQ agent is highly configurable, it will also serve to make it easier to collect a wide range of metrics that can be consumed by a wide variety of management tools, Kelly added.

    The company will employ observIQ to collect metrics for observIQ Cloud, a SaaS platform currently in beta, in addition to continuing to invest in BindPlane, an IT operations data management platform developed by Blue Medora that delivers a relationship-aware stream of metrics and logs in real-time.

    Kelly said IT teams need access to a highly configurable agent software because the types of metrics that need to be collected continue to expand and evolve.

    At the same time, Kelly notes IT teams need to be able to affordably instrument many more applications, which becomes difficult to achieve when relying on commercial agent software.

    Finally, as IT environments become more complex, organizations are embracing best DevOps practices to manage applications, which requires organizations to increase dramatically the level of observability they have into any application environment. The issue DevOps teams encounter is they wind up having to support multiple agents, so if an open source project leads to fewer agents that need to be maintained and updated, that might be a benefit that resonates.

    In the meantime, as log management heads into the cloud along with most other IT management services it’s also becoming easier for IT teams to manage applications from anywhere. Most IT teams are going to be working remotely much more often than they did prior to the onslaught of the COVID-19 pandemic. That transition is easier to achieve when the tools being relied on reside in the cloud.

    Many IT teams also tend to limit their use of log analytics because of the costs associated with storing and collecting data. To limit those costs many IT organizations opt to only store log data for a limited time or only collect metrics for their most critical applications. The challenge is, the percentage of applications deemed critical continues to increase.

    Of course, competition across providers of log management tools that can be accessed as a service is already fierce, so observIQ will have its work cut out establishing another platform. However, if DevOps teams adopt its open source agent software, there’s naturally a much higher chance organizations will wind up employing observIQ Cloud.