devopsy.kinsta.cloud

Tag: cloud-native

  • DevOps Chats: Database for Cloud-Native Apps with MemSQL

    DevOps Chats: Database for Cloud-Native Apps with MemSQL

    Data and database technologies are experiencing disruption. The volume of data collected is exploding, creating new potential for disruptive services and new companies. Data is distributed across locations in the cloud and the data center. Also, cloud technologies bring a plethora of database types, distribution options, analytics capabilities and AI/ML processing. Any company not leveraging data to the benefit of its customers and business are just asking to be the next victim of disruption.

    Nikita Shamgunov, MemSQL co-CEO/co-founder, joins DevOps Chat to discuss how these disruptive factors changed how containerized and cloud-native applications approach data collection and storage. Speed, high volume data collection, distributed design and scale are all challenges Nikita sees as vital for cloud-native applications of today and the future.

    As usual, the streaming audio is immediately below, followed by the transcript of our conversation.

    Transcript

    Mitch Ashley: Hi, everyone, this is Mitch Ashley with devopsy.kinsta.cloud, and you’re listening to another DevOps Chat podcast. Today, I’m joined by Nikita Shamgunov, co-CEO and co-founder of MemSQL. The topic today is really looking at the state of database and the market and technology as we think about database now. Nikita, welcome to DevOps Chat.

    Nikita Shamgunov: Thank you, thank you—excited to be here.

    Ashley: It’s a pleasure to have you on. Thanks for joining us. Would you start by introducing yourself, tell us a little bit about what you do and a little bit about MemSQL?

    Shamgunov: Absolutely. My name is Nikita Shamgunov, I’m the co-CEO and co-founder of a database company, MemSQL. Prior to MemSQL, I worked at Facebook, and prior to that at Microsoft on the SQL server kernel. So, I’m a kernel engineer by training. And before that, I graduated with the Ph.D. program from St. Petersburg, Russia.

    Ashley: Excellent, excellent. Well, let’s jump right into it. We were talking a little bit before we started the podcast recording. You know, databases have been around for a long time, and you know, the technologies created decades ago are still in production in many areas, but there’s a lot of changes that are happening, too. Why don’t you talk a little bit about some of the market forces that are happening to change the database environment?

    Shamgunov: Absolutely. So, there’s several market forces and industry forces that are, I would say, piercing through the market and technology. And the first one is that the data volumes are growing, right?

    Ashley: Mm-hmm.

    Shamgunov: And a lot of companies—and you don’t have to go far for examples, you know, companies like Uber and Netflix and Amazon and Google drive most of their value from data, right? That somebody in the ’80s would describe any of those companies as a glorified database application—but, of course, they are a lot more than just that.

    And with the data volumes and variety of that data growing, what separates winners from losers is how well people can capture and act on data.

    Ashley: Mm-hmm.

    Shamgunov: Right? A great example is, you know, when you’re hailing an Uber, you know, you pop it in your phone and then you know exactly where the cars are, how fast they’re gonna arrive, how long it’s gonna take you to the destination.

    And that’s really, really powerful. We’re all used to this right now, but 10 years ago, that was not a reality and not a possibility. So, that is the force number one, right? People see who are those companies that are advancing in the market and they wanna take on the same opportunity in their state, in their category.

    They also are afraid of being disrupted by the next Uber, the next Facebook, the next Amazon—particularly Amazon.

    Ashley: Exactly.

    Shamgunov: So, the second market force that is going through the market is architectural. And, because the data volumes are growing and the Moore’s Law is not working any more, so we cannot just rely on better, cheaper hardware, we have to build distributed systems.

    And, in the database world, distributed systems, for the most part, have been exotic until quite recently. And we’re starting to see distributed database systems—and the reality is, it’s just very hard to build a distributed kinda system of record transactional system and make it production ready, because the amount of technology that goes in there is enormous, and different, kinda other database products that are successful in the market have been products for 30 plus years. You know, thinking about Oracle, thinking about SQL Server.

    Ashley: Mm-hmm.

    Shamgunov: And when this massive architectural change comes in where it’s a business reality that you need to re-architect those systems, then it becomes very, very hard for the vendors to actually deliver those. So, that’s the second market force. Transition [Crosstalk]—

    Ashley: Let me ask you a little bit about that, Nikita, if I could, about the—

    Shamgunov: —towards distributed systems—go ahead.

    Ashley: The distributed architectures, do you see that happening largely because of the displacement between on-prem data centers and private clouds and public clouds? Is it more pushing content to the edge and doing caching? Is it pushing content toward a geographical location? What’s forcing the distribution of databases?

    Shamgunov: This is a wonderful question, first and foremost, because I was about to say that the third mega market force that is happening right now is moved to the cloud.

    Ashley: Mm-hmm.

    Shamgunov: Right, and cloud, if you make it parallel with electricity, right, the majority of the world consumes electricity in the clouds, you know? You just plug your stuff into a socket and here comes your electricity.

    Ashley: Mm-hmm.

    Shamgunov: On premises, generators still exist at important places—manufacturing, health care, hospitals—but it’s a tiny percentage of the global electricity consumption.

    Similarly, it just makes too much sense to consume IT in the cloud, and within IT, you know, there’s applications, analytics, databases, machine learning—you know, all the whole spectrum of services that typically a cloud provider now offers. Now, if we buy into this worldview—and certainly, we see what happened with electricity, and this is the same thing that’s happening in IT—then you can split the same into different workloads. And once you zoom in on the database workloads, you will realize that, yes, you wanna take workloads that currently run in data centers and shift them into the cloud.

    Cloud is different, you know? The most important thing in the cloud is that software just runs as a service. But also, it creates an opportunity to rebuild every single piece of IT for the cloud. And that’s already happening. In our space, it’s already happened in data warehousing, but it also happened with identity because of Okta. It happened with cloud storage, with services like S3 on Amazon or Dropbox as a higher level service. So, basically, as infrastructure IT is shifting towards the cloud, we are rebuilding it for the cloud. And now, it just so happens that, through that rebuild, you want to cater to the next generation workloads. That’s where distributed systems comes in. And also, you’re architecting for the cloud because the cost equation in the cloud is different.

    So, being scalable and elastic allows you to provide a better cost equation, as opposed to legacy architectures. So, if you take an Oracle and—yeah, you can run it in the cloud, in EDM, no problem. Well, actually, it’s gonna be way too slow and way too expensive as opposed to the system that’s architected specifically for the cloud that scales both for cost, right, so you can only—you will only consume as many resources as you need from the storage and compute standpoint. Storage compute network bandwidth—whatever you need to run the workload. But it also is gonna be a lot more efficient, because running a database as EDM, everybody will tell you this is not such a great idea, but if it’s a database service architected for the cloud, then you can build for and around—around limitations and for all the services and resources that are available for the cloud.

    Ashley: What I’d like to ask you about that is, it seems like there’s a parallel between lift and shift for applications and lift and shift for database. It isn’t architected for the cloud if you’re just taking the application into a cloud environment, but you’ve gotta re-architect, rebuild, take advantage of things like database as a service, for example. Is that correct?

    Shamgunov: Absolutely. Alright, and then in the application space, what’s different than the cloud is that certain applications can be more popular than others. And if you just put those apps as you had on premises in VMs—so, very likely, you both overprovision and underprovision resources.

    For less popular applications you are wasting the whole VM to run an app, and for a very popular application you don’t have enough network bandwidth or CPU to run this app at scale. And that’s why Kubernetes really changed the game, and if you build against Kubernetes, you can pre-release scaled, stateless applications. And then, if you run it in the cloud, of course, the database that you use to power those applications, you consume as a service.

    Ashley: Say a little bit more about that, because we’ve talked on other webinars and podcasts about the statefulness of data versus the more dynamic nature of building software and rebuilding software that may not match up with the state of the database. How do you handle that in a Kubernetes environment?

    Shamgunov: So, there are two pieces to this. The first one is, you run your application in the Kubernetes environment and you consume data through a database as a service. So, that is a blueprint of a cloud architecture today. Let’s say you run your Kubernetes on Amazon, your application is written in whatever language, but let’s say Node.js, and you fire up DynamoDB, using an Amazon example, as a service, you connect to that database from the application and there it is, there’s your app.

    Now, in the world of relational databases—which, by the way, is a much larger slice of the market than object databases—

    Ashley: Sure, yeah.

    Shamgunov: And for a very good reason. You can run it in a similar fashion. You know, in this case, you can launch MemSQL as a service, you know, run  your application inside a Kubernetes cluster.

    The other thing that you can do is, you can bring MemSQL with you into the Kubernetes environment. And that gives you control and cloud portability. Now, not only do you deploy your application into Kubernetes and the application scales depending on the demands and the popularity of the app, but also the database that stores the data and powers the app can be deployed in the same Kubernetes cluster, and it will do the same thing. It will scale or contract, depending on the needs of your application.

    So, what it gives you, it gives you cloud portability. So, you can pack your bags and go from AWS to Google, for example, and the other thing it allows you to do is to take it and bring it down, right, from cloud to on premises. And today, the reason to do so is usually twofold. It’s either security and compliance or it’s cost, right? If the workloads are pretty static and you cannot drive a good amount of efficiency through the elasticity of your both application and data infrastructure, then on premises may still be cheaper, and then that creates an incentive for you to bring it down on the ground.

    Ashley: How important is it to have kind of a single tool to do ETL versus separating those as you move not only within one cloud but across clouds?

    Shamgunov: Well, I think, at the end of the day, we are talking about lock in, and we’re talking about developer productivity. Those are completely different. Now, lock in is something that a lot of people in our space experience with Oracle as a vendor.

    Ashley: Sure.

    Shamgunov: Where they go through what they describe as extortion cycles. Every time an Oracle renewal comes in, then you end up negotiating with Oracle, and if you wanna reduce the amount of the database spent, it becomes a very difficult conversation. You know, Oracle runs an audit, which they have the right to, so they inspect and find all the applications that consume Oracle. If you reduce the amount of consumption, Oracle jacks up the maintenance fee or jacks up the next license fee.

    So, it’s really, really hard to get out of spending a lot of money on this vendor. We hear that time and time again in major, major enterprises. I had a conversation with the CIO of a major bank, and the Oracle spend there is $1 billion over several years.

    Ashley: Not surprising, yeah.

    Shamgunov: And this is absolutely insane, if you think about it, right? The reason for that is that Oracle has been a great partner for delivering their services, you know, cost aside. You know, banking runs—in their bank, it’s a consumer bank, and it runs on Oracle.

    But at the same time, competitors catch up to the functionality, right? MemSQL comes with a very strong system of record capabilities and, to this day, that has been the stronghold of Oracle, and you know, with a much  more attractive price, with a much more flexible deployment model and the ability to run in the cloud as a database as a service where we guarantee SLAs enough time. And with an integration with AI and ML, we see that, slowly but surely, that equation is shifting. And, you know, people seriously ask questions, “Why would we spend so much money on this technology and can we not do this? We would save hundreds and hundreds of millions of dollars and drive shareholder value, here. In addition to that, we’ll have our development teams move a lot faster,” right?

    So, back to your question about across clouds and developer productivity, right? The first one is lock in, and I gave you an example of an Oracle lock in. But now, the CIOs don’t want to fall into the same trap by getting married to a single cloud provider. And, you know, thank God, we have multiple cloud providers available on the market, of which three in the United States are very, very strong, and you know, for the most part, similar in capabilities—GTP, Azure, and AWS.

    So, it’s very typical for a key enterprise to choose two or three cloud vendors. So, that prevents the lock in. However, in order to truly avoid lock in, you’ve gotta choose service providers that allow to run their software the same way on all the clouds.

    Ashley: Exactly.

    Shamgunov: And ideally, they deliver their services as a path service, you know, like a database as a service, and that service is available on every cloud. And if that’s the case, then, you know, you take a dependency on a vendor, but you are not taking a dependency on the mega vendor, which is one of the clouds. So, that is something that is top of mind of every CIO that I know.

    Ashley: You make a good point about Oracle, kinda going back to the lock in of the 2000s and maybe ‘90s. You know, for VLDB, that was really the best game in town, arguably. You could also Microsoft, maybe.

    Shamgunov: Mm-hmm.

    Ashley: But that isn’t true today. You’ve got many options when you go to cloud providers and folks like yourselves, I would imagine, live in all of those cloud environments or multiple of them. So, you can also serve customers that, of course, are gonna have a multi-cloud strategy.

    Shamgunov: Absolutely, you know, I couldn’t have said it better.

    Ashley: Okay, great. [Laughter] Lock that one in—great. What’s the number one challenge as folks move out of—let’s pick Oracle again—move out of that traditional VLDB environment and move into the cloud. What’s the paradigm shift that you have to make to really think about how you can leverage the flexibility of the cloud?

    Shamgunov: There is a three step strategy if you wanna get out of Oracle. The biggest stickiness of any database technology is all the applications that are built against the database and they’re using and sharing that data across the applications. And then different database technologies can either be app compatible—you know, an example of that is MySQL and AWS Aurora, right?

    Ashley: Mm-hmm.

    Shamgunov: Aurora is a new database, but the compute piece of it is actually MySQL code, so they’re app compatible. If an app works against MySQL, it’s gonna work against Aurora.

    So, that’s why, in the low end of the market, in which both MySQL and Aurora play, this can be a viable strategy. And it’s a very good one, right, because it’s so easy to just point your app at a different database. And it’s very hard for a different database product to achieve application level compatibility between the two databases.

    The second one is workload compatibility. So, all our workloads that are running on Oracle can be moved to another database, assuming there is a good enough reason. You know, assuming the database you’re moving the workload to has all the same set of features and provides strong system of record guarantees and so on and so forth.

    So, if you are workload compatible, then you can move from one database into another, but you need to put in some work, right? You need to augment and potentially rewrite certain pieces of the application.

    Ashley: Mm-hmm.

    Shamgunov: And, in our space, there’s a massive piece of the market that is called tier one workloads. Those tier one workloads typically run on Oracle, and they have either very, very strong system of record requirements or they have performance requirements, or they have availability requirements, or they have very strong concurrency requirements.

    And because of our architecture, we are practically the only game in town that can be the destination for moving workloads from Oracle, and especially systems like Oracle—you know, heavy Oracle systems running on enterprise storage or systems like Oracle Rack or Oracle Exadata over to running on MemSQL. And we have countless examples of how we quote-unquote liberated our customers from Oracle.

    So—and this is, frankly, our biggest opportunity. So you will have to put some work, but if the pain that you’re experiencing is so high that you’re willing to put that work, we are a fantastic destination for moving your Oracle workloads.

    Ashley: Well, excellent. You’ve been a great guest and a fascinating conversation. I wish we had more time to chat more. Maybe we can do that on another podcast.

    Well, thank you, and with my audience, I’d also like to thank you, Nikita Shamgunov, for joining us, co-CEO and co-founder of MemSQL.

    Shamgunov: Yeah, very happy to be here. Thank you so much.

    Ashley: You bet. And thank you to our listeners. This is Mitch Ashley with devopsy.kinsta.cloud and you’ve listened to another DevOps Chat podcast. Be careful out there.

    — Mitchell Ashley

  • What Are AWS Application Patterns?

    What Are AWS Application Patterns?

    The cloud-native approach is not just about compute power and cloud services. A “set it and forget it” attitude won’t work with cloud native; there are far too many moving pieces and there is just way too much complexity. Instead, this approach requires constant testing, optimization and improvement. This is epitomized in the two key phrases of the DevOps methodology: continuous integration and continuous delivery.

    To make this a reality, however, it takes a strategic approach to defining application patterns that will govern the way in which cloud-native applications are built and shipped, rather than a one-off effort. Since AWS is the leading cloud provider, we should discuss which application patterns are relevant to the AWS platform. There are many aspects to consider when it comes to AWS application patterns, and we will cover them here.

    Operational Excellence

    The best starting point for application patterns is a commitment to operational excellence. Typically, the term “Ops” is used to identify the IT team, but in this case, operational excellence rests on the shoulders of Dev, QA and Ops teams. In cross-functional DevOps teams, every member should strive for operational excellence at every step–even in the smallest of tasks.

    In their white paper on operational excellence, AWS encourages its users to manage operations as code. This helps avoid human error, enables automation of complex processes and allows for quick and easy rollback when errors occur. Failure, according to AWS, is an opportunity for learning. They even advise performing “pre-mortems” that try to anticipate potential points of failure before a deployment.

    AWS Trusted Advisor is a useful tool that enables this type of accountability and shared responsibility within a team. It checks and reviews your new and existing AWS workloads looking for opportunities to save costs, improve performance, tighten security or follow best practices for scaling operations.

    Security

    Security is a pattern that is non-negotiable. The term “DevOps” has inspired another variation called DevSecOps that signifies this fact. Security operations take center stage when talking about DevOps.

    The interesting thing about DevSecOps is it actually results in more freedom, flexibility and trust across teams when done right. This is unlike traditional approaches to security which created silos, stifled creativity and slowed down innovation. Security is now a key piece of the puzzle, not an afterthought. AWS has many machine learning-based security solutions such as Macie which automatically identifies and secures personally identifiable information (PII).

    Along with this, configuring role-based access controls using AWS IAM is essential to have greater control over who or which applications access data and for how long. Access Control Lists govern access to data stored in AWS S3 buckets.

    The news is full of instances of data breaches due to weak security policies and access controls. Capital One is a recent example of a data breach that affected 100 million of its users. While security in the cloud is more complex than before, AWS has the necessary tools and systems in place. It’s up to organizations to implement a security strategy that prevents misuse of their data.

    Reliability

    Today, there is a low tolerance for downtime because of the potential loss of revenue. With the stakes so high, reliability is a pattern any application custodian would have high on their list of priorities. AWS has multiple reporting and health monitoring tools to keep you constantly on the pulse of your applications.

    Architectural patterns like microservices enable better resilience by decomposing monolithic applications into more manageable chunks called services. This “divide and conquer” approach leads to higher availability, which is critical for SaaS applications.

    Patterns like chaos engineering seek to intentionally kill random instances and services and force the DevOps team to figure out workarounds and backups for every scenario.

    Application Performance

    With the explosion of data, applications are always dealing with lots of data to transfer, store and process. An application’s performance depends on three factors: the backend data storage layer, the middleware components and the frontend.

    The backend data layer is the most critical component when it comes to performance. Choosing the right storage formats, database types, storage devices and organization patterns all contribute to the speed of data access. AWS has an amazing array of storage services, including S3, RedShift, Aurora and Glacier. AWS gives its users every type of storage solution, all of which should be considered to improve data management.

    AWS is devising new ways to query large datasets without bearing the load on your existing systems. RedShift Spectrum is one example of this. It is a great way to “minimize the need to scale Redshift with a new node, which can be expensive.” If you’d still rather not have to manage any RedShift clusters on your own AWS Athena is a great serverless alternative

    That gives you all the power you need to query large datasets in S3 without having to maintain any clusters of your own.

    This pay-per-use model is making its ways across the entire AWS stack with services such as ECS and Fargate offering completely serverless experiences when operating containers at scale. What started out with AWS Lambda is catching on fast, and many AWS services like the ones mentioned above enable serverless experiences with a focus on application performance.

    Cost Optimization

    When moving to the cloud, pricing is completely different. Rather than paying for perpetual licenses, you now pay a small fee for every small resource that you use. Though small on its own, you’ll soon realize that cloud costs can shoot up easily if you’re not paying close attention to your spending.

    Cost optimization is necessary as you scale operations in the cloud. Cost optimization includes looking for more efficient services, whether that’s within the same vendor platform or a different platform. At a minimum it may require a lift-and-shift approach to transition from expensive old infrastructure to inexpensive new alternatives. For example, shifting workloads from VMs to containers to serverless within the same platform can show great cost savings, since containers and serverless are generally more cost-effective than VMs.

    It pays to leverage the expertise of experts in this area. They should ideally be people who have spent years using AWS, understand Well-Architected best practices and have found ways to cut costs and derive more value from the same budget.

    Application Patterns Enable Extensibility

    Microservice applications adopt a pluggable architecture. This enables them to be easily integrated with external applications. Typically, API-based communication patterns are most common. AWS API Gateway is a service that manages API design for AWS applications. In some cases, an SDK is the better choice–for example, when you need more control over the developer experience and integrations. It’s important to know when to choose either option.

    Application Patterns Enable Portability

    Portability is the ability to move workloads across instances without interrupting their functioning. As instances change, the workloads which they support shouldn’t be affected. This is only possible if there is a clear separation layer between the workloads and the infrastructure that runs them.

    Application Patterns Enable Shift-Left

    The DevOps methodology is known to encourage a shift-left approach to software delivery. This means that not only Dev, but also QA and Ops contribute to the design and development stage of an application. This fosters deeper collaboration across teams and breaks down silos.

    A shift-right approach holds true as well. Shift-right means developers are responsible for the code they write. They are equal owners with operations teams for the resilience of the apps they build and also the bugs they introduce into the system. This practice of putting ownership and responsibility where it belongs is very healthy for a DevOps team.

    AWS facilitates pipeline management using tools such as CodePipeline, CodeCommit and CodeDeploy. They each perform similar but complementary roles and help build a CI/CD pipeline. They make it easy to get started with CI/CD within the AWS platform.

    For organizations further down the line in their cloud journey, AWS’ pipeline management tools may seem too simplistic for something like GitOps. Instead, they’d want to look outside AWS for tools such as Jenkins X or Weave Flux which are designed with GitOps automation in mind. Whether it’s the simple or advanced route, a pipeline brings great benefits over a manual development process.

    Conclusion

    Though tools enable the implementation of helpful application patterns, on their own, they are ineffective. Application patterns need a deliberate strategy and an approach for a start. Only after a strategy is in place, can you move to deciding which tools to use. With a strong strategy in place, any of the tools mentioned here can be swapped out for other better tools and the results will be the same. It’s important to not be distracted by the next shiny tool but keep a keen focus on the strategy for implementing AWS application patterns.

    As we’ve seen here, application patterns are diverse, but they all build upon each other. Without even one of these pillars, organizations would not be able to realize the goal of cloud-native applications. On the other hand, if they give careful thought to each of these application patterns, organizations would deliver cutting-edge applications that can only get better with the passing of time.

    This sponsored article was written on behalf of Eplexity.

    — Twain Taylor

  • Data Monoliths Go Cloud-Native with Kubernetes

    Data Monoliths Go Cloud-Native with Kubernetes

    Breaking monolithic applications into independently scalable services is the core of the cloud-native application model. By making cloud-native apps easy to deploy and manage, Kubernetes has contributed to a big change in IT practices.

    Initial cloud-native conquests started with stateless services and quickly grew to encompass applications that manage data. Database management systems have long relied on centralized servers, but cloud-native architectures are now reaching even the largest monoliths. Among the most interesting cloud-native converts are analytic databases, which hold data arranged in storage formats optimized for extremely fast analytic queries over billions ,or even trillions, of rows.

    The transformation of databases to cloud-native models has profound implications for data-driven applications. Intersecting trends in analytic database architecture and Kubernetes promise substantial benefits to the users of such applications as well as companies that operate them.

    The Rising Importance of Analytics

    Over the last decade the quantity of data that businesses generate has decoupled from the size of the business itself. Ad networks, website analytics and service observability applications often ingest 1 million events per second (or far more) and may have tables running into trillions of records. Businesses need to analyze this information rapidly–for example, to diagnose faults in cloud applications or to retrain models for displaying online ads. Flexible, real-time analytics have gone from nice-to-have to business critical.

    Meanwhile, new analytic engines have emerged that eliminate dependencies on specialized hardware and complex networking required by legacy data warehouses. Druid, ClickHouse or Elasticsearch run on commodity hardware and depend on basic compute, network and storage. Everything else, such as high availability (HA), is built into the application. Newer analytic databases offer built-in horizontal scaling using fleets of server processes. The server processes themselves typically run well in containers.

    With these changes, analytic databases are increasingly cloud-friendly if not cloud-native. It is a short hop to deployment on Kubernetes.

    Kubernetes Operators for Databases

    At the same time, as analytic databases have shifted their operational model, Kubernetes itself has become more welcoming for databases.

    One particularly important innovation for the database is custom resource definitions, or CRDs, which extend the Kubernetes API to define new types of manageable resources. Using CRDs makes it possible to have a single definition that shows the desired state of a complex application such as a data warehouse.

    Another addition is Kubernetes operators, which manage custom resource definitions. Operators track custom resource definitions and take actions when they change. They can also track other information that might be relevant to managing the custom resource, such as changes to pods, storage or even related cloud-native applications. It’s like having a human operator who is constantly watching the application, except the rules are programmed into software that runs inside Kubernetes itself.

    The operator pattern is a perfect match for analytic databases, which are complex distributed applications and may require definition of dozens or even hundreds of individual Kubernetes resources in a running system. Operators encapsulate these details as well as important procedures such as upgrade, scale-out and other operations that require careful manipulation of resources to prevent outages.

    It’s not surprising operators are popular with database developers. The OperatorHub.io registry for database operators alone has 19 entries covering databases of every kind, including analytic databases such as ClickHouse and Elastic. Many more exist outside the registry.

    A Cloud-Native Way to Consume Analytic Databases

    New data warehouse architectures and Kubernetes operators means users can now use analytic databases very differently from legacy data warehouses. Our work on ClickHouse, including development of the ClickHouse Kubernetes Operator, prompts two observations.

    First, operators make spinning up analytic databases simple and fast for all users. Amazon Redshift kicked open the door to data warehouse-as-a-service in the public cloud. Kubernetes operators enable similar capabilities not just in public clouds but any Kubernetes cluster. Individual developers can cycle through dozens of configurations in a single day while testing new analytic applications. Thanks to quick pod spin-up new analytic databases can be running in minutes or even less.

    More profoundly, users can use Kubernetes to move to an on-demand model for analytics. Rather than forcing all applications to compete for resources on a single shared cluster, each application can have an analytic database tuned to its specific requirements. Analytic databases can integrate tightly with applications in the same locations where those applications already run. Businesses can scale databases flexibly to deliver quality analytics at exactly the time they are needed and at reasonable cost. This is the heart of the cloud-native programming model.

    Modern database architectures combined with Kubernetes features, such as the operator pattern, are creating a new path to manage enormous volumes of data. For readers who are hesitant about managing data on Kubernetes, it’s time for another look.

    To learn more about containerized infrastructure and cloud native technologies, consider coming to KubeCon + CloudNativeCon NA, November 18-21 in San Diego.

    — Robert Hodges

  • Cloud Native Tracing and Observability: Why You Care

    Cloud Native Tracing and Observability: Why You Care

    Is observability the new monitoring? Or is observability, and tracing, fundamentally different? Like any IT industry trend, it can be difficult to discern as many jump on the trend bandwagon, appropriately or not. The Splunk .conf 2019 event presented Splunk with the opportunity to bring clarity to the terms and the reasoning behind the acquisitions of SignalFx and Omnition.

    Monitoring is mostly about taking events, telemetry data and established data points and thresholds into monitoring, alerting and problem management processes and tools. Its tried and true, and has progressed with the evolution of typical monolith, distributed apps and systems. Enter cloud-native applications, composed containers, microservices and service meshes. With great flexibility comes complexity, and cloud native isn’t immune to such an adage.

    What would have been a monolithic application is now shattered into hundreds or even thousands of smaller pieces of application and app technology software. The complexity of externally monitoring cloud native quickly surpasses the capabilities of most monitoring approaches.

    Observability, as recently acquired SignalFx CTO Arijit Mukherji shared this week, is built upon three pillars: metrics, tracing and logs. Metrics show when you have a problem, tracing points you to the problem and logs help find the root cause—a reasonable way to define and segment observability. It also, not surprisingly, fits well with the logic of Splunk plus SignalFx plus Omnition.

    To move beyond monitoring requires instrumentation, built into the software and APIs as part of the software creation process, a DevOps process. SignalFx in part brought Splunk auto-instrumentation, which during run-time, identifies frameworks and libraries in use within applications and can capture tracing instrumentation. Omnition brings even deeper tracing chops to perform tracing across large service meshes of microservices. Add Splunk’s capability to correlate data across the business, applications and operations data and you complete the picture with the ultimate goal of making observability easy for developers.

    The above might explain why developers, ops and DevOps care about observability and tracing, but should the business care? SignalFx CEO Karthik Rau connected the dots nicely during a conversation this week. Digital transformation strategies require speed and agility, but also demand more risk-taking. Confidence in risk-taking comes when accompanied by the ability to respond rapidly to changes and failures. A software deployed 10 minutes ago may need to be backed out or corrected rapidly when the users’ experience goes negative. That requires a rapid determination of what the problems is, and the ability to take immediate corrective action, including automated action.

    Cloud native and DevOps not only enable disruptive, digital transformation strategies but must be accompanied by rapid and automated responses when negative business and customer impacting conditions occur. Customers don’t care when CPUs are taxed, but they do care when a mobile app’s responses fail or are slow. The move to cloud native is served well when backed up by such capabilities to respond in near real-time when problems occur.

    — Mitchell Ashley

  • From Pets to X-as-Code: Enabling Control and Predictability for ITOps at Scale

    From Pets to X-as-Code: Enabling Control and Predictability for ITOps at Scale

    Amy Wheelus, vice president of network cloud at AT&T, recently spoke about declarative predictability at the Open Networking Summit in Antwerp. AT&T is the world’s largest telecommunications company providing mission critical communication services where people’s jobs–and lives–can be on the line. As they move away from labor-intensive IT operations toward cloud-native automation, there is a need to trust the declared IT infrastructure state can be predictably controlled and reproduced. This requirement to build predictability, reproducibility and trust into IT systems is not new but has grown in both scale and complexity as IT departments went from a few servers blinking in the basement to the core around which every business rotates. 

    With every customer interaction and business function becoming more digital, companies must be able to rely upon their IT departments if they hope to continue to keep pace in today’s rapidly changing world. This article traces the evolution of IT operations from Pets towards X-as-code with each improvement bringing companies closer to declarative predictability. These best practices enable enterprises to reliably deliver core business value through their IT department and keep pace in the cloud native era.

    Yesterday: Pets Versus Cattle and Infrastructure as Code

    Back when the IT department was just servers in the back room, it was all hands on deck when anything went wrong because work ground to a halt. The CEO couldn’t get their emails and the sales department couldn’t book new orders. From these early days, it was learned that in order to trust IT with critical business services, high availability and reliability needed to be built into the system. This lead to the creation of the Pets vs. Cattle analogy where each infrastructure component is interchangeable and can quickly be replaced when anything goes wrong. This allows companies to quickly recover from or even avoid outages.  

    While treating servers as cattle solves some problems, as any rancher can tell you, handling a whole heard brings its own set of challenges. As the size of managed infrastructure grows, it can be difficult to keep track of where, when and how it is actually deployed. In addition, changes to production systems to keep them healthy can lead to configuration drift creating unique snowflake servers which are pets rather than cattle. It becomes a difficult task to predict how a system will react or rely upon it in critical situations when its actual state is unknown.

    To help eliminate this problem and control how infrastructure is configured and deployed, ITOps teams moved to infrastructure-as-code. In this way of working, applications and servers are never modified once deployed. If something needs to be changed, a new version is created and the old one is decommissioned. Each deployment can be versioned to create an overview of what is in production and how it is deployed.

    Managing cattle rather than pets and using infrastructure-as-code has helped companies move from ticket-based operations with long lived machines to more scalable and dynamic infrastructure. These practices build reusability, scalability, automation and versioning into IT systems to create a simpler, more predictable deployment and management process. By being able to declare the state of its infrastructure, companies are able to predict and trust the outcome.

    Today: Cloud-Native Infrastructure

    Looking at the definition of cloud native from the Cloud Native Computing Foundation, it quickly becomes clear why many companies are turning toward these best practices. They help companies build and run resilient and manageable infrastructure that–combined with robust automation–creates dynamic, yet predictable systems. Declarative APIs and immutable infrastructure will be used as two examples to understand how cloud native builds upon and expands the techniques covered above.

    Declarative APIs make infrastructure as code and automation possible. For example, with Kubernetes, operations teams are able to declare the desired state of the system and automate the reconciliation with the actual state until they match. It originally began with just container orchestration but has since expanded to cover many other types of cloud resources including the physical infrastructure itself through Custom Resources and Cluster API respectively. Declarative APIs make infrastructure management simpler and more reproducible by automating operations and eliminating errors in production. Declarative APIs can be leveraged to create immutable infrastructure.

    Immutable infrastructure goes beyond just servers as cattle and only treating the infrastructure as code. Immutable infrastructure covers every higher layer of the stack until each one is codified and managed like cattle. Only exact copies are deployed and they can be completely replaced at any time. If anything needs to be changed, a new deployment is made and the old ones are decommissioned. Immutable infrastructure works for the whole stack to eliminate variations leading to fewer deployment failures, consistency across environments, easy horizontal scaling and a simple rollback and recovery process.

    As these two examples show, cloud native enables companies to reliably scale out dynamic IT services without having to simultaneously scale out their operations team. Building declarative predictability into systems engenders trust in automatic processes, even when critical services are running on them.

    Tomorrow: X-as-Code

    While cloud-native best practices provide many benefits, applications and infrastructure are just the beginning of the story for enterprise IT teams. In today’s world, they are presented with the Herculean tasks to manage a dynamic and sprawling environment while still keeping in compliance with internal policy along with national and international laws. This challenge moves far beyond mere servers and treating them like cattle to compliance and governance across multiple clouds and hundreds or thousands of developers. With these challenges in mind, the question still remains on how to best deliver on these demands at the enterprise scale.

    Looking across architecture, configuration, blueprinting, policy and governance, if each of these is simply viewed as another layer of the stack above the application, their operations can be radically simplified. By applying the principles of immutable infrastructure and cattle to this further layer of the stack, each of these can be defined and managed using an X-as-code model. These types of management models can already be seen taking root across the cloud-native ecosystem.

    Kubernetes creates blueprints-as-code through the use of Admission Controllers while policy-as-code can be defined in Pod Security Policies and Network Policies. The CNCF even has a policy engine, OPA, as a sandbox project that gives operators fine-grained yet flexible policy control across their entire stack. By using code to decouple policy and other layers from the actual implementation, the resulting stack is easier to understand, flexible enough to handle future requirements and less expensive to maintain.

    At Loodse, our customers were struggling to manage configuration, services and policy across their clusters. Leveraging the Kubernetes controller paradigm in Kubermatic, we are able to automate the deployment and lifecycle of each of these add-ons. This gives users the ability to declare, enforce and automate the required state of not only the infrastructure but the configuration and policy, too. Kubernetes controllers bring predictability and control to dynamic and scalable cloud-native IT even as it begins to handle more and more mission critical services.

    Cloud Native in Production: Declarative Predictability

    With IT becoming the core of every business, bringing cloud-native best practices into production enables operators to predictably declare, run and manage their infrastructure at the enterprise scale. Being able to trust in these automated operations frees up resources and opens up many new possibilities. Developers can be given guardrails and become empowered to do their job rather than waiting in a ticket queue. Operators can easily create highly available self-healing infrastructure that meets governance standards allowing them to focus on delivering a platform with differentiated business value.

    At each step of the development of modern IT practices, from cattle to X-as-code, operational processes have become more streamlined allowing both developers and operators to drive more value to the bottom line. The future of IT infrastructure and operations will be a trusted platform that automates undifferentiated heavy lifting, allowing the whole business to focus on customer value.

    To learn more about containerized infrastructure and cloud native technologies, consider coming to KubeCon + CloudNativeCon NA, November 18-21 in San Diego.

    — Bill Mulligan

  • Kong Employs Machine Learning to Extend Services Control

    Kong Employs Machine Learning to Extend Services Control

    At the Kong Summit 2019 conference, Kong today announced it has acquired Insomnia, an open source testing tool for application programming interfaces (APIs). In addition, it has added support for REST, Kafka Streams, gRPC, GraphQL and plugins written in Go to its service control platform.

    Kong also added more capabilities based on machine learning algorithms in the form of a visual service map a Kong Immunity module that detects anomalies across services spanning multiple platforms.

    Additionally, the company is delivering Kong Studio, a suite of tools to design, build and maintain APIs for REST and GraphQL endpoints based on the API testing and debugging tool Insomnia, which then can be published to Kong Gateway, proxy software for deploying and managing distributed APIs. The 2.0 release of Kong Gateway separates the data and control plane for the platform and provides the ability to write plugins in Go via Kong’s Plugin Development Kit.

    Kong CTO Marco Palladino said collectively, this approach makes it easier to address a raft of potential API issues at design time rather than after an API has been pushed into a production environment.

    Finally, Kong also updated Kuma, an open source control plane for a service mesh based on Envoy proxy software developed under the auspices of the Cloud Native Computing Foundation (CNCF).

    Kong has been making the case for a service control plane dubbed Kong Brain, which spans both cloud-native and legacy monolithic environments and makes extensive use of machine learning algorithms. Palladino said as more services proliferate across existing monolithic and cloud-native application environments, there is no way IT organizations will be able to keep pace without relying more on machine learning algorithms to manage events and processes occurring within milliseconds of one another across the much more complex IT environment.

    In fact, Palladino noted that rather than expecting one environment to supplant the other, IT organizations should expect to see services proliferate across both monolithic and cloud-native application environments. Managing all those different classes of services will require a service control platform capable of spanning multiple environments, he said.

    Over time, machine learning algorithms will enable service control platforms to become smarter as IT environments transition to managing services end to end versus continuing to focus on individual applications. Naturally, there’s a high correlation between organizations that have made that transition and those that have embraced DevOps to build, deploy and manage those services, said Palladino, noting the most successful organizations employ DevOps to deploy and update APIs as if they were a standalone application rather than a set of add-on capabilities that tend to be forgotten over time.

    It remains to be seen how the confluence of APIs and DevOps ultimately will change how IT is managed. The only thing that is for certain is the days when applications were managed in isolation are finally coming to a much-needed close.

    — Mike Vizard

  • DevSecOps Becomes a Higher Cloud-Native Priority

    DevSecOps Becomes a Higher Cloud-Native Priority

    At the Cloud Native Security Summit, Enterprise Strategy Group (ESG) today revealed the results of a survey of 600 senior IT leaders that finds organizations are looking at DevSecOps as a way to address the complexities of managing and securing cloud-native applications.

    According to the survey results, 43% of respondents said their biggest challenge with cloud-native applications is maintaining consistency across disparate infrastructures. As a result, the same number of respondents said DevSecOps automation as their highest cloud security priority.

    Commissioned by Capsule8, Obsidian and Signal Sciences, the survey also finds 90% of respondents are concerned about not having visibility into misconfigured cloud services, server workloads, network security or privileged accounts. Another 83% are worried about the misuse of privileged accounts by insiders.

    Two-thirds (66%) say IT is more complex than it was two years ago, with more than a third (35%) citing the need to manage multiple cybersecurity controls as a major source of that complexity.

    Doug Cahill, a senior analyst and group director at ESG, said it’s clear that as organizations embrace best DevOps practices to build and deploy cloud-native applications, those processes now are being extended to include security controls as part of the quality assurance process. Rather than bolting on security, Cahill said, more organizations are building cybersecurity controls into their software from the ground up.

    Capsule8 CEO John Viega noted much of that shift is being driven by necessity. When applications were deployed mainly in on-premises IT environments, it was easier to secure the environment by deploying appliances. Now organizations need to secure applications on infrastructure they don’t control as part of a shared responsibility model that often spans multiple cloud service providers. The cloud may be more agile and less expensive, but Viega noted it’s not uncommon for hundreds of cloud accounts to have been set up by individuals in the same organization, each with very different levels of cybersecurity expertise.

    Obsidian Security CTO Ben Johnson added the rate at which cloud-native applications are being deployed and updated is overwhelming the ability of cybersecurity teams to keep pace. In fact, many organizations are still underestimating the scope of the cloud-native cybersecurity challenge they face.

    Finally, Hala Al-Adwan, vice president of technology for Signal Sciences, observed as organizations embrace DevSecOps, cybersecurity professionals still will play a key role in terms of their consulting expertise. However, responsibility for implementing cybersecurity will continue to shift left toward DevOps teams. The challenge those teams will face is the need to replace legacy cybersecurity infrastructure designed for cybersecurity administrators with programmable tools that fit neatly within a continuous integration/continuous deployment environment, said Al-Adwan.

    It’s not clear precisely when a desire to embrace DevSecOps will result in more secure applications. In theory, cloud-native applications should be a lot more secure than legacy monolithic applications. However, the degree to which higher levels of security will motivate organizations to replace monolithic applications remains to be seen.

    In the meantime, cultural issues associated with embracing DevSecOps will abound. There are simply not enough cybersecurity professionals available to participate in every application development scrum session. The real issue now is finding a way to embed that knowledge into an application development process operating at industrial scale.

    — Mike Vizard

  • ‘DevOps in the Multicloud World’ Virtual Summit Puts IBM Z in the Spotlight

    ‘DevOps in the Multicloud World’ Virtual Summit Puts IBM Z in the Spotlight

    Got mainframes on the mind? Wonder how you can unlock the value of the platform for mission critical workloads? Trying to figure out where IBM Z fits in an increasingly cloud native IT environment? Those questions and others will be discussed in an upcoming virtual summit that explores the role of IBM Z —within the context of delivering higher efficiency and greater productivity for hybrid multicloud environments.

    “DevOps in the Multicloud World” virtual summit is scheduled to take place Oct. 24, starting at 10 a.m. Eastern. The day-long event features a passel of IBM experts, partners and customers sharing their knowledge and offering a deep dive into how to build and manage hybrid applications for agility in a multicloud world of continuous delivery. Panels and presentations will be held on a multitude of topics ranging from open source and DevOps to provisioning and user experience—all with a focus on the IBM Z systems.

    In addition, IBM customers will discuss how they’ve successfully included IBM Z in their multicloud strategies to improve their operations and bring real value to their end-user customers.

    Featured speakers include:

      • Barry Baker, Vice President, IBM Z Software
      • Rosalind Radcliffe, Distinguished Engineer, Chief Architect for CLM and DevOps, Member of the Academy of Technology, IBM
      • Mike Fulton, IBM Distinguished Engineer, CTO Enterprise DevOps, IBM Systems
      • Keith Wilson, Principal Technical Architect, USAA
      • Patrick Wawrzoszek, Senior Programmer Analyst, Scotiabank
      • James Gill, Mainframe DevOps Engineer, Triton Consulting (Lloyds Banking Group)
      • Arun Sharma, Product Manager Core Banking, DXC Technology
      • and more….

    In between sessions, attendees can roam the virtual conference hall, where they can chat and socialize with other attendees including IBM staff, and visit the virtual expo floor, featuring virtual solution zone stands equipped with video, downloads and even prizes for attendees.

    Perhaps best of all? Because this is a virtual event, you don’t have to leave the comfort of your own environs. Feel free to log in wearing your pajamas, even—we don’t judge!

    This event is free, but space is limited, so register now to reserve your spot today!

    — Charlene O’Hanlon

  • Going Cloud-Native with IBM Z as a Hybrid Cloud Platform

    Going Cloud-Native with IBM Z as a Hybrid Cloud Platform

    When people use the phrase “they don’t make ‘em like that anymore,” 100-year-old bridges, Jon Snow’s sword and American muscle cars often come to mind. If the topic of discussion happens to be computer technology, however, mainframes are without a doubt the Valyrian steel of the enterprise and lucky for us, are still in production today. While the cloud computing concept as we know it has been around for only about a decade, mainframes have been delivering on-demand access to compute and storage resources that are safe, scalable, highly available and resilient for half a century.

    Big Iron Advantage

    In today’s enterprise scenario, the mission to gain an edge over the competition in terms of pure horsepower can be elusive, to say the least. Although 20% of the easy enterprise workloads have moved to cloud platforms, it is estimated a large amount of data and applications continue to be run on-premise as enterprises determine their hybrid cloud strategy. Far from being irrelevant, mainframe architecture now finds itself at the center of this digital battle, causing disruption and giving its users a much-coveted and undeniable edge over the competition. In addition to running over 8,000 VMs on a single system, mainframes offer some of the most scalable storage in the industry.

    In case that isn’t enough, couple that with pervasive encryption, multi-factor authentication, a co-processor dedicated to cryptography and the most trusted and secure Linux environment in the industry today. What you get in the end is a major breakthrough with regard to cloud limitations that many early adopters of cloud technology found out about the hard way. These limitations are especially true with regard to cybersecurity, and though private clouds help alleviate these risks to an extent, IBM Z removes these risks by enabling all-encompassing encryption.

    Enabling the Hybrid Cloud

    While most people were skeptical of mainframes making a comeback until about a year ago, the majority seem to have changed their tune. There are two reasons for this: One is IBM’s new take on the mainframe, called IBM Z; and the other reason is its applications with regard to the hybrid cloud. Hybrid basically refers to organizations using a broad combination of public, private, IaaS and Paas in the same application solution. Mainframes are proving an important part of hybrid infrastructure thanks to the massive compute, storage and scalability features they can deliver.

    Another advantage mainframes have in a hybrid environment is simplicity. Every engineer knows the more moving parts, the higher the risk of a breakdown, and the same holds true for the hybrid cloud. In a complex, multi-cloud environment that often spans more than one public cloud and a variety of on-prem resources, the more discrete devices involved in a network, the higher the risk of something failing. With a mainframe like the IBM Z, you have one device that delivers broad functionality at high efficiency and low cost. And it does it for a few decades, without complaining.

    IBM Z and the Cloud

    IBM’s Z models are lean, mean, transacting machines. Recent innovations in hardware and software design have made it possible to run not just native mainframe apps, but also most Linux-based operating systems and applications on these systems. IBM Z also features machine learning and AI frameworks, as well as support for Apache SparkML, TensorFlow and languages such as Scala and Python. Capable of running more than 12 billion fully encrypted transactions per day on a single machine and 850 million on IBM’s skinny version, the z14 has a 35% capacity increase for workloads compared to the z13, and it’s being touted as the most powerful transaction machine in existence today.

    Combining mainframes and cloud platforms in today’s hybrid environments may sound like old wine in a new bottle, but it effectively gives you the best of both worlds–the scalability of the cloud, along with the power, security and dependability of the mainframe. Additionally, the mainframe’s ability to house both data and applications on the same device, while dedicating processors to specific operations independently, make it ideal for ensuring speed in such complex and distributed environments. IBM Z also features a multiple processing structure that makes it ideal for dealing with the varying bandwidth needs of modern applications. It can process Java workloads up to 50% faster than x86 servers.

    Cost Efficiency

    Forty-four of the top 50 banks in the world run on IBM Z servers right now. They understand that running their applications on a mainframe provides security and scalability for a compelling return on investment. Large enterprise systems have had a perception of being expensive, but when you factor in such things as space, power and cooling efficiency, as well as the number of personnel required to administer the system, the total cost of ownership is a value-add when leveraging Z in your hybrid cloud strategy.

    — Twain Taylor

  • Bringing 5G-Enabled Digital Services to Life Calls for Cloud-Native Operations

    Bringing 5G-Enabled Digital Services to Life Calls for Cloud-Native Operations

    Spotify, Netflix, IoT connectivity and cloud storage are a few examples of the digital services providing new options for fun and function to consumers and businesses alike, while representing a growing boon for communications service providers (CSPs).

    Gartner predicts by 2020, nearly one quarter (23%) of CSPs’ large enterprise business communication revenues will be driven by digital services—up a staggering 10 percentage points from 13% of revenues in 2017. This gain should come as no shock to developers who favor building in the cloud or using tools such as Microsoft Teams to collaborate, or who stream video from HBO Go or skim stories in Apple News during their commutes. Today’s customers (consumers or businesses) expect value-driven, digitally enabled services that inspire personalized and meaningful experiences with their brands of choice.

    The good news is companies are eager to meet demand: CSPs want to deploy services in the cloud to address the challenges of web-scale companies. In fact, 85% of companies surveyed say launching new digital services is critical to their business strategy.

    What’s the catch? To do it, they’ll need to be agile. Although telecom is not an industry known for its agility, and is often thought of as the opposite, it has been transforming—like many industries—through adopting more agile approaches in business strategy. CSPs are striving to maximize resources and reduce inefficiencies, while at the same time sparking innovation. Nothing has been more of a stimulus for agility than the cloud, especially for the web-scale companies whose services pose the greatest threat to the telco industry.

    A key requirement will be cloud-native applications that improve the agility of network functions. However, being cloud-native involves a lot more than just shoehorning code into a container previously ported from an appliance to a virtual machine.

    Rather, at Oracle, cloud-native means faster development and deployment through DevOps and continuous integration and continuous delivery (CI/CD) pipeline. CSPs can only achieve an agile cloud with an environment enabling developers to innovate true cloud-native solutions if they implement an operational framework that embraces a key set of principles. Specifically, we at Oracle have identified the principles for successful cloud-native operations.

    System Immutability

    Everything, including software and configuration, is code. All changes are made through CI/CD, where they are deployed as immutable artifacts. No manual configurations or customizations are allowed. This makes it easier to implement the principle of least privilege as there is no need to run scripts in the production environment. Also, any changes not coming though the delivery pipeline can be considered malicious.

    Automate Everything

    All aspects of build, test, verification and deployment are automated. This includes activities such as backup, recovery, password/key rotation, etc. Fully automating the DevOps pipeline, including verification and testing, removes much of the potential for human error. This automation also allows changes to be applied to the environment with confidence and provides for rapid repair.

    Disposability

    All services are transient and treated as short-lived. Instead of focusing on never failing, services are designed to go up and down quickly without service interruption. Regular repaving (re-deployment) of the environment ensures failed or failing services are removed and new ones deployed.

    Externalized Configuration

    Configuration (including passwords, credentials, location of backing stores, etc.) is decoupled from the software image, and like software, can be treated as a build artifact in a controlled and versioned manner. Versioned configuration enables development and production parity as an artifact and can eliminate costly operational errors.

    Logs as Event Streams and Constant Telemetry

    Everything needed to debug or diagnose any functional, operational or security issue is found in logs, traces or metrics data. These are treated as a stream of time-ordered events and stored in a centralized collector outside the system, enabling better threat monitoring, forensics and diagnostics through event correlation or analyzing the aggregated and holistic view. Constant telemetry is fundamentally enabled through logs as event streams and, thus, the two should always be considered together.

    Delegated Governance

    Some shared aspects of the environment are centrally managed like networking, identity management or infrastructure, but in a true DevOps fashion, teams delivering a service are responsible for operating the service. This is allowed only with strict governance, enforced through checks in the continuous delivery pipelines, giving greater control over rate of change. Business agility is improved as applications have better visibility into the operations of their service with tighter feedback loops that ultimately improve quality.

    Independent Lifecycle

    Independently upgrading, scaling and deploying each microservice is paramount for supporting other cloud-native principles as well as minimizing the amount of change in the system at a given time. Furthermore, such decoupling makes other principles, such as repaving, easier and promotes easier isolation of issues.

    As 5G introduces increased connectivity and speed, along with the need for more security and reliability, service providers will need the agility, flexibility and security of the cloud and cloud-native principles. These principles are enablers for better cloud operations. Together, they provide the ideal context in which to implement a broader view of the Cloud Native Computing Foundation (CNCF) landscape, giving developers a robust Platform-as-a-Service of backing services beyond just containers.

    As 5G moves from a fabled future to a reality, the market for digital services that reap its benefits is likely to continue to grow. By leveraging these principles, along with DevOps and the CI/CD pipeline, organizations will be able to evolve and prepare for 5G’s promise for innovative business models and new revenue streams—be they digital services or beyond.

    — Shirin Esfandiari